Develop/AIclinicalresearch
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
master
AIclinicalresearch/backend/test-result.txt
HaHafeng beb7f7f559 feat(asl): Implement full-text screening core LLM service and validation system (Day 1-3) 
Core Components:
- PDFStorageService with Dify/OSS adapters
- LLM12FieldsService with Nougat-first + dual-model + 3-layer JSON parsing
- PromptBuilder for dynamic prompt assembly
- MedicalLogicValidator with 5 rules + fault tolerance
- EvidenceChainValidator for citation integrity
- ConflictDetectionService for dual-model comparison

Prompt Engineering:
- System Prompt (6601 chars, Section-Aware strategy)
- User Prompt template (PICOS context injection)
- JSON Schema (12 fields constraints)
- Cochrane standards (not loaded in MVP)

Key Innovations:
- 3-layer JSON parsing (JSON.parse + json-repair + code block extraction)
- Promise.allSettled for dual-model fault tolerance
- safeGetFieldValue for robust field extraction
- Mixed CN/EN token calculation

Integration Tests:
- integration-test.ts (full test)
- quick-test.ts (quick test)
- cached-result-test.ts (fault tolerance test)

Documentation Updates:
- Development record (Day 2-3 summary)
- Quality assurance strategy (full-text screening)
- Development plan (progress update)
- Module status (v1.1 update)
- Technical debt (10 new items)

Test Results:
- JSON parsing success rate: 100%
- Medical logic validation: 5/5 passed
- Dual-model parallel processing: OK
- Cost per PDF: CNY 0.10

Files: 238 changed, 14383 insertions(+), 32 deletions(-)
Docs: docs/03-涓氬姟妯″潡/ASL-AI鏅鸿兘鏂囩尞/05-寮€鍙戣褰?2025-11-22_Day2-Day3_LLM鏈嶅姟涓庨獙璇佺郴缁熷紑鍙?md
2025-11-22 22:21:12 +08:00

222 lines
12 KiB
Plaintext
Raw Permalink Blame History

This file contains invisible Unicode characters
This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
[dotenv@17.2.3] injecting env (13) from .env -- tip: 👥 sync secrets across teammates & machines: https://dotenvx.com/ops
[Config] Loaded .env file for development
[CacheFactory] Using MemoryCacheAdapter
🚀 开始测试 Day 2 Prompt 和服务...
📋 测试1: PromptBuilder基础功能
────────────────────────────────────────────────────────────
[1.1] 加载基础System Prompt...
2025-11-22 20:12:31.222 [aiclinical-backend] info: Building system prompt...
{
"env": "development",
"instance": "LAPTOP-UH2VURTB",
"pid": 53216
}
2025-11-22 20:12:31.224 [aiclinical-backend] info: System prompt loaded from D:\MyCursor\AIclinicalresearch\backend\src\modules\asl\fulltext-screening\prompts\system_prompt.md
{
"env": "development",
"instance": "LAPTOP-UH2VURTB",
"pid": 53216
}
✅ 成功！长度: 6699 字符
- 包含"Cochrane": ✅
- 包含"Lost in the Middle": ✅
- 包含"逐段阅读": ✅
[1.2] 加载Cochrane标准...
2025-11-22 20:12:31.224 [aiclinical-backend] info: Building system prompt...
{
"env": "development",
"instance": "LAPTOP-UH2VURTB",
"pid": 53216
}
2025-11-22 20:12:31.225 [aiclinical-backend] info: Cochrane standard loaded: 随机化方法
{
"env": "development",
"instance": "LAPTOP-UH2VURTB",
"pid": 53216
}
2025-11-22 20:12:31.225 [aiclinical-backend] info: Cochrane standard loaded: 盲法
{
"env": "development",
"instance": "LAPTOP-UH2VURTB",
"pid": 53216
}
2025-11-22 20:12:31.225 [aiclinical-backend] info: Cochrane standard loaded: 结果完整性
{
"env": "development",
"instance": "LAPTOP-UH2VURTB",
"pid": 53216
}
2025-11-22 20:12:31.225 [aiclinical-backend] info: System prompt built successfully, total length: 31538 chars
{
"env": "development",
"instance": "LAPTOP-UH2VURTB",
"pid": 53216
}
✅ 成功！长度: 31538 字符（增加了 24839 字符）
- 包含"随机化方法": ✅
- 包含"盲法": ✅
- 包含"结果完整性": ✅
[1.3] 加载Few-shot案例...
2025-11-22 20:12:31.225 [aiclinical-backend] info: Building system prompt...
{
"env": "development",
"instance": "LAPTOP-UH2VURTB",
"pid": 53216
}
2025-11-22 20:12:31.226 [aiclinical-backend] info: Few-shot example loaded: 信息在中间位置案例
{
"env": "development",
"instance": "LAPTOP-UH2VURTB",
"pid": 53216
}
2025-11-22 20:12:31.226 [aiclinical-backend] info: System prompt built successfully, total length: 14669 chars
{
"env": "development",
"instance": "LAPTOP-UH2VURTB",
"pid": 53216
}
✅ 成功！长度: 14669 字符
- 包含"信息在中间位置": ✅
[1.4] MVP完整配置（Cochrane + Few-shot）...
2025-11-22 20:12:31.226 [aiclinical-backend] info: Building system prompt...
{
"env": "development",
"instance": "LAPTOP-UH2VURTB",
"pid": 53216
}
2025-11-22 20:12:31.226 [aiclinical-backend] info: System prompt built successfully, total length: 39508 chars
{
"env": "development",
"instance": "LAPTOP-UH2VURTB",
"pid": 53216
}
✅ 成功！长度: 39508 字符
- 预估Token: 13169
[1.5] 加载JSON Schema...
✅ 成功！
- 类型: object
- 必需字段: fields, processing_log, verification, metadata
- 12字段: ✅
[1.6] 构建User Prompt...
2025-11-22 20:12:31.226 [aiclinical-backend] info: Building user prompt...
{
"env": "development",
"instance": "LAPTOP-UH2VURTB",
"pid": 53216
}
2025-11-22 20:12:31.227 [aiclinical-backend] info: User prompt template loaded from D:\MyCursor\AIclinicalresearch\backend\src\modules\asl\fulltext-screening\prompts\user_prompt_template.md
{
"env": "development",
"instance": "LAPTOP-UH2VURTB",
"pid": 53216
}
2025-11-22 20:12:31.227 [aiclinical-backend] info: User prompt built successfully, total length: 4425 chars
{
"env": "development",
"instance": "LAPTOP-UH2VURTB",
"pid": 53216
}
✅ 成功！长度: 4425 字符
- 包含PICOS: ✅
- 包含全文: ✅
[1.7] 构建完整Prompt（System + User）...
2025-11-22 20:12:31.227 [aiclinical-backend] info: Building system prompt...
{
"env": "development",
"instance": "LAPTOP-UH2VURTB",
"pid": 53216
}
2025-11-22 20:12:31.227 [aiclinical-backend] info: Building user prompt...
{
"env": "development",
"instance": "LAPTOP-UH2VURTB",
"pid": 53216
}
2025-11-22 20:12:31.227 [aiclinical-backend] info: User prompt built successfully, total length: 4425 chars
{
"env": "development",
"instance": "LAPTOP-UH2VURTB",
"pid": 53216
}
2025-11-22 20:12:31.227 [aiclinical-backend] info: System prompt built successfully, total length: 39508 chars
{
"env": "development",
"instance": "LAPTOP-UH2VURTB",
"pid": 53216
}
✅ 成功！
- System Prompt: 39508 字符
- User Prompt: 4425 字符
- 总计: 43933 字符
- 预估Token: 14644
💾 完整Prompt已保存到: D:\MyCursor\AIclinicalresearch\backend\test-output/
✅ PromptBuilder测试全部通过！
📋 测试2: LLM12FieldsService（模拟模式）
────────────────────────────────────────────────────────────
[2.1] 创建LLM12FieldsService实例...
✅ 成功！
[2.2] 测试Nougat提取策略...
⏭️ 跳过（需要实际PDF文件）
[2.3] 测试缓存Key生成...
✅ 缓存机制已集成（通过process12Fields验证）
[2.4] 测试成本计算...
测试Token数: 10000
- deepseek-v3: ¥0.0100
- qwen-max: ¥0.0400
- gpt-4o: ¥0.3000
✅ 成本计算逻辑正确
✅ LLM12FieldsService测试完成（模拟模式）！
⚠️ 注意: 实际LLM调用需要：
1. 配置API Key（DeepSeek/Qwen）
2. 准备测试PDF文件
3. 运行集成测试
📋 测试3: Prompt文件完整性检查
────────────────────────────────────────────────────────────
检查必需文件:
✅ system_prompt.md
大小: 10KB
✅ user_prompt_template.md
大小: 5KB
✅ json_schema.json
大小: 14KB
✅ cochrane_standards/随机化方法.md
大小: 8KB
✅ cochrane_standards/盲法.md
大小: 11KB
✅ cochrane_standards/结果完整性.md
大小: 12KB
✅ few_shot_examples/信息在中间位置案例.md
大小: 10KB
✅ 所有必需文件都存在！
============================================================
🎉 所有测试完成！Day 2 Prompt和服务验证通过！
============================================================
📋 下一步建议:
1. 查看生成的Prompt文件（test-output/目录）
2. 配置API Key准备集成测试
3. 继续Day 3开发（验证服务）
Reference in New Issue View Git Blame Copy Permalink