feat(asl): Complete Week 4 - Results display and Excel export with hybrid solution
Features: - Backend statistics API (cloud-native Prisma aggregation) - Results page with hybrid solution (AI consensus + human final decision) - Excel export (frontend generation, zero disk write, cloud-native) - PRISMA-style exclusion reason analysis with bar chart - Batch selection and export (3 export methods) - Fixed logic contradiction (inclusion does not show exclusion reason) - Optimized table width (870px, no horizontal scroll) Components: - Backend: screeningController.ts - add getProjectStatistics API - Frontend: ScreeningResults.tsx - complete results page (hybrid solution) - Frontend: excelExport.ts - Excel export utility (40 columns full info) - Frontend: ScreeningWorkbench.tsx - add navigation button - Utils: get-test-projects.mjs - quick test tool Architecture: - Cloud-native: backend aggregation reduces network transfer - Cloud-native: frontend Excel generation (zero file persistence) - Reuse platform: global prisma instance, logger - Performance: statistics API < 500ms, Excel export < 3s (1000 records) Documentation: - Update module status guide (add Week 4 features) - Update task breakdown (mark Week 4 completed) - Update API design spec (add statistics API) - Update database design (add field usage notes) - Create Week 4 development plan - Create Week 4 completion report - Create technical debt list Test: - End-to-end flow test passed - All features verified - Performance test passed - Cloud-native compliance verified Ref: Week 4 Development Plan Scope: ASL Module MVP - Title Abstract Screening Results Cloud-Native: Backend aggregation + Frontend Excel generation
This commit is contained in:
378
docs/03-业务模块/ASL-AI智能文献/05-开发记录/2025-11-21-真实LLM集成完成报告.md
Normal file
378
docs/03-业务模块/ASL-AI智能文献/05-开发记录/2025-11-21-真实LLM集成完成报告.md
Normal file
@@ -0,0 +1,378 @@
|
||||
# 真实LLM集成完成报告
|
||||
|
||||
**日期**: 2025-11-21
|
||||
**任务**: 将Mock AI替换为真实LLM调用
|
||||
**状态**: ✅ 完成
|
||||
|
||||
---
|
||||
|
||||
## 📋 背景
|
||||
|
||||
### 之前的状态
|
||||
- ✅ 已完成 Prompt 设计（v1.0.0-MVP）
|
||||
- ✅ 已实现 `llmScreeningService.ts`（真实LLM调用）
|
||||
- ✅ 已完成测试框架和质量验证
|
||||
- ❌ **问题**: `screeningService.ts` 中使用 `mockAIScreening` 生成假数据
|
||||
|
||||
### 用户需求
|
||||
从"设置与启动"页面上传真实文献数据后，**使用真实的 DeepSeek 和 Qwen API 进行筛选**，而不是模拟数据。
|
||||
|
||||
---
|
||||
|
||||
## ✅ 完成内容
|
||||
|
||||
### 1. 修改 `screeningService.ts`
|
||||
|
||||
**文件**: `backend/src/modules/asl/services/screeningService.ts`
|
||||
|
||||
#### 核心改动
|
||||
|
||||
**引入真实LLM服务**:
|
||||
```typescript
|
||||
import { llmScreeningService } from './llmScreeningService.js';
|
||||
```
|
||||
|
||||
**替换处理逻辑**:
|
||||
```typescript
|
||||
// ❌ 旧代码（Mock）
|
||||
const result = await mockAIScreening(projectId, literature);
|
||||
|
||||
// ✅ 新代码（真实LLM）
|
||||
const screeningResult = await llmScreeningService.dualModelScreening(
|
||||
literature.id,
|
||||
literature.title,
|
||||
literature.abstract,
|
||||
picoCriteria,
|
||||
inclusionCriteria,
|
||||
exclusionCriteria,
|
||||
[models[0], models[1]],
|
||||
screeningConfig?.style || 'standard',
|
||||
literature.authors,
|
||||
literature.journal,
|
||||
literature.publicationYear
|
||||
);
|
||||
```
|
||||
|
||||
#### 新增功能
|
||||
|
||||
1. **从项目读取PICOS标准**:
|
||||
```typescript
|
||||
const project = await prisma.aslScreeningProject.findUnique({
|
||||
where: { id: projectId },
|
||||
});
|
||||
|
||||
const picoCriteria = project.picoCriteria;
|
||||
const inclusionCriteria = project.inclusionCriteria;
|
||||
const exclusionCriteria = project.exclusionCriteria;
|
||||
```
|
||||
|
||||
2. **支持自定义模型选择**:
|
||||
```typescript
|
||||
const models = screeningConfig?.models || ['deepseek-chat', 'qwen-max'];
|
||||
```
|
||||
|
||||
3. **详细日志记录**:
|
||||
```typescript
|
||||
logger.info('Processing literature', {
|
||||
literatureId: literature.id,
|
||||
title: literature.title?.substring(0, 50) + '...',
|
||||
});
|
||||
```
|
||||
|
||||
4. **结果映射到数据库格式**:
|
||||
```typescript
|
||||
const dbResult = {
|
||||
projectId,
|
||||
literatureId: literature.id,
|
||||
// DeepSeek结果
|
||||
dsModelName: screeningResult.deepseekModel,
|
||||
dsPJudgment: screeningResult.deepseek.judgment.P,
|
||||
// ... 完整的字段映射
|
||||
};
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔄 完整流程
|
||||
|
||||
### 用户操作流程
|
||||
```
|
||||
1. 访问"设置与启动"页面
|
||||
↓
|
||||
2. 填写 PICOS 标准
|
||||
↓
|
||||
3. 上传 Excel 文献列表（例如：199篇）
|
||||
↓
|
||||
4. 点击"开始AI初筛"
|
||||
↓
|
||||
5. 后端自动处理：
|
||||
a. 创建项目
|
||||
b. 导入文献
|
||||
c. 启动筛选任务
|
||||
↓
|
||||
6. 真实LLM处理（每篇约10-15秒）
|
||||
a. 调用 DeepSeek API
|
||||
b. 调用 Qwen API
|
||||
c. 对比结果，检测冲突
|
||||
d. 保存到数据库
|
||||
↓
|
||||
7. 前端自动跳转到"审核工作台"
|
||||
↓
|
||||
8. 显示真实的AI筛选结果
|
||||
```
|
||||
|
||||
### 技术流程
|
||||
|
||||
```
|
||||
前端: TitleScreeningSettings.tsx
|
||||
↓ POST /api/v1/asl/literatures/import
|
||||
|
||||
后端: literatureController.ts
|
||||
↓ importLiteratures()
|
||||
↓ startScreeningTask()
|
||||
|
||||
后端: screeningService.ts
|
||||
↓ processLiteraturesInBackground()
|
||||
↓ for each literature:
|
||||
↓ llmScreeningService.dualModelScreening()
|
||||
|
||||
后端: llmScreeningService.ts
|
||||
↓ Promise.all([
|
||||
screenWithModel('deepseek-chat', ...),
|
||||
screenWithModel('qwen-max', ...),
|
||||
])
|
||||
|
||||
后端: LLMFactory
|
||||
↓ getAdapter('deepseek-v3')
|
||||
↓ getAdapter('qwen3-72b')
|
||||
|
||||
真实API调用
|
||||
↓ DeepSeek API
|
||||
↓ Qwen API
|
||||
|
||||
结果保存
|
||||
↓ AslScreeningResult 表
|
||||
|
||||
前端: ScreeningWorkbench.tsx
|
||||
↓ GET /api/v1/asl/projects/:projectId/screening-results
|
||||
↓ 显示真实结果
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ⏱️ 性能预期
|
||||
|
||||
### 单篇文献处理时间
|
||||
| 步骤 | 耗时（串行） |
|
||||
|-----|------------|
|
||||
| DeepSeek API 调用 | 5-10秒 |
|
||||
| Qwen API 调用 | 5-10秒 |
|
||||
| 结果保存 | 0.1秒 |
|
||||
| **总计** | **10-20秒** |
|
||||
|
||||
### 批量处理时间（199篇）
|
||||
| 模式 | 耗时 | 说明 |
|
||||
|-----|------|-----|
|
||||
| **串行处理** | 33-66分钟 | 当前实现（避免API限流）|
|
||||
| 并发处理（3个） | 11-22分钟 | 可选优化（需测试） |
|
||||
| 并发处理（10个） | 3-7分钟 | 风险：可能触发API限额 |
|
||||
|
||||
**当前策略**: 串行处理（稳定优先）
|
||||
|
||||
---
|
||||
|
||||
## 🎯 与Mock数据的对比
|
||||
|
||||
### Mock 数据（旧）
|
||||
```javascript
|
||||
// ❌ 假数据
|
||||
dsPEvidence: "模拟证据: 研究人群与PICO中的P标准匹配"
|
||||
dsReason: "基于标题和摘要分析，该文献符合纳入标准。"
|
||||
dsConclusion: randomConclusion() // 随机！
|
||||
|
||||
// 特点：
|
||||
- 1秒完成199篇
|
||||
- 证据都是"模拟证据"
|
||||
- 判断结果随机生成
|
||||
```
|
||||
|
||||
### 真实LLM（新）
|
||||
```javascript
|
||||
// ✅ 真实数据
|
||||
dsPEvidence: "This study included adult patients with type 2 diabetes mellitus aged 18 years or older, which matches the population criteria."
|
||||
dsReason: "The study population consists of T2DM patients, the intervention is an SGLT2 inhibitor (empagliflozin), the comparator is placebo, and the study design is a randomized controlled trial. All PICO criteria are met. The study reports on cardiovascular outcomes including MACE, heart failure hospitalization, and cardiovascular death, which are the outcomes of interest."
|
||||
dsConclusion: "include" // AI真实判断！
|
||||
|
||||
// 特点：
|
||||
- 33-66分钟完成199篇
|
||||
- 证据引用文献原文
|
||||
- 判断基于Prompt v1.0.0-MVP
|
||||
- 准确率：60%（首次测试）
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔍 数据验证
|
||||
|
||||
### 验证方法
|
||||
```bash
|
||||
cd AIclinicalresearch/backend
|
||||
node check-data.mjs
|
||||
```
|
||||
|
||||
### 预期输出（真实数据）
|
||||
```
|
||||
🔬 筛选结果样本:
|
||||
[1] 文献: Assessment of Thrombectomy versus Combined...
|
||||
DeepSeek: include (P:match, I:partial, C:mismatch, S:match)
|
||||
Qwen: exclude (P:mismatch, I:mismatch, C:partial, S:match)
|
||||
冲突状态: conflict
|
||||
是否有证据: DeepSeek=true, Qwen=true ✅
|
||||
|
||||
证据示例:
|
||||
- dsPEvidence: "The study population consists of..."
|
||||
- qwenPEvidence: "Patients with acute ischemic stroke..."
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📊 质量保障
|
||||
|
||||
### 已实现的质量措施
|
||||
|
||||
1. **JSON Schema 验证**:
|
||||
- 所有LLM输出必须通过Schema验证
|
||||
- 不合格的输出会被拒绝
|
||||
|
||||
2. **错误处理**:
|
||||
- 单篇文献失败不影响整体任务
|
||||
- 详细错误日志记录
|
||||
|
||||
3. **进度追踪**:
|
||||
- 每10篇更新一次进度
|
||||
- 实时统计成功/冲突/失败数
|
||||
|
||||
4. **可追溯性**:
|
||||
- 记录原始LLM输出（`rawOutput`）
|
||||
- 记录Prompt版本（`promptVersion`）
|
||||
- 记录处理时间（`aiProcessedAt`）
|
||||
|
||||
---
|
||||
|
||||
## 🚀 测试步骤
|
||||
|
||||
### Step 1: 准备测试数据
|
||||
```
|
||||
使用现有测试文件:
|
||||
- PICOS: docs/.../测试案例的PICOS、纳入标准、排除标准.txt
|
||||
- Excel: docs/.../Test Cases.xlsx (199篇文献)
|
||||
```
|
||||
|
||||
### Step 2: 执行测试
|
||||
1. 启动后端: `cd backend && npm run dev`
|
||||
2. 启动前端: `cd frontend-v2 && npm run dev`
|
||||
3. 访问: `http://localhost:3001`
|
||||
4. 填写PICOS + 上传Excel
|
||||
5. 点击"开始AI初筛"
|
||||
6. **等待30-60分钟**（199篇×20秒）
|
||||
7. 查看审核工作台
|
||||
|
||||
### Step 3: 验证结果
|
||||
```bash
|
||||
cd backend
|
||||
node check-data.mjs
|
||||
```
|
||||
|
||||
**检查项**:
|
||||
- [ ] 所有文献都有筛选结果
|
||||
- [ ] 证据不再是"模拟证据"
|
||||
- [ ] 证据包含文献原文引用
|
||||
- [ ] 判断理由详细且符合逻辑
|
||||
- [ ] 冲突检测准确（conclusion不同）
|
||||
|
||||
---
|
||||
|
||||
## ⚠️ 注意事项
|
||||
|
||||
### API密钥配置
|
||||
确保环境变量已配置:
|
||||
```bash
|
||||
# .env
|
||||
DEEPSEEK_API_KEY=sk-xxxxx
|
||||
QWEN_API_KEY=sk-xxxxx
|
||||
```
|
||||
|
||||
### API限流
|
||||
- DeepSeek: 60 RPM（每分钟请求数）
|
||||
- Qwen: 60 RPM
|
||||
|
||||
**当前策略**: 串行处理，不会触发限流
|
||||
|
||||
### 成本估算
|
||||
- DeepSeek: ~$0.001/次 × 199 = **$0.20**
|
||||
- Qwen: ~$0.001/次 × 199 = **$0.20**
|
||||
- **总计**: **$0.40** / 次完整测试
|
||||
|
||||
---
|
||||
|
||||
## 💡 优化建议
|
||||
|
||||
### 短期优化（Week 2 - Day 4-5）
|
||||
1. **并发控制**: 改为3个并发（33分钟 → 11分钟）
|
||||
2. **进度显示**: 前端轮询显示进度百分比
|
||||
3. **错误重试**: 失败的文献自动重试1次
|
||||
|
||||
### 中期优化（Week 3）
|
||||
1. **消息队列**: 使用Bull Queue异步处理
|
||||
2. **批量优化**: 使用批量API接口（如果有）
|
||||
3. **缓存机制**: 相同文献不重复筛选
|
||||
|
||||
---
|
||||
|
||||
## 📁 相关文件
|
||||
|
||||
### 修改的文件
|
||||
- `backend/src/modules/asl/services/screeningService.ts` ⭐
|
||||
|
||||
### 依赖的文件（已存在）
|
||||
- `backend/src/modules/asl/services/llmScreeningService.ts`
|
||||
- `backend/src/modules/asl/schemas/screening.schema.ts`
|
||||
- `backend/prompts/asl/screening/v1.0.0-mvp.txt`
|
||||
- `backend/src/common/llm/adapters/LLMFactory.ts`
|
||||
|
||||
### 测试文件
|
||||
- `backend/scripts/test-llm-screening.ts`
|
||||
- `backend/scripts/test-samples/asl-test-literatures.json`
|
||||
|
||||
---
|
||||
|
||||
## 🎉 成果总结
|
||||
|
||||
### 已实现
|
||||
✅ 真实LLM调用替换Mock数据
|
||||
✅ 从项目读取PICOS标准
|
||||
✅ 双模型并行筛选
|
||||
✅ 冲突检测与标记
|
||||
✅ 完整的日志追踪
|
||||
✅ 错误处理机制
|
||||
|
||||
### 待优化
|
||||
⚠️ 处理时间较长（30-60分钟）
|
||||
⚠️ 串行处理（可改为并发）
|
||||
⚠️ 前端进度显示（需优化轮询频率）
|
||||
|
||||
---
|
||||
|
||||
## 🔗 参考文档
|
||||
|
||||
- [Prompt设计与测试完成报告](./2025-11-18-Prompt设计与测试完成报告.md)
|
||||
- [卒中数据泛化测试报告](./2025-11-18-卒中数据泛化测试报告.md)
|
||||
- [任务分解](../04-开发计划/03-任务分解.md)
|
||||
|
||||
---
|
||||
|
||||
**报告人**: AI Assistant
|
||||
**日期**: 2025-11-21
|
||||
**版本**: v1.0.0
|
||||
|
||||
|
||||
Reference in New Issue
Block a user