feat(aia): Complete AIA V2.0 with universal streaming capabilities
Major Changes: - Add StreamingService with OpenAI Compatible format - Upgrade Chat component V2 with Ant Design X integration - Implement AIA module with 12 intelligent agents - Update API routes to unified /api/v1 prefix - Update system documentation Backend (~1300 lines): - common/streaming: OpenAI Compatible adapter - modules/aia: 12 agents, conversation service, streaming integration - Update route versions (RVW, PKB to v1) Frontend (~3500 lines): - modules/aia: AgentHub + ChatWorkspace (100% prototype restoration) - shared/Chat: AIStreamChat, ThinkingBlock, useAIStream Hook - Update API endpoints to v1 Documentation: - AIA module status guide - Universal capabilities catalog - System overview updates - All module documentation sync Tested: Stream response verified, authentication working Status: AIA V2.0 core completed (85%)
This commit is contained in:
@@ -1,11 +1,10 @@
|
||||
# 数据库设计文档 - 工具B(病历结构化机器人)
|
||||
# <EFBFBD>唳旿摨栞挽霈⊥<EFBFBD>獢?- 撌亙<E6928C>B嚗<42><E59A97><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>箏膥鈭綽<E988AD>
|
||||
|
||||
> **璅∪<E79285>**: DC<44>唳旿皜<E697BF><E79A9C><EFBFBD>渡<EFBFBD> - 撌亙<E6928C>B
|
||||
> **<2A><>𧋦**: V2.0 (MVP)
|
||||
> **Schema**: `dc_schema`
|
||||
> **<2A>湔鰵<E6B994>交<EFBFBD>**: 2025-12-03
|
||||
> **状态**: ✅ MVP完成(已验证可用,真实数据测试通过)
|
||||
|
||||
> **<EFBFBD>嗆<EFBFBD>?*: <EFBFBD>?MVP摰峕<EFBFBD>嚗<EFBFBD>歇撉諹<EFBFBD><EFBFBD>舐鍂嚗𣬚<EFBFBD>摰墧㺭<EFBFBD>格<EFBFBD>霂閖<EFBFBD>朞<EFBFBD>嚗?
|
||||
---
|
||||
|
||||
## <20><> <20>桀<EFBFBD>
|
||||
@@ -13,49 +12,37 @@
|
||||
- [銝<><E98A9D><EFBFBD><EFBFBD>餈財(#銝<EFBFBD>璁<EFBFBD>膩)
|
||||
- [鈭䎚<EFBFBD><EFBFBD>chema霈曇恣<EFBFBD>笔<EFBFBD>](#鈭众chema霈曇恣<E69B87>笔<EFBFBD>)
|
||||
- [銝剹<EFBFBD><EFBFBD>㺭<EFBFBD>株”霈曇恣](#銝㗇㺭<E39787>株”霈曇恣)
|
||||
- [四、索引设计](#四索引设计)
|
||||
- [五、外键约束](#五外键约束)
|
||||
- [六、数据生命周期](#六数据生命周期)
|
||||
- [<EFBFBD>䜘<EFBFBD><EFBFBD>揣撘閗挽霈((#<23>𤤿揣撘閗挽霈?
|
||||
- [鈭𢛵<EFBFBD><EFBFBD><EFBFBD><EFBFBD>桃漲<EFBFBD>篏(#鈭𥪜<EFBFBD><EFBFBD>桃漲<EFBFBD>?
|
||||
- [<EFBFBD>准<EFBFBD><EFBFBD>㺭<EFBFBD>桃<EFBFBD><EFBFBD>賢𪂹<EFBFBD>篏(#<23>剜㺭<E5899C>桃<EFBFBD><E6A183>賢𪂹<E8B3A2>?
|
||||
|
||||
---
|
||||
|
||||
## 一、概述
|
||||
|
||||
## 銝<EFBFBD><EFBFBD><EFBFBD><EFBFBD>餈?
|
||||
### 1.1 霈曇恣<E69B87>格<EFBFBD>
|
||||
|
||||
工具B的数据库设计旨在支持:
|
||||
- ✅ 双大模型交叉验证的文本结构化
|
||||
- ✅ 大规模异步任务处理(1000+条记录)
|
||||
- ✅ 冲突检测与人工裁决
|
||||
- ✅ 预设模板管理与复用
|
||||
- ✅ 健康检查缓存优化
|
||||
|
||||
撌亙<EFBFBD>B<EFBFBD><EFBFBD>㺭<EFBFBD>桀<EFBFBD>霈曇恣<EFBFBD>典銁<EFBFBD>舀<EFBFBD>嚗?- <20>?<3F><>之璅∪<E79285>鈭文<E988AD>撉諹<E69289><E8ABB9><EFBFBD><EFBFBD><EFBFBD>祉<EFBFBD><E7A589><EFBFBD><EFBFBD>
|
||||
- <EFBFBD>?憭扯<E686AD>璅∪<E79285>甇乩遙<E4B9A9>∪<EFBFBD><E288AA><EFBFBD><EFBFBD>1000+<2B>∟扇敶𤏪<E695B6>
|
||||
- <EFBFBD>?<3F>脩<EFBFBD>璉<EFBFBD>瘚衤<E7989A>鈭箏極鋆<E6A5B5><E98B86>
|
||||
- <EFBFBD>?憸<>挽璅⊥踎蝞∠<E89D9E>銝𤾸<E98A9D><F0A4BEB8>?- <20>?<3F>亙熒璉<E78692><E79289>亦<EFBFBD>摮䀝<E691AE><E4809D>?
|
||||
### 1.2 銵典<E98AB5>蝟餅<E89D9F>餉<EFBFBD>
|
||||
|
||||
```
|
||||
dc_schema ✅ 已创建并运行中
|
||||
├── dc_health_checks [健康检查缓存] ✅ 运行正常
|
||||
├── dc_templates [预设模板] ✅ 3个预设模板可用
|
||||
├── dc_extraction_tasks [提取任务] ✅ 已完成多个任务
|
||||
│ └── dc_extraction_items [提取记录] (1:N) ✅ 双模型结果正常保存
|
||||
```
|
||||
dc_schema <EFBFBD>?撌脣<E6928C>撱箏僎餈鞱<E9A488>銝?<3F>鎿<EFBFBD><E98EBF><EFBFBD> dc_health_checks [<5B>亙熒璉<E78692><E79289>亦<EFBFBD>摮𤊓 <20>?餈鞱<E9A488>甇<EFBFBD>虜
|
||||
<EFBFBD>鎿<EFBFBD><EFBFBD><EFBFBD> dc_templates [憸<>挽璅⊥踎] <20>?3銝芷<E98A9D>霈暹芋<E69AB9>踹虾<E8B8B9>?<3F>鎿<EFBFBD><E98EBF><EFBFBD> dc_extraction_tasks [<5B>𣂼<EFBFBD>隞餃𦛚] <20>?撌脣<E6928C><E884A3>𣂼<EFBFBD>銝芯遙<E88AAF>?<3F>? <20>婙<EFBFBD><E5A999><EFBFBD> dc_extraction_items [<5B>𣂼<EFBFBD>霈啣<E99C88>] (1:N) <20>?<3F>峕芋<E5B395>讠<EFBFBD><E8AEA0>𨀣迤撣訾<E692A3>摮?```
|
||||
|
||||
**✅ MVP完成状态(2025-12-03)**:
|
||||
- 所有表正常工作,已处理多个真实任务
|
||||
- 3个预设模板:肺癌病理报告、糖尿病入院记录、高血压门诊病历
|
||||
- 真实测试:9条病理数据提取成功,100%成功率
|
||||
- 双模型结果:resultA、resultB、finalResult字段正常保存
|
||||
**<2A>?MVP摰峕<E691B0><E5B395>嗆<EFBFBD><E59786><EFBFBD>2025-12-03嚗?*嚗?- <20><><EFBFBD>㕑”甇<E2809D>虜撌乩<E6928C>嚗<EFBFBD>歇憭<E6AD87><E686AD>憭帋葵<E5B88B>笔<EFBFBD>隞餃𦛚
|
||||
- 3銝芷<E98A9D>霈暹芋<E69AB9>選<EFBFBD><E981B8>箇<EFBFBD><E7AE87><EFBFBD><EFBFBD><EFBFBD>亙<EFBFBD><E4BA99><EFBFBD><EFBFBD>撠輻<E692A0><E8BCBB>仿堺霈啣<E99C88><E595A3><EFBFBD><EFBFBD>銵<EFBFBD><E98AB5>钅秄霂羓<E99C82><E7BE93>?- <20>笔<EFBFBD>瘚贝<E7989A>嚗?<3F>∠<EFBFBD><E288A0><EFBFBD>㺭<EFBFBD>格<EFBFBD><E6A0BC>𡝗<EFBFBD><F0A19D97><EFBFBD><EFBFBD>100%<25>𣂼<EFBFBD><F0A382BC>?- <20>峕芋<E5B395>讠<EFBFBD><E8AEA0>頣<EFBFBD>resultA<74><41>esultB<74><42>inalResult摮埈挾甇<E68CBE>虜靽嘥<E99DBD>
|
||||
- Token蝏蠘恣嚗魩otalTokens摮埈挾甇<E68CBE>虜蝝臬<E89D9D>
|
||||
- <20>脩<EFBFBD>璉<EFBFBD>瘚页<E7989A>conflictFields<64>啁<EFBFBD>甇<EFBFBD>虜撌乩<E6928C>
|
||||
- 撉諹<E69289><E8ABB9>𡁏𧋦嚗䫤backend/scripts/check-task-progress.mjs`
|
||||
|
||||
### 1.3 <20><><EFBFBD>舀<EFBFBD>
|
||||
|
||||
- **数据库**: PostgreSQL 15
|
||||
- **<2A>唳旿摨?*: PostgreSQL 15
|
||||
- **ORM**: Prisma 6
|
||||
- **Schema<6D>𠉛氖**: `dc_schema`嚗<>𡠺蝡见𦶢<E8A781>滨征<E6BBA8>湛<EFBFBD>
|
||||
- **JSON字段**: 使用JSONB类型(高性能查询)
|
||||
|
||||
- **JSON摮埈挾**: 雿輻鍂JSONB蝐餃<EFBFBD>嚗<EFBFBD><EFBFBD><EFBFBD>扯<EFBFBD><EFBFBD>亥砭嚗?
|
||||
---
|
||||
|
||||
## 鈭䎚<E988AD><E48E9A>chema霈曇恣<E69B87>笔<EFBFBD>
|
||||
@@ -68,10 +55,9 @@ CREATE TABLE "dc_schema"."dc_health_checks" (...);
|
||||
CREATE TABLE "dc_schema"."dc_extraction_tasks" (...);
|
||||
```
|
||||
|
||||
**优势**:
|
||||
- ✅ 与其他模块完全隔离(platform_schema、asl_schema等)
|
||||
- ✅ 数据安全,避免误操作
|
||||
- ✅ 便于模块化管理和迁移
|
||||
**隡睃飵**嚗?- <20>?銝𤾸<E98A9D>隞𡝗芋<F0A19D97>堒<EFBFBD><E5A092>券<EFBFBD>蝳鳴<E89DB3>platform_schema<6D><61>sl_schema蝑㚁<E89D91>
|
||||
- <EFBFBD>?<3F>唳旿摰匧<E691B0>嚗屸<E59A97><E5B1B8>滩秤<E6BBA9>滢<EFBFBD>
|
||||
- <EFBFBD>?靘蹂<E99D98>璅∪<E79285><E288AA>𣇉恣<F0A38789><E681A3><EFBFBD>餈<EFBFBD>宏
|
||||
|
||||
### 2.2 <20>賢<EFBFBD>閫<EFBFBD><E996AB>
|
||||
|
||||
@@ -79,24 +65,23 @@ CREATE TABLE "dc_schema"."dc_extraction_tasks" (...);
|
||||
|------|------|------|
|
||||
| **銵典<E98AB5><E585B8>滨<EFBFBD>** | `dc_` | `dc_extraction_tasks` |
|
||||
| **摮埈挾<E59F88>賢<EFBFBD>** | snake_case | `user_id`, `source_file_key` |
|
||||
| **时间戳** | 统一后缀 | `created_at`, `started_at` |
|
||||
| **<EFBFBD>園𡢿<EFBFBD>?* | 蝏煺<E89D8F><E785BA>𡒊<EFBFBD> | `created_at`, `started_at` |
|
||||
| **憭㚚睸** | 摰硺<E691B0><E7A1BA>䇭id | `task_id`, `user_id` |
|
||||
|
||||
### 2.3 JSONB摮埈挾雿輻鍂<E8BCBB>箸艶
|
||||
|
||||
| 摮埈挾 | 蝐餃<E89D90> | <20>笔<EFBFBD> |
|
||||
|------|------|------|
|
||||
| `target_fields` | JSONB | 灵活的字段配置 |
|
||||
| `result_a/result_b` | JSONB | 动态提取结果 |
|
||||
| `final_result` | JSONB | 最终裁决结果 |
|
||||
| `target_fields` | JSONB | <EFBFBD>菜暑<EFBFBD><EFBFBD><EFBFBD>畾菟<EFBFBD>蝵?|
|
||||
| `result_a/result_b` | JSONB | <EFBFBD>冽<EFBFBD><EFBFBD><EFBFBD><EFBFBD>𣇉<EFBFBD><EFBFBD>?|
|
||||
| `final_result` | JSONB | <EFBFBD><EFBFBD>蝏<EFBFBD><EFBFBD><EFBFBD>喟<EFBFBD><EFBFBD>?|
|
||||
|
||||
---
|
||||
|
||||
## 銝剹<E98A9D><E589B9>㺭<EFBFBD>株”霈曇恣
|
||||
|
||||
### 3.1 dc_health_checks(健康检查缓存表)
|
||||
|
||||
**用途**: 缓存健康检查结果,避免重复计算
|
||||
### 3.1 dc_health_checks嚗<EFBFBD><EFBFBD>摨瑟<EFBFBD><EFBFBD>亦<EFBFBD>摮䁅”嚗?
|
||||
**<2A>券<EFBFBD>?*: 蝻枏<E89DBB><E69E8F>亙熒璉<E78692><E79289>亦<EFBFBD><E4BAA6>頣<EFBFBD><E9A0A3>踹<EFBFBD><E8B8B9>滚<EFBFBD>霈∠<E99C88>
|
||||
|
||||
```sql
|
||||
CREATE TABLE "dc_schema"."dc_health_checks" (
|
||||
@@ -111,46 +96,39 @@ CREATE TABLE "dc_schema"."dc_health_checks" (
|
||||
"total_rows" INTEGER NOT NULL,
|
||||
"estimated_tokens" INTEGER NOT NULL,
|
||||
|
||||
-- 检查结果
|
||||
"status" TEXT NOT NULL, -- 'good' | 'bad'
|
||||
-- 璉<EFBFBD><EFBFBD>亦<EFBFBD><EFBFBD>? "status" TEXT NOT NULL, -- 'good' | 'bad'
|
||||
"message" TEXT NOT NULL,
|
||||
|
||||
"created_at" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP
|
||||
);
|
||||
```
|
||||
|
||||
**字段说明**:
|
||||
|
||||
**摮埈挾霂湔<EFBFBD>**嚗?
|
||||
| 摮埈挾 | 蝐餃<E89D90> | 霂湔<E99C82> | 蝷箔<E89DB7> |
|
||||
|------|------|------|------|
|
||||
| `id` | TEXT | UUID銝駁睸 | `uuid()` |
|
||||
| `user_id` | TEXT | <20>冽<EFBFBD>ID | `user-123` |
|
||||
| `file_name` | TEXT | 文件名 | `患者数据.xlsx` |
|
||||
| `file_name` | TEXT | <EFBFBD><EFBFBD>辣<EFBFBD>?| `<60><><EFBFBD><EFBFBD>㺭<EFBFBD>?xlsx` |
|
||||
| `column_name` | TEXT | 璉<><E79289>亦<EFBFBD><E4BAA6>堒<EFBFBD> | `<60><><EFBFBD><EFBFBD><EFBFBD>𧋦` |
|
||||
| `empty_rate` | DOUBLE | 蝛箏<E89D9B>潛<EFBFBD> (0-1) | 0.15 (15%) |
|
||||
| `avg_length` | DOUBLE | 撟喳<E6929F><E596B3><EFBFBD>𧋦<EFBFBD>踹漲 | 256.8 |
|
||||
| `total_rows` | INT | 总行数 | 500 |
|
||||
| `estimated_tokens` | INT | 预估Token数 | 150000 |
|
||||
| `status` | TEXT | 健康状态 | `good` / `bad` |
|
||||
| `total_rows` | INT | <EFBFBD>餉<EFBFBD><EFBFBD>?| 500 |
|
||||
| `estimated_tokens` | INT | 憸<EFBFBD>摯Token<EFBFBD>?| 150000 |
|
||||
| `status` | TEXT | <EFBFBD>亙熒<EFBFBD>嗆<EFBFBD>?| `good` / `bad` |
|
||||
| `message` | TEXT | <20>鞟內靽⊥<E99DBD> | `<60>亙熒摨西<E691A8>憟窯 |
|
||||
|
||||
**索引**:
|
||||
```sql
|
||||
**蝝W<EFBFBD>**嚗?```sql
|
||||
CREATE INDEX "dc_health_checks_user_id_file_name_idx"
|
||||
ON "dc_schema"."dc_health_checks"("user_id", "file_name");
|
||||
```
|
||||
|
||||
**业务规则**:
|
||||
- 空值率 > 80% → `status = 'bad'`
|
||||
- 平均长度 < 10 → `status = 'bad'`
|
||||
- 缓存有效期:24小时(应用层实现)
|
||||
|
||||
**銝𡁜𦛚閫<EFBFBD><EFBFBD>**嚗?- 蝛箏<E89D9B>潛<EFBFBD> > 80% <20>?`status = 'bad'`
|
||||
- 撟喳<E6929F><E596B3>踹漲 < 10 <20>?`status = 'bad'`
|
||||
- 蝻枏<E89DBB><E69E8F>㗇<EFBFBD><E39787><EFBFBD><EFBFBD>24撠𤩺𧒄嚗<F0A79284><E59A97><EFBFBD>典<EFBFBD>摰䂿緵嚗?
|
||||
---
|
||||
|
||||
### 3.2 dc_templates(预设模板表)
|
||||
|
||||
**用途**: 存储疾病类型的预设提取模板
|
||||
|
||||
### 3.2 dc_templates嚗<EFBFBD><EFBFBD>霈暹芋<EFBFBD>輯”嚗?
|
||||
**<2A>券<EFBFBD>?*: 摮睃<E691AE><E79D83>曄<EFBFBD>蝐餃<E89D90><E9A483><EFBFBD><EFBFBD>霈暹<E99C88><E69AB9>𡝗芋<F0A19D97>?
|
||||
```sql
|
||||
CREATE TABLE "dc_schema"."dc_templates" (
|
||||
"id" TEXT NOT NULL PRIMARY KEY,
|
||||
@@ -168,18 +146,16 @@ CREATE TABLE "dc_schema"."dc_templates" (
|
||||
);
|
||||
```
|
||||
|
||||
**字段说明**:
|
||||
|
||||
**摮埈挾霂湔<EFBFBD>**嚗?
|
||||
| 摮埈挾 | 蝐餃<E89D90> | 霂湔<E99C82> | 蝷箔<E89DB7> |
|
||||
|------|------|------|------|
|
||||
| `disease_type` | TEXT | <20>曄<EFBFBD>蝐餃<E89D90> | `lung_cancer` |
|
||||
| `report_type` | TEXT | <20>亙<EFBFBD>蝐餃<E89D90> | `pathology` |
|
||||
| `display_name` | TEXT | <20>曄內<E69B84>滨妍 | `<EFBFBD>箇<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>亙<EFBFBD>` |
|
||||
| `fields` | JSONB | 提取字段配置 | 见下方示例 |
|
||||
| `prompt_template` | TEXT | Prompt模板 | `请从以下病理报告中提取...` |
|
||||
| `fields` | JSONB | <EFBFBD>𣂼<EFBFBD>摮埈挾<EFBFBD>滨蔭 | 閫<><E996AB><EFBFBD>寧內靘?|
|
||||
| `prompt_template` | TEXT | Prompt璅⊥踎 | `霂瑚<EFBFBD>隞乩<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>亙<EFBFBD>銝剜<EFBFBD><EFBFBD>?..` |
|
||||
|
||||
**fields字段结构**:
|
||||
```json
|
||||
**fields摮埈挾蝏𤘪<EFBFBD>**嚗?```json
|
||||
[
|
||||
{
|
||||
"name": "<22><><EFBFBD>蝐餃<E89D90>",
|
||||
@@ -188,23 +164,20 @@ CREATE TABLE "dc_schema"."dc_templates" (
|
||||
},
|
||||
{
|
||||
"name": "<22><><EFBFBD>蝔见漲",
|
||||
"desc": "高/中/低分化",
|
||||
"desc": "擃?銝?雿𤾸<E99BBF><F0A4BEB8>?,
|
||||
"width": "w-32"
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
**唯一约束**:
|
||||
```sql
|
||||
**<EFBFBD>臭<EFBFBD>蝥行<EFBFBD>**嚗?```sql
|
||||
UNIQUE ("disease_type", "report_type")
|
||||
```
|
||||
同一疾病+报告类型组合只能有一个模板
|
||||
|
||||
<EFBFBD>䔶<EFBFBD><EFBFBD>曄<EFBFBD>+<2B>亙<EFBFBD>蝐餃<E89D90>蝏<EFBFBD><E89D8F><EFBFBD>芾<EFBFBD><E88ABE>劐<EFBFBD>銝芣芋<E88AA3>?
|
||||
---
|
||||
|
||||
### 3.3 dc_extraction_tasks(提取任务表)
|
||||
|
||||
**用途**: 管理批量提取任务,追踪进度和成本
|
||||
### 3.3 dc_extraction_tasks嚗<EFBFBD><EFBFBD><EFBFBD>碶遙<EFBFBD>∟”嚗?
|
||||
**<2A>券<EFBFBD>?*: 蝞∠<E89D9E><E288A0>寥<EFBFBD><E5AFA5>𣂼<EFBFBD>隞餃𦛚嚗諹蕭頦芾<E9A0A6>摨血<E691A8><E8A180>鞉𧋦
|
||||
|
||||
```sql
|
||||
CREATE TABLE "dc_schema"."dc_extraction_tasks" (
|
||||
@@ -219,12 +192,10 @@ CREATE TABLE "dc_schema"."dc_extraction_tasks" (
|
||||
"report_type" TEXT NOT NULL,
|
||||
"target_fields" JSONB NOT NULL,
|
||||
|
||||
-- 双模型配置
|
||||
"model_a" TEXT NOT NULL DEFAULT 'deepseek-v3',
|
||||
-- <EFBFBD>峕芋<EFBFBD>钅<EFBFBD>蝵? "model_a" TEXT NOT NULL DEFAULT 'deepseek-v3',
|
||||
"model_b" TEXT NOT NULL DEFAULT 'qwen3-72b',
|
||||
|
||||
-- 任务状态
|
||||
"status" TEXT NOT NULL DEFAULT 'pending',
|
||||
-- 隞餃𦛚<EFBFBD>嗆<EFBFBD>? "status" TEXT NOT NULL DEFAULT 'pending',
|
||||
"total_count" INTEGER NOT NULL DEFAULT 0,
|
||||
"processed_count" INTEGER NOT NULL DEFAULT 0,
|
||||
"clean_count" INTEGER NOT NULL DEFAULT 0,
|
||||
@@ -238,47 +209,41 @@ CREATE TABLE "dc_schema"."dc_extraction_tasks" (
|
||||
-- <20>躰秤靽⊥<E99DBD>
|
||||
"error" TEXT,
|
||||
|
||||
-- 时间戳
|
||||
"created_at" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
-- <EFBFBD>園𡢿<EFBFBD>? "created_at" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||||
"started_at" TIMESTAMP(3),
|
||||
"completed_at" TIMESTAMP(3)
|
||||
);
|
||||
```
|
||||
|
||||
**字段说明**:
|
||||
|
||||
**摮埈挾霂湔<EFBFBD>**嚗?
|
||||
| 摮埈挾 | 蝐餃<E89D90> | 霂湔<E99C82> | 蝷箔<E89DB7> |
|
||||
|------|------|------|------|
|
||||
| `source_file_key` | TEXT | Storage頝臬<E9A09D> | `uploads/user123/data.xlsx` |
|
||||
| `text_column` | TEXT | <20><>𧋦<EFBFBD>堒<EFBFBD> | `<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>𧋦` |
|
||||
| `target_fields` | JSONB | <20>𣂼<EFBFBD>摮埈挾 | `[{name, desc}]` |
|
||||
| `status` | TEXT | 任务状态 | `pending/processing/completed/failed` |
|
||||
| `status` | TEXT | 隞餃𦛚<EFBFBD>嗆<EFBFBD>?| `pending/processing/completed/failed` |
|
||||
| `total_count` | INT | <20>餉扇敶閙㺭 | 500 |
|
||||
| `processed_count` | INT | 撌脣<E6928C><E884A3><EFBFBD>㺭 | 250 |
|
||||
| `clean_count` | INT | 銝<><E98A9D>湔㺭 | 200 |
|
||||
| `conflict_count` | INT | 冲突数 | 45 |
|
||||
| `failed_count` | INT | 失败数 | 5 |
|
||||
| `total_tokens` | INT | 总Token数 | 150000 |
|
||||
| `total_cost` | DOUBLE | 总成本($) | 0.27 |
|
||||
| `conflict_count` | INT | <EFBFBD>脩<EFBFBD><EFBFBD>?| 45 |
|
||||
| `failed_count` | INT | 憭梯揖<EFBFBD>?| 5 |
|
||||
| `total_tokens` | INT | <EFBFBD>蓉oken<EFBFBD>?| 150000 |
|
||||
| `total_cost` | DOUBLE | <EFBFBD>餅<EFBFBD><EFBFBD>?$) | 0.27 |
|
||||
|
||||
**状态流转**:
|
||||
```
|
||||
pending → processing → completed
|
||||
→ failed
|
||||
**<EFBFBD>嗆<EFBFBD><EFBFBD><EFBFBD>頧?*嚗?```
|
||||
pending <20>?processing <20>?completed
|
||||
<20>?failed
|
||||
```
|
||||
|
||||
**索引**:
|
||||
```sql
|
||||
**蝝W<EFBFBD>**嚗?```sql
|
||||
CREATE INDEX "dc_extraction_tasks_user_id_status_idx"
|
||||
ON "dc_schema"."dc_extraction_tasks"("user_id", "status");
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 3.4 dc_extraction_items(提取记录表)
|
||||
|
||||
**用途**: 存储每条记录的双模型提取结果和冲突状态
|
||||
|
||||
### 3.4 dc_extraction_items嚗<EFBFBD><EFBFBD><EFBFBD>𤥁扇敶閗”嚗?
|
||||
**<2A>券<EFBFBD>?*: 摮睃<E691AE>瘥𤩺辺霈啣<E99C88><E595A3><EFBFBD><EFBFBD>璅∪<E79285><E288AA>𣂼<EFBFBD>蝏𤘪<E89D8F><F0A498AA><EFBFBD><EFBFBD>蝒<EFBFBD>𠶖<EFBFBD>?
|
||||
```sql
|
||||
CREATE TABLE "dc_schema"."dc_extraction_items" (
|
||||
"id" TEXT NOT NULL PRIMARY KEY,
|
||||
@@ -288,16 +253,13 @@ CREATE TABLE "dc_schema"."dc_extraction_items" (
|
||||
"row_index" INTEGER NOT NULL,
|
||||
"original_text" TEXT NOT NULL,
|
||||
|
||||
-- 双模型结果
|
||||
"result_a" JSONB,
|
||||
-- <EFBFBD>峕芋<EFBFBD>讠<EFBFBD><EFBFBD>? "result_a" JSONB,
|
||||
"result_b" JSONB,
|
||||
|
||||
-- 冲突检测
|
||||
"status" TEXT NOT NULL DEFAULT 'pending',
|
||||
-- <EFBFBD>脩<EFBFBD>璉<EFBFBD>瘚? "status" TEXT NOT NULL DEFAULT 'pending',
|
||||
"conflict_fields" TEXT[] DEFAULT ARRAY[]::TEXT[],
|
||||
|
||||
-- 最终结果
|
||||
"final_result" JSONB,
|
||||
-- <EFBFBD><EFBFBD>蝏<EFBFBD><EFBFBD><EFBFBD>? "final_result" JSONB,
|
||||
|
||||
-- Token蝏蠘恣
|
||||
"tokens_a" INTEGER NOT NULL DEFAULT 0,
|
||||
@@ -316,78 +278,63 @@ CREATE TABLE "dc_schema"."dc_extraction_items" (
|
||||
);
|
||||
```
|
||||
|
||||
**字段说明**:
|
||||
|
||||
**摮埈挾霂湔<EFBFBD>**嚗?
|
||||
| 摮埈挾 | 蝐餃<E89D90> | 霂湔<E99C82> | 蝷箔<E89DB7> |
|
||||
|------|------|------|------|
|
||||
| `row_index` | INT | Excel銵<6C>噡 | 5 |
|
||||
| `original_text` | TEXT | 原始病历文本 | `患者,男,45岁...` |
|
||||
| `original_text` | TEXT | <EFBFBD>笔<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>𧋦 | `<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>瘀<EFBFBD>45撗?..` |
|
||||
| `result_a` | JSONB | DeepSeek蝏𤘪<E89D8F> | `{"<22>輻𠈔憭批<E686AD>": "3cm"}` |
|
||||
| `result_b` | JSONB | Qwen蝏𤘪<E89D8F> | `{"<22>輻𠈔憭批<E686AD>": "3.0cm"}` |
|
||||
| `status` | TEXT | 处理状态 | `clean/conflict/resolved/failed` |
|
||||
| `status` | TEXT | 憭<EFBFBD><EFBFBD><EFBFBD>嗆<EFBFBD>?| `clean/conflict/resolved/failed` |
|
||||
| `conflict_fields` | TEXT[] | <20>脩<EFBFBD>摮埈挾<E59F88>𡑒” | `["<22>輻𠈔憭批<E686AD>"]` |
|
||||
| `final_result` | JSONB | 最终裁决结果 | `{"肿瘤大小": "3cm"}` |
|
||||
| `final_result` | JSONB | <EFBFBD><EFBFBD>蝏<EFBFBD><EFBFBD><EFBFBD>喟<EFBFBD><EFBFBD>?| `{"<22>輻𠈔憭批<E686AD>": "3cm"}` |
|
||||
|
||||
**result_a/result_b结构示例**:
|
||||
```json
|
||||
**result_a/result_b蝏𤘪<EFBFBD>蝷箔<EFBFBD>**嚗?```json
|
||||
{
|
||||
"病理类型": "浸润性腺癌",
|
||||
"分化程度": "中分化",
|
||||
"<22><><EFBFBD>蝐餃<E89D90>": "瘚豢隋<E8B1A2>扯<EFBFBD><E689AF>?,
|
||||
"<22><><EFBFBD>蝔见漲": "銝剖<E98A9D><E58996>?,
|
||||
"<22>輻𠈔憭批<E686AD>": "3cm",
|
||||
"淋巴结转移": "无"
|
||||
"瘛见毀蝏栞蓮蝘?: "<22>?
|
||||
}
|
||||
```
|
||||
|
||||
**状态说明**:
|
||||
- `pending`: 等待处理
|
||||
- `clean`: 双模型结果一致
|
||||
- `conflict`: 存在冲突,需人工裁决
|
||||
- `resolved`: 冲突已解决
|
||||
- `failed`: 提取失败
|
||||
**<EFBFBD>嗆<EFBFBD><EFBFBD>秩<EFBFBD>?*嚗?- `pending`: 蝑匧<E89D91>憭<EFBFBD><E686AD>
|
||||
- `clean`: <20>峕芋<E5B395>讠<EFBFBD><E8AEA0>靝<EFBFBD><E99D9D>?- `conflict`: 摮睃銁<E79D83>脩<EFBFBD>嚗屸<E59A97>鈭箏極鋆<E6A5B5><E98B86>
|
||||
- `resolved`: <20>脩<EFBFBD>撌脰圾<E884B0>?- `failed`: <20>𣂼<EFBFBD>憭梯揖
|
||||
|
||||
**索引**:
|
||||
```sql
|
||||
**蝝W<EFBFBD>**嚗?```sql
|
||||
CREATE INDEX "dc_extraction_items_task_id_status_idx"
|
||||
ON "dc_schema"."dc_extraction_items"("task_id", "status");
|
||||
```
|
||||
|
||||
**外键约束**:
|
||||
- `ON DELETE CASCADE`: 删除任务时自动删除所有记录
|
||||
|
||||
**憭㚚睸蝥行<EFBFBD>**嚗?- `ON DELETE CASCADE`: <20>𣳇膄隞餃𦛚<E9A483>嗉䌊<E59789>典<EFBFBD><E585B8>斗<EFBFBD><E69697>㕑扇敶?
|
||||
---
|
||||
|
||||
## 四、索引设计
|
||||
|
||||
## <EFBFBD>䜘<EFBFBD><EFBFBD>揣撘閗挽霈?
|
||||
### 4.1 蝝W<E89D9D><EFBCB7>𡑒”
|
||||
|
||||
| 表名 | 索引字段 | 类型 | 用途 |
|
||||
| 銵典<EFBFBD> | 蝝W<E89D9D>摮埈挾 | 蝐餃<E89D90> | <20>券<EFBFBD>?|
|
||||
|------|---------|------|------|
|
||||
| `dc_health_checks` | `(user_id, file_name)` | 复合 | 查询用户的历史检查 |
|
||||
| `dc_health_checks` | `(user_id, file_name)` | 憭滚<EFBFBD> | <20>亥砭<E4BAA5>冽<EFBFBD><E586BD><EFBFBD><EFBFBD><EFBFBD>脫<EFBFBD><E884AB>?|
|
||||
| `dc_templates` | `(disease_type, report_type)` | <20>臭<EFBFBD> | <20>脫迫<E884AB>滚<EFBFBD>璅⊥踎 |
|
||||
| `dc_extraction_tasks` | `(user_id, status)` | 复合 | 查询用户的任务列表 |
|
||||
| `dc_extraction_items` | `(task_id, status)` | 复合 | 查询任务的记录列表 |
|
||||
| `dc_extraction_tasks` | `(user_id, status)` | 憭滚<EFBFBD> | <20>亥砭<E4BAA5>冽<EFBFBD><E586BD><EFBFBD>遙<EFBFBD>∪<EFBFBD>銵?|
|
||||
| `dc_extraction_items` | `(task_id, status)` | 憭滚<EFBFBD> | <20>亥砭隞餃𦛚<E9A483><F0A69B9A>扇敶訫<E695B6>銵?|
|
||||
|
||||
### 4.2 <20>扯<EFBFBD><E689AF><EFBFBD><EFBFBD>
|
||||
|
||||
**查询优化**:
|
||||
```sql
|
||||
-- 高效查询:利用索引
|
||||
SELECT * FROM dc_extraction_tasks
|
||||
**<EFBFBD>亥砭隡睃<EFBFBD>**嚗?```sql
|
||||
-- 擃䀹<E69383><E480B9>亥砭嚗𡁜⏚<F0A1819C>函揣撘?SELECT * FROM dc_extraction_tasks
|
||||
WHERE user_id = 'user123' AND status = 'processing';
|
||||
|
||||
-- 高效查询:利用索引
|
||||
SELECT * FROM dc_extraction_items
|
||||
-- 擃䀹<E69383><E480B9>亥砭嚗𡁜⏚<F0A1819C>函揣撘?SELECT * FROM dc_extraction_items
|
||||
WHERE task_id = 'task456' AND status = 'conflict';
|
||||
```
|
||||
|
||||
**避免全表扫描**:
|
||||
- ✅ 始终在WHERE子句中包含索引字段
|
||||
- ✅ 使用`status`字段过滤可以显著减少扫描行数
|
||||
**<EFBFBD>踹<EFBFBD><EFBFBD>刻”<EFBFBD>急<EFBFBD>**嚗?- <20>?憪讠<E686AA><E8AEA0>汾HERE摮𣂼蘂銝剖<E98A9D><E58996>怎揣撘訫<E69298>畾?- <20>?雿輻鍂`status`摮埈挾餈<E68CBE>誘<EFBFBD>臭誑<E887AD>曇<EFBFBD><E69B87>誩<EFBFBD><E8AAA9>急<EFBFBD>銵峕㺭
|
||||
|
||||
---
|
||||
|
||||
## 五、外键约束
|
||||
|
||||
## 鈭𢛵<EFBFBD><EFBFBD><EFBFBD><EFBFBD>桃漲<EFBFBD>?
|
||||
### 5.1 蝥扯<E89DA5><E689AF>𣳇膄
|
||||
|
||||
```sql
|
||||
@@ -398,46 +345,34 @@ REFERENCES "dc_schema"."dc_extraction_tasks"("id")
|
||||
ON DELETE CASCADE;
|
||||
```
|
||||
|
||||
**行为**:
|
||||
- 删除任务 → 自动删除所有关联的提取记录
|
||||
- 保证数据一致性
|
||||
|
||||
### 5.2 无外键的表
|
||||
|
||||
- `dc_health_checks`: 独立表,无外键
|
||||
- `dc_templates`: 独立表,无外键
|
||||
- `dc_extraction_tasks`: 无外键(user_id仅为标识,不强制关联)
|
||||
|
||||
**原因**:
|
||||
- ✅ 减少跨Schema依赖
|
||||
- ✅ 提高模块独立性
|
||||
- ✅ 简化迁移和回滚
|
||||
**銵䔶蛹**嚗?- <20>𣳇膄隞餃𦛚 <20>?<3F>芸𢆡<E88AB8>𣳇膄<F0A3B387><E88684><EFBFBD>匧<EFBFBD><E58CA7>𠉛<EFBFBD><F0A0899B>𣂼<EFBFBD>霈啣<E99C88>
|
||||
- 靽肽<E99DBD><E882BD>唳旿銝<E697BF><E98A9D>湔<EFBFBD>?
|
||||
### 5.2 <20>惩<EFBFBD><E683A9>桃<EFBFBD>銵?
|
||||
- `dc_health_checks`: <20>祉<EFBFBD>銵剁<E98AB5><E58981>惩<EFBFBD><E683A9>?- `dc_templates`: <20>祉<EFBFBD>銵剁<E98AB5><E58981>惩<EFBFBD><E683A9>?- `dc_extraction_tasks`: <20>惩<EFBFBD><E683A9>殷<EFBFBD>user_id隞<64>蛹<EFBFBD><E89BB9><EFBFBD>嚗䔶<E59A97>撘箏<E69298><E7AE8F>唾<EFBFBD>嚗?
|
||||
**<2A>笔<EFBFBD>**嚗?- <20>?<3F>誩<EFBFBD>頝沒chema靘肽<E99D98>
|
||||
- <20>?<3F>鞾<EFBFBD>璅∪<E79285><E288AA>祉<EFBFBD><E7A589>?- <20>?蝞<><E89D9E>𤥁<EFBFBD>蝘餃<E89D98><E9A483>墧<EFBFBD>
|
||||
|
||||
---
|
||||
|
||||
## 六、数据生命周期
|
||||
|
||||
## <EFBFBD>准<EFBFBD><EFBFBD>㺭<EFBFBD>桃<EFBFBD><EFBFBD>賢𪂹<EFBFBD>?
|
||||
### 6.1 <20>唳旿靽萘<E99DBD>蝑𣇉裦
|
||||
|
||||
| 銵典<E98AB5> | 靽萘<E99DBD><E89098>園𡢿 | 皜<><E79A9C>蝑𣇉裦 |
|
||||
|------|---------|---------|
|
||||
| `dc_health_checks` | 7天 | 定期清理旧记录 |
|
||||
| `dc_health_checks` | 7憭?| 摰𡁏<E691B0>皜<EFBFBD><E79A9C><EFBFBD>扯扇敶?|
|
||||
| `dc_templates` | 瘞訾<E7989E> | <20>见𢆡蝞∠<E89D9E> |
|
||||
| `dc_extraction_tasks` | 90天 | 归档后删除 |
|
||||
| `dc_extraction_items` | 90天 | 随任务删除 |
|
||||
| `dc_extraction_tasks` | 90憭?| 敶埝﹝<E59F9D>𤾸<EFBFBD><F0A4BEB8>?|
|
||||
| `dc_extraction_items` | 90憭?| <20>譍遙<E8AD8D>∪<EFBFBD><E288AA>?|
|
||||
|
||||
### 6.2 敶埝﹝蝑𣇉裦
|
||||
|
||||
**大任务归档** (> 1000条记录):
|
||||
1. 任务完成后,导出结果到CSV/Excel
|
||||
**憭找遙<EFBFBD>∪<EFBFBD>獢?* (> 1000<30>∟扇敶?嚗?1. 隞餃𦛚摰峕<E691B0><E5B395>𠬍<EFBFBD>撖澆枂蝏𤘪<E89D8F><F0A498AA>蚓SV/Excel
|
||||
2. 銝𠹺<E98A9D><F0A0B9BA>訕torage嚗<65>偶銋<E581B6><E98A8B>摮矋<E691AE>
|
||||
3. 删除数据库记录(释放空间)
|
||||
|
||||
3. <20>𣳇膄<F0A3B387>唳旿摨栞扇敶𤏪<E695B6><F0A48FAA>𦠜𦆮蝛粹𡢿嚗?
|
||||
### 6.3 皜<><E79A9C><EFBFBD>𡁏𧋦嚗<F0A78BA6>內靘页<E99D98>
|
||||
|
||||
```typescript
|
||||
// 清理7天前的健康检查记录
|
||||
await prisma.dCHealthCheck.deleteMany({
|
||||
// 皜<><E79A9C>7憭拙<E686AD><E68B99><EFBFBD><EFBFBD>摨瑟<E691A8><E7919F>亥扇敶?await prisma.dCHealthCheck.deleteMany({
|
||||
where: {
|
||||
createdAt: {
|
||||
lt: new Date(Date.now() - 7 * 24 * 60 * 60 * 1000)
|
||||
@@ -456,8 +391,7 @@ const oldTasks = await prisma.dCExtractionTask.findMany({
|
||||
include: { items: true }
|
||||
});
|
||||
|
||||
// 导出后删除
|
||||
for (const task of oldTasks) {
|
||||
// 撖澆枂<EFBFBD>𤾸<EFBFBD><EFBFBD>?for (const task of oldTasks) {
|
||||
await exportTaskToStorage(task);
|
||||
await prisma.dCExtractionTask.delete({ where: { id: task.id } });
|
||||
}
|
||||
@@ -465,52 +399,39 @@ for (const task of oldTasks) {
|
||||
|
||||
---
|
||||
|
||||
## 七、数据安全
|
||||
|
||||
## 銝<EFBFBD><EFBFBD><EFBFBD>㺭<EFBFBD>桀<EFBFBD><EFBFBD>?
|
||||
### 7.1 PII靽脲擪
|
||||
|
||||
**敏感字段**:
|
||||
- `original_text`: 可能包含患者姓名、身份证号
|
||||
- `result_a/result_b/final_result`: 可能包含结构化的敏感信息
|
||||
**<EFBFBD>𤩺<EFBFBD>摮埈挾**嚗?- `original_text`: <20>航<EFBFBD><E888AA><EFBFBD>鉄<EFBFBD><E98984><EFBFBD><EFBFBD><EFBFBD><EFBFBD>溻<EFBFBD><E6BABB>澈隞質<E99A9E><E8B3AA>?- `result_a/result_b/final_result`: <20>航<EFBFBD><E888AA><EFBFBD>鉄蝏𤘪<E89D8F><F0A498AA>𣇉<EFBFBD><F0A38789>𤩺<EFBFBD>靽⊥<E99DBD>
|
||||
|
||||
**保护措施**:
|
||||
- ✅ 发送LLM前自动脱敏(PIIMaskUtil)
|
||||
- ✅ 数据库加密(PostgreSQL SSL)
|
||||
- ✅ 定期清理历史数据
|
||||
**靽脲擪<EFBFBD>芣鴌**嚗?- <20>?<3F>煾<EFBFBD><E785BE>LM<4C>滩䌊<E6BBA9>刻<EFBFBD><E588BB>𧶏<EFBFBD>PIIMaskUtil嚗?- <20>?<3F>唳旿摨枏<E691A8>撖<EFBFBD><E69296>PostgreSQL SSL嚗?- <20>?摰𡁏<E691B0>皜<EFBFBD><E79A9C><EFBFBD><EFBFBD>蟮<EFBFBD>唳旿
|
||||
|
||||
### 7.2 <20>冽<EFBFBD><E586BD>𠉛氖
|
||||
|
||||
**机制**:
|
||||
- 所有表包含`user_id`字段
|
||||
**<EFBFBD>箏<EFBFBD>**嚗?- <20><><EFBFBD>㕑”<E39591><E2809D>鉄`user_id`摮埈挾
|
||||
- 摨𠉛鍂撅<E98D82>撩<EFBFBD>嗉<EFBFBD>皛歹<E79A9B>`WHERE user_id = currentUserId`
|
||||
- 永不跨用户查询
|
||||
|
||||
- 瘞訾<E7989E>頝函鍂<E587BD>瑟䰻霂?
|
||||
---
|
||||
|
||||
## 八、附录
|
||||
|
||||
## <EFBFBD>怒<EFBFBD><EFBFBD><EFBFBD>敶?
|
||||
### 8.1 摰峕㟲Schema DDL
|
||||
|
||||
完整的Schema创建脚本位于:
|
||||
```
|
||||
摰峕㟲<EFBFBD><EFBFBD>chema<EFBFBD>𥕦遣<EFBFBD>𡁏𧋦雿滢<EFBFBD>嚗?```
|
||||
backend/prisma/migrations/20251127_add_dc_tool_b_tables/migration.sql
|
||||
```
|
||||
|
||||
### 8.2 Prisma璅∪<E79285>摰帋<E691B0>
|
||||
|
||||
完整的Prisma模型定义位于:
|
||||
```
|
||||
摰峕㟲<EFBFBD><EFBFBD>risma璅∪<EFBFBD>摰帋<EFBFBD>雿滢<EFBFBD>嚗?```
|
||||
backend/prisma/schema.prisma
|
||||
```
|
||||
搜索 `dc_schema` 查看所有模型。
|
||||
|
||||
<EFBFBD>𦦵揣 `dc_schema` <20>亦<EFBFBD><E4BAA6><EFBFBD><EFBFBD>㗇芋<E39787>卝<EFBFBD>?
|
||||
### 8.3 <20>䀹凒<E480B9><E58792>蟮
|
||||
|
||||
| <20><>𧋦 | <20>交<EFBFBD> | <20>䀹凒<E480B9><E58792>捆 |
|
||||
|------|------|---------|
|
||||
| V1.0 | 2025-11-27 | 初始版本,4个表 |
|
||||
| V1.0 | 2025-11-27 | <EFBFBD>嘥<EFBFBD><EFBFBD><EFBFBD>𧋦嚗?銝芾” |
|
||||
|
||||
---
|
||||
|
||||
**文档结束** ✅
|
||||
|
||||
**<EFBFBD><EFBFBD>﹝蝏𤘪<EFBFBD>** <20>?
|
||||
|
||||
Reference in New Issue
Block a user