Architecture transformation: - Replace Fan-out (Manager->Child->Last Child Wins) with Scatter+Aggregator pattern - API layer directly dispatches N independent jobs (no Manager) - Worker only writes its own Result row, never touches Task table (zero row-lock) - Aggregator polls groupBy for completion + zombie cleanup (replaces Sweeper) - Reduce red lines from 13 to 9, eliminate distributed complexity Documents updated (10 files): - 08-Tool3 main architecture doc: v2.0 rewrite (schema, Task 2.3/2.4, red lines, risks) - 08d-Code patterns: rewrite sections 4.1-4.6 (API dispatch, SingleWorker, Aggregator) - 08a-M1 sprint: rewrite M1-3 core (Worker+Aggregator), red lines, acceptance criteria - 08b-M2 sprint: simplify SSE (NOTIFY/LISTEN downgraded to P2 optional) - 08c-M3 sprint: milestone table wording update - New: Scatter+Polling Aggregator pattern guide v1.1 (Level 2 cookbook) - New: V2.0 architecture deep review and gap-fix report - Updated: ASL module status, system status, capability layer index Co-authored-by: Cursor <cursoragent@cursor.com>
998 lines
25 KiB
Markdown
998 lines
25 KiB
Markdown
# 通用能力层清单
|
||
|
||
> **文档版本:** v2.4
|
||
> **创建日期:** 2026-01-14
|
||
> **最后更新:** 2026-01-21
|
||
> **文档目的:** 列出所有通用能力模块,提供快速调用指南
|
||
> **本次更新:** RAG 引擎完整实现(替代 Dify)+ 文档处理引擎增强
|
||
|
||
---
|
||
|
||
## 📚 概述
|
||
|
||
通用能力层是平台的核心基础设施,提供可复用的技术能力,避免业务模块重复开发。
|
||
|
||
### 设计原则
|
||
|
||
1. **高复用性** - 一次开发,多处使用
|
||
2. **标准化** - 统一接口,降低学习成本
|
||
3. **解耦合** - 业务模块无需关心底层实现
|
||
4. **云原生** - 支持多环境部署(本地/阿里云)
|
||
|
||
---
|
||
|
||
## 🎯 能力清单
|
||
|
||
### 后端通用能力
|
||
|
||
| 能力 | 路径 | 状态 | 说明 |
|
||
|------|------|------|------|
|
||
| **存储服务** | `common/storage/` | ✅ | 文件存储抽象层(本地/OSS) |
|
||
| **日志服务** | `common/logging/` | ✅ | 结构化日志(Pino) |
|
||
| **缓存服务** | `common/cache/` | ✅ | 缓存抽象层(内存/Redis/Postgres) |
|
||
| **异步任务** | `common/jobs/` | ✅ | 队列服务(Memory/PgBoss) |
|
||
| **🆕批量任务模式** | 设计模式指南 | ✅ | 散装派发 + 轮询收口(Level 2) |
|
||
| **LLM网关** | `common/llm/` | ✅ | 统一LLM适配器(5个模型) |
|
||
| **流式响应** | `common/streaming/` | ✅ 🆕 | OpenAI Compatible流式输出 |
|
||
| **🎉RAG引擎** | `common/rag/` | ✅ 🆕 | **完整实现!pgvector+DeepSeek+Rerank** |
|
||
| **文档处理** | `extraction_service/` | ✅ V2 | pymupdf4llm (全文) + **PDF 表格提取引擎** (多引擎可插拔) |
|
||
| **认证授权** | `common/auth/` | ✅ | JWT认证 + 权限控制 |
|
||
| **Prompt管理** | `common/prompt/` | ✅ | 动态Prompt配置 |
|
||
| **🆕R统计引擎** | `r-statistics-service/` | ✅ | Docker化R统计服务(plumber) |
|
||
| **🆕DeepResearch引擎** | `common/deepsearch/` | ✅ | Unifuncs DeepSearch API封装(9站点验证) |
|
||
|
||
### 前端通用能力
|
||
|
||
| 能力 | 路径 | 状态 | 说明 |
|
||
|------|------|------|------|
|
||
| **Chat组件** | `shared/components/Chat/` | ✅ 🆕 | AI对话通用组件(V2) |
|
||
| **FileCard组件** | `@ant-design/x` | ✅ 🆕 | 文件卡片展示(附件) |
|
||
| **认证API** | `framework/auth/api.ts` | ✅ | Token管理 |
|
||
| **API Client** | `common/api/axios.ts` | ✅ | 带认证的axios实例 |
|
||
| **通用布局** | `shared/layouts/` | ✅ | 主布局、侧边栏 |
|
||
|
||
---
|
||
|
||
## 📦 详细文档
|
||
|
||
### 1. 流式响应服务(🆕 重点推荐)
|
||
|
||
**路径:** `backend/src/common/streaming/`
|
||
|
||
**功能:** 提供 OpenAI Compatible 格式的流式响应,支持深度思考
|
||
|
||
**使用方式:**
|
||
|
||
```typescript
|
||
import { createStreamingService } from '../../../common/streaming';
|
||
|
||
// 在控制器中
|
||
async function handler(request, reply) {
|
||
const service = createStreamingService(reply, {
|
||
model: 'deepseek-v3',
|
||
temperature: 0.7,
|
||
maxTokens: 4096,
|
||
enableDeepThinking: true,
|
||
userId: 'xxx',
|
||
conversationId: 'xxx',
|
||
});
|
||
|
||
const result = await service.streamGenerate(messages, {
|
||
onContent: (delta) => console.log('内容:', delta),
|
||
onThinking: (delta) => console.log('思考:', delta),
|
||
onComplete: (content, thinking) => {
|
||
// 保存到数据库
|
||
},
|
||
});
|
||
}
|
||
```
|
||
|
||
**输出格式:**
|
||
```
|
||
data: {"id":"chatcmpl-xxx","choices":[{"delta":{"content":"你好"}}]}\n\n
|
||
data: {"id":"chatcmpl-xxx","choices":[{"delta":{"reasoning_content":"让我想想"}}]}\n\n
|
||
data: [DONE]\n\n
|
||
```
|
||
|
||
**已使用模块:**
|
||
- ✅ AIA - AI智能问答
|
||
- 🔜 PKB - 个人知识库(待迁移)
|
||
|
||
---
|
||
|
||
### 2. Prompt 管理服务
|
||
|
||
**路径:** `backend/src/common/prompt/`
|
||
|
||
**功能:** 动态 Prompt 配置、灰度预览、版本管理
|
||
|
||
**核心特性:**
|
||
- 灰度预览(DRAFT/ACTIVE 分发)
|
||
- Handlebars 模板渲染
|
||
- 变量提取与校验
|
||
- 三级容灾(数据库→缓存→兜底)
|
||
|
||
**使用方式:**
|
||
|
||
```typescript
|
||
import { getPromptService } from '../../../common/prompt';
|
||
import { prisma } from '../../../config/database';
|
||
|
||
// 获取 Prompt(支持灰度预览)
|
||
const promptService = getPromptService(prisma);
|
||
const { content, isDraft, version } = await promptService.get(
|
||
'AIA_SCIENTIFIC_QUESTION', // Prompt Code
|
||
{}, // 变量(如 { userName: '张三' })
|
||
{ userId } // 用于灰度判断
|
||
);
|
||
|
||
if (isDraft) {
|
||
console.log('使用 DRAFT 版本(调试模式)');
|
||
}
|
||
```
|
||
|
||
**已集成模块:**
|
||
- ✅ RVW - 稿件审查(2个 Prompt)
|
||
- ✅ AIA - AI智能问答(10个 Prompt)🆕 2026-01-18
|
||
|
||
**AIA 模块 Prompt Code 列表:**
|
||
|
||
| Prompt Code | 智能体 |
|
||
|-------------|--------|
|
||
| `AIA_SCIENTIFIC_QUESTION` | 科学问题梳理 |
|
||
| `AIA_PICO_ANALYSIS` | PICO 梳理 |
|
||
| `AIA_TOPIC_EVALUATION` | 选题评价 |
|
||
| `AIA_OUTCOME_DESIGN` | 观察指标设计 |
|
||
| `AIA_CRF_DESIGN` | 病例报告表设计 |
|
||
| `AIA_SAMPLE_SIZE` | 样本量计算 |
|
||
| `AIA_PROTOCOL_WRITING` | 临床研究方案撰写 |
|
||
| `AIA_METHODOLOGY_REVIEW` | 方法学评审智能体 |
|
||
| `AIA_PAPER_POLISH` | 论文润色 |
|
||
| `AIA_PAPER_TRANSLATE` | 论文翻译 |
|
||
|
||
---
|
||
|
||
### 3. Chat 通用组件(🆕 重点推荐)
|
||
|
||
**路径:** `frontend-v2/src/shared/components/Chat/`
|
||
|
||
**功能:** 现代感AI对话组件,支持流式响应、深度思考、会话管理
|
||
|
||
**核心组件:**
|
||
|
||
```typescript
|
||
import {
|
||
AIStreamChat, // 流式对话组件(推荐)
|
||
ThinkingBlock, // 深度思考展示
|
||
ConversationList, // 会话列表
|
||
useAIStream, // 流式响应Hook
|
||
useConversations, // 会话管理Hook
|
||
} from '@/shared/components/Chat';
|
||
```
|
||
|
||
**快速使用:**
|
||
|
||
```tsx
|
||
<AIStreamChat
|
||
apiEndpoint="/api/v1/xxx/chat/stream"
|
||
conversationId={conversationId}
|
||
agentId="xxx"
|
||
enableDeepThinking={true}
|
||
welcome={{
|
||
title: '智能助手',
|
||
description: '有什么可以帮您的?',
|
||
}}
|
||
/>
|
||
```
|
||
|
||
**已使用模块:**
|
||
- ✅ AIA - AI智能问答
|
||
- ✅ DC Tool C - 科研数据编辑器
|
||
- 🔜 PKB - 个人知识库(待迁移)
|
||
|
||
**详细文档:** [Chat组件README](../../frontend-v2/src/shared/components/Chat/README.md)
|
||
|
||
---
|
||
|
||
### 4. Ant Design X FileCard 组件(🆕 2026-01-18)
|
||
|
||
**来源:** `@ant-design/x` (Ant Design X 2.1)
|
||
|
||
**功能:** 紧凑的文件卡片展示,适用于聊天界面附件显示
|
||
|
||
**安装:**
|
||
```bash
|
||
npm install @ant-design/x
|
||
```
|
||
|
||
**使用方式:**
|
||
|
||
```tsx
|
||
import { Attachments } from '@ant-design/x';
|
||
|
||
const { FileCard } = Attachments;
|
||
|
||
// 单个文件卡片
|
||
<FileCard
|
||
item={{
|
||
uid: 'file-1',
|
||
name: '研究方案.pdf',
|
||
size: 1024000, // 字节
|
||
}}
|
||
/>
|
||
|
||
// 文件卡片列表(水平排列)
|
||
<FileCard.List
|
||
items={[
|
||
{ uid: '1', name: '方案.pdf', size: 512000 },
|
||
{ uid: '2', name: '数据.xlsx', size: 256000 },
|
||
]}
|
||
/>
|
||
```
|
||
|
||
**在聊天界面中使用:**
|
||
|
||
```tsx
|
||
// 将附件与工具栏放在同一行
|
||
<div style={{ display: 'flex', alignItems: 'center', gap: 8 }}>
|
||
<Button icon={<PaperClipOutlined />}>上传附件</Button>
|
||
<Button icon={<BulbOutlined />}>深度思考</Button>
|
||
|
||
{/* 附件列表 */}
|
||
{attachments.length > 0 && (
|
||
<FileCard.List
|
||
items={attachments.map(att => ({
|
||
uid: att.id,
|
||
name: att.filename,
|
||
size: att.size,
|
||
}))}
|
||
/>
|
||
)}
|
||
</div>
|
||
```
|
||
|
||
**样式特点:**
|
||
- 自动识别文件类型图标(PDF/Word/Excel/图片等)
|
||
- 自动格式化文件大小(KB/MB)
|
||
- 紧凑设计,适合水平排列
|
||
- 支持删除回调
|
||
|
||
**已使用模块:**
|
||
- ✅ AIA - AI智能问答(附件上传预览)
|
||
|
||
**官方文档:** https://x.ant.design/components/attachments-cn
|
||
|
||
---
|
||
|
||
### 5. LLM 网关
|
||
|
||
**路径:** `backend/src/common/llm/`
|
||
|
||
**功能:** 统一LLM调用接口,支持5个主流模型
|
||
|
||
**支持模型:**
|
||
- DeepSeek-V3(直连)
|
||
- Qwen3-72B(阿里云)
|
||
- Qwen-Long(1M上下文)
|
||
- GPT-5-Pro(CloseAI代理)
|
||
- Claude-4.5(CloseAI代理)
|
||
|
||
**使用方式:**
|
||
|
||
```typescript
|
||
import { LLMFactory } from '../../../common/llm/adapters/LLMFactory';
|
||
|
||
// 获取适配器
|
||
const llm = LLMFactory.getAdapter('deepseek-v3');
|
||
|
||
// 非流式调用
|
||
const response = await llm.chat(messages, { temperature: 0.7 });
|
||
|
||
// 流式调用
|
||
const stream = llm.chatStream(messages, { temperature: 0.7 });
|
||
for await (const chunk of stream) {
|
||
console.log(chunk.content);
|
||
}
|
||
```
|
||
|
||
**已使用模块:**
|
||
- ✅ AIA - AI智能问答
|
||
- ✅ PKB - 个人知识库
|
||
- ✅ DC Tool B - 病历结构化
|
||
- ✅ DC Tool C - 数据编辑器
|
||
|
||
---
|
||
|
||
### 4. 存储服务
|
||
|
||
**路径:** `backend/src/common/storage/`
|
||
|
||
**功能:** 文件存储抽象层,零代码切换本地/OSS
|
||
|
||
**使用方式:**
|
||
|
||
```typescript
|
||
import { storage } from '../../../common/storage';
|
||
|
||
// 上传文件
|
||
const url = await storage.upload('path/to/file.pdf', buffer, {
|
||
contentType: 'application/pdf',
|
||
});
|
||
|
||
// 下载文件
|
||
const buffer = await storage.download('path/to/file.pdf');
|
||
|
||
// 删除文件
|
||
await storage.delete('path/to/file.pdf');
|
||
|
||
// 获取签名URL(OSS)
|
||
const signedUrl = await storage.getSignedUrl('path/to/file.pdf', 3600);
|
||
```
|
||
|
||
**环境配置:**
|
||
```env
|
||
# 本地开发
|
||
STORAGE_PROVIDER=local
|
||
UPLOAD_DIR=./uploads
|
||
|
||
# 生产环境(阿里云OSS)
|
||
STORAGE_PROVIDER=oss
|
||
OSS_REGION=oss-cn-hangzhou
|
||
OSS_BUCKET=your-bucket
|
||
OSS_ACCESS_KEY_ID=xxx
|
||
OSS_ACCESS_KEY_SECRET=xxx
|
||
```
|
||
|
||
**已使用模块:**
|
||
- ✅ PKB - 文档上传
|
||
- ✅ DC Tool B - 病历文件
|
||
- ✅ AIA - 附件上传(待完成)
|
||
|
||
---
|
||
|
||
### 5. 异步任务队列
|
||
|
||
**路径:** `backend/src/common/jobs/`
|
||
|
||
**功能:** Postgres-Only 异步任务处理,基于 PgBoss
|
||
|
||
**使用方式:**
|
||
|
||
```typescript
|
||
import { JobFactory } from '../../../common/jobs';
|
||
|
||
// 获取队列实例
|
||
const queue = JobFactory.getQueue();
|
||
|
||
// 创建任务
|
||
const jobId = await queue.createJob('extraction-job', {
|
||
taskId: 'xxx',
|
||
fileUrl: 'xxx',
|
||
options: {},
|
||
});
|
||
|
||
// 注册 Worker
|
||
queue.registerWorker('extraction-job', async (job) => {
|
||
console.log('处理任务:', job.data);
|
||
// 执行任务逻辑
|
||
return { success: true };
|
||
});
|
||
|
||
// 启动队列
|
||
await queue.start();
|
||
```
|
||
|
||
**已使用模块:**
|
||
- ✅ DC Tool B - 病历批量提取
|
||
- ✅ DC Tool C - 数据异步处理
|
||
|
||
**详细文档:** [Postgres-Only异步任务指南](./Postgres-Only异步任务处理指南.md)
|
||
|
||
---
|
||
|
||
### 6. 缓存服务
|
||
|
||
**路径:** `backend/src/common/cache/`
|
||
|
||
**功能:** 缓存抽象层(内存/Redis/Postgres)
|
||
|
||
**使用方式:**
|
||
|
||
```typescript
|
||
import { cache } from '../../../common/cache';
|
||
|
||
// 设置缓存(TTL: 3600秒)
|
||
await cache.set('key', { data: 'value' }, 3600);
|
||
|
||
// 获取缓存
|
||
const value = await cache.get('key');
|
||
|
||
// 删除缓存
|
||
await cache.delete('key');
|
||
|
||
// 批量删除(支持通配符)
|
||
await cache.clear('prefix:*');
|
||
```
|
||
|
||
**已使用模块:**
|
||
- ✅ AIA - 智能体配置缓存
|
||
- ✅ Prompt管理 - 模板缓存
|
||
|
||
---
|
||
|
||
### 7. 日志服务
|
||
|
||
**路径:** `backend/src/common/logging/`
|
||
|
||
**功能:** 结构化日志,支持多级别输出
|
||
|
||
**使用方式:**
|
||
|
||
```typescript
|
||
import { logger } from '../../../common/logging';
|
||
|
||
// 不同级别
|
||
logger.debug('调试信息', { detail: 'xxx' });
|
||
logger.info('普通日志', { userId: 'xxx' });
|
||
logger.warn('警告信息', { issue: 'xxx' });
|
||
logger.error('错误信息', { error, stack: error.stack });
|
||
|
||
// 结构化字段
|
||
logger.info('[ModuleName] 操作描述', {
|
||
userId: 'xxx',
|
||
conversationId: 'xxx',
|
||
duration: 1234,
|
||
});
|
||
```
|
||
|
||
**日志格式:**
|
||
```
|
||
2026-01-14 18:44:27.500 [aiclinical-backend] info: [AIA:ConversationService] 消息发送完成
|
||
{
|
||
"conversationId": "xxx",
|
||
"tokens": 1234,
|
||
"hasThinking": true
|
||
}
|
||
```
|
||
|
||
**已使用模块:** 所有模块
|
||
|
||
---
|
||
|
||
### 8. 🎉 RAG 引擎(✅ 2026-01-21 完整实现)
|
||
|
||
**路径:** `backend/src/common/rag/` + `ekb_schema`
|
||
|
||
**功能:** 完整的 RAG 检索引擎(替代 Dify)
|
||
|
||
**核心组件:**
|
||
- ✅ EmbeddingService - 文本向量化(text-embedding-v4)
|
||
- ✅ ChunkService - 智能文本分块
|
||
- ✅ VectorSearchService - 向量检索 + 混合检索
|
||
- ✅ RerankService - qwen3-rerank 重排序
|
||
- ✅ QueryRewriter - DeepSeek V3 查询理解
|
||
- ✅ DocumentIngestService - 文档入库完整流程
|
||
|
||
**技术栈:**
|
||
```
|
||
PostgreSQL + pgvector (向量存储)
|
||
↓
|
||
text-embedding-v4 1024维 (向量化)
|
||
↓
|
||
DeepSeek V3 (查询理解 + 中英翻译)
|
||
↓
|
||
向量检索 + 关键词检索 → RRF 融合
|
||
↓
|
||
qwen3-rerank (精排序)
|
||
```
|
||
|
||
**Brain-Hand 架构使用方式:**
|
||
|
||
```typescript
|
||
import { getVectorSearchService, QueryRewriter } from '@/common/rag';
|
||
|
||
// 业务层:查询理解(The Brain)
|
||
const rewriter = new QueryRewriter();
|
||
const result = await rewriter.rewrite('K药副作用');
|
||
const queries = [result.original, ...result.rewritten];
|
||
// ["K药副作用", "Keytruda AE", "Pembrolizumab side effects"]
|
||
|
||
// 引擎层:执行检索(The Hand)
|
||
const searchService = getVectorSearchService(prisma);
|
||
const results = await searchService.searchWithQueries(queries, {
|
||
topK: 10,
|
||
filter: { kbId: 'your-kb-id' }
|
||
});
|
||
|
||
// Rerank 精排
|
||
const final = await searchService.rerank(queries[0], results, { topK: 5 });
|
||
```
|
||
|
||
**性能指标:**
|
||
- 延迟:2.5秒/次(包含查询理解)
|
||
- 成本:¥0.0025/次
|
||
- 准确率:+20.5%(跨语言场景)
|
||
|
||
**已使用模块:**
|
||
- ✅ PKB - 个人知识库(双轨模式:pgvector/dify)
|
||
- 🔜 AIA - AI智能问答
|
||
- 🔜 ASL - AI智能文献
|
||
|
||
**详细文档:**
|
||
- 📖 [RAG 引擎使用指南](./03-RAG引擎/05-RAG引擎使用指南.md) ⭐ **推荐阅读**
|
||
- [知识库引擎架构设计](./03-RAG引擎/01-知识库引擎架构设计.md)
|
||
- [数据模型设计](./03-RAG引擎/04-数据模型设计.md)
|
||
- [分阶段实施方案](./03-RAG引擎/03-分阶段实施方案.md)
|
||
|
||
---
|
||
|
||
### 9. 🎉 文档处理引擎(✅ V2 — 2026-02-23 表格提取引擎升级)
|
||
|
||
**路径:** `extraction_service/` (Python 微服务) + `backend/src/common/document/tableExtraction/` (TypeScript)
|
||
|
||
**功能:** 将各类文档统一转换为 LLM 友好的 Markdown 格式 + **PDF 结构化表格提取**
|
||
|
||
**V2 分层架构 — 全文文本 + 结构化表格 分离:**
|
||
| 引擎层 | 定位 | 输出 | 状态 |
|
||
|--------|------|------|------|
|
||
| **pymupdf4llm** | 全文文本提取 | Markdown | ✅ 已有 |
|
||
| **PDF 表格提取引擎** | 结构化表格提取 (统一抽象层) | ExtractedTable[] | ✅ V2 新增 |
|
||
|
||
**PDF 表格提取引擎 — 候选引擎 (可插拔):**
|
||
| 引擎 | 状态 | 特点 |
|
||
|------|------|------|
|
||
| MinerU Cloud API (VLM) | ✅ 已接入 (当前默认) | 综合 4.6/5 |
|
||
| Qwen3-VL | 📋 待评测 | 多模态理解最强 |
|
||
| PaddleOCR-VL 1.5 | 📋 待评测 | 医学场景案例多 |
|
||
| Qwen-OCR + Qwen-Long | 📋 待评测 | 成本最低 |
|
||
| Docling (IBM) | 📋 待评测 | MIT 开源,离线部署 |
|
||
|
||
**核心 API:**
|
||
```
|
||
POST http://localhost:8000/api/document/to-markdown
|
||
Content-Type: multipart/form-data
|
||
|
||
参数:file (PDF/Word/Excel/PPT等)
|
||
返回:{ success: true, text: "Markdown内容", metadata: {...} }
|
||
```
|
||
|
||
**支持格式:**
|
||
| 格式 | 工具 | 输出质量 | 状态 |
|
||
|------|------|----------|------|
|
||
| PDF (全文) | pymupdf4llm | Markdown 文本 | ✅ |
|
||
| PDF (表格) | **MinerU VLM** | HTML 结构化表格 | ✅ V2 |
|
||
| Word | mammoth | 结构完整 | ✅ |
|
||
| Excel/CSV | pandas | 上下文丰富 | ✅ |
|
||
| PPT | python-pptx | 按页拆分 | ✅ |
|
||
| 纯文本 | 直接读取 | 原样输出 | ✅ |
|
||
|
||
**使用方式(Node.js 调用):**
|
||
|
||
```typescript
|
||
// 在 RAG 引擎入库时自动调用
|
||
import { getDocumentIngestService } from '@/common/rag';
|
||
|
||
const ingestService = getDocumentIngestService(prisma);
|
||
const result = await ingestService.ingestDocument(
|
||
{ filename: 'paper.pdf', fileBuffer: pdfBuffer },
|
||
{ kbId: 'your-kb-id' }
|
||
);
|
||
// DocumentIngestService 内部会调用 Python 微服务转换
|
||
|
||
# 转换任意文档为 Markdown
|
||
md = processor.to_markdown("research_paper.pdf")
|
||
md = processor.to_markdown("report.docx")
|
||
md = processor.to_markdown("data.xlsx")
|
||
```
|
||
|
||
**后端调用(TypeScript):**
|
||
|
||
```typescript
|
||
import { ExtractionClient } from '../../../common/document/ExtractionClient';
|
||
|
||
const client = new ExtractionClient();
|
||
|
||
// 提取文本(返回 Markdown)
|
||
const markdown = await client.extractText(buffer, 'pdf');
|
||
```
|
||
|
||
**已使用模块:**
|
||
- ✅ PKB - 文档上传
|
||
- ✅ DC Tool B - 病历文本提取
|
||
- ✅ ASL - 文献 PDF 提取
|
||
- 🔜 AIA - 附件处理
|
||
|
||
**详细文档:**
|
||
- 📖 [PDF 表格提取引擎使用指南](./02-文档处理引擎/04-PDF表格提取引擎使用指南.md) ⭐ **5 秒上手 + 实战场景**
|
||
- 📖 [PDF 表格提取引擎设计方案](./02-文档处理引擎/03-PDF表格提取引擎设计方案.md) — 统一抽象 + 多引擎可插拔
|
||
- 📖 [文档处理引擎使用指南](./02-文档处理引擎/02-文档处理引擎使用指南.md)
|
||
- [文档处理引擎设计方案](./02-文档处理引擎/01-文档处理引擎设计方案.md)
|
||
|
||
---
|
||
|
||
### 10. 认证授权
|
||
|
||
**路径:** `backend/src/common/auth/`
|
||
|
||
**功能:** JWT认证 + 基于角色的访问控制
|
||
|
||
**后端使用:**
|
||
|
||
```typescript
|
||
import { authenticate, requirePermission } from '../../../common/auth/auth.middleware';
|
||
|
||
// 路由添加认证
|
||
fastify.get('/api', { preHandler: [authenticate] }, handler);
|
||
|
||
// 要求特定权限
|
||
fastify.post('/api', {
|
||
preHandler: [authenticate, requirePermission('module:action')]
|
||
}, handler);
|
||
|
||
// 控制器获取用户ID
|
||
function getUserId(request: FastifyRequest): string {
|
||
const userId = (request as any).user?.userId;
|
||
if (!userId) throw new Error('User not authenticated');
|
||
return userId;
|
||
}
|
||
```
|
||
|
||
**前端使用:**
|
||
|
||
```typescript
|
||
import { getAccessToken } from '@/framework/auth/api';
|
||
|
||
// 方式1: 使用 apiClient(推荐)
|
||
import apiClient from '@/common/api/axios';
|
||
const response = await apiClient.get('/api/xxx');
|
||
|
||
// 方式2: 原生 fetch
|
||
const response = await fetch('/api/xxx', {
|
||
headers: {
|
||
'Content-Type': 'application/json',
|
||
'Authorization': `Bearer ${getAccessToken()}`,
|
||
},
|
||
});
|
||
```
|
||
|
||
**已使用模块:** 所有模块
|
||
|
||
**详细文档:** [模块认证规范](../04-开发规范/10-模块认证规范.md)
|
||
|
||
---
|
||
|
||
### 11. Prompt 管理系统
|
||
|
||
**路径:** `backend/src/common/prompt/`
|
||
|
||
**功能:** 动态Prompt配置、版本管理、A/B测试
|
||
|
||
**使用方式:**
|
||
|
||
```typescript
|
||
import { PromptService } from '../../../common/prompt/prompt.service';
|
||
|
||
const promptService = new PromptService();
|
||
|
||
// 获取激活的Prompt
|
||
const prompt = await promptService.getActivePrompt('template_code');
|
||
|
||
// 渲染Prompt(变量替换)
|
||
const rendered = promptService.renderPrompt(template.content, {
|
||
userName: '张三',
|
||
context: '研究背景...',
|
||
});
|
||
```
|
||
|
||
**已使用模块:**
|
||
- ✅ DC Tool C - 代码生成Prompt
|
||
- 🔜 AIA - 智能体Prompt(待迁移)
|
||
|
||
---
|
||
|
||
## 🎨 前端 Chat 组件(V2)
|
||
|
||
### 概述
|
||
|
||
基于 **Ant Design X 2.1** 构建的现代感AI对话组件,支持流式响应、深度思考、会话管理。
|
||
|
||
### 核心组件
|
||
|
||
#### 1. AIStreamChat - 流式对话组件(推荐)
|
||
|
||
```tsx
|
||
import { AIStreamChat } from '@/shared/components/Chat';
|
||
|
||
<AIStreamChat
|
||
apiEndpoint="/api/v1/aia/conversations/xxx/messages/stream"
|
||
conversationId="xxx"
|
||
agentId="TOPIC_01"
|
||
enableDeepThinking={true}
|
||
welcome={{
|
||
title: 'AI 助手',
|
||
description: '有什么可以帮您的?',
|
||
}}
|
||
onMessageSent={(msg) => console.log('发送:', msg)}
|
||
onMessageReceived={(msg) => console.log('接收:', msg)}
|
||
/>
|
||
```
|
||
|
||
**特性:**
|
||
- ✅ 逐字流式显示(打字机效果)
|
||
- ✅ 深度思考展示(可折叠)
|
||
- ✅ 欢迎页配置
|
||
- ✅ 快捷提示
|
||
- ✅ 附件上传
|
||
- ✅ 现代感设计
|
||
|
||
#### 2. ThinkingBlock - 深度思考组件
|
||
|
||
```tsx
|
||
import { ThinkingBlock } from '@/shared/components/Chat';
|
||
|
||
<ThinkingBlock
|
||
content={thinkingContent}
|
||
isThinking={true}
|
||
duration={2.5}
|
||
defaultExpanded={true}
|
||
/>
|
||
```
|
||
|
||
#### 3. ConversationList - 会话列表
|
||
|
||
```tsx
|
||
import { ConversationList } from '@/shared/components/Chat';
|
||
|
||
<ConversationList
|
||
groups={groupedConversations}
|
||
currentId={currentId}
|
||
onSelect={setCurrent}
|
||
onNew={createConversation}
|
||
onDelete={deleteConversation}
|
||
/>
|
||
```
|
||
|
||
### Hooks
|
||
|
||
#### useAIStream - 流式响应处理
|
||
|
||
```tsx
|
||
import { useAIStream } from '@/shared/components/Chat';
|
||
|
||
const {
|
||
content, // 当前内容
|
||
thinking, // 思考内容
|
||
status, // 流式状态
|
||
isStreaming, // 是否正在流式输出
|
||
isThinking, // 是否正在思考
|
||
error, // 错误信息
|
||
sendMessage, // 发送消息
|
||
abort, // 中断请求
|
||
reset, // 重置状态
|
||
} = useAIStream({
|
||
apiEndpoint: '/api/v1/aia/chat/stream',
|
||
headers: { Authorization: `Bearer ${token}` },
|
||
});
|
||
```
|
||
|
||
#### useConversations - 会话管理
|
||
|
||
```tsx
|
||
import { useConversations } from '@/shared/components/Chat';
|
||
|
||
const {
|
||
conversations, // 会话列表
|
||
groupedConversations, // 分组会话(今天/昨天/更早)
|
||
current, // 当前会话
|
||
setCurrent, // 设置当前会话
|
||
addConversation, // 添加会话
|
||
updateConversation, // 更新会话
|
||
deleteConversation, // 删除会话
|
||
} = useConversations();
|
||
```
|
||
|
||
### 向后兼容
|
||
|
||
保留了 V1 版本的组件:
|
||
- `ChatContainer` - 传统对话容器(非流式)
|
||
- `MessageRenderer` - 消息渲染器
|
||
- `CodeBlockRenderer` - 代码块渲染器(Tool C专用)
|
||
|
||
---
|
||
|
||
## 🔗 能力组合使用
|
||
|
||
### 场景1:AI对话模块(如AIA)
|
||
|
||
**前端:**
|
||
```tsx
|
||
import { AIStreamChat } from '@/shared/components/Chat';
|
||
|
||
<AIStreamChat
|
||
apiEndpoint="/api/v1/aia/conversations/xxx/messages/stream"
|
||
enableDeepThinking={true}
|
||
/>
|
||
```
|
||
|
||
**后端:**
|
||
```typescript
|
||
import { createStreamingService } from '../../../common/streaming';
|
||
import { LLMFactory } from '../../../common/llm/adapters/LLMFactory';
|
||
|
||
async function sendMessageStream(userId, conversationId, request, reply) {
|
||
const service = createStreamingService(reply);
|
||
await service.streamGenerate(messages);
|
||
}
|
||
```
|
||
|
||
### 场景2:知识库问答(如PKB)
|
||
|
||
**前端:**
|
||
```tsx
|
||
<AIStreamChat
|
||
apiEndpoint="/api/v1/pkb/chat/stream"
|
||
enableAttachment={true} // 支持文档上传
|
||
/>
|
||
```
|
||
|
||
**后端:**
|
||
```typescript
|
||
import { DifyClient } from '../../../common/rag/DifyClient';
|
||
import { createStreamingService } from '../../../common/streaming';
|
||
|
||
// RAG检索 + 流式输出
|
||
const ragResults = await dify.retrievalSearch(query, options);
|
||
const messages = buildMessagesWithRAG(query, ragResults);
|
||
await service.streamGenerate(messages);
|
||
```
|
||
|
||
### 场景3:代码生成工具(如Tool C)
|
||
|
||
**前端:**
|
||
```tsx
|
||
<ChatContainer
|
||
conversationType="tool-c"
|
||
customMessageRenderer={(msg) => (
|
||
<MessageRenderer
|
||
messageInfo={msg}
|
||
onExecuteCode={handleExecute}
|
||
/>
|
||
)}
|
||
/>
|
||
```
|
||
|
||
**后端:**
|
||
```typescript
|
||
import { LLMFactory } from '../../../common/llm/adapters/LLMFactory';
|
||
import { PromptService } from '../../../common/prompt/prompt.service';
|
||
|
||
// Prompt管理 + LLM调用
|
||
const prompt = await promptService.getActivePrompt('code_gen');
|
||
const llm = LLMFactory.getAdapter('deepseek-v3');
|
||
const response = await llm.chat(messages);
|
||
```
|
||
|
||
---
|
||
|
||
## 📊 使用统计
|
||
|
||
### 能力使用矩阵
|
||
|
||
| 能力 | AIA | PKB | DC-B | DC-C | ASL | RVW |
|
||
|------|-----|-----|------|------|-----|-----|
|
||
| **StreamingService** | ✅ | 🔜 | ❌ | ❌ | ❌ | ❌ |
|
||
| **Chat组件V2** | ✅ | 🔜 | ❌ | ✅ | ❌ | ❌ |
|
||
| **LLM网关** | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ |
|
||
| **存储服务** | 🔜 | ✅ | ✅ | ✅ | ✅ | ✅ |
|
||
| **异步任务** | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ |
|
||
| **知识库引擎** | 🔜 | ✅ | ❌ | ❌ | 🔜 | 🔜 |
|
||
| **文档处理** | 🔜 | ✅ | ✅ | ❌ | ❌ | ❌ |
|
||
| **认证授权** | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
|
||
| **Prompt管理** | 🔜 | ❌ | ❌ | ✅ | ❌ | ❌ |
|
||
|
||
图例:✅ 已使用 | 🔜 计划使用 | ❌ 不需要
|
||
|
||
---
|
||
|
||
## 🚀 最佳实践
|
||
|
||
### 1. 优先使用通用能力
|
||
|
||
❌ **错误做法:**
|
||
```typescript
|
||
// 自己实现流式响应
|
||
reply.raw.write(`data: ${JSON.stringify(chunk)}\n\n`);
|
||
```
|
||
|
||
✅ **正确做法:**
|
||
```typescript
|
||
// 使用 StreamingService
|
||
const service = createStreamingService(reply);
|
||
await service.streamGenerate(messages);
|
||
```
|
||
|
||
### 2. 标准化API格式
|
||
|
||
❌ **错误做法:**
|
||
```typescript
|
||
// 自定义SSE格式
|
||
event: delta\ndata: {"content":"xxx"}\n\n
|
||
```
|
||
|
||
✅ **正确做法:**
|
||
```typescript
|
||
// OpenAI Compatible格式
|
||
data: {"choices":[{"delta":{"content":"xxx"}}]}\n\n
|
||
```
|
||
|
||
### 3. 云原生设计
|
||
|
||
❌ **错误做法:**
|
||
```typescript
|
||
// 直接使用本地文件系统
|
||
fs.writeFileSync('/tmp/file.txt', content);
|
||
```
|
||
|
||
✅ **正确做法:**
|
||
```typescript
|
||
// 使用存储抽象层
|
||
await storage.upload('path/file.txt', buffer);
|
||
```
|
||
|
||
---
|
||
|
||
## 📚 学习路径
|
||
|
||
### 新手入门
|
||
|
||
1. 阅读 [系统整体设计](../00-系统总体设计/00-系统当前状态与开发指南.md)
|
||
2. 阅读 [云原生开发规范](../04-开发规范/08-云原生开发规范.md)
|
||
3. 参考 AIA 或 DC Tool C 的实现
|
||
|
||
### 进阶学习
|
||
|
||
1. 深入 [Chat组件文档](../../frontend-v2/src/shared/components/Chat/README.md)
|
||
2. 研究 [StreamingService 源码](../../backend/src/common/streaming/)
|
||
3. 了解 [Postgres-Only架构](./Postgres-Only异步任务处理指南.md)
|
||
|
||
---
|
||
|
||
## 🔄 版本历史
|
||
|
||
### V2.0 (2026-01-14) 🎉
|
||
|
||
**通用能力层建设:**
|
||
- 🆕 StreamingService - OpenAI Compatible流式响应
|
||
- 🆕 Chat组件V2 - 现代感UI + 完整功能
|
||
- 🆕 useAIStream Hook - 流式响应处理
|
||
- 🆕 ThinkingBlock - 深度思考展示
|
||
|
||
**已应用模块:**
|
||
- ✅ AIA - 首个使用者
|
||
- 🔜 PKB - 计划迁移
|
||
- 🔜 其他AI对话场景
|
||
|
||
### V1.0 (历史版本)
|
||
|
||
- 基础LLM网关
|
||
- 简单Chat组件
|
||
- 各模块独立实现
|
||
|
||
---
|
||
|
||
## 📞 获取帮助
|
||
|
||
- **通用能力层问题:** 查看本文档或各子模块README
|
||
- **业务模块问题:** 查看对应模块的状态文档
|
||
- **架构设计问题:** 查看系统总体设计文档
|
||
|
||
---
|
||
|
||
**维护说明:** 新增通用能力时,请及时更新本文档。
|
||
|
||
|
||
|
||
|
||
|
||
|