feat(aia): Complete AIA V2.0 with universal streaming capabilities

Major Changes:
- Add StreamingService with OpenAI Compatible format
- Upgrade Chat component V2 with Ant Design X integration
- Implement AIA module with 12 intelligent agents
- Update API routes to unified /api/v1 prefix
- Update system documentation

Backend (~1300 lines):
- common/streaming: OpenAI Compatible adapter
- modules/aia: 12 agents, conversation service, streaming integration
- Update route versions (RVW, PKB to v1)

Frontend (~3500 lines):
- modules/aia: AgentHub + ChatWorkspace (100% prototype restoration)
- shared/Chat: AIStreamChat, ThinkingBlock, useAIStream Hook
- Update API endpoints to v1

Documentation:
- AIA module status guide
- Universal capabilities catalog
- System overview updates
- All module documentation sync

Tested: Stream response verified, authentication working
Status: AIA V2.0 core completed (85%)
This commit is contained in:
2026-01-14 19:15:01 +08:00
parent 3d35e9c58b
commit 1b53ab9d52
386 changed files with 52096 additions and 65238 deletions

View File

@@ -1,20 +1,20 @@
# <20>𡁶鍂<F0A181B6><EFBFBD><EFBFBD><E69285><EFBFBD><EFBFBD>箏𦛚皜<F0A69B9A><E79A9C>
> **文档版本:** v1.0
> **创建日期:** 2025-12-22
> **<EFBFBD><EFBFBD><EFBFBD><EFBFBD>𧋦嚗?* v1.0
> **<EFBFBD>𥕦遣<EFBFBD><EFBFBD>嚗?* 2025-12-22
> **蝏湔擪<E6B994><E693AA><EFBFBD>** 撟喳蝱<E596B3><EFBFBD><E59786><EFBFBD>
> **文档目的:** 记录通用能力层待优化项,指导未来迭代
> **<EFBFBD><EFBFBD><EFBFBD><EFBFBD>嚗?* 霈啣<E99C88><E595A3>𡁶鍂<F0A181B6><EFBFBD><EFBFBD><E69285>隡睃<E99AA1>憿對<E686BF><E5B08D><EFBFBD><EFBFBD>芣䔉餈凋誨
---
## <20><><>
本文档基于 **DC Tool C 异步架构实践**2025-12-22总结发现的可以抽象为通用能力的模式。
<EFBFBD><EFBFBD><EFBFBD>抅鈭?**DC Tool C <EFBFBD><EFBFBD><EFBFBD>摰噼殿**嚗?025-12-22嚗㚁<E59A97><E39A81><EFBFBD><E9A48C>𤑳緵<F0A491B3><E7B7B5>虾隞交𡂝鞊∩蛹<E288A9>𡁶鍂<F0A181B6><EFBFBD><E8B3A2><EFBFBD>芋撘譌<E69298>?
**核心思想**
- **当前分层已经很好**(通用能力层提供完整的基础设施)
- **业务模块正确使用了这些能力**
- ⏭️ **未来可以进一步抽象**(锦上添花,非必需)
**<EFBFBD><EFBFBD><EFBFBD><EFBFBD>**嚗?
- <EFBFBD>?**敶枏<E695B6><E69E8F><EFBFBD><EFBFBD>撌脩<E6928C><EFBFBD>末**嚗<><E59A97>𡁶鍂<F0A181B6><EFBFBD><EFBFBD><E69285>靘𥕦<E99D98><F0A595A6><EFBFBD><E6B8A1><EFBFBD>霈暹鴌嚗?
- <EFBFBD>?**銝𡁜𦛚璅<E79285><EFBFBD>雿輻鍂鈭<E98D82><E988AD>鈭𥡝<E988AD><F0A5A19D>?*
- <EFBFBD><EFBFBD> **<EFBFBD>芣䔉<EFBFBD>臭誑餈𥕢<EFBFBD>甇交𡂝鞊?*嚗<>釺銝𦠜溶<F0A6A09C><EFBFBD><E6A2A7>𧼮<EFBFBD><F0A7BCAE><EFBFBD>嚗?
---
@@ -22,18 +22,18 @@
### TD-COMMON-001: <20>滨垢<E6BBA8>𡁶鍂頧株砭Hook
**优先级**:⭐⭐⭐ P2
**工作量**0.5天
**隡睃<EFBFBD>蝥?*嚗尠<E59A97>潃鐥<E6BD83> P2嚗<32>葉嚗?
**撌乩<EFBFBD><EFBFBD>?*嚗?.5憭?
**憸<><E686B8><EFBFBD><EFBFBD>**嚗帋誨<E5B88B><E8AAA8><EFBFBD><EFBFBD><EFBFBD><EFBFBD><E689B3><EFBFBD><EFBFBD><E79285><E288AA><EFBFBD><EFBFBD>
#### <20><EFBFBD><E6A185>讛膩
**当前状态**
**敶枏<EFBFBD><EFBFBD><EFBFBD>?*嚗?
- Tool C 摰䂿緵嚗䫤dc/tool-c/hooks/useSessionStatus.ts`
- ASL 摰䂿緵嚗䫤asl/hooks/useScreeningTask.ts`
-<><E99A9E><EFBFBD><EFBFBD>摨佗<E691A8>70%
**重复的逻辑**
**<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>**嚗?
```typescript
// 瘥譍葵璅<E79285><E288AA><EFBFBD><E8B3AA>嗵掩隡潛<E99AA1><EFBFBD><E99A9E>
useQuery({
@@ -51,7 +51,7 @@ useQuery({
#### 閫<><E996AB><EFBFBD><EFBFBD>
**抽象为通用Hook**
**<EFBFBD>質情銝粹<EFBFBD>𡁶鍂Hook**嚗?
```typescript
// frontend-v2/src/common/hooks/useAsyncTaskPolling.ts嚗<73>鰵撱綽<E692B1>
@@ -65,7 +65,7 @@ interface UseAsyncTaskPollingOptions<T> {
/** <20><EFBFBD><E59786>䰻霂PI<50>賣㺭 */
queryFn: (taskId: string) => Promise<T>;
/** 状态提取函数 */
/** <EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>硋遆<EFBFBD>?*/
getStatus: (data: T) => 'pending' | 'processing' | 'ready' | 'error' | string;
/** 餈𥕦漲<F0A595A6>𣂼<EFBFBD><F0A382BC>賣㺭嚗<E3BAAD><EFBFBD><EFBFBD> */
@@ -74,7 +74,7 @@ interface UseAsyncTaskPollingOptions<T> {
/** <20>臬炏<E887AC>舐鍂 */
enabled?: boolean;
/** 轮询间隔毫秒默认2000 */
/** 頧株砭<EFBFBD><EFBFBD><EFBFBD>神蝘𡜐<EFBFBD>嚗屸<EFBFBD>霈?000 */
pollingInterval?: number;
}
@@ -122,7 +122,7 @@ export function useAsyncTaskPolling<T>({
}
```
**使用示例**
**雿輻鍂蝷箔<EFBFBD>**嚗?
```typescript
// Tool C 雿輻鍂
@@ -144,33 +144,33 @@ const { status, isReady } = useAsyncTaskPolling({
#### 敶勗<E695B6><E58B97><EFBFBD>
- Tool C: `useSessionStatus.ts` 可简化
- ASL: `useScreeningTask.ts` 可简化
- Tool B: 未来前端可直接使用
- Tool C: `useSessionStatus.ts` <EFBFBD><EFBFBD><EFBFBD>?
- ASL: `useScreeningTask.ts` <EFBFBD><EFBFBD><EFBFBD>?
- Tool B: <EFBFBD>芣䔉<EFBFBD>滨垢<EFBFBD>舐凒<EFBFBD>乩蝙<EFBFBD>?
- <20><EFBFBD><E79285>: <20>湔𦻖憭滨鍂
---
### TD-COMMON-002: Clean Data 蝻枏<E89DBB><E69E8F>滚𦛚
**优先级**:⭐⭐⭐⭐ P1
**工作量**1天
**预期收益**性能提升99%,所有模块受益
**隡睃<EFBFBD>蝥?*嚗尠<E59A97>潃鐥<E6BD83>潃?P1嚗<31><E59A97>嚗?
**撌乩<EFBFBD><EFBFBD>?*嚗?憭?
**<EFBFBD><EFBFBD><EFBFBD><EFBFBD>**嚗𡁏<E59A97><EFBFBD><E689AF>𣂼<EFBFBD>99%嚗峕<E59A97><E5B395>㗇芋<E39787><EFBFBD><E5A092>?
#### <20><EFBFBD><E6A185>讛膩
**当前状态**
- Tool C 实现:保存 `${fileKey}_clean.json`
- ASL、Tool B:未实现,仍然每次重新解析
**敶枏<EFBFBD><EFBFBD><EFBFBD>?*嚗?
- Tool C 摰䂿緵嚗帋<EFBFBD>摮?`${fileKey}_clean.json`
- ASL<EFBFBD><EFBFBD>ool B嚗𡁏𧊋摰䂿緵嚗䔶<EFBFBD><EFBFBD><EFBFBD>甈⊿<EFBFBD><EFBFBD>啗圾<EFBFBD>?
**重复计算问题**
- ASL 文献筛选:每次从 OSS 下载 PDF重新解析5-10秒/篇)
- Tool B 数据提取:每次重新读取 Excel
- Tool C 操作已优化clean data缓存)
**<EFBFBD><EFBFBD>霈∠<EFBFBD><EFBFBD><EFBFBD>**嚗?
- ASL <EFBFBD><EFBFBD>讃蝑偦<EFBFBD><EFBFBD>瘥𤩺活隞?OSS 銝贝蝸 PDF嚗屸<EFBFBD><EFBFBD>啗圾<EFBFBD><EFBFBD>5-10蝘?蝭<><E89DAD>
- Tool B <EFBFBD>唳旿<EFBFBD>𣂼<EFBFBD>嚗𡁏<EFBFBD>甈⊿<EFBFBD><EFBFBD>啗粉<EFBFBD>?Excel
- Tool C <EFBFBD><EFBFBD>嚗𡁜歇隡睃<EFBFBD>嚗Ếlean data蝻枏<EFBFBD>嚗?
#### 閫<><E996AB><EFBFBD><EFBFBD>
**抽象为通用服务**
**<EFBFBD>質情銝粹<EFBFBD>𡁶鍂<EFBFBD>滚𦛚**嚗?
```typescript
// backend/src/common/services/DataCacheService.ts嚗<73>鰵撱綽<E692B1>
@@ -180,12 +180,12 @@ import { prisma } from '../../config/database';
import { logger } from '../logging';
/**
* 数据缓存服务(通用)
* <EFBFBD>唳旿蝻枏<EFBFBD><EFBFBD>滚𦛚嚗<EFBFBD><EFBFBD>𡁶鍂嚗?
*
* <20><EFBFBD><EFBFBD>
* - Worker 解析后保存处理结果
* - Service 优先读取缓存,避免重复计算
* - 操作后同步更新缓存
* - Worker <EFBFBD><EFBFBD><EFBFBD>𦒘<EFBFBD>摮睃<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>?
* - Service 隡睃<EFBFBD>霂餃<EFBFBD>蝻枏<EFBFBD>嚗屸<EFBFBD><EFBFBD><EFBFBD>憭滩恣蝞?
* - <EFBFBD><EFBFBD><EFBFBD>𤾸<EFBFBD>甇交凒<EFBFBD><EFBFBD>摮?
*/
export class DataCacheService {
/**
@@ -193,8 +193,8 @@ export class DataCacheService {
*
* @param originalKey <20><EFBFBD><E7AC94><EFBFBD>辣key
* @param cleanData 皜<><E79A9C><EFBFBD>𡒊<EFBFBD><F0A1928A>唳旿
* @param suffix 后缀(默认 '_clean.json'
* @returns clean data OSS key
* @param suffix <EFBFBD>𡒊<EFBFBD><EFBFBD><EFBFBD>霈?'_clean.json'嚗?
* @returns clean data <EFBFBD>?OSS key
*/
async saveCleanData(
originalKey: string,
@@ -223,7 +223,7 @@ export class DataCacheService {
/**
* 霂餃<E99C82><EFBFBD><E79A9C><EFBFBD>𡒊<EFBFBD><F0A1928A>唳旿
*
* @param cleanDataKey clean data OSS key
* @param cleanDataKey clean data <EFBFBD>?OSS key
* @returns 皜<><E79A9C><EFBFBD>𡒊<EFBFBD><F0A1928A>唳旿
*/
async getCleanData(cleanDataKey: string): Promise<any> {
@@ -242,8 +242,8 @@ export class DataCacheService {
/**
* <20>湔鰵皜<E9B0B5><E79A9C><EFBFBD>𡒊<EFBFBD><F0A1928A>唳旿
*
* @param cleanDataKey clean data OSS key
* @param newData 新数据
* @param cleanDataKey clean data <EFBFBD>?OSS key
* @param newData <EFBFBD>唳㺭<EFBFBD>?
*/
async updateCleanData(cleanDataKey: string, newData: any): Promise<void> {
logger.info('[DataCacheService] Updating clean data', {
@@ -260,7 +260,7 @@ export class DataCacheService {
/**
* <20>𣳇膄皜<E88684><E79A9C><EFBFBD>𡒊<EFBFBD><F0A1928A>唳旿
*
* @param cleanDataKey clean data OSS key
* @param cleanDataKey clean data <EFBFBD>?OSS key
*/
async deleteCleanData(cleanDataKey: string): Promise<void> {
try {
@@ -278,46 +278,46 @@ export class DataCacheService {
export const dataCacheService = new DataCacheService();
```
**使用示例**
**雿輻鍂蝷箔<EFBFBD>**嚗?
```typescript
// Worker
// Worker 銝?
const cleanDataKey = await dataCacheService.saveCleanData(fileKey, cleanedData);
await prisma.update({ where: { id }, data: { cleanDataKey } });
// Service
// Service 銝?
if (record.cleanDataKey) {
return await dataCacheService.getCleanData(record.cleanDataKey);
}
// 操作后更新
// <EFBFBD><EFBFBD><EFBFBD>擧凒<EFBFBD>?
await dataCacheService.updateCleanData(record.cleanDataKey, newData);
```
#### 敶勗<E695B6><E58B97><EFBFBD>
- Tool C: 简化现有代码
- ASL: 文献解析结果缓存提升99%
- Tool C: <EFBFBD><EFBFBD>𣇉緵<EFBFBD>劐誨<EFBFBD>?
- ASL: <EFBFBD><EFBFBD>讃閫<EFBFBD><EFBFBD>蝏𤘪<EFBFBD>蝻枏<EFBFBD><EFBFBD><EFBFBD><EFBFBD>?9%嚗?
- Tool B: <20>唳旿<E594B3>𣂼<EFBFBD>蝏𤘪<E89D8F>蝻枏<E89DBB>
- 所有模块: 统一的缓存机制
- <EFBFBD><EFBFBD><EFBFBD>㗇芋<EFBFBD>? 蝏煺<E89D8F><E785BA><EFBFBD><EFBFBD>摮䀹㦤<E480B9>?
---
### TD-COMMON-003: 智能清洗算法通用化
### TD-COMMON-003: <EFBFBD><EFBFBD><EFBFBD><EFBFBD>蝞埈<EFBFBD><EFBFBD>𡁶鍂<EFBFBD>?
**优先级**:⭐⭐ P2
**工作量**0.5天
**隡睃<EFBFBD>蝥?*嚗尠<E59A97>潃?P2嚗<32>葉嚗?
**撌乩<EFBFBD><EFBFBD>?*嚗?.5憭?
**憸<><E686B8><EFBFBD><EFBFBD>**嚗帋誨<E5B88B><E8AAA8><EFBFBD><EFBFBD><EFBFBD><E58981><EFBFBD><E8B8B9><EFBFBD>摰䂿緵
#### <20><EFBFBD><E6A185>讛膩
**当前状态**
- Tool C 实现:`intelligentCleanData`(边界检测+安全阀)
**敶枏<EFBFBD><EFBFBD><EFBFBD>?*嚗?
- Tool C 摰䂿緵嚗䫤intelligentCleanData`<EFBFBD><EFBFBD><EFBFBD>瘚?摰匧<E691B0><E58CA7><EFBFBD>嚗?
- <20><EFBFBD><E79285>嚗𡁏𧊋摰䂿緵蝐颱撮<E9A2B1><EFBFBD>
**可以通用化的算法**
**<EFBFBD>臭誑<EFBFBD>𡁶鍂<EFBFBD>𣇉<EFBFBD>蝞埈<EFBFBD>**嚗?
1. 撟賜<E6929F><E8B39C><EFBFBD>瘚页<E7989A>颲寧<E9A2B2><EFBFBD>瘚页<E7989A>
2. 幽灵行过滤
2. 撟賜<EFBFBD>銵諹<EFBFBD>皛?
3. 摰匧<E691B0><E58CA7><EFBFBD><EFBFBD><E59A97>憭批<E686AD><E689B9><EFBFBD><E5959C><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><E59C88><EFBFBD>
#### 閫<><E996AB><EFBFBD><EFBFBD>
@@ -327,7 +327,7 @@ await dataCacheService.updateCleanData(record.cleanDataKey, newData);
export interface CleaningOptions {
maxCols?: number; // <20><>憭批<E686AD><E689B9><EFBFBD>暺䁅恕3000
maxCells?: number; // 最大单元格数,默认500
maxCells?: number; // <EFBFBD><EFBFBD>憭批<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>暺䁅恕500銝?
removeEmptyRows?: boolean; // <20>臬炏<E887AC>𣳇膄蝛箄<E89D9B>嚗屸<E59A97>霈川rue
removeEmptyCols?: boolean; // <20>臬炏<E887AC>𣳇膄蝛箏<E89D9B>嚗屸<E59A97>霈川rue
}
@@ -341,7 +341,7 @@ export function intelligentCleanData(
}
export function isValidValue(value: any): boolean {
// 统一的空值判断
// 蝏煺<EFBFBD><EFBFBD><EFBFBD><EFBFBD>澆ế<EFBFBD>?
// ...
}
```
@@ -349,19 +349,19 @@ export function isValidValue(value: any): boolean {
#### 敶勗<E695B6><E58B97><EFBFBD>
- Tool C: 憭滨鍂<E6BBA8>𡁶鍂摰䂿緵
- 其他上传Excel的功能: 直接使用
- <EFBFBD><EFBFBD>銝𠹺<EFBFBD>Excel<EFBFBD><EFBFBD><EFBFBD><EFBFBD>? <20>湔𦻖雿輻鍂
---
### TD-COMMON-004: Worker瘜典<E7989C><EFBFBD>𨭌撌亙<E6928C>
**优先级**:⭐ P3
**工作量**0.3天
**隡睃<EFBFBD>蝥?*嚗尠<E59A97> P3嚗<33><E59A97>嚗?
**撌乩<EFBFBD><EFBFBD>?*嚗?.3憭?
**憸<><E686B8><EFBFBD><EFBFBD>**嚗𡁻<E59A97>雿竝orker瘜典<E7989C><EFBFBD><E99A9E><EFBFBD><EFBFBD>
#### <20><EFBFBD><E6A185>讛膩
**当前状态**
**敶枏<EFBFBD><EFBFBD><EFBFBD>?*嚗?
- 瘥譍葵璅<E79285><E288AA><EFBFBD><E8B3AA><EFBFBD> Worker 瘜典<E7989C><EFBFBD><E99A9E>
- <20>躰秤憭<E7A7A4><E686AD><EFBFBD><EFBFBD><E9A489><EFBFBD>
@@ -403,11 +403,11 @@ export function registerWorker(options: RegisterWorkerOptions) {
}
});
logger.info(`[WorkerHelper] Worker registered: ${queueName}`);
logger.info(`[WorkerHelper] <EFBFBD>?Worker registered: ${queueName}`);
}
```
**使用示例**
**雿輻鍂蝷箔<EFBFBD>**嚗?
```typescript
// 蝞<><E89D9E>𣇉<EFBFBD>瘜典<E7989C><EFBFBD><E99A9E>
@@ -418,7 +418,7 @@ registerWorker({
// 銝𡁜𦛚<F0A1819C><EFBFBD>
return result;
},
onStart: (job) => console.log(`开始处理: ${job.id}`),
onStart: (job) => console.log(`<EFBFBD>憪见<EFBFBD><EFBFBD>? ${job.id}`),
onComplete: (job, result) => console.log(`摰峕<EFBFBD>: ${result}`),
onError: (job, error) => console.error(`憭梯揖: ${error}`),
});
@@ -428,36 +428,36 @@ registerWorker({
### TD-COMMON-005: <20>唳旿<E594B3><E697BF>𧋦蝞∠<E89D9E>蝟餌<E89D9F>
**优先级**:⭐⭐⭐⭐ P1
**工作量**3天
**预期收益**支持链式操作、undo功能、导出历史版本
**隡睃<EFBFBD>蝥?*嚗尠<E59A97>潃鐥<E6BD83>潃?P1嚗<31><E59A97>嚗?
**撌乩<EFBFBD><EFBFBD>?*嚗?憭?
**<EFBFBD><EFBFBD><EFBFBD><EFBFBD>**嚗𡁏𣈲<F0A1818F><F0A388B2>曎撘𤩺<E69298>雿栶<E99BBF><E6A0B6>ndo<64><EFBFBD><E8A098><EFBFBD><EFBFBD><EFBFBD><E7AE8F><EFBFBD><E884A9>?
#### <20><EFBFBD><E6A185>讛膩
**当前限制**
- 操作不是累积的(每次基于原始数据)
- 无法回退到某个操作前的状态
- 无法导出中间版本的数据
**敶枏<EFBFBD><EFBFBD>𣂼<EFBFBD>**嚗?
- <EFBFBD><EFBFBD>銝齿糓蝝舐妖<EFBFBD><EFBFBD><EFBFBD>瘥𤩺活<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>唳旿嚗?
- <EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>銝芣<EFBFBD>雿𨅯<EFBFBD><EFBFBD><EFBFBD>𠶖<EFBFBD>?
- <EFBFBD><EFBFBD>撖澆枂銝剝𡢿<EFBFBD><EFBFBD>𧋦<EFBFBD><EFBFBD><EFBFBD>?
**用户期望的工作流**
**<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>極雿𨀣<EFBFBD>**嚗?
```
銝𠹺<EFBFBD>嚗ǒ0: 100銵䕘<E98AB5>
筛选(v1: 50行)← 可以回退到这里
数值映射v2: 50行有映射← 可以回退到这里
Pivotv3: 不同结构)
导出:可以导出 v0、v1、v2、v3 任意版本
<EFBFBD>?
蝑偦<EFBFBD><EFBFBD>v1: 50銵䕘<EFBFBD><EFBFBD>?<3F>臭誑<E887AD><EFBFBD><E9AE8B><EFBFBD><EFBFBD><E59597>?
<EFBFBD>?
<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>v2: 50銵䕘<E98AB5><E49598><EFBFBD><EFBFBD><E692A0><EFBFBD>?<3F>臭誑<E887AD><EFBFBD><E9AE8B><EFBFBD><EFBFBD><E59597>?
<EFBFBD>?
Pivot嚗ǒ3: 銝滚<EFBFBD>蝏𤘪<EFBFBD>嚗?
<EFBFBD>?
撖澆枂嚗𡁜虾隞亙紡<EFBFBD>?v0<76><30>1<EFBFBD><31>2<EFBFBD><32>3 隞餅<E99A9E><E9A485><EFBFBD>𧋦
```
#### 閫<><E996AB><EFBFBD><EFBFBD>
**Prisma Schema设计**
**Prisma Schema霈曇恣**嚗?
```prisma
// 版本管理表(通用)
// <EFBFBD><EFBFBD>𧋦蝞∠<EFBFBD>銵剁<EFBFBD><EFBFBD>𡁶鍂嚗?
model DataVersion {
id String @id @default(uuid())
@@ -466,19 +466,19 @@ model DataVersion {
entityId String // Session ID | Project ID | ...
// <20><>𧋦靽⊥<E99DBD>
versionNumber Int // 0=原始, 1=第1次操作后, 2=第2次操作后...
versionNumber Int // 0=<EFBFBD><EFBFBD>, 1=蝚?甈⊥<E79488>雿𨅯<E99BBF>, 2=蝚?甈⊥<E79488>雿𨅯<E99BBF>...
dataKey String // OSS銝剔<E98A9D><E58994>唳旿<E594B3><E697BF>辣key
// <20><EFBFBD>霈啣<E99C88>
operation String? // 'upload' | 'filter' | 'pivot' | 'recode' ...
operationParams Json? // <20><EFBFBD><E6BBA2><EFBFBD>
// 元数据
// <EFBFBD><EFBFBD><EFBFBD>?
totalRows Int
totalCols Int
columns Json
// 时间戳
// <EFBFBD>園𡢿<EFBFBD>?
createdAt DateTime @default(now())
createdBy String // <20><EFBFBD>ID
@@ -488,22 +488,22 @@ model DataVersion {
@@schema("platform_schema")
}
// 业务表添加字段
// 銝𡁜𦛚銵冽溶<EFBFBD><EFBFBD>畾?
model DcToolCSession {
// ...<2E><EFBFBD>摮埈挾
currentVersion Int @default(0) // 当前版本号
currentVersion Int @default(0) // 敶枏<EFBFBD><EFBFBD><EFBFBD>𧋦<EFBFBD>?
}
```
**Service实现**
**Service摰䂿緵**嚗?
```typescript
// backend/src/common/services/DataVersionService.ts嚗<73>鰵撱綽<E692B1>
export class DataVersionService {
/**
* 创建新版本
* <EFBFBD>𥕦遣<EFBFBD><EFBFBD><EFBFBD>?
*/
async createVersion(
entityType: string,
@@ -513,7 +513,7 @@ export class DataVersionService {
operation?: string,
params?: any
): Promise<string> {
// 保存数据到 OSS
// 靽嘥<EFBFBD><EFBFBD>唳旿<EFBFBD>?OSS
const dataKey = `versions/${entityType}/${entityId}/v${versionNumber}.json`;
await storage.upload(dataKey, JSON.stringify(data));
@@ -553,21 +553,21 @@ export class DataVersionService {
}
});
if (!version) throw new Error('版本不存在');
if (!version) throw new Error('<EFBFBD><EFBFBD>𧋦銝滚<EFBFBD><EFBFBD>?);
const buffer = await storage.download(version.dataKey);
return JSON.parse(buffer.toString('utf-8'));
}
/**
* 回退到指定版本
* <EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>摰𡁶<EFBFBD><EFBFBD>?
*/
async rollbackToVersion(
entityType: string,
entityId: string,
versionNumber: number
): Promise<void> {
// 更新当前版本号
// <EFBFBD>湔鰵敶枏<EFBFBD><EFBFBD><EFBFBD>𧋦<EFBFBD>?
await this.updateCurrentVersion(entityType, entityId, versionNumber);
}
@@ -583,16 +583,16 @@ export class DataVersionService {
}
```
**QuickAction 集成**
**QuickAction <EFBFBD><EFBFBD><EFBFBD>**嚗?
```typescript
// QuickAction <20><EFBFBD><E689AF>舘䌊<E88898><EFBFBD>撱箸鰵<E7AEB8><E9B0B5>𧋦
const result = await python.execute(fullData, params);
// 读取当前版本号
// 霂餃<EFBFBD>敶枏<EFBFBD><EFBFBD><EFBFBD>𧋦<EFBFBD>?
const currentVersion = session.currentVersion || 0;
// 创建新版本
// <EFBFBD>𥕦遣<EFBFBD><EFBFBD><EFBFBD>?
await dataVersionService.createVersion(
'dc_toolc_session',
sessionId,
@@ -602,7 +602,7 @@ await dataVersionService.createVersion(
params
);
// 更新当前版本号
// <EFBFBD>湔鰵敶枏<EFBFBD><EFBFBD><EFBFBD>𧋦<EFBFBD>?
await prisma.update({
where: { id: sessionId },
data: { currentVersion: currentVersion + 1 }
@@ -611,28 +611,28 @@ await prisma.update({
#### 敶勗<E695B6><E58B97><EFBFBD>
- Tool C: 支持链式操作、undo、导出历史
- ASL: 文献筛选的多阶段结果管理
- Tool B: 数据提取的版本管理
- 所有模块: 统一的版本管理能力
- Tool C: <EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ndo<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>?
- ASL: <EFBFBD><EFBFBD>讃蝑偦<EFBFBD><EFBFBD>憭𡁻𧫴畾萇<EFBFBD><EFBFBD>𦦵恣<EFBFBD>?
- Tool B: <EFBFBD>唳旿<EFBFBD>𣂼<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>祉恣<EFBFBD>?
- <EFBFBD><EFBFBD><EFBFBD>㗇芋<EFBFBD>? 蝏煺<E89D8F><E785BA><EFBFBD><EFBFBD><EFBFBD>祉恣<E7A589><E681A3><EFBFBD><EFBFBD>?
---
### TD-COMMON-006: 幽灵列/行检测算法库
### TD-COMMON-006: 撟賜<EFBFBD><EFBFBD>?銵峕<E98AB5>瘚讠<E7989A>瘜訫<E7989C>
**优先级**:⭐⭐ P2
**工作量**0.5天
**隡睃<EFBFBD>蝥?*嚗尠<E59A97>潃?P2嚗<32>葉嚗?
**撌乩<EFBFBD><EFBFBD>?*嚗?.5憭?
**憸<><E686B8><EFBFBD><EFBFBD>**嚗鍃xcel憭<6C><E686AD>韐券<E99F90><E588B8>𣂼<EFBFBD>
#### <20><EFBFBD><E6A185>讛膩
**当前状态**
- Tool C 实现:边界检测算法
**敶枏<EFBFBD><EFBFBD><EFBFBD>?*嚗?
- Tool C 摰䂿緵嚗朞器<EFBFBD><EFBFBD>瘚讠<EFBFBD>瘜?
- <20><EFBFBD><E79285>嚗𡁏𧊋摰䂿緵
**Excel格式污染问题**
- 用户刷颜色到16384列 → 解析出16384列实际只有151列有效
- 用户删除数据未清理 → 解析出大量空行
**Excel<EFBFBD><EFBFBD>瘙⊥<EFBFBD><EFBFBD><EFBFBD>**嚗?
- <EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>16384<EFBFBD>?<3F>?閫<><E996AB><EFBFBD>?6384<38><EFBFBD>摰鮋<E691B0><E9AE8B><EFBFBD>151<35><EFBFBD><E59F88><EFBFBD><EFBFBD>
- <EFBFBD><EFBFBD><EFBFBD>𣳇膄<EFBFBD>唳旿<EFBFBD><EFBFBD><EFBFBD>?<3F>?閫<><E996AB><EFBFBD>箏之<E7AE8F>讐征銵?
#### 閫<><E996AB><EFBFBD><EFBFBD>
@@ -654,23 +654,23 @@ export function cleanExcelData(
rawData: any[],
options?: CleaningOptions
): CleaningResult {
// 实现通用的清洗算法
// 摰䂿緵<EFBFBD>𡁶鍂<EFBFBD><EFBFBD><EFBFBD>瘣㛖<EFBFBD>瘜?
// ...
}
```
---
### TD-COMMON-007: 前端进度条组件
### TD-COMMON-007: <EFBFBD>滨垢餈𥕦漲<EFBFBD><EFBFBD>隞?
**优先级**:⭐⭐ P2
**工作量**0.3天
**预期收益**UI统一用户体验提升
**隡睃<EFBFBD>蝥?*嚗尠<E59A97>潃?P2嚗<32>葉嚗?
**撌乩<EFBFBD><EFBFBD>?*嚗?.3憭?
**<EFBFBD><EFBFBD><EFBFBD><EFBFBD>**嚗䦧I蝏煺<E89D8F>嚗𣬚鍂<F0A3AC9A><EFBFBD>撉峕<E69289><E5B395>?
#### <20><EFBFBD><E6A185>讛膩
**当前状态**
- Tool C 实现:内联进度条(蓝色,Header下方)
**敶枏<EFBFBD><EFBFBD><EFBFBD>?*嚗?
- Tool C 摰䂿緵嚗𡁜<EFBFBD><EFBFBD><EFBFBD>摨行辺嚗<EFBFBD><EFBFBD><EFBFBD><EFBFBD>Header銝𧢲䲮嚗?
- <20><EFBFBD><E79285>嚗𡁏𧊋摰䂿緵<E482BF><EFBFBD>蝏煺<E89D8F>
#### 閫<><E996AB><EFBFBD><EFBFBD>
@@ -714,63 +714,64 @@ export const AsyncProgressBar: React.FC<AsyncProgressBarProps> = ({
## <20><> 摰墧鴌霈<E99C88>
| 技术债务 | 优先级 | 工作量 | 建议时机 | 收益 |
| <EFBFBD><EFBFBD><EFBFBD><EFBFBD>箏𦛚 | 隡睃<E99AA1>蝥?| 撌乩<E6928C><E4B9A9>?| 撱箄悅<E7AE84>嗆㦤 | <20><EFBFBD> |
|---------|--------|--------|---------|------|
| TD-COMMON-001 前端轮询Hook | P2 | 0.5天 | 下次迭代 | 代码统一 |
| TD-COMMON-002 Clean Data服务 | **P1** | 1| **下次迭代** | **性能提升99%** |
| TD-COMMON-003 智能清洗算法 | P2 | 0.5天 | 需要时 | 质量提升 |
| TD-COMMON-004 Worker注册辅助 | P3 | 0.3天 | 可选 | 代码简化 |
| TD-COMMON-005 数据版本管理 | **P1** | 3| **下次迭代** | **链式操作** |
| TD-COMMON-006 幽灵列/行检测 | P2 | 0.5天 | 需要时 | 质量提升 |
| TD-COMMON-007 进度条组件 | P2 | 0.3天 | 需要时 | UI统一 |
| TD-COMMON-001 <EFBFBD>滨垢頧株砭Hook | P2 | 0.5憭?| 銝𧢲活餈凋誨 | 隞<><E99A9E>蝏煺<E89D8F> |
| TD-COMMON-002 Clean Data<EFBFBD>滚𦛚 | **P1** | 1憭?| **銝𧢲活餈凋誨** | **<EFBFBD><EFBFBD><EFBFBD>𣂼<EFBFBD>99%** |
| TD-COMMON-003 <EFBFBD><EFBFBD><EFBFBD><EFBFBD>蝞埈<EFBFBD> | P2 | 0.5憭?| <20><><EFBFBD>𧒄 | 韐券<E99F90><E588B8>𣂼<EFBFBD> |
| TD-COMMON-004 Worker瘜典<EFBFBD><EFBFBD>𨭌 | P3 | 0.3憭?| <20><EFBFBD>?| 隞<><E99A9E><EFBFBD><E89D9E>?|
| TD-COMMON-005 <EFBFBD>唳旿<EFBFBD><EFBFBD>𧋦蝞∠<EFBFBD> | **P1** | 3憭?| **銝𧢲活餈凋誨** | **<EFBFBD><EFBFBD><EFBFBD><EFBFBD>** |
| TD-COMMON-006 撟賜<EFBFBD><EFBFBD>?銵峕<E98AB5>瘚?| P2 | 0.5憭?| <20><><EFBFBD>𧒄 | 韐券<E99F90><E588B8>𣂼<EFBFBD> |
| TD-COMMON-007 餈𥕦漲<EFBFBD><EFBFBD>隞?| P2 | 0.3憭?| <20><><EFBFBD>𧒄 | UI蝏煺<EFBFBD> |
**推荐优先级**
1. ⭐⭐⭐⭐⭐ TD-COMMON-002Clean Data服务)
2. ⭐⭐⭐⭐⭐ TD-COMMON-005(数据版本管理)
3. ⭐⭐⭐ TD-COMMON-001前端轮询Hook
**<EFBFBD><EFBFBD>隡睃<EFBFBD>蝥?*嚗?
1. 潃鐥<EFBFBD>潃鐥<EFBFBD>潃?TD-COMMON-002嚗㇃lean Data<EFBFBD>滚𦛚嚗?
2. 潃鐥<EFBFBD>潃鐥<EFBFBD>潃?TD-COMMON-005<EFBFBD><EFBFBD><EFBFBD><EFBFBD>祉恣<EFBFBD><EFBFBD><EFBFBD>
3. 潃鐥<EFBFBD>潃?TD-COMMON-001<EFBFBD><EFBFBD>蝡航蔭霂ook嚗?
---
## <20><> 敶枏<E695B6><E69E8F><EFBFBD><EFBFBD>
### ✅ 已经很好的部分
### <EFBFBD>?撌脩<E6928C><EFBFBD><EFBFBD><E69CAB><EFBFBD><EFBFBD>?
1. **通用能力层基础设施完整**
- jobQueuepg-boss队列)
1. <EFBFBD>?**<2A>𡁶鍂<F0A181B6><EFBFBD><EFBFBD>抅蝖<E68A85>霈暹鴌摰峕㟲**
- jobQueue嚗īg-boss<EFBFBD><EFBFBD>嚗?
- CheckpointService嚗<65><EFBFBD>寧賒隡𩤃<E99AA1>
- 隞餃𦛚<E9A483><F0A69B9A><EFBFBD>撌亙<E6928C>
- storage<67><65>ogger<65><72>ache
2. **业务模块正确使用通用能力**
2. <EFBFBD>?**銝𡁜𦛚璅<E79285><EFBFBD>雿輻鍂<E8BCBB>𡁶鍂<F0A181B6><EFBFBD>**
- Tool C<><43>SL<53><4C>ool B 蝏煺<E89D8F>雿輻鍂 jobQueue
- Platform-Only璅<E79285>
- 銝滚銁銝𡁜𦛚銵其葉摮睃<E691AE>隞餃𦛚蝞∠<E89D9E>靽⊥<E99DBD>
3. **分层清晰,职责明确**
3. <EFBFBD>?**<2A><><EFBFBD><EFBFBD>苊嚗諹<E59A97><EFBFBD><E99F90>蝖?*
- <20>𡁶鍂<F0A181B6><EFBFBD><EFBFBD><E69285><EFBFBD><EFBFBD>霈暹鴌
- 业务模块:具体实现
- 銝𡁜𦛚璅<EFBFBD>嚗𡁜<EFBFBD>雿枏<EFBFBD><EFBFBD>?
### ⏭️ 可以改进的部分(非必需)
### <EFBFBD><EFBFBD> <20>臭誑<E887AD><EFBFBD><E5AFA1><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>𧼮<EFBFBD><F0A7BCAE><EFBFBD>嚗?
1. <20><EFBFBD> <20>滨垢頧株砭璅<E79285><E288AA>臭誑<E887AD><EFBFBD><EFBFBD>
2. <20><EFBFBD> clean data蝻枏<E89DBB><E69E8F>臭誑<E887AD><EFBFBD>𡁶鍂
3. ⏭️ 数据版本管理待建立
3. <EFBFBD><EFBFBD> <20>唳旿<E594B3><E697BF>𧋦蝞∠<E89D9E><EFBFBD>遣蝡?
**蝏栞捏**嚗𡁜<E59A97><F0A1819C>齿沲<E9BDBF><E6B2B2>歇蝏讐泵<E8AE90><E6B3B5><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>芣䔉隡睃<E99AA1><E79D83>舫釺銝𦠜溶<F0A6A09C><EFBFBD>
---
## 📚 参考文档
## <EFBFBD><EFBFBD> <20><><EFBFBD><EFBFBD><EFBFBD>獢?
- [Postgres-Only撘<79>郊隞餃𦛚憭<F0A69B9A><E686AD><EFBFBD><EFBFBD><EFBFBD>](./Postgres-Only撘<79>郊隞餃𦛚憭<F0A69B9A><E686AD><EFBFBD><EFBFBD><EFBFBD>.md) - 摰峕㟲摰噼殿
- [云原生开发规范](../04-开发规范/08-云原生开发规范.md) - 开发规范
- [DC Tool C状态](../03-业务模块/DC-数据清洗整理/00-工具C当前状态与开发指南.md) - Day 10实践
- [鈭穃<EFBFBD><EFBFBD><EFBFBD><EFBFBD>𤏸<EFBFBD><EFBFBD><EFBFBD>(../04-撘<><E69298>𤏸<EFBFBD><F0A48FB8>?08-鈭穃<E988AD><E7A983><EFBFBD><E7AC94>𤏸<EFBFBD><F0A48FB8>?md) - <EFBFBD><EFBFBD>𤏸<EFBFBD><EFBFBD>?
- [DC Tool C<EFBFBD><EFBFBD><EFBFBD>(../03-銝𡁜𦛚璅<E79285>/DC-<2D>唳旿皜<E697BF><E79A9C><EFBFBD><EFBFBD>/00-撌亙<E6928C>C敶枏<E695B6><E69E8F><EFBFBD><E59786><EFBFBD><EFBFBD><E69298><EFBFBD><E78390>?md) - Day 10摰噼殿
---
**维护者**: 平台架构团队
**最后更新**: 2025-12-22
**文档状态**: ✅ 初始版本
**蝏湔擪<EFBFBD>?*: 撟喳蝱<E596B3><EFBFBD><E59786><EFBFBD>
**<EFBFBD><EFBFBD><EFBFBD>擧凒<EFBFBD>?*: 2025-12-22
**<EFBFBD><EFBFBD><EFBFBD><EFBFBD>?*: <20>?<3F><EFBFBD><E598A5><EFBFBD>𧋦