feat(admin): Add user management and upgrade to module permission system

Features - User Management (Phase 4.1):
- Database: Add user_modules table for fine-grained module permissions
- Database: Add 4 user permissions (view/create/edit/delete) to role_permissions
- Backend: UserService (780 lines) - CRUD with tenant isolation
- Backend: UserController + UserRoutes (648 lines) - 13 API endpoints
- Backend: Batch import users from Excel
- Frontend: UserListPage (412 lines) - list/filter/search/pagination
- Frontend: UserFormPage (341 lines) - create/edit with module config
- Frontend: UserDetailPage (393 lines) - details/tenant/module management
- Frontend: 3 modal components (592 lines) - import/assign/configure
- API: GET/POST/PUT/DELETE /api/admin/users/* endpoints

Architecture Upgrade - Module Permission System:
- Backend: Add getUserModules() method in auth.service
- Backend: Login API returns modules array in user object
- Frontend: AuthContext adds hasModule() method
- Frontend: Navigation filters modules based on user.modules
- Frontend: RouteGuard checks requiredModule instead of requiredVersion
- Frontend: Remove deprecated version-based permission system
- UX: Only show accessible modules in navigation (clean UI)
- UX: Smart redirect after login (avoid 403 for regular users)

Fixes:
- Fix UTF-8 encoding corruption in ~100 docs files
- Fix pageSize type conversion in userService (String to Number)
- Fix authUser undefined error in TopNavigation
- Fix login redirect logic with role-based access check
- Update Git commit guidelines v1.2 with UTF-8 safety rules

Database Changes:
- CREATE TABLE user_modules (user_id, tenant_id, module_code, is_enabled)
- ADD UNIQUE CONSTRAINT (user_id, tenant_id, module_code)
- INSERT 4 permissions + role assignments
- UPDATE PUBLIC tenant with 8 module subscriptions

Technical:
- Backend: 5 new files (~2400 lines)
- Frontend: 10 new files (~2500 lines)
- Docs: 1 development record + 2 status updates + 1 guideline update
- Total: ~4900 lines of code

Status: User management 100% complete, module permission system operational
This commit is contained in:
2026-01-16 13:42:10 +08:00
parent 98d862dbd4
commit 66255368b7
560 changed files with 70424 additions and 52353 deletions

View File

@@ -1,20 +1,20 @@
# 通用能力层技术债务清单
> **<EFBFBD><EFBFBD><EFBFBD><EFBFBD>𧋦嚗?* v1.0
> **<EFBFBD>𥕦遣<EFBFBD><EFBFBD>嚗?* 2025-12-22
> **文档版本:** v1.0
> **创建日期:** 2025-12-22
> **维护者:** 平台架构团队
> **<EFBFBD><EFBFBD><EFBFBD><EFBFBD>嚗?* 霈啣<E99C88><E595A3>𡁶鍂<F0A181B6><EFBFBD><EFBFBD><E69285>隡睃<E99AA1>憿對<E686BF><E5B08D><EFBFBD><EFBFBD>芣䔉餈凋誨
> **文档目的:** 记录通用能力层待优化项,指导未来迭代
---
## 📋 概述
<EFBFBD><EFBFBD><EFBFBD>抅鈭?**DC Tool C <EFBFBD><EFBFBD><EFBFBD>摰噼殿**嚗?025-12-22嚗㚁<E59A97><E39A81><EFBFBD><E9A48C>𤑳緵<F0A491B3><E7B7B5>虾隞交𡂝鞊∩蛹<E288A9>𡁶鍂<F0A181B6><EFBFBD><E8B3A2><EFBFBD>芋撘譌<E69298>?
本文档基于 **DC Tool C 异步架构实践**2025-12-22总结发现的可以抽象为通用能力的模式。
**<EFBFBD><EFBFBD><EFBFBD><EFBFBD>**嚗?
- <EFBFBD>?**敶枏<E695B6><E69E8F><EFBFBD><EFBFBD>撌脩<E6928C><EFBFBD>末**嚗<><E59A97>𡁶鍂<F0A181B6><EFBFBD><EFBFBD><E69285>靘𥕦<E99D98><F0A595A6><EFBFBD><E6B8A1><EFBFBD>霈暹鴌嚗?
- <EFBFBD>?**銝𡁜𦛚璅<E79285><EFBFBD>雿輻鍂鈭<E98D82><E988AD>鈭𥡝<E988AD><F0A5A19D>?*
- <EFBFBD><EFBFBD> **<EFBFBD>芣䔉<EFBFBD>臭誑餈𥕢<EFBFBD>甇交𡂝鞊?*嚗<>釺銝𦠜溶<F0A6A09C><EFBFBD><E6A2A7>𧼮<EFBFBD><F0A7BCAE><EFBFBD>嚗?
**核心思想**
- **当前分层已经很好**(通用能力层提供完整的基础设施)
- **业务模块正确使用了这些能力**
- ⏭️ **未来可以进一步抽象**(锦上添花,非必需)
---
@@ -22,18 +22,18 @@
### TD-COMMON-001: 前端通用轮询Hook
**隡睃<EFBFBD>蝥?*嚗尠<E59A97>潃鐥<E6BD83> P2嚗<32>葉嚗?
**撌乩<EFBFBD><EFBFBD>?*嚗?.5憭?
**优先级**:⭐⭐⭐ P2
**工作量**0.5天
**预期收益**:代码复用性提升,模块间统一
#### 问题描述
**敶枏<EFBFBD><EFBFBD><EFBFBD>?*嚗?
**当前状态**
- Tool C 实现:`dc/tool-c/hooks/useSessionStatus.ts`
- ASL 实现:`asl/hooks/useScreeningTask.ts`
- 代码重复度70%
**<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>**嚗?
**重复的逻辑**
```typescript
// 每个模块都要写类似的代码
useQuery({
@@ -51,7 +51,7 @@ useQuery({
#### 解决方案
**<EFBFBD>質情銝粹<EFBFBD>𡁶鍂Hook**嚗?
**抽象为通用Hook**
```typescript
// frontend-v2/src/common/hooks/useAsyncTaskPolling.ts新建
@@ -65,7 +65,7 @@ interface UseAsyncTaskPollingOptions<T> {
/** 状态查询API函数 */
queryFn: (taskId: string) => Promise<T>;
/** <EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>硋遆<EFBFBD>?*/
/** 状态提取函数 */
getStatus: (data: T) => 'pending' | 'processing' | 'ready' | 'error' | string;
/** 进度提取函数(可选) */
@@ -74,7 +74,7 @@ interface UseAsyncTaskPollingOptions<T> {
/** 是否启用 */
enabled?: boolean;
/** 頧株砭<EFBFBD><EFBFBD><EFBFBD>神蝘𡜐<EFBFBD>嚗屸<EFBFBD>霈?000 */
/** 轮询间隔毫秒默认2000 */
pollingInterval?: number;
}
@@ -122,7 +122,7 @@ export function useAsyncTaskPolling<T>({
}
```
**雿輻鍂蝷箔<EFBFBD>**嚗?
**使用示例**
```typescript
// Tool C 使用
@@ -144,33 +144,33 @@ const { status, isReady } = useAsyncTaskPolling({
#### 影响范围
- Tool C: `useSessionStatus.ts` <EFBFBD><EFBFBD><EFBFBD>?
- ASL: `useScreeningTask.ts` <EFBFBD><EFBFBD><EFBFBD>?
- Tool B: <EFBFBD>芣䔉<EFBFBD>滨垢<EFBFBD>舐凒<EFBFBD>乩蝙<EFBFBD>?
- Tool C: `useSessionStatus.ts` 可简化
- ASL: `useScreeningTask.ts` 可简化
- Tool B: 未来前端可直接使用
- 其他模块: 直接复用
---
### TD-COMMON-002: Clean Data 缓存服务
**隡睃<EFBFBD>蝥?*嚗尠<E59A97>潃鐥<E6BD83>潃?P1嚗<31><E59A97>嚗?
**撌乩<EFBFBD><EFBFBD>?*嚗?憭?
**<EFBFBD><EFBFBD><EFBFBD><EFBFBD>**嚗𡁏<E59A97><EFBFBD><E689AF>𣂼<EFBFBD>99%嚗峕<E59A97><E5B395>㗇芋<E39787><EFBFBD><E5A092>?
**优先级**:⭐⭐⭐⭐ P1
**工作量**1天
**预期收益**性能提升99%,所有模块受益
#### 问题描述
**敶枏<EFBFBD><EFBFBD><EFBFBD>?*嚗?
- Tool C 摰䂿緵嚗帋<EFBFBD>摮?`${fileKey}_clean.json`
- ASL<EFBFBD><EFBFBD>ool B嚗𡁏𧊋摰䂿緵嚗䔶<EFBFBD><EFBFBD><EFBFBD>甈⊿<EFBFBD><EFBFBD>啗圾<EFBFBD>?
**当前状态**
- Tool C 实现:保存 `${fileKey}_clean.json`
- ASL、Tool B:未实现,仍然每次重新解析
**<EFBFBD><EFBFBD>霈∠<EFBFBD><EFBFBD><EFBFBD>**嚗?
- ASL <EFBFBD><EFBFBD>讃蝑偦<EFBFBD><EFBFBD>瘥𤩺活隞?OSS 銝贝蝸 PDF嚗屸<EFBFBD><EFBFBD>啗圾<EFBFBD><EFBFBD>5-10蝘?蝭<><E89DAD>
- Tool B <EFBFBD>唳旿<EFBFBD>𣂼<EFBFBD>嚗𡁏<EFBFBD>甈⊿<EFBFBD><EFBFBD>啗粉<EFBFBD>?Excel
- Tool C <EFBFBD><EFBFBD>嚗𡁜歇隡睃<EFBFBD>嚗Ếlean data蝻枏<EFBFBD>嚗?
**重复计算问题**
- ASL 文献筛选:每次从 OSS 下载 PDF重新解析5-10秒/篇)
- Tool B 数据提取:每次重新读取 Excel
- Tool C 操作已优化clean data缓存)
#### 解决方案
**<EFBFBD>質情銝粹<EFBFBD>𡁶鍂<EFBFBD>滚𦛚**嚗?
**抽象为通用服务**
```typescript
// backend/src/common/services/DataCacheService.ts新建
@@ -180,12 +180,12 @@ import { prisma } from '../../config/database';
import { logger } from '../logging';
/**
* <EFBFBD>唳旿蝻枏<EFBFBD><EFBFBD>滚𦛚嚗<EFBFBD><EFBFBD>𡁶鍂嚗?
* 数据缓存服务(通用)
*
* 用途:
* - Worker <EFBFBD><EFBFBD><EFBFBD>𦒘<EFBFBD>摮睃<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>?
* - Service 隡睃<EFBFBD>霂餃<EFBFBD>蝻枏<EFBFBD>嚗屸<EFBFBD><EFBFBD><EFBFBD>憭滩恣蝞?
* - <EFBFBD><EFBFBD><EFBFBD>𤾸<EFBFBD>甇交凒<EFBFBD><EFBFBD>摮?
* - Worker 解析后保存处理结果
* - Service 优先读取缓存,避免重复计算
* - 操作后同步更新缓存
*/
export class DataCacheService {
/**
@@ -193,8 +193,8 @@ export class DataCacheService {
*
* @param originalKey 原始文件key
* @param cleanData 清洗后的数据
* @param suffix <EFBFBD>𡒊<EFBFBD><EFBFBD><EFBFBD>霈?'_clean.json'嚗?
* @returns clean data <EFBFBD>?OSS key
* @param suffix 后缀(默认 '_clean.json'
* @returns clean data OSS key
*/
async saveCleanData(
originalKey: string,
@@ -223,7 +223,7 @@ export class DataCacheService {
/**
* 读取清洗后的数据
*
* @param cleanDataKey clean data <EFBFBD>?OSS key
* @param cleanDataKey clean data OSS key
* @returns 清洗后的数据
*/
async getCleanData(cleanDataKey: string): Promise<any> {
@@ -242,8 +242,8 @@ export class DataCacheService {
/**
* 更新清洗后的数据
*
* @param cleanDataKey clean data <EFBFBD>?OSS key
* @param newData <EFBFBD>唳㺭<EFBFBD>?
* @param cleanDataKey clean data OSS key
* @param newData 新数据
*/
async updateCleanData(cleanDataKey: string, newData: any): Promise<void> {
logger.info('[DataCacheService] Updating clean data', {
@@ -260,7 +260,7 @@ export class DataCacheService {
/**
* 删除清洗后的数据
*
* @param cleanDataKey clean data <EFBFBD>?OSS key
* @param cleanDataKey clean data OSS key
*/
async deleteCleanData(cleanDataKey: string): Promise<void> {
try {
@@ -278,46 +278,46 @@ export class DataCacheService {
export const dataCacheService = new DataCacheService();
```
**雿輻鍂蝷箔<EFBFBD>**嚗?
**使用示例**
```typescript
// Worker 銝?
// Worker
const cleanDataKey = await dataCacheService.saveCleanData(fileKey, cleanedData);
await prisma.update({ where: { id }, data: { cleanDataKey } });
// Service 銝?
// Service
if (record.cleanDataKey) {
return await dataCacheService.getCleanData(record.cleanDataKey);
}
// <EFBFBD><EFBFBD><EFBFBD>擧凒<EFBFBD>?
// 操作后更新
await dataCacheService.updateCleanData(record.cleanDataKey, newData);
```
#### 影响范围
- Tool C: <EFBFBD><EFBFBD>𣇉緵<EFBFBD>劐誨<EFBFBD>?
- ASL: <EFBFBD><EFBFBD>讃閫<EFBFBD><EFBFBD>蝏𤘪<EFBFBD>蝻枏<EFBFBD><EFBFBD><EFBFBD><EFBFBD>?9%嚗?
- Tool C: 简化现有代码
- ASL: 文献解析结果缓存提升99%
- Tool B: 数据提取结果缓存
- <EFBFBD><EFBFBD><EFBFBD>㗇芋<EFBFBD>? 蝏煺<E89D8F><E785BA><EFBFBD><EFBFBD>摮䀹㦤<E480B9>?
- 所有模块: 统一的缓存机制
---
### TD-COMMON-003: <EFBFBD><EFBFBD><EFBFBD><EFBFBD>蝞埈<EFBFBD><EFBFBD>𡁶鍂<EFBFBD>?
### TD-COMMON-003: 智能清洗算法通用化
**隡睃<EFBFBD>蝥?*嚗尠<E59A97>潃?P2嚗<32>葉嚗?
**撌乩<EFBFBD><EFBFBD>?*嚗?.5憭?
**优先级**:⭐⭐ P2
**工作量**0.5天
**预期收益**:代码复用,避免重复实现
#### 问题描述
**敶枏<EFBFBD><EFBFBD><EFBFBD>?*嚗?
- Tool C 摰䂿緵嚗䫤intelligentCleanData`<EFBFBD><EFBFBD><EFBFBD>瘚?摰匧<E691B0><E58CA7><EFBFBD>嚗?
**当前状态**
- Tool C 实现:`intelligentCleanData`(边界检测+安全阀)
- 其他模块:未实现类似功能
**<EFBFBD>臭誑<EFBFBD>𡁶鍂<EFBFBD>𣇉<EFBFBD>蝞埈<EFBFBD>**嚗?
**可以通用化的算法**
1. 幽灵列检测(边界检测)
2. 撟賜<EFBFBD>銵諹<EFBFBD>皛?
2. 幽灵行过滤
3. 安全阀(最大列数、单元格数限制)
#### 解决方案
@@ -327,7 +327,7 @@ await dataCacheService.updateCleanData(record.cleanDataKey, newData);
export interface CleaningOptions {
maxCols?: number; // 最大列数默认3000
maxCells?: number; // <EFBFBD><EFBFBD>憭批<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>暺䁅恕500銝?
maxCells?: number; // 最大单元格数,默认500
removeEmptyRows?: boolean; // 是否删除空行默认true
removeEmptyCols?: boolean; // 是否删除空列默认true
}
@@ -341,7 +341,7 @@ export function intelligentCleanData(
}
export function isValidValue(value: any): boolean {
// 蝏煺<EFBFBD><EFBFBD><EFBFBD><EFBFBD>澆ế<EFBFBD>?
// 统一的空值判断
// ...
}
```
@@ -349,19 +349,19 @@ export function isValidValue(value: any): boolean {
#### 影响范围
- Tool C: 复用通用实现
- <EFBFBD><EFBFBD>銝𠹺<EFBFBD>Excel<EFBFBD><EFBFBD><EFBFBD><EFBFBD>? <20>湔𦻖雿輻鍂
- 其他上传Excel的功能: 直接使用
---
### TD-COMMON-004: Worker注册辅助工具
**隡睃<EFBFBD>蝥?*嚗尠<E59A97> P3嚗<33><E59A97>嚗?
**撌乩<EFBFBD><EFBFBD>?*嚗?.3憭?
**优先级**:⭐ P3
**工作量**0.3天
**预期收益**降低Worker注册代码重复
#### 问题描述
**敶枏<EFBFBD><EFBFBD><EFBFBD>?*嚗?
**当前状态**
- 每个模块都要手写 Worker 注册代码
- 错误处理逻辑重复
@@ -403,11 +403,11 @@ export function registerWorker(options: RegisterWorkerOptions) {
}
});
logger.info(`[WorkerHelper] <EFBFBD>?Worker registered: ${queueName}`);
logger.info(`[WorkerHelper] Worker registered: ${queueName}`);
}
```
**雿輻鍂蝷箔<EFBFBD>**嚗?
**使用示例**
```typescript
// 简化的注册代码
@@ -418,7 +418,7 @@ registerWorker({
// 业务逻辑
return result;
},
onStart: (job) => console.log(`<EFBFBD>憪见<EFBFBD><EFBFBD>? ${job.id}`),
onStart: (job) => console.log(`开始处理: ${job.id}`),
onComplete: (job, result) => console.log(`完成: ${result}`),
onError: (job, error) => console.error(`失败: ${error}`),
});
@@ -428,36 +428,36 @@ registerWorker({
### TD-COMMON-005: 数据版本管理系统
**隡睃<EFBFBD>蝥?*嚗尠<E59A97>潃鐥<E6BD83>潃?P1嚗<31><E59A97>嚗?
**撌乩<EFBFBD><EFBFBD>?*嚗?憭?
**<EFBFBD><EFBFBD><EFBFBD><EFBFBD>**嚗𡁏𣈲<F0A1818F><F0A388B2>曎撘𤩺<E69298>雿栶<E99BBF><E6A0B6>ndo<64><EFBFBD><E8A098><EFBFBD><EFBFBD><EFBFBD><E7AE8F><EFBFBD><E884A9>?
**优先级**:⭐⭐⭐⭐ P1
**工作量**3天
**预期收益**支持链式操作、undo功能、导出历史版本
#### 问题描述
**敶枏<EFBFBD><EFBFBD>𣂼<EFBFBD>**嚗?
- <EFBFBD><EFBFBD>銝齿糓蝝舐妖<EFBFBD><EFBFBD><EFBFBD>瘥𤩺活<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>唳旿嚗?
- <EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>銝芣<EFBFBD>雿𨅯<EFBFBD><EFBFBD><EFBFBD>𠶖<EFBFBD>?
- <EFBFBD><EFBFBD>撖澆枂銝剝𡢿<EFBFBD><EFBFBD>𧋦<EFBFBD><EFBFBD><EFBFBD>?
**当前限制**
- 操作不是累积的(每次基于原始数据)
- 无法回退到某个操作前的状态
- 无法导出中间版本的数据
**<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>極雿𨀣<EFBFBD>**嚗?
**用户期望的工作流**
```
上传v0: 100行
<EFBFBD>?
蝑偦<EFBFBD><EFBFBD>v1: 50銵䕘<EFBFBD><EFBFBD>?<3F>臭誑<E887AD><EFBFBD><E9AE8B><EFBFBD><EFBFBD><E59597>?
<EFBFBD>?
<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>v2: 50銵䕘<E98AB5><E49598><EFBFBD><EFBFBD><E692A0><EFBFBD>?<3F>臭誑<E887AD><EFBFBD><E9AE8B><EFBFBD><EFBFBD><E59597>?
<EFBFBD>?
Pivot嚗ǒ3: 銝滚<EFBFBD>蝏𤘪<EFBFBD>嚗?
<EFBFBD>?
撖澆枂嚗𡁜虾隞亙紡<EFBFBD>?v0<76><30>1<EFBFBD><31>2<EFBFBD><32>3 隞餅<E99A9E><E9A485><EFBFBD>𧋦
筛选(v1: 50行)← 可以回退到这里
数值映射v2: 50行有映射← 可以回退到这里
Pivotv3: 不同结构)
导出:可以导出 v0、v1、v2、v3 任意版本
```
#### 解决方案
**Prisma Schema霈曇恣**嚗?
**Prisma Schema设计**
```prisma
// <EFBFBD><EFBFBD>𧋦蝞∠<EFBFBD>銵剁<EFBFBD><EFBFBD>𡁶鍂嚗?
// 版本管理表(通用)
model DataVersion {
id String @id @default(uuid())
@@ -466,19 +466,19 @@ model DataVersion {
entityId String // Session ID | Project ID | ...
// 版本信息
versionNumber Int // 0=<EFBFBD><EFBFBD>, 1=蝚?甈⊥<E79488>雿𨅯<E99BBF>, 2=蝚?甈⊥<E79488>雿𨅯<E99BBF>...
versionNumber Int // 0=原始, 1=第1次操作后, 2=第2次操作后...
dataKey String // OSS中的数据文件key
// 操作记录
operation String? // 'upload' | 'filter' | 'pivot' | 'recode' ...
operationParams Json? // 操作参数
// <EFBFBD><EFBFBD><EFBFBD>?
// 元数据
totalRows Int
totalCols Int
columns Json
// <EFBFBD>園𡢿<EFBFBD>?
// 时间戳
createdAt DateTime @default(now())
createdBy String // 用户ID
@@ -488,22 +488,22 @@ model DataVersion {
@@schema("platform_schema")
}
// 銝𡁜𦛚銵冽溶<EFBFBD><EFBFBD>畾?
// 业务表添加字段
model DcToolCSession {
// ...现有字段
currentVersion Int @default(0) // 敶枏<EFBFBD><EFBFBD><EFBFBD>𧋦<EFBFBD>?
currentVersion Int @default(0) // 当前版本号
}
```
**Service摰䂿緵**嚗?
**Service实现**
```typescript
// backend/src/common/services/DataVersionService.ts新建
export class DataVersionService {
/**
* <EFBFBD>𥕦遣<EFBFBD><EFBFBD><EFBFBD>?
* 创建新版本
*/
async createVersion(
entityType: string,
@@ -513,7 +513,7 @@ export class DataVersionService {
operation?: string,
params?: any
): Promise<string> {
// 靽嘥<EFBFBD><EFBFBD>唳旿<EFBFBD>?OSS
// 保存数据到 OSS
const dataKey = `versions/${entityType}/${entityId}/v${versionNumber}.json`;
await storage.upload(dataKey, JSON.stringify(data));
@@ -553,21 +553,21 @@ export class DataVersionService {
}
});
if (!version) throw new Error('<EFBFBD><EFBFBD>𧋦銝滚<EFBFBD><EFBFBD>?);
if (!version) throw new Error('版本不存在');
const buffer = await storage.download(version.dataKey);
return JSON.parse(buffer.toString('utf-8'));
}
/**
* <EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>摰𡁶<EFBFBD><EFBFBD>?
* 回退到指定版本
*/
async rollbackToVersion(
entityType: string,
entityId: string,
versionNumber: number
): Promise<void> {
// <EFBFBD>湔鰵敶枏<EFBFBD><EFBFBD><EFBFBD>𧋦<EFBFBD>?
// 更新当前版本号
await this.updateCurrentVersion(entityType, entityId, versionNumber);
}
@@ -583,16 +583,16 @@ export class DataVersionService {
}
```
**QuickAction <EFBFBD><EFBFBD><EFBFBD>**嚗?
**QuickAction 集成**
```typescript
// QuickAction 执行后自动创建新版本
const result = await python.execute(fullData, params);
// 霂餃<EFBFBD>敶枏<EFBFBD><EFBFBD><EFBFBD>𧋦<EFBFBD>?
// 读取当前版本号
const currentVersion = session.currentVersion || 0;
// <EFBFBD>𥕦遣<EFBFBD><EFBFBD><EFBFBD>?
// 创建新版本
await dataVersionService.createVersion(
'dc_toolc_session',
sessionId,
@@ -602,7 +602,7 @@ await dataVersionService.createVersion(
params
);
// <EFBFBD>湔鰵敶枏<EFBFBD><EFBFBD><EFBFBD>𧋦<EFBFBD>?
// 更新当前版本号
await prisma.update({
where: { id: sessionId },
data: { currentVersion: currentVersion + 1 }
@@ -611,28 +611,28 @@ await prisma.update({
#### 影响范围
- Tool C: <EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ndo<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>?
- ASL: <EFBFBD><EFBFBD>讃蝑偦<EFBFBD><EFBFBD>憭𡁻𧫴畾萇<EFBFBD><EFBFBD>𦦵恣<EFBFBD>?
- Tool B: <EFBFBD>唳旿<EFBFBD>𣂼<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>祉恣<EFBFBD>?
- <EFBFBD><EFBFBD><EFBFBD>㗇芋<EFBFBD>? 蝏煺<E89D8F><E785BA><EFBFBD><EFBFBD><EFBFBD>祉恣<E7A589><E681A3><EFBFBD><EFBFBD>?
- Tool C: 支持链式操作、undo、导出历史
- ASL: 文献筛选的多阶段结果管理
- Tool B: 数据提取的版本管理
- 所有模块: 统一的版本管理能力
---
### TD-COMMON-006: 撟賜<EFBFBD><EFBFBD>?銵峕<E98AB5>瘚讠<E7989A>瘜訫<E7989C>
### TD-COMMON-006: 幽灵列/行检测算法库
**隡睃<EFBFBD>蝥?*嚗尠<E59A97>潃?P2嚗<32>葉嚗?
**撌乩<EFBFBD><EFBFBD>?*嚗?.5憭?
**优先级**:⭐⭐ P2
**工作量**0.5天
**预期收益**Excel处理质量提升
#### 问题描述
**敶枏<EFBFBD><EFBFBD><EFBFBD>?*嚗?
- Tool C 摰䂿緵嚗朞器<EFBFBD><EFBFBD>瘚讠<EFBFBD>瘜?
**当前状态**
- Tool C 实现:边界检测算法
- 其他模块:未实现
**Excel<EFBFBD><EFBFBD>瘙⊥<EFBFBD><EFBFBD><EFBFBD>**嚗?
- <EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>16384<EFBFBD>?<3F>?閫<><E996AB><EFBFBD>?6384<38><EFBFBD>摰鮋<E691B0><E9AE8B><EFBFBD>151<35><EFBFBD><E59F88><EFBFBD><EFBFBD>
- <EFBFBD><EFBFBD><EFBFBD>𣳇膄<EFBFBD>唳旿<EFBFBD><EFBFBD><EFBFBD>?<3F>?閫<><E996AB><EFBFBD>箏之<E7AE8F>讐征銵?
**Excel格式污染问题**
- 用户刷颜色到16384列 → 解析出16384列实际只有151列有效
- 用户删除数据未清理 → 解析出大量空行
#### 解决方案
@@ -654,23 +654,23 @@ export function cleanExcelData(
rawData: any[],
options?: CleaningOptions
): CleaningResult {
// 摰䂿緵<EFBFBD>𡁶鍂<EFBFBD><EFBFBD><EFBFBD>瘣㛖<EFBFBD>瘜?
// 实现通用的清洗算法
// ...
}
```
---
### TD-COMMON-007: <EFBFBD>滨垢餈𥕦漲<EFBFBD><EFBFBD>隞?
### TD-COMMON-007: 前端进度条组件
**隡睃<EFBFBD>蝥?*嚗尠<E59A97>潃?P2嚗<32>葉嚗?
**撌乩<EFBFBD><EFBFBD>?*嚗?.3憭?
**<EFBFBD><EFBFBD><EFBFBD><EFBFBD>**嚗䦧I蝏煺<E89D8F>嚗𣬚鍂<F0A3AC9A><EFBFBD>撉峕<E69289><E5B395>?
**优先级**:⭐⭐ P2
**工作量**0.3天
**预期收益**UI统一用户体验提升
#### 问题描述
**敶枏<EFBFBD><EFBFBD><EFBFBD>?*嚗?
- Tool C 摰䂿緵嚗𡁜<EFBFBD><EFBFBD><EFBFBD>摨行辺嚗<EFBFBD><EFBFBD><EFBFBD><EFBFBD>Header銝𧢲䲮嚗?
**当前状态**
- Tool C 实现:内联进度条(蓝色,Header下方)
- 其他模块:未实现或不统一
#### 解决方案
@@ -714,64 +714,63 @@ export const AsyncProgressBar: React.FC<AsyncProgressBarProps> = ({
## 📅 实施计划
| <EFBFBD><EFBFBD><EFBFBD><EFBFBD>箏𦛚 | 隡睃<E99AA1>蝥?| 撌乩<E6928C><E4B9A9>?| 撱箄悅<E7AE84>嗆㦤 | <20><EFBFBD> |
| 技术债务 | 优先级 | 工作量 | 建议时机 | 收益 |
|---------|--------|--------|---------|------|
| TD-COMMON-001 <EFBFBD>滨垢頧株砭Hook | P2 | 0.5憭?| 銝𧢲活餈凋誨 | 隞<><E99A9E>蝏煺<E89D8F> |
| TD-COMMON-002 Clean Data<EFBFBD>滚𦛚 | **P1** | 1憭?| **銝𧢲活餈凋誨** | **<EFBFBD><EFBFBD><EFBFBD>𣂼<EFBFBD>99%** |
| TD-COMMON-003 <EFBFBD><EFBFBD><EFBFBD><EFBFBD>蝞埈<EFBFBD> | P2 | 0.5憭?| <20><><EFBFBD>𧒄 | 韐券<E99F90><E588B8>𣂼<EFBFBD> |
| TD-COMMON-004 Worker瘜典<EFBFBD><EFBFBD>𨭌 | P3 | 0.3憭?| <20><EFBFBD>?| 隞<><E99A9E><EFBFBD><E89D9E>?|
| TD-COMMON-005 <EFBFBD>唳旿<EFBFBD><EFBFBD>𧋦蝞∠<EFBFBD> | **P1** | 3憭?| **銝𧢲活餈凋誨** | **<EFBFBD><EFBFBD><EFBFBD><EFBFBD>** |
| TD-COMMON-006 撟賜<EFBFBD><EFBFBD>?銵峕<E98AB5>瘚?| P2 | 0.5憭?| <20><><EFBFBD>𧒄 | 韐券<E99F90><E588B8>𣂼<EFBFBD> |
| TD-COMMON-007 餈𥕦漲<EFBFBD><EFBFBD>隞?| P2 | 0.3憭?| <20><><EFBFBD>𧒄 | UI蝏煺<EFBFBD> |
| TD-COMMON-001 前端轮询Hook | P2 | 0.5天 | 下次迭代 | 代码统一 |
| TD-COMMON-002 Clean Data服务 | **P1** | 1| **下次迭代** | **性能提升99%** |
| TD-COMMON-003 智能清洗算法 | P2 | 0.5天 | 需要时 | 质量提升 |
| TD-COMMON-004 Worker注册辅助 | P3 | 0.3天 | 可选 | 代码简化 |
| TD-COMMON-005 数据版本管理 | **P1** | 3| **下次迭代** | **链式操作** |
| TD-COMMON-006 幽灵列/行检测 | P2 | 0.5天 | 需要时 | 质量提升 |
| TD-COMMON-007 进度条组件 | P2 | 0.3天 | 需要时 | UI统一 |
**<EFBFBD><EFBFBD>隡睃<EFBFBD>蝥?*嚗?
1. 潃鐥<EFBFBD>潃鐥<EFBFBD>潃?TD-COMMON-002嚗㇃lean Data<EFBFBD>滚𦛚嚗?
2. 潃鐥<EFBFBD>潃鐥<EFBFBD>潃?TD-COMMON-005<EFBFBD><EFBFBD><EFBFBD><EFBFBD>祉恣<EFBFBD><EFBFBD><EFBFBD>
3. 潃鐥<EFBFBD>潃?TD-COMMON-001<EFBFBD><EFBFBD>蝡航蔭霂ook嚗?
**推荐优先级**
1. ⭐⭐⭐⭐⭐ TD-COMMON-002Clean Data服务)
2. ⭐⭐⭐⭐⭐ TD-COMMON-005(数据版本管理)
3. ⭐⭐⭐ TD-COMMON-001前端轮询Hook
---
## 📊 当前架构评价
### <EFBFBD>?撌脩<E6928C><EFBFBD><EFBFBD><E69CAB><EFBFBD><EFBFBD>?
### ✅ 已经很好的部分
1. <EFBFBD>?**<2A>𡁶鍂<F0A181B6><EFBFBD><EFBFBD>抅蝖<E68A85>霈暹鴌摰峕㟲**
- jobQueue嚗īg-boss<EFBFBD><EFBFBD>嚗?
1. **通用能力层基础设施完整**
- jobQueuepg-boss队列)
- CheckpointService断点续传
- 任务拆分工具
- storage、logger、cache
2. <EFBFBD>?**銝𡁜𦛚璅<E79285><EFBFBD>雿輻鍂<E8BCBB>𡁶鍂<F0A181B6><EFBFBD>**
2. **业务模块正确使用通用能力**
- Tool C、ASL、Tool B 统一使用 jobQueue
- Platform-Only模式
- 不在业务表中存储任务管理信息
3. <EFBFBD>?**<2A><><EFBFBD><EFBFBD>苊嚗諹<E59A97><EFBFBD><E99F90>蝖?*
3. **分层清晰,职责明确**
- 通用能力层:基础设施
- 銝𡁜𦛚璅<EFBFBD>嚗𡁜<EFBFBD>雿枏<EFBFBD><EFBFBD>?
- 业务模块:具体实现
### <EFBFBD><EFBFBD> <20>臭誑<E887AD><EFBFBD><E5AFA1><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>𧼮<EFBFBD><F0A7BCAE><EFBFBD>嚗?
### ⏭️ 可以改进的部分(非必需)
1. ⏭️ 前端轮询模式可以更统一
2. ⏭️ clean data缓存可以更通用
3. <EFBFBD><EFBFBD> <20>唳旿<E594B3><E697BF>𧋦蝞∠<E89D9E><EFBFBD>遣蝡?
3. ⏭️ 数据版本管理待建立
**结论**:当前架构已经符合规范,未来优化是锦上添花!
---
## <EFBFBD><EFBFBD> <20><><EFBFBD><EFBFBD><EFBFBD>獢?
## 📚 参考文档
- [Postgres-Only异步任务处理指南](./Postgres-Only异步任务处理指南.md) - 完整实践
- [鈭穃<EFBFBD><EFBFBD><EFBFBD><EFBFBD>𤏸<EFBFBD><EFBFBD><EFBFBD>(../04-撘<><E69298>𤏸<EFBFBD><F0A48FB8>?08-鈭穃<E988AD><E7A983><EFBFBD><E7AC94>𤏸<EFBFBD><F0A48FB8>?md) - <EFBFBD><EFBFBD>𤏸<EFBFBD><EFBFBD>?
- [DC Tool C<EFBFBD><EFBFBD><EFBFBD>(../03-銝𡁜𦛚璅<E79285>/DC-<2D>唳旿皜<E697BF><E79A9C><EFBFBD><EFBFBD>/00-撌亙<E6928C>C敶枏<E695B6><E69E8F><EFBFBD><E59786><EFBFBD><EFBFBD><E69298><EFBFBD><E78390>?md) - Day 10摰噼殿
- [云原生开发规范](../04-开发规范/08-云原生开发规范.md) - 开发规范
- [DC Tool C状态](../03-业务模块/DC-数据清洗整理/00-工具C当前状态与开发指南.md) - Day 10实践
---
**蝏湔擪<EFBFBD>?*: 撟喳蝱<E596B3><EFBFBD><E59786><EFBFBD>
**<EFBFBD><EFBFBD><EFBFBD>擧凒<EFBFBD>?*: 2025-12-22
**<EFBFBD><EFBFBD><EFBFBD><EFBFBD>?*: <20>?<3F><EFBFBD><E598A5><EFBFBD>𧋦
**维护者**: 平台架构团队
**最后更新**: 2025-12-22
**文档状态**: ✅ 初始版本