feat(admin): Add user management and upgrade to module permission system
Features - User Management (Phase 4.1): - Database: Add user_modules table for fine-grained module permissions - Database: Add 4 user permissions (view/create/edit/delete) to role_permissions - Backend: UserService (780 lines) - CRUD with tenant isolation - Backend: UserController + UserRoutes (648 lines) - 13 API endpoints - Backend: Batch import users from Excel - Frontend: UserListPage (412 lines) - list/filter/search/pagination - Frontend: UserFormPage (341 lines) - create/edit with module config - Frontend: UserDetailPage (393 lines) - details/tenant/module management - Frontend: 3 modal components (592 lines) - import/assign/configure - API: GET/POST/PUT/DELETE /api/admin/users/* endpoints Architecture Upgrade - Module Permission System: - Backend: Add getUserModules() method in auth.service - Backend: Login API returns modules array in user object - Frontend: AuthContext adds hasModule() method - Frontend: Navigation filters modules based on user.modules - Frontend: RouteGuard checks requiredModule instead of requiredVersion - Frontend: Remove deprecated version-based permission system - UX: Only show accessible modules in navigation (clean UI) - UX: Smart redirect after login (avoid 403 for regular users) Fixes: - Fix UTF-8 encoding corruption in ~100 docs files - Fix pageSize type conversion in userService (String to Number) - Fix authUser undefined error in TopNavigation - Fix login redirect logic with role-based access check - Update Git commit guidelines v1.2 with UTF-8 safety rules Database Changes: - CREATE TABLE user_modules (user_id, tenant_id, module_code, is_enabled) - ADD UNIQUE CONSTRAINT (user_id, tenant_id, module_code) - INSERT 4 permissions + role assignments - UPDATE PUBLIC tenant with 8 module subscriptions Technical: - Backend: 5 new files (~2400 lines) - Frontend: 10 new files (~2500 lines) - Docs: 1 development record + 2 status updates + 1 guideline update - Total: ~4900 lines of code Status: User management 100% complete, module permission system operational
This commit is contained in:
@@ -1,53 +1,53 @@
|
||||
# Postgres-Only 异步任务处理指南
|
||||
|
||||
> **譁<EFBFBD>。」迚域悽<EFBFBD>?* v1.0
|
||||
> **蛻帛サコ譌・譛滂シ?* 2025-12-22
|
||||
> **文档版本:** v1.0
|
||||
> **创建日期:** 2025-12-22
|
||||
> **维护者:** 平台架构团队
|
||||
> **騾ら畑蝨コ譎ッ<EFBFBD>?* 髟ソ譌カ髣エ莉サ蜉。<E89C89><EFBDA1>>30遘抵シ峨∝、ァ譁<EFBDA7>サカ螟<EFBDB6>炊縲∝錘蜿ーWorker
|
||||
> **适用场景:** 长时间任务(>30秒)、大文件处理、后台Worker
|
||||
> **参考实现:** DC Tool C Excel解析、ASL文献筛选、DC Tool B数据提取
|
||||
|
||||
---
|
||||
|
||||
## 📋 概述
|
||||
|
||||
譛ャ譁<EFBFBD>。」蝓コ莠?**DC Tool C Excel隗」譫仙粥閭ス** 逧<>ョ梧紛螳櫁キオ<EFBDB7>梧サ扈<EFBDBB> Postgres-Only 譫カ譫<EFBDB6>ク句シよュ・莉サ蜉。螟<EFBDA1>炊逧<E7828A><E980A7><EFBFBD>㊥讓。蠑上?
|
||||
本文档基于 **DC Tool C Excel解析功能** 的完整实践,总结 Postgres-Only 架构下异步任务处理的标准模式。
|
||||
|
||||
### 譬ク蠢<EFBFBD>サキ蛟?
|
||||
### 核心价值
|
||||
|
||||
1. 笨?**驕ソ蜈工TTP雜<50>慮**<2A>壻ク贋シ<E8B48B>謗・蜿?遘定ソ泌屓<E6B38C>瑚ァ」譫仙惠蜷主床螳梧<E89EB3><E6A2A7><EFBFBD>30-60遘抵シ<EFBFBD>
|
||||
2. 笨?**逕ィ謌キ菴馴ェ御シ倡ァ**<2A>壼ョ樊慮霑帛コヲ蜿埼ヲ茨シ御ク埼怙隕∝そ遲?
|
||||
3. 笨?**隨ヲ蜷井コ大次逕溯ァ<E6BAAF><EFBDA7>?*<2A>啀latform-Only讓。蠑擾シ継g-boss髦溷<EFBFBD>
|
||||
4. 笨?**諤ァ閭ス莨伜喧**<2A>喞lean data郛灘ュ假シ碁∩蜈埼㍾螟崎ョ。邂暦シ<E69AA6>-99%閠玲慮<E78EB2>?
|
||||
1. ✅ **避免HTTP超时**:上传接口3秒返回,解析在后台完成(30-60秒)
|
||||
2. ✅ **用户体验优秀**:实时进度反馈,不需要傻等
|
||||
3. ✅ **符合云原生规范**:Platform-Only模式,pg-boss队列
|
||||
4. ✅ **性能优化**:clean data缓存,避免重复计算(-99%耗时)
|
||||
|
||||
---
|
||||
|
||||
## <EFBFBD>女<EFBFBD>?譫カ譫<EFBDB6>ョセ隶。
|
||||
## 🏗️ 架构设计
|
||||
|
||||
### 三层架构
|
||||
|
||||
```
|
||||
笏娯楳笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏?
|
||||
笏? 蜑咲ォッ螻ゑシ<E38291>eact + React Query<EFBFBD>? 笏?
|
||||
笏? - 荳贋シ<EFBFBD>譁<EFBFBD>サカ<EFBFBD>育ォ句叉霑泌<EFBFBD>?sessionId + jobId<EFBFBD>? 笏?
|
||||
笏? - 霓ョ隸「迥カ諤<EFBFBD>シ<EFBFBD>seQuery + refetchInterval<EFBFBD>瑚<EFBFBD>蜉ィ荳イ陦鯉シ<EFBFBD> 笏?
|
||||
笏? - 逶大成 status='ready'<EFBFBD>悟刈霓ス謨ー謐? 笏?
|
||||
笏披楳笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏?
|
||||
竊?HTTP
|
||||
笏娯楳笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏?
|
||||
笏? 蜷守ォッ螻ゑシ<E38291>astify + Prisma<EFBFBD>? 笏?
|
||||
笏? - 蠢ォ騾滉ク贋シ<EFBFBD>蛻ー OSS<53>?-3遘抵シ<E68AB5> 笏?
|
||||
笏? - 蛻帛サコ Session<EFBFBD>育憾諤<EFBFBD>シ嗔rocessing<EFBFBD>? 笏?
|
||||
笏? - 謗ィ騾∽ササ蜉。蛻ー pg-boss<73>育ォ句叉霑泌屓<E6B38C><E5B193> 笏?
|
||||
笏? - 謠蝉セ帷憾諤∵衍隸?API 笏?
|
||||
笏披楳笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏?
|
||||
竊?pg-boss
|
||||
笏娯楳笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏?
|
||||
笏? Worker螻ゑシ<EFBFBD>g-boss + Platform螻ゑシ<EFBFBD> 笏?
|
||||
笏? - 莉朱弌蛻怜叙莉サ蜉。<EFBFBD>郁<EFBFBD>蜉ィ荳イ陦鯉シ<EFBFBD> 笏?
|
||||
笏? - 謇ァ陦瑚玲慮謫堺ス懶シ郁ァ」譫舌∵ク<EFBFBD>エ励∫サ溯ョ。<EFBFBD><EFBFBD> 笏?
|
||||
笏? - 菫晏ュ倡サ捺棡<EFBFBD><EFBFBD>lean data 蛻?OSS<EFBFBD>? 笏?
|
||||
笏? - 譖エ譁ー Session<EFBFBD>亥。ォ蜈<EFBFBD><EFBFBD>謨ー謐ョ<EFBFBD>? 笏?
|
||||
笏披楳笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏?
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ 前端层(React + React Query) │
|
||||
│ - 上传文件(立即返回 sessionId + jobId) │
|
||||
│ - 轮询状态(useQuery + refetchInterval,自动串行) │
|
||||
│ - 监听 status='ready',加载数据 │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
↓ HTTP
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ 后端层(Fastify + Prisma) │
|
||||
│ - 快速上传到 OSS(2-3秒) │
|
||||
│ - 创建 Session(状态:processing) │
|
||||
│ - 推送任务到 pg-boss(立即返回) │
|
||||
│ - 提供状态查询 API │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
↓ pg-boss
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ Worker层(pg-boss + Platform层) │
|
||||
│ - 从队列取任务(自动串行) │
|
||||
│ - 执行耗时操作(解析、清洗、统计) │
|
||||
│ - 保存结果(clean data 到 OSS) │
|
||||
│ - 更新 Session(填充元数据) │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
@@ -63,15 +63,15 @@ model YourBusinessTable {
|
||||
userId String
|
||||
fileKey String // OSS原始文件
|
||||
|
||||
// 笨?諤ァ閭ス莨伜喧<E4BC9C>壻ソ晏ュ伜、<E4BC9C>炊扈捺<E68988>?
|
||||
// ✅ 性能优化:保存处理结果
|
||||
cleanDataKey String? // 清洗/处理后的数据(避免重复计算)
|
||||
|
||||
// 謨ー謐ョ蜈<EFBFBD>ソ。諱ッ<EFBFBD>亥シよュ・蝪ォ蜈<EFBFBD>シ?
|
||||
// 数据元信息(异步填充)
|
||||
totalRows Int?
|
||||
totalCols Int?
|
||||
columns Json?
|
||||
|
||||
// 譌カ髣エ謌?
|
||||
// 时间戳
|
||||
createdAt DateTime
|
||||
updatedAt DateTime
|
||||
expiresAt DateTime
|
||||
@@ -80,13 +80,13 @@ model YourBusinessTable {
|
||||
}
|
||||
```
|
||||
|
||||
**蜈ウ髞ョ轤?*<2A>?
|
||||
- 笶?荳崎ヲ∵キサ蜉<EFBDBB> `status`縲~progress`縲~errorMessage` 遲我ササ蜉。邂。逅<EFBFBD>ュ玲ョ?
|
||||
- 笨?霑吩コ帛ュ玲ョオ逕?pg-boss 逧?`job` 陦ィ邂。逅?
|
||||
**关键点**:
|
||||
- ❌ 不要添加 `status`、`progress`、`errorMessage` 等任务管理字段
|
||||
- ✅ 这些字段由 pg-boss 的 `job` 表管理
|
||||
|
||||
---
|
||||
|
||||
### 豁・鬪、2<EFBFBD>售ervice螻?- 蠢ォ騾滉ク贋シ?謗ィ騾∽ササ蜉?
|
||||
### 步骤2:Service层 - 快速上传+推送任务
|
||||
|
||||
```typescript
|
||||
// backend/src/modules/your-module/services/YourService.ts
|
||||
@@ -97,13 +97,13 @@ import { prisma } from '@/config/database';
|
||||
|
||||
export class YourService {
|
||||
/**
|
||||
* 蛻帛サコ莉サ蜉。蟷カ謗ィ騾∝芦髦溷<EFBFBD><EFBFBD><EFBFBD>ostgres-Only譫カ譫<EFBFBD>シ?
|
||||
* 创建任务并推送到队列(Postgres-Only架构)
|
||||
*
|
||||
* 笨?Platform-Only 讓。蠑擾シ?
|
||||
* - 遶句叉荳贋シ<EFBFBD>譁<EFBFBD>サカ蛻?OSS
|
||||
* - 蛻帛サコ荳壼苅隶ー蠖包シ亥<EFBFBD>謨ー謐ョ荳コnull<EFBFBD>?
|
||||
* ✅ Platform-Only 模式:
|
||||
* - 立即上传文件到 OSS
|
||||
* - 创建业务记录(元数据为null)
|
||||
* - 推送任务到队列
|
||||
* - 遶句叉霑泌屓<EFBFBD>井ク埼仆蝪櫁ッキ豎ゑシ?
|
||||
* - 立即返回(不阻塞请求)
|
||||
*/
|
||||
async createTask(userId: string, fileName: string, fileBuffer: Buffer) {
|
||||
// 1. 验证文件
|
||||
@@ -111,34 +111,34 @@ export class YourService {
|
||||
throw new Error('文件太大');
|
||||
}
|
||||
|
||||
// 2. 笞?遶句叉荳贋シ<E8B48B>蛻?OSS<53>?-3遘抵シ<E68AB5>
|
||||
// 2. ⚡ 立即上传到 OSS(2-3秒)
|
||||
const fileKey = `path/${userId}/${Date.now()}-${fileName}`;
|
||||
await storage.upload(fileKey, fileBuffer);
|
||||
|
||||
// 3. 笞?蛻帛サコ荳壼苅隶ー蠖包シ亥<EFBDBC>謨ー謐ョ荳コnull<6C>檎ュ姥orker蝪ォ蜈<EFBDAB>シ?
|
||||
// 3. ⚡ 创建业务记录(元数据为null,等Worker填充)
|
||||
const record = await prisma.yourTable.create({
|
||||
data: {
|
||||
userId,
|
||||
fileName,
|
||||
fileKey,
|
||||
// 笞<EFBFBD><EFBFBD><EFBFBD> 螟<>炊扈捺棡蟄玲ョオ荳?null
|
||||
// ⚠️ 处理结果字段为 null
|
||||
totalRows: null,
|
||||
columns: null,
|
||||
expiresAt: new Date(Date.now() + 10 * 60 * 1000),
|
||||
},
|
||||
});
|
||||
|
||||
// 4. 笞?謗ィ騾∽ササ蜉。蛻ー pg-boss<EFBFBD><EFBFBD>latform-Only<EFBFBD>?
|
||||
// 4. ⚡ 推送任务到 pg-boss(Platform-Only)
|
||||
const job = await jobQueue.push('your_module_process', {
|
||||
recordId: record.id,
|
||||
fileKey,
|
||||
userId,
|
||||
});
|
||||
|
||||
// 5. 笞?遶句叉霑泌屓<E6B38C>域サ閠玲慮<3遘抵シ<E68AB5>
|
||||
// 5. ⚡ 立即返回(总耗时<3秒)
|
||||
return {
|
||||
...record,
|
||||
jobId: job.id, // 笨?霑泌屓 jobId 萓帛燕遶ッ霓ョ隸?
|
||||
jobId: job.id, // ✅ 返回 jobId 供前端轮询
|
||||
};
|
||||
}
|
||||
}
|
||||
@@ -146,7 +146,7 @@ export class YourService {
|
||||
|
||||
---
|
||||
|
||||
### 豁・鬪、3<EFBFBD>啗orker螻?- 蜷主床螟<E5BA8A>炊
|
||||
### 步骤3:Worker层 - 后台处理
|
||||
|
||||
```typescript
|
||||
// backend/src/modules/your-module/workers/yourWorker.ts
|
||||
@@ -163,7 +163,7 @@ interface YourJob {
|
||||
}
|
||||
|
||||
/**
|
||||
* 豕ィ蜀<EFBFBD> Worker 蛻ー髦溷<EFBFBD>?
|
||||
* 注册 Worker 到队列
|
||||
*/
|
||||
export function registerYourWorker() {
|
||||
logger.info('[YourWorker] Registering worker');
|
||||
@@ -175,14 +175,14 @@ export function registerYourWorker() {
|
||||
logger.info('[YourWorker] Processing job', { jobId: job.id, recordId });
|
||||
|
||||
try {
|
||||
// 1. 莉?OSS 荳玖スス譁<EFBFBD>サカ
|
||||
// 1. 从 OSS 下载文件
|
||||
const buffer = await storage.download(fileKey);
|
||||
|
||||
// 2. 执行耗时操作(解析、处理、计算)
|
||||
const result = await yourLongTimeProcess(buffer);
|
||||
const { processedData, totalRows, columns } = result;
|
||||
|
||||
// 3. 笨?菫晏ュ伜、<E4BC9C>炊扈捺棡蛻?OSS<53>磯∩蜈埼㍾螟崎ョ。邂暦シ<E69AA6>
|
||||
// 3. ✅ 保存处理结果到 OSS(避免重复计算)
|
||||
const cleanDataKey = `${fileKey}_clean.json`;
|
||||
const cleanDataBuffer = Buffer.from(JSON.stringify(processedData), 'utf-8');
|
||||
await storage.upload(cleanDataKey, cleanDataBuffer);
|
||||
@@ -191,36 +191,36 @@ export function registerYourWorker() {
|
||||
size: `${(cleanDataBuffer.length / 1024).toFixed(2)} KB`
|
||||
});
|
||||
|
||||
// 4. 譖エ譁ー荳壼苅隶ー蠖包シ亥。ォ蜈<EFBFBD><EFBFBD>謨ー謐ョ<EFBFBD>?
|
||||
// 4. 更新业务记录(填充元数据)
|
||||
await prisma.yourTable.update({
|
||||
where: { id: recordId },
|
||||
data: {
|
||||
cleanDataKey, // 笨?菫晏ュ<E6998F> clean data 菴咲スョ
|
||||
cleanDataKey, // ✅ 保存 clean data 位置
|
||||
totalRows,
|
||||
columns,
|
||||
updatedAt: new Date(),
|
||||
},
|
||||
});
|
||||
|
||||
logger.info('[YourWorker] 笨?Job completed', { jobId: job.id });
|
||||
logger.info('[YourWorker] ✅ Job completed', { jobId: job.id });
|
||||
|
||||
return { success: true, recordId, totalRows };
|
||||
} catch (error: any) {
|
||||
logger.error('[YourWorker] 笶?Job failed', {
|
||||
logger.error('[YourWorker] ❌ Job failed', {
|
||||
jobId: job.id,
|
||||
error: error.message
|
||||
});
|
||||
throw error; // 隶?pg-boss 螟<EFBFBD>炊驥崎ッ<EFBFBD>
|
||||
throw error; // 让 pg-boss 处理重试
|
||||
}
|
||||
});
|
||||
|
||||
logger.info('[YourWorker] 笨?Worker registered: your_module_process');
|
||||
logger.info('[YourWorker] ✅ Worker registered: your_module_process');
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 豁・鬪、4<EFBFBD>咾ontroller螻?- 迥カ諤∵衍隸「API
|
||||
### 步骤4:Controller层 - 状态查询API
|
||||
|
||||
```typescript
|
||||
// backend/src/modules/your-module/controllers/YourController.ts
|
||||
@@ -229,10 +229,10 @@ import { jobQueue } from '@/common/jobs';
|
||||
|
||||
export class YourController {
|
||||
/**
|
||||
* 闔キ蜿紋ササ蜉。迥カ諤<EFBFBD>シ<EFBFBD>latform-Only讓。蠑擾シ?
|
||||
* 获取任务状态(Platform-Only模式)
|
||||
*
|
||||
* GET /api/v1/your-module/tasks/:id/status
|
||||
* Query: jobId (蜿ッ騾?
|
||||
* Query: jobId (可选)
|
||||
*/
|
||||
async getTaskStatus(request, reply) {
|
||||
const { id: recordId } = request.params;
|
||||
@@ -244,18 +244,18 @@ export class YourController {
|
||||
});
|
||||
|
||||
if (!record) {
|
||||
return reply.code(404).send({ success: false, error: '隶ー蠖穂ク榊ュ伜<EFBFBD>? });
|
||||
return reply.code(404).send({ success: false, error: '记录不存在' });
|
||||
}
|
||||
|
||||
// 2. 蛻、譁ュ迥カ諤?
|
||||
// - 螯よ棡 totalRows 荳堺クコ null<EFBFBD>瑚ッエ譏主、<EFBFBD>炊螳梧<EFBFBD>?
|
||||
// - 蜷ヲ蛻呎衍隸「 job 迥カ諤?
|
||||
// 2. 判断状态
|
||||
// - 如果 totalRows 不为 null,说明处理完成
|
||||
// - 否则查询 job 状态
|
||||
if (record.totalRows !== null) {
|
||||
return reply.send({
|
||||
success: true,
|
||||
data: {
|
||||
recordId,
|
||||
status: 'ready', // 笨?螟<>炊螳梧<E89EB3>
|
||||
status: 'ready', // ✅ 处理完成
|
||||
progress: 100,
|
||||
record,
|
||||
},
|
||||
@@ -274,7 +274,7 @@ export class YourController {
|
||||
});
|
||||
}
|
||||
|
||||
// 4. 莉?pg-boss 譟・隸「 job 迥カ諤?
|
||||
// 4. 从 pg-boss 查询 job 状态
|
||||
const job = await jobQueue.getJob(jobId);
|
||||
|
||||
const status = job?.status === 'completed' ? 'ready' :
|
||||
@@ -299,7 +299,7 @@ export class YourController {
|
||||
|
||||
---
|
||||
|
||||
### 豁・鬪、5<EFBFBD>壼燕遶?- React Query 霓ョ隸「
|
||||
### 步骤5:前端 - React Query 轮询
|
||||
|
||||
```typescript
|
||||
// frontend-v2/src/modules/your-module/hooks/useTaskStatus.ts
|
||||
@@ -308,12 +308,12 @@ import { useQuery } from '@tanstack/react-query';
|
||||
import * as api from '../api';
|
||||
|
||||
/**
|
||||
* 莉サ蜉。迥カ諤∬スョ隸?Hook
|
||||
* 任务状态轮询 Hook
|
||||
*
|
||||
* 迚ケ轤ケ<EFBFBD>?
|
||||
* 特点:
|
||||
* - 自动串行轮询(React Query 内置防并发)
|
||||
* - 閾ェ蜉ィ貂<EFBFBD>炊<EFBFBD>育サ<EFBFBD>サカ蜊ク霓ス譌カ蛛懈ュ「<EFBFBD>?
|
||||
* - 譚。莉カ蛛懈ュ「<EFBFBD>亥ョ梧<EFBFBD>?螟ア雍・譌カ閾ェ蜉ィ蛛懈ュ「<EFBDAD><EFBDA2>
|
||||
* - 自动清理(组件卸载时停止)
|
||||
* - 条件停止(完成/失败时自动停止)
|
||||
*/
|
||||
export function useTaskStatus({
|
||||
recordId,
|
||||
@@ -327,15 +327,15 @@ export function useTaskStatus({
|
||||
refetchInterval: (query) => {
|
||||
const status = query.state.data?.data?.status;
|
||||
|
||||
// 笨?螳梧<E89EB3>謌門、ア雍・譌カ蛛懈ュ「霓ョ隸「
|
||||
// ✅ 完成或失败时停止轮询
|
||||
if (status === 'ready' || status === 'error') {
|
||||
return false;
|
||||
}
|
||||
|
||||
// 笨?螟<>炊荳ュ譌カ豈?遘定スョ隸「<E99AB8>郁<EFBFBD>蜉ィ荳イ陦鯉シ?
|
||||
// ✅ 处理中时每2秒轮询(自动串行)
|
||||
return 2000;
|
||||
},
|
||||
staleTime: 0, // 蟋狗サ郁ァ<EFBFBD>クコ霑<EFBFBD>慮<EFBFBD>檎。ョ菫晁スョ隸?
|
||||
staleTime: 0, // 始终视为过时,确保轮询
|
||||
retry: 1,
|
||||
});
|
||||
|
||||
@@ -356,7 +356,7 @@ export function useTaskStatus({
|
||||
|
||||
---
|
||||
|
||||
### 豁・鬪、6<EFBFBD>壼燕遶ッ扈<EFBFBD>サ?- 菴ソ逕ィHook
|
||||
### 步骤6:前端组件 - 使用Hook
|
||||
|
||||
```typescript
|
||||
// frontend-v2/src/modules/your-module/pages/YourPage.tsx
|
||||
@@ -369,17 +369,17 @@ const YourPage = () => {
|
||||
jobId: string;
|
||||
} | null>(null);
|
||||
|
||||
// 笨?菴ソ逕ィ React Query Hook 閾ェ蜉ィ霓ョ隸「
|
||||
// ✅ 使用 React Query Hook 自动轮询
|
||||
const { status, progress, isReady } = useTaskStatus({
|
||||
recordId: pollingInfo?.recordId || null,
|
||||
jobId: pollingInfo?.jobId || null,
|
||||
enabled: !!pollingInfo,
|
||||
});
|
||||
|
||||
// 笨?逶大成迥カ諤∝序蛹?
|
||||
// ✅ 监听状态变化
|
||||
useEffect(() => {
|
||||
if (isReady && pollingInfo) {
|
||||
console.log('笨?螟<EFBFBD>炊螳梧<EFBFBD><EFBFBD>悟刈霓ス謨ー謐?);
|
||||
console.log('✅ 处理完成,加载数据');
|
||||
|
||||
// 停止轮询
|
||||
setPollingInfo(null);
|
||||
@@ -394,13 +394,13 @@ const YourPage = () => {
|
||||
const result = await api.uploadFile(file);
|
||||
const { recordId, jobId } = result.data;
|
||||
|
||||
// 笨?蜷ッ蜉ィ霓ョ隸「<E99AB8>郁ョセ鄂ョ迥カ諤<EFBDB6>シ軍eact Query閾ェ蜉ィ蠑蟋具シ<E585B7>
|
||||
// ✅ 启动轮询(设置状态,React Query自动开始)
|
||||
setPollingInfo({ recordId, jobId });
|
||||
};
|
||||
|
||||
return (
|
||||
<div>
|
||||
{/* 霑帛コヲ譚?*/}
|
||||
{/* 进度条 */}
|
||||
{pollingInfo && (
|
||||
<div className="progress-bar">
|
||||
<div style={{ width: `${progress}%` }} />
|
||||
@@ -421,16 +421,16 @@ const YourPage = () => {
|
||||
|
||||
### 1. 队列名称规范
|
||||
|
||||
**髞呵ッッ**<EFBFBD>?
|
||||
**错误**:
|
||||
```typescript
|
||||
笶?'asl:screening:batch' // 蛹<EFBFBD>性蜀貞捷<EFBFBD>継g-boss荳肴髪謖?
|
||||
笶?'dc.toolc.parse' // 蛹<EFBFBD>性轤ケ蜿キ<EFBFBD>御ク肴耳闕<EFBFBD>
|
||||
❌ 'asl:screening:batch' // 包含冒号,pg-boss不支持
|
||||
❌ 'dc.toolc.parse' // 包含点号,不推荐
|
||||
```
|
||||
|
||||
**豁」遑ョ**<EFBFBD>?
|
||||
**正确**:
|
||||
```typescript
|
||||
笨?'asl_screening_batch' // 荳句<EFBFBD>郤?
|
||||
笨?'dc_toolc_parse_excel' // 荳句<EFBFBD>郤?
|
||||
✅ 'asl_screening_batch' // 下划线
|
||||
✅ 'dc_toolc_parse_excel' // 下划线
|
||||
```
|
||||
|
||||
---
|
||||
@@ -440,22 +440,22 @@ const YourPage = () => {
|
||||
```typescript
|
||||
// backend/src/index.ts
|
||||
|
||||
await jobQueue.start(); // 竊?蠢<>。サ蜈亥星蜉ィ髦溷<E9ABA6>?
|
||||
await jobQueue.start(); // ← 必须先启动队列
|
||||
|
||||
registerYourWorker(); // 竊?蜀肴ウィ蜀?Worker
|
||||
registerYourWorker(); // ← 再注册 Worker
|
||||
registerOtherWorker();
|
||||
|
||||
// 笨?遲牙セ<E78999>3遘抵シ檎。ョ菫晏シよュ・豕ィ蜀悟ョ梧<EFBDAE>
|
||||
// ✅ 等待3秒,确保异步注册完成
|
||||
await new Promise(resolve => setTimeout(resolve, 3000));
|
||||
|
||||
logger.info('笨?All workers registered');
|
||||
logger.info('✅ All workers registered');
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 3. clean data 缓存机制
|
||||
|
||||
**逶ョ逧<EFBFBD>**<EFBFBD>夐∩蜈埼㍾螟崎ョ。邂暦シ域ァ閭ス謠仙合99%<EFBFBD>?
|
||||
**目的**:避免重复计算(性能提升99%)
|
||||
|
||||
```typescript
|
||||
// Worker 保存 clean data
|
||||
@@ -465,17 +465,17 @@ await storage.upload(cleanDataKey, JSON.stringify(processedData));
|
||||
await prisma.update({
|
||||
where: { id },
|
||||
data: {
|
||||
cleanDataKey, // 竊?隶ー蠖穂ス咲スョ
|
||||
cleanDataKey, // ← 记录位置
|
||||
totalRows,
|
||||
columns,
|
||||
}
|
||||
});
|
||||
|
||||
// Service 隸サ蜿匁焚謐ョ<EFBFBD>井シ伜<EFBFBD>?clean data<EFBFBD>?
|
||||
// Service 读取数据(优先 clean data)
|
||||
async getFullData(recordId) {
|
||||
const record = await prisma.findUnique({ where: { id: recordId } });
|
||||
|
||||
// 笨?莨伜<E88EA8>隸サ蜿<EFBDBB> clean data<EFBFBD>?1遘抵シ<E68AB5>
|
||||
// ✅ 优先读取 clean data(<1秒)
|
||||
if (record.cleanDataKey) {
|
||||
const buffer = await storage.download(record.cleanDataKey);
|
||||
return JSON.parse(buffer.toString('utf-8'));
|
||||
@@ -486,19 +486,19 @@ async getFullData(recordId) {
|
||||
return parseFile(buffer);
|
||||
}
|
||||
|
||||
// 笞<EFBFBD><EFBFBD><EFBFBD> 驥崎ヲ<E5B48E>シ壽桃菴懷錘隕∝酔豁・譖エ譁?clean data
|
||||
// ⚠️ 重要:操作后要同步更新 clean data
|
||||
async saveProcessedData(recordId, newData) {
|
||||
const record = await getRecord(recordId);
|
||||
|
||||
// 隕<EFBFBD>尠蜴滓枚莉?
|
||||
// 覆盖原文件
|
||||
await storage.upload(record.fileKey, toExcel(newData));
|
||||
|
||||
// 笨?蜷梧慮譖エ譁ー clean data
|
||||
// ✅ 同时更新 clean data
|
||||
if (record.cleanDataKey) {
|
||||
await storage.upload(record.cleanDataKey, JSON.stringify(newData));
|
||||
}
|
||||
|
||||
// 譖エ譁ー蜈<EFBFBD>焚謐?
|
||||
// 更新元数据
|
||||
await prisma.update({ where: { id: recordId }, data: { ... } });
|
||||
}
|
||||
```
|
||||
@@ -507,15 +507,15 @@ async saveProcessedData(recordId, newData) {
|
||||
|
||||
### 4. React Query 轮询(推荐)
|
||||
|
||||
**莨倡せ**<EFBFBD>?
|
||||
- 笨?閾ェ蜉ィ荳イ陦鯉シ磯亟蟷カ蜿鷹」取垓<E58F96>?
|
||||
- 笨?閾ェ蜉ィ蜴サ驥搾シ亥酔荳queryKey蜿ェ譛我ク荳ェ隸キ豎ゑシ<E38291>
|
||||
- 笨?閾ェ蜉ィ貂<EFBDA8>炊<EFBFBD>育サ<E882B2>サカ蜊ク霓ス譌カ蛛懈ュ「<EFBDAD>?
|
||||
- 笨?譚。莉カ蛛懈ュ「<EFBDAD>亥勘諤∵而蛻カ<E89BBB><EFBDB6>
|
||||
**优点**:
|
||||
- ✅ 自动串行(防并发风暴)
|
||||
- ✅ 自动去重(同一queryKey只有一个请求)
|
||||
- ✅ 自动清理(组件卸载时停止)
|
||||
- ✅ 条件停止(动态控制)
|
||||
|
||||
**荳崎ヲ∽スソ逕ィ setInterval**<EFBFBD>?
|
||||
**不要使用 setInterval**:
|
||||
```typescript
|
||||
笶?const pollInterval = setInterval(() => {
|
||||
❌ const pollInterval = setInterval(() => {
|
||||
api.getStatus(); // 可能并发
|
||||
}, 2000);
|
||||
```
|
||||
@@ -524,16 +524,16 @@ async saveProcessedData(recordId, newData) {
|
||||
|
||||
## 📊 性能对比
|
||||
|
||||
### DC Tool C 螳樣刔謨ー謐ョ<EFBFBD>?339陦古?51蛻玲枚莉カ<E88E89><EFBDB6>
|
||||
### DC Tool C 实际数据(3339行×151列文件)
|
||||
|
||||
| 指标 | 同步处理 | 异步处理 | 改善 |
|
||||
|------|---------|---------|------|
|
||||
| **荳贋シ<EFBFBD>閠玲慮** | 47遘抵シ磯仆蝪橸シ?| 3遘抵シ育ォ句叉霑泌屓<E6B38C>?| 笨?-94% |
|
||||
| **HTTP雜<EFBFBD>慮** | 笶?扈丞クク雜<EFBDB8>慮 | 笨?荳堺シ夊カ<E5A48A>慮 | 笨?100% |
|
||||
| **getPreviewData** | 43遘抵シ磯㍾螟崎ァ」譫撰シ?| 0.5遘抵シ育シ灘ュ假シ?| 笨?-99% |
|
||||
| **getFullData** | 43遘抵シ磯㍾螟崎ァ」譫撰シ?| 0.5遘抵シ育シ灘ュ假シ?| 笨?-99% |
|
||||
| **QuickAction謫堺ス<EFBFBD>** | 43遘?+ Python | 0.5遘?+ Python | 笨?-95% |
|
||||
| **蟷カ蜿題ッキ豎<EFBFBD>** | 15+荳?| 1荳ェ<EFBFBD>井クイ陦鯉シ?| 笨?-93% |
|
||||
| **上传耗时** | 47秒(阻塞) | 3秒(立即返回) | ✅ -94% |
|
||||
| **HTTP超时** | ❌ 经常超时 | ✅ 不会超时 | ✅ 100% |
|
||||
| **getPreviewData** | 43秒(重复解析) | 0.5秒(缓存) | ✅ -99% |
|
||||
| **getFullData** | 43秒(重复解析) | 0.5秒(缓存) | ✅ -99% |
|
||||
| **QuickAction操作** | 43秒 + Python | 0.5秒 + Python | ✅ -95% |
|
||||
| **并发请求** | 15+个 | 1个(串行) | ✅ -93% |
|
||||
|
||||
---
|
||||
|
||||
@@ -541,50 +541,49 @@ async saveProcessedData(recordId, newData) {
|
||||
|
||||
### Q1: Worker 注册了但不工作?
|
||||
|
||||
**譽譟?*<2A>?
|
||||
- 髦溷<EFBFBD>蜷咲ァー譏ッ蜷ヲ蛹<EFBFBD>性蜀貞捷<EFBFBD><EFBFBD>:`<60>会シ滓隼荳コ荳句<E88DB3>郤ソ<E983A4><EFBDBF>_`<EFBFBD>?
|
||||
- 邇ッ蠅<EFBFBD>序驥<EFBFBD> `QUEUE_TYPE=pgboss` 譏ッ蜷ヲ隶セ鄂ョ<EFBFBD>?
|
||||
- Worker 豕ィ蜀梧弍蜷ヲ蝨?`jobQueue.start()` 荵句錘<EFBFBD>?
|
||||
**检查**:
|
||||
- 队列名称是否包含冒号(`:`)?改为下划线(`_`)
|
||||
- 环境变量 `QUEUE_TYPE=pgboss` 是否设置?
|
||||
- Worker 注册是否在 `jobQueue.start()` 之后?
|
||||
|
||||
### Q2: 霓ョ隸「鬟取垓<EFBFBD>亥、壻クェ蟷カ蜿題ッキ豎ゑシ会シ?
|
||||
### Q2: 轮询风暴(多个并发请求)?
|
||||
|
||||
**隗」蜀ウ**<EFBFBD>壻スソ逕?React Query<EFBFBD>御ク崎ヲ∫畑 setInterval
|
||||
**解决**:使用 React Query,不要用 setInterval
|
||||
|
||||
### Q3: 导出数据不对(是原始数据)?
|
||||
|
||||
**原因**:`saveProcessedData` 没有更新 clean data
|
||||
**隗」蜀ウ**<2A>壼酔譌カ譖エ譁?fileKey 蜥?cleanDataKey
|
||||
**解决**:同时更新 fileKey 和 cleanDataKey
|
||||
|
||||
---
|
||||
|
||||
## <EFBFBD>答 蜿り<E3828A>ョ樒<EFBDAE>?
|
||||
## 📚 参考实现
|
||||
|
||||
| 模块 | Worker | 前端Hook | 文档 |
|
||||
|------|--------|---------|------|
|
||||
| **DC Tool C** | `parseExcelWorker.ts` | `useSessionStatus.ts` | 本指南基础 |
|
||||
| **ASL 譎コ閭ス譁<EFBFBD>鍵** | `screeningWorker.ts` | `useScreeningTask.ts` | [ASL讓。蝮礼憾諤‐(../03-荳壼苅讓。蝮<EFBDA1>/ASL-AI譎コ閭ス譁<EFBDBD>鍵/00-讓。蝮怜ス灘燕迥カ諤∽ク主シ蜿第欠蜊?md) |
|
||||
| **DC Tool B** | `extractionWorker.ts` | - | [DC讓。蝮礼憾諤‐(../03-荳壼苅讓。蝮<EFBDA1>/DC-謨ー謐ョ貂<EFBDAE>エ玲紛逅<E7B49B>/00-讓。蝮怜ス灘燕迥カ諤∽ク主シ蜿第欠蜊?md) |
|
||||
| **ASL 智能文献** | `screeningWorker.ts` | `useScreeningTask.ts` | [ASL模块状态](../03-业务模块/ASL-AI智能文献/00-模块当前状态与开发指南.md) |
|
||||
| **DC Tool B** | `extractionWorker.ts` | - | [DC模块状态](../03-业务模块/DC-数据清洗整理/00-模块当前状态与开发指南.md) |
|
||||
|
||||
---
|
||||
|
||||
## 笨?譽譟・貂<EFBDA5><E8B282>?
|
||||
## ✅ 检查清单
|
||||
|
||||
蝨ィ螳樊命蠑よュ・莉サ蜉。蜑搾シ瑚ッキ遑ョ隶、<EFBFBD>?
|
||||
在实施异步任务前,请确认:
|
||||
|
||||
- [ ] 荳壼苅陦ィ蜿ェ蟄倅ク壼苅菫。諱ッ<E8ABB1>井ク榊桁蜷?status 遲牙ュ玲ョオ<EFBDAE><EFBDB5>
|
||||
- [ ] 髦溷<E9ABA6>蜷咲ァー菴ソ逕ィ荳句<E88DB3>郤ソ<E983A4>井ク榊性蜀貞捷<E8B29E>?
|
||||
- [ ] 邇ッ蠅<EFBDAF>序驥<E5BA8F> `QUEUE_TYPE=pgboss` 蟾イ隶セ鄂?
|
||||
- [ ] Worker 蝨?`jobQueue.start()` 荵句錘豕ィ蜀<EFBDA8>
|
||||
- [ ] 业务表只存业务信息(不包含 status 等字段)
|
||||
- [ ] 队列名称使用下划线(不含冒号)
|
||||
- [ ] 环境变量 `QUEUE_TYPE=pgboss` 已设置
|
||||
- [ ] Worker 在 `jobQueue.start()` 之后注册
|
||||
- [ ] 前端使用 React Query 轮询
|
||||
- [ ] Service 优先读取 clean data
|
||||
- [ ] saveProcessedData 同步更新 clean data
|
||||
|
||||
---
|
||||
|
||||
**扈エ謚、閠?*: 蟷ウ蜿ー譫カ譫<EFBDB6>屬髦<E5B1AC>
|
||||
**譛蜷取峩譁?*: 2025-12-22
|
||||
**譁<EFBFBD>。」迥カ諤?*: 笨?蟾イ螳梧<E89EB3>?
|
||||
|
||||
**维护者**: 平台架构团队
|
||||
**最后更新**: 2025-12-22
|
||||
**文档状态**: ✅ 已完成
|
||||
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user