feat(admin): Add user management and upgrade to module permission system
Features - User Management (Phase 4.1): - Database: Add user_modules table for fine-grained module permissions - Database: Add 4 user permissions (view/create/edit/delete) to role_permissions - Backend: UserService (780 lines) - CRUD with tenant isolation - Backend: UserController + UserRoutes (648 lines) - 13 API endpoints - Backend: Batch import users from Excel - Frontend: UserListPage (412 lines) - list/filter/search/pagination - Frontend: UserFormPage (341 lines) - create/edit with module config - Frontend: UserDetailPage (393 lines) - details/tenant/module management - Frontend: 3 modal components (592 lines) - import/assign/configure - API: GET/POST/PUT/DELETE /api/admin/users/* endpoints Architecture Upgrade - Module Permission System: - Backend: Add getUserModules() method in auth.service - Backend: Login API returns modules array in user object - Frontend: AuthContext adds hasModule() method - Frontend: Navigation filters modules based on user.modules - Frontend: RouteGuard checks requiredModule instead of requiredVersion - Frontend: Remove deprecated version-based permission system - UX: Only show accessible modules in navigation (clean UI) - UX: Smart redirect after login (avoid 403 for regular users) Fixes: - Fix UTF-8 encoding corruption in ~100 docs files - Fix pageSize type conversion in userService (String to Number) - Fix authUser undefined error in TopNavigation - Fix login redirect logic with role-based access check - Update Git commit guidelines v1.2 with UTF-8 safety rules Database Changes: - CREATE TABLE user_modules (user_id, tenant_id, module_code, is_enabled) - ADD UNIQUE CONSTRAINT (user_id, tenant_id, module_code) - INSERT 4 permissions + role assignments - UPDATE PUBLIC tenant with 8 module subscriptions Technical: - Backend: 5 new files (~2400 lines) - Frontend: 10 new files (~2500 lines) - Docs: 1 development record + 2 status updates + 1 guideline update - Total: ~4900 lines of code Status: User management 100% complete, module permission system operational
This commit is contained in:
@@ -1,34 +1,37 @@
|
||||
# **<EFBFBD><EFBFBD><EFBFBD>航挽霈⊥<EFBFBD>獢<EFBFBD><EFBFBD>撌亙<EFBFBD> C \- 蝘𤑳<E89D98><F0A491B3>唳旿蝻𤥁<E89DBB><F0A4A581>?(The Research Editor)**
|
||||
# **技术设计文档：工具 C \- 科研数据编辑器 (The Research Editor)**
|
||||
|
||||
| 文档类型 | Technical Design Document (TDD) |
|
||||
| :---- | :---- |
|
||||
| **对应 PRD** | **PRD\_工具C\_科研数据编辑器\_V2.1.md** |
|
||||
| **<EFBFBD><EFBFBD>𧋦** | **V2.1** (<EFBFBD>啣<EFBFBD> Pivot 蝞埈<EFBFBD>銝?Web Worker <EFBFBD>嗆<EFBFBD>) |
|
||||
| **<EFBFBD>嗆<EFBFBD>?* | Final Draft |
|
||||
| **<EFBFBD>詨<EFBFBD><EFBFBD>格<EFBFBD>** | <EFBFBD><EFBFBD>遣銝<EFBFBD>銝芷<EFBFBD><EFBFBD>扯<EFBFBD><EFBFBD>?Web 蝡舀㺭<E88880>桃<EFBFBD>颲穃膥嚗峕𣈲<E5B395>?5 銝<><E98A9D>蝥扳㺭<E689B3>桃<EFBFBD>摰墧𧒄皜<F0A79284><E79A9C><EFBFBD><EFBFBD><EFBFBD><EFBFBD>誩<EFBFBD>撌伐<E6928C><E4BC90>恍鵭摰質蓮<E8B3AA>ｇ<EFBFBD>銝𡡞<E98A9D>餉<EFBFBD>瘝餌<E7989D>嚗峕<E59A97>靘𥕞<E99D98>𣈯妟撱嗉<E692B1><E59789>脲<EFBFBD>雿靝<E99BBF>撉䎚<E69289>?|
|
||||
| **版本** | **V2.1** (新增 Pivot 算法与 Web Worker 架构) |
|
||||
| **状态** | Final Draft |
|
||||
| **核心目标** | 构建一个高性能的 Web 端数据编辑器，支持 5 万行级数据的实时清洗、变量加工（含长宽转换）与逻辑治理，提供“零延迟”操作体验。 |
|
||||
|
||||
## **1\. 总体架构设计 (Architecture Overview)**
|
||||
|
||||
銝箔<EFBFBD>皛∟雲 **PRD V2.1** 銝凌<EFBFBD>𨅯朖<EFBFBD>嗅<EFBFBD>擐<EFBFBD><EFBFBD>腈<EFBFBD><EFBFBD><EFBFBD>𨀣伃<EFBFBD><EFBFBD><EFBFBD>滚<EFBFBD><EFBFBD>苷誑<EFBFBD>𠰴<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>𣈯鵭摰質蓮<EFBFBD>Ｔ<EFBFBD>嗪<EFBFBD>瘙<EFBFBD><EFBFBD>撌亙<EFBFBD> C <20><>鍂 **"Local-First" (<EFBFBD>砍𧑐隡睃<EFBFBD>)** <20>嗆<EFBFBD><E59786>?
|
||||
<EFBFBD>詨<EFBFBD>蝑𣇉裦嚗?
|
||||
1. **<EFBFBD>唳旿撽餌<EFBFBD>嚗?* <20>唳旿<E594B3>㰘蝸<E3B098>𦒘蜓閬<E89C93><E996AC><EFBFBD>典銁瘚讛<E7989A><E8AE9B>函<EFBFBD> **IndexedDB (Dexie.js)** <20>?**<2A><><EFBFBD> (Zustand)** 銝准<E98A9D>?
|
||||
2. **霈∠<E99C88>銝𧢲𦆮嚗?* 憭齿<E686AD><E9BDBF><EFBFBD>恣蝞烾<E89D9E>餉<EFBFBD>嚗<EFBFBD><E59A97> Pivot<6F><74><EFBFBD>撘讛圾<E8AE9B>琜<EFBFBD>銝𧢲𦆮<F0A7A2B2>?**Web Worker**嚗屸<E59A97><E5B1B8>漤獈憛?UI 銝餌瑪蝔卝<E89D94>?
|
||||
### **1.1 蝟餌<E89D9F><E9A48C>嗆<EFBFBD><E59786>?*
|
||||
为了满足 **PRD V2.1** 中“即时反馈”、“撤销重做”以及复杂的“长宽转换”需求，工具 C 采用 **"Local-First" (本地优先)** 架构。
|
||||
|
||||
核心策略：
|
||||
|
||||
1. **数据驻留：** 数据加载后主要存储在浏览器的 **IndexedDB (Dexie.js)** 和 **内存 (Zustand)** 中。
|
||||
2. **计算下放：** 复杂的计算逻辑（如 Pivot、公式解析）下放至 **Web Worker**，避免阻塞 UI 主线程。
|
||||
|
||||
### **1.1 系统架构图**
|
||||
|
||||
graph TD
|
||||
subgraph Browser\_Layer \[浏览器端 (React SPA)\]
|
||||
UI\_Shell\[UI 憯喳<EFBFBD>: <20><>像<EFBFBD>?Toolbar \+ <EFBFBD>箄<EFBFBD> Sidebar\]
|
||||
UI\_Shell\[UI 壳层: 扁平化 Toolbar \+ 智能 Sidebar\]
|
||||
|
||||
subgraph Core\_Engine \[核心引擎\]
|
||||
GridComponent\[AG Grid (閫<EFBFBD>㦛撅?\]
|
||||
GridComponent\[AG Grid (视图层)\]
|
||||
StateManager\[Zustand Store (状态层)\]
|
||||
|
||||
subgraph Worker\_Thread \[Web Worker 线程\]
|
||||
ComputeEngine\[计算引擎 (Math.js / Pivot Alg)\]
|
||||
StatEngine\[蝏蠘恣撘閙<EFBFBD> (<28>湔䲮<E6B994>?憸烐活)\]
|
||||
StatEngine\[统计引擎 (直方图/频次)\]
|
||||
end
|
||||
|
||||
HistoryManager\[Immer Patches (<EFBFBD>日<EFBFBD><EFBFBD>?\]
|
||||
HistoryManager\[Immer Patches (撤销栈)\]
|
||||
end
|
||||
|
||||
subgraph Local\_Storage \[持久化层\]
|
||||
@@ -36,13 +39,13 @@ graph TD
|
||||
end
|
||||
end
|
||||
|
||||
subgraph Server\_Layer \[<EFBFBD>滚𦛚蝡?(Node.js)\]
|
||||
subgraph Server\_Layer \[服务端 (Node.js)\]
|
||||
API\[Fastify API\]
|
||||
S3\[对象存储 (MinIO/OSS)\]
|
||||
end
|
||||
|
||||
User \--1.操作(如Pivot)--\> UI\_Shell
|
||||
UI\_Shell \--2.<EFBFBD>煾<EFBFBD><EFBFBD><EFBFBD><EFBFBD>?postMessage)--\> Worker\_Thread
|
||||
UI\_Shell \--2.发送消息(postMessage)--\> Worker\_Thread
|
||||
Worker\_Thread \--3.计算结果--\> StateManager
|
||||
StateManager \--4.更新视图--\> GridComponent
|
||||
StateManager \--5.异步备份--\> Dexie
|
||||
@@ -51,14 +54,14 @@ graph TD
|
||||
|
||||
## **2\. 技术选型 (Tech Stack)**
|
||||
|
||||
| 撅<EFBFBD>漣 | <20><><EFBFBD>舐<EFBFBD>隞?| <20>匧<EFBFBD><E58CA7><EFBFBD>眏 |
|
||||
| 层级 | 技术组件 | 选型理由 |
|
||||
| :---- | :---- | :---- |
|
||||
| **銵冽聢<EFBFBD>詨<EFBFBD>** | **AG Grid Community** | <EFBFBD>臭<EFBFBD><EFBFBD>賢<EFBFBD>韐寞𣈲<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>具<EFBFBD><EFBFBD><EFBFBD><EFBFBD>𡝗嗻<EFBFBD><EFBFBD><EFBFBD><EFBFBD>扯<EFBFBD>皜脫<EFBFBD><EFBFBD>?React 銵冽聢摨瓐<E691A8>?|
|
||||
| **<EFBFBD>砍𧑐<EFBFBD>唳旿摨?* | **Dexie.js (IndexedDB)** | <EFBFBD>豢<EFBFBD> localStorage (5MB<EFBFBD>𣂼<EFBFBD>)嚗䬠ndexedDB 摰寥<EFBFBD>憭找<EFBFBD>撘<EFBFBD>郊嚗屸<EFBFBD><EFBFBD><EFBFBD>摮睃<EFBFBD> 5銝? 銵𣬚<E98AB5> JSON <20>唳旿<E594B3><E697BF><EFBFBD>?|
|
||||
| **<EFBFBD>嗆<EFBFBD><EFBFBD>恣<EFBFBD>?* | **Zustand \+ Immer** | Zustand 頧駁<EFBFBD>擃䀹<EFBFBD>嚗熘mmer <20>其<EFBFBD>憭<EFBFBD><E686AD>銝滚虾<E6BB9A>䀹㺭<E480B9>桃<EFBFBD><E6A183><EFBFBD><EFBFBD><EFBFBD>?produce <EFBFBD>?patches <EFBFBD>蠘<EFBFBD><EFBFBD>臬<EFBFBD><EFBFBD>?Undo/Redo <EFBFBD><EFBFBD>瓲敹<EFBFBD><EFBFBD>?|
|
||||
| **霈∠<EFBFBD>撘閙<EFBFBD>** | **Math.js \+ Web Worker** | 閫<EFBFBD><EFBFBD> JS 瘚桃<EFBFBD><EFBFBD>啁移摨阡䔮憸?(0.1+0.2\!=0.3)嚗𢫕eb Worker <EFBFBD>其<EFBFBD>撠?Pivot 蝑厰<EFBFBD>霈∠<EFBFBD>蝘餃枂銝餌瑪蝔卝<EFBFBD>?|
|
||||
| **<EFBFBD>唳旿憭<EFBFBD><EFBFBD>** | **Lodash** | <EFBFBD>箇<EFBFBD><EFBFBD><EFBFBD>㺭<EFBFBD>格<EFBFBD>雿頣<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>皛扎<EFBFBD><EFBFBD>楛<EFBFBD>瑁<EFBFBD>嚗剹<EFBFBD>?|
|
||||
| **<EFBFBD>航<EFBFBD><EFBFBD>?* | **Ant Design Charts** | <EFBFBD>冽惣<EFBFBD>賭儒颲寞<EFBFBD>銝剔<EFBFBD><EFBFBD>嗥凒<EFBFBD>孵㦛 (Histogram) <EFBFBD>屸<EFBFBD>甈∪㦛 (Bar)<EFBFBD>?|
|
||||
| **表格核心** | **AG Grid Community** | 唯一能免费支持虚拟滚动、列拖拽、高性能渲染的 React 表格库。 |
|
||||
| **本地数据库** | **Dexie.js (IndexedDB)** | 相比 localStorage (5MB限制)，IndexedDB 容量大且异步，适合存储 5万+ 行的 JSON 数据集。 |
|
||||
| **状态管理** | **Zustand \+ Immer** | Zustand 轻量高效；Immer 用于处理不可变数据结构，其 produce 和 patches 功能是实现 Undo/Redo 的核心。 |
|
||||
| **计算引擎** | **Math.js \+ Web Worker** | 解决 JS 浮点数精度问题 (0.1+0.2\!=0.3)；Web Worker 用于将 Pivot 等重计算移出主线程。 |
|
||||
| **数据处理** | **Lodash** | 基础的数据操作（分组、过滤、深拷贝）。 |
|
||||
| **可视化** | **Ant Design Charts** | 在智能侧边栏中绘制直方图 (Histogram) 和频次图 (Bar)。 |
|
||||
|
||||
## **3\. 核心模块详细设计**
|
||||
|
||||
@@ -66,68 +69,74 @@ graph TD
|
||||
|
||||
#### **A. 长宽转换 (Pivot / Reshaping Algorithm) \- V2.1 核心难点**
|
||||
|
||||
餈蹱糓<EFBFBD><EFBFBD>憭齿<EFBFBD><EFBFBD><EFBFBD>恣蝞𦯀遙<EFBFBD>∴<EFBFBD>敹<EFBFBD>◆<EFBFBD>?Web Worker 銝剜<E98A9D>銵䕘<E98AB5><E49598>血<EFBFBD>憿菟𢒰隡𡁜㨃甇颯<E79487>?
|
||||
* **颲枏<E9A2B2><E69E8F><EFBFBD>㺭嚗?*
|
||||
这是最复杂的计算任务，必须在 Web Worker 中执行，否则页面会卡死。
|
||||
|
||||
* **输入参数：**
|
||||
* data: 原始对象数组 Row\[\]
|
||||
* indexCol: 銝駁睸<EFBFBD>堒<EFBFBD> (e.g., 'patient\_id') \- 蝖桀<EFBFBD><EFBFBD>𡏭<EFBFBD><EFBFBD>?
|
||||
* pivotKeyCol: <EFBFBD>箏<EFBFBD><EFBFBD>堒<EFBFBD> (e.g., 'visit\_date') \- 蝖桀<EFBFBD><EFBFBD>𨅯<EFBFBD><EFBFBD>𡒊<EFBFBD><EFBFBD>?
|
||||
* valueCols: <EFBFBD>澆<EFBFBD><EFBFBD>齿㺭蝏?(e.g., \['wbc', 'bmi'\]) \- 蝖桀<EFBFBD><EFBFBD>𨅯‵<EFBFBD><EFBFBD><EFBFBD>潑<EFBFBD>?
|
||||
* **蝞埈<EFBFBD><EFBFBD>餉<EFBFBD>嚗?*
|
||||
1. **憸<EFBFBD><EFBFBD><EFBFBD>?(Guard):** 霈∠<EFBFBD> Unique(pivotKeyCol).length \* valueCols.length<EFBFBD><EFBFBD><EFBFBD><EFBFBD>𦦵<EFBFBD><EFBFBD>鞟<EFBFBD>瞏𨅯銁<EFBFBD>埈㺭 \> 1000嚗峕<E59A97><E5B395>粹<EFBFBD>霂胼<E99C82>𦦵<EFBFBD><F0A6A6B5>鞟<EFBFBD><E99E9F>埈㺭餈<E3BAAD><E9A488>嚗諹窈<E8ABB9><E7AA88><EFBFBD><EFBFBD>㗇㺭<E39787>栽<EFBFBD>腈<EFBFBD>?
|
||||
2. **<EFBFBD><EFBFBD><EFBFBD> (Grouping):** 雿輻鍂 \_.groupBy(data, indexCol) <EFBFBD>劐蜓<EFBFBD>桀<EFBFBD>蝏<EFBFBD><EFBFBD>?
|
||||
3. **頧祆揢 (Transformation):** <EFBFBD>滚<EFBFBD>瘥讐<EFBFBD><EFBFBD>唳旿嚗?
|
||||
* <EFBFBD>𥕦遣銝<EFBFBD>銝芣鰵銵<EFBFBD>笆鞊∴<EFBFBD>靽萘<EFBFBD>銝駁睸<EFBFBD>?
|
||||
* <EFBFBD>滚<EFBFBD>霂亦<EFBFBD><EFBFBD><EFBFBD><EFBFBD>銝<EFBFBD><EFBFBD>∟扇敶𤏪<EFBFBD><EFBFBD>瑕<EFBFBD> pivotKeyCol <EFBFBD><EFBFBD><EFBFBD>潘<EFBFBD>靘见<EFBFBD> "2023-01-01"嚗剹<EFBFBD>?
|
||||
* <EFBFBD>滚<EFBFBD> valueCols嚗<EFBFBD><EFBFBD><EFBFBD>潭<EFBFBD>撠<EFBFBD>蛹 ValueCol\_PivotKey (靘见<EFBFBD> "wbc\_2023-01-01")<EFBFBD>?
|
||||
4. **Schema<EFBFBD><EFBFBD><EFBFBD>:** <EFBFBD>冽<EFBFBD><EFBFBD><EFBFBD><EFBFBD>鞉鰵<EFBFBD>?ColumnDefs<EFBFBD>?
|
||||
* **颲枏枂嚗?* { newRows, newColumnDefs }
|
||||
* indexCol: 主键列名 (e.g., 'patient\_id') \- 确定“行”
|
||||
* pivotKeyCol: 区分列名 (e.g., 'visit\_date') \- 确定“列后缀”
|
||||
* valueCols: 值列名数组 (e.g., \['wbc', 'bmi'\]) \- 确定“填充值”
|
||||
* **算法逻辑：**
|
||||
1. **预检查 (Guard):** 计算 Unique(pivotKeyCol).length \* valueCols.length。如果生成的潜在列数 \> 1000，抛出错误“生成的列数过多，请先筛选数据”。
|
||||
2. **分组 (Grouping):** 使用 \_.groupBy(data, indexCol) 按主键分组。
|
||||
3. **转换 (Transformation):** 遍历每组数据：
|
||||
* 创建一个新行对象，保留主键。
|
||||
* 遍历该组的每一条记录，获取 pivotKeyCol 的值（例如 "2023-01-01"）。
|
||||
* 遍历 valueCols，将值映射为 ValueCol\_PivotKey (例如 "wbc\_2023-01-01")。
|
||||
4. **Schema生成:** 动态生成新的 ColumnDefs。
|
||||
* **输出：** { newRows, newColumnDefs }
|
||||
|
||||
#### **B. 公式变量 (Formula)**
|
||||
|
||||
* 雿輻鍂 math.evaluate(formula, row)<EFBFBD>?
|
||||
* **摰匧<EFBFBD>瘝嗵拳嚗?* <20>𣂼<EFBFBD><F0A382BC>砍<EFBFBD>銝剖虾霈輸䔮<E8BCB8><E494AE><EFBFBD><EFBFBD>譍<EFBFBD>銝箏<E98A9D><E7AE8F>滩<EFBFBD><E6BBA9><EFBFBD>㺭<EFBFBD>殷<EFBFBD><E6AEB7>脫迫 XSS<EFBFBD>?
|
||||
* **撘<EFBFBD>虜憭<EFBFBD><EFBFBD>嚗?* 憭<><E686AD><EFBFBD>支誑<E694AF>?(Infinity) <20>屸<EFBFBD><E5B1B8>啣<EFBFBD>霈∠<E99C88> (NaN) <20><><EFBFBD><EFBFBD>蛛<EFBFBD>蝏煺<E89D8F>餈𥪜<E9A488> null <20>㚚<EFBFBD>霂舀<E99C82>霈啜<E99C88>?
|
||||
### **3.2 <20>箄<EFBFBD>靘扯器<E689AF>誩<EFBFBD><E8AAA9>?(Insight Engine)**
|
||||
* 使用 math.evaluate(formula, row)。
|
||||
* **安全沙箱：** 限制公式中可访问的变量仅为当前行的数据，防止 XSS。
|
||||
* **异常处理：** 处理除以零 (Infinity) 和非数字计算 (NaN) 的情况，统一返回 null 或错误标记。
|
||||
|
||||
* **閫血<E996AB>:** <20>穃𨯬 AG Grid <20>?onColumnHeaderClicked 鈭衤辣<E8A1A4>?
|
||||
* **<2A>餅<EFBFBD> (Debounce):** 200ms 撱嗉<E692B1>霈∠<E99C88>嚗屸俈甇Ｗ翰<EFBCB7>笔<EFBFBD><E7AC94>Ｗ<EFBFBD><EFBCB7>?UI <20>芰<EFBFBD><E88AB0>?
|
||||
### **3.2 智能侧边栏引擎 (Insight Engine)**
|
||||
|
||||
* **触发:** 监听 AG Grid 的 onColumnHeaderClicked 事件。
|
||||
* **去抖 (Debounce):** 200ms 延迟计算，防止快速切换列时 UI 闪烁。
|
||||
* **统计逻辑:**
|
||||
* **<EFBFBD>啣<EFBFBD>澆<EFBFBD>:** 霈∠<EFBFBD> Min, Max, Mean, SD嚗<EFBFBD>僎雿輻鍂 Freedman-Diaconis 閫<EFBFBD><EFBFBD>霈∠<EFBFBD><EFBFBD>湔䲮<EFBFBD>曄<EFBFBD> Bins<EFBFBD>?
|
||||
* **<EFBFBD><EFBFBD>𧋦<EFBFBD>?** 霈∠<E99C88> Top 10 憸𤑳<E686B8><F0A491B3><EFBFBD>擃条<E69383>霂溻<E99C82>?
|
||||
* **数值列:** 计算 Min, Max, Mean, SD，并使用 Freedman-Diaconis 规则计算直方图的 Bins。
|
||||
* **文本列:** 计算 Top 10 频率最高的词。
|
||||
|
||||
### **3.3 历史记录与撤销 (History Manager)**
|
||||
|
||||
* **Undo/Redo 策略:**
|
||||
* **<EFBFBD>桅<EFBFBD>𡁏<EFBFBD>雿?(蝻𤥁<E89DBB>/<2F>踵揢):** 霈啣<EFBFBD> patches (Immer)<EFBFBD>?
|
||||
* **蝏𤘪<EFBFBD><EFBFBD>扳<EFBFBD>雿?(Pivot/<2F><><EFBFBD>/<2F><><EFBFBD><EFBFBD>啣<EFBFBD><E595A3>?:** <20>曹<EFBFBD>銵函<E98AB5><E587BD><EFBFBD><EFBFBD><EFBFBD>冽㺿<E586BD>矋<EFBFBD>霈啣<E99C88> patches <20>鞉𧋦餈<F0A78BA6><E9A488>銝娪𠗕隞亙<E99A9E>皛𠾼<E79A9B><F0A0BEBC><EFBFBD><EFBFBD>交㺿銝綽<E98A9D>**<2A>冽<EFBFBD>銵峕迨蝐餅<E89D90>雿𨅯<E99BBF>嚗<EFBFBD>撩<EFBFBD>嗡<EFBFBD>摮䀝<E691AE>銝芸<E98A9D><E88AB8>誩翰<E8AAA9>?(Checkpoint)**<2A><>伃<EFBFBD><E4BC83><EFBFBD>嗥凒<E597A5>仿<EFBFBD>頧賢翰<E8B3A2>扼<EFBFBD>?
|
||||
* **普通操作 (编辑/替换):** 记录 patches (Immer)。
|
||||
* **结构性操作 (Pivot/拆分/生成新变量):** 由于表结构完全改变，记录 patches 成本过高且难以回滚。策略改为：**在执行此类操作前，强制保存一个全量快照 (Checkpoint)**。撤销时直接重载快照。
|
||||
|
||||
## **4\. 数据流与存储设计**
|
||||
|
||||
### **4.1 浏览器端存储 (Dexie Schema)**
|
||||
|
||||
<EFBFBD>其<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>冽<EFBFBD>甇<EFBFBD>銁蝻𤥁<EFBFBD><EFBFBD><EFBFBD>㺭<EFBFBD>殷<EFBFBD>摰䂿緵<EFBFBD>𡏭䌊<EFBFBD>典翰<EFBFBD>把<EFBFBD>嘥<EFBFBD><EFBFBD>𨅯援皞<EFBFBD><EFBFBD>憭𨧀<EFBFBD>腈<EFBFBD>?
|
||||
用于暂存用户正在编辑的数据，实现“自动快照”和“崩溃恢复”。
|
||||
|
||||
const db \= new Dexie('ResearchEditorDB');
|
||||
db.version(2).stores({
|
||||
// 憿寧𤌍<EFBFBD><EFBFBD>㺭<EFBFBD>?
|
||||
// 项目元数据
|
||||
projects: '++id, name, lastModified, rowCount',
|
||||
|
||||
// <EFBFBD>唳旿<EFBFBD>?(Chunks): 撠?5銝<35><E98A9D><EFBFBD>唳旿<E594B3><E697BF><EFBFBD>銝箏<E98A9D>銝?Chunk 摮睃<E691AE>嚗屸<E59A97><E5B1B8>滚<EFBFBD>甈∟粉<E2889F>躰<EFBFBD>憭批紡<E689B9>湔<EFBFBD>閫<EFBFBD>膥撏拇<E6928F>
|
||||
// 数据块 (Chunks): 将 5万行数据切分为多个 Chunk 存储，避免单次读写过大导致浏览器崩溃
|
||||
dataChunks: '\[projectId+chunkIndex\], projectId',
|
||||
|
||||
// 操作历史 (用于恢复现场)
|
||||
history: 'projectId, stack',
|
||||
|
||||
// 摰峕㟲敹怎<EFBFBD> (<28>其<EFBFBD> Pivot 蝑匧之<E58CA7>滢<EFBFBD><E6BBA2><EFBFBD><EFBFBD>皛?
|
||||
// 完整快照 (用于 Pivot 等大操作的回滚)
|
||||
checkpoints: '++id, projectId, createdAt'
|
||||
});
|
||||
|
||||
### **4.2 后端存储 (PostgreSQL \+ OSS)**
|
||||
|
||||
<EFBFBD>𡒊垢隞<EFBFBD><EFBFBD>韐<EFBFBD><EFBFBD><EFBFBD>兩<EFBFBD>𨅯歇靽嘥<EFBFBD><EFBFBD>萘<EFBFBD>敹怎<EFBFBD>嚗䔶<EFBFBD><EFBFBD><EFBFBD><EFBFBD>摰墧𧒄蝻𤥁<EFBFBD><EFBFBD>?
|
||||
后端仅负责存储“已保存”的快照，不参与实时编辑。
|
||||
|
||||
model DatasetSnapshot {
|
||||
id String @id @default(uuid())
|
||||
taskId String // 关联任务
|
||||
version Int // <EFBFBD><EFBFBD>𧋦<EFBFBD>?
|
||||
version Int // 版本号
|
||||
|
||||
// 摮睃<EFBFBD>銝箏之<EFBFBD>?JSON Blob嚗峕<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>?OSS <20><>辣頝臬<E9A09D> (<28>刻<EFBFBD> OSS)
|
||||
// 存储为大的 JSON Blob，或者指向 OSS 文件路径 (推荐 OSS)
|
||||
// 内容包含：rows\[\], columnDefs\[\], metadata
|
||||
ossKey String
|
||||
|
||||
@@ -136,31 +145,32 @@ model DatasetSnapshot {
|
||||
|
||||
## **5\. API 接口定义**
|
||||
|
||||
* POST /api/editor/init: <EFBFBD>嘥<EFBFBD><EFBFBD>𣇉<EFBFBD>颲穃膥隡朞<EFBFBD>嚗䔶<EFBFBD> OSS <20>㰘蝸<E3B098>笔<EFBFBD><E7AC94><EFBFBD>辣嚗<E8BEA3><E59A97><EFBFBD>𨀣糓隞𤾸極<F0A4BEB8>?A/B 瘚<>蓮餈<E893AE>䔉<EFBFBD><E49489><EFBFBD><EFBFBD>?
|
||||
* POST /api/editor/save: 靽嘥<EFBFBD>敶枏<EFBFBD>敹怎<EFBFBD><EFBFBD>?
|
||||
* POST /api/editor/export: 霂瑟<EFBFBD><EFBFBD>𡒊垢<EFBFBD><EFBFBD><EFBFBD> Excel/SPSS <EFBFBD><EFBFBD>辣<EFBFBD>?
|
||||
* POST /api/editor/init: 初始化编辑器会话，从 OSS 加载原始文件（如果是从工具 A/B 流转过来的）。
|
||||
* POST /api/editor/save: 保存当前快照。
|
||||
* POST /api/editor/export: 请求后端生成 Excel/SPSS 文件。
|
||||
* *Payload:* { rows: \[...\], format: 'spss' }
|
||||
* *霂湔<EFBFBD>:* 憒<><E68692><EFBFBD>唳旿<E594B3>誩<EFBFBD>嚗𣬚凒<F0A3AC9A>亙<EFBFBD>蝡?SheetJS <20><><EFBFBD>嚗𥟇㺭<F0A59F87>桅<EFBFBD>憭?(\>5MB) <20>𤑳<EFBFBD><F0A491B3>𡒊垢<F0A1928A><E59EA2><EFBFBD><EFBFBD>?
|
||||
## **6\. <20>扯<EFBFBD><E689AF><EFBFBD><EFBFBD>銝舘器<E88898>?(Performance Guardrails)**
|
||||
* *说明:* 如果数据量小，直接前端 SheetJS 生成；数据量大 (\>5MB) 发给后端生成。
|
||||
|
||||
## **6\. 性能准入与边界 (Performance Guardrails)**
|
||||
|
||||
| 数据量级 | 策略 |
|
||||
| :---- | :---- |
|
||||
| **\< 50,000 銵?* | **<EFBFBD>券<EFBFBD><EFBFBD>㰘蝸璅∪<EFBFBD>**<2A><><EFBFBD><EFBFBD>㗇㺭<E39787>桅<EFBFBD><E6A185>典<EFBFBD>摮?IndexedDB嚗峕<E59A97>雿𨀣<E99BBF>敹怒<E695B9>?|
|
||||
| **\> 50,000 銵?* | **<EFBFBD>漤<EFBFBD><EFBFBD>瑟芋撘?(Downsampling)**<2A><><EFBFBD>蝡臭<E89DA1><E887AD>㰘蝸<E3B098>?5 銝<><E98A9D><EFBFBD>其<EFBFBD>憸<EFBFBD><E686B8><EFBFBD>諹<EFBFBD><E8ABB9>坔<EFBFBD>摰𠾼<E691B0><F0A0BEBC>紡<EFBFBD>箸𧒄嚗<F0A79284><E59A97>皜<EFBFBD><E79A9C>閫<EFBFBD><E996AB>嚗㇌ecipe嚗匧<E59A97><E58CA7><EFBFBD><EFBFBD><EFBFBD>𡒊垢嚗𣬚眏<F0A3AC9A>𡒊垢 Worker 撖孵<E69296><E5ADB5>𤩺㺭<F0A4A9BA>株<EFBFBD>銵峕鸌憭<E9B88C><E686AD><EFBFBD>?|
|
||||
| **\< 50,000 行** | **全量加载模式**。所有数据都在内存/IndexedDB，操作极快。 |
|
||||
| **\> 50,000 行** | **降采样模式 (Downsampling)**。前端仅加载前 5 万行用于预览和规则制定。导出时，将清洗规则（Recipe）发送给后端，由后端 Worker 对全量数据进行批处理。 |
|
||||
|
||||
## **7\. 撘<EFBFBD><EFBFBD>𤏸恣<EFBFBD>?(Milestones)**
|
||||
## **7\. 开发计划 (Milestones)**
|
||||
|
||||
1. **Week 1: <EFBFBD>詨<EFBFBD>蝵烐聢銝𤾸<EFBFBD><EFBFBD>?*
|
||||
* <EFBFBD>剖遣 React \+ AG Grid <EFBFBD>臬<EFBFBD><EFBFBD>?
|
||||
* 摰䂿緵 SheetJS 撖澆<EFBFBD>銝?Dexie.js <EFBFBD><EFBFBD><EFBFBD><EFBFBD>㚚<EFBFBD>餉<EFBFBD><EFBFBD>?
|
||||
2. **Week 2: <EFBFBD><EFBFBD>像<EFBFBD>硋極<EFBFBD>瑟<EFBFBD>銝?Web Worker**
|
||||
* <EFBFBD>剖遣 Web Worker <EFBFBD>帋縑<EFBFBD>嗆<EFBFBD><EFBFBD>?
|
||||
* 摰䂿緵 Formula 霈∠<EFBFBD><EFBFBD>?Math.js <EFBFBD><EFBFBD><EFBFBD><EFBFBD>?
|
||||
* 摰䂿緵 Undo/Redo <EFBFBD><EFBFBD><EFBFBD>Immer嚗剹<EFBFBD>?
|
||||
1. **Week 1: 核心网格与存储**
|
||||
* 搭建 React \+ AG Grid 环境。
|
||||
* 实现 SheetJS 导入与 Dexie.js 持久化逻辑。
|
||||
2. **Week 2: 扁平化工具栏与 Web Worker**
|
||||
* 搭建 Web Worker 通信架构。
|
||||
* 实现 Formula 计算和 Math.js 集成。
|
||||
* 实现 Undo/Redo 栈（Immer）。
|
||||
3. **Week 3: 复杂计算 (Pivot)**
|
||||
* **<EFBFBD>滨<EFBFBD><EFBFBD>餃<EFBFBD>嚗?* <20>?Web Worker 銝剖<EFBFBD><EFBFBD>?Pivot 蝞埈<EFBFBD><EFBFBD>?
|
||||
* 摰䂿緵 Pivot <EFBFBD>?UI <EFBFBD>滨蔭撘寧<EFBFBD><EFBFBD>?
|
||||
* **重点攻坚：** 在 Web Worker 中实现 Pivot 算法。
|
||||
* 实现 Pivot 的 UI 配置弹窗。
|
||||
4. **Week 4: 智能侧边栏与导出**
|
||||
* 撘<EFBFBD><EFBFBD>𤑳凒<EFBFBD>孵㦛/憸烐活<E78390>曄<EFBFBD>隞?(AntD Charts)<EFBFBD>?
|
||||
* 摰䂿緵<EFBFBD><EFBFBD>拳<EFBFBD><EFBFBD><EFBFBD>撠<EFBFBD><EFBFBD><EFBFBD>‵銵亦撩憭勗<EFBFBD>潮<EFBFBD>餉<EFBFBD><EFBFBD>?
|
||||
* 撖寞𦻖<EFBFBD>𡒊垢靽嘥<EFBFBD><EFBFBD>亙藁<EFBFBD>
|
||||
* 开发直方图/频次图组件 (AntD Charts)。
|
||||
* 实现分箱、映射、填补缺失值逻辑。
|
||||
* 对接后端保存接口。
|
||||
Reference in New Issue
Block a user