HaHafeng
4c6eaaecbf
Summary: - Implement async file upload processing (Platform-Only pattern) - Add parseExcelWorker with pg-boss queue - Implement React Query polling mechanism - Add clean data caching (avoid duplicate parsing) - Fix pivot single-value column tuple issue - Optimize performance by 99 percent Technical Details: 1. Async Architecture (Postgres-Only): - SessionService.createSession: Fast upload + push to queue (3s) - parseExcelWorker: Background parsing + save clean data (53s) - SessionController.getSessionStatus: Status query API for polling - React Query Hook: useSessionStatus (auto-serial polling) - Frontend progress bar with real-time feedback 2. Performance Optimization: - Clean data caching: Worker saves processed data to OSS - getPreviewData: Read from clean data cache (0.5s vs 43s, -99 percent) - getFullData: Read from clean data cache (0.5s vs 43s, -99 percent) - Intelligent cleaning: Boundary detection + ghost column/row removal - Safety valve: Max 3000 columns, 5M cells 3. Bug Fixes: - Fix pivot column name tuple issue for single value column - Fix queue name format (colon to underscore: asl:screening -> asl_screening) - Fix polling storm (15+ concurrent requests -> 1 serial request) - Fix QUEUE_TYPE environment variable (memory -> pgboss) - Fix logger import in PgBossQueue - Fix formatSession to return cleanDataKey - Fix saveProcessedData to update clean data synchronously 4. Database Changes: - ALTER TABLE dc_tool_c_sessions ADD COLUMN clean_data_key VARCHAR(1000) - ALTER TABLE dc_tool_c_sessions ALTER COLUMN total_rows DROP NOT NULL - ALTER TABLE dc_tool_c_sessions ALTER COLUMN total_cols DROP NOT NULL - ALTER TABLE dc_tool_c_sessions ALTER COLUMN columns DROP NOT NULL 5. Documentation: - Create Postgres-Only async task processing guide (588 lines) - Update Tool C status document (Day 10 summary) - Update DC module status document - Update system overview document - Update cloud-native development guide Performance Improvements: - Upload + preview: 96s -> 53.5s (-44 percent) - Filter operation: 44s -> 2.5s (-94 percent) - Pivot operation: 45s -> 2.5s (-94 percent) - Concurrent requests: 15+ -> 1 (-93 percent) - Complete workflow (upload + 7 ops): 404s -> 70.5s (-83 percent) Files Changed: - Backend: 15 files (Worker, Service, Controller, Schema, Config) - Frontend: 4 files (Hook, Component, API) - Docs: 4 files (Guide, Status, Overview, Spec) - Database: 4 column modifications - Total: ~1388 lines of new/modified code Status: Fully tested and verified, production ready
ASL - AI智能文献
模块代号： ASL (AI Smart Literature)
开发状态： ⏳ 下一步开发（Week 2-4）
商业价值： ⭐⭐⭐⭐⭐ 可独立售卖
独立性： ⭐⭐⭐⭐⭐
优先级： P0
📋 模块概述
AI智能文献筛选系统，帮助研究者快速筛选和分析文献。
核心价值： 核心差异化功能，可独立售卖
🎯 核心功能（6个模块）
- ✅ 标题摘要初筛 - 双模型AI判断
- ✅ 全文复筛 - PDF全文分析
- ⏳ 全文解析与数据提取
- ⏳ 数据分析与报告生成
- ⏳ 系统评价与Meta分析
- ⏳ 文献管理
本周重点： 标题摘要初筛 + 全文复筛
📂 文档结构
ASL-AI智能文献/
├── [AI对接] ASL快速上下文.md # ⏳ 待创建
├── 00-项目概述/
│ ├── 01-产品需求文档(PRD).md # ⏳ 待合并（3个PRD）
│ └── ...
├── 01-设计文档/
│ ├── 02-数据库设计.md
│ ├── 03-API设计.md
│ └── 07-UI设计/
│ ├── 标题摘要初筛原型.html
│ └── 全文复筛原型.html
└── README.md # ✅ 当前文档
🔗 依赖的通用能力
- LLM网关 - 双模型AI判断
- 文档处理引擎 - PDF全文提取
- RAG引擎 - 文献内容检索
🎯 商业模式
目标客户： 系统评价研究者、循证医学中心
售卖方式： 独立产品
定价策略： 按项目数或按月订阅
最后更新： 2025-11-06
维护人： 技术架构师