7c3cc12b2e
feat(iit): Complete CRA Agent V3.0 P1 - ChatOrchestrator with LLM Function Calling
...
P1 Architecture: Lightweight ReAct (Function Calling loop, max 3 rounds)
Core changes:
- Add ToolDefinition/ToolCall types to LLM adapters (DeepSeek + CloseAI + Claude)
- Replace 6 old tools with 4 semantic tools: read_report, look_up_data, check_quality, search_knowledge
- Create ChatOrchestrator (~160 lines) replacing ChatService (1,442 lines)
- Wire WechatCallbackController to ChatOrchestrator, deprecate ChatService
- Fix nullable content (string | null) across 12+ LLM consumer files
E2E test results: 8/8 scenarios passed (100%)
- QC report query, critical issues, patient data, trend, on-demand QC
- Knowledge base search, project overview, data modification refusal
Net code reduction: ~1,100 lines
Tested: E2E P1 chat test 8/8 passed with DeepSeek API
Made-with: Cursor
2026-02-26 14:27:09 +08:00
371fa53956
docs(asl): Upgrade Tool 3 architecture from Fan-out to Scatter+Aggregator (v2.0)
...
Architecture transformation:
- Replace Fan-out (Manager->Child->Last Child Wins) with Scatter+Aggregator pattern
- API layer directly dispatches N independent jobs (no Manager)
- Worker only writes its own Result row, never touches Task table (zero row-lock)
- Aggregator polls groupBy for completion + zombie cleanup (replaces Sweeper)
- Reduce red lines from 13 to 9, eliminate distributed complexity
Documents updated (10 files):
- 08-Tool3 main architecture doc: v2.0 rewrite (schema, Task 2.3/2.4, red lines, risks)
- 08d-Code patterns: rewrite sections 4.1-4.6 (API dispatch, SingleWorker, Aggregator)
- 08a-M1 sprint: rewrite M1-3 core (Worker+Aggregator), red lines, acceptance criteria
- 08b-M2 sprint: simplify SSE (NOTIFY/LISTEN downgraded to P2 optional)
- 08c-M3 sprint: milestone table wording update
- New: Scatter+Polling Aggregator pattern guide v1.1 (Level 2 cookbook)
- New: V2.0 architecture deep review and gap-fix report
- Updated: ASL module status, system status, capability layer index
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-02-24 22:11:09 +08:00
dc6b292308
docs(asl): Complete Tool 3 extraction workbench V2.0 development plan (v1.5)
...
ASL Tool 3 Development Plan:
- Architecture blueprint v1.5 (6 rounds of architecture review, 13 red lines)
- M1/M2/M3 sprint checklists (Skeleton Pipeline / HITL Workbench / Dynamic Template Engine)
- Code patterns cookbook (9 chapters: Fan-out, Prompt engineering, ACL, SSE dual-track, etc.)
- Key patterns: Fan-out with Last Child Wins, Optimistic Locking, teamConcurrency throttling
- PKB ACL integration (anti-corruption layer), MinerU Cache-Aside, NOTIFY/LISTEN cross-pod SSE
- Data consistency snapshot for long-running extraction tasks
Platform capability:
- Add distributed Fan-out task pattern development guide (7 patterns + 10 anti-patterns)
- Add system-level async architecture risk analysis blueprint
- Add PDF table extraction engine design and usage guide (MinerU integration)
- Add table extraction source code (TableExtractionManager + MinerU engine)
Documentation updates:
- Update ASL module status with Tool 3 V2.0 plan readiness
- Update system status document (v6.2) with latest milestones
- Add V2.0 product requirements, prototypes, and data dictionary specs
- Add architecture review documents (4 rounds of review feedback)
- Add test PDF files for extraction validation
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-02-23 22:49:16 +08:00
8f06d4f929
feat(asl): Complete Deep Research V2.0 core development
...
Backend:
- Add SSE streaming client (unifuncsSseClient) replacing async polling
- Add paragraph-based reasoning parser with mergeConsecutiveThinking
- Add requirement expansion service (DeepSeek-V3 PICOS+MeSH)
- Add Word export service with Pandoc, inline hyperlinks, reference link expansion
- Add deep research V2 worker with 2s log flush and Chinese source prompt
- Add 5 curated data sources config (PubMed/ClinicalTrials/Cochrane/CNKI/MedJournals)
- Add 4 API endpoints (generate-requirement/tasks/task-status/export-word)
- Update Prisma schema with 6 new V2.0 fields on AslResearchTask
- Add DB migration for V2.0 fields
- Simplify ASL_DEEP_RESEARCH_EXPANSION prompt (remove strategy section)
Frontend:
- Add waterfall-flow DeepResearchPage (phase 0-4 progressive reveal)
- Add LandingView, SetupPanel, StrategyConfirm, AgentTerminal, ResultsView
- Add react-markdown + remark-gfm for report rendering
- Add custom link component showing visible URLs after references
- Add useDeepResearchTask polling hook
- Add deep research TypeScript types
Tests:
- Add E2E test, smoke test, and Chinese data source test scripts
Docs:
- Update ASL module status (v2.0 - core features complete)
- Update system status (v6.1 - ASL V2.0 milestone)
- Update Unifuncs DeepSearch API guide (v2.0 - SSE mode + Chinese source results)
- Update module auth specification (test script guidelines)
- Update V2.0 development plan
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-02-23 13:21:52 +08:00
b06daecacd
feat(asl): Add Deep Research V2.0 development plan and Unifuncs API site coverage testing
...
Completed:
- Unifuncs DeepSearch API site coverage test (18 medical sites, 9 tier-1 available)
- ClinicalTrials.gov dedicated test (4 strategies, English query + depth>=10 works best)
- Deep Research V2.0 development plan (5-day phased delivery)
- DeepResearch engine capability guide (docs/02-common-capability/)
- Test scripts: test-unifuncs-site-coverage.ts, test-unifuncs-clinicaltrials.ts
Key findings:
- Tier-1 sites: PubMed(28), ClinicalTrials(38), NCBI(18), Scholar(10), Cochrane(4), CNKI(7), SinoMed(9), GeenMedical(5), VIP(1)
- Paid databases (WoS/Embase/Scopus/Ovid) cannot be accessed (no credential support)
- ClinicalTrials.gov requires English queries with max_depth>=10
Updated: ASL module status doc, system status doc, common capability list
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-02-22 22:44:41 +08:00
3446909ff7
feat(ssa): Complete Phase I-IV intelligent dialogue and tool system development
...
Phase I - Session Blackboard + READ Layer:
- SessionBlackboardService with Postgres-Only cache
- DataProfileService for data overview generation
- PicoInferenceService for LLM-driven PICO extraction
- Frontend DataContextCard and VariableDictionaryPanel
- E2E tests: 31/31 passed
Phase II - Conversation Layer LLM + Intent Router:
- ConversationService with SSE streaming
- IntentRouterService (rule-first + LLM fallback, 6 intents)
- SystemPromptService with 6-segment dynamic assembly
- TokenTruncationService for context management
- ChatHandlerService as unified chat entry
- Frontend SSAChatPane and useSSAChat hook
- E2E tests: 38/38 passed
Phase III - Method Consultation + AskUser Standardization:
- ToolRegistryService with Repository Pattern
- MethodConsultService with DecisionTable + LLM enhancement
- AskUserService with global interrupt handling
- Frontend AskUserCard component
- E2E tests: 13/13 passed
Phase IV - Dialogue-Driven Analysis + QPER Integration:
- ToolOrchestratorService (plan/execute/report)
- analysis_plan SSE event for WorkflowPlan transmission
- Dual-channel confirmation (ask_user card + workspace button)
- PICO as optional hint for LLM parsing
- E2E tests: 25/25 passed
R Statistics Service:
- 5 new R tools: anova_one, baseline_table, fisher, linear_reg, wilcoxon
- Enhanced guardrails and block helpers
- Comprehensive test suite (run_all_tools_test.js)
Documentation:
- Updated system status document (v5.9)
- Updated SSA module status and development plan (v1.8)
Total E2E: 107/107 passed (Phase I: 31, Phase II: 38, Phase III: 13, Phase IV: 25)
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-02-22 18:53:39 +08:00
bf10dec4c8
docs(ssa): Complete intelligent dialogue and tool system architecture design
...
Architecture Design:
- Add intent recognition and dialogue architecture design (Intent Router + DataContext)
- Add tool system planning (4-layer 7-tool fusion solution: READ/INTERACT/THINK/ACT)
- Add 4-layer 7-tool implementation mechanism details (Conversation Layer LLM + Node.js orchestration)
Development Plan (v1.2):
- Create 6-phase development plan (134h/22 days) for intelligent dialogue system
- Add 8 architectural constraints (C1-C8): no Function Calling, Postgres-Only cache,
streaming output, context guard, Zod dynamic validation
- Correct Session Blackboard to use CacheFactory (Postgres-Only, no Redis)
Status Updates:
- Update SSA module status: QPER complete + dialogue architecture design complete
- Update system-level status: add SSA architecture design milestone
Other:
- R tools minor fixes (chi_square, correlation, logistic_binary, mann_whitney, t_test_paired)
- Frontend AIA chat workspace style adjustment
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-02-22 10:05:14 +08:00
371e1c069c
feat(ssa): Complete QPER architecture - Query, Planner, Execute, Reflection layers
...
Implement the full QPER intelligent analysis pipeline:
- Phase E+: Block-based standardization for all 7 R tools, DynamicReport renderer, Word export enhancement
- Phase Q: LLM intent parsing with dynamic Zod validation against real column names, ClarificationCard component, DataProfile is_id_like tagging
- Phase P: ConfigLoader with Zod schema validation and hot-reload API, DecisionTableService (4-dimension matching), FlowTemplateService with EPV protection, PlannedTrace audit output
- Phase R: ReflectionService with statistical slot injection, sensitivity analysis conflict rules, ConclusionReport with section reveal animation, conclusion caching API, graceful R error classification
End-to-end test: 40/40 passed across two complete analysis scenarios.
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-02-21 18:15:53 +08:00
428a22adf2
feat(ssa): Complete Phase 2A frontend integration - multi-step workflow end-to-end
...
Phase 2A: WorkflowPlannerService, WorkflowExecutorService, Python data quality, 6 bug fixes, DescriptiveResultView, multi-step R code/Word export, MVP UI reuse. V11 UI: Gemini-style, multi-task, single-page scroll, Word export. Architecture: Block-based rendering consensus (4 block types). New R tools: chi_square, correlation, descriptive, logistic_binary, mann_whitney, t_test_paired. Docs: dev summary, block-based plan, status updates, task list v2.0.
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-02-20 23:09:27 +08:00
49b5c37cb1
feat(ssa): Complete T-test end-to-end testing with 9 bug fixes - Phase 1 core 85% complete. R service: missing value auto-filter. Backend: error handling, variable matching, dynamic filename. Frontend: module activation, session isolation, error propagation. Full flow verified.
...
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-02-19 20:57:00 +08:00
8137e3cde2
feat(ssa): Complete SSA-Pro MVP development plan v1.3
...
Summary:
- Add PRD and architecture design V4 (Brain-Hand model)
- Complete 5 development guide documents
- Pass 3 rounds of team review (v1.0 -> v1.3)
- Add module status guide document
- Update system status document
Key Features:
- Brain-Hand architecture: Node.js + R Docker
- Statistical guardrails with auto degradation
- HITL workflow: PlanCard -> ExecutionTrace -> ResultCard
- Mixed data protocol: inline vs OSS
- Reproducible R code delivery
MVP Scope: 10 statistical tools
Status: Design 100%, ready for development
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-02-18 21:58:37 +08:00
f9ed0c2528
feat(rvw): Complete V2.0 Week 3 - Statistical validation extension and UX improvements
...
Week 3 Development Summary:
- Implement negative sign normalization (6 Unicode variants)
- Enhance T-test validation with smart sample size extraction
- Enhance SE triangle and CI-P consistency validation with subrow support
- Add precise sub-cell highlighting for P-values in multi-line cells
- Add frontend issue type Chinese translations (6 new types)
- Add file format tips for PDF/DOC uploads
Technical improvements:
- Add _clean_statistical_text() in extractor.py
- Add _safe_float() wrapper in validator.py
- Add ForensicsReport.tsx component
- Update ISSUE_TYPE_LABELS translations
Documentation:
- Add 2026-02-18 development record
- Update RVW module status (v5.1)
- Update system status (v5.2)
Status: Week 3 complete, ready for Week 4 testing
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-02-18 18:26:16 +08:00
9f256c4a02
feat(rvw): implement Skills architecture (Day 7-10)
...
- Add Skills core framework (types, registry, executor, profile, context)
- Implement DataForensicsSkill with DI, path security, graceful degradation
- Implement EditorialSkill and MethodologySkill wrapping existing services
- Extend ExtractionClient with IExtractionClient interface and analyzeDocx
- Refactor reviewWorker to support V1/V2 architecture switching
- Add Zod config validation and generic type support
- Update development docs and module status
Day 7: Skills core framework (~700 lines)
Day 8: DataForensicsSkill + ExtractionClient extension (~400 lines)
Day 9: EditorialSkill + MethodologySkill (~350 lines)
Day 10: ReviewWorker integration (~280 lines)
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-02-18 10:09:40 +08:00
e785969e54
feat(rvw): Implement RVW V2.0 Data Forensics Module - Day 6 StatValidator
...
Summary:
- Implement L2 Statistical Validator (CI-P consistency, T-test reverse)
- Implement L2.5 Consistency Forensics (SE Triangle, SD>Mean check)
- Add error/warning severity classification with tolerance thresholds
- Support 5+ CI formats parsing (parentheses, brackets, 95% CI prefix)
- Complete Python forensics service (types, config, validator, extractor)
V2.0 Development Progress (Week 2 Day 6):
- Day 1-5: Python service setup, Word table extraction, L1 arithmetic validator
- Day 6: L2 StatValidator + L2.5 consistency forensics (promoted from V2.1)
Test Results:
- Unit tests: 4/4 passed (CI-P, SE Triangle, SD>Mean, T-test)
- Real document tests: 5/5 successful, 2 reasonable WARNINGs
Status: Day 6 completed, ready for Day 7 (Skills Framework)
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-02-17 22:15:27 +08:00
7a299e8562
feat(iit): Implement event-level QC architecture V3.1 with dynamic rule filtering, report deduplication and AI intent enhancement
...
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-02-08 21:22:11 +08:00
45c7b32dbb
docs(iit): Add QC system UI and LLM format optimization plan
...
- Add development plan: 07-QC system UI and LLM format optimization
- Phase 1: PromptBuilder + XML clinical slice format (~5.5h)
- Phase 2: QC cockpit + stat cards + risk heatmap (~9.5h)
- Phase 3: Variable tagging system (~5h)
- Add CRA Agent design document and prototype
- Update module status: design 100%, code 45%
- Update system status: real-time QC complete + plan ready
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-02-08 09:59:44 +08:00
5db4a7064c
feat(iit): Implement real-time quality control system
...
Summary:
- Add 4 new database tables: iit_field_metadata, iit_qc_logs, iit_record_summary, iit_qc_project_stats
- Implement pg-boss debounce mechanism in WebhookController
- Refactor QC Worker for dual output: QC logs + record summary
- Enhance HardRuleEngine to support form-based rule filtering
- Create QcService for QC data queries
- Optimize ChatService with new intents: query_enrollment, query_qc_status
- Add admin batch operations: one-click full QC + one-click full summary
- Create IIT Admin management module: project config, QC rules, user mapping
Status: Code complete, pending end-to-end testing
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-02-07 21:56:11 +08:00
4b9b90ffb8
feat(iit): Complete REDCap production deployment on Alibaba Cloud ECS
...
Summary:
- Deploy REDCap 15.8.0 on ECS with Docker CE 26.1.3
- Configure RDS MySQL 8.0 database (redcap_prod)
- Setup Nginx reverse proxy with HTTPS/SSL
- Domain configured: https://redcap.xunzhengyixue.com/
Documentation:
- Add ECS deployment guide
- Add deployment info record
- Update system status document (v4.5 -> v4.6)
Status: REDCap production environment fully operational
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-02-02 22:27:05 +08:00
aaa29ea9d3
feat(admin): Implement Prompt knowledge base integration
...
Features:
- PromptService enhancement: enhanceWithKnowledge(), loadFullKnowledge(), ragSearch()
- FULL mode: Load entire knowledge base content
- RAG mode: Vector search based on user query
- Knowledge config API: PUT /:code/knowledge-config
- Test render with knowledge injection support
- Frontend: Knowledge config UI in Prompt editor
Bug fixes:
- Fix knowledge config not returned in getPromptDetail
- Fix publish button 400 error (empty request body)
- Fix cache not invalidated after publish
- Add detailed logging for debugging
Documentation:
- Add development record 2026-01-28
- Update ADMIN module status to Phase 4.6
- Update system status document to v4.5
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-02-01 20:26:20 +08:00
0d9e6b9922
feat(admin): Implement System Knowledge Base management module
...
Features:
- Backend: SystemKbService with full CRUD (knowledge bases + documents)
- Backend: 8 RESTful API endpoints (list/detail/create/update/delete/upload/download)
- Backend: OSS storage integration (system/knowledge-bases/{kbId}/{docId})
- Backend: RAG engine integration (document parsing, chunking, vectorization)
- Frontend: SystemKbListPage with card-based layout
- Frontend: SystemKbDetailPage with document management table
- Frontend: Master-Detail UX pattern for better user experience
- Document upload (single/batch), download (preserving original filename), delete
Technical:
- Database migration for system_knowledge_bases and system_kb_documents tables
- OSSAdapter.getSignedUrl with Content-Disposition for original filename
- Reuse RAG engine from common/rag for document processing
Tested: Local environment verified, all features working
2026-01-28 21:57:44 +08:00
bbf98c4d5c
fix(backend): Resolve PgBoss infinite loop issue and cleanup unused files
...
Backend fixes:
- Fix PgBoss task infinite loop on SAE (root cause: missing queue table constraints)
- Add singletonKey to prevent duplicate job enqueueing
- Add idempotency check in reviewWorker (skip completed tasks)
- Add optimistic locking in reviewService (atomic status update)
Frontend fixes:
- Add isSubmitting state to prevent duplicate submissions in RVW Dashboard
- Fix API baseURL in knowledgeBaseApi (relative path)
Cleanup (removed):
- Old frontend/ directory (migrated to frontend-v2)
- python-microservice/ (unused, replaced by extraction_service)
- Root package.json and node_modules (accidentally created)
- redcap-docker-dev/ (external dependency)
- Various temporary files and outdated docs in root
New documentation:
- docs/07-运维文档/01-PgBoss队列监控与维护.md
- docs/07-运维文档/02-故障预防检查清单.md
- docs/07-运维文档/03-数据库迁移注意事项.md
Database fix applied to RDS:
- Added PRIMARY KEY to platform_schema.queue
- Added 3 missing foreign key constraints
Tested: Local build passed, RDS constraints verified
2026-01-27 18:16:22 +08:00
2481b786d8
deploy: Complete 0126-27 deployment - database upgrade, services update, code recovery
...
Major Changes:
- Database: Install pg_bigm/pgvector plugins, create test database
- Python service: v1.0 -> v1.1, add pymupdf4llm/openpyxl/pypandoc
- Node.js backend: v1.3 -> v1.7, fix pino-pretty and ES Module imports
- Frontend: v1.2 -> v1.3, skip TypeScript check for deployment
- Code recovery: Restore empty files from local backup
Technical Fixes:
- Fix pino-pretty error in production (conditional loading)
- Fix ES Module import paths (add .js extensions)
- Fix OSSAdapter TypeScript errors
- Update Prisma Schema (63 models, 16 schemas)
- Update environment variables (DATABASE_URL, EXTRACTION_SERVICE_URL, OSS)
- Remove deprecated variables (REDIS_URL, DIFY_API_URL, DIFY_API_KEY)
Documentation:
- Create 0126 deployment folder with 8 documents
- Update database development standards v2.0
- Update SAE deployment status records
Deployment Status:
- PostgreSQL: ai_clinical_research_test with plugins
- Python: v1.1 @ 172.17.173.84:8000
- Backend: v1.7 @ 172.17.173.89:3001
- Frontend: v1.3 @ 172.17.173.90:80
Tested: All services running successfully on SAE
2026-01-27 08:13:27 +08:00
01a17f1e6f
feat(admin): Implement operational monitoring MVP and login optimization
...
Summary:
- Add SimpleLog table for activity tracking (admin_schema)
- Implement ActivityService with fire-and-forget pattern
- Add stats API endpoints (overview/live-feed/user-overview/cleanup)
- Complete activity logging for 7 modules (SYSTEM/AIA/PKB/ASL/DC/RVW/IIT)
- Update Admin Dashboard with DAU/DAT metrics and live feed
- Fix user module permission display logic
- Fix login redirect to /ai-qa instead of homepage
- Replace top navigation LOGO with brand image
- Fix PKB workspace layout CSS conflict (rename to .pa-chat-container)
New files:
- backend/src/common/services/activity.service.ts
- backend/src/modules/admin/controllers/statsController.ts
- backend/src/modules/admin/routes/statsRoutes.ts
- frontend-v2/src/modules/admin/api/statsApi.ts
- docs/03-.../04-operational-monitoring-mvp-plan.md
- docs/03-.../04-operational-monitoring-mvp-implementation.md
Tested: All features verified locally
2026-01-25 22:16:16 +08:00
303dd78c54
feat(aia): Protocol Agent MVP complete with one-click generation and Word export
...
- Add one-click research protocol generation with streaming output
- Implement Word document export via Pandoc integration
- Add dynamic dual-panel layout with resizable split pane
- Implement collapsible content for StatePanel stages
- Add conversation history management with title auto-update
- Fix scroll behavior, markdown rendering, and UI layout issues
- Simplify conversation creation logic for reliability
2026-01-25 19:16:36 +08:00
596f2dfc02
docs: Update RAG engine docs - pg_bigm v1.2 installation completed
...
Summary:
- pg_bigm v1.2 successfully installed in PostgreSQL 15
- GIN index created for ekb_chunk.content
- Keyword search performance improved 10-100x
Documentation updates:
- RAG engine guide v1.0 -> v1.1 (update Q3, add performance data)
- System status guide v4.1 -> v4.2 (mark pg_bigm as installed)
- pg_bigm installation guide v1.0 -> v1.1 (mark as completed)
Status: Production ready with full RAG capabilities
2026-01-24 19:17:24 +08:00
96290d2f76
feat(aia): Implement Protocol Agent MVP with reusable Agent framework
...
Sprint 1-3 Completed (Backend + Frontend):
Backend (Sprint 1-2):
- Implement 5-layer Agent framework (Query->Planner->Executor->Tools->Reflection)
- Create agent_schema with 6 tables (agent_definitions, stages, prompts, sessions, traces, reflexion_rules)
- Create protocol_schema with 2 tables (protocol_contexts, protocol_generations)
- Implement Protocol Agent core services (Orchestrator, ContextService, PromptBuilder)
- Integrate LLM service adapter (DeepSeek/Qwen/GPT-5/Claude)
- 6 API endpoints with full authentication
- 10/10 API tests passed
Frontend (Sprint 3):
- Add Protocol Agent entry in AgentHub (indigo theme card)
- Implement ProtocolAgentPage with 3-column layout
- Collapsible sidebar (Gemini style, 48px <-> 280px)
- StatePanel with 5 stage cards (scientific_question, pico, study_design, sample_size, endpoints)
- ChatArea with sync button and action cards integration
- 100% prototype design restoration (608 lines CSS)
- Detailed endpoints structure: baseline, exposure, outcomes, confounders
Features:
- 5-stage dialogue flow for research protocol design
- Conversation-driven interaction with sync-to-protocol button
- Real-time context state management
- One-click protocol generation button (UI ready, backend pending)
Database:
- agent_schema: 6 tables for reusable Agent framework
- protocol_schema: 2 tables for Protocol Agent
- Seed data: 1 agent + 5 stages + 9 prompts + 4 reflexion rules
Code Stats:
- Backend: 13 files, 4338 lines
- Frontend: 14 files, 2071 lines
- Total: 27 files, 6409 lines
Status: MVP core functionality completed, pending frontend-backend integration testing
Next: Sprint 4 - One-click protocol generation + Word export
2026-01-24 17:29:24 +08:00
9c96f75c52
feat(storage): integrate Alibaba Cloud OSS for file persistence - Add OSSAdapter and LocalAdapter with StorageFactory pattern - Integrate PKB module with OSS upload - Rename difyDocumentId to storageKey - Create 4 OSS buckets and development specification
2026-01-22 22:02:20 +08:00
483c62fb6f
feat(pkb): Replace Dify with self-developed pgvector RAG engine
...
Major milestone: Successfully replaced Dify external service with PostgreSQL + pgvector RAG engine
Backend changes:
- Refactor ragService.ts: Remove dual-track mode, keep only pgvector
- Refactor knowledgeBaseService.ts: Remove Dify creation logic
- Refactor documentService.ts: Remove Dify upload/polling logic
- DifyClient.ts: Convert to deprecated stub file (for legacy compatibility)
- common/rag/index.ts: Update exports
- common/rag/types.ts: Remove Dify types, keep generic RAG types
- config/env.ts: Remove Dify configuration
Frontend changes:
- DashboardPage.tsx: Add delete knowledge base dropdown menu
- KnowledgeBaseList.tsx: Enhance quota warning display
- CreateKBDialog.tsx: Add quota exceeded modal with guidance
- knowledgeBaseApi.ts: Add auth interceptor
Documentation:
- Update PKB module status guide (v2.3)
- Update system status guide (v4.0)
Performance metrics:
- Single query latency: 2.5s
- Single query cost: 0.0025 CNY
- Cross-language accuracy improvement: +20.5%
Remaining tasks:
- OSS storage integration
- pg_bigm extension installation
Tested: End-to-end test passed (create KB -> upload doc -> vector search)
2026-01-21 22:35:50 +08:00
40c2f8e148
feat(rag): Complete RAG engine implementation with pgvector
...
Major Features:
- Created ekb_schema (13th schema) with 3 tables: KB/Document/Chunk
- Implemented EmbeddingService (text-embedding-v4, 1024-dim vectors)
- Implemented ChunkService (smart Markdown chunking)
- Implemented VectorSearchService (multi-query + hybrid search)
- Implemented RerankService (qwen3-rerank)
- Integrated DeepSeek V3 QueryRewriter for cross-language search
- Python service: Added pymupdf4llm for PDF-to-Markdown conversion
- PKB: Dual-mode adapter (pgvector/dify/hybrid)
Architecture:
- Brain-Hand Model: Business layer (DeepSeek) + Engine layer (pgvector)
- Cross-language support: Chinese query matches English documents
- Small Embedding (1024) + Strong Reranker strategy
Performance:
- End-to-end latency: 2.5s
- Cost per query: 0.0025 RMB
- Accuracy improvement: +20.5% (cross-language)
Tests:
- test-embedding-service.ts: Vector embedding verified
- test-rag-e2e.ts: Full pipeline tested
- test-rerank.ts: Rerank quality validated
- test-query-rewrite.ts: Cross-language search verified
- test-pdf-ingest.ts: Real PDF document tested (Dongen 2003.pdf)
Documentation:
- Added 05-RAG-Engine-User-Guide.md
- Added 02-Document-Processing-User-Guide.md
- Updated system status documentation
Status: Production ready
2026-01-21 20:24:29 +08:00
dfc0fe0b9a
feat(pkb): Integrate pgvector and create Dify replacement plan
...
Summary:
- Migrate PostgreSQL to pgvector/pgvector:pg15 Docker image
- Successfully install and verify pgvector 0.8.1 extension
- Create comprehensive Dify-to-pgvector migration plan
- Update PKB module documentation with pgvector status
- Update system documentation with pgvector integration
Key changes:
- docker-compose.yml: Switch to pgvector/pgvector:pg15 image
- Add EkbDocument and EkbChunk data model design
- Design R-C-R-G hybrid retrieval architecture
- Add clinical data JSONB fields (pico, studyDesign, regimen, safety, criteria, endpoints)
- Create detailed 10-day implementation roadmap
Documentation updates:
- PKB module status: pgvector RAG infrastructure ready
- System status: pgvector 0.8.1 integrated
- New: Dify replacement development plan (01-Dify替换为pgvector开发计划.md)
- New: Enterprise medical knowledge base solution V2
Tested: PostgreSQL with pgvector verified, frontend and backend functionality confirmed
2026-01-20 00:00:58 +08:00
1ece9a4ae8
feat(asl): Add DeepSearch smart literature retrieval MVP
...
Features:
- Integrate unifuncs DeepSearch API (OpenAI compatible protocol)
- SSE real-time streaming for AI thinking process display
- Natural language input, auto-generate PubMed search strategy
- Extract and display PubMed literature links
- Database storage for task records (asl_research_tasks)
Backend:
- researchService.ts - Core business logic with SSE streaming
- researchController.ts - SSE stream endpoint
- researchWorker.ts - Async task worker (backup mode)
- schema.prisma - AslResearchTask model
Frontend:
- ResearchSearch.tsx - Search page with unified content stream
- ResearchSearch.css - Styling (unifuncs-inspired simple design)
- ASLLayout.tsx - Enable menu item
- api/index.ts - Add research API functions
API Endpoints:
- POST /api/v1/asl/research/stream - SSE streaming search
- POST /api/v1/asl/research/tasks - Async task creation
- GET /api/v1/asl/research/tasks/:taskId/status - Task status
Documentation:
- Development record for DeepSearch integration
- Update ASL module status (v1.5)
- Update system status (v3.7)
Known limitations:
- SSE mode, task interrupts when leaving page
- Cost ~0.3 RMB per search (unifuncs API)
2026-01-18 19:15:55 +08:00
57fdc6ef00
feat(aia): Integrate PromptService for 10 AI agents
...
Features:
- Migrate 10 agent prompts from hardcoded to database
- Add grayscale preview support (DRAFT/ACTIVE distribution)
- Implement 3-tier fallback (DB -> Cache -> Hardcoded)
- Add version management and rollback capability
Files changed:
- backend/scripts/migrate-aia-prompts.ts (new migration script)
- backend/src/common/prompt/prompt.fallbacks.ts (add AIA fallbacks)
- backend/src/modules/aia/services/agentService.ts (integrate PromptService)
- backend/src/modules/aia/services/conversationService.ts (pass userId)
- backend/src/modules/aia/types/index.ts (fix AgentStage type)
Documentation:
- docs/03-业务模块/AIA-AI智能问答/06-开发记录/2026-01-18-Prompt管理系统集成.md
- docs/02-通用能力层/00-通用能力层清单.md (add FileCard, Prompt management)
- docs/00-系统总体设计/00-系统当前状态与开发指南.md (update to v3.6)
Prompt codes:
- AIA_SCIENTIFIC_QUESTION, AIA_PICO_ANALYSIS, AIA_TOPIC_EVALUATION
- AIA_OUTCOME_DESIGN, AIA_CRF_DESIGN, AIA_SAMPLE_SIZE
- AIA_PROTOCOL_WRITING, AIA_METHODOLOGY_REVIEW
- AIA_PAPER_POLISH, AIA_PAPER_TRANSLATE
Tested: Migration script executed, all 10 prompts inserted successfully
2026-01-18 15:48:53 +08:00
66255368b7
feat(admin): Add user management and upgrade to module permission system
...
Features - User Management (Phase 4.1):
- Database: Add user_modules table for fine-grained module permissions
- Database: Add 4 user permissions (view/create/edit/delete) to role_permissions
- Backend: UserService (780 lines) - CRUD with tenant isolation
- Backend: UserController + UserRoutes (648 lines) - 13 API endpoints
- Backend: Batch import users from Excel
- Frontend: UserListPage (412 lines) - list/filter/search/pagination
- Frontend: UserFormPage (341 lines) - create/edit with module config
- Frontend: UserDetailPage (393 lines) - details/tenant/module management
- Frontend: 3 modal components (592 lines) - import/assign/configure
- API: GET/POST/PUT/DELETE /api/admin/users/* endpoints
Architecture Upgrade - Module Permission System:
- Backend: Add getUserModules() method in auth.service
- Backend: Login API returns modules array in user object
- Frontend: AuthContext adds hasModule() method
- Frontend: Navigation filters modules based on user.modules
- Frontend: RouteGuard checks requiredModule instead of requiredVersion
- Frontend: Remove deprecated version-based permission system
- UX: Only show accessible modules in navigation (clean UI)
- UX: Smart redirect after login (avoid 403 for regular users)
Fixes:
- Fix UTF-8 encoding corruption in ~100 docs files
- Fix pageSize type conversion in userService (String to Number)
- Fix authUser undefined error in TopNavigation
- Fix login redirect logic with role-based access check
- Update Git commit guidelines v1.2 with UTF-8 safety rules
Database Changes:
- CREATE TABLE user_modules (user_id, tenant_id, module_code, is_enabled)
- ADD UNIQUE CONSTRAINT (user_id, tenant_id, module_code)
- INSERT 4 permissions + role assignments
- UPDATE PUBLIC tenant with 8 module subscriptions
Technical:
- Backend: 5 new files (~2400 lines)
- Frontend: 10 new files (~2500 lines)
- Docs: 1 development record + 2 status updates + 1 guideline update
- Total: ~4900 lines of code
Status: User management 100% complete, module permission system operational
2026-01-16 13:42:10 +08:00
3d35e9c58b
feat(aia): Complete AIA V2.0 with universal streaming capabilities
...
Major Updates:
- Add StreamingService with OpenAI Compatible format (backend/common/streaming)
- Upgrade Chat component V2 with Ant Design X integration
- Implement AIA module with 12 intelligent agents
- Create AgentHub with 100% prototype V11 restoration
- Create ChatWorkspace with streaming response support
- Add ThinkingBlock for deep thinking display
- Add useAIStream Hook for OpenAI Compatible stream handling
Backend Common Capabilities (~400 lines):
- OpenAIStreamAdapter: SSE adapter with OpenAI format
- StreamingService: unified streaming service
- Support content and reasoning_content dual streams
- Deep thinking tag processing (<think>...</think>)
Frontend Common Capabilities (~2000 lines):
- AIStreamChat: modern streaming chat component
- ThinkingBlock: collapsible deep thinking display
- ConversationList: conversation management with grouping
- useAIStream: OpenAI Compatible stream handler Hook
- useConversations: conversation state management Hook
- Modern design styles (Ultramodern theme)
AIA Module Frontend (~1500 lines):
- AgentHub: 12 agent cards with timeline design
- ChatWorkspace: fullscreen immersive chat interface
- AgentCard: theme-colored cards (blue/yellow/teal/purple)
- 5 phases, 12 agents configuration
- Responsive layout (desktop + mobile)
AIA Module Backend (~900 lines):
- agentService: 12 agents config with system prompts
- conversationService: refactored with StreamingService
- attachmentService: file upload skeleton (30k token limit)
- 12 API endpoints with authentication
- Full CRUD for conversations and messages
Documentation:
- AIA module status and development guide
- Universal capabilities catalog (11 services)
- Quick reference card for developers
- System overview updates
Testing:
- Stream response verified (HTTP 200)
- Authentication working correctly
- Auto conversation creation working
- Deep thinking display working
- Message input and send working
Status: Core features completed (85%), attachment and history loading pending
2026-01-14 19:09:28 +08:00
5523ef36ea
feat(admin): Complete Phase 3.5.1-3.5.4 Prompt Management System (83%)
...
Summary:
- Implement Prompt management infrastructure and core services
- Build admin portal frontend with light theme
- Integrate CodeMirror 6 editor for non-technical users
Phase 3.5.1: Infrastructure Setup
- Create capability_schema for Prompt storage
- Add prompt_templates and prompt_versions tables
- Add prompt:view/edit/debug/publish permissions
- Migrate RVW prompts to database (RVW_EDITORIAL, RVW_METHODOLOGY)
Phase 3.5.2: PromptService Core
- Implement gray preview logic (DRAFT for debuggers, ACTIVE for users)
- Module-level debug control (setDebugMode)
- Handlebars template rendering
- Variable extraction and validation (extractVariables, validateVariables)
- Three-level disaster recovery (database -> cache -> hardcoded fallback)
Phase 3.5.3: Management API
- 8 RESTful endpoints (/api/admin/prompts/*)
- Permission control (PROMPT_ENGINEER can edit, SUPER_ADMIN can publish)
Phase 3.5.4: Frontend Management UI
- Build admin portal architecture (AdminLayout, OrgLayout)
- Add route system (/admin/*, /org/*)
- Implement PromptListPage (filter, search, debug switch)
- Implement PromptEditor (CodeMirror 6 simplified for clinical users)
- Implement PromptEditorPage (edit, save, publish, test, version history)
Technical Details:
- Backend: 6 files, ~2044 lines (prompt.service.ts 596 lines)
- Frontend: 9 files, ~1735 lines (PromptEditorPage.tsx 399 lines)
- CodeMirror 6: Line numbers, auto-wrap, variable highlight, search, undo/redo
- Chinese-friendly: 15px font, 1.8 line-height, system fonts
Next Step: Phase 3.5.5 - Integrate RVW module with PromptService
Tested: Backend API tests passed (8/8), Frontend pending user testing
Status: Ready for Phase 3.5.5 RVW integration
2026-01-11 21:25:16 +08:00
cdfbc9927a
docs: Update RVW module status to 95% after schema migration
...
RVW Module (v3.2):
- Phase 6 complete: Schema isolation (public -> rvw_schema)
- Module progress: 95% complete
- All technical debt resolved except PDF export
System Status (v3.1):
- RVW module updated to 95% completion
- Schema isolation highlighted as technical achievement
Verified: All RVW functions working after migration
2026-01-10 23:18:01 +08:00
440f75255e
feat(rvw): Complete Phase 4-5 - Bug fixes and Word export
...
Summary:
- Fix methodology score display issue in task list (show score instead of 'warn')
- Add methodology_score field to database schema
- Fix report display when only methodology agent is selected
- Implement Word document export using docx library
- Update documentation to v3.0/v3.1
Backend changes:
- Add methodologyScore to Prisma schema and TaskSummary type
- Update reviewWorker to save methodologyScore
- Update getTaskList to return methodologyScore
Frontend changes:
- Install docx and file-saver libraries
- Implement handleExportReport with Word generation
- Fix activeTab auto-selection based on available data
- Add proper imports for docx components
Documentation:
- Update RVW module status to 90% (Phase 1-5 complete)
- Update system status document to v3.0
Tested: All review workflows verified, Word export functional
2026-01-10 22:52:15 +08:00
179afa2c6b
feat(rvw): Complete RVW module development Phase 1-3
...
Summary:
- Migrate backend to modules/rvw with v2 API routes (/api/v2/rvw)
- Add new database fields: selectedAgents, editorialScore, methodologyStatus, picoExtract, isArchived
- Create frontend module in frontend-v2/src/modules/rvw
- Implement Dashboard with task list, filtering, batch operations
- Implement ReportDetail with dual tabs (editorial/methodology)
- Implement AgentModal for intelligent agent selection
- Register RVW module in moduleRegistry.ts
- Add navigation entry in TopNavigation
- Update documentation for RVW module status (v3.0)
- Update system status document (v2.9)
Features:
- User can select agents: editorial, methodology, or both
- Support batch task execution
- Task status filtering
- Replace console.log with logger service
- Maintain v1 API backward compatibility
Tested: Frontend and backend verified locally
Status: 85% complete (Phase 1-3 done)
2026-01-07 22:39:08 +08:00
06028c6952
feat(pkb): implement complete batch processing workflow and frontend optimization
...
- Frontend V3 architecture migration to modules/pkb
- Implement three work modes: full-text reading, deep reading, batch processing
- Complete batch processing: template selection, progress display, result export (CSV)
- Integrate Ant Design X Chat component with streaming support
- Add document upload modal with drag-and-drop support
- Optimize UI: multi-line table display, citation formatting, auto-scroll
- Fix 10+ technical issues: API mapping, state sync, form clearing
- Update documentation: development records and module status
Performance: 3 docs batch processing ~17-28s
Status: PKB module now production-ready (90% complete)
2026-01-07 18:23:43 +08:00
e59676342a
docs(pkb): Add development records and update system status
...
Summary:
- Add PKB module development record for 2026-01-07
- Create PKB module status document (00-模块当前状态与开发指南.md)
- Update system status document to v2.7
Documents added:
- docs/03-业务模块/PKB-个人知识库/06-开发记录/2026-01-07_PKB模块前端V3设计实现.md
- docs/03-业务模块/PKB-个人知识库/00-模块当前状态与开发指南.md
Documents updated:
- docs/00-系统总体设计/00-系统当前状态与开发指南.md
PKB module progress: 75% complete
- Frontend Dashboard: 90%
- Frontend Workspace: 85%
- 3 work modes implemented
- Batch processing API pending debug
2026-01-07 10:35:03 +08:00
b47079b387
feat(iit): Phase 1.5 AI对话集成REDCap真实数据完成
...
- feat: ChatService集成DeepSeek-V3实现AI对话（390行）
- feat: SessionMemory实现上下文记忆（最近3轮对话，170行）
- feat: 意图识别支持REDCap数据查询（关键词匹配）
- feat: REDCap数据注入LLM（queryRedcapRecord, countRedcapRecords, getProjectInfo）
- feat: 解决LLM幻觉问题（基于真实数据回答，明确system prompt）
- feat: 即时反馈（正在查询...提示）
- test: REDCap查询测试通过（test0102项目，10条记录，ID 7患者详情）
- docs: 创建Phase1.5开发完成记录（313行）
- docs: 更新Phase1.5开发计划（标记完成）
- docs: 更新MVP开发任务清单（Phase 1.5完成）
- docs: 更新模块当前状态（60%完成度）
- docs: 更新系统总体设计文档（v2.6）
- chore: 删除测试脚本（test-redcap-query-for-ai.ts, check-env-config.ts）
- chore: 移除REDCap测试环境变量（REDCAP_TEST_*）
技术亮点：
- AI基于REDCap真实数据对话，不编造信息
- 从数据库读取项目配置，不使用环境变量
- 企业微信端测试通过，用户体验良好
测试通过：
- 查询项目记录总数（10条）
- 查询特定患者详情（ID 7）
- 项目信息查询
- 上下文记忆（3轮对话）
- 即时反馈提示
影响范围：IIT Manager Agent模块
2026-01-03 22:48:10 +08:00
2eef7522a1
feat(iit): Complete Day 2 - REDCap real-time integration
...
Summary:
- Implement RedcapAdapter (271 lines, 7 API methods)
- Implement WebhookController (327 lines, <10ms response)
- Implement SyncManager (398 lines, incremental/full sync)
- Register Workers (iit_quality_check + iit_redcap_poll)
- Configure routes with form-urlencoded parser
- Add 3 integration test scripts (912 lines total)
- Complete development documentation
Technical Highlights:
- REDCap DET real-time trigger (0ms delay)
- Webhook + scheduled polling dual mechanism
- Form-urlencoded format support for REDCap DET
- Postgres-Only architecture with pg-boss queue
- Full compliance with team development standards
Test Results:
- Integration tests: 12/12 passed
- Real scenario validation: PASSED
- Performance: Webhook response <10ms
- Data accuracy: 100%
Progress:
- Module completion: 18% -> 35%
- Day 2 development: COMPLETED
- Production ready: YES
2026-01-02 18:20:18 +08:00
4c5bb3d174
feat(iit): Initialize IIT Manager Agent MVP - Day 1 complete
...
- Add iit_schema with 5 tables
- Create module structure and types (223 lines)
- WeChat integration verified (Access Token success)
- Update system docs to v2.4
- Add REDCap source folders to .gitignore
- Day 1/14 complete (11/11 tasks)
2025-12-31 18:35:05 +08:00
decff0bb1f
docs(deploy): Complete full system deployment to Aliyun SAE
...
Summary:
- Successfully deployed complete system to Aliyun SAE (2025-12-25)
- All services running: Python microservice + Node.js backend + Frontend Nginx + CLB
- Public access available at http://8.140.53.236/
Major Achievements:
1. Python microservice deployed (v1.0, internal IP: 172.17.173.66:8000)
2. Node.js backend deployed (v1.3, internal IP: 172.17.173.73:3001)
- Fixed 4 critical issues: bash path, config directory, pino-pretty, ES Module
3. Frontend Nginx deployed (v1.0, internal IP: 172.17.173.72:80)
4. CLB load balancer configured (public IP: 8.140.53.236)
New Documentation (9 docs):
- 11-Node.js backend SAE deployment config checklist (21 env vars)
- 12-Node.js backend SAE deployment operation manual
- 13-Node.js backend image fix record (config directory)
- 14-Node.js backend pino-pretty fix
- 15-Node.js backend deployment success summary
- 16-Frontend Nginx deployment success summary
- 17-Complete deployment practical manual 2025 edition (1800 lines)
- 18-Deployment documentation usage guide
- 19-Daily update quick operation manual (670 lines)
Key Fixes:
- Environment variable name correction: EXTRACTION_SERVICE_URL (not PYTHON_SERVICE_URL)
- Dockerfile fix: added COPY config ./config
- Logger configuration: conditional pino-pretty for dev only
- Health check fix: ES Module compatibility (require -> import)
Updated Files:
- System status document updated with full deployment info
- Deployment progress overview updated with latest IPs
- All 3 Docker services' Dockerfiles and configs refined
Verification:
- All health checks passed
- Tool C 7 features working correctly
- Literature screening module functional
- Response time < 1 second
BREAKING CHANGE: Node.js backend internal IP changed from 172.17.173.71 to 172.17.173.73
Closes #deployment-milestone
2025-12-25 21:24:37 +08:00
691dc2bc98
docs(deploy): Update deployment documentation for Node.js backend
...
Summary:
- Created Node.js backend Docker image build guide
- Updated deployment progress overview with backend status
- Updated system status documentation
Backend build achievements:
- Fixed 200+ TypeScript compilation errors (200+ to 0)
- Completed Prisma reverse sync (32 models from RDS)
- Manually added 30+ Prisma relation fields
- Successfully built Docker image (838MB)
- Pushed image to ACR (v1.0 + latest tags)
Documentation updates:
- Added 10-Node.js后端-Docker镜像构建手册.md
- Updated 00-部署进度总览.md with backend deployment status
- Updated 00-系统当前状态与开发指南.md with latest progress
- Fixed date format (2024 -> 2025)
Next steps:
- Deploy Node.js backend to SAE
- Configure environment variables
- Test end-to-end functionality
Status: Backend Docker image ready for SAE deployment
2025-12-25 08:21:21 +08:00
ef967d7d7c
build(backend): Complete Node.js backend deployment preparation
...
Major changes:
- Add Docker configuration (Dockerfile, .dockerignore)
- Fix 200+ TypeScript compilation errors
- Add Prisma schema relations for all models (30+ relations)
- Update tsconfig.json to relax non-critical checks
- Optimize Docker build with local dist strategy
Technical details:
- Exclude test files from TypeScript compilation
- Add manual relations for ASL, PKB, DC, AIA modules
- Use type assertions for JSON/Buffer compatibility
- Fix pg-boss, extractionWorker, and other legacy code issues
Build result:
- Docker image: 838MB (compressed ~186MB)
- Successfully pushed to ACR
- Zero TypeScript compilation errors
Related docs:
- Update deployment documentation
- Add Python microservice SAE deployment guide
2025-12-24 22:12:00 +08:00
b64896a307
feat(deploy): Complete PostgreSQL migration and Docker image build
...
Summary:
- PostgreSQL database migration to RDS completed (90MB SQL, 11 schemas)
- Frontend Nginx Docker image built and pushed to ACR (v1.0, ~50MB)
- Python microservice Docker image built and pushed to ACR (v1.0, 1.12GB)
- Created 3 deployment documentation files
Docker Configuration Files:
- frontend-v2/Dockerfile: Multi-stage build with nginx:alpine
- frontend-v2/.dockerignore: Optimize build context
- frontend-v2/nginx.conf: SPA routing and API proxy
- frontend-v2/docker-entrypoint.sh: Dynamic env injection
- extraction_service/Dockerfile: Multi-stage build with Aliyun Debian mirror
- extraction_service/.dockerignore: Optimize build context
- extraction_service/requirements-prod.txt: Production dependencies (removed Nougat)
Deployment Documentation:
- docs/05-部署文档/00-部署进度总览.md: One-stop deployment status overview
- docs/05-部署文档/07-前端Nginx-SAE部署操作手册.md: Frontend deployment guide
- docs/05-部署文档/08-PostgreSQL数据库部署操作手册.md: Database deployment guide
- docs/00-系统总体设计/00-系统当前状态与开发指南.md: Updated with deployment status
Database Migration:
- RDS instance: pgm-2zex1m2y3r23hdn5 (2C4G, PostgreSQL 15.0)
- Database: ai_clinical_research
- Schemas: 11 business schemas migrated successfully
- Data: 3 users, 2 projects, 1204 literatures verified
- Backup: rds_init_20251224_154529.sql (90MB)
Docker Images:
- Frontend: crpi-cd5ij4pjt65mweeo.cn-beijing.personal.cr.aliyuncs.com/ai-clinical/ai-clinical_frontend-nginx:v1.0
- Python: crpi-cd5ij4pjt65mweeo.cn-beijing.personal.cr.aliyuncs.com/ai-clinical/python-extraction:v1.0
Key Achievements:
- Resolved Docker Hub network issues (using generic tags)
- Fixed 30 TypeScript compilation errors
- Removed Nougat OCR to reduce image size by 1.5GB
- Used Aliyun Debian mirror to resolve apt-get network issues
- Implemented multi-stage builds for optimization
Next Steps:
- Deploy Python microservice to SAE
- Build Node.js backend Docker image
- Deploy Node.js backend to SAE
- Deploy frontend Nginx to SAE
- End-to-end verification testing
Status: Docker images ready, SAE deployment pending
2025-12-24 18:21:55 +08:00
4c6eaaecbf
feat(dc): Implement Postgres-Only async architecture and performance optimization
...
Summary:
- Implement async file upload processing (Platform-Only pattern)
- Add parseExcelWorker with pg-boss queue
- Implement React Query polling mechanism
- Add clean data caching (avoid duplicate parsing)
- Fix pivot single-value column tuple issue
- Optimize performance by 99 percent
Technical Details:
1. Async Architecture (Postgres-Only):
- SessionService.createSession: Fast upload + push to queue (3s)
- parseExcelWorker: Background parsing + save clean data (53s)
- SessionController.getSessionStatus: Status query API for polling
- React Query Hook: useSessionStatus (auto-serial polling)
- Frontend progress bar with real-time feedback
2. Performance Optimization:
- Clean data caching: Worker saves processed data to OSS
- getPreviewData: Read from clean data cache (0.5s vs 43s, -99 percent)
- getFullData: Read from clean data cache (0.5s vs 43s, -99 percent)
- Intelligent cleaning: Boundary detection + ghost column/row removal
- Safety valve: Max 3000 columns, 5M cells
3. Bug Fixes:
- Fix pivot column name tuple issue for single value column
- Fix queue name format (colon to underscore: asl:screening -> asl_screening)
- Fix polling storm (15+ concurrent requests -> 1 serial request)
- Fix QUEUE_TYPE environment variable (memory -> pgboss)
- Fix logger import in PgBossQueue
- Fix formatSession to return cleanDataKey
- Fix saveProcessedData to update clean data synchronously
4. Database Changes:
- ALTER TABLE dc_tool_c_sessions ADD COLUMN clean_data_key VARCHAR(1000)
- ALTER TABLE dc_tool_c_sessions ALTER COLUMN total_rows DROP NOT NULL
- ALTER TABLE dc_tool_c_sessions ALTER COLUMN total_cols DROP NOT NULL
- ALTER TABLE dc_tool_c_sessions ALTER COLUMN columns DROP NOT NULL
5. Documentation:
- Create Postgres-Only async task processing guide (588 lines)
- Update Tool C status document (Day 10 summary)
- Update DC module status document
- Update system overview document
- Update cloud-native development guide
Performance Improvements:
- Upload + preview: 96s -> 53.5s (-44 percent)
- Filter operation: 44s -> 2.5s (-94 percent)
- Pivot operation: 45s -> 2.5s (-94 percent)
- Concurrent requests: 15+ -> 1 (-93 percent)
- Complete workflow (upload + 7 ops): 404s -> 70.5s (-83 percent)
Files Changed:
- Backend: 15 files (Worker, Service, Controller, Schema, Config)
- Frontend: 4 files (Hook, Component, API)
- Docs: 4 files (Guide, Status, Overview, Spec)
- Database: 4 column modifications
- Total: ~1388 lines of new/modified code
Status: Fully tested and verified, production ready
2025-12-22 21:30:31 +08:00
9b81aef9a7
feat(dc): Add multi-metric transformation feature (direction 1+2)
...
Summary:
- Implement intelligent multi-metric grouping detection algorithm
- Add direction 1: timepoint-as-row, metric-as-column (analysis format)
- Add direction 2: timepoint-as-column, metric-as-row (display format)
- Fix column name pattern detection (FMA___ issue)
- Maintain original Record ID order in output
- Add full-select/clear buttons in UI
- Integrate into TransformDialog with Radio selection
- Update 3 documentation files
Technical Details:
- Python: detect_metric_groups(), apply_multi_metric_to_long(), apply_multi_metric_to_matrix()
- Backend: 3 new methods in QuickActionService
- Frontend: MultiMetricPanel.tsx (531 lines)
- Total: ~1460 lines of new code
Status: Fully tested and verified, ready for production
2025-12-21 15:06:15 +08:00
19f9c5ea93
docs(deployment): Fix 8 critical deployment issues and enhance documentation
...
Summary of fixes:
- Fix service discovery address (change .sae domain to internal IP)
- Unify timezone configuration (Asia/Shanghai for all services)
- Enhance ECS security group configuration (Redis/Weaviate port binding)
- Add image pull strategy best practices
- Add Python service memory management guidelines
- Update Dify API Key deployment strategy (avoid deadlock)
- Add SSH tunnel for RDS database access
- Add NAT gateway cost optimization explanation
Modified files (7 docs):
- 00-部署架构总览.md (enhanced with 7 sections)
- 03-Dify-ECS部署完全指南.md (security hardening)
- 04-Python微服务-SAE容器部署指南.md (timezone + service discovery)
- 05-Node.js后端-SAE容器部署指南.md (timezone configuration)
- PostgreSQL部署策略-摸底报告.md (timezone best practice)
- 07-关键配置补充说明.md (3 new sections)
- 08-部署检查清单.md (service address fix)
New files:
- 文档修正报告-20251214.md (comprehensive fix report)
- Review documents from technical team
Impact:
- Fixed 3 P0/P1 critical issues (100% connection failure risk)
- Fixed 3 P2 important issues (stability and maintainability)
- Added 2 P3 best practices (developer convenience)
Status: All deployment documents reviewed and corrected, ready for production deployment
2025-12-14 13:25:28 +08:00
