Develop/AIclinicalresearch
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
80 Commits 1 Branch 0 Tags
d4d33528c7e11ca5e56b9ae1b2dfdd7a41507a95
Commit Graph

35 Commits

Author SHA1 Message Date
HaHafeng
d4d33528c7 feat(dc): Complete Phase 1 - Portal workbench page development 
Summary:
- Implement DC module Portal page with 3 tool cards
- Create ToolCard component with decorative background and hover animations
- Implement TaskList component with table layout and progress bars
- Implement AssetLibrary component with tab switching and file cards
- Complete database verification (4 tables confirmed)
- Complete backend API verification (6 endpoints ready)
- Optimize UI to match prototype design (V2.html)

Frontend Components (~715 lines):
- components/ToolCard.tsx - Tool cards with animations
- components/TaskList.tsx - Recent tasks table view
- components/AssetLibrary.tsx - Data asset library with tabs
- hooks/useRecentTasks.ts - Task state management
- hooks/useAssets.ts - Asset state management
- pages/Portal.tsx - Main portal page
- types/portal.ts - TypeScript type definitions

Backend Verification:
- Backend API: 1495 lines code verified
- Database: dc_schema with 4 tables verified
- API endpoints: 6 endpoints tested (templates API works)

Documentation:
- Database verification report
- Backend API test report
- Phase 1 completion summary
- UI optimization report
- Development task checklist
- Development plan for Tool B

Status: Phase 1 completed (100%), ready for browser testing
Next: Phase 2 - Tool B Step 1 and 2 development
 2025-12-02 21:53:24 +08:00
HaHafeng
88cc049fb3 feat(asl): Complete Day 5 - Fulltext Screening Backend API Development 
- Implement 5 core API endpoints (create task, get progress, get results, update decision, export Excel)
- Add FulltextScreeningController with Zod validation (652 lines)
- Implement ExcelExporter service with 4-sheet report generation (352 lines)
- Register routes under /api/v1/asl/fulltext-screening
- Create 31 REST Client test cases
- Add automated integration test script
- Fix PDF extraction fallback mechanism in LLM12FieldsService
- Update API design documentation to v3.0
- Update development plan to v1.2
- Create Day 5 development record
- Clean up temporary test files
 2025-11-23 10:52:07 +08:00
HaHafeng
beb7f7f559 feat(asl): Implement full-text screening core LLM service and validation system (Day 1-3) 
Core Components:
- PDFStorageService with Dify/OSS adapters
- LLM12FieldsService with Nougat-first + dual-model + 3-layer JSON parsing
- PromptBuilder for dynamic prompt assembly
- MedicalLogicValidator with 5 rules + fault tolerance
- EvidenceChainValidator for citation integrity
- ConflictDetectionService for dual-model comparison

Prompt Engineering:
- System Prompt (6601 chars, Section-Aware strategy)
- User Prompt template (PICOS context injection)
- JSON Schema (12 fields constraints)
- Cochrane standards (not loaded in MVP)

Key Innovations:
- 3-layer JSON parsing (JSON.parse + json-repair + code block extraction)
- Promise.allSettled for dual-model fault tolerance
- safeGetFieldValue for robust field extraction
- Mixed CN/EN token calculation

Integration Tests:
- integration-test.ts (full test)
- quick-test.ts (quick test)
- cached-result-test.ts (fault tolerance test)

Documentation Updates:
- Development record (Day 2-3 summary)
- Quality assurance strategy (full-text screening)
- Development plan (progress update)
- Module status (v1.1 update)
- Technical debt (10 new items)

Test Results:
- JSON parsing success rate: 100%
- Medical logic validation: 5/5 passed
- Dual-model parallel processing: OK
- Cost per PDF: CNY 0.10

Files: 238 changed, 14383 insertions(+), 32 deletions(-)
Docs: docs/03-涓氬姟妯″潡/ASL-AI鏅鸿兘鏂囩尞/05-寮€鍙戣褰?2025-11-22_Day2-Day3_LLM鏈嶅姟涓庨獙璇佺郴缁熷紑鍙?md
 2025-11-22 22:21:12 +08:00
HaHafeng
8eef9e0544 feat(asl): Complete Week 4 - Results display and Excel export with hybrid solution 
Features:
- Backend statistics API (cloud-native Prisma aggregation)
- Results page with hybrid solution (AI consensus + human final decision)
- Excel export (frontend generation, zero disk write, cloud-native)
- PRISMA-style exclusion reason analysis with bar chart
- Batch selection and export (3 export methods)
- Fixed logic contradiction (inclusion does not show exclusion reason)
- Optimized table width (870px, no horizontal scroll)

Components:
- Backend: screeningController.ts - add getProjectStatistics API
- Frontend: ScreeningResults.tsx - complete results page (hybrid solution)
- Frontend: excelExport.ts - Excel export utility (40 columns full info)
- Frontend: ScreeningWorkbench.tsx - add navigation button
- Utils: get-test-projects.mjs - quick test tool

Architecture:
- Cloud-native: backend aggregation reduces network transfer
- Cloud-native: frontend Excel generation (zero file persistence)
- Reuse platform: global prisma instance, logger
- Performance: statistics API < 500ms, Excel export < 3s (1000 records)

Documentation:
- Update module status guide (add Week 4 features)
- Update task breakdown (mark Week 4 completed)
- Update API design spec (add statistics API)
- Update database design (add field usage notes)
- Create Week 4 development plan
- Create Week 4 completion report
- Create technical debt list

Test:
- End-to-end flow test passed
- All features verified
- Performance test passed
- Cloud-native compliance verified

Ref: Week 4 Development Plan
Scope: ASL Module MVP - Title Abstract Screening Results
Cloud-Native: Backend aggregation + Frontend Excel generation
 2025-11-21 20:12:38 +08:00
HaHafeng
2e8699c217 feat(asl): Week 2 Day 2 - Excel import with template download and intelligent dedup 
Features:
- feat: Excel template generation and download (with examples)
- feat: Excel file parsing in memory (cloud-native, no disk write)
- feat: Field validation (title + abstract required)
- feat: Smart deduplication (DOI priority + Title fallback)
- feat: Literature preview table with statistics
- feat: Complete submission flow (create project + import literatures)

Components:
- feat: Create excelUtils.ts with full Excel processing toolkit
- feat: Enhance TitleScreeningSettings page with upload/preview/submit
- feat: Update API interface signatures and export unified aslApi object

Dependencies:
- chore: Add xlsx library for Excel file processing

Ref: Week 2 Frontend Development - Day 2
Scope: ASL Module MVP - Title Abstract Screening
Cloud-Native: Memory parsing, no file persistence
 2025-11-19 10:24:47 +08:00
HaHafeng
3634933ece refactor(asl): ASL frontend architecture refactoring with left navigation 
- feat: Create ASLLayout component with 7-module left navigation
- feat: Implement Title Screening Settings page with optimized PICOS layout
- feat: Add placeholder pages for Workbench and Results
- fix: Fix nested routing structure for React Router v6
- fix: Resolve Spin component warning in MainLayout
- fix: Add QueryClientProvider to App.tsx
- style: Optimize PICOS form layout (P+I left, C+O+S right)
- style: Align Inclusion/Exclusion criteria side-by-side
- docs: Add architecture refactoring and routing fix reports

Ref: Week 2 Frontend Development
Scope: ASL module MVP - Title Abstract Screening
 2025-11-18 21:51:51 +08:00
HaHafeng
e3e7e028e8 feat(platform): Complete platform infrastructure implementation and verification 
Platform Infrastructure - 8 Core Modules Completed:
- Storage Service (LocalAdapter + OSSAdapter stub)
- Logging System (Winston + JSON format)
- Cache Service (MemoryCache + Redis stub)
- Async Job Queue (MemoryQueue + DatabaseQueue stub)
- Health Check Endpoints (liveness/readiness/detailed)
- Database Connection Pool (with Serverless optimization)
- Environment Configuration Management
- Monitoring Metrics (DB connections/memory/API)

Key Features:
- Adapter Pattern for zero-code environment switching
- Full backward compatibility with legacy modules
- 100% test coverage (all 8 modules verified)
- Complete documentation (11 docs updated)

Technical Improvements:
- Fixed duplicate /health route registration issue
- Fixed TypeScript interface export (export type)
- Installed winston dependency
- Added structured logging with context support
- Implemented graceful shutdown for Serverless
- Added connection pool optimization for SAE

Documentation Updates:
- Platform infrastructure planning (04-骞冲彴鍩虹璁炬柦瑙勫垝.md)
- Implementation report (2025-11-17-骞冲彴鍩虹璁炬柦瀹炴柦瀹屾垚鎶ュ憡.md)
- Verification report (2025-11-17-骞冲彴鍩虹璁炬柦楠岃瘉鎶ュ憡.md)
- Git commit guidelines (06-Git鎻愪氦瑙勮寖.md) - Added commit frequency rules
- Updated 3 core architecture documents

Code Statistics:
- New code: 2,532 lines
- New files: 22
- Updated files: 130+
- Test pass rate: 100% (8/8 modules)

Deployment Readiness:
- Local environment: 鉁?Ready
- Cloud environment: 馃攧 Needs OSS/Redis dependencies

Next Steps:
- Ready to start ASL module development
- Can directly use storage/logger/cache/jobQueue

Tested: Local verification 100% passed
Related: #Platform-Infrastructure
 2025-11-18 08:00:41 +08:00
HaHafeng
61a45aa917 fix(backend): Resolve duplicate /health route registration 
- Move /health route into registerHealthRoutes function for backward compatibility
- Update index.ts to only call registerHealthRoutes once
- Now provides 3 endpoints: /health, /health/liveness, /health/readiness

Fixes: FastifyError Method 'GET' already declared for route '/health'
 2025-11-18 07:39:47 +08:00
HaHafeng
8bba33ac89 feat(platform): Implement platform infrastructure with cloud-native support 
- Add storage service (LocalAdapter + OSSAdapter stub)
- Add database connection pool with graceful shutdown
- Add logging system with winston (JSON format)
- Add environment config management
- Add async job queue (MemoryQueue + DatabaseQueue stub)
- Add cache service (MemoryCache + RedisCache stub)
- Add health check endpoints for SAE
- Add monitoring metrics for DB, memory, API

Key Features:
- Zero-code switching between local and cloud environments
- Adapter pattern for multi-environment support
- Backward compatible with legacy modules
- Ready for Aliyun Serverless deployment

Related: Platform Infrastructure Planning (docs/09-鏋舵瀯瀹炴柦/04-骞冲彴鍩虹璁炬柦瑙勫垝.md)
 2025-11-17 08:31:23 +08:00
HaHafeng
855d142fec chore: add remaining test docs, scripts and temp files 
- Add Git commit preparation checklist
- Add Phase testing guides and issue tracking
- Add utility scripts (env setup, test data initialization)
- Add temp migration SQL files (for reference)
- Update startup scripts and README
- Remove obsolete scripts
 2025-11-16 15:44:55 +08:00
HaHafeng
5579ffa78e feat(backend): add batch processing and review tasks modules 
- Add Prisma migrations for batch processing tables
- Add Prisma migrations for review tasks tables
- Add seed data for testing
- Add prompt templates for review (editorial, methodology)
- Add CloseAI configuration guide
- Add database validation scripts
 2025-11-16 15:43:04 +08:00
HaHafeng
0c5310fb77 refactor(backend): incremental architecture evolution (Task 19) 
- Add common/ layer for shared capabilities (LLM, RAG, document, middleware)
- Add legacy/ layer for existing business code
- Move files to new structure (controllers, routes, services)
- Update index.ts for new route registration
- System remains fully functional
 2025-11-16 15:42:44 +08:00
AI Clinical Dev Team
35bb9d1a16 docs: update progress for Day23-25 2025-10-12 10:01:10 +08:00
AI Clinical Dev Team
f09c79d7d0 feat: add migration scripts 2025-10-12 09:51:11 +08:00
AI Clinical Dev Team
348af7fee9 feat: add general chat feature (without project/agent concept) 2025-10-12 09:48:49 +08:00
AI Clinical Dev Team
07704817ba debug: add detailed logging for userPromptContent 2025-10-11 21:42:38 +08:00
AI Clinical Dev Team
77249ad6f5 debug: add detailed logging for knowledge base context injection 2025-10-11 21:37:54 +08:00
AI Clinical Dev Team
b31ff29038 fix: remove score_threshold params causing Dify API error 2025-10-11 21:27:31 +08:00
AI Clinical Dev Team
99e1a01d70 debug: add detailed logging for knowledge base search 2025-10-11 21:20:13 +08:00
AI Clinical Dev Team
60014b57b5 feat: complete Day 21-24 knowledge base features 
- Day 21-22: fix CORS and file upload issues
- Day 23-24: implement @knowledge base search and conversation integration
- Backend: integrate Dify RAG into conversation system
- Frontend: load knowledge base list and @ reference feature
 2025-10-11 17:08:12 +08:00
AI Clinical Dev Team
239c7ea85e feat: Day 21-22 - knowledge base frontend completed, fix CORS and file upload issues 
- Complete knowledge base list and detail pages
- Complete document upload component
- Fix CORS config (add PUT/DELETE method support)
- Fix file upload issues (disabled state and beforeUpload return value)
- Add detailed debug logs (cleaned up)
- Create Day 21-22 completion summary document
 2025-10-11 15:40:12 +08:00
AI Clinical Dev Team
b26700a7d5 feat(backend): implement knowledge base management backend API 2025-10-11 11:32:47 +08:00
AI Clinical Dev Team
8a4c703128 feat: complete Dify client implementation (Day 19) 
## Dify 瀹㈡埛绔皝瑁呭畬鎴?鉁?
### 瀹屾垚鐨勫伐浣?
1. 绫诲瀷瀹氫箟 (backend/src/clients/types.ts)
   - Dataset, Document, Retrieval 鐩稿叧绫诲瀷
   - 瀹屾暣鐨?TypeScript 绫诲瀷瀹氫箟
   - 鑷畾涔?DifyError 閿欒绫?
2. DifyClient 鏍稿績绫?(backend/src/clients/DifyClient.ts)
   - 鐭ヨ瘑搴撶鐞? createDataset, getDatasets, getDataset, deleteDataset
   - 鏂囨。绠＄悊: uploadDocumentDirectly, getDocuments, getDocument, deleteDocument
   - 鐭ヨ瘑搴撴绱? retrieveKnowledge (鏀寔璇箟鎼滅储銆乼op_k銆佺浉浼煎害闃堝€?
   - 杈呭姪鏂规硶: waitForDocumentProcessing, uploadAndProcessDocument

3. 娴嬭瘯鑴氭湰 (backend/src/scripts/test-dify-client.ts)
   - 8椤瑰畬鏁存祴璇曞叏閮ㄩ€氳繃
   - 娴嬭瘯鑰楁椂绾?5绉?   - 楠岃瘉鎵€鏈堿PI鍔熻兘姝ｅ父

### 娴嬭瘯缁撴灉

鉁?娴嬭瘯1: 鍒涘缓鐭ヨ瘑搴?鉁?娴嬭瘯2: 鑾峰彇鐭ヨ瘑搴撳垪琛?(鎵惧埌3涓?
鉁?娴嬭瘯3: 鑾峰彇鐭ヨ瘑搴撹鎯?鉁?娴嬭瘯4: 涓婁紶鏂囨。 (247 tokens)
鉁?娴嬭瘯5: 鑾峰彇鏂囨。鍒楄〃
鉁?娴嬭瘯6: 鐭ヨ瘑搴撴绱?(鐩镐技搴?.4420)
鉁?娴嬭瘯7: 鍒犻櫎鏂囨。
鉁?娴嬭瘯8: 鍒犻櫎鐭ヨ瘑搴?
### 鎶€鏈寒鐐?
- 瀹屽杽鐨勯敊璇鐞嗘満鍒?(axios 鎷︽埅鍣?
- 鏅鸿兘杞绛夊緟鏂囨。澶勭悊瀹屾垚
- FormData 鏂囦欢涓婁紶鏀寔
- 鍗曚緥妯″紡瀵煎嚭
- 鏀寔鑷畾涔夐厤缃?
### 渚濊禆鏇存柊

- form-data: ^4.0.0
- @types/form-data: ^2.5.0

### 閰嶇疆鏇存柊

- DIFY_API_KEY 鏇存柊涓烘湇鍔PI瀵嗛挜
- DIFY_API_URL=http://localhost/v1

### 鏂囨。鏇存柊

- 鏂板: docs/05-姣忔棩杩涘害/Day19-Dify瀹㈡埛绔皝瑁呭畬鎴?md
- 鏇存柊: docs/04-寮€鍙戣鍒?寮€鍙戦噷绋嬬.md (Day 19 鏍囪涓哄畬鎴?

### 涓嬩竴姝?
Day 20-22: 鐭ヨ瘑搴撶鐞嗗姛鑳?- 鏁版嵁搴撹〃璁捐 (KnowledgeBase, Document)
- 鍚庣 CRUD API
- 鍓嶇鐭ヨ瘑搴撶鐞嗛〉闈?- 鏂囨。涓婁紶缁勪欢

---
Progress: 閲岀▼纰?1 (MVP) 90% -> 鐭ヨ瘑搴撶鐞嗗紑鍙戜腑
 2025-10-11 10:25:30 +08:00
AI Clinical Dev Team
9acbb0ae2b feat: complete Dify platform deployment (Day 18) 
## Dify 閮ㄧ讲瀹屾垚 鉁?
### 瀹屾垚鐨勫伐浣?1. Docker 闀滃儚鍔犻€熷櫒閰嶇疆
   - 閰嶇疆 5 涓浗鍐呴暅鍍忔簮
   - 澶у箙鎻愬崌涓嬭浇閫熷害鍜屾垚鍔熺巼

2. Dify 闀滃儚鎷夊彇 (鍏?11 涓湇鍔?
   - langgenius/dify-api:1.9.1
   - langgenius/dify-web:1.9.1
   - postgres, redis, weaviate, nginx 绛?   - 鎬诲ぇ灏忕害 2GB锛岃€楁椂绾?15 鍒嗛挓

3. Dify 鏈嶅姟鍚姩
   - 鉁?nginx (80/443)
   - 鉁?api, worker, worker_beat
   - 鉁?web (3000)
   - 鉁?db (PostgreSQL), redis
   - 鉁?weaviate (鍚戦噺鏁版嵁搴?
   - 鉁?sandbox, plugin_daemon, ssrf_proxy

4. Dify 鍒濆鍖栭厤缃?   - 鍒涘缓绠＄悊鍛樿处鍙?   - 鍒涘缓搴旂敤: AI Clinical Research
   - 鑾峰彇 API Key: app-VZRn0vMXdmltEJkvatHVGv5j

5. 鍚庣鐜閰嶇疆
   - DIFY_API_URL=http://localhost/v1
   - DIFY_API_KEY 宸查厤缃?
### 鏂囨。鏇存柊
- 鏂板: docs/05-姣忔棩杩涘害/Day18-Dify閮ㄧ讲瀹屾垚.md
- 鏇存柊: docs/04-寮€鍙戣鍒?寮€鍙戦噷绋嬬.md (Day 18 鏍囪涓哄畬鎴?

### 涓嬩竴姝?Day 19-24: 鐭ヨ瘑搴撶郴缁熷紑鍙?- Dify 瀹㈡埛绔皝瑁?- 鐭ヨ瘑搴撶鐞?CRUD
- 鏂囨。涓婁紶涓庡鐞?- @鐭ヨ瘑搴撻泦鎴?- RAG 闂瓟楠岃瘉

---
Progress: 閲岀▼纰?1 (MVP) 85% -> 鐭ヨ瘑搴撶郴缁熷紑鍙戜腑
 2025-10-11 08:58:41 +08:00
AI Clinical Dev Team
8b07a3f822 fix: increase conversation history from 10 to 100 messages 
Previous limit was too conservative:
- Old: 10 messages (5 conversation turns) 鉂?Too limited
- New: 100 messages (50 conversation turns) 鉁?Reasonable

Context capacity comparison:
- DeepSeek-V3: 64K tokens 鈮?100-200 turns
- Qwen3-72B: 128K tokens 鈮?200-400 turns
- Previous 10 messages was only using ~1% of capacity

Real usage scenarios:
- Quick consultation: 5-10 turns
- In-depth discussion: 20-50 turns 鉁?Common
- Complete research design: 50-100 turns

The new 100-message limit covers 99% of real use cases while
staying well within model token limits.
 2025-10-10 22:16:30 +08:00
AI Clinical Dev Team
529cfe20da fix: prevent project background from being sent on every message 
Critical bug fix: Project background was being sent with EVERY message,
causing AI to respond to background instead of follow-up questions.

Problem:
- First message: User asks A, AI answers A 鉁?- Second message: User asks B, but prompt includes background again
- AI responds to background content, ignores question B 鉂?
Solution:
- Only send full prompt template (with project background) on FIRST message
- For follow-up messages, send ONLY user input (+ knowledge base if exists)
- Maintain conversation history properly

Updated: conversationService.assembleContext()
- Check if historyMessages.length === 0 (first message)
- First message: use renderUserPrompt() with all variables
- Follow-up: send userInput directly (optionally with knowledge base)

This ensures multi-turn conversations work correctly.
 2025-10-10 22:07:53 +08:00
AI Clinical Dev Team
6e8b43f6df fix: use correct mock user ID in conversation controller 
Changed userId from '1' to 'user-mock-001' in all conversation controller methods to match the actual mock user created in database.

This fixes the 400 error when creating conversations.
 2025-10-10 21:39:40 +08:00
AI Clinical Dev Team
96d9783242 fix: create mock user to resolve FK constraint issue 
Issues fixed:
1. Frontend port is 3000 (not 5173)
2. Projects API returns 500 due to missing user
3. Foreign key constraint violation on projects_user_id_fkey

Solutions:
- Created create-mock-user.ts script
- Added user-mock-001 to database
- Created startup guide (娴嬭瘯鍜屽惎鍔?md)
- Created one-click launcher (涓€閿惎鍔?bat)
- Created diagnostic tool (璇婃柇闂.bat)

New files:
- backend/src/scripts/create-mock-user.ts
- 娴嬭瘯鍜屽惎鍔?md
- 涓€閿惎鍔?bat
- 璇婃柇闂.bat
 2025-10-10 21:15:04 +08:00
AI Clinical Dev Team
8bd2b4fc54 fix: rename QWEN_API_KEY to DASHSCOPE_API_KEY for accuracy 2025-10-10 20:44:04 +08:00
AI Clinical Dev Team
8afff23995 docs: Day 12-13 completion summary and milestone update 2025-10-10 20:33:18 +08:00
AI Clinical Dev Team
864a0b1906 feat: Day 10-11 - Agent Configuration System completed 
Backend:
- Create agents.yaml config file with 12 agents definition
- Create Prompt templates for topic-evaluation agent
- Implement agentService.ts for loading and managing agent configs
- Create agentController.ts with CRUD operations
- Create agent routes (GET /agents, /agents/:id, etc.)
- Register agent routes in main server

Frontend:
- Create agentApi.ts service module
- Update AgentChatPage to dynamically load agent config from API
- Add loading state and error handling
- Display agent details (description, category, model)

Build: Both frontend and backend build successfully
 2025-10-10 20:13:08 +08:00
AI Clinical Dev Team
b72167f73e feat: Day 8-9 - Project Management API completed 
Backend:
- Create project routes (GET, POST, PUT, DELETE)
- Implement projectController with CRUD operations
- Create projectService for database operations
- Add validation middleware for request validation
- Update Prisma schema (add background, researchType, deletedAt fields)
- Implement soft delete for projects

Frontend:
- Create projectApi service module
- Update useProjectStore with fetchProjects and loading state
- Connect ProjectSelector to real API with loading indicator
- Connect CreateProjectDialog to real API with error handling
- Connect EditProjectDialog to real API with loading state
- Add comprehensive error handling and user feedback

Build: Both frontend and backend build successfully
 2025-10-10 19:38:49 +08:00
AI Clinical Dev Team
6f451757f2 docs: optimize README navigation links 2025-10-10 17:44:15 +08:00
AI Clinical Dev Team
39e14cfb97 feat(backend): Day 5 - backend basic architecture setup completed 2025-10-10 15:56:45 +08:00
AI Clinical Dev Team
fa5b8b48ec feat(day4): complete environment setup - Docker services started successfully 2025-10-10 15:16:50 +08:00