205932bb3f
feat(asl): Complete Tool 4 SR Chart Generator and Tool 5 Meta Analysis Engine
...
Tool 4 - SR Chart Generator:
- PRISMA 2020 flow diagram with Chinese/English toggle (SVG)
- Baseline characteristics table (Table 1)
- Dual data source: project pipeline API + Excel upload
- SVG/PNG export support
- Backend: ChartingService with Prisma aggregation
- Frontend: PrismaFlowDiagram, BaselineTable, DataSourceSelector
Tool 5 - Meta Analysis Engine:
- 3 data types: HR (metagen), dichotomous (metabin), continuous (metacont)
- Random and fixed effects models
- Multiple effect measures: HR / OR / RR
- Forest plot + funnel plot (base64 PNG from R)
- Heterogeneity statistics: I2, Q, p-value, Tau2
- Data input via Excel upload or project pipeline
- R Docker image updated with meta package (13 tools total)
- E2E test: 36/36 passed
- Key fix: exp() back-transformation for log-scale ratio measures
Also includes:
- IIT CRA Agent V3.0 routing and AI chat page integration
- Updated ASL module status guide (v2.3)
- Updated system status guide (v6.3)
- Updated R statistics engine guide (v1.4)
Tested: Frontend renders correctly, backend APIs functional, E2E tests passed
Made-with: Cursor
2026-02-26 21:51:02 +08:00
f0736dbca1
feat(asl/extraction): Complete Tool 3 M1+M2 - skeleton pipeline and HITL workbench
...
M1 Skeleton Pipeline:
- Scatter-dispatch + Aggregator polling pattern (PgBoss)
- PKB ACL bridge (PkbBridgeService -> PkbExportService DTOs)
- ExtractionSingleWorker with DeepSeek-V3 LLM extraction
- PermanentExtractionError for non-retryable failures
- Phantom Retry Guard (idempotent worker)
- 3-step minimal frontend (Setup -> Progress -> Workbench)
- 4 new DB tables (extraction_templates, project_templates, tasks, results)
- 3 system templates seed (RCT, Cohort, QC)
- M1 integration test suite
M2 HITL Workbench:
- MinerU VLM integration for high-fidelity table extraction
- XML-isolated DynamicPromptBuilder with flat JSON output template
- fuzzyQuoteMatch validator (3-tier confidence scoring)
- SSE real-time logging via ExtractionEventBus
- Schema-driven ExtractionDrawer (dynamic field rendering from template)
- Excel wide-table export with flattenModuleData normalization
- M2 integration test suite
Critical Fixes (data normalization):
- DynamicPromptBuilder: explicit flat key-value output format with example
- ExtractionExcelExporter: handle both array and flat data formats
- ExtractionDrawer: schema-driven rendering instead of hardcoded fields
- ExtractionValidator: array-format quote verification support
- SSE route: Fastify register encapsulation to bypass auth for EventSource
- LLM JSON sanitizer: strip illegal control chars before JSON.parse
Also includes: RVW stats verification spec, SSA expert config guide
Tested: M1 pipeline test + M2 HITL test + manual frontend verification
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-02-25 18:29:20 +08:00
371fa53956
docs(asl): Upgrade Tool 3 architecture from Fan-out to Scatter+Aggregator (v2.0)
...
Architecture transformation:
- Replace Fan-out (Manager->Child->Last Child Wins) with Scatter+Aggregator pattern
- API layer directly dispatches N independent jobs (no Manager)
- Worker only writes its own Result row, never touches Task table (zero row-lock)
- Aggregator polls groupBy for completion + zombie cleanup (replaces Sweeper)
- Reduce red lines from 13 to 9, eliminate distributed complexity
Documents updated (10 files):
- 08-Tool3 main architecture doc: v2.0 rewrite (schema, Task 2.3/2.4, red lines, risks)
- 08d-Code patterns: rewrite sections 4.1-4.6 (API dispatch, SingleWorker, Aggregator)
- 08a-M1 sprint: rewrite M1-3 core (Worker+Aggregator), red lines, acceptance criteria
- 08b-M2 sprint: simplify SSE (NOTIFY/LISTEN downgraded to P2 optional)
- 08c-M3 sprint: milestone table wording update
- New: Scatter+Polling Aggregator pattern guide v1.1 (Level 2 cookbook)
- New: V2.0 architecture deep review and gap-fix report
- Updated: ASL module status, system status, capability layer index
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-02-24 22:11:09 +08:00
dc6b292308
docs(asl): Complete Tool 3 extraction workbench V2.0 development plan (v1.5)
...
ASL Tool 3 Development Plan:
- Architecture blueprint v1.5 (6 rounds of architecture review, 13 red lines)
- M1/M2/M3 sprint checklists (Skeleton Pipeline / HITL Workbench / Dynamic Template Engine)
- Code patterns cookbook (9 chapters: Fan-out, Prompt engineering, ACL, SSE dual-track, etc.)
- Key patterns: Fan-out with Last Child Wins, Optimistic Locking, teamConcurrency throttling
- PKB ACL integration (anti-corruption layer), MinerU Cache-Aside, NOTIFY/LISTEN cross-pod SSE
- Data consistency snapshot for long-running extraction tasks
Platform capability:
- Add distributed Fan-out task pattern development guide (7 patterns + 10 anti-patterns)
- Add system-level async architecture risk analysis blueprint
- Add PDF table extraction engine design and usage guide (MinerU integration)
- Add table extraction source code (TableExtractionManager + MinerU engine)
Documentation updates:
- Update ASL module status with Tool 3 V2.0 plan readiness
- Update system status document (v6.2) with latest milestones
- Add V2.0 product requirements, prototypes, and data dictionary specs
- Add architecture review documents (4 rounds of review feedback)
- Add test PDF files for extraction validation
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-02-23 22:49:16 +08:00
8f06d4f929
feat(asl): Complete Deep Research V2.0 core development
...
Backend:
- Add SSE streaming client (unifuncsSseClient) replacing async polling
- Add paragraph-based reasoning parser with mergeConsecutiveThinking
- Add requirement expansion service (DeepSeek-V3 PICOS+MeSH)
- Add Word export service with Pandoc, inline hyperlinks, reference link expansion
- Add deep research V2 worker with 2s log flush and Chinese source prompt
- Add 5 curated data sources config (PubMed/ClinicalTrials/Cochrane/CNKI/MedJournals)
- Add 4 API endpoints (generate-requirement/tasks/task-status/export-word)
- Update Prisma schema with 6 new V2.0 fields on AslResearchTask
- Add DB migration for V2.0 fields
- Simplify ASL_DEEP_RESEARCH_EXPANSION prompt (remove strategy section)
Frontend:
- Add waterfall-flow DeepResearchPage (phase 0-4 progressive reveal)
- Add LandingView, SetupPanel, StrategyConfirm, AgentTerminal, ResultsView
- Add react-markdown + remark-gfm for report rendering
- Add custom link component showing visible URLs after references
- Add useDeepResearchTask polling hook
- Add deep research TypeScript types
Tests:
- Add E2E test, smoke test, and Chinese data source test scripts
Docs:
- Update ASL module status (v2.0 - core features complete)
- Update system status (v6.1 - ASL V2.0 milestone)
- Update Unifuncs DeepSearch API guide (v2.0 - SSE mode + Chinese source results)
- Update module auth specification (test script guidelines)
- Update V2.0 development plan
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-02-23 13:21:52 +08:00
b06daecacd
feat(asl): Add Deep Research V2.0 development plan and Unifuncs API site coverage testing
...
Completed:
- Unifuncs DeepSearch API site coverage test (18 medical sites, 9 tier-1 available)
- ClinicalTrials.gov dedicated test (4 strategies, English query + depth>=10 works best)
- Deep Research V2.0 development plan (5-day phased delivery)
- DeepResearch engine capability guide (docs/02-common-capability/)
- Test scripts: test-unifuncs-site-coverage.ts, test-unifuncs-clinicaltrials.ts
Key findings:
- Tier-1 sites: PubMed(28), ClinicalTrials(38), NCBI(18), Scholar(10), Cochrane(4), CNKI(7), SinoMed(9), GeenMedical(5), VIP(1)
- Paid databases (WoS/Embase/Scopus/Ovid) cannot be accessed (no credential support)
- ClinicalTrials.gov requires English queries with max_depth>=10
Updated: ASL module status doc, system status doc, common capability list
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-02-22 22:44:41 +08:00
1ece9a4ae8
feat(asl): Add DeepSearch smart literature retrieval MVP
...
Features:
- Integrate unifuncs DeepSearch API (OpenAI compatible protocol)
- SSE real-time streaming for AI thinking process display
- Natural language input, auto-generate PubMed search strategy
- Extract and display PubMed literature links
- Database storage for task records (asl_research_tasks)
Backend:
- researchService.ts - Core business logic with SSE streaming
- researchController.ts - SSE stream endpoint
- researchWorker.ts - Async task worker (backup mode)
- schema.prisma - AslResearchTask model
Frontend:
- ResearchSearch.tsx - Search page with unified content stream
- ResearchSearch.css - Styling (unifuncs-inspired simple design)
- ASLLayout.tsx - Enable menu item
- api/index.ts - Add research API functions
API Endpoints:
- POST /api/v1/asl/research/stream - SSE streaming search
- POST /api/v1/asl/research/tasks - Async task creation
- GET /api/v1/asl/research/tasks/:taskId/status - Task status
Documentation:
- Development record for DeepSearch integration
- Update ASL module status (v1.5)
- Update system status (v3.7)
Known limitations:
- SSE mode, task interrupts when leaving page
- Cost ~0.3 RMB per search (unifuncs API)
2026-01-18 19:15:55 +08:00
66255368b7
feat(admin): Add user management and upgrade to module permission system
...
Features - User Management (Phase 4.1):
- Database: Add user_modules table for fine-grained module permissions
- Database: Add 4 user permissions (view/create/edit/delete) to role_permissions
- Backend: UserService (780 lines) - CRUD with tenant isolation
- Backend: UserController + UserRoutes (648 lines) - 13 API endpoints
- Backend: Batch import users from Excel
- Frontend: UserListPage (412 lines) - list/filter/search/pagination
- Frontend: UserFormPage (341 lines) - create/edit with module config
- Frontend: UserDetailPage (393 lines) - details/tenant/module management
- Frontend: 3 modal components (592 lines) - import/assign/configure
- API: GET/POST/PUT/DELETE /api/admin/users/* endpoints
Architecture Upgrade - Module Permission System:
- Backend: Add getUserModules() method in auth.service
- Backend: Login API returns modules array in user object
- Frontend: AuthContext adds hasModule() method
- Frontend: Navigation filters modules based on user.modules
- Frontend: RouteGuard checks requiredModule instead of requiredVersion
- Frontend: Remove deprecated version-based permission system
- UX: Only show accessible modules in navigation (clean UI)
- UX: Smart redirect after login (avoid 403 for regular users)
Fixes:
- Fix UTF-8 encoding corruption in ~100 docs files
- Fix pageSize type conversion in userService (String to Number)
- Fix authUser undefined error in TopNavigation
- Fix login redirect logic with role-based access check
- Update Git commit guidelines v1.2 with UTF-8 safety rules
Database Changes:
- CREATE TABLE user_modules (user_id, tenant_id, module_code, is_enabled)
- ADD UNIQUE CONSTRAINT (user_id, tenant_id, module_code)
- INSERT 4 permissions + role assignments
- UPDATE PUBLIC tenant with 8 module subscriptions
Technical:
- Backend: 5 new files (~2400 lines)
- Frontend: 10 new files (~2500 lines)
- Docs: 1 development record + 2 status updates + 1 guideline update
- Total: ~4900 lines of code
Status: User management 100% complete, module permission system operational
2026-01-16 13:42:10 +08:00
1b53ab9d52
feat(aia): Complete AIA V2.0 with universal streaming capabilities
...
Major Changes:
- Add StreamingService with OpenAI Compatible format
- Upgrade Chat component V2 with Ant Design X integration
- Implement AIA module with 12 intelligent agents
- Update API routes to unified /api/v1 prefix
- Update system documentation
Backend (~1300 lines):
- common/streaming: OpenAI Compatible adapter
- modules/aia: 12 agents, conversation service, streaming integration
- Update route versions (RVW, PKB to v1)
Frontend (~3500 lines):
- modules/aia: AgentHub + ChatWorkspace (100% prototype restoration)
- shared/Chat: AIStreamChat, ThinkingBlock, useAIStream Hook
- Update API endpoints to v1
Documentation:
- AIA module status guide
- Universal capabilities catalog
- System overview updates
- All module documentation sync
Tested: Stream response verified, authentication working
Status: AIA V2.0 core completed (85%)
2026-01-14 19:15:01 +08:00
fa72beea6c
feat(platform): Complete Postgres-Only architecture refactoring (Phase 1-7)
...
Major Changes:
- Implement Platform-Only architecture pattern (unified task management)
- Add PostgresCacheAdapter for unified caching (platform_schema.app_cache)
- Add PgBossQueue for job queue management (platform_schema.job)
- Implement CheckpointService using job.data (generic for all modules)
- Add intelligent threshold-based dual-mode processing (THRESHOLD=50)
- Add task splitting mechanism (auto chunk size recommendation)
- Refactor ASL screening service with smart mode selection
- Refactor DC extraction service with smart mode selection
- Register workers for ASL and DC modules
Technical Highlights:
- All task management data stored in platform_schema.job.data (JSONB)
- Business tables remain clean (no task management fields)
- CheckpointService is generic (shared by all modules)
- Zero code duplication (DRY principle)
- Follows 3-layer architecture principle
- Zero additional cost (no Redis needed, save 8400 CNY/year)
Code Statistics:
- New code: ~1750 lines
- Modified code: ~500 lines
- Test code: ~1800 lines
- Documentation: ~3000 lines
Testing:
- Unit tests: 8/8 passed
- Integration tests: 2/2 passed
- Architecture validation: passed
- Linter errors: 0
Files:
- Platform layer: PostgresCacheAdapter, PgBossQueue, CheckpointService, utils
- ASL module: screeningService, screeningWorker
- DC module: ExtractionController, extractionWorker
- Tests: 11 test files
- Docs: Updated 4 key documents
Status: Phase 1-7 completed, Phase 8-9 pending
2025-12-13 16:10:04 +08:00
f240aa9236
docs(asl): Update module and system status documentation
...
ASL Module Status Update (v1.2 -> v1.3):
- Update development stage: backend completed (Day 2-5)
- Add fulltext-screening backend structure (controllers, services, routes, tests)
- Add 5 new API endpoints for fulltext screening
- Update milestone: Day 4-5 completed (database + batch service + API)
- Mark Day 6-8 as pending (frontend development)
System Status Update (v2.4.0 -> v2.5.0):
- Update ASL module progress across all sections
- Update database schema: 4 tables -> 6 tables (add fulltext screening tables)
- Update API endpoints: 10 -> 15 (add 5 fulltext screening APIs)
- Update backend structure to include fulltext-screening module
- Reflect 2500+ lines of code added in Day 2-5
2025-11-23 11:36:30 +08:00
88cc049fb3
feat(asl): Complete Day 5 - Fulltext Screening Backend API Development
...
- Implement 5 core API endpoints (create task, get progress, get results, update decision, export Excel)
- Add FulltextScreeningController with Zod validation (652 lines)
- Implement ExcelExporter service with 4-sheet report generation (352 lines)
- Register routes under /api/v1/asl/fulltext-screening
- Create 31 REST Client test cases
- Add automated integration test script
- Fix PDF extraction fallback mechanism in LLM12FieldsService
- Update API design documentation to v3.0
- Update development plan to v1.2
- Create Day 5 development record
- Clean up temporary test files
2025-11-23 10:52:07 +08:00
beb7f7f559
feat(asl): Implement full-text screening core LLM service and validation system (Day 1-3)
...
Core Components:
- PDFStorageService with Dify/OSS adapters
- LLM12FieldsService with Nougat-first + dual-model + 3-layer JSON parsing
- PromptBuilder for dynamic prompt assembly
- MedicalLogicValidator with 5 rules + fault tolerance
- EvidenceChainValidator for citation integrity
- ConflictDetectionService for dual-model comparison
Prompt Engineering:
- System Prompt (6601 chars, Section-Aware strategy)
- User Prompt template (PICOS context injection)
- JSON Schema (12 fields constraints)
- Cochrane standards (not loaded in MVP)
Key Innovations:
- 3-layer JSON parsing (JSON.parse + json-repair + code block extraction)
- Promise.allSettled for dual-model fault tolerance
- safeGetFieldValue for robust field extraction
- Mixed CN/EN token calculation
Integration Tests:
- integration-test.ts (full test)
- quick-test.ts (quick test)
- cached-result-test.ts (fault tolerance test)
Documentation Updates:
- Development record (Day 2-3 summary)
- Quality assurance strategy (full-text screening)
- Development plan (progress update)
- Module status (v1.1 update)
- Technical debt (10 new items)
Test Results:
- JSON parsing success rate: 100%
- Medical logic validation: 5/5 passed
- Dual-model parallel processing: OK
- Cost per PDF: CNY 0.10
Files: 238 changed, 14383 insertions(+), 32 deletions(-)
Docs: docs/03-涓氬姟妯″潡/ASL-AI鏅鸿兘鏂囩尞/05-寮€鍙戣褰?2025-11-22_Day2-Day3_LLM鏈嶅姟涓庨獙璇佺郴缁熷紑鍙?md
2025-11-22 22:21:12 +08:00
8eef9e0544
feat(asl): Complete Week 4 - Results display and Excel export with hybrid solution
...
Features:
- Backend statistics API (cloud-native Prisma aggregation)
- Results page with hybrid solution (AI consensus + human final decision)
- Excel export (frontend generation, zero disk write, cloud-native)
- PRISMA-style exclusion reason analysis with bar chart
- Batch selection and export (3 export methods)
- Fixed logic contradiction (inclusion does not show exclusion reason)
- Optimized table width (870px, no horizontal scroll)
Components:
- Backend: screeningController.ts - add getProjectStatistics API
- Frontend: ScreeningResults.tsx - complete results page (hybrid solution)
- Frontend: excelExport.ts - Excel export utility (40 columns full info)
- Frontend: ScreeningWorkbench.tsx - add navigation button
- Utils: get-test-projects.mjs - quick test tool
Architecture:
- Cloud-native: backend aggregation reduces network transfer
- Cloud-native: frontend Excel generation (zero file persistence)
- Reuse platform: global prisma instance, logger
- Performance: statistics API < 500ms, Excel export < 3s (1000 records)
Documentation:
- Update module status guide (add Week 4 features)
- Update task breakdown (mark Week 4 completed)
- Update API design spec (add statistics API)
- Update database design (add field usage notes)
- Create Week 4 development plan
- Create Week 4 completion report
- Create technical debt list
Test:
- End-to-end flow test passed
- All features verified
- Performance test passed
- Cloud-native compliance verified
Ref: Week 4 Development Plan
Scope: ASL Module MVP - Title Abstract Screening Results
Cloud-Native: Backend aggregation + Frontend Excel generation
2025-11-21 20:12:38 +08:00
