Major fixes:
- Fix pivot transformation with special characters in column names
- Fix compute column validation for Chinese punctuation
- Fix recode dialog to fetch unique values from full dataset via new API
- Add column mapping mechanism to handle special characters
Database migration:
- Add column_mapping field to dc_tool_c_sessions table
- Migration file: 20251208_add_column_mapping
UX improvements:
- Darken table grid lines for better visibility
- Reduce column width by 40% with tooltip support
- Insert new columns next to source columns
- Preserve original row order after operations
- Add notice about 50-row preview limit
Modified files:
- Backend: SessionService, SessionController, QuickActionService, routes
- Python: pivot.py, compute.py, recode.py, binning.py, conditional.py
- Frontend: DataGrid, RecodeDialog, index.tsx, ag-grid-custom.css
- Database: schema.prisma, migration SQL
Status: Code complete, database migrated, ready for testing
Summary:
- Update Tool C MVP Development Plan (V1.3)
* Clarify Python execution as core feature
* Add 15 real medical data cleaning scenarios (basic/medium/advanced)
* Enhance System Prompt with 10 Few-shot examples
* Discover existing Python service (extraction_service)
* Update to extend existing service instead of rebuilding
- Create Tool C MVP Development TODO List
* 3-week plan with 30 tasks (Day 1-15)
* 4 core milestones with clear acceptance criteria
* Daily checklist and risk management
* Detailed task breakdown for each day
Key Changes:
- Python service: Extend existing extraction_service instead of new setup
- Test scenarios: 15 scenarios (5 basic + 5 medium + 5 advanced)
- Success criteria: Basic >90%, Medium >80%, Advanced >60%, Total >80%
- Development time: Reduced from 3 weeks to 2 weeks (reuse infrastructure)
Status: Planning complete, ready to start Day 1 development
- Implement Tool B 5-step workflow (upload, schema, processing, verify, result)
- Add back navigation button to Portal
- Optimize Step 2 field list styling to match prototype
- Fix step 3 label: 'dual-blind' to 'dual-model'
- Create API service layer with 7 endpoints
- Integrate Tool B route into DC module
- Add comprehensive TypeScript types
Components (~1100 lines):
- index.tsx: Main Tool B entry with state management
- Step1Upload.tsx: File upload and health check
- Step2Schema.tsx: Smart template configuration
- Step3Processing.tsx: Dual-model extraction progress
- Step4Verify.tsx: Conflict verification workbench
- Step5Result.tsx: Result display
- StepIndicator.tsx: Step progress component
- api/toolB.ts: API service layer
Status: Frontend complete, ready for API integration
