HaHafeng
74cf346453
Major features: 1. Missing value imputation (6 simple methods + MICE): - Mean/Median/Mode/Constant imputation - Forward fill (ffill) and Backward fill (bfill) for time series - MICE multivariate imputation (in progress, shape issue to fix) 2. Auto precision detection: - Automatically match decimal places of original data - Prevent false precision (e.g. 13.57 instead of 13.566716417910449) 3. Categorical variable detection: - Auto-detect and skip categorical columns in MICE - Show warnings for unsuitable columns - Suggest mode imputation for categorical data 4. UI improvements: - Rename button: "Delete Missing" to "Missing Value Handling" - Remove standalone "Dedup" and "MICE" buttons - 3-tab dialog: Delete / Fill / Advanced Fill - Display column statistics and recommended methods - Extended warning messages (8 seconds for skipped columns) 5. Bug fixes: - Fix sessionService.updateSessionData -> saveProcessedData - Fix OperationResult interface (add message and stats) - Fix Toolbar button labels and removal Modified files: Python: operations/fillna.py (new, 556 lines), main.py (3 new endpoints) Backend: QuickActionService.ts, QuickActionController.ts, routes/index.ts Frontend: MissingValueDialog.tsx (new, 437 lines), Toolbar.tsx, index.tsx Tests: test_fillna_operations.py (774 lines), test scripts and docs Docs: 5 documentation files updated Known issues: - MICE imputation has DataFrame shape mismatch issue (under debugging) - Workaround: Use 6 simple imputation methods first Status: Development complete, MICE debugging in progress Lines added: ~2000 lines across 3 tiers
41 lines
1.3 KiB
SQL
41 lines
1.3 KiB
SQL
-- 创建 Tool C Session 表
|
||
-- 日期: 2025-12-06
|
||
-- 用途: 科研数据编辑器会话管理
|
||
|
||
CREATE TABLE IF NOT EXISTS dc_schema.dc_tool_c_sessions (
|
||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||
user_id VARCHAR(255) NOT NULL,
|
||
file_name VARCHAR(500) NOT NULL,
|
||
file_key VARCHAR(500) NOT NULL,
|
||
|
||
-- 数据元信息
|
||
total_rows INTEGER NOT NULL,
|
||
total_cols INTEGER NOT NULL,
|
||
columns JSONB NOT NULL,
|
||
encoding VARCHAR(50),
|
||
file_size INTEGER NOT NULL,
|
||
|
||
-- 时间戳
|
||
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||
updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
||
expires_at TIMESTAMP NOT NULL
|
||
);
|
||
|
||
-- 创建索引
|
||
CREATE INDEX IF NOT EXISTS idx_dc_tool_c_sessions_user_id ON dc_schema.dc_tool_c_sessions(user_id);
|
||
CREATE INDEX IF NOT EXISTS idx_dc_tool_c_sessions_expires_at ON dc_schema.dc_tool_c_sessions(expires_at);
|
||
|
||
-- 添加注释
|
||
COMMENT ON TABLE dc_schema.dc_tool_c_sessions IS 'Tool C (科研数据编辑器) Session会话表';
|
||
COMMENT ON COLUMN dc_schema.dc_tool_c_sessions.file_key IS 'OSS存储路径: dc/tool-c/sessions/{timestamp}-{fileName}';
|
||
COMMENT ON COLUMN dc_schema.dc_tool_c_sessions.columns IS '列名数组 ["age", "gender", "diagnosis"]';
|
||
COMMENT ON COLUMN dc_schema.dc_tool_c_sessions.expires_at IS '过期时间（创建后10分钟）';
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|