feat(dc/tool-c): Add missing value imputation feature with 6 methods and MICE
Major features: 1. Missing value imputation (6 simple methods + MICE): - Mean/Median/Mode/Constant imputation - Forward fill (ffill) and Backward fill (bfill) for time series - MICE multivariate imputation (in progress, shape issue to fix) 2. Auto precision detection: - Automatically match decimal places of original data - Prevent false precision (e.g. 13.57 instead of 13.566716417910449) 3. Categorical variable detection: - Auto-detect and skip categorical columns in MICE - Show warnings for unsuitable columns - Suggest mode imputation for categorical data 4. UI improvements: - Rename button: "Delete Missing" to "Missing Value Handling" - Remove standalone "Dedup" and "MICE" buttons - 3-tab dialog: Delete / Fill / Advanced Fill - Display column statistics and recommended methods - Extended warning messages (8 seconds for skipped columns) 5. Bug fixes: - Fix sessionService.updateSessionData -> saveProcessedData - Fix OperationResult interface (add message and stats) - Fix Toolbar button labels and removal Modified files: Python: operations/fillna.py (new, 556 lines), main.py (3 new endpoints) Backend: QuickActionService.ts, QuickActionController.ts, routes/index.ts Frontend: MissingValueDialog.tsx (new, 437 lines), Toolbar.tsx, index.tsx Tests: test_fillna_operations.py (774 lines), test scripts and docs Docs: 5 documentation files updated Known issues: - MICE imputation has DataFrame shape mismatch issue (under debugging) - Workaround: Use 6 simple imputation methods first Status: Development complete, MICE debugging in progress Lines added: ~2000 lines across 3 tiers
This commit is contained in:
@@ -302,3 +302,5 @@ Changes:
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -374,3 +374,5 @@ cd path; command
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -603,3 +603,5 @@ import { logger } from '../../../../common/logging/index.js';
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -607,3 +607,5 @@ Content-Length: 45234
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -259,3 +259,5 @@ Response:
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -412,3 +412,5 @@ Response:
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -406,3 +406,5 @@ import { ChatContainer } from '@/shared/components/Chat';
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -316,3 +316,5 @@ const initialMessages = defaultMessages.length > 0 ? defaultMessages : [{
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -356,3 +356,5 @@ python main.py
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -604,3 +604,5 @@ http://localhost:5173/data-cleaning/tool-c
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -214,3 +214,5 @@ Day 5 (6-8小时):
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -392,3 +392,5 @@ Docs: docs/03-业务模块/DC-数据清洗整理/06-开发记录/DC模块重建
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -367,3 +367,5 @@ const mockAssets: Asset[] = [
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -351,3 +351,5 @@ frontend-v2/src/modules/dc/
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -311,3 +311,5 @@
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -265,3 +265,5 @@ ConflictDetectionService // 冲突检测（字段级对比）
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -314,3 +314,5 @@
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -277,3 +277,5 @@
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -341,3 +341,5 @@
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -429,3 +429,5 @@ Tool B后端代码**100%复用**了平台通用能力层，无任何重复开发
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -275,3 +275,5 @@
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -206,3 +206,5 @@ $ node scripts/check-dc-tables.mjs
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user