feat(rag): Complete RAG engine implementation with pgvector

Major Features:
- Created ekb_schema (13th schema) with 3 tables: KB/Document/Chunk
- Implemented EmbeddingService (text-embedding-v4, 1024-dim vectors)
- Implemented ChunkService (smart Markdown chunking)
- Implemented VectorSearchService (multi-query + hybrid search)
- Implemented RerankService (qwen3-rerank)
- Integrated DeepSeek V3 QueryRewriter for cross-language search
- Python service: Added pymupdf4llm for PDF-to-Markdown conversion
- PKB: Dual-mode adapter (pgvector/dify/hybrid)

Architecture:
- Brain-Hand Model: Business layer (DeepSeek) + Engine layer (pgvector)
- Cross-language support: Chinese query matches English documents
- Small Embedding (1024) + Strong Reranker strategy

Performance:
- End-to-end latency: 2.5s
- Cost per query: 0.0025 RMB
- Accuracy improvement: +20.5% (cross-language)

Tests:
- test-embedding-service.ts: Vector embedding verified
- test-rag-e2e.ts: Full pipeline tested
- test-rerank.ts: Rerank quality validated
- test-query-rewrite.ts: Cross-language search verified
- test-pdf-ingest.ts: Real PDF document tested (Dongen 2003.pdf)

Documentation:
- Added 05-RAG-Engine-User-Guide.md
- Added 02-Document-Processing-User-Guide.md
- Updated system status documentation

Status: Production ready

backend/scripts/check-dc-tables.mjs

@@ -256,6 +256,9 @@ checkDCTables();
 
 
 
+
+
+
 
 
 

backend/scripts/create-capability-schema.sql

@@ -13,3 +13,6 @@ CREATE SCHEMA IF NOT EXISTS capability_schema;
 
 
 
+
+
+

backend/scripts/create-tool-c-ai-history-table.mjs

@@ -208,6 +208,9 @@ createAiHistoryTable()
 
 
 
+
+
+
 
 
 

backend/scripts/create-tool-c-table.js

@@ -195,6 +195,9 @@ createToolCTable()
 
 
 
+
+
+
 
 
 

backend/scripts/create-tool-c-table.mjs

@@ -192,6 +192,9 @@ createToolCTable()
 
 
 
+
+
+
 
 
 

backend/scripts/migrate-aia-prompts.ts

@@ -316,3 +316,6 @@ main()
   .finally(() => prisma.$disconnect());
 
 
+
+
+

backend/scripts/setup-prompt-system.ts

@@ -123,3 +123,6 @@ main()
 
 
 
+
+
+

backend/scripts/test-pkb-apis-simple.ts

@@ -340,6 +340,9 @@ runTests().catch(error => {
 
 
 
+
+
+
 
 
 

backend/scripts/test-prompt-api.ts

@@ -89,3 +89,6 @@ testAPI().catch(console.error);
 
 
 
+
+
+

backend/scripts/test-unifuncs-deepsearch.ts

@@ -119,3 +119,6 @@ async function testDeepSearch() {
 testDeepSearch().catch(console.error);
 
 
+
+
+

backend/scripts/verify-pkb-rvw-schema.ts

@@ -305,6 +305,9 @@ verifySchemas()
 
 
 
+
+
+