Summary: - Migrate PostgreSQL to pgvector/pgvector:pg15 Docker image - Successfully install and verify pgvector 0.8.1 extension - Create comprehensive Dify-to-pgvector migration plan - Update PKB module documentation with pgvector status - Update system documentation with pgvector integration Key changes: - docker-compose.yml: Switch to pgvector/pgvector:pg15 image - Add EkbDocument and EkbChunk data model design - Design R-C-R-G hybrid retrieval architecture - Add clinical data JSONB fields (pico, studyDesign, regimen, safety, criteria, endpoints) - Create detailed 10-day implementation roadmap Documentation updates: - PKB module status: pgvector RAG infrastructure ready - System status: pgvector 0.8.1 integrated - New: Dify replacement development plan (01-Dify替换为pgvector开发计划.md) - New: Enterprise medical knowledge base solution V2 Tested: PostgreSQL with pgvector verified, frontend and backend functionality confirmed
330 lines
7.7 KiB
Markdown
330 lines
7.7 KiB
Markdown
# 数据库开发规范
|
||
|
||
> 版本: v1.0
|
||
> 更新日期: 2026-01-11
|
||
> 编写背景: 2026-01-11 数据库事故后总结
|
||
|
||
---
|
||
|
||
## 1. 核心原则
|
||
|
||
### 1.1 安全第一
|
||
|
||
```
|
||
⚠️ 黄金法则:任何数据库操作前,必须先备份!
|
||
```
|
||
|
||
### 1.2 禁止使用的危险命令
|
||
|
||
| 命令 | 危险等级 | 说明 |
|
||
|------|----------|------|
|
||
| `prisma db push --force-reset` | 🔴 **极高** | 会删除所有数据和非Prisma管理的对象 |
|
||
| `prisma migrate reset` | 🔴 **极高** | 重置整个数据库 |
|
||
| `DROP DATABASE` | 🔴 **极高** | 删除整个数据库 |
|
||
| `TRUNCATE TABLE` | 🟠 高 | 清空表数据 |
|
||
|
||
### 1.3 推荐的安全命令
|
||
|
||
| 命令 | 用途 | 安全性 |
|
||
|------|------|--------|
|
||
| `prisma migrate dev` | 开发环境迁移 | ✅ 安全 |
|
||
| `prisma migrate deploy` | 生产环境迁移 | ✅ 安全 |
|
||
| `prisma db push` (无 --force-reset) | 同步schema到数据库 | ⚠️ 谨慎使用 |
|
||
| `prisma generate` | 生成客户端 | ✅ 安全 |
|
||
|
||
---
|
||
|
||
## 2. 数据库备份规范
|
||
|
||
### 2.1 备份命令
|
||
|
||
```bash
|
||
# 通过 Docker 备份(推荐)
|
||
docker exec ai-clinical-postgres pg_dump -U postgres -d ai_clinical_research > backup_$(date +%Y%m%d_%H%M%S).sql
|
||
|
||
# PowerShell 版本
|
||
$timestamp = Get-Date -Format "yyyyMMdd_HHmmss"
|
||
docker exec ai-clinical-postgres pg_dump -U postgres -d ai_clinical_research > "backup_$timestamp.sql"
|
||
```
|
||
|
||
### 2.2 备份时机
|
||
|
||
| 时机 | 是否必须 |
|
||
|------|----------|
|
||
| 执行任何 `prisma migrate` 前 | ✅ 必须 |
|
||
| 执行 `prisma db push` 前 | ✅ 必须 |
|
||
| 部署到生产环境前 | ✅ 必须 |
|
||
| 每日自动备份 | ✅ 推荐 |
|
||
| 重大功能发布前 | ✅ 必须 |
|
||
|
||
### 2.3 备份文件管理
|
||
|
||
```
|
||
AIclinicalresearch/
|
||
├── backup_20260111_131506.sql # 日期_时间命名
|
||
├── rds_init_20251224_154529.sql # 历史备份
|
||
└── ...
|
||
```
|
||
|
||
---
|
||
|
||
## 3. Schema 变更流程
|
||
|
||
### 3.1 标准流程
|
||
|
||
```mermaid
|
||
graph TD
|
||
A[修改 schema.prisma] --> B[备份数据库]
|
||
B --> C[运行 prisma migrate dev]
|
||
C --> D{迁移成功?}
|
||
D -->|是| E[测试功能]
|
||
D -->|否| F[从备份恢复]
|
||
E --> G{测试通过?}
|
||
G -->|是| H[提交代码]
|
||
G -->|否| F
|
||
```
|
||
|
||
### 3.2 具体步骤
|
||
|
||
```bash
|
||
# 1. 备份数据库
|
||
docker exec ai-clinical-postgres pg_dump -U postgres -d ai_clinical_research > backup_before_migration.sql
|
||
|
||
# 2. 修改 schema.prisma
|
||
|
||
# 3. 创建迁移
|
||
npx prisma migrate dev --name describe_your_change
|
||
|
||
# 4. 检查生成的迁移 SQL
|
||
cat prisma/migrations/xxx_describe_your_change/migration.sql
|
||
|
||
# 5. 测试
|
||
|
||
# 6. 如果失败,恢复备份
|
||
cat backup_before_migration.sql | docker exec -i ai-clinical-postgres psql -U postgres -d ai_clinical_research
|
||
```
|
||
|
||
---
|
||
|
||
## 4. Prisma 与数据库不一致问题
|
||
|
||
### 4.1 Prisma 不管理的对象
|
||
|
||
以下数据库对象不在 `schema.prisma` 中定义,需要单独管理:
|
||
|
||
| 对象 | 类型 | 来源 | 恢复脚本 |
|
||
|------|------|------|----------|
|
||
| `platform_schema.job_common` | 表 | pg-boss 运行时创建 | `restore_job_common.sql` |
|
||
| `platform_schema.create_queue()` | 函数 | pg-boss 初始化 | `restore_pgboss_functions.sql` |
|
||
| `platform_schema.delete_queue()` | 函数 | pg-boss 初始化 | `restore_pgboss_functions.sql` |
|
||
|
||
### 4.2 恢复非 Prisma 管理的对象
|
||
|
||
```bash
|
||
# 如果误删了 pg-boss 相关对象,执行:
|
||
npx prisma db execute --file restore_job_common.sql --schema prisma/schema.prisma
|
||
npx prisma db execute --file restore_pgboss_functions.sql --schema prisma/schema.prisma
|
||
```
|
||
|
||
### 4.3 检查数据库与 Prisma 一致性
|
||
|
||
```bash
|
||
# 查看数据库中的函数
|
||
SELECT routine_name FROM information_schema.routines WHERE routine_schema = 'platform_schema';
|
||
|
||
# 查看数据库中的表
|
||
SELECT table_name FROM information_schema.tables WHERE table_schema = 'platform_schema';
|
||
|
||
# 对比 schema.prisma 定义
|
||
```
|
||
|
||
---
|
||
|
||
## 5. 多 Schema 架构规范
|
||
|
||
### 5.1 Schema 命名规范
|
||
|
||
| Schema | 用途 | 示例表 |
|
||
|--------|------|--------|
|
||
| `platform_schema` | 平台基础设施 | users, tenants, app_cache |
|
||
| `admin_schema` | 运营管理 | admin_operation_logs |
|
||
| `aia_schema` | AI智能问答 | conversations, messages |
|
||
| `asl_schema` | 文献筛选 | screening_projects, literatures |
|
||
| `dc_schema` | 数据清洗 | dc_templates, dc_extraction_tasks |
|
||
| `pkb_schema` | 个人知识库 | knowledge_bases, documents |
|
||
| `iit_schema` | IIT项目 | projects, audit_logs |
|
||
| `rvw_schema` | 论文预审 | review_tasks |
|
||
| `capability_schema` | 通用能力 | prompt_templates |
|
||
| `public` | 旧数据/兼容 | users (旧), admin_logs |
|
||
|
||
### 5.2 表命名规范
|
||
|
||
```
|
||
{schema_name}.{module_prefix}_{entity_name}
|
||
|
||
示例:
|
||
- dc_schema.dc_templates
|
||
- dc_schema.dc_extraction_tasks
|
||
- asl_schema.screening_projects
|
||
```
|
||
|
||
---
|
||
|
||
## 6. 外键与数据完整性
|
||
|
||
### 6.1 跨 Schema 外键
|
||
|
||
```prisma
|
||
// ✅ 正确:明确指定关系
|
||
model ReviewTask {
|
||
userId String @map("user_id")
|
||
user PublicUser @relation(fields: [userId], references: [id])
|
||
|
||
@@schema("rvw_schema")
|
||
}
|
||
|
||
model PublicUser {
|
||
id String @id
|
||
reviewTasks ReviewTask[]
|
||
|
||
@@schema("public")
|
||
}
|
||
```
|
||
|
||
### 6.2 外键指向检查
|
||
|
||
在使用 `prisma db push` 后,检查外键是否正确:
|
||
|
||
```sql
|
||
SELECT
|
||
tc.table_schema, tc.table_name, kcu.column_name,
|
||
ccu.table_schema AS foreign_schema, ccu.table_name AS foreign_table
|
||
FROM information_schema.table_constraints AS tc
|
||
JOIN information_schema.key_column_usage AS kcu ON tc.constraint_name = kcu.constraint_name
|
||
JOIN information_schema.constraint_column_usage AS ccu ON ccu.constraint_name = tc.constraint_name
|
||
WHERE tc.constraint_type = 'FOREIGN KEY';
|
||
```
|
||
|
||
---
|
||
|
||
## 7. 事故恢复流程
|
||
|
||
### 7.1 误删数据恢复
|
||
|
||
```bash
|
||
# 1. 停止应用
|
||
# 2. 从备份恢复
|
||
cat backup_xxx.sql | docker exec -i ai-clinical-postgres psql -U postgres -d ai_clinical_research
|
||
|
||
# 3. 验证数据
|
||
npx tsx verify_system.ts
|
||
|
||
# 4. 重启应用
|
||
```
|
||
|
||
### 7.2 Schema 不一致恢复
|
||
|
||
```bash
|
||
# 1. 检查差异
|
||
npx tsx compare_schema_db.ts
|
||
|
||
# 2. 恢复缺失的对象
|
||
npx prisma db execute --file restore_xxx.sql --schema prisma/schema.prisma
|
||
|
||
# 3. 验证
|
||
```
|
||
|
||
---
|
||
|
||
## 8. 开发环境 vs 生产环境
|
||
|
||
### 8.1 开发环境
|
||
|
||
- 可以使用 `prisma migrate dev`
|
||
- 可以使用 `prisma db push`(谨慎)
|
||
- 定期同步生产数据库结构
|
||
|
||
### 8.2 生产环境
|
||
|
||
- **只能**使用 `prisma migrate deploy`
|
||
- **禁止**使用任何 `--force` 或 `--reset` 参数
|
||
- 变更前必须有回滚计划
|
||
- 必须有最新备份
|
||
|
||
---
|
||
|
||
## 9. 检查清单
|
||
|
||
### 9.1 数据库变更前检查
|
||
|
||
- [ ] 已备份数据库
|
||
- [ ] 已审查 schema.prisma 变更
|
||
- [ ] 已检查是否影响非 Prisma 管理的对象
|
||
- [ ] 已准备回滚方案
|
||
- [ ] 已在开发环境测试
|
||
|
||
### 9.2 部署后检查
|
||
|
||
- [ ] 应用正常启动
|
||
- [ ] 数据库连接正常
|
||
- [ ] pg-boss 队列正常工作
|
||
- [ ] 核心功能测试通过
|
||
|
||
---
|
||
|
||
## 10. 常用脚本
|
||
|
||
### 10.1 验证脚本位置
|
||
|
||
```
|
||
backend/
|
||
├── verify_system.ts # 系统完整性验证
|
||
├── compare_schema_db.ts # Schema与数据库对比
|
||
├── check_iit_asl_data.ts # 检查模块数据
|
||
├── restore_job_common.sql # 恢复 job_common 表
|
||
└── restore_pgboss_functions.sql # 恢复 pg-boss 函数
|
||
```
|
||
|
||
### 10.2 快速验证命令
|
||
|
||
```bash
|
||
# 设置环境变量
|
||
$env:DATABASE_URL="postgresql://postgres:postgres123@localhost:5432/ai_clinical_research"
|
||
|
||
# 验证系统
|
||
npx tsx verify_system.ts
|
||
|
||
# 检查数据
|
||
npx tsx check_iit_asl_data.ts
|
||
```
|
||
|
||
---
|
||
|
||
## 附录:事故案例
|
||
|
||
### 案例1:2026-01-11 数据库重置事故
|
||
|
||
**原因**:使用 `prisma db push --force-reset` 导致非 Prisma 管理的对象丢失
|
||
|
||
**影响**:
|
||
- pg-boss 函数丢失,队列无法注册
|
||
- job_common 表丢失
|
||
- 用户数据丢失(已通过 seed 恢复)
|
||
|
||
**教训**:
|
||
1. 永远不要使用 `--force-reset`
|
||
2. 操作前必须备份
|
||
3. 了解 Prisma 的管理边界
|
||
|
||
详见:`docs/08-项目管理/2026-01-11-数据库事故总结.md`
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|
||
|