deploy: Complete 0126-27 deployment - database upgrade, services update, code recovery
Major Changes: - Database: Install pg_bigm/pgvector plugins, create test database - Python service: v1.0 -> v1.1, add pymupdf4llm/openpyxl/pypandoc - Node.js backend: v1.3 -> v1.7, fix pino-pretty and ES Module imports - Frontend: v1.2 -> v1.3, skip TypeScript check for deployment - Code recovery: Restore empty files from local backup Technical Fixes: - Fix pino-pretty error in production (conditional loading) - Fix ES Module import paths (add .js extensions) - Fix OSSAdapter TypeScript errors - Update Prisma Schema (63 models, 16 schemas) - Update environment variables (DATABASE_URL, EXTRACTION_SERVICE_URL, OSS) - Remove deprecated variables (REDIS_URL, DIFY_API_URL, DIFY_API_KEY) Documentation: - Create 0126 deployment folder with 8 documents - Update database development standards v2.0 - Update SAE deployment status records Deployment Status: - PostgreSQL: ai_clinical_research_test with plugins - Python: v1.1 @ 172.17.173.84:8000 - Backend: v1.7 @ 172.17.173.89:3001 - Frontend: v1.3 @ 172.17.173.90:80 Tested: All services running successfully on SAE
This commit is contained in:
375
docs/05-部署文档/0126部署/02-OSS环境配置方案.md
Normal file
375
docs/05-部署文档/0126部署/02-OSS环境配置方案.md
Normal file
@@ -0,0 +1,375 @@
|
||||
# 📦 OSS存储环境配置方案
|
||||
|
||||
> **文档版本**：v1.0
|
||||
> **创建日期**：2026-01-26
|
||||
> **适用范围**：阿里云OSS对象存储
|
||||
> **变更类型**：环境分离 + 新Bucket创建
|
||||
|
||||
---
|
||||
|
||||
## 📋 一、变更概述
|
||||
|
||||
### 1.1 变更内容
|
||||
|
||||
| 变更项 | 描述 | 优先级 |
|
||||
|--------|------|--------|
|
||||
| **创建开发环境Bucket** | ai-clinical-data-dev, ai-clinical-static-dev | 🔴 高 |
|
||||
| **创建生产环境Bucket** | ai-clinical-data, ai-clinical-static | 🔴 高 |
|
||||
| **更新环境变量** | OSS_BUCKET, OSS_BUCKET_STATIC, OSS_INTERNAL | 🔴 高 |
|
||||
| **配置RAM权限** | 确保SAE服务可访问新Bucket | 🔴 高 |
|
||||
|
||||
### 1.2 当前OSS状态
|
||||
|
||||
```yaml
|
||||
现有Bucket: ai-clinical-research
|
||||
用途: 所有环境混用
|
||||
RAM用户: oss-bucket-put-object@1991407246109125.onaliyun.com
|
||||
AccessKeyId: LTAI5tB2Dt3NdvBL3G7nYGv7
|
||||
```
|
||||
|
||||
### 1.3 目标状态
|
||||
|
||||
```yaml
|
||||
开发环境:
|
||||
数据Bucket: ai-clinical-data-dev
|
||||
静态Bucket: ai-clinical-static-dev
|
||||
|
||||
生产环境:
|
||||
数据Bucket: ai-clinical-data
|
||||
静态Bucket: ai-clinical-static
|
||||
|
||||
保留:
|
||||
原有Bucket: ai-clinical-research（保留，逐步迁移）
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🗂️ 二、Bucket规划
|
||||
|
||||
### 2.1 Bucket用途说明
|
||||
|
||||
| Bucket名称 | 环境 | 用途 | 存储内容 |
|
||||
|------------|------|------|---------|
|
||||
| `ai-clinical-data-dev` | 开发 | 业务数据 | PDF、文档、临时文件 |
|
||||
| `ai-clinical-static-dev` | 开发 | 静态资源 | 头像、图片、公共资源 |
|
||||
| `ai-clinical-data` | 生产 | 业务数据 | PDF、文档、临时文件 |
|
||||
| `ai-clinical-static` | 生产 | 静态资源 | 头像、图片、公共资源 |
|
||||
|
||||
### 2.2 目录结构规划
|
||||
|
||||
```
|
||||
ai-clinical-data[-dev]/
|
||||
├── uploads/
|
||||
│ ├── pdfs/ # PDF文件上传
|
||||
│ ├── docx/ # Word文档上传
|
||||
│ ├── txt/ # 文本文件上传
|
||||
│ └── excel/ # Excel文件上传
|
||||
├── exports/ # 导出文件临时存储
|
||||
├── temp/ # 临时文件（定期清理）
|
||||
└── backups/ # 备份文件
|
||||
|
||||
ai-clinical-static[-dev]/
|
||||
├── avatars/ # 用户头像
|
||||
├── images/ # 公共图片
|
||||
├── templates/ # 模板文件
|
||||
└── public/ # 公开资源
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔧 三、创建Bucket操作步骤
|
||||
|
||||
### Step 1：登录OSS控制台
|
||||
|
||||
```
|
||||
地址：https://oss.console.aliyun.com/
|
||||
地域：华北2（北京）
|
||||
```
|
||||
|
||||
### Step 2：创建开发数据Bucket
|
||||
|
||||
1. 点击"创建Bucket"
|
||||
2. 填写配置：
|
||||
- **Bucket名称**：`ai-clinical-data-dev`
|
||||
- **地域**：华北2（北京）
|
||||
- **存储类型**：标准存储
|
||||
- **存储冗余类型**：同城冗余存储
|
||||
- **版本控制**：关闭
|
||||
- **读写权限**：私有
|
||||
- **服务端加密**：无
|
||||
3. 点击"确定"
|
||||
|
||||
### Step 3：创建开发静态Bucket
|
||||
|
||||
1. 点击"创建Bucket"
|
||||
2. 填写配置：
|
||||
- **Bucket名称**：`ai-clinical-static-dev`
|
||||
- **地域**：华北2（北京）
|
||||
- **存储类型**：标准存储
|
||||
- **存储冗余类型**：同城冗余存储
|
||||
- **读写权限**：私有
|
||||
3. 点击"确定"
|
||||
|
||||
### Step 4：创建生产数据Bucket
|
||||
|
||||
1. 点击"创建Bucket"
|
||||
2. 填写配置：
|
||||
- **Bucket名称**：`ai-clinical-data`
|
||||
- **地域**：华北2（北京）
|
||||
- **存储类型**：标准存储
|
||||
- **存储冗余类型**：同城冗余存储
|
||||
- **读写权限**：私有
|
||||
3. 点击"确定"
|
||||
|
||||
### Step 5：创建生产静态Bucket
|
||||
|
||||
1. 点击"创建Bucket"
|
||||
2. 填写配置：
|
||||
- **Bucket名称**：`ai-clinical-static`
|
||||
- **地域**：华北2（北京）
|
||||
- **存储类型**：标准存储
|
||||
- **存储冗余类型**：同城冗余存储
|
||||
- **读写权限**：私有
|
||||
3. 点击"确定"
|
||||
|
||||
---
|
||||
|
||||
## 🔐 四、RAM权限配置
|
||||
|
||||
### 4.1 当前RAM用户
|
||||
|
||||
```yaml
|
||||
RAM用户名: oss-bucket-put-object@1991407246109125.onaliyun.com
|
||||
AccessKeyId: LTAI5tB2Dt3NdvBL3G7nYGv7
|
||||
AccessKeySecret: 1iSN9k39RkApP93QjUhC1DcPIeMG4V # 敏感信息
|
||||
```
|
||||
|
||||
### 4.2 更新RAM策略
|
||||
|
||||
需要为RAM用户添加新Bucket的访问权限。
|
||||
|
||||
**方式1：更新自定义策略**
|
||||
|
||||
登录RAM控制台，找到对应策略，更新为：
|
||||
|
||||
```json
|
||||
{
|
||||
"Version": "1",
|
||||
"Statement": [
|
||||
{
|
||||
"Effect": "Allow",
|
||||
"Action": [
|
||||
"oss:PutObject",
|
||||
"oss:GetObject",
|
||||
"oss:DeleteObject",
|
||||
"oss:ListObjects",
|
||||
"oss:GetObjectAcl",
|
||||
"oss:PutObjectAcl"
|
||||
],
|
||||
"Resource": [
|
||||
"acs:oss:*:*:ai-clinical-research",
|
||||
"acs:oss:*:*:ai-clinical-research/*",
|
||||
"acs:oss:*:*:ai-clinical-data",
|
||||
"acs:oss:*:*:ai-clinical-data/*",
|
||||
"acs:oss:*:*:ai-clinical-data-dev",
|
||||
"acs:oss:*:*:ai-clinical-data-dev/*",
|
||||
"acs:oss:*:*:ai-clinical-static",
|
||||
"acs:oss:*:*:ai-clinical-static/*",
|
||||
"acs:oss:*:*:ai-clinical-static-dev",
|
||||
"acs:oss:*:*:ai-clinical-static-dev/*"
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**方式2：使用AliyunOSSFullAccess策略（简单但权限较大）**
|
||||
|
||||
```bash
|
||||
# 在RAM控制台为用户附加策略
|
||||
AliyunOSSFullAccess
|
||||
```
|
||||
|
||||
### 4.3 验证权限
|
||||
|
||||
```bash
|
||||
# 使用ossutil测试上传
|
||||
ossutil64 cp test.txt oss://ai-clinical-data-dev/test/test.txt \
|
||||
-i LTAI5tB2Dt3NdvBL3G7nYGv7 \
|
||||
-k 1iSN9k39RkApP93QjUhC1DcPIeMG4V \
|
||||
-e oss-cn-beijing.aliyuncs.com
|
||||
|
||||
# 测试下载
|
||||
ossutil64 cat oss://ai-clinical-data-dev/test/test.txt \
|
||||
-i LTAI5tB2Dt3NdvBL3G7nYGv7 \
|
||||
-k 1iSN9k39RkApP93QjUhC1DcPIeMG4V \
|
||||
-e oss-cn-beijing.aliyuncs.com
|
||||
|
||||
# 清理测试文件
|
||||
ossutil64 rm oss://ai-clinical-data-dev/test/test.txt \
|
||||
-i LTAI5tB2Dt3NdvBL3G7nYGv7 \
|
||||
-k 1iSN9k39RkApP93QjUhC1DcPIeMG4V \
|
||||
-e oss-cn-beijing.aliyuncs.com
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ⚙️ 五、环境变量配置
|
||||
|
||||
### 5.1 本地开发环境（backend/.env）
|
||||
|
||||
```bash
|
||||
# OSS配置 - 开发环境
|
||||
STORAGE_TYPE=oss
|
||||
OSS_REGION=oss-cn-beijing
|
||||
OSS_BUCKET=ai-clinical-data-dev
|
||||
OSS_BUCKET_STATIC=ai-clinical-static-dev
|
||||
OSS_ACCESS_KEY_ID=LTAI5tB2Dt3NdvBL3G7nYGv7
|
||||
OSS_ACCESS_KEY_SECRET=1iSN9k39RkApP93QjUhC1DcPIeMG4V
|
||||
OSS_INTERNAL=false # 本地开发用公网
|
||||
```
|
||||
|
||||
### 5.2 SAE测试环境
|
||||
|
||||
```bash
|
||||
# OSS配置 - SAE测试环境
|
||||
STORAGE_TYPE=oss
|
||||
OSS_REGION=oss-cn-beijing
|
||||
OSS_BUCKET=ai-clinical-data-dev
|
||||
OSS_BUCKET_STATIC=ai-clinical-static-dev
|
||||
OSS_ACCESS_KEY_ID=LTAI5tB2Dt3NdvBL3G7nYGv7
|
||||
OSS_ACCESS_KEY_SECRET=1iSN9k39RkApP93QjUhC1DcPIeMG4V
|
||||
OSS_INTERNAL=true # SAE使用内网Endpoint
|
||||
```
|
||||
|
||||
### 5.3 SAE生产环境
|
||||
|
||||
```bash
|
||||
# OSS配置 - SAE生产环境
|
||||
STORAGE_TYPE=oss
|
||||
OSS_REGION=oss-cn-beijing
|
||||
OSS_BUCKET=ai-clinical-data
|
||||
OSS_BUCKET_STATIC=ai-clinical-static
|
||||
OSS_ACCESS_KEY_ID=LTAI5tB2Dt3NdvBL3G7nYGv7
|
||||
OSS_ACCESS_KEY_SECRET=1iSN9k39RkApP93QjUhC1DcPIeMG4V
|
||||
OSS_INTERNAL=true # SAE使用内网Endpoint
|
||||
```
|
||||
|
||||
### 5.4 环境变量对比表
|
||||
|
||||
| 变量名 | 开发环境 | SAE测试 | SAE生产 |
|
||||
|--------|---------|---------|---------|
|
||||
| `STORAGE_TYPE` | oss | oss | oss |
|
||||
| `OSS_REGION` | oss-cn-beijing | oss-cn-beijing | oss-cn-beijing |
|
||||
| `OSS_BUCKET` | ai-clinical-data-dev | ai-clinical-data-dev | ai-clinical-data |
|
||||
| `OSS_BUCKET_STATIC` | ai-clinical-static-dev | ai-clinical-static-dev | ai-clinical-static |
|
||||
| `OSS_INTERNAL` | false | true | true |
|
||||
|
||||
---
|
||||
|
||||
## 🔄 六、SAE环境变量更新步骤
|
||||
|
||||
### Step 1：更新Node.js后端环境变量
|
||||
|
||||
1. 登录SAE控制台：https://sae.console.aliyun.com/
|
||||
2. 进入应用：`nodejs-backend-test`
|
||||
3. 点击【应用配置】→【环境变量】
|
||||
4. 添加/修改以下变量：
|
||||
```
|
||||
OSS_BUCKET=ai-clinical-data（生产）或 ai-clinical-data-dev（测试）
|
||||
OSS_BUCKET_STATIC=ai-clinical-static（生产）或 ai-clinical-static-dev（测试）
|
||||
OSS_INTERNAL=true
|
||||
```
|
||||
5. 点击【保存】
|
||||
6. 点击【重启应用】（注意：是重启，不是部署！）
|
||||
|
||||
### Step 2：验证配置
|
||||
|
||||
```bash
|
||||
# 通过公网测试API
|
||||
curl http://8.140.53.236/api/v1/health
|
||||
|
||||
# 检查日志确认OSS配置加载正确
|
||||
# SAE控制台 → 日志查询
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📊 七、Endpoint说明
|
||||
|
||||
### 7.1 OSS Endpoint
|
||||
|
||||
| 类型 | Endpoint | 使用场景 |
|
||||
|------|----------|---------|
|
||||
| **公网Endpoint** | `oss-cn-beijing.aliyuncs.com` | 本地开发、外部访问 |
|
||||
| **内网Endpoint** | `oss-cn-beijing-internal.aliyuncs.com` | SAE/ECS内网访问 |
|
||||
| **VPC Endpoint** | `oss-cn-beijing-internal.aliyuncs.com` | VPC网络访问 |
|
||||
|
||||
### 7.2 代码中的Endpoint选择
|
||||
|
||||
```typescript
|
||||
// 根据OSS_INTERNAL环境变量选择Endpoint
|
||||
const endpoint = process.env.OSS_INTERNAL === 'true'
|
||||
? 'oss-cn-beijing-internal.aliyuncs.com' // 内网（免流量费）
|
||||
: 'oss-cn-beijing.aliyuncs.com'; // 公网
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ✅ 八、验证清单
|
||||
|
||||
### 8.1 Bucket创建验证
|
||||
|
||||
- [ ] `ai-clinical-data-dev` Bucket创建成功
|
||||
- [ ] `ai-clinical-static-dev` Bucket创建成功
|
||||
- [ ] `ai-clinical-data` Bucket创建成功
|
||||
- [ ] `ai-clinical-static` Bucket创建成功
|
||||
|
||||
### 8.2 权限验证
|
||||
|
||||
- [ ] RAM用户可访问所有新Bucket
|
||||
- [ ] 上传测试文件成功
|
||||
- [ ] 下载测试文件成功
|
||||
- [ ] 删除测试文件成功
|
||||
|
||||
### 8.3 SAE配置验证
|
||||
|
||||
- [ ] Node.js后端环境变量已更新
|
||||
- [ ] 服务重启后正常运行
|
||||
- [ ] 文件上传功能正常
|
||||
- [ ] 文件下载功能正常
|
||||
|
||||
---
|
||||
|
||||
## ⚠️ 九、注意事项
|
||||
|
||||
### 9.1 成本控制
|
||||
|
||||
- 内网访问免流量费
|
||||
- 设置生命周期规则清理临时文件
|
||||
- 定期检查存储用量
|
||||
|
||||
### 9.2 安全建议
|
||||
|
||||
- AccessKeySecret妥善保管
|
||||
- 不要提交到Git仓库
|
||||
- 定期轮换AccessKey
|
||||
|
||||
### 9.3 数据迁移
|
||||
|
||||
如需从旧Bucket迁移数据：
|
||||
```bash
|
||||
# 使用ossutil迁移
|
||||
ossutil64 cp -r oss://ai-clinical-research/uploads/ oss://ai-clinical-data/uploads/ \
|
||||
--include "*.pdf"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
> **最后更新**：2026-01-26
|
||||
> **维护人员**：开发团队
|
||||
> **参考文档**：[阿里云OSS文档](https://help.aliyun.com/product/31815.html)
|
||||
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user