feat(aia): Complete AIA V2.0 with universal streaming capabilities
Major Changes: - Add StreamingService with OpenAI Compatible format - Upgrade Chat component V2 with Ant Design X integration - Implement AIA module with 12 intelligent agents - Update API routes to unified /api/v1 prefix - Update system documentation Backend (~1300 lines): - common/streaming: OpenAI Compatible adapter - modules/aia: 12 agents, conversation service, streaming integration - Update route versions (RVW, PKB to v1) Frontend (~3500 lines): - modules/aia: AgentHub + ChatWorkspace (100% prototype restoration) - shared/Chat: AIStreamChat, ThinkingBlock, useAIStream Hook - Update API endpoints to v1 Documentation: - AIA module status guide - Universal capabilities catalog - System overview updates - All module documentation sync Tested: Stream response verified, authentication working Status: AIA V2.0 core completed (85%)
This commit is contained in:
@@ -1,44 +1,38 @@
|
||||
# <20>臬<EFBFBD><E887AC>滨蔭<E6BBA8><E894AD><EFBFBD>
|
||||
|
||||
> **文档版本:** v1.0
|
||||
> **创建日期:** 2025-11-09
|
||||
> **维护者:** 技术团队
|
||||
> **<EFBFBD><EFBFBD>﹝<EFBFBD><EFBFBD>𧋦嚗?* v1.0
|
||||
> **<EFBFBD>𥕦遣<EFBFBD>交<EFBFBD>嚗?* 2025-11-09
|
||||
> **蝏湔擪<EFBFBD><EFBFBD><EFBFBD>** <20><><EFBFBD>臬𣪧<E887AC>?
|
||||
> **<2A><><EFBFBD>擧凒<E693A7>堆<EFBFBD>** 2025-11-09
|
||||
|
||||
---
|
||||
|
||||
## <20><> <20><>﹝霂湔<E99C82>
|
||||
|
||||
本文档记录系统运行所需的全部环境变量配置,包括:
|
||||
- 数据库连接信息
|
||||
- LLM API密钥配置
|
||||
- 第三方服务配置
|
||||
- 安全相关配置
|
||||
<EFBFBD>祆<EFBFBD>獢<EFBFBD>扇敶閧頂蝏蠘<EFBFBD>銵峕<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>函㴓憓<EFBFBD><EFBFBD><EFBFBD>誯<EFBFBD>蝵殷<EFBFBD><EFBFBD><EFBFBD>𡠺嚗?- <20>唳旿摨栞<E691A8><E6A09E>乩縑<E4B9A9>?- LLM API撖<49>𤨎<EFBFBD>滨蔭
|
||||
- 蝚砌<EFBFBD><EFBFBD>寞<EFBFBD><EFBFBD>⊿<EFBFBD>蝵?- 摰匧<E691B0><E58CA7>詨<EFBFBD><E8A9A8>滨蔭
|
||||
|
||||
---
|
||||
|
||||
## 🗄️ 数据库配置
|
||||
|
||||
## <EFBFBD><EFBFBD>儭?<3F>唳旿摨㯄<E691A8>蝵?
|
||||
### PostgreSQL餈墧𦻖靽⊥<E99DBD>
|
||||
|
||||
**当前配置:**
|
||||
**敶枏<EFBFBD><EFBFBD>滨蔭嚗?*
|
||||
```env
|
||||
DATABASE_URL=postgresql://postgres:postgres@localhost:5432/ai_clinical_research
|
||||
```
|
||||
|
||||
**连接参数说明:**
|
||||
| 参数 | 当前值 | 说明 |
|
||||
**餈墧𦻖<EFBFBD><EFBFBD>㺭霂湔<EFBFBD>嚗?*
|
||||
| <EFBFBD><EFBFBD>㺭 | 敶枏<E695B6><E69E8F>?| 霂湔<E99C82> |
|
||||
|------|--------|------|
|
||||
| 用户名 | postgres | PostgreSQL默认用户 |
|
||||
| 密码 | postgres | 本地开发密码 |
|
||||
| 主机 | localhost | 本地数据库 |
|
||||
| <EFBFBD>冽<EFBFBD><EFBFBD>?| postgres | PostgreSQL暺䁅恕<EFBFBD>冽<EFBFBD> |
|
||||
| 撖<EFBFBD><EFBFBD> | postgres | <EFBFBD>砍𧑐撘<EFBFBD><EFBFBD>穃<EFBFBD><EFBFBD>?|
|
||||
| 銝餅㦤 | localhost | <EFBFBD>砍𧑐<EFBFBD>唳旿摨?|
|
||||
| 蝡臬藁 | 5432 | PostgreSQL暺䁅恕蝡臬藁 |
|
||||
| 数据库名 | ai_clinical_research | 项目数据库 |
|
||||
| <EFBFBD>唳旿摨枏<EFBFBD> | ai_clinical_research | 憿寧𤌍<EFBFBD>唳旿摨?|
|
||||
|
||||
**Schema信息:**
|
||||
- **当前Schema:** `public`(所有13个表)
|
||||
- **计划Schema:** 10个隔离Schema(Week 1实施)
|
||||
- platform_schema
|
||||
**Schema靽⊥<EFBFBD>嚗?*
|
||||
- **敶枏<EFBFBD>Schema嚗?* `public`嚗<EFBFBD><EFBFBD><EFBFBD>?3銝芾”嚗?- **霈∪<E99C88>Schema嚗?* 10銝芷<E98A9D>蝳艋chema嚗Áeek 1摰墧鴌嚗? - platform_schema
|
||||
- common_schema
|
||||
- asl_schema
|
||||
- aia_schema
|
||||
@@ -55,50 +49,45 @@ DATABASE_URL=postgresql://postgres:postgres@localhost:5432/ai_clinical_research
|
||||
|
||||
### 1. DeepSeek API
|
||||
|
||||
**用途:** 主力大模型(推理、对话、批处理)
|
||||
|
||||
**配置:**
|
||||
**<EFBFBD>券<EFBFBD>䈑<EFBFBD>** 銝餃<E98A9D>憭扳芋<E689B3>页<EFBFBD><E9A1B5>函<EFBFBD><E587BD><EFBFBD>笆霂腈<E99C82><E88588>鸌憭<E9B88C><E686AD>嚗?
|
||||
**<EFBFBD>滨蔭嚗?*
|
||||
```env
|
||||
DEEPSEEK_API_KEY=sk-your-deepseek-key
|
||||
```
|
||||
|
||||
**获取方式:**
|
||||
**<EFBFBD>瑕<EFBFBD><EFBFBD>孵<EFBFBD>嚗?*
|
||||
1. 霈輸䔮嚗冴ttps://platform.deepseek.com
|
||||
2. 瘜典<E7989C>/<2F>餃<EFBFBD>韐血噡
|
||||
3. 餈𥕦<E9A488>API Keys憿菟𢒰
|
||||
4. <20>𥕦遣<F0A595A6>啁<EFBFBD>API Key
|
||||
|
||||
**使用场景:**
|
||||
**雿輻鍂<EFBFBD>箸艶嚗?*
|
||||
- AI<41>箄<EFBFBD><E7AE84>桃<EFBFBD>嚗<EFBFBD><E59A97>厰<EFBFBD>霂<EFBFBD>遠<EFBFBD><E981A0>ICO璇喟<E79287>蝑㚁<E89D91>
|
||||
- 批处理任务
|
||||
- 标题摘要初筛
|
||||
- <EFBFBD>孵<EFBFBD><EFBFBD><EFBFBD>遙<EFBFBD>?- <20><><EFBFBD><EFBFBD>䁅<EFBFBD><E48185>萘<EFBFBD>
|
||||
- <20>冽<EFBFBD>憭滨<E686AD>
|
||||
|
||||
**定价:**
|
||||
- DeepSeek-V3: ¥1/M tokens(输入),¥2/M tokens(输出)
|
||||
**摰帋遠嚗?*
|
||||
- DeepSeek-V3: 瞼1/M tokens嚗<EFBFBD><EFBFBD><EFBFBD>伐<EFBFBD>嚗𡐿?/M tokens嚗<EFBFBD><EFBFBD><EFBFBD>綽<EFBFBD>
|
||||
|
||||
---
|
||||
|
||||
### 2. <20>帋<EFBFBD><E5B88B><EFBFBD>䔮嚗㇋wen嚗𡅅PI
|
||||
|
||||
**用途:** 备用模型、特定场景优化
|
||||
|
||||
**配置:**
|
||||
**<EFBFBD>券<EFBFBD>䈑<EFBFBD>** 憭<>鍂璅∪<E79285><E288AA><EFBFBD>鸌摰𡁜㦤<F0A1819C>臭<EFBFBD><E887AD>?
|
||||
**<EFBFBD>滨蔭嚗?*
|
||||
```env
|
||||
DASHSCOPE_API_KEY=sk-your-qwen-key
|
||||
```
|
||||
|
||||
**获取方式:**
|
||||
**<EFBFBD>瑕<EFBFBD><EFBFBD>孵<EFBFBD>嚗?*
|
||||
1. 霈輸䔮嚗冴ttps://dashscope.console.aliyun.com
|
||||
2. 撘<><E69298>鋽ashScope<70>滚𦛚
|
||||
3. <20>𥕦遣API Key
|
||||
|
||||
**使用场景:**
|
||||
- 长文本处理(Qwen-Long)
|
||||
- 特定领域任务
|
||||
- 模型对比筛选
|
||||
|
||||
**定价:**
|
||||
**雿輻鍂<EFBFBD>箸艶嚗?*
|
||||
- <EFBFBD>踵<EFBFBD><EFBFBD>砍<EFBFBD><EFBFBD><EFBFBD><EFBFBD>Qwen-Long嚗?- <20>孵<EFBFBD>憸<EFBFBD><E686B8>隞餃𦛚
|
||||
- 璅∪<EFBFBD>撖寞<EFBFBD>蝑偦<EFBFBD>?
|
||||
**摰帋遠嚗?*
|
||||
- Qwen-Max: 瞼0.04/1K tokens
|
||||
- Qwen-Long: 瞼0.005/1K tokens
|
||||
|
||||
@@ -108,39 +97,36 @@ DASHSCOPE_API_KEY=sk-your-qwen-key
|
||||
|
||||
**<EFBFBD>券<EFBFBD>䈑<EFBFBD>** <20>朞<EFBFBD>隞<EFBFBD><E99A9E>撟喳蝱蝔喳<E89D94>霈輸䔮OpenAI<41>龦laude API
|
||||
|
||||
**为什么使用CloseAI?**
|
||||
- ✅ 国内稳定访问,无需科学上网
|
||||
- ✅ 一个账号同时使用OpenAI和Claude
|
||||
- ✅ 兼容OpenAI SDK标准接口
|
||||
- ✅ 最新模型支持:GPT-5-Pro、Claude-Sonnet-4.5
|
||||
**銝箔<EFBFBD>銋<EFBFBD>蝙<EFBFBD>每loseAI嚗?*
|
||||
- <EFBFBD>?<3F>賢<EFBFBD>蝔喳<E89D94>霈輸䔮嚗峕<E59A97><E5B395><EFBFBD>蝘穃郎銝羓<E98A9D>
|
||||
- <EFBFBD>?銝<>銝芾揭<E88ABE>瑕<EFBFBD><E79195>嗡蝙<E597A1>汰penAI<EFBFBD>龦laude
|
||||
- <EFBFBD>?<3F>澆捆OpenAI SDK<EFBFBD><EFBFBD><EFBFBD><EFBFBD>亙藁
|
||||
- <EFBFBD>?<3F><><EFBFBD>唳芋<E594B3>𧢲𣈲<F0A7A2B2><F0A388B2><EFBFBD>GPT-5-Pro<EFBFBD><EFBFBD>laude-Sonnet-4.5
|
||||
|
||||
**配置:**
|
||||
**<EFBFBD>滨蔭嚗?*
|
||||
```env
|
||||
# CloseAI蝏煺<E89D8F>API Key
|
||||
CLOSEAI_API_KEY=sk-cu0iepbXYGGx2jc7BqP6ogtSWmP6fk918qV3RUdtGC3Edlpo
|
||||
|
||||
# OpenAI端点(通过CloseAI代理)
|
||||
CLOSEAI_OPENAI_BASE_URL=https://api.openai-proxy.org/v1
|
||||
# OpenAI蝡舐<EFBFBD>嚗<EFBFBD><EFBFBD>朞<EFBFBD>CloseAI隞<EFBFBD><EFBFBD>嚗?CLOSEAI_OPENAI_BASE_URL=https://api.openai-proxy.org/v1
|
||||
|
||||
# Claude端点(通过CloseAI代理)
|
||||
CLOSEAI_CLAUDE_BASE_URL=https://api.openai-proxy.org/anthropic
|
||||
# Claude蝡舐<EFBFBD>嚗<EFBFBD><EFBFBD>朞<EFBFBD>CloseAI隞<EFBFBD><EFBFBD>嚗?CLOSEAI_CLAUDE_BASE_URL=https://api.openai-proxy.org/anthropic
|
||||
```
|
||||
|
||||
**<EFBFBD>舀<EFBFBD><EFBFBD><EFBFBD>芋<EFBFBD>页<EFBFBD>**
|
||||
|
||||
| 模型系列 | 最新模型 | 说明 |
|
||||
| 璅∪<EFBFBD>蝟餃<EFBFBD> | <20><><EFBFBD>唳芋<E594B3>?| 霂湔<E99C82> |
|
||||
|---------|---------|------|
|
||||
| **OpenAI** | `gpt-5-pro` | 最新GPT-5模型 ⭐ |
|
||||
| **OpenAI** | `gpt-5-pro` | <EFBFBD><EFBFBD><EFBFBD>蚣PT-5璅∪<E79285> 潃?|
|
||||
| OpenAI | `gpt-4-turbo-preview` | GPT-4擃䀹<E69383>扯<EFBFBD><E689AF><EFBFBD>𧋦 |
|
||||
| OpenAI | `gpt-3.5-turbo` | 快速经济版本 |
|
||||
| **Claude** | `claude-sonnet-4-5-20250929` | 最新Claude-4.5 ⭐ |
|
||||
| Claude | `claude-3-5-sonnet-20241022` | Claude-3.5稳定版 |
|
||||
| Claude | `claude-3-opus-20240229` | Claude-3最强版本 |
|
||||
| OpenAI | `gpt-3.5-turbo` | 敹恍<EFBFBD>毺<EFBFBD>瘚𡒊<EFBFBD><EFBFBD>?|
|
||||
| **Claude** | `claude-sonnet-4-5-20250929` | <EFBFBD><EFBFBD><EFBFBD>蚓laude-4.5 潃?|
|
||||
| Claude | `claude-3-5-sonnet-20241022` | Claude-3.5蝔喳<EFBFBD><EFBFBD>?|
|
||||
| Claude | `claude-3-opus-20240229` | Claude-3<EFBFBD><EFBFBD>撘箇<EFBFBD><EFBFBD>?|
|
||||
|
||||
**获取方式:**
|
||||
**<EFBFBD>瑕<EFBFBD><EFBFBD>孵<EFBFBD>嚗?*
|
||||
1. 霈輸䔮嚗冴ttps://platform.openai-proxy.org
|
||||
2. 注册账号并充值
|
||||
3. 在控制台获取API Key
|
||||
2. 瘜典<EFBFBD>韐血噡撟嗅<EFBFBD><EFBFBD>?3. <20>冽綉<E586BD>嗅蝱<E59785>瑕<EFBFBD>API Key
|
||||
4. 銝<>銝服PI Key<65>臬<EFBFBD><E887AC>嗉<EFBFBD><E59789>汰penAI<41>龦laude
|
||||
|
||||
**隞<><E99A9E>蝷箔<E89DB7>嚗㇍ypeScript嚗㚁<E59A97>**
|
||||
@@ -170,49 +156,44 @@ const claudeResponse = await claudeClient.chat.completions.create({
|
||||
});
|
||||
```
|
||||
|
||||
**使用场景(AI智能文献 ⭐):**
|
||||
- **GPT-5-Pro:** 文献精准筛选、质量控制
|
||||
- **Claude-4.5:** 第三方仲裁、结构化输出
|
||||
- **双模型对比:** DeepSeek + GPT-5-Pro 快速+高质量
|
||||
- **三模型共识:** 冲突时启用Claude仲裁
|
||||
**雿輻鍂<EFBFBD>箸艶嚗㇁I<EFBFBD>箄<EFBFBD><EFBFBD><EFBFBD>讃 潃琜<E6BD83>嚗?*
|
||||
- **GPT-5-Pro嚗?* <20><>讃蝎曉<E89D8E>蝑偦<E89D91>剹<EFBFBD><E589B9>捶<EFBFBD>𤩺綉<F0A4A9BA>?- **Claude-4.5嚗?* 蝚砌<E89D9A><E7A08C>嫣輔鋆<E8BC94><E98B86><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>颲枏枂
|
||||
- **<EFBFBD>峕芋<EFBFBD>见笆瘥䈑<EFBFBD>** DeepSeek + GPT-5-Pro 敹恍<E695B9>?擃䁅捶<E48185>?- **銝㗇芋<E39787>见<EFBFBD>霂<EFBFBD><E99C82>** <20>脩<EFBFBD><E884A9>嗅鍳<E59785>每laude隞脰<E99A9E>
|
||||
|
||||
**定价(参考):**
|
||||
**摰帋遠嚗<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>嚗?*
|
||||
- GPT-5-Pro: ~瞼0.10/1K tokens嚗<73><E59A97><EFBFBD>伐<EFBFBD>嚗斋瞼0.20/1K tokens嚗<73><E59A97><EFBFBD>綽<EFBFBD>
|
||||
- Claude-4.5-Sonnet: ~瞼0.021/1K tokens嚗<73><E59A97><EFBFBD>伐<EFBFBD>嚗斋瞼0.105/1K tokens嚗<73><E59A97><EFBFBD>綽<EFBFBD>
|
||||
|
||||
**注意事项:**
|
||||
**瘜冽<EFBFBD>鈭钅★嚗?*
|
||||
- <20>𩤃<EFBFBD> API Key<65><79>鉄<EFBFBD>𤩺<EFBFBD>靽⊥<E99DBD>嚗䔶<E59A97>閬<EFBFBD><E996AC>鈭文<E988AD>Git
|
||||
- <20>𩤃<EFBFBD> 撱箄悅摰𡁏<E691B0><F0A1818F>湔揢API Key
|
||||
- <20>𩤃<EFBFBD> <20>煺漣<E785BA>臬<EFBFBD>雿輻鍂<E8BCBB>祉<EFBFBD><E7A589><EFBFBD>PI Key
|
||||
|
||||
---
|
||||
|
||||
## 🔧 Dify配置(RAG引擎)
|
||||
|
||||
## <EFBFBD>圲 Dify<EFBFBD>滨蔭嚗㇌AG撘閙<EFBFBD>嚗?
|
||||
**<2A>券<EFBFBD>䈑<EFBFBD>** <20>亥<EFBFBD>摨枏<E691A8><E69E8F>𤩺<EFBFBD>蝝U<E89D9D><EFBCB5>AG<41>桃<EFBFBD>
|
||||
|
||||
**配置:**
|
||||
**<EFBFBD>滨蔭嚗?*
|
||||
```env
|
||||
DIFY_API_KEY=app-your-dify-key
|
||||
DIFY_API_URL=http://localhost/v1
|
||||
```
|
||||
|
||||
**部署信息:**
|
||||
- **本地部署:** Docker Compose
|
||||
- **访问地址:** http://localhost
|
||||
- **管理后台:** http://localhost/install
|
||||
**<EFBFBD>函蔡靽⊥<EFBFBD>嚗?*
|
||||
- **<EFBFBD>砍𧑐<EFBFBD>函蔡嚗?* Docker Compose
|
||||
- **霈輸䔮<EFBFBD>啣<EFBFBD>嚗?* http://localhost
|
||||
- **蝞∠<EFBFBD><EFBFBD>𤾸蝱嚗?* http://localhost/install
|
||||
- **<2A>煾<EFBFBD><E785BE>唳旿摨橒<E691A8>** Qdrant嚗<74><E59A97>蝵殷<E89DB5>
|
||||
|
||||
**获取API Key:**
|
||||
**<EFBFBD>瑕<EFBFBD>API Key嚗?*
|
||||
1. 霈輸䔮 Dify 蝞∠<E89D9E><E288A0>𤾸蝱
|
||||
2. 餈𥕦<E9A488>"摨𠉛鍂"憿菟𢒰
|
||||
3. 创建"知识库应用"
|
||||
3. <EFBFBD>𥕦遣"<22>亥<EFBFBD>摨枏<E691A8><E69E8F>?
|
||||
4. 憭滚<E686AD>API Key
|
||||
|
||||
**使用场景:**
|
||||
- 个人知识库文档上传
|
||||
- @知识库问答
|
||||
- 智能引用功能
|
||||
**雿輻鍂<EFBFBD>箸艶嚗?*
|
||||
- 銝芯犖<EFBFBD>亥<EFBFBD>摨𤘪<EFBFBD>獢<EFBFBD><EFBFBD>隡?- @<40>亥<EFBFBD>摨㯄䔮蝑?- <20>箄<EFBFBD>撘閧鍂<E996A7>蠘<EFBFBD>
|
||||
|
||||
---
|
||||
|
||||
@@ -220,18 +201,17 @@ DIFY_API_URL=http://localhost/v1
|
||||
|
||||
### JWT撖<54>𤨎
|
||||
|
||||
**配置:**
|
||||
**<EFBFBD>滨蔭嚗?*
|
||||
```env
|
||||
JWT_SECRET=your-secret-key-change-in-production
|
||||
JWT_EXPIRES_IN=7d
|
||||
```
|
||||
|
||||
**说明:**
|
||||
**霂湔<EFBFBD>嚗?*
|
||||
- <20>其<EFBFBD><E585B6>冽<EFBFBD>霈方<E99C88>Token蝑曉<E89D91>
|
||||
- 撱箄悅雿輻鍂32雿滢誑銝𢠃<E98A9D><F0A2A083>箏<EFBFBD>蝚虫葡
|
||||
- 过期时间:7天
|
||||
|
||||
**生成强密钥(可选):**
|
||||
- 餈<EFBFBD><EFBFBD><EFBFBD>園𡢿嚗?憭?
|
||||
**<2A><><EFBFBD>撘箏<E69298><E7AE8F>伐<EFBFBD><E4BC90>舫<EFBFBD>㚁<EFBFBD>嚗?*
|
||||
```bash
|
||||
# Node.js<6A><73><EFBFBD>
|
||||
node -e "console.log(require('crypto').randomBytes(32).toString('hex'))"
|
||||
@@ -246,7 +226,7 @@ openssl rand -hex 32
|
||||
|
||||
### <20>𡒊垢<F0A1928A>滚𦛚
|
||||
|
||||
**配置:**
|
||||
**<EFBFBD>滨蔭嚗?*
|
||||
```env
|
||||
PORT=3001
|
||||
HOST=0.0.0.0
|
||||
@@ -254,24 +234,24 @@ NODE_ENV=development
|
||||
LOG_LEVEL=info
|
||||
```
|
||||
|
||||
**说明:**
|
||||
| 参数 | 值 | 说明 |
|
||||
**霂湔<EFBFBD>嚗?*
|
||||
| <EFBFBD><EFBFBD>㺭 | <20>?| 霂湔<E99C82> |
|
||||
|------|-----|------|
|
||||
| PORT | 3001 | <20>𡒊垢<F0A1928A>滚𦛚蝡臬藁 |
|
||||
| HOST | 0.0.0.0 | 监听所有网卡 |
|
||||
| NODE_ENV | development | 开发环境 |
|
||||
| HOST | 0.0.0.0 | <EFBFBD>穃𨯬<EFBFBD><EFBFBD><EFBFBD>厩<EFBFBD><EFBFBD>?|
|
||||
| NODE_ENV | development | 撘<EFBFBD><EFBFBD>𤑳㴓憓?|
|
||||
| LOG_LEVEL | info | <20>亙<EFBFBD>蝥批<E89DA5> |
|
||||
|
||||
---
|
||||
|
||||
### CORS<52>滨蔭
|
||||
|
||||
**配置:**
|
||||
**<EFBFBD>滨蔭嚗?*
|
||||
```env
|
||||
CORS_ORIGIN=http://localhost:5173
|
||||
```
|
||||
|
||||
**说明:**
|
||||
**霂湔<EFBFBD>嚗?*
|
||||
- <20><>捂<EFBFBD>滨垢頝典<E9A09D>霈輸䔮
|
||||
- <20>滨垢撘<E59EA2><E69298>烐<EFBFBD><E78390>∪膥嚗冴ttp://localhost:5173
|
||||
|
||||
@@ -279,27 +259,25 @@ CORS_ORIGIN=http://localhost:5173
|
||||
|
||||
### Redis<69>滨蔭嚗<E894AD>虾<EFBFBD>㚁<EFBFBD>
|
||||
|
||||
**配置:**
|
||||
**<EFBFBD>滨蔭嚗?*
|
||||
```env
|
||||
REDIS_URL=redis://localhost:6379
|
||||
```
|
||||
|
||||
**说明:**
|
||||
- 用于缓存和会话管理
|
||||
- 当前未强制要求
|
||||
|
||||
**霂湔<EFBFBD>嚗?*
|
||||
- <EFBFBD>其<EFBFBD>蝻枏<EFBFBD><EFBFBD>䔶<EFBFBD>霂萘恣<EFBFBD>?- 敶枏<E695B6><E69E8F>芸撩<E88AB8>嗉<EFBFBD>瘙?
|
||||
---
|
||||
|
||||
## <20><> <20><>辣銝𠹺<E98A9D><F0A0B9BA>滨蔭
|
||||
|
||||
**配置:**
|
||||
**<EFBFBD>滨蔭嚗?*
|
||||
```env
|
||||
UPLOAD_MAX_SIZE=10485760
|
||||
UPLOAD_DIR=./uploads
|
||||
```
|
||||
|
||||
**说明:**
|
||||
| 参数 | 值 | 说明 |
|
||||
**霂湔<EFBFBD>嚗?*
|
||||
| <EFBFBD><EFBFBD>㺭 | <20>?| 霂湔<E99C82> |
|
||||
|------|-----|------|
|
||||
| UPLOAD_MAX_SIZE | 10485760 | 10MB嚗<42><E59A97><EFBFBD><EFBFBD><EFBFBD> |
|
||||
| UPLOAD_DIR | ./uploads | 銝𠹺<E98A9D><F0A0B9BA><EFBFBD>辣摮睃<E691AE><E79D83>桀<EFBFBD> |
|
||||
@@ -308,20 +286,19 @@ UPLOAD_DIR=./uploads
|
||||
|
||||
## <20><> <20>滨蔭甇仿炊
|
||||
|
||||
### 1. 检查环境变量文件
|
||||
|
||||
**位置:** `backend/.env`
|
||||
### 1. 璉<EFBFBD><EFBFBD>亦㴓憓<EFBFBD><EFBFBD><EFBFBD>𤩺<EFBFBD>隞?
|
||||
**雿滨蔭嚗?* `backend/.env`
|
||||
|
||||
憒<EFBFBD><EFBFBD><EFBFBD><EFBFBD>辣銝滚<EFBFBD><EFBFBD>剁<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>誑銝𧢲芋<EFBFBD>踹<EFBFBD>撱綽<EFBFBD>
|
||||
|
||||
```env
|
||||
# ==================== 服务器配置 ====================
|
||||
# ==================== <EFBFBD>滚𦛚<EFBFBD>券<EFBFBD>蝵?====================
|
||||
PORT=3001
|
||||
HOST=0.0.0.0
|
||||
NODE_ENV=development
|
||||
LOG_LEVEL=info
|
||||
|
||||
# ==================== 数据库配置 ====================
|
||||
# ==================== <EFBFBD>唳旿摨㯄<EFBFBD>蝵?====================
|
||||
DATABASE_URL=postgresql://postgres:postgres@localhost:5432/ai_clinical_research
|
||||
|
||||
# ==================== Redis<69>滨蔭 ====================
|
||||
@@ -335,8 +312,7 @@ JWT_EXPIRES_IN=7d
|
||||
# DeepSeek
|
||||
DEEPSEEK_API_KEY=sk-your-deepseek-key
|
||||
|
||||
# 通义千问(阿里云DashScope)
|
||||
DASHSCOPE_API_KEY=sk-your-qwen-key
|
||||
# <EFBFBD>帋<EFBFBD><EFBFBD><EFBFBD>䔮嚗<EFBFBD>燵<EFBFBD>䔶<EFBFBD>DashScope嚗?DASHSCOPE_API_KEY=sk-your-qwen-key
|
||||
|
||||
# Gemini嚗<69>虾<EFBFBD>㚁<EFBFBD>
|
||||
GEMINI_API_KEY=your-gemini-key
|
||||
@@ -357,7 +333,7 @@ CORS_ORIGIN=http://localhost:5173
|
||||
|
||||
### 2. 撉諹<E69289><E8ABB9>滨蔭
|
||||
|
||||
**启动后端服务:**
|
||||
**<EFBFBD>臬𢆡<EFBFBD>𡒊垢<EFBFBD>滚𦛚嚗?*
|
||||
```bash
|
||||
cd backend
|
||||
npm run dev
|
||||
@@ -365,10 +341,10 @@ npm run dev
|
||||
|
||||
**璉<><E79289>亙鍳<E4BA99>冽𠯫敹梹<E695B9>**
|
||||
```
|
||||
✓ Prisma schema loaded
|
||||
✓ Environment variables loaded from .env
|
||||
✓ Datasource "db": PostgreSQL database "ai_clinical_research"
|
||||
✓ Server running on http://0.0.0.0:3001
|
||||
<EFBFBD>?Prisma schema loaded
|
||||
<EFBFBD>?Environment variables loaded from .env
|
||||
<EFBFBD>?Datasource "db": PostgreSQL database "ai_clinical_research"
|
||||
<EFBFBD>?Server running on http://0.0.0.0:3001
|
||||
```
|
||||
|
||||
**撉諹<E69289><E8ABB9>唳旿摨栞<E691A8><E6A09E>伐<EFBFBD>**
|
||||
@@ -377,7 +353,7 @@ cd backend
|
||||
npx prisma migrate status
|
||||
```
|
||||
|
||||
**预期输出:**
|
||||
**憸<EFBFBD><EFBFBD>颲枏枂嚗?*
|
||||
```
|
||||
Environment variables loaded from .env
|
||||
Datasource "db": PostgreSQL database "ai_clinical_research", schema "public" at "localhost:5432"
|
||||
@@ -390,20 +366,17 @@ Database schema is up to date!
|
||||
|
||||
#### <20>桅<EFBFBD>1嚗𡁏㺭<F0A1818F>桀<EFBFBD>餈墧𦻖憭梯揖
|
||||
|
||||
**错误信息:**
|
||||
**<EFBFBD>躰秤靽⊥<EFBFBD>嚗?*
|
||||
```
|
||||
Error: Can't reach database server at localhost:5432
|
||||
```
|
||||
|
||||
**解决方案:**
|
||||
**閫<EFBFBD><EFBFBD><EFBFBD>寞<EFBFBD>嚗?*
|
||||
1. 璉<><E79289>仙ostgreSQL<51>臬炏<E887AC>臬𢆡
|
||||
2. 验证端口5432是否被占用
|
||||
3. 确认用户名密码是否正确
|
||||
|
||||
2. 撉諹<EFBFBD>蝡臬藁5432<EFBFBD>臬炏鋡怠<EFBFBD><EFBFBD>?3. 蝖株恕<E6A0AA>冽<EFBFBD><E586BD>滚<EFBFBD><E6BB9A><EFBFBD>糓<EFBFBD>行迤蝖?
|
||||
**璉<><E79289>仙ostgreSQL<51>嗆<EFBFBD><E59786><EFBFBD>Windows嚗㚁<E59A97>**
|
||||
```bash
|
||||
# 查看服务状态
|
||||
Get-Service postgresql*
|
||||
# <EFBFBD>亦<EFBFBD><EFBFBD>滚𦛚<EFBFBD>嗆<EFBFBD>?Get-Service postgresql*
|
||||
|
||||
# <20>臬𢆡<E887AC>滚𦛚
|
||||
Start-Service postgresql-x64-15
|
||||
@@ -413,42 +386,39 @@ Start-Service postgresql-x64-15
|
||||
|
||||
#### <20>桅<EFBFBD>2嚗匁LM API靚<49>鍂憭梯揖
|
||||
|
||||
**错误信息:**
|
||||
**<EFBFBD>躰秤靽⊥<EFBFBD>嚗?*
|
||||
```
|
||||
Error: Invalid API key
|
||||
```
|
||||
|
||||
**解决方案:**
|
||||
**閫<EFBFBD><EFBFBD><EFBFBD>寞<EFBFBD>嚗?*
|
||||
1. 璉<><E79289>丕PI Key<65>臬炏甇<E7828F>&憭滚<E686AD>
|
||||
2. 确认API Key是否已激活
|
||||
3. 检查账户余额是否充足
|
||||
|
||||
2. 蝖株恕API Key<EFBFBD>臬炏撌脫<EFBFBD>瘣?3. 璉<><E79289>亥揭<E4BAA5>瑚<EFBFBD>憸脲糓<E884B2>血<EFBFBD>頞?
|
||||
---
|
||||
|
||||
#### <20>桅<EFBFBD>3嚗鋽ify餈墧𦻖憭梯揖
|
||||
|
||||
**错误信息:**
|
||||
**<EFBFBD>躰秤靽⊥<EFBFBD>嚗?*
|
||||
```
|
||||
Error: connect ECONNREFUSED 127.0.0.1:80
|
||||
```
|
||||
|
||||
**解决方案:**
|
||||
**閫<EFBFBD><EFBFBD><EFBFBD>寞<EFBFBD>嚗?*
|
||||
1. 璉<><E79289>主ify<66>臬炏<E887AC>臬𢆡嚗䫤docker-compose ps`
|
||||
2. <20>臬𢆡Dify嚗䫤docker-compose up -d`
|
||||
3. 撉諹<E69289>霈輸䔮嚗𡁏<E59A97>閫<EFBFBD>膥<EFBFBD>枏<EFBFBD> http://localhost
|
||||
|
||||
---
|
||||
|
||||
## 🌟 平台基础设施配置(2025-11-16 新增)
|
||||
|
||||
> **⭐ 重要更新**:为支持云原生部署,新增平台基础设施环境变量
|
||||
## <EFBFBD><EFBFBD> 撟喳蝱<E596B3>箇<EFBFBD>霈暹鴌<E69AB9>滨蔭嚗?025-11-16 <EFBFBD>啣<EFBFBD>嚗?
|
||||
> **潃?<3F>滩<EFBFBD><E6BBA9>湔鰵**嚗帋蛹<E5B88B>舀<EFBFBD>鈭穃<E988AD><E7A983>罸<EFBFBD>蝵莎<E89DB5><E88E8E>啣<EFBFBD>撟喳蝱<E596B3>箇<EFBFBD>霈暹鴌<E69AB9>臬<EFBFBD><E887AC>㗛<EFBFBD>
|
||||
> **霂衣<E99C82><E8A1A3><EFBFBD>﹝**嚗靀撟喳蝱<E596B3>箇<EFBFBD>霈暹鴌閫<E9B48C><E996AB>](../09-<2D>嗆<EFBFBD>摰墧鴌/04-撟喳蝱<E596B3>箇<EFBFBD>霈暹鴌閫<E9B48C><E996AB>.md)
|
||||
|
||||
---
|
||||
|
||||
### 1. 摮睃<E691AE><E79D83>滚𦛚<E6BB9A>滨蔭
|
||||
|
||||
#### **本地开发环境**
|
||||
#### **<EFBFBD>砍𧑐撘<EFBFBD><EFBFBD>𤑳㴓憓?*
|
||||
|
||||
```bash
|
||||
# backend/.env.development
|
||||
@@ -456,10 +426,10 @@ STORAGE_TYPE=local
|
||||
BASE_URL=http://localhost:3001
|
||||
```
|
||||
|
||||
#### **生产环境(阿里云OSS)**
|
||||
#### **<EFBFBD>煺漣<EFBFBD>臬<EFBFBD>嚗<EFBFBD>燵<EFBFBD>䔶<EFBFBD>OSS嚗?*
|
||||
|
||||
```bash
|
||||
# SAE控制台 -> 环境变量配置
|
||||
# SAE<EFBFBD>批<EFBFBD><EFBFBD>?-> <20>臬<EFBFBD><E887AC>㗛<EFBFBD><E3979B>滨蔭
|
||||
STORAGE_TYPE=oss
|
||||
OSS_REGION=oss-cn-hangzhou
|
||||
OSS_BUCKET=aiclinical-prod
|
||||
@@ -470,11 +440,11 @@ OSS_ENDPOINT=https://oss-cn-hangzhou.aliyuncs.com
|
||||
|
||||
#### **<2A>滨蔭霂湔<E99C82>**
|
||||
|
||||
| 变量名 | 必需 | 默认值 | 说明 |
|
||||
| <EFBFBD>㗛<EFBFBD><EFBFBD>?| 敹<><E695B9> | 暺䁅恕<E48185>?| 霂湔<E99C82> |
|
||||
|--------|------|--------|------|
|
||||
| `STORAGE_TYPE` | ✅ | `local` | 存储类型:`local` 或 `oss` |
|
||||
| `STORAGE_TYPE` | <EFBFBD>?| `local` | 摮睃<EFBFBD>蝐餃<EFBFBD>嚗䫤local` <20>?`oss` |
|
||||
| `BASE_URL` | <20>砍𧑐 | `http://localhost:3001` | <20>砍𧑐摮睃<E691AE>霈輸䔮URL |
|
||||
| `OSS_REGION` | 生产 | - | OSS区域(如:oss-cn-hangzhou) |
|
||||
| `OSS_REGION` | <EFBFBD>煺漣 | - | OSS<EFBFBD>箏<EFBFBD>嚗<EFBFBD><EFBFBD>嚗駅ss-cn-hangzhou嚗?|
|
||||
| `OSS_BUCKET` | <20>煺漣 | - | OSS Bucket<65>滨妍 |
|
||||
| `OSS_ACCESS_KEY_ID` | <20>煺漣 | - | <20>輸<EFBFBD>鈭飊ccessKey ID |
|
||||
| `OSS_ACCESS_KEY_SECRET` | <20>煺漣 | - | <20>輸<EFBFBD>鈭飊ccessKey Secret |
|
||||
@@ -483,17 +453,17 @@ OSS_ENDPOINT=https://oss-cn-hangzhou.aliyuncs.com
|
||||
|
||||
### 2. 蝻枏<E89DBB><E69E8F>滚𦛚<E6BB9A>滨蔭
|
||||
|
||||
#### **本地开发环境**
|
||||
#### **<EFBFBD>砍𧑐撘<EFBFBD><EFBFBD>𤑳㴓憓?*
|
||||
|
||||
```bash
|
||||
# backend/.env.development
|
||||
CACHE_TYPE=memory
|
||||
```
|
||||
|
||||
#### **生产环境(阿里云Redis)**
|
||||
#### **<EFBFBD>煺漣<EFBFBD>臬<EFBFBD>嚗<EFBFBD>燵<EFBFBD>䔶<EFBFBD>Redis嚗?*
|
||||
|
||||
```bash
|
||||
# SAE控制台 -> 环境变量配置
|
||||
# SAE<EFBFBD>批<EFBFBD><EFBFBD>?-> <20>臬<EFBFBD><E887AC>㗛<EFBFBD><E3979B>滨蔭
|
||||
CACHE_TYPE=redis
|
||||
REDIS_HOST=r-***.redis.aliyuncs.com
|
||||
REDIS_PORT=6379
|
||||
@@ -503,31 +473,30 @@ REDIS_DB=0
|
||||
|
||||
#### **<2A>滨蔭霂湔<E99C82>**
|
||||
|
||||
| 变量名 | 必需 | 默认值 | 说明 |
|
||||
| <EFBFBD>㗛<EFBFBD><EFBFBD>?| 敹<><E695B9> | 暺䁅恕<E48185>?| 霂湔<E99C82> |
|
||||
|--------|------|--------|------|
|
||||
| `CACHE_TYPE` | ✅ | `memory` | 缓存类型:`memory` 或 `redis` |
|
||||
| `CACHE_TYPE` | <EFBFBD>?| `memory` | 蝻枏<EFBFBD>蝐餃<EFBFBD>嚗䫤memory` <EFBFBD>?`redis` |
|
||||
| `REDIS_HOST` | Redis | - | Redis銝餅㦤<E9A485>啣<EFBFBD> |
|
||||
| `REDIS_PORT` | Redis | `6379` | Redis蝡臬藁 |
|
||||
| `REDIS_PASSWORD` | Redis | - | Redis撖<73><E69296> |
|
||||
| `REDIS_DB` | Redis | `0` | Redis数据库编号 |
|
||||
| `REDIS_DB` | Redis | `0` | Redis<EFBFBD>唳旿摨梶<EFBFBD><EFBFBD>?|
|
||||
|
||||
---
|
||||
|
||||
### 3. <20>唳旿摨栞<E691A8><E6A09E>交<EFBFBD><E4BAA4>滨蔭
|
||||
|
||||
#### **本地开发环境**
|
||||
#### **<EFBFBD>砍𧑐撘<EFBFBD><EFBFBD>𤑳㴓憓?*
|
||||
|
||||
```bash
|
||||
# backend/.env.development
|
||||
DATABASE_URL=postgresql://postgres:postgres@localhost:5432/ai_clinical_research
|
||||
|
||||
# 本地开发无需配置连接池
|
||||
```
|
||||
# <EFBFBD>砍𧑐撘<EFBFBD><EFBFBD>烐<EFBFBD><EFBFBD><EFBFBD><EFBFBD>滨蔭餈墧𦻖瘙?```
|
||||
|
||||
#### **生产环境(阿里云RDS)**
|
||||
#### **<EFBFBD>煺漣<EFBFBD>臬<EFBFBD>嚗<EFBFBD>燵<EFBFBD>䔶<EFBFBD>RDS嚗?*
|
||||
|
||||
```bash
|
||||
# SAE控制台 -> 环境变量配置
|
||||
# SAE<EFBFBD>批<EFBFBD><EFBFBD>?-> <20>臬<EFBFBD><E887AC>㗛<EFBFBD><E3979B>滨蔭
|
||||
DATABASE_URL=postgresql://user:password@rm-xxx.aliyuncs.com:5432/prod_db
|
||||
DB_MAX_CONNECTIONS=400 # RDS<44><53>憭扯<E686AD><E689AF>交㺭
|
||||
MAX_INSTANCES=20 # SAE<41><45>憭批<E686AD>靘𧢲㺭
|
||||
@@ -535,16 +504,15 @@ MAX_INSTANCES=20 # SAE最大实例数
|
||||
|
||||
#### **<2A>滨蔭霂湔<E99C82>**
|
||||
|
||||
| 变量名 | 必需 | 默认值 | 说明 |
|
||||
| <EFBFBD>㗛<EFBFBD><EFBFBD>?| 敹<><E695B9> | 暺䁅恕<E48185>?| 霂湔<E99C82> |
|
||||
|--------|------|--------|------|
|
||||
| `DATABASE_URL` | ✅ | - | PostgreSQL连接字符串 |
|
||||
| `DATABASE_URL` | <EFBFBD>?| - | PostgreSQL餈墧𦻖摮㛖泵銝?|
|
||||
| `DB_MAX_CONNECTIONS` | <20>煺漣 | `400` | RDS<44><53>憭扯<E686AD><E689AF>交㺭 |
|
||||
| `MAX_INSTANCES` | <20>煺漣 | `20` | SAE<41><45>憭批<E686AD>靘𧢲㺭 |
|
||||
|
||||
**连接数计算**:
|
||||
```
|
||||
**餈墧𦻖<EFBFBD>啗恣蝞?*嚗?```
|
||||
瘥誩<EFBFBD>靘贝<EFBFBD><EFBFBD>交㺭 = DB_MAX_CONNECTIONS / MAX_INSTANCES
|
||||
示例:400 / 20 = 20连接/实例
|
||||
蝷箔<EFBFBD>嚗?00 / 20 = 20餈墧𦻖/摰硺<E691B0>
|
||||
```
|
||||
|
||||
---
|
||||
@@ -559,33 +527,30 @@ NODE_ENV=development # development | production | test
|
||||
|
||||
#### **<2A>滨蔭霂湔<E99C82>**
|
||||
|
||||
| 变量名 | 必需 | 默认值 | 说明 |
|
||||
| <EFBFBD>㗛<EFBFBD><EFBFBD>?| 敹<><E695B9> | 暺䁅恕<E48185>?| 霂湔<E99C82> |
|
||||
|--------|------|--------|------|
|
||||
| `LOG_LEVEL` | ✅ | `info` | 日志级别 |
|
||||
| `NODE_ENV` | ✅ | `development` | 运行环境 |
|
||||
| `LOG_LEVEL` | <EFBFBD>?| `info` | <EFBFBD>亙<EFBFBD>蝥批<EFBFBD> |
|
||||
| `NODE_ENV` | <EFBFBD>?| `development` | 餈鞱<EFBFBD><EFBFBD>臬<EFBFBD> |
|
||||
|
||||
---
|
||||
|
||||
### 5. 功能开关配置
|
||||
|
||||
### 5. <EFBFBD>蠘<EFBFBD>撘<EFBFBD><EFBFBD>喲<EFBFBD>蝵?
|
||||
```bash
|
||||
# 启用的业务模块(逗号分隔)
|
||||
ENABLED_MODULES=ASL,AIA,PKB,DC,SSA,ST
|
||||
# <20>舐鍂<E88890><E98D82><EFBFBD><EFBFBD>⊥芋<E28AA5>梹<EFBFBD><E6A2B9>堒噡<E5A092><E599A1><EFBFBD>嚗?ENABLED_MODULES=ASL,AIA,PKB,DC,SSA,ST
|
||||
|
||||
# 或启用全部
|
||||
ENABLED_MODULES=*
|
||||
# <EFBFBD>硋鍳<EFBFBD>典<EFBFBD><EFBFBD>?ENABLED_MODULES=*
|
||||
```
|
||||
|
||||
#### **<2A>滨蔭霂湔<E99C82>**
|
||||
|
||||
| 模块代码 | 模块名称 | 状态 |
|
||||
| 璅∪<EFBFBD>隞<EFBFBD><EFBFBD> | 璅∪<E79285><E288AA>滨妍 | <20>嗆<EFBFBD>?|
|
||||
|---------|---------|------|
|
||||
| `ASL` | AI<41>箄<EFBFBD><E7AE84><EFBFBD>讃 | 撘<><E69298>睲葉 |
|
||||
| `AIA` | AI智能问答 | 已完成 |
|
||||
| `PKB` | 个人知识库 | 已完成 |
|
||||
| `DC` | 数据清洗 | 计划中 |
|
||||
| `SSA` | 智能统计分析 | 计划中 |
|
||||
| `ST` | 统计工具 | 计划中 |
|
||||
| `AIA` | AI<EFBFBD>箄<EFBFBD><EFBFBD>桃<EFBFBD> | 撌脣<E6928C><E884A3>?|
|
||||
| `PKB` | 銝芯犖<EFBFBD>亥<EFBFBD>摨?| 撌脣<E6928C><E884A3>?|
|
||||
| `DC` | <20>唳旿皜<E697BF><E79A9C> | 霈∪<E99C88>銝?|
|
||||
| `SSA` | <EFBFBD>箄<EFBFBD>蝏蠘恣<EFBFBD><EFBFBD><EFBFBD> | 霈∪<E99C88>銝?|
|
||||
| `ST` | 蝏蠘恣撌亙<E6928C> | 霈∪<E99C88>銝?|
|
||||
|
||||
---
|
||||
|
||||
@@ -616,7 +581,7 @@ NODE_ENV=development
|
||||
PORT=3001
|
||||
BASE_URL=http://localhost:3001
|
||||
|
||||
# ==================== 数据库配置 ====================
|
||||
# ==================== <EFBFBD>唳旿摨㯄<EFBFBD>蝵?====================
|
||||
DATABASE_URL=postgresql://postgres:postgres@localhost:5432/ai_clinical_research
|
||||
|
||||
# ==================== 摮睃<E691AE><E79D83>滚𦛚 ====================
|
||||
@@ -643,7 +608,7 @@ JWT_EXPIRES_IN=7d
|
||||
# ==================== CORS<52>滨蔭 ====================
|
||||
CORS_ORIGIN=http://localhost:5173
|
||||
|
||||
# ==================== 功能开关 ====================
|
||||
# ==================== <EFBFBD>蠘<EFBFBD>撘<EFBFBD><EFBFBD>?====================
|
||||
ENABLED_MODULES=*
|
||||
```
|
||||
|
||||
@@ -654,7 +619,7 @@ ENABLED_MODULES=*
|
||||
NODE_ENV=production
|
||||
PORT=3001
|
||||
|
||||
# ==================== 数据库配置 ====================
|
||||
# ==================== <EFBFBD>唳旿摨㯄<EFBFBD>蝵?====================
|
||||
DATABASE_URL=postgresql://user:password@rm-xxx.aliyuncs.com:5432/prod_db
|
||||
DB_MAX_CONNECTIONS=400
|
||||
MAX_INSTANCES=20
|
||||
@@ -685,66 +650,50 @@ DIFY_API_BASE_URL=https://api.dify.ai/v1
|
||||
DIFY_API_KEY=app-***
|
||||
|
||||
# ==================== JWT<57>滨蔭 ====================
|
||||
JWT_SECRET=<生产环境强密钥>
|
||||
JWT_SECRET=<<3C>煺漣<E785BA>臬<EFBFBD>撘箏<E69298><E7AE8F>?
|
||||
JWT_EXPIRES_IN=7d
|
||||
|
||||
# ==================== CORS<52>滨蔭 ====================
|
||||
CORS_ORIGIN=https://app.yourdomain.com
|
||||
|
||||
# ==================== 功能开关 ====================
|
||||
# ==================== <EFBFBD>蠘<EFBFBD>撘<EFBFBD><EFBFBD>?====================
|
||||
ENABLED_MODULES=ASL,AIA,PKB,DC,SSA,ST
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📊 配置检查清单
|
||||
|
||||
## <EFBFBD><EFBFBD> <20>滨蔭璉<E894AD><E79289>交<EFBFBD><E4BAA4>?
|
||||
雿輻鍂隞乩<EFBFBD>皜<EFBFBD><EFBFBD>撉諹<EFBFBD><EFBFBD>滨蔭摰峕㟲<EFBFBD>改<EFBFBD>
|
||||
|
||||
### **基础配置(必需)**
|
||||
- [ ] ✅ 数据库连接成功(`npx prisma migrate status`)
|
||||
- [ ] ✅ 存储类型已配置(`STORAGE_TYPE`)
|
||||
- [ ] ✅ 日志级别已配置(`LOG_LEVEL`)
|
||||
- [ ] ✅ JWT密钥已配置
|
||||
|
||||
### **LLM配置(必需)**
|
||||
- [ ] ✅ DeepSeek API Key配置且可用
|
||||
- [ ] ✅ Qwen API Key配置且可用
|
||||
- [ ] ✅ Dify服务运行中(`docker-compose ps`)
|
||||
- [ ] ✅ Dify API Key已配置
|
||||
|
||||
### **平台基础设施(云原生)**
|
||||
- [ ] ✅ 存储服务配置正确(本地/OSS)
|
||||
- [ ] ✅ 缓存服务配置正确(Memory/Redis)
|
||||
- [ ] ✅ 连接池参数已配置(生产环境)
|
||||
- [ ] ✅ 功能开关已配置(ENABLED_MODULES)
|
||||
|
||||
### **应用运行(必需)**
|
||||
- [ ] ✅ 后端服务启动成功(端口3001)
|
||||
- [ ] ✅ 前端服务启动成功(端口5173)
|
||||
- [ ] ✅ CORS配置正确(前端可访问后端API)
|
||||
- [ ] ✅ 可正常使用平台服务(storage/logger/cache等)
|
||||
### **<EFBFBD>箇<EFBFBD><EFBFBD>滨蔭嚗<EFBFBD><EFBFBD><EFBFBD><EFBFBD>嚗?*
|
||||
- [ ] <20>?<3F>唳旿摨栞<E691A8><E6A09E>交<EFBFBD><E4BAA4><EFBFBD><EFBFBD>`npx prisma migrate status`嚗?- [ ] <20>?摮睃<E691AE>蝐餃<E89D90>撌脤<E6928C>蝵殷<E89DB5>`STORAGE_TYPE`嚗?- [ ] <20>?<3F>亙<EFBFBD>蝥批<E89DA5>撌脤<E6928C>蝵殷<E89DB5>`LOG_LEVEL`嚗?- [ ] <20>?JWT撖<54>𤨎撌脤<E6928C>蝵?
|
||||
### **LLM<4C>滨蔭嚗<E894AD><E59A97><EFBFBD><EFBFBD>嚗?*
|
||||
- [ ] <20>?DeepSeek API Key<65>滨蔭銝𥪜虾<F0A5AA9C>?- [ ] <20>?Qwen API Key<65>滨蔭銝𥪜虾<F0A5AA9C>?- [ ] <20>?Dify<66>滚𦛚餈鞱<E9A488>銝哨<E98A9D>`docker-compose ps`嚗?- [ ] <20>?Dify API Key撌脤<E6928C>蝵?
|
||||
### **撟喳蝱<E596B3>箇<EFBFBD>霈暹鴌嚗<E9B48C><E59A97><EFBFBD>毺<EFBFBD>嚗?*
|
||||
- [ ] <20>?摮睃<E691AE><E79D83>滚𦛚<E6BB9A>滨蔭甇<E894AD>&嚗<EFBC86>𧋦<EFBFBD>?OSS嚗?- [ ] <20>?蝻枏<E89DBB><E69E8F>滚𦛚<E6BB9A>滨蔭甇<E894AD>&嚗㇈emory/Redis嚗?- [ ] <20>?餈墧𦻖瘙惩<E79899><E683A9>啣歇<E595A3>滨蔭嚗<E894AD><E59A97>鈭抒㴓憓<E3B493><E68693>
|
||||
- [ ] <20>?<3F>蠘<EFBFBD>撘<EFBFBD><E69298>喳歇<E596B3>滨蔭嚗𠄌NABLED_MODULES嚗?
|
||||
### **摨𠉛鍂餈鞱<E9A488>嚗<EFBFBD><E59A97><EFBFBD><EFBFBD>嚗?*
|
||||
- [ ] <EFBFBD>?<3F>𡒊垢<F0A1928A>滚𦛚<E6BB9A>臬𢆡<E887AC>𣂼<EFBFBD>嚗<EFBFBD>垢<EFBFBD>?001嚗?- [ ] <20>?<3F>滨垢<E6BBA8>滚𦛚<E6BB9A>臬𢆡<E887AC>𣂼<EFBFBD>嚗<EFBFBD>垢<EFBFBD>?173嚗?- [ ] <20>?CORS<52>滨蔭甇<E894AD>&嚗<EFBC86><E59A97>蝡臬虾霈輸䔮<E8BCB8>𡒊垢API嚗?- [ ] <20>?<3F>舀迤撣訾蝙<E8A8BE>典像<E585B8>唳<EFBFBD><E594B3>∴<EFBFBD>storage/logger/cache蝑㚁<E89D91>
|
||||
|
||||
---
|
||||
|
||||
## <20><> <20>滨蔭<E6BBA8>湔鰵霈啣<E99C88>
|
||||
|
||||
| 日期 | 更新内容 | 更新人 |
|
||||
| <EFBFBD>交<EFBFBD> | <20>湔鰵<E6B994><E9B0B5>捆 | <20>湔鰵鈭?|
|
||||
|------|---------|--------|
|
||||
| 2025-11-09 | 初始配置文档创建 | 技术团队 |
|
||||
| 2025-11-16 | 新增平台基础设施配置章节 | 技术团队 |
|
||||
| 2025-11-09 | <EFBFBD>嘥<EFBFBD><EFBFBD>滨蔭<EFBFBD><EFBFBD>﹝<EFBFBD>𥕦遣 | <20><><EFBFBD>臬𣪧<E887AC>?|
|
||||
| 2025-11-16 | <EFBFBD>啣<EFBFBD>撟喳蝱<EFBFBD>箇<EFBFBD>霈暹鴌<EFBFBD>滨蔭蝡㰘<EFBFBD> | <20><><EFBFBD>臬𣪧<E887AC>?|
|
||||
|
||||
---
|
||||
|
||||
## <20><> <20>詨<EFBFBD><E8A9A8><EFBFBD>﹝
|
||||
|
||||
- [数据库连接配置](../09-架构实施/02-数据库连接配置.md)
|
||||
- [<EFBFBD>唳旿摨栞<EFBFBD><EFBFBD>仿<EFBFBD>蝵孫(../09-<2D>嗆<EFBFBD>摰墧鴌/02-<2D>唳旿摨栞<E691A8><E6A09E>仿<EFBFBD>蝵?md)
|
||||
- [<EFBFBD>函蔡<EFBFBD>嗆<EFBFBD>霈曇恣](../05-<2D>函蔡<E587BD><E894A1>﹝/01-<2D>函蔡<E587BD>嗆<EFBFBD>霈曇恣.md)
|
||||
- [蝟餌<EFBFBD><EFBFBD>嗆<EFBFBD><EFBFBD>餉<EFBFBD>](../00-憿寧𤌍璁<F0A48C8D>膩/<2F><><EFBFBD>舀沲<E88880><E6B2B2><EFBFBD>餉<EFBFBD>.md)
|
||||
|
||||
---
|
||||
|
||||
**文档版本:** v1.0
|
||||
**<EFBFBD><EFBFBD>﹝<EFBFBD><EFBFBD>𧋦嚗?* v1.0
|
||||
**<EFBFBD><EFBFBD><EFBFBD>擧凒<EFBFBD>堆<EFBFBD>** 2025-11-09
|
||||
**维护者:** 技术团队
|
||||
|
||||
**蝏湔擪<EFBFBD><EFBFBD><EFBFBD>** <20><><EFBFBD>臬𣪧<E887AC>?
|
||||
|
||||
@@ -1,18 +1,18 @@
|
||||
# <20>臬<EFBFBD><E887AC>㗛<EFBFBD><E3979B>滨蔭璅⊥踎 (.env)
|
||||
|
||||
> **文档说明:** 本文档提供完整的 `.env` 配置模板
|
||||
> **使用方式:** 复制以下内容到 `backend/.env` 文件中,并填入真实配置值
|
||||
> **创建日期:** 2025-11-09
|
||||
> **<EFBFBD><EFBFBD>﹝霂湔<EFBFBD>嚗?* <20>祆<EFBFBD>獢<EFBFBD><E78DA2>靘𥕦<E99D98><F0A595A6>渡<EFBFBD> `.env` <EFBFBD>滨蔭璅⊥踎
|
||||
> **雿輻鍂<EFBFBD>孵<EFBFBD>嚗?* 憭滚<E686AD>隞乩<E99A9E><E4B9A9><EFBFBD>捆<EFBFBD>?`backend/.env` <20><>辣銝哨<E98A9D>撟嗅‵<E59785>亦<EFBFBD>摰鮋<E691B0>蝵桀<E89DB5>?
|
||||
> **<EFBFBD>𥕦遣<EFBFBD>交<EFBFBD>嚗?* 2025-11-09
|
||||
|
||||
---
|
||||
|
||||
## <20><> 摰峕㟲<E5B395>滨蔭璅⊥踎
|
||||
|
||||
将以下内容复制到 `AIclinicalresearch/backend/.env` 文件:
|
||||
撠<EFBFBD>誑銝见<EFBFBD>摰孵<EFBFBD><EFBFBD>嗅<EFBFBD> `AIclinicalresearch/backend/.env` <EFBFBD><EFBFBD>辣嚗?
|
||||
|
||||
```env
|
||||
# ================================
|
||||
# 服务器配置
|
||||
# <EFBFBD>滚𦛚<EFBFBD>券<EFBFBD>蝵?
|
||||
# ================================
|
||||
PORT=3001
|
||||
HOST=0.0.0.0
|
||||
@@ -20,7 +20,7 @@ NODE_ENV=development
|
||||
LOG_LEVEL=info
|
||||
|
||||
# ================================
|
||||
# 数据库配置
|
||||
# <EFBFBD>唳旿摨㯄<EFBFBD>蝵?
|
||||
# ================================
|
||||
DATABASE_URL=postgresql://postgres:your_password@localhost:5432/ai_clinical_research
|
||||
|
||||
@@ -46,7 +46,7 @@ DEEPSEEK_BASE_URL=https://api.deepseek.com
|
||||
# ---------- Qwen (<28>朞<EFBFBD><E69C9E>輸<EFBFBD>鈭飇ashScope) ----------
|
||||
DASHSCOPE_API_KEY=sk-your-dashscope-api-key
|
||||
|
||||
# ---------- Gemini (可选) ----------
|
||||
# ---------- Gemini (<EFBFBD>舫<EFBFBD>? ----------
|
||||
GEMINI_API_KEY=your-gemini-api-key
|
||||
|
||||
# ================================
|
||||
@@ -55,7 +55,7 @@ GEMINI_API_KEY=your-gemini-api-key
|
||||
# CloseAI<41>臭<EFBFBD>銝服PI隞<49><E99A9E>撟喳蝱嚗峕<E59A97>靘𤤿迅摰𡁶<E691B0>OpenAI<41>龦laude霈輸䔮
|
||||
# 摰条<E691B0>嚗冴ttps://platform.openai-proxy.org
|
||||
|
||||
# 统一API Key(同时用于OpenAI和Claude)
|
||||
# 蝏煺<EFBFBD>API Key嚗<EFBFBD><EFBFBD><EFBFBD>嗥鍂鈭窻penAI<EFBFBD>龦laude嚗?
|
||||
CLOSEAI_API_KEY=sk-your-closeai-api-key
|
||||
|
||||
# OpenAI蝡舐<E89DA1>
|
||||
@@ -69,7 +69,7 @@ CLOSEAI_CLAUDE_BASE_URL=https://api.openai-proxy.org/anthropic
|
||||
# - Claude: claude-sonnet-4-5-20250929, claude-3-5-sonnet-20241022
|
||||
|
||||
# ================================
|
||||
# Dify配置(知识库RAG引擎)
|
||||
# Dify<EFBFBD>滨蔭嚗<EFBFBD>䰻霂<EFBFBD><EFBFBD>RAG撘閙<EFBFBD>嚗?
|
||||
# ================================
|
||||
DIFY_API_KEY=app-your-dify-api-key
|
||||
DIFY_API_URL=http://localhost/v1
|
||||
@@ -88,7 +88,7 @@ CORS_ORIGIN=http://localhost:5173
|
||||
# ================================
|
||||
# 瘜冽<E7989C>鈭钅★
|
||||
# ================================
|
||||
# 1. 请将 your-* 占位符替换为真实的配置值
|
||||
# 1. 霂瑕<EFBFBD> your-* <EFBFBD>牐<EFBFBD>蝚行𤜯<EFBFBD>V蛹<EFBFBD>笔<EFBFBD><EFBFBD><EFBFBD><EFBFBD>蝵桀<EFBFBD>?
|
||||
# 2. .env <20><>辣<EFBFBD><E8BEA3>鉄<EFBFBD>𤩺<EFBFBD>靽⊥<E99DBD>嚗䔶<E59A97>閬<EFBFBD><E996AC>鈭文<E988AD> git 隞枏<E99A9E>
|
||||
# 3. <20>煺漣<E785BA>臬<EFBFBD>霂瑚蝙<E7919A>典撩撖<E692A9><E69296><EFBFBD>𣬚𡠺蝡讠<E89DA1> API Key
|
||||
# 4. JWT_SECRET 撱箄悅雿輻鍂 32 雿滢誑銝𢠃<E98A9D><F0A2A083>箏<EFBFBD>蝚虫葡
|
||||
@@ -106,7 +106,7 @@ CLOSEAI_OPENAI_BASE_URL=https://api.openai-proxy.org/v1
|
||||
CLOSEAI_CLAUDE_BASE_URL=https://api.openai-proxy.org/anthropic
|
||||
```
|
||||
|
||||
**可用模型:**
|
||||
**<EFBFBD>舐鍂璅∪<EFBFBD>嚗?*
|
||||
- OpenAI: `gpt-5-pro`
|
||||
- Claude: `claude-sonnet-4-5-20250929`
|
||||
|
||||
@@ -120,18 +120,18 @@ CLOSEAI_CLAUDE_BASE_URL=https://api.openai-proxy.org/anthropic
|
||||
cd AIclinicalresearch/backend
|
||||
# 憭滚<E686AD>璅⊥踎
|
||||
copy .env.example .env # Windows
|
||||
# 或
|
||||
# <EFBFBD>?
|
||||
cp .env.example .env # Linux/Mac
|
||||
```
|
||||
|
||||
### 2. 憛怠<E6869B><E680A0>笔<EFBFBD><E7AC94>滨蔭
|
||||
|
||||
打开 `backend/.env` 文件,替换占位符:
|
||||
<EFBFBD>枏<EFBFBD> `backend/.env` <EFBFBD><EFBFBD>辣嚗峕𤜯<EFBFBD>W<EFBFBD>雿滨泵嚗?
|
||||
|
||||
**必须配置:**
|
||||
- `DATABASE_URL` - 数据库连接
|
||||
**敹<EFBFBD>◆<EFBFBD>滨蔭嚗?*
|
||||
- `DATABASE_URL` - <EFBFBD>唳旿摨栞<EFBFBD><EFBFBD>?
|
||||
- `DEEPSEEK_API_KEY` - DeepSeek API嚗<49>蜓<EFBFBD>𥟇芋<F0A59F87>页<EFBFBD>
|
||||
- `CLOSEAI_API_KEY` - CloseAI API(OpenAI+Claude)
|
||||
- `CLOSEAI_API_KEY` - CloseAI API嚗㇉penAI+Claude嚗?
|
||||
|
||||
**<EFBFBD>舫<EFBFBD>厰<EFBFBD>蝵殷<EFBFBD>**
|
||||
- `DASHSCOPE_API_KEY` - Qwen璅∪<E79285>
|
||||
@@ -154,7 +154,7 @@ npm run dev
|
||||
|
||||
### 銝滩<E98A9D><E6BBA9>𣂷漱<F0A382B7>蚣it
|
||||
|
||||
确认 `.gitignore` 包含:
|
||||
蝖株恕 `.gitignore` <EFBFBD><EFBFBD>鉄嚗?
|
||||
```gitignore
|
||||
# <20>臬<EFBFBD><E887AC>㗛<EFBFBD><E3979B><EFBFBD>辣
|
||||
.env
|
||||
@@ -164,14 +164,14 @@ npm run dev
|
||||
|
||||
### API Key摰匧<E691B0>
|
||||
|
||||
1. **定期轮换:** 每3个月更换一次API Key
|
||||
2. **权限最小化:** 只授予必要的权限
|
||||
3. **独立密钥:** 开发/测试/生产使用不同的Key
|
||||
4. **监控使用:** 定期检查API调用量和费用
|
||||
1. **摰𡁏<EFBFBD>頧格揢嚗?* 瘥?銝芣<E98A9D><E88AA3>湔揢銝<E68FA2>甈,PI Key
|
||||
2. **<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>撠誩<EFBFBD>嚗?* <20>芣<EFBFBD>鈭<EFBFBD><E988AD>閬<EFBFBD><E996AC><EFBFBD><EFBFBD><EFBFBD>
|
||||
3. **<EFBFBD>祉<EFBFBD>撖<EFBFBD>𤨎嚗?* 撘<><E69298>?瘚贝<E7989A>/<2F>煺漣雿輻鍂銝滚<E98A9D><E6BB9A><EFBFBD>ey
|
||||
4. **<EFBFBD>烐綉雿輻鍂嚗?* 摰𡁏<E691B0>璉<EFBFBD><E79289>丕PI靚<49>鍂<EFBFBD>誩<EFBFBD>韐寧鍂
|
||||
|
||||
### 泄露应急
|
||||
### 瘜<EFBFBD>蠧摨娍<EFBFBD>?
|
||||
|
||||
如果API Key不慎泄露:
|
||||
憒<EFBFBD><EFBFBD>API Key銝齿<EFBFBD>瘜<EFBFBD>蠧嚗?
|
||||
1. 蝡见朖<E8A781>冽<EFBFBD><E586BD>∪<EFBFBD><E288AA>𤾸蝱蝳<E89DB1>鍂/<2F>𣳇膄霂仕ey
|
||||
2. <20><><EFBFBD><EFBFBD>啁<EFBFBD>API Key
|
||||
3. <20>湔鰵 `.env` <20><>辣
|
||||
@@ -181,21 +181,21 @@ npm run dev
|
||||
|
||||
## <20><> <20>滨蔭撉諹<E69289>皜<EFBFBD><E79A9C>
|
||||
|
||||
部署前请确认:
|
||||
<EFBFBD>函蔡<EFBFBD>滩窈蝖株恕嚗?
|
||||
|
||||
- [ ] ✅ DATABASE_URL 配置正确且可连接
|
||||
- [ ] ✅ DEEPSEEK_API_KEY 已配置
|
||||
- [ ] ✅ CLOSEAI_API_KEY 已配置(用于GPT-5和Claude-4.5)
|
||||
- [ ] ✅ JWT_SECRET 已修改为强密码
|
||||
- [ ] ✅ CORS_ORIGIN 已设置正确的前端地址
|
||||
- [ ] ✅ .env 文件已添加到 .gitignore
|
||||
- [ ] ✅ 所有敏感信息未提交到Git
|
||||
- [ ] <EFBFBD>?DATABASE_URL <EFBFBD>滨蔭甇<EFBFBD>&銝𥪜虾餈墧𦻖
|
||||
- [ ] <EFBFBD>?DEEPSEEK_API_KEY 撌脤<EFBFBD>蝵?
|
||||
- [ ] <EFBFBD>?CLOSEAI_API_KEY 撌脤<EFBFBD>蝵殷<EFBFBD><EFBFBD>其<EFBFBD>GPT-5<EFBFBD>龦laude-4.5嚗?
|
||||
- [ ] <EFBFBD>?JWT_SECRET 撌脖耨<EFBFBD>嫣蛹撘箏<EFBFBD><EFBFBD>?
|
||||
- [ ] <EFBFBD>?CORS_ORIGIN 撌脰挽蝵格迤蝖桃<EFBFBD><EFBFBD>滨垢<EFBFBD>啣<EFBFBD>
|
||||
- [ ] <EFBFBD>?.env <EFBFBD><EFBFBD>辣撌脫溶<EFBFBD>惩<EFBFBD> .gitignore
|
||||
- [ ] <EFBFBD>?<3F><><EFBFBD>㗇<EFBFBD><E39787>煺縑<E785BA>舀𧊋<E88880>𣂷漱<F0A382B7>蚣it
|
||||
|
||||
---
|
||||
|
||||
**<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>獢<EFBFBD><EFBFBD>**
|
||||
- [01-环境配置指南.md](./01-环境配置指南.md) - 详细的配置说明
|
||||
- [数据库连接配置](../09-架构实施/02-数据库连接配置.md) - 数据库专项配置
|
||||
- [01-<EFBFBD>臬<EFBFBD><EFBFBD>滨蔭<EFBFBD><EFBFBD><EFBFBD>.md](./01-<EFBFBD>臬<EFBFBD><EFBFBD>滨蔭<EFBFBD><EFBFBD><EFBFBD>.md) - 霂衣<E99C82><E8A1A3><EFBFBD><EFBFBD>蝵株秩<E6A0AA>?
|
||||
- [<EFBFBD>唳旿摨栞<EFBFBD><EFBFBD>仿<EFBFBD>蝵孫(../09-<2D>嗆<EFBFBD>摰墧鴌/02-<2D>唳旿摨栞<E691A8><E6A09E>仿<EFBFBD>蝵?md) - <EFBFBD>唳旿摨㮖<EFBFBD>憿寥<EFBFBD>蝵?
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -1,23 +1,23 @@
|
||||
# SAE<41>臬<EFBFBD><E887AC>㗛<EFBFBD><E3979B>滨蔭<E6BBA8><E894AD><EFBFBD>
|
||||
|
||||
> **文档版本:** v1.0
|
||||
> **创建日期:** 2025-12-11
|
||||
> **适用场景:** 阿里云SAE部署环境变量配置
|
||||
> **使用方法:** 在SAE控制台逐行配置
|
||||
> **<EFBFBD><EFBFBD>﹝<EFBFBD><EFBFBD>𧋦嚗?* v1.0
|
||||
> **<EFBFBD>𥕦遣<EFBFBD>交<EFBFBD>嚗?* 2025-12-11
|
||||
> **<EFBFBD><EFBFBD>鍂<EFBFBD>箸艶嚗?* <20>輸<EFBFBD>鈭騌AE<41>函蔡<E587BD>臬<EFBFBD><E887AC>㗛<EFBFBD><E3979B>滨蔭
|
||||
> **雿輻鍂<EFBFBD>寞<EFBFBD>嚗?* <20>沒AE<41>批<EFBFBD><E689B9>圈<EFBFBD>鞱<EFBFBD><E99EB1>滨蔭
|
||||
|
||||
---
|
||||
|
||||
## <20><> <20>滨蔭霂湔<E99C82>
|
||||
|
||||
在阿里云SAE控制台配置环境变量时,按照以下顺序逐行添加:
|
||||
<EFBFBD>券燵<EFBFBD>䔶<EFBFBD>SAE<EFBFBD>批<EFBFBD><EFBFBD>圈<EFBFBD>蝵桃㴓憓<EFBFBD><EFBFBD><EFBFBD>𤩺𧒄嚗峕<EFBFBD><EFBFBD>找誑銝钅◇摨誯<EFBFBD>鞱<EFBFBD>瘛餃<EFBFBD>嚗?
|
||||
|
||||
### <20>滢<EFBFBD>甇仿炊
|
||||
|
||||
1. <20>餃<EFBFBD><E9A483>輸<EFBFBD>鈭烐綉<E78390>嗅蝱
|
||||
2. 餈𥕦<E9A488> Serverless摨𠉛鍂撘閙<E69298>SAE
|
||||
3. 选择应用 → 配置管理 → 环境变量
|
||||
4. 点击「添加环境变量」
|
||||
5. 逐行复制以下内容(替换所有"你的XXX")
|
||||
3. <EFBFBD>㗇𥋘摨𠉛鍂 <20>?<3F>滨蔭蝞∠<E89D9E> <20>?<3F>臬<EFBFBD><E887AC>㗛<EFBFBD>
|
||||
4. <EFBFBD>孵稬<EFBFBD>峕溶<EFBFBD>删㴓憓<EFBFBD><EFBFBD><EFBFBD>譌<EFBFBD>?
|
||||
5. <EFBFBD>鞱<EFBFBD>憭滚<EFBFBD>隞乩<EFBFBD><EFBFBD><EFBFBD>捆嚗<EFBFBD>𤜯<EFBFBD>X<EFBFBD><EFBFBD>?雿删<E99BBF>XXX"嚗?
|
||||
|
||||
---
|
||||
|
||||
@@ -32,21 +32,21 @@ SERVICE_NAME=aiclinical-backend-dev
|
||||
LOG_LEVEL=debug
|
||||
```
|
||||
|
||||
### 数据库配置
|
||||
### <EFBFBD>唳旿摨㯄<EFBFBD>蝵?
|
||||
|
||||
```bash
|
||||
# 格式:postgresql://用户名:密码@地址:端口/数据库名
|
||||
# <EFBFBD>澆<EFBFBD>嚗䮝ostgresql://<EFBFBD>冽<EFBFBD><EFBFBD>?撖<><E69296>@<40>啣<EFBFBD>:蝡臬藁/<2F>唳旿摨枏<E691A8>
|
||||
# 蝷箔<E89DB7>嚗䮝ostgresql://aiclinical:MyPass123@rm-bp1xxxx.mysql.rds.aliyuncs.com:5432/aiclinical_dev
|
||||
DATABASE_URL=postgresql://aiclinical:雿删<E99BBF>撖<EFBFBD><E69296>@雿删<E99BBF>RDS<44><53><EFBFBD><EFBFBD>啣<EFBFBD>:5432/aiclinical_dev
|
||||
|
||||
# Serverless连接池优化
|
||||
# Serverless餈墧𦻖瘙牐<EFBFBD><EFBFBD>?
|
||||
DB_MAX_CONNECTIONS=400
|
||||
MAX_INSTANCES=10
|
||||
```
|
||||
|
||||
**获取RDS地址:**
|
||||
1. RDS控制台 → 实例列表 → 点击实例ID
|
||||
2. 基本信息 → 内网地址(复制)
|
||||
**<EFBFBD>瑕<EFBFBD>RDS<EFBFBD>啣<EFBFBD>嚗?*
|
||||
1. RDS<EFBFBD>批<EFBFBD><EFBFBD>?<3F>?摰硺<E691B0><E7A1BA>𡑒” <20>?<3F>孵稬摰硺<E691B0>ID
|
||||
2. <EFBFBD>箸𧋦靽⊥<EFBFBD> <20>?<3F><><EFBFBD><EFBFBD>啣<EFBFBD>嚗<EFBFBD><E59A97><EFBFBD>塚<EFBFBD>
|
||||
3. 蝷箔<E89DB7>嚗䫤rm-bp1abcd1234.mysql.rds.aliyuncs.com`
|
||||
|
||||
### OSS摮睃<E691AE><E79D83>滨蔭
|
||||
@@ -59,10 +59,10 @@ OSS_ACCESS_KEY_ID=你的AccessKeyId
|
||||
OSS_ACCESS_KEY_SECRET=雿删<EFBFBD>AccessKeySecret
|
||||
```
|
||||
|
||||
**获取OSS密钥:**
|
||||
1. 访问控制RAM → 用户 → `aiclinical-oss`
|
||||
**<EFBFBD>瑕<EFBFBD>OSS撖<EFBFBD>𤨎嚗?*
|
||||
1. 霈輸䔮<EFBFBD>批<EFBFBD>RAM <20>?<3F>冽<EFBFBD> <20>?`aiclinical-oss`
|
||||
2. 憒<><E68692>敹䁅扇撖<E68987>𤨎嚗屸<E59A97>閬<EFBFBD><E996AC><EFBFBD>啣<EFBFBD>撱態ccessKey
|
||||
3. **重要:** 密钥只显示一次,立即保存!
|
||||
3. **<EFBFBD>滩<EFBFBD>嚗?* 撖<>𤨎<EFBFBD>芣遬蝷箔<E89DB7>甈∴<E79488>蝡见朖靽嘥<E99DBD>嚗?
|
||||
|
||||
### LLM API<50>滨蔭
|
||||
|
||||
@@ -71,7 +71,7 @@ OSS_ACCESS_KEY_SECRET=你的AccessKeySecret
|
||||
DEEPSEEK_API_KEY=sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
||||
DEEPSEEK_BASE_URL=https://api.deepseek.com
|
||||
|
||||
# 通义千问(阿里云)
|
||||
# <EFBFBD>帋<EFBFBD><EFBFBD><EFBFBD>䔮嚗<EFBFBD>燵<EFBFBD>䔶<EFBFBD>嚗?
|
||||
DASHSCOPE_API_KEY=sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
||||
|
||||
# CloseAI隞<49><E99A9E>嚗<EFBFBD>虾<EFBFBD>㚁<EFBFBD>
|
||||
@@ -80,14 +80,14 @@ CLOSEAI_OPENAI_BASE_URL=https://api.openai-proxy.org/v1
|
||||
CLOSEAI_CLAUDE_BASE_URL=https://api.openai-proxy.org/anthropic
|
||||
```
|
||||
|
||||
**至少配置一个LLM API Key!**
|
||||
**<EFBFBD>喳<EFBFBD><EFBFBD>滨蔭銝<EFBFBD>銝林LM API Key嚗?*
|
||||
|
||||
### 摰匧<E691B0><E58CA7>滨蔭
|
||||
|
||||
```bash
|
||||
# JWT密钥(必须修改!)
|
||||
# JWT撖<EFBFBD>𤨎嚗<EFBFBD><EFBFBD>憿颱耨<EFBFBD>對<EFBFBD>嚗?
|
||||
# <20><><EFBFBD>撌亙<E6928C>嚗冴ttps://www.random.org/strings/
|
||||
JWT_SECRET=请改为32位以上随机字符串abcdefg123456
|
||||
JWT_SECRET=霂瑟㺿銝?2雿滢誑銝𢠃<E98A9D><F0A2A083>箏<EFBFBD>蝚虫葡abcdefg123456
|
||||
JWT_EXPIRES_IN=7d
|
||||
|
||||
# CORS<52>滨蔭
|
||||
@@ -100,16 +100,16 @@ CORS_ORIGIN=*
|
||||
|
||||
## <20>辷<EFBFBD> <20>刻<EFBFBD><E588BB>滨蔭
|
||||
|
||||
### 缓存配置(初期不使用Redis)
|
||||
### 蝻枏<EFBFBD><EFBFBD>滨蔭嚗<EFBFBD><EFBFBD><EFBFBD>煺<EFBFBD>雿輻鍂Redis嚗?
|
||||
|
||||
```bash
|
||||
CACHE_TYPE=memory
|
||||
QUEUE_TYPE=memory
|
||||
```
|
||||
|
||||
**说明:** 初期用户量小,使用内存缓存足够
|
||||
**霂湔<EFBFBD>嚗?* <20>脲<EFBFBD><E884B2>冽<EFBFBD><E586BD>誩<EFBFBD>嚗䔶蝙<E494B6>典<EFBFBD>摮条<E691AE>摮䁅雲憭?
|
||||
|
||||
**未来需要Redis时,改为:**
|
||||
**<EFBFBD>芣䔉<EFBFBD><EFBFBD>閬<EFBFBD>edis<EFBFBD>塚<EFBFBD><EFBFBD>嫣蛹嚗?*
|
||||
```bash
|
||||
CACHE_TYPE=redis
|
||||
REDIS_HOST=r-bp1xxxx.redis.rds.aliyuncs.com
|
||||
@@ -125,9 +125,9 @@ DIFY_API_KEY=app-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
||||
DIFY_API_URL=https://api.dify.ai/v1
|
||||
```
|
||||
|
||||
**说明:** 如果使用Dify提供RAG服务,需要配置
|
||||
**霂湔<EFBFBD>嚗?* 憒<><E68692>雿輻鍂Dify<66>𣂷<EFBFBD>RAG<41>滚𦛚嚗屸<E59A97>閬<EFBFBD><E996AC>蝵?
|
||||
|
||||
### Python微服务配置
|
||||
### Python敺格<EFBFBD><EFBFBD>⊿<EFBFBD>蝵?
|
||||
|
||||
```bash
|
||||
# 銝湔𧒄<E6B994>寞<EFBFBD>嚗𡁜<E59A97><F0A1819C>典<EFBFBD>蝵穃𧑐<E7A983><F0A79190>
|
||||
@@ -143,47 +143,47 @@ EXTRACTION_SERVICE_URL=http://你的临时地址:8000
|
||||
UPLOAD_MAX_SIZE=104857600
|
||||
```
|
||||
|
||||
**说明:** 100MB = 104857600 bytes
|
||||
**霂湔<EFBFBD>嚗?* 100MB = 104857600 bytes
|
||||
|
||||
---
|
||||
|
||||
## ✅ 配置检查清单
|
||||
## <EFBFBD>?<3F>滨蔭璉<E894AD><E79289>交<EFBFBD><E4BAA4>?
|
||||
|
||||
### 第一步:复制粘贴检查
|
||||
### 蝚砌<EFBFBD>甇伐<EFBFBD>憭滚<EFBFBD>蝎䁅斐璉<EFBFBD><EFBFBD>?
|
||||
|
||||
- [ ] <20><><EFBFBD>厩㴓憓<E3B493><E68693><EFBFBD>誩歇瘛餃<E7989B><E9A483>訕AE
|
||||
- [ ] 所有"你的XXX"已替换为真实值
|
||||
- [ ] 没有遗漏任何必填项
|
||||
- [ ] <EFBFBD><EFBFBD><EFBFBD>?雿删<E99BBF>XXX"撌脫𤜯<E884AB>V蛹<EFBCB6>笔<EFBFBD><E7AC94>?
|
||||
- [ ] 瘝⊥<EFBFBD><EFBFBD>埈<EFBFBD>隞颱<EFBFBD>敹<EFBFBD>‵憿?
|
||||
|
||||
### 第二步:格式检查
|
||||
### 蝚砌<EFBFBD>甇伐<EFBFBD><EFBFBD>澆<EFBFBD>璉<EFBFBD><EFBFBD>?
|
||||
|
||||
- [ ] DATABASE_URL <20>澆<EFBFBD>甇<EFBFBD>&
|
||||
```
|
||||
postgresql://用户名:密码@地址:端口/数据库
|
||||
✅ 正确:postgresql://aiclinical:MyPass@rm-xxx.com:5432/db
|
||||
❌ 错误:postgresql://aiclinical@rm-xxx.com:5432/db(缺少密码)
|
||||
postgresql://<EFBFBD>冽<EFBFBD><EFBFBD>?撖<><E69296>@<40>啣<EFBFBD>:蝡臬藁/<2F>唳旿摨?
|
||||
<EFBFBD>?甇<>&嚗䮝ostgresql://aiclinical:MyPass@rm-xxx.com:5432/db
|
||||
<EFBFBD>?<3F>躰秤嚗䮝ostgresql://aiclinical@rm-xxx.com:5432/db嚗<EFBFBD>撩撠穃<EFBFBD><EFBFBD><EFBFBD><EFBFBD>
|
||||
```
|
||||
|
||||
- [ ] 密码中没有特殊字符(`@ # $ % & 空格`)
|
||||
- [ ] 撖<EFBFBD><EFBFBD>銝剜瓷<EFBFBD>厩鸌畾𠰴<EFBFBD>蝚佗<EFBFBD>`@ # $ % & 蝛箸聢`嚗?
|
||||
```
|
||||
✅ 推荐:MyPassword123
|
||||
❌ 避免:My@Pass#123(包含@和#)
|
||||
<EFBFBD>?<3F>刻<EFBFBD>嚗鐝yPassword123
|
||||
<EFBFBD>?<3F>踹<EFBFBD>嚗鐝y@Pass#123嚗<EFBFBD><EFBFBD><EFBFBD>陂<EFBFBD>?嚗?
|
||||
```
|
||||
|
||||
- [ ] JWT_SECRET 撌脖耨<E88496>對<EFBFBD>銝齿糓暺䁅恕<E48185>潘<EFBFBD>
|
||||
- [ ] OSS_REGION 格式正确(带 `oss-` 前缀)
|
||||
- [ ] OSS_REGION <EFBFBD>澆<EFBFBD>甇<EFBFBD>&嚗<EFBFBD>蒂 `oss-` <EFBFBD>滨<EFBFBD>嚗?
|
||||
```
|
||||
✅ 正确:oss-cn-hangzhou
|
||||
❌ 错误:cn-hangzhou
|
||||
<EFBFBD>?甇<>&嚗駅ss-cn-hangzhou
|
||||
<EFBFBD>?<3F>躰秤嚗䬙n-hangzhou
|
||||
```
|
||||
|
||||
### 第三步:密钥有效性检查
|
||||
### 蝚砌<EFBFBD>甇伐<EFBFBD>撖<EFBFBD>𤨎<EFBFBD>㗇<EFBFBD><EFBFBD>扳<EFBFBD><EFBFBD>?
|
||||
|
||||
- [ ] RDS密码正确(可以用数据库客户端测试连接)
|
||||
- [ ] RDS撖<EFBFBD><EFBFBD>甇<EFBFBD>&嚗<EFBFBD>虾隞亦鍂<EFBFBD>唳旿摨枏恥<EFBFBD>瑞垢瘚贝<EFBFBD>餈墧𦻖嚗?
|
||||
- [ ] OSS AccessKey<65>㗇<EFBFBD>嚗<EFBFBD>銁RAM<41>批<EFBFBD><E689B9>啁&霈歹<E99C88>
|
||||
- [ ] LLM API Key有效(可以用curl测试)
|
||||
- [ ] LLM API Key<EFBFBD>㗇<EFBFBD>嚗<EFBFBD>虾隞亦鍂curl瘚贝<EFBFBD>嚗?
|
||||
|
||||
**测试LLM API Key:**
|
||||
**瘚贝<EFBFBD>LLM API Key嚗?*
|
||||
```bash
|
||||
curl https://api.deepseek.com/v1/models \
|
||||
-H "Authorization: Bearer sk-雿删<E99BBF>撖<EFBFBD>𤨎"
|
||||
@@ -191,7 +191,7 @@ curl https://api.deepseek.com/v1/models \
|
||||
|
||||
---
|
||||
|
||||
## 📝 配置示例(脱敏版)
|
||||
## <EFBFBD><EFBFBD> <20>滨蔭蝷箔<E89DB7>嚗<EFBFBD><E59A97><EFBFBD>讐<EFBFBD>嚗?
|
||||
|
||||
```bash
|
||||
NODE_ENV=development
|
||||
@@ -228,21 +228,21 @@ UPLOAD_MAX_SIZE=104857600
|
||||
|
||||
---
|
||||
|
||||
## 🔐 安全最佳实践
|
||||
## <EFBFBD><EFBFBD> 摰匧<E691B0><E58CA7><EFBFBD>雿喳<E99BBF>頝?
|
||||
|
||||
### 撖<>𤨎蝞∠<E89D9E>
|
||||
|
||||
1. **銝滩<E98A9D>撠<EFBFBD><E692A0><EFBFBD>交<EFBFBD>鈭文<E988AD>Git**
|
||||
- ❌ 不要创建 `.env.production` 文件
|
||||
- ❌ 不要在代码中硬编码密钥
|
||||
- ✅ 只在SAE控制台配置
|
||||
- <EFBFBD>?銝滩<E98A9D><E6BBA9>𥕦遣 `.env.production` <EFBFBD><EFBFBD>辣
|
||||
- <EFBFBD>?銝滩<E98A9D><E6BBA9>其誨<E585B6><E8AAA8>葉蝖祉<E89D96><E7A589><EFBFBD><EFBFBD><EFBFBD>?
|
||||
- <EFBFBD>?<3F>芸銁SAE<41>批<EFBFBD><E689B9>圈<EFBFBD>蝵?
|
||||
|
||||
2. **摰𡁏<E691B0><F0A1818F>湔揢撖<E68FA2>𤨎**
|
||||
- 每3-6个月更换一次
|
||||
- 瘥?-6銝芣<E98A9D><E88AA3>湔揢銝<E68FA2>甈?
|
||||
- <20>𤑳緵瘜<E7B7B5>蠧蝡见朖<E8A781>湔揢
|
||||
|
||||
3. **使用密码管理器**
|
||||
- 推荐:1Password、LastPass、Bitwarden
|
||||
3. **雿輻鍂撖<EFBFBD><EFBFBD>蝞∠<EFBFBD><EFBFBD>?*
|
||||
- <EFBFBD>刻<EFBFBD>嚗?Password<EFBFBD><EFBFBD>astPass<EFBFBD><EFBFBD>itwarden
|
||||
- 靽嘥<E99DBD><E598A5><EFBFBD><EFBFBD>匧<EFBFBD><E58CA7>亙<EFBFBD><E4BA99>滨蔭靽⊥<E99DBD>
|
||||
|
||||
### <20>臬<EFBFBD><E887AC>𠉛氖
|
||||
@@ -251,15 +251,15 @@ UPLOAD_MAX_SIZE=104857600
|
||||
撘<EFBFBD><EFBFBD>𤑳㴓憓<EFBFBD><EFBFBD>
|
||||
- Bucket: aiclinical-dev
|
||||
- Database: aiclinical_dev
|
||||
- JWT_SECRET: 独立的密钥
|
||||
- JWT_SECRET: <EFBFBD>祉<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>?
|
||||
|
||||
生产环境:
|
||||
<EFBFBD>煺漣<EFBFBD>臬<EFBFBD>嚗?
|
||||
- Bucket: aiclinical-prod
|
||||
- Database: aiclinical_prod
|
||||
- JWT_SECRET: 不同的密钥
|
||||
- JWT_SECRET: 銝滚<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>?
|
||||
```
|
||||
|
||||
**永远不要在生产环境使用开发环境的密钥!**
|
||||
**瘞貉<EFBFBD>銝滩<EFBFBD><EFBFBD>函<EFBFBD>鈭抒㴓憓<EFBFBD>蝙<EFBFBD>典<EFBFBD><EFBFBD>𤑳㴓憓<EFBFBD><EFBFBD>撖<EFBFBD>𤨎嚗?*
|
||||
|
||||
---
|
||||
|
||||
@@ -267,56 +267,56 @@ UPLOAD_MAX_SIZE=104857600
|
||||
|
||||
### Q1: 敹䁅扇鈭<E68987>DS撖<53><E69296><EFBFBD>𦒘<EFBFBD><F0A69298>痹<EFBFBD>
|
||||
|
||||
**解决方法:**
|
||||
1. RDS控制台 → 账号管理
|
||||
**閫<EFBFBD><EFBFBD><EFBFBD>寞<EFBFBD>嚗?*
|
||||
1. RDS<EFBFBD>批<EFBFBD><EFBFBD>?<3F>?韐血噡蝞∠<E89D9E>
|
||||
2. <20>曉<EFBFBD><E69B89>冽<EFBFBD> `aiclinical`
|
||||
3. 点击「重置密码」
|
||||
4. 设置新密码
|
||||
3. <EFBFBD>孵稬<EFBFBD>屸<EFBFBD>蝵桀<EFBFBD><EFBFBD><EFBFBD><EFBFBD>?
|
||||
4. 霈曄蔭<EFBFBD>啣<EFBFBD><EFBFBD>?
|
||||
5. <20>湔鰵SAE<41>臬<EFBFBD><E887AC>㗛<EFBFBD>銝剔<E98A9D> `DATABASE_URL`
|
||||
|
||||
### Q2: OSS AccessKey瘜<79>蠧鈭<E8A0A7><E988AD>𦒘<EFBFBD><F0A69298>痹<EFBFBD>
|
||||
|
||||
**解决方法:**
|
||||
1. RAM控制台 → 用户 → `aiclinical-oss`
|
||||
**閫<EFBFBD><EFBFBD><EFBFBD>寞<EFBFBD>嚗?*
|
||||
1. RAM<EFBFBD>批<EFBFBD><EFBFBD>?<3F>?<3F>冽<EFBFBD> <20>?`aiclinical-oss`
|
||||
2. 蝳<>鍂<EFBFBD>硋<EFBFBD><E7A18B>斗<EFBFBD><E69697>脩<EFBFBD>AccessKey
|
||||
3. <20>𥕦遣<F0A595A6>啁<EFBFBD>AccessKey
|
||||
4. <20>湔鰵SAE<41>臬<EFBFBD><E887AC>㗛<EFBFBD>
|
||||
|
||||
### Q3: 如何验证环境变量配置正确?
|
||||
### Q3: 憒<EFBFBD><EFBFBD>撉諹<EFBFBD><EFBFBD>臬<EFBFBD><EFBFBD>㗛<EFBFBD><EFBFBD>滨蔭甇<EFBFBD>&嚗?
|
||||
|
||||
**解决方法:**
|
||||
**閫<EFBFBD><EFBFBD><EFBFBD>寞<EFBFBD>嚗?*
|
||||
1. <20>函蔡摨𠉛鍂<F0A0899B>𠬍<EFBFBD><F0A0AC8D>亦<EFBFBD>摰墧𧒄<E5A2A7>亙<EFBFBD>
|
||||
2. 看到以下日志表示配置正确:
|
||||
2. <EFBFBD>见<EFBFBD>隞乩<EFBFBD><EFBFBD>亙<EFBFBD>銵函內<EFBFBD>滨蔭甇<EFBFBD>&嚗?
|
||||
```
|
||||
✅ [Config] Environment validation passed
|
||||
✅ [Database] 数据库连接成功
|
||||
📦 [Storage] 使用阿里云 OSS 存储
|
||||
<EFBFBD>?[Config] Environment validation passed
|
||||
<EFBFBD>?[Database] <EFBFBD>唳旿摨栞<EFBFBD><EFBFBD>交<EFBFBD><EFBFBD>?
|
||||
<EFBFBD>𣑐 [Storage] 雿輻鍂<EFBFBD>輸<EFBFBD>鈭?OSS 摮睃<EFBFBD>
|
||||
```
|
||||
|
||||
### Q4: DATABASE_URL銝剖<E98A9D><E58996><EFBFBD><EFBFBD><EFBFBD>怎鸌畾𠰴<E795BE>蝚行<E89D9A>𦒘<EFBFBD><F0A69298>痹<EFBFBD>
|
||||
|
||||
**解决方法:**
|
||||
如果密码包含 `@ # $ % & 空格` 等特殊字符,需要URL编码:
|
||||
**閫<EFBFBD><EFBFBD><EFBFBD>寞<EFBFBD>嚗?*
|
||||
憒<EFBFBD><EFBFBD>撖<EFBFBD><EFBFBD><EFBFBD><EFBFBD>鉄 `@ # $ % & 蝛箸聢` 蝑厩鸌畾𠰴<E795BE>蝚佗<E89D9A><E4BD97><EFBFBD>閬<EFBFBD>RL蝻𣇉<E89DBB>嚗?
|
||||
|
||||
```
|
||||
<EFBFBD>笔<EFBFBD><EFBFBD><EFBFBD><EFBFBD>My@Pass#123
|
||||
蝻𣇉<EFBFBD><EFBFBD>𠬍<EFBFBD>My%40Pass%23123
|
||||
|
||||
完整URL:
|
||||
摰峕㟲URL嚗?
|
||||
postgresql://aiclinical:My%40Pass%23123@rm-xxx.com:5432/aiclinical_dev
|
||||
```
|
||||
|
||||
**蝻𣇉<E89DBB>撖寧<E69296>銵剁<E98AB5>**
|
||||
```
|
||||
@ → %40
|
||||
# → %23
|
||||
$ → %24
|
||||
% → %25
|
||||
& → %26
|
||||
空格 → %20
|
||||
@ <EFBFBD>?%40
|
||||
# <EFBFBD>?%23
|
||||
$ <EFBFBD>?%24
|
||||
% <EFBFBD>?%25
|
||||
& <EFBFBD>?%26
|
||||
蝛箸聢 <20>?%20
|
||||
```
|
||||
|
||||
**推荐:** 重新设置不包含特殊字符的密码更简单
|
||||
**<EFBFBD>刻<EFBFBD>嚗?* <20>齿鰵霈曄蔭銝滚<E98A9D><E6BB9A>怎鸌畾𠰴<E795BE>蝚衣<E89D9A>撖<EFBFBD><E69296><EFBFBD>渡<EFBFBD><E6B8A1>?
|
||||
|
||||
---
|
||||
|
||||
@@ -331,9 +331,9 @@ $ → %24
|
||||
// 隡朞䌊<E69C9E>冽<EFBFBD><E586BD>交<EFBFBD><E4BAA4>匧<EFBFBD>憛恍★
|
||||
```
|
||||
|
||||
**日志输出示例:**
|
||||
**<EFBFBD>亙<EFBFBD>颲枏枂蝷箔<EFBFBD>嚗?*
|
||||
```
|
||||
✅ [Config] Environment validation passed
|
||||
<EFBFBD>?[Config] Environment validation passed
|
||||
[Config] Application configuration:
|
||||
- Environment: development
|
||||
- Port: 3001
|
||||
@@ -346,7 +346,7 @@ $ → %24
|
||||
### <20>见𢆡撉諹<E69289>
|
||||
|
||||
```bash
|
||||
# 访问健康检查接口
|
||||
# 霈輸䔮<EFBFBD>亙熒璉<EFBFBD><EFBFBD>交𦻖<EFBFBD>?
|
||||
curl http://雿删<E99BBF>SAE<41>啣<EFBFBD>:3001/health
|
||||
|
||||
# 憸<><E686B8>餈𥪜<E9A488>
|
||||
@@ -361,10 +361,11 @@ curl http://你的SAE地址:3001/health
|
||||
|
||||
---
|
||||
|
||||
**文档版本:** v1.0
|
||||
**<EFBFBD><EFBFBD>﹝<EFBFBD><EFBFBD>𧋦嚗?* v1.0
|
||||
**<EFBFBD><EFBFBD><EFBFBD>擧凒<EFBFBD>堆<EFBFBD>** 2025-12-11
|
||||
**蝏湔擪<E6B994><E693AA><EFBFBD>** <20><><EFBFBD>舀沲<E88880><E6B2B2><EFBFBD>
|
||||
**相关文档:** [SAE部署完全指南](../05-部署文档/02-SAE部署完全指南(产品经理版).md)
|
||||
**<EFBFBD>詨<EFBFBD><EFBFBD><EFBFBD>﹝嚗?* [SAE<EFBFBD>函蔡摰<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>](../05-<EFBFBD>函蔡<EFBFBD><EFBFBD>﹝/02-SAE<EFBFBD>函蔡摰<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>(鈭批<E988AD>蝏讐<E89D8F><E8AE90>?.md)
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,18 +1,18 @@
|
||||
# Redis蝻枏<E89DBB> vs Redis<69>笔<EFBFBD> - 霂衣<E99C82>霂湔<E99C82>
|
||||
|
||||
> **文档版本:** V1.0
|
||||
> **创建日期:** 2025-12-12
|
||||
> **目标读者:** 产品经理、开发人员
|
||||
> **目的:** 澄清Redis缓存和Redis队列的区别
|
||||
> **<EFBFBD><EFBFBD>﹝<EFBFBD><EFBFBD>𧋦嚗?* V1.0
|
||||
> **<EFBFBD>𥕦遣<EFBFBD>交<EFBFBD>嚗?* 2025-12-12
|
||||
> **<EFBFBD>格<EFBFBD>霂餉<EFBFBD><EFBFBD><EFBFBD>** 鈭批<E988AD>蝏讐<E89D8F><E8AE90><EFBFBD><EFBFBD><EFBFBD>睲犖<E79DB2>?
|
||||
> **<EFBFBD>桃<EFBFBD>嚗?* 瞉<><E79E89>Redis蝻枏<E89DBB><E69E8F>朙edis<69>笔<EFBFBD><E7AC94><EFBFBD>躹<EFBFBD>?
|
||||
|
||||
---
|
||||
|
||||
## <20><> <20>桀<EFBFBD>
|
||||
|
||||
1. [<EFBFBD>詨<EFBFBD>璁<EFBFBD>艙](#1-<2D>詨<EFBFBD>璁<EFBFBD>艙)
|
||||
2. [当前系统真实状态](#2-当前系统真实状态)
|
||||
2. [敶枏<EFBFBD>蝟餌<EFBFBD><EFBFBD>笔<EFBFBD><EFBFBD>嗆<EFBFBD><EFBFBD>(#2-敶枏<E695B6>蝟餌<E89D9F><E9A48C>笔<EFBFBD><E7AC94>嗆<EFBFBD>?
|
||||
3. [Redis蝻枏<EFBFBD> vs Redis<69>笔<EFBFBD>撖寞<E69296>](#3-redis蝻枏<E89DBB>-vs-redis<69>笔<EFBFBD>撖寞<E69296>)
|
||||
4. [只用Redis缓存的影响](#4-只用redis缓存的影响)
|
||||
4. [<EFBFBD>芰鍂Redis蝻枏<EFBFBD><EFBFBD><EFBFBD>蔣<EFBFBD>䓞(#4-<2D>芰鍂redis蝻枏<E89DBB><E69E8F><EFBFBD>蔣<EFBFBD>?
|
||||
5. [<EFBFBD>舐鍂Redis<EFBFBD>笔<EFBFBD><EFBFBD><EFBFBD>𤣰<EFBFBD>𠹺<EFBFBD>憌𡡞埯](#5-<2D>舐鍂redis<69>笔<EFBFBD><E7AC94><EFBFBD>𤣰<EFBFBD>𠹺<EFBFBD>憌𡡞埯)
|
||||
6. [<EFBFBD>刻<EFBFBD><EFBFBD>寞<EFBFBD>](#6-<2D>刻<EFBFBD><E588BB>寞<EFBFBD>)
|
||||
|
||||
@@ -20,60 +20,60 @@
|
||||
|
||||
## 1. <20>詨<EFBFBD>璁<EFBFBD>艙
|
||||
|
||||
### 1.1 什么是Redis?
|
||||
### 1.1 隞<EFBFBD>銋<EFBFBD>糓Redis嚗?
|
||||
|
||||
**Redis = 銝<>銝芷<E98A9D><E88AB7>扯<EFBFBD><E689AF><EFBFBD><EFBFBD>摮䀹㺭<E480B9>桀<EFBFBD>**
|
||||
|
||||
<EFBFBD>臭誑<EFBFBD><EFBFBD>圾銝綽<EFBFBD>
|
||||
- 🏪 **超大的共享内存存储空间**
|
||||
- 🚀 **读写速度极快**(1ms内)
|
||||
- 💾 **支持数据持久化**
|
||||
- 🌐 **多个服务器可以共享**
|
||||
- <EFBFBD>蘨 **頞<>之<EFBFBD><E4B98B><EFBFBD>鈭怠<E988AD>摮睃<E691AE><E79D83>函征<E587BD>?*
|
||||
- <EFBFBD><EFBFBD> **霂餃<E99C82><E9A483>笔漲<E7AC94><E6BCB2>翰**嚗?ms<6D><73><EFBFBD>
|
||||
- <EFBFBD>𠒣 **<EFBFBD>舀<EFBFBD><EFBFBD>唳旿<EFBFBD><EFBFBD><EFBFBD><EFBFBD>?*
|
||||
- <EFBFBD><EFBFBD> **憭帋葵<E5B88B>滚𦛚<E6BB9A>典虾隞亙<E99A9E>鈭?*
|
||||
|
||||
Redis本身是一个**工具箱**,里面有很多工具(数据结构):
|
||||
- String(字符串)—— 用于缓存
|
||||
- List(列表)—— 用于队列
|
||||
Redis<EFBFBD>祈澈<EFBFBD>臭<EFBFBD>銝?*撌亙<E6928C>蝞?*嚗屸<E59A97><E5B1B8>X<EFBFBD>敺<EFBFBD><E695BA>撌亙<E6928C>嚗<EFBFBD>㺭<EFBFBD>桃<EFBFBD><E6A183><EFBFBD><EFBFBD>嚗?
|
||||
- String嚗<EFBFBD><EFBFBD>蝚虫葡嚗争<EFBFBD>婙<EFBFBD>?<3F>其<EFBFBD>蝻枏<E89DBB>
|
||||
- List嚗<EFBFBD><EFBFBD>銵剁<EFBFBD><EFBFBD>婙<EFBFBD>?<3F>其<EFBFBD><E585B6>笔<EFBFBD>
|
||||
- Set嚗<74><E59A97><EFBFBD><EFBFBD><EFBFBD>
|
||||
- Hash(哈希表)
|
||||
- Hash嚗<EFBFBD><EFBFBD>撣諹”嚗?
|
||||
- Sorted Set嚗<74><E59A97>摨誯<E691A8><E8AAAF><EFBFBD><EFBFBD>
|
||||
|
||||
### 1.2 Redis蝻枏<E89DBB> vs Redis<69>笔<EFBFBD>
|
||||
|
||||
这两个概念是**使用Redis的不同方式**,不是两个不同的产品!
|
||||
餈嗘舅銝芣<EFBFBD>敹菜糓**雿輻鍂Redis<69><73><EFBFBD><EFBFBD>峕䲮撘?*嚗䔶<E59A97><E494B6>臭舅銝芯<E98A9D><E88AAF>𣬚<EFBFBD>鈭批<E988AD>嚗?
|
||||
|
||||
```
|
||||
<EFBFBD>輸<EFBFBD>鈭駵edis摰硺<EFBFBD>嚗<EFBFBD><EFBFBD>韐凋僭<EFBFBD><EFBFBD><EFBFBD>
|
||||
↓
|
||||
可以同时用于:
|
||||
├─ Redis缓存(用String数据结构)
|
||||
<EFBFBD>?
|
||||
<EFBFBD>臭誑<EFBFBD>峕𧒄<EFBFBD>其<EFBFBD>嚗?
|
||||
<EFBFBD>鎿<EFBFBD> Redis蝻枏<EFBFBD>嚗<EFBFBD>鍂String<EFBFBD>唳旿蝏𤘪<EFBFBD>嚗?
|
||||
<20>婙<EFBFBD> Redis<69>笔<EFBFBD>嚗<EFBFBD>鍂List<73>唳旿蝏𤘪<E89D8F> + BullMQ摨橒<E691A8>
|
||||
```
|
||||
|
||||
**类比理解**:
|
||||
**蝐餅<EFBFBD><EFBFBD><EFBFBD>圾**嚗?
|
||||
```
|
||||
Redis = 一栋大楼(您购买的)
|
||||
Redis = 銝<EFBFBD><EFBFBD>见之璆潘<EFBFBD><EFBFBD>刻揚銋啁<EFBFBD>嚗?
|
||||
|
||||
Redis蝻枏<EFBFBD> = 憭扳未<E689B3>𣬚<EFBFBD>"敹恍<E695B9>埝<EFBFBD>"
|
||||
- 摮䀝<E691AE>镼踴<E995BC><E8B8B4><EFBFBD>銝𡏭正
|
||||
- 有过期时间
|
||||
- 用途:快速查询
|
||||
- <EFBFBD>㕑<EFBFBD><EFBFBD><EFBFBD>𧒄<EFBFBD>?
|
||||
- <EFBFBD>券<EFBFBD>䈑<EFBFBD>敹恍<EFBFBD><EFBFBD>䰻霂?
|
||||
|
||||
Redis<EFBFBD>笔<EFBFBD> = 憭扳未<E689B3>𣬚<EFBFBD>"隡𣳇<E99AA1><F0A3B387>蒂"
|
||||
- 隞餃𦛚<E9A483>㘾<EFBFBD>
|
||||
- 按顺序处理
|
||||
- <EFBFBD>厰◇摨誩<EFBFBD><EFBFBD>?
|
||||
- <20>券<EFBFBD>䈑<EFBFBD>撘<EFBFBD>郊隞餃𦛚
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. 当前系统真实状态
|
||||
## 2. 敶枏<EFBFBD>蝟餌<EFBFBD><EFBFBD>笔<EFBFBD><EFBFBD>嗆<EFBFBD>?
|
||||
|
||||
### 2.1 代码检查结果 ✅
|
||||
### 2.1 隞<EFBFBD><EFBFBD>璉<EFBFBD><EFBFBD>亦<EFBFBD><EFBFBD>?<3F>?
|
||||
|
||||
#### **发现1:BullMQ已安装但未使用**
|
||||
#### **<EFBFBD>𤑳緵1嚗鋳ullMQ撌脣<EFBFBD>鋆<EFBFBD><EFBFBD><EFBFBD>芯蝙<EFBFBD>?*
|
||||
```json
|
||||
// backend/package.json (第36行)
|
||||
"bullmq": "^5.65.0", // ← 已安装但代码中未使用
|
||||
// backend/package.json (蝚?6銵?
|
||||
"bullmq": "^5.65.0", // <EFBFBD>?撌脣<E6928C>鋆<EFBFBD><E98B86>隞<EFBFBD><E99A9E>銝剜𧊋雿輻鍂
|
||||
```
|
||||
|
||||
#### **<2A>𤑳緵2嚗𡁜<E59A97><F0A1819C>滢蝙<E6BBA2>刻䌊<E588BB>𠉛<EFBFBD>MemoryQueue**
|
||||
@@ -81,23 +81,23 @@ Redis队列 = 大楼里的"传送带"
|
||||
// backend/src/common/jobs/JobFactory.ts
|
||||
export class JobFactory {
|
||||
private static createQueue(): JobQueue {
|
||||
const queueType = process.env.QUEUE_TYPE || 'memory'; // ← 默认memory
|
||||
const queueType = process.env.QUEUE_TYPE || 'memory'; // <EFBFBD>?暺䁅恕memory
|
||||
|
||||
switch (queueType) {
|
||||
case 'memory':
|
||||
return this.createMemoryQueue(); // ← 当前使用这个
|
||||
return this.createMemoryQueue(); // <EFBFBD>?敶枏<E695B6>雿輻鍂餈嗘葵
|
||||
|
||||
case 'database':
|
||||
// TODO: 实现DatabaseQueue // ← 还没实现
|
||||
// TODO: 摰䂿緵DatabaseQueue // <EFBFBD>?餈䀹瓷摰䂿緵
|
||||
return this.createMemoryQueue();
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### **发现3:jobQueue在8个文件中被引用,共50次**
|
||||
#### **<EFBFBD>𤑳緵3嚗餷obQueue<EFBFBD>?銝芣<E98A9D>隞嗡葉鋡怠<E98BA1><E680A0>剁<EFBFBD><E58981>?0甈?*
|
||||
```typescript
|
||||
// 主要使用位置:
|
||||
// 銝餉<EFBFBD>雿輻鍂雿滨蔭嚗?
|
||||
1. ExtractionController.ts (TODO瘜券<EFBFBD>)
|
||||
2. DualModelExtractionService.ts (TODO瘜券<EFBFBD>)
|
||||
3. test-platform-api.ts (瘚贝<EFBFBD>隞<EFBFBD><EFBFBD>)
|
||||
@@ -106,17 +106,17 @@ export class JobFactory {
|
||||
|
||||
### 2.2 <20>笔<EFBFBD><E7AC94><EFBFBD><EFBFBD><EFBFBD>餌<EFBFBD>
|
||||
|
||||
| 组件 | 安装状态 | 使用状态 | 说明 |
|
||||
| 蝏<EFBFBD>辣 | 摰㕑<E691B0><E39591>嗆<EFBFBD>?| 雿輻鍂<E8BCBB>嗆<EFBFBD>?| 霂湔<E99C82> |
|
||||
|------|---------|---------|------|
|
||||
| **Redis缓存** | ❌ 未配置 | ❌ 未使用 | 目前用的是MemoryCache |
|
||||
| **Redis队列** | ❌ 未配置 | ❌ 未使用 | BullMQ已安装但未启用 |
|
||||
| **内存缓存** | ✅ 已实现 | ✅ 使用中 | HealthCheck、LLM结果 |
|
||||
| **内存队列** | ✅ 已实现 | ✅ 使用中 | MemoryQueue |
|
||||
| **Redis蝻枏<EFBFBD>** | <EFBFBD>?<3F>芷<EFBFBD>蝵?| <20>?<3F>芯蝙<E88AAF>?| <20>桀<EFBFBD><E6A180>函<EFBFBD><E587BD>烘emoryCache |
|
||||
| **Redis<EFBFBD>笔<EFBFBD>** | <EFBFBD>?<3F>芷<EFBFBD>蝵?| <20>?<3F>芯蝙<E88AAF>?| BullMQ撌脣<E6928C>鋆<EFBFBD><E98B86><EFBFBD>芸鍳<E88AB8>?|
|
||||
| **<EFBFBD><EFBFBD><EFBFBD>蝻枏<EFBFBD>** | <EFBFBD>?撌脣<E6928C><E884A3>?| <20>?雿輻鍂銝?| HealthCheck<EFBFBD><EFBFBD>LM蝏𤘪<EFBFBD> |
|
||||
| **<EFBFBD><EFBFBD><EFBFBD><EFBFBD>笔<EFBFBD>** | <EFBFBD>?撌脣<E6928C><E884A3>?| <20>?雿輻鍂銝?| MemoryQueue |
|
||||
|
||||
**结论**:
|
||||
- ✅ 您的系统已经预留了队列架构(MemoryQueue)
|
||||
- ✅ 已经安装了BullMQ依赖
|
||||
- ❌ 但实际还没有真正使用Redis(无论是缓存还是队列)
|
||||
**蝏栞捏**嚗?
|
||||
- <EFBFBD>?<3F>函<EFBFBD>蝟餌<E89D9F>撌脩<E6928C>憸<EFBFBD><E686B8>鈭<EFBFBD><E988AD><EFBFBD>埈沲<E59F88><E6B2B2><EFBFBD>MemoryQueue嚗?
|
||||
- <EFBFBD>?撌脩<E6928C>摰㕑<E691B0>鈭<EFBFBD>ullMQ靘肽<EFBFBD>
|
||||
- <EFBFBD>?雿<><E99BBF><EFBFBD><EFBFBD><EFBFBD>瘝⊥<E7989D><E28AA5><EFBFBD>迤雿輻鍂Redis嚗<73><E59A97>霈箸糓蝻枏<E89DBB>餈䀹糓<E480B9>笔<EFBFBD>嚗?
|
||||
|
||||
---
|
||||
|
||||
@@ -126,43 +126,43 @@ export class JobFactory {
|
||||
|
||||
| 蝏游漲 | Redis蝻枏<E89DBB> | Redis<69>笔<EFBFBD> |
|
||||
|------|----------|----------|
|
||||
| **用途** | 存储临时数据 | 异步任务管理 |
|
||||
| **数据结构** | String(字符串) | List(列表) |
|
||||
| **<EFBFBD>券<EFBFBD>?* | 摮睃<E691AE>銝湔𧒄<E6B994>唳旿 | 撘<>郊隞餃𦛚蝞∠<E89D9E> |
|
||||
| **<EFBFBD>唳旿蝏𤘪<EFBFBD>** | String嚗<EFBFBD><EFBFBD>蝚虫葡嚗?| List嚗<74><E59A97>銵剁<E98AB5> |
|
||||
| **<EFBFBD>滢<EFBFBD>** | GET / SET / DELETE | PUSH / POP / ACK |
|
||||
| **过期机制** | TTL自动过期 | 任务完成后删除 |
|
||||
| **读写模式** | 随机读写 | 顺序处理(FIFO) |
|
||||
| **典型场景** | LLM结果缓存、Session | 文献筛选任务、批量处理 |
|
||||
| **库/工具** | 直接用ioredis | BullMQ(基于ioredis) |
|
||||
| **餈<EFBFBD><EFBFBD><EFBFBD>箏<EFBFBD>** | TTL<EFBFBD>芸𢆡餈<EFBFBD><EFBFBD> | 隞餃𦛚摰峕<E691B0><E5B395>𤾸<EFBFBD><F0A4BEB8>?|
|
||||
| **霂餃<EFBFBD>璅∪<EFBFBD>** | <EFBFBD>𤩺㦤霂餃<EFBFBD> | 憿箏<E686BF>憭<EFBFBD><E686AD>嚗㇅IFO嚗?|
|
||||
| **<EFBFBD>詨<EFBFBD><EFBFBD>箸艶** | LLM蝏𤘪<EFBFBD>蝻枏<EFBFBD><EFBFBD><EFBFBD>ession | <EFBFBD><EFBFBD>讃蝑偦<EFBFBD>劐遙<EFBFBD>~<EFBFBD><EFBFBD>鸌<EFBFBD>誩<EFBFBD><EFBFBD>?|
|
||||
| **摨?撌亙<E6928C>** | <EFBFBD>湔𦻖<EFBFBD>甬oredis | BullMQ嚗<EFBFBD>抅鈭箸oredis嚗?|
|
||||
|
||||
### 3.2 <20>冽<EFBFBD>蝟餌<E89D9F>銝剔<E98A9D>摰鮋<E691B0>摨𠉛鍂
|
||||
|
||||
#### **Redis缓存的用途**
|
||||
#### **Redis蝻枏<EFBFBD><EFBFBD><EFBFBD>鍂<EFBFBD>?*
|
||||
|
||||
```typescript
|
||||
// <20>箸艶1嚗匁LM蝏𤘪<E89D8F>蝻枏<E89DBB>
|
||||
const cacheKey = `fulltext:${literatureId}:${model}`;
|
||||
await cache.set(cacheKey, llmResult, 3600); // 蝻枏<E89DBB>1撠𤩺𧒄
|
||||
|
||||
// 下次相同PDF再提取时:
|
||||
// 銝𧢲活<EFBFBD>詨<EFBFBD>PDF<EFBFBD>齿<EFBFBD><EFBFBD>𡝗𧒄嚗?
|
||||
const cached = await cache.get(cacheKey);
|
||||
if (cached) {
|
||||
return cached; // ✅ 直接返回,节省API费用
|
||||
return cached; // <EFBFBD>?<3F>湔𦻖餈𥪜<E9A488>嚗諹<E59A97><E8ABB9><EFBFBD>PI韐寧鍂
|
||||
}
|
||||
```
|
||||
|
||||
**好处**:
|
||||
- ✅ 避免重复调用DeepSeek API(¥0.43/篇)
|
||||
- ✅ 响应速度快(从1分钟降到1ms)
|
||||
- ✅ 多实例共享缓存
|
||||
**憟賢<EFBFBD>**嚗?
|
||||
- <EFBFBD>?<3F>踹<EFBFBD><E8B8B9>滚<EFBFBD>靚<EFBFBD>鍂DeepSeek API嚗<EFBFBD>?.43/蝭<EFBFBD><EFBFBD>
|
||||
- <EFBFBD>?<3F>滚<EFBFBD><E6BB9A>笔漲敹恬<E695B9>隞?<3F><><EFBFBD><EFBFBD>滚<EFBFBD>1ms嚗?
|
||||
- <EFBFBD>?憭𡁜<E686AD>靘见<E99D98>鈭怎<E988AD>摮?
|
||||
|
||||
---
|
||||
|
||||
#### **Redis队列的用途**
|
||||
#### **Redis<EFBFBD>笔<EFBFBD><EFBFBD><EFBFBD>鍂<EFBFBD>?*
|
||||
|
||||
```typescript
|
||||
// 场景2:文献筛选任务(199篇,需要30-60分钟)
|
||||
// <EFBFBD>箸艶2嚗𡁏<EFBFBD><EFBFBD>桃<EFBFBD><EFBFBD>劐遙<EFBFBD>∴<EFBFBD>199蝭<EFBFBD><EFBFBD><EFBFBD><EFBFBD>閬?0-60<36><30><EFBFBD>嚗?
|
||||
export async function startScreening(projectId) {
|
||||
// 1. 创建任务(不阻塞请求)
|
||||
// 1. <EFBFBD>𥕦遣隞餃𦛚嚗<EFBFBD><EFBFBD><EFBFBD>餃<EFBFBD>霂瑟<EFBFBD>嚗?
|
||||
const task = await prisma.aslScreeningTask.create({...});
|
||||
|
||||
// 2. <20>券<EFBFBD><E588B8><EFBFBD><EFBFBD>笔<EFBFBD>嚗<EFBFBD><E59A97>甇亙<E79487><E4BA99><EFBFBD><EFBFBD>
|
||||
@@ -185,15 +185,15 @@ jobQueue.process('asl:screening', async (job) => {
|
||||
});
|
||||
```
|
||||
|
||||
**好处**:
|
||||
- ✅ 任务持久化(实例重启不丢失)
|
||||
- ✅ 支持任务优先级
|
||||
- ✅ 支持任务重试
|
||||
- ✅ 多实例间任务分配
|
||||
**憟賢<EFBFBD>**嚗?
|
||||
- <EFBFBD>?隞餃𦛚<E9A483><F0A69B9A><EFBFBD><EFBFBD>吔<EFBFBD>摰硺<E691B0><E7A1BA>滚鍳銝滢腺憭梧<E686AD>
|
||||
- <EFBFBD>?<3F>舀<EFBFBD>隞餃𦛚隡睃<E99AA1>蝥?
|
||||
- <EFBFBD>?<3F>舀<EFBFBD>隞餃𦛚<E9A483>滩<EFBFBD>
|
||||
- <EFBFBD>?憭𡁜<E686AD>靘钅𡢿隞餃𦛚<E9A483><F0A69B9A><EFBFBD>
|
||||
|
||||
---
|
||||
|
||||
### 3.3 技术实现对比
|
||||
### 3.3 <EFBFBD><EFBFBD><EFBFBD>臬<EFBFBD><EFBFBD>啣笆瘥?
|
||||
|
||||
```typescript
|
||||
// ==================== Redis蝻枏<E89DBB> ====================
|
||||
@@ -234,84 +234,84 @@ const worker = new Worker('asl:screening', async (job) => {
|
||||
});
|
||||
```
|
||||
|
||||
**关键点**:
|
||||
- ✅ 两者都连接到同一个Redis实例
|
||||
- ✅ 可以同时使用(互不干扰)
|
||||
- ✅ BullMQ内部也是用ioredis实现的
|
||||
**<EFBFBD>喲睸<EFBFBD>?*嚗?
|
||||
- <EFBFBD>?銝方<E98A9D><E696B9><EFBFBD>餈墧𦻖<E5A2A7>啣<EFBFBD>銝<EFBFBD>銝析edis摰硺<EFBFBD>
|
||||
- <EFBFBD>?<3F>臭誑<E887AD>峕𧒄雿輻鍂嚗<E98D82><E59A97>銝滚僕<E6BB9A>堆<EFBFBD>
|
||||
- <EFBFBD>?BullMQ<EFBFBD><EFBFBD><EFBFBD>銋<EFBFBD>糓<EFBFBD>甬oredis摰䂿緵<EFBFBD>?
|
||||
|
||||
---
|
||||
|
||||
## 4. 只用Redis缓存的影响
|
||||
## 4. <EFBFBD>芰鍂Redis蝻枏<EFBFBD><EFBFBD><EFBFBD>蔣<EFBFBD>?
|
||||
|
||||
### 4.1 可以满足的需求 ✅
|
||||
### 4.1 <EFBFBD>臭誑皛∟雲<EFBFBD><EFBFBD><EFBFBD>瘙?<3F>?
|
||||
|
||||
1. **LLM结果缓存** ✅
|
||||
1. **LLM蝏𤘪<EFBFBD>蝻枏<EFBFBD>** <EFBFBD>?
|
||||
- <20>踹<EFBFBD><E8B8B9>滚<EFBFBD>API靚<49>鍂
|
||||
- <20><><EFBFBD><EFBFBD>鞉𧋦
|
||||
- <20>𣂼<EFBFBD><F0A382BC>滚<EFBFBD><E6BB9A>笔漲
|
||||
|
||||
2. **Session管理** ✅
|
||||
2. **Session蝞∠<EFBFBD>** <EFBFBD>?
|
||||
- 憭𡁜<E686AD>靘见<E99D98>鈭俟ession
|
||||
- 用户状态同步
|
||||
- <EFBFBD>冽<EFBFBD><EFBFBD>嗆<EFBFBD><EFBFBD><EFBFBD>甇?
|
||||
|
||||
3. **健康检查缓存** ✅
|
||||
3. **<EFBFBD>亙熒璉<EFBFBD><EFBFBD>亦<EFBFBD>摮?* <20>?
|
||||
- <20>踹<EFBFBD><E8B8B9>滚<EFBFBD>閫<EFBFBD><E996AB>Excel
|
||||
|
||||
4. **短期任务** ✅
|
||||
4. **<EFBFBD>剜<EFBFBD>隞餃𦛚** <20>?
|
||||
- < 10蝘垍<E89D98>隞餃𦛚
|
||||
- 直接在HTTP请求中处理
|
||||
- <EFBFBD>湔𦻖<EFBFBD>沈TTP霂瑟<EFBFBD>銝剖<EFBFBD><EFBFBD>?
|
||||
|
||||
### 4.2 无法满足的需求 ❌
|
||||
### 4.2 <EFBFBD>䭾<EFBFBD>皛∟雲<EFBFBD><EFBFBD><EFBFBD>瘙?<3F>?
|
||||
|
||||
1. **长时间任务持久化** ❌
|
||||
1. **<EFBFBD>踵𧒄<EFBFBD>港遙<EFBFBD>⊥<EFBFBD>銋<EFBFBD><EFBFBD>** <20>?
|
||||
```
|
||||
场景:用户提交199篇文献筛选(需要30-60分钟)
|
||||
<EFBFBD>箸艶嚗𡁶鍂<EFBFBD>瑟<EFBFBD>鈭?99蝭<39><E89DAD><EFBFBD>桃<EFBFBD><E6A183>㚁<EFBFBD><E39A81><EFBFBD>閬?0-60<36><30><EFBFBD>嚗?
|
||||
|
||||
如果只用Redis缓存:
|
||||
- 任务状态存在内存 → 实例重启丢失
|
||||
- 无法恢复中断的任务
|
||||
憒<EFBFBD><EFBFBD><EFBFBD>芰鍂Redis蝻枏<EFBFBD>嚗?
|
||||
- 隞餃𦛚<EFBFBD>嗆<EFBFBD><EFBFBD><EFBFBD><EFBFBD>典<EFBFBD>摮?<3F>?摰硺<E691B0><E7A1BA>滚鍳銝W仃
|
||||
- <EFBFBD>䭾<EFBFBD><EFBFBD>W<EFBFBD>銝剜鱏<EFBFBD><EFBFBD>遙<EFBFBD>?
|
||||
- <20>冽<EFBFBD><E586BD>见<EFBFBD>隞餃𦛚瘨<F0A69B9A>仃
|
||||
```
|
||||
|
||||
2. **任务队列管理** ❌
|
||||
2. **隞餃𦛚<EFBFBD>笔<EFBFBD>蝞∠<EFBFBD>** <EFBFBD>?
|
||||
```
|
||||
场景:10个用户同时提交批量任务
|
||||
<EFBFBD>箸艶嚗?0銝芰鍂<E88AB0>瑕<EFBFBD><E79195>嗆<EFBFBD>鈭斗鸌<E69697>譍遙<E8AD8D>?
|
||||
|
||||
如果只用Redis缓存:
|
||||
- 无法排队(先来先服务)
|
||||
憒<EFBFBD><EFBFBD><EFBFBD>芰鍂Redis蝻枏<EFBFBD>嚗?
|
||||
- <EFBFBD>䭾<EFBFBD><EFBFBD>㘾<EFBFBD>嚗<EFBFBD><EFBFBD><EFBFBD>亙<EFBFBD><EFBFBD>滚𦛚嚗?
|
||||
- <20>䭾<EFBFBD><E4ADBE>𣂼<EFBFBD>撟嗅<E6929F>
|
||||
- <20>滚𦛚<E6BB9A>典虾<E585B8>質◤<E8B3AA>梶<EFBFBD>
|
||||
```
|
||||
|
||||
3. **任务重试** ❌
|
||||
3. **隞餃𦛚<EFBFBD>滩<EFBFBD>** <EFBFBD>?
|
||||
```
|
||||
场景:某篇文献处理失败
|
||||
<EFBFBD>箸艶嚗𡁏<EFBFBD>蝭<EFBFBD><EFBFBD><EFBFBD>桀<EFBFBD><EFBFBD><EFBFBD>仃韐?
|
||||
|
||||
如果只用Redis缓存:
|
||||
憒<EFBFBD><EFBFBD><EFBFBD>芰鍂Redis蝻枏<EFBFBD>嚗?
|
||||
- <20>䭾<EFBFBD><E4ADBE>芸𢆡<E88AB8>滩<EFBFBD>
|
||||
- 需要用户重新提交
|
||||
- <EFBFBD><EFBFBD>閬<EFBFBD>鍂<EFBFBD>琿<EFBFBD><EFBFBD>唳<EFBFBD>鈭?
|
||||
```
|
||||
|
||||
4. **分布式任务分配** ❌
|
||||
4. **<EFBFBD><EFBFBD><EFBFBD>撘譍遙<EFBFBD>∪<EFBFBD><EFBFBD>?* <20>?
|
||||
```
|
||||
场景:SAE有3个实例
|
||||
<EFBFBD>箸艶嚗锭AE<EFBFBD>?銝芸<E98A9D>靘?
|
||||
|
||||
如果只用Redis缓存:
|
||||
憒<EFBFBD><EFBFBD><EFBFBD>芰鍂Redis蝻枏<EFBFBD>嚗?
|
||||
- <20>䭾<EFBFBD><E4ADBE>讛<EFBFBD><E8AE9B>芯葵摰硺<E691B0>憭<EFBFBD><E686AD><EFBFBD>芯葵隞餃𦛚
|
||||
- <20>航<EFBFBD><E888AA>滚<EFBFBD>憭<EFBFBD><E686AD>
|
||||
```
|
||||
|
||||
### 4.3 摰鮋<E691B0>敶勗<E695B6>霂<EFBFBD>摯
|
||||
|
||||
#### **如果只启用Redis缓存:**
|
||||
#### **憒<EFBFBD><EFBFBD><EFBFBD>芸鍳<EFBFBD>沖edis蝻枏<EFBFBD>嚗?*
|
||||
|
||||
| 敶勗<E695B6>蝏游漲 | 霂<><E99C82> | 霂湔<E99C82> |
|
||||
|---------|------|------|
|
||||
| **LLM成本控制** | ✅ 优秀 | 缓存命中,节省30-50%费用 |
|
||||
| **多实例支持** | ✅ 优秀 | 缓存共享 |
|
||||
| **长任务可靠性** | ⚠️ 一般 | 仍然依赖MemoryQueue |
|
||||
| **任务管理能力** | ⚠️ 一般 | 无优先级、重试 |
|
||||
| **系统可扩展性** | 🟡 中等 | 单实例可以,多实例有问题 |
|
||||
| **LLM<EFBFBD>鞉𧋦<EFBFBD>批<EFBFBD>** | <20>?隡条<E99AA1> | 蝻枏<E89DBB><E69E8F>賭葉嚗諹<E59A97><E8ABB9>?0-50%韐寧鍂 |
|
||||
| **憭𡁜<EFBFBD>靘𧢲𣈲<EFBFBD>?* | <20>?隡条<E99AA1> | 蝻枏<E89DBB><E69E8F>曹澈 |
|
||||
| **<EFBFBD>蹂遙<EFBFBD>∪虾<EFBFBD>䭾<EFBFBD>?* | <20>𩤃<EFBFBD> 銝<><E98A9D>?| 隞滨<E99A9E>靘肽<E99D98>MemoryQueue |
|
||||
| **隞餃𦛚蝞∠<EFBFBD><EFBFBD>賢<EFBFBD>** | <20>𩤃<EFBFBD> 銝<><E98A9D>?| <20>牐<EFBFBD><E78990><EFBFBD>漣<EFBFBD><E6BCA3><EFBFBD>霂?|
|
||||
| **蝟餌<EFBFBD><EFBFBD>舀<EFBFBD>撅閙<EFBFBD>?* | <20>椬 銝剔<E98A9D> | <20>訫<EFBFBD>靘见虾隞伐<E99A9E>憭𡁜<E686AD>靘𧢲<E99D98><F0A7A2B2>桅<EFBFBD> |
|
||||
|
||||
---
|
||||
|
||||
@@ -323,24 +323,24 @@ const worker = new Worker('asl:screening', async (job) => {
|
||||
```
|
||||
敶枏<EFBFBD>嚗㇈emoryQueue嚗㚁<EFBFBD>
|
||||
- <20>冽<EFBFBD><E586BD>𣂷漱<F0A382B7>寥<EFBFBD>隞餃𦛚
|
||||
- 实例重启 → ❌ 任务丢失
|
||||
- 摰硺<EFBFBD><EFBFBD>滚鍳 <20>?<3F>?隞餃𦛚銝W仃
|
||||
- <20>冽<EFBFBD><E586BD>閗<EFBFBD><E99697><EFBFBD><EFBFBD>5-10%
|
||||
|
||||
改用Redis队列:
|
||||
<EFBFBD>寧鍂Redis<EFBFBD>笔<EFBFBD>嚗?
|
||||
- 隞餃𦛚靽嘥<E99DBD><E598A5>沖edis
|
||||
- 实例重启 → ✅ 任务继续
|
||||
- 摰硺<EFBFBD><EFBFBD>滚鍳 <20>?<3F>?隞餃𦛚蝏抒賒
|
||||
- <20>冽<EFBFBD><E586BD>閗<EFBFBD><E99697><EFBFBD><EFBFBD>< 1%
|
||||
```
|
||||
|
||||
#### **收益2:任务重试**
|
||||
#### **<EFBFBD>嗥<EFBFBD>2嚗帋遙<EFBFBD>⊿<EFBFBD>霂?*
|
||||
```
|
||||
敶枏<EFBFBD>嚗㇈emoryQueue嚗㚁<EFBFBD>
|
||||
- LLM调用失败 → ❌ 任务标记为失败
|
||||
- 需要用户重新提交
|
||||
- LLM靚<EFBFBD>鍂憭梯揖 <20>?<3F>?隞餃𦛚<E9A483><F0A69B9A>扇銝箏仃韐?
|
||||
- <EFBFBD><EFBFBD>閬<EFBFBD>鍂<EFBFBD>琿<EFBFBD><EFBFBD>唳<EFBFBD>鈭?
|
||||
|
||||
改用Redis队列:
|
||||
- LLM调用失败 → ✅ 自动重试3次
|
||||
- 指数退避(2秒、4秒、8秒)
|
||||
<EFBFBD>寧鍂Redis<EFBFBD>笔<EFBFBD>嚗?
|
||||
- LLM靚<EFBFBD>鍂憭梯揖 <20>?<3F>?<3F>芸𢆡<E88AB8>滩<EFBFBD>3甈?
|
||||
- <EFBFBD><EFBFBD>㺭<EFBFBD><EFBFBD><EFBFBD>選<EFBFBD>2蝘鉝<EFBFBD>?蝘鉝<E89D98>?蝘𡜐<E89D98>
|
||||
```
|
||||
|
||||
#### **<2A>嗥<EFBFBD>3嚗𡁜<E59A97>撣<EFBFBD><E692A3>隞餃𦛚<E9A483><F0A69B9A><EFBFBD>**
|
||||
@@ -349,19 +349,19 @@ const worker = new Worker('asl:screening', async (job) => {
|
||||
- 瘥譍葵摰硺<E691B0><E7A1BA>祉<EFBFBD>憭<EFBFBD><E686AD>隞餃𦛚
|
||||
- <20>䭾<EFBFBD><E4ADBE>讛<EFBFBD>
|
||||
|
||||
改用Redis队列:
|
||||
<EFBFBD>寧鍂Redis<EFBFBD>笔<EFBFBD>嚗?
|
||||
- 憭帋葵Worker<65>W<EFBFBD>隞餃𦛚
|
||||
- <20>芸𢆡韐蠘蝸<E8A098><E89DB8>﹛
|
||||
- <20>𣂷葵Worker<65><72><EFBFBD>嚗<EFBFBD><E59A97>隞頟orker<65>亦恣
|
||||
```
|
||||
|
||||
#### **收益4:任务监控**
|
||||
#### **<EFBFBD>嗥<EFBFBD>4嚗帋遙<EFBFBD>∠<EFBFBD><EFBFBD>?*
|
||||
```
|
||||
敶枏<EFBFBD>嚗㇈emoryQueue嚗㚁<EFBFBD>
|
||||
- 任务状态在内存中
|
||||
- 隞餃𦛚<EFBFBD>嗆<EFBFBD><EFBFBD>銁<EFBFBD><EFBFBD><EFBFBD>銝?
|
||||
- <20>䭾<EFBFBD><E4ADBE>亦<EFBFBD><E4BAA6><EFBFBD>蟮隞餃𦛚
|
||||
|
||||
改用Redis队列:
|
||||
<EFBFBD>寧鍂Redis<EFBFBD>笔<EFBFBD>嚗?
|
||||
- 隞餃𦛚<E9A483><F0A69B9A>蟮靽嘥<E99DBD>
|
||||
- <20>臭誑<E887AD>亦<EFBFBD>憭梯揖<E6A2AF>笔<EFBFBD>
|
||||
- 蝏蠘恣憭<E681A3><E686AD><EFBFBD>園鵭
|
||||
@@ -371,76 +371,76 @@ const worker = new Worker('asl:screening', async (job) => {
|
||||
|
||||
#### **憌𡡞埯1嚗𡁜<E59A97><F0A1819C>惩<EFBFBD><E683A9><EFBFBD>漲** <20>椬 銝剔<E98A9D>
|
||||
```
|
||||
当前:
|
||||
敶枏<EFBFBD>嚗?
|
||||
- MemoryQueue嚗𡁶<E59A97><F0A181B6>𨰻<EFBFBD><F0A8B0BB><EFBFBD><EFBFBD><EFBFBD>圾
|
||||
- 代码量:~200行
|
||||
- 隞<EFBFBD><EFBFBD><EFBFBD>𧶏<EFBFBD>~200銵?
|
||||
|
||||
改为BullMQ:
|
||||
<EFBFBD>嫣蛹BullMQ嚗?
|
||||
- <20><>閬<EFBFBD>郎銋泎ullMQ API
|
||||
- 代码量:~500行
|
||||
- 隞<EFBFBD><EFBFBD><EFBFBD>𧶏<EFBFBD>~500銵?
|
||||
- <20><>閬<EFBFBD><E996AC>蝵娠orker
|
||||
```
|
||||
|
||||
**缓解**:
|
||||
- ✅ BullMQ文档完善
|
||||
- ✅ 社区活跃(GitHub 15k+ stars)
|
||||
- ✅ 我们已经安装了依赖
|
||||
**蝻栞圾**嚗?
|
||||
- <EFBFBD>?BullMQ<EFBFBD><EFBFBD>﹝摰<EFBFBD><EFBFBD>
|
||||
- <EFBFBD>?蝷曉躹瘣餉<E798A3>嚗𠃑itHub 15k+ stars嚗?
|
||||
- <EFBFBD>?<3F>睲賑撌脩<E6928C>摰㕑<E691B0>鈭<EFBFBD><E988AD>韏?
|
||||
|
||||
---
|
||||
|
||||
#### **风险2:依赖Redis稳定性** 🟡 中等
|
||||
#### **憌𡡞埯2嚗帋<EFBFBD>韏餸edis蝔喳<EFBFBD><EFBFBD>?* <20>椬 銝剔<E98A9D>
|
||||
```
|
||||
<EFBFBD>箸艶嚗鑹edis<EFBFBD><EFBFBD><EFBFBD>
|
||||
- Redis蝻枏<E89DBB>嚗𡁜虾隞仿<E99A9E>蝥批<E89DA5><E689B9><EFBFBD><EFBFBD>
|
||||
- Redis<69>笔<EFBFBD>嚗𡁏<E59A97>瘜閖<E7989C>蝥改<E89DA5><E694B9>笔<EFBFBD>敹<EFBFBD>◆<EFBFBD><E29786><EFBFBD><EFBFBD>吔<EFBFBD>
|
||||
```
|
||||
|
||||
**缓解**:
|
||||
- ✅ 您购买的是高可用版Redis(99.95%)
|
||||
- ✅ 主从自动切换
|
||||
- ✅ 实际风险很低
|
||||
**蝻栞圾**嚗?
|
||||
- <EFBFBD>?<3F>刻揚銋啁<E98A8B><E59581>舫<EFBFBD><E888AB>舐鍂<E88890>㇌edis嚗?9.95%嚗?
|
||||
- <EFBFBD>?銝颱<E98A9D><E9A2B1>芸𢆡<E88AB8><F0A286A1>揢
|
||||
- <EFBFBD>?摰鮋<E691B0>憌𡡞埯敺<E59FAF><E695BA>
|
||||
|
||||
---
|
||||
|
||||
#### **风险3:调试难度增加** 🟢 低
|
||||
#### **憌𡡞埯3嚗朞<EFBFBD>霂閖𠗕摨血<EFBFBD><EFBFBD>?* <20>叚 雿?
|
||||
```
|
||||
当前:
|
||||
敶枏<EFBFBD>嚗?
|
||||
- console.log() <20>喳虾
|
||||
- 隞餃𦛚<E9A483>典<EFBFBD>摮䀝葉
|
||||
|
||||
改为BullMQ:
|
||||
<EFBFBD>嫣蛹BullMQ嚗?
|
||||
- <20><>閬<EFBFBD>䰻<EFBFBD>苄edis<69>唳旿
|
||||
- 需要用BullBoard(可视化工具)
|
||||
- <EFBFBD><EFBFBD>閬<EFBFBD>鍂BullBoard嚗<EFBFBD>虾閫<EFBFBD><EFBFBD>撌亙<EFBFBD>嚗?
|
||||
```
|
||||
|
||||
**缓解**:
|
||||
- ✅ BullBoard提供Web UI
|
||||
- ✅ 可以看到任务状态、重试次数
|
||||
- ✅ 日志更详细
|
||||
**蝻栞圾**嚗?
|
||||
- <EFBFBD>?BullBoard<EFBFBD>𣂷<EFBFBD>Web UI
|
||||
- <EFBFBD>?<3F>臭誑<E887AD>见<EFBFBD>隞餃𦛚<E9A483>嗆<EFBFBD><E59786><EFBFBD><EFBFBD><EFBFBD>霂閙活<E99699>?
|
||||
- <EFBFBD>?<3F>亙<EFBFBD><E4BA99>渲祕蝏?
|
||||
|
||||
---
|
||||
|
||||
#### **风险4:内存占用** 🟢 低
|
||||
#### **憌𡡞埯4嚗𡁜<EFBFBD>摮睃<EFBFBD><EFBFBD>?* <20>叚 雿?
|
||||
```
|
||||
<EFBFBD><EFBFBD><EFBFBD>嚗鑹edis<EFBFBD>笔<EFBFBD>隡𡁜<EFBFBD><EFBFBD>典<EFBFBD>憭𡁜<EFBFBD>摮矋<EFBFBD>
|
||||
|
||||
实际:
|
||||
摰鮋<EFBFBD>嚗?
|
||||
- <20>蓥葵隞餃𦛚<E9A483>唳旿嚗鰺1KB
|
||||
- 100銝芸僎<E88AB8>睲遙<E79DB2>∴<EFBFBD>100KB
|
||||
- 1000銝芯遙<E88AAF>∪<EFBFBD><E288AA>莎<EFBFBD>1MB
|
||||
- 对256MB Redis影响很小
|
||||
- 撖?56MB Redis敶勗<EFBFBD>敺<EFBFBD><EFBFBD>
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 5.3 憌𡡞埯<F0A1A19E>拚猐
|
||||
|
||||
| 风险 | 严重性 | 概率 | 缓解难度 | 建议 |
|
||||
| 憌𡡞埯 | 銝仿<E98A9D><E4BBBF>?| 璁<><E79281> | 蝻栞圾<E6A09E>曉漲 | 撱箄悅 |
|
||||
|------|--------|------|----------|------|
|
||||
| **增加复杂度** | 🟡 中 | 🔴 高 | ✅ 易 | 提供培训文档 |
|
||||
| **Redis依赖** | 🔴 高 | 🟢 低 | ✅ 易 | 高可用版 |
|
||||
| **调试困难** | 🟢 低 | 🟡 中 | ✅ 易 | 使用BullBoard |
|
||||
| **内存占用** | 🟢 低 | 🟢 低 | ✅ 易 | 监控即可 |
|
||||
| **憓𧼮<EFBFBD>憭齿<EFBFBD>摨?* | <20>椬 銝?| <20>𣞁 擃?| <20>?<3F>?| <20>𣂷<EFBFBD><F0A382B7>寡悌<E5AFA1><E6828C>﹝ |
|
||||
| **Redis靘肽<EFBFBD>** | <EFBFBD>𣞁 擃?| <20>叚 雿?| <20>?<3F>?| 擃睃虾<E79D83>函<EFBFBD> |
|
||||
| **靚<EFBFBD><EFBFBD><EFBFBD>圈𠗕** | <20>叚 雿?| <20>椬 銝?| <20>?<3F>?| 雿輻鍂BullBoard |
|
||||
| **<EFBFBD><EFBFBD><EFBFBD><EFBFBD>删鍂** | <20>叚 雿?| <20>叚 雿?| <20>?<3F>?| <20>烐綉<E78390>喳虾 |
|
||||
|
||||
**蝏栞捏嚗𡁻<E59A97><F0A181BB>拙虾<E68B99>改<EFBFBD><E694B9>嗥<EFBFBD>憭找<E686AD>憌𡡞埯**
|
||||
|
||||
@@ -453,19 +453,19 @@ const worker = new Worker('asl:screening', async (job) => {
|
||||
```
|
||||
<EFBFBD>嗆挾1嚗<EFBFBD>𧋦<EFBFBD>剁<EFBFBD>: <20>舐鍂Redis蝻枏<E89DBB>
|
||||
<EFBFBD>鎿<EFBFBD> <20>格<EFBFBD>嚗朞圾<E69C9E>蚯LM<4C>鞉𧋦<E99E89>桅<EFBFBD>
|
||||
├─ 工作量:2天
|
||||
├─ 风险:低(有降级方案)
|
||||
└─ 收益:节省30-50% API费用
|
||||
<EFBFBD>鎿<EFBFBD> 撌乩<E6928C><E4B9A9>𧶏<EFBFBD>2憭?
|
||||
<EFBFBD>鎿<EFBFBD> 憌𡡞埯嚗帋<E59A97>嚗<EFBFBD><E59A97><EFBFBD>滨漣<E6BBA8>寞<EFBFBD>嚗?
|
||||
<EFBFBD>婙<EFBFBD> <20>嗥<EFBFBD>嚗朞<E59A97><E69C9E>?0-50% API韐寧鍂
|
||||
|
||||
<EFBFBD>嗆挾2嚗<EFBFBD><EFBFBD><EFBFBD>剁<EFBFBD>: <20>舐鍂Redis<69>笔<EFBFBD>
|
||||
├─ 目标:解决长任务可靠性
|
||||
├─ 工作量:3天
|
||||
├─ 风险:中(但可控)
|
||||
└─ 收益:任务丢失率降低10倍
|
||||
<EFBFBD>鎿<EFBFBD> <20>格<EFBFBD>嚗朞圾<E69C9E>喲鵭隞餃𦛚<E9A483>舫<EFBFBD><E888AB>?
|
||||
<EFBFBD>鎿<EFBFBD> 撌乩<E6928C><E4B9A9>𧶏<EFBFBD>3憭?
|
||||
<EFBFBD>鎿<EFBFBD> 憌𡡞埯嚗帋葉嚗<E89189><E59A97><EFBFBD>舀綉嚗?
|
||||
<EFBFBD>婙<EFBFBD> <20>嗥<EFBFBD>嚗帋遙<E5B88B>∩腺憭梁<E686AD><E6A281>滢<EFBFBD>10<31>?
|
||||
|
||||
阶段3(未来): 优化与监控
|
||||
├─ BullBoard可视化
|
||||
├─ 任务优先级
|
||||
<EFBFBD>嗆挾3嚗<EFBFBD>𧊋<EFBFBD>伐<EFBFBD>: 隡睃<E99AA1>銝𡒊<E98A9D><F0A1928A>?
|
||||
<EFBFBD>鎿<EFBFBD> BullBoard<EFBFBD>航<EFBFBD><EFBFBD>?
|
||||
<EFBFBD>鎿<EFBFBD> 隞餃𦛚隡睃<E99AA1>蝥?
|
||||
<EFBFBD>鎿<EFBFBD> <20>扯<EFBFBD><E689AF>烐綉
|
||||
<EFBFBD>婙<EFBFBD> <20>𡃏郎<F0A1838F>箏<EFBFBD>
|
||||
```
|
||||
@@ -476,21 +476,21 @@ const worker = new Worker('asl:screening', async (job) => {
|
||||
|
||||
**Day 1-2嚗𡁜<E59A97><F0A1819C>訌edisCacheAdapter**
|
||||
```bash
|
||||
# 参考文档:04-Redis改造实施计划.md
|
||||
# <EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>獢<EFBFBD><EFBFBD>04-Redis<69>寥<EFBFBD>惩<EFBFBD><E683A9>質恣<E8B3AA>?md
|
||||
# Phase 1-3
|
||||
|
||||
✅ 安装ioredis
|
||||
✅ 实现RedisCacheAdapter
|
||||
✅ 添加降级策略
|
||||
✅ 本地测试
|
||||
<EFBFBD>?摰㕑<E691B0>ioredis
|
||||
<EFBFBD>?摰䂿緵RedisCacheAdapter
|
||||
<EFBFBD>?瘛餃<E7989B><E9A483>滨漣蝑𣇉裦
|
||||
<EFBFBD>?<3F>砍𧑐瘚贝<E7989A>
|
||||
```
|
||||
|
||||
**验收标准**:
|
||||
**撉峕𤣰<EFBFBD><EFBFBD><EFBFBD>**嚗?
|
||||
```bash
|
||||
✅ HealthCheckService缓存命中
|
||||
✅ LLM12FieldsService缓存命中
|
||||
✅ 实例重启缓存不丢失
|
||||
✅ Redis挂了系统仍可用(降级)
|
||||
<EFBFBD>?HealthCheckService蝻枏<EFBFBD><EFBFBD>賭葉
|
||||
<EFBFBD>?LLM12FieldsService蝻枏<EFBFBD><EFBFBD>賭葉
|
||||
<EFBFBD>?摰硺<E691B0><E7A1BA>滚鍳蝻枏<E89DBB>銝滢腺憭?
|
||||
<EFBFBD>?Redis<EFBFBD><EFBFBD><EFBFBD>蝟餌<EFBFBD>隞滚虾<EFBFBD>剁<EFBFBD><EFBFBD>滨漣嚗?
|
||||
```
|
||||
|
||||
---
|
||||
@@ -507,13 +507,13 @@ export class RedisQueue implements JobQueue {
|
||||
}
|
||||
```
|
||||
|
||||
**Day 2:迁移业务代码**
|
||||
**Day 2嚗朞<EFBFBD>蝘颱<EFBFBD><EFBFBD>∩誨<EFBFBD>?*
|
||||
```typescript
|
||||
// 靽格㺿嚗窃creeningService.ts
|
||||
// 隞𠬍<E99A9E>
|
||||
processLiteraturesInBackground(task.id, ...);
|
||||
|
||||
// 改为:
|
||||
// <EFBFBD>嫣蛹嚗?
|
||||
await jobQueue.push('asl:screening', {
|
||||
taskId: task.id,
|
||||
projectId,
|
||||
@@ -523,109 +523,109 @@ await jobQueue.push('asl:screening', {
|
||||
|
||||
**Day 3嚗𡁏<E59A97>霂蓥<E99C82>銝羓瑪**
|
||||
```bash
|
||||
✅ 单元测试
|
||||
✅ 集成测试
|
||||
✅ 压力测试
|
||||
✅ 故障模拟测试
|
||||
✅ 灰度发布
|
||||
<EFBFBD>?<3F>訫<EFBFBD>瘚贝<E7989A>
|
||||
<EFBFBD>?<3F><><EFBFBD>瘚贝<E7989A>
|
||||
<EFBFBD>?<3F>见<EFBFBD>瘚贝<E7989A>
|
||||
<EFBFBD>?<3F><><EFBFBD>璅⊥<E79285>瘚贝<E7989A>
|
||||
<EFBFBD>?<3F>啣漲<E595A3>穃<EFBFBD>
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 6.3 最小改动方案(如果资源有限)
|
||||
### 6.3 <EFBFBD><EFBFBD>撠𤩺㺿<EFBFBD>冽䲮獢<EFBFBD><EFBFBD>憒<EFBFBD><EFBFBD>韏<EFBFBD><EFBFBD><EFBFBD>厰<EFBFBD>嚗?
|
||||
|
||||
**如果只有1周时间,建议:**
|
||||
**憒<EFBFBD><EFBFBD><EFBFBD>芣<EFBFBD>1<EFBFBD>冽𧒄<EFBFBD>湛<EFBFBD>撱箄悅嚗?*
|
||||
|
||||
```
|
||||
优先级1(必须):Redis缓存
|
||||
隡睃<EFBFBD>蝥?嚗<><E59A97>憿鳴<E686BF>嚗鑹edis蝻枏<EFBFBD>
|
||||
- 閫<><E996AB>LLM<4C>鞉𧋦<E99E89>桅<EFBFBD>
|
||||
- 工作量:2天
|
||||
- 撌乩<EFBFBD><EFBFBD>𧶏<EFBFBD>2憭?
|
||||
|
||||
优先级2(可选):Redis队列
|
||||
隡睃<EFBFBD>蝥?嚗<>虾<EFBFBD>㚁<EFBFBD>嚗鑹edis<EFBFBD>笔<EFBFBD>
|
||||
- <20><>𧒄靽脲<E99DBD>MemoryQueue
|
||||
- 霈啣<E99C88><E595A3><EFBFBD><EFBFBD>臬<EFBFBD>箏𦛚
|
||||
- 等用户规模增长后再改造
|
||||
- 蝑厩鍂<EFBFBD>瑁<EFBFBD>璅∪<EFBFBD><EFBFBD>踹<EFBFBD><EFBFBD>齿㺿<EFBFBD>?
|
||||
```
|
||||
|
||||
**判断标准**:
|
||||
**<EFBFBD>斗鱏<EFBFBD><EFBFBD><EFBFBD>**嚗?
|
||||
```
|
||||
如果满足以下条件,可以暂缓Redis队列:
|
||||
✅ 用户数 < 20
|
||||
✅ SAE只有1个实例
|
||||
✅ 批量任务不频繁(每天 < 5个)
|
||||
✅ 可以接受偶尔任务丢失
|
||||
憒<EFBFBD><EFBFBD>皛∟雲隞乩<EFBFBD><EFBFBD>∩辣嚗<EFBFBD>虾隞交<EFBFBD>蝻𣫮edis<EFBFBD>笔<EFBFBD>嚗?
|
||||
<EFBFBD>?<3F>冽<EFBFBD><E586BD>?< 20
|
||||
<EFBFBD>?SAE<EFBFBD>芣<EFBFBD>1銝芸<EFBFBD>靘?
|
||||
<EFBFBD>?<3F>寥<EFBFBD>隞餃𦛚銝漤<E98A9D>蝜<EFBFBD><E89D9C>瘥誩予 < 5銝迎<EFBFBD>
|
||||
<EFBFBD>?<3F>臭誑<E887AD>亙<EFBFBD><E4BA99>嗅<EFBFBD>隞餃𦛚銝W仃
|
||||
|
||||
<EFBFBD>血<EFBFBD>嚗<EFBFBD>遣霈桀偷敹怠鍳<EFBFBD>沖edis<EFBFBD>笔<EFBFBD>
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 6.4 为什么BullMQ已经安装但未使用?
|
||||
### 6.4 銝箔<EFBFBD>銋㇂ullMQ撌脩<EFBFBD>摰㕑<EFBFBD>雿<EFBFBD>𧊋雿輻鍂嚗?
|
||||
|
||||
**推测原因**:
|
||||
1. **计划使用但未实施**:开发时计划用BullMQ,先安装了依赖
|
||||
2. **测试过但未启用**:可能在测试环境验证过
|
||||
3. **依赖传递**:其他包依赖了BullMQ
|
||||
**<EFBFBD>冽<EFBFBD><EFBFBD>笔<EFBFBD>**嚗?
|
||||
1. **霈∪<EFBFBD>雿輻鍂雿<EFBFBD>𧊋摰墧鴌**嚗𡁜<E59A97><F0A1819C>烐𧒄霈∪<E99C88><E288AA>步ullMQ嚗<51><E59A97>摰㕑<E691B0>鈭<EFBFBD><E988AD>韏?
|
||||
2. **瘚贝<EFBFBD>餈<EFBFBD><EFBFBD><EFBFBD>芸鍳<EFBFBD>?*嚗𡁜虾<F0A1819C>賢銁瘚贝<E7989A><E8B49D>臬<EFBFBD>撉諹<E69289>餈?
|
||||
3. **靘肽<EFBFBD>隡𣳇<EFBFBD>?*嚗𡁜<E59A97>隞硋<E99A9E>靘肽<E99D98>鈭<EFBFBD>ullMQ
|
||||
|
||||
**建议**:
|
||||
- ✅ 保留BullMQ依赖(已经安装了)
|
||||
- ✅ 在阶段2时直接使用(无需重新安装)
|
||||
- ✅ 如果暂时不用,也不用删除(不影响性能)
|
||||
**撱箄悅**嚗?
|
||||
- <EFBFBD>?靽萘<E99DBD>BullMQ靘肽<EFBFBD>嚗<EFBFBD>歇蝏誩<EFBFBD>鋆<EFBFBD><EFBFBD>嚗?
|
||||
- <EFBFBD>?<3F>券𧫴畾?<3F>嗥凒<E597A5>乩蝙<E4B9A9>剁<EFBFBD><E58981>𣳇<EFBFBD><F0A3B387>齿鰵摰㕑<E691B0>嚗?
|
||||
- <EFBFBD>?憒<><E68692><EFBFBD><EFBFBD>𧒄銝滨鍂嚗䔶<E59A97>銝滨鍂<E6BBA8>𣳇膄嚗<E88684><E59A97>敶勗<E695B6><E58B97>扯<EFBFBD>嚗?
|
||||
|
||||
---
|
||||
|
||||
## 7. 撣貉<E692A3><E8B289>桅<EFBFBD> FAQ
|
||||
|
||||
### Q1:Redis缓存和Redis队列能同时使用吗?
|
||||
**A1**:可以!它们使用同一个Redis实例,但是不同的数据结构。
|
||||
### Q1嚗鑹edis蝻枏<EFBFBD><EFBFBD>朙edis<EFBFBD>笔<EFBFBD><EFBFBD>賢<EFBFBD><EFBFBD>嗡蝙<EFBFBD>典<EFBFBD>嚗?
|
||||
**A1**嚗𡁜虾隞伐<EFBFBD>摰<EFBFBD>賑雿輻鍂<EFBFBD>䔶<EFBFBD>銝析edis摰硺<EFBFBD>嚗䔶<EFBFBD><EFBFBD>臭<EFBFBD><EFBFBD>𣬚<EFBFBD><EFBFBD>唳旿蝏𤘪<EFBFBD><EFBFBD>?
|
||||
|
||||
```
|
||||
Redis实例(256MB)
|
||||
Redis摰硺<EFBFBD>嚗?56MB嚗?
|
||||
<EFBFBD>鎿<EFBFBD> 蝻枏<E89DBB><E69E8F>唳旿嚗𠄎tring嚗㚁<E59A97><E39A81>删鍂 ~50MB
|
||||
<EFBFBD>婙<EFBFBD> <20>笔<EFBFBD><E7AC94>唳旿嚗𡿨ist嚗㚁<E59A97><E39A81>删鍂 ~10MB
|
||||
------------------------------------
|
||||
总计:~60MB / 256MB = 23% 使用率
|
||||
<EFBFBD>餉恣嚗鰺60MB / 256MB = 23% 雿輻鍂<EFBFBD>?
|
||||
```
|
||||
|
||||
### Q2:不用Redis队列,用数据库队列可以吗?
|
||||
**A2**:可以但不推荐。
|
||||
### Q2嚗帋<EFBFBD><EFBFBD>沖edis<EFBFBD>笔<EFBFBD>嚗𣬚鍂<EFBFBD>唳旿摨㯄<EFBFBD><EFBFBD>堒虾隞亙<EFBFBD>嚗?
|
||||
**A2**嚗𡁜虾隞乩<EFBFBD>銝齿綫<EFBFBD>僐<EFBFBD>?
|
||||
|
||||
| <20>寞<EFBFBD> | 隡睃飵 | <20><>飵 |
|
||||
|------|------|------|
|
||||
| **Redis队列** | 快(< 1ms)、功能完善 | 需要Redis |
|
||||
| **数据库队列** | 不需要额外依赖 | 慢(> 10ms)、功能简陋 |
|
||||
| **Redis<EFBFBD>笔<EFBFBD>** | 敹恬<EFBFBD>< 1ms嚗剹<EFBFBD><EFBFBD><EFBFBD><EFBFBD>賢<EFBFBD><EFBFBD>?| <20><>閬<EFBFBD>edis |
|
||||
| **<EFBFBD>唳旿摨㯄<EFBFBD><EFBFBD>?* | 銝漤<E98A9D>閬<EFBFBD><E996AC>憭碶<E686AD>韏?| <20>g<EFBFBD>> 10ms嚗剹<E59A97><E589B9><EFBFBD><EFBFBD>賜<EFBFBD><E8B39C>?|
|
||||
|
||||
您已经购买了Redis,建议直接用Redis队列。
|
||||
<EFBFBD>典歇蝏讛揚銋唬<EFBFBD>Redis嚗<EFBFBD>遣霈桃凒<EFBFBD>亦鍂Redis<EFBFBD>笔<EFBFBD><EFBFBD>?
|
||||
|
||||
### Q3:如果只启用Redis缓存,系统会有问题吗?
|
||||
**A3**:短期内不会有大问题,但存在风险。
|
||||
### Q3嚗𡁜<EFBFBD><EFBFBD>𨅯蘨<EFBFBD>舐鍂Redis蝻枏<EFBFBD>嚗𣬚頂蝏煺<EFBFBD><EFBFBD>厰䔮憸睃<EFBFBD>嚗?
|
||||
**A3**嚗𡁶<EFBFBD><EFBFBD>笔<EFBFBD>銝滢<EFBFBD><EFBFBD>匧之<EFBFBD>桅<EFBFBD>嚗䔶<EFBFBD>摮睃銁憌𡡞埯<EFBFBD>?
|
||||
|
||||
```
|
||||
✅ 可以正常运行:
|
||||
- LLM成本控制 ✅
|
||||
- 多实例缓存共享 ✅
|
||||
<EFBFBD>?<3F>臭誑甇<E8AA91>虜餈鞱<E9A488>嚗?
|
||||
- LLM<EFBFBD>鞉𧋦<EFBFBD>批<EFBFBD> <20>?
|
||||
- 憭𡁜<EFBFBD>靘讠<EFBFBD>摮睃<EFBFBD>鈭?<3F>?
|
||||
|
||||
⚠️ 潜在风险:
|
||||
- 长任务可能丢失(实例重启)
|
||||
<EFBFBD>𩤃<EFBFBD> 瞏𨅯銁憌𡡞埯嚗?
|
||||
- <EFBFBD>蹂遙<EFBFBD>∪虾<EFBFBD>賭腺憭梧<EFBFBD>摰硺<EFBFBD><EFBFBD>滚鍳嚗?
|
||||
- <20>䭾<EFBFBD><E4ADBE>滩<EFBFBD>憭梯揖隞餃𦛚
|
||||
- <20>冽<EFBFBD>雿㯄<E99BBF>銝滢蔔
|
||||
```
|
||||
|
||||
### Q4:启用Redis队列后,能否回退到MemoryQueue?
|
||||
**A4**:可以!修改环境变量即可。
|
||||
### Q4嚗𡁜鍳<EFBFBD>沖edis<EFBFBD>笔<EFBFBD><EFBFBD>𠬍<EFBFBD><EFBFBD>賢炏<EFBFBD>鮋<EFBFBD><EFBFBD><EFBFBD>衽emoryQueue嚗?
|
||||
**A4**嚗𡁜虾隞伐<EFBFBD>靽格㺿<EFBFBD>臬<EFBFBD><EFBFBD>㗛<EFBFBD><EFBFBD>喳虾<EFBFBD>?
|
||||
|
||||
```bash
|
||||
# <20><>揢<EFBFBD>訌edis<69>笔<EFBFBD>
|
||||
QUEUE_TYPE=redis
|
||||
|
||||
# 回退到内存队列
|
||||
# <EFBFBD>鮋<EFBFBD><EFBFBD><EFBFBD>啣<EFBFBD>摮㗛<EFBFBD><EFBFBD>?
|
||||
QUEUE_TYPE=memory
|
||||
```
|
||||
|
||||
但注意:**Redis中的未完成任务会丢失**。
|
||||
雿<EFBFBD>釣<EFBFBD>𧶏<EFBFBD>**Redis銝剔<EFBFBD><EFBFBD>芸<EFBFBD><EFBFBD>𣂷遙<EFBFBD>∩<EFBFBD>銝W仃**<2A>?
|
||||
|
||||
### Q5嚗鋳ullMQ<4D>曉郎<E69B89>梹<EFBFBD>
|
||||
**A5**:不难,核心API只有5个。
|
||||
**A5**嚗帋<EFBFBD><EFBFBD>橘<EFBFBD><EFBFBD>詨<EFBFBD>API<EFBFBD>芣<EFBFBD>5銝芥<EFBFBD>?
|
||||
|
||||
```typescript
|
||||
// 1. <20>𥕦遣<F0A595A6>笔<EFBFBD>
|
||||
@@ -645,7 +645,7 @@ worker.on('failed', ...);
|
||||
const job = await queue.getJob(jobId);
|
||||
```
|
||||
|
||||
预计学习时间:**半天**
|
||||
憸<EFBFBD>恣摮虫<EFBFBD><EFBFBD>園𡢿嚗?*<2A>𠰴予**
|
||||
|
||||
---
|
||||
|
||||
@@ -653,51 +653,52 @@ const job = await queue.getJob(jobId);
|
||||
|
||||
### 8.1 <20>詨<EFBFBD>閬<EFBFBD><E996AC>
|
||||
|
||||
1. **Redis缓存 ≠ Redis队列**
|
||||
1. **Redis蝻枏<EFBFBD> <20>?Redis<EFBFBD>笔<EFBFBD>**
|
||||
- <20>賣糓雿輻鍂<E8BCBB>䔶<EFBFBD>銝析edis摰硺<E691B0>
|
||||
- <20>芣糓雿輻鍂<E8BCBB>孵<EFBFBD>銝滚<E98A9D>
|
||||
|
||||
2. **<2A>函<EFBFBD>蝟餌<E89D9F><E9A48C>啁𠶖**
|
||||
- BullMQ已安装但未使用
|
||||
- BullMQ撌脣<EFBFBD>鋆<EFBFBD><EFBFBD><EFBFBD>芯蝙<EFBFBD>?
|
||||
- 敶枏<E695B6><E69E8F>決emoryQueue嚗<65>䌊<EFBFBD>𥪜<EFBFBD>摮㗛<E691AE><E3979B>梹<EFBFBD>
|
||||
- <20><>閬<EFBFBD>㺿<EFBFBD>䭾<EFBFBD><E4ADBE>賢鍳<E8B3A2>沖edis<69>笔<EFBFBD>
|
||||
|
||||
3. **<2A>刻<EFBFBD><E588BB>寞<EFBFBD>**
|
||||
- ✅ 先启用Redis缓存(本周,2天)
|
||||
- ✅ 再启用Redis队列(下周,3天)
|
||||
- ✅ 渐进式实施,降低风险
|
||||
- <EFBFBD>?<3F><>鍳<EFBFBD>沖edis蝻枏<E89DBB>嚗<EFBFBD>𧋦<EFBFBD>剁<EFBFBD>2憭抬<E686AD>
|
||||
- <EFBFBD>?<3F>滚鍳<E6BB9A>沖edis<69>笔<EFBFBD>嚗<EFBFBD><E59A97><EFBFBD>剁<EFBFBD>3憭抬<E686AD>
|
||||
- <EFBFBD>?皜鞱<E79A9C>撘誩<E69298><E8AAA9>踝<EFBFBD><E8B89D>滢<EFBFBD>憌𡡞埯
|
||||
|
||||
4. **只用Redis缓存的影响**
|
||||
- ✅ 可以解决LLM成本问题
|
||||
- ⚠️ 长任务可靠性仍有风险
|
||||
- 📊 建议:根据用户规模决定是否启用队列
|
||||
4. **<EFBFBD>芰鍂Redis蝻枏<EFBFBD><EFBFBD><EFBFBD>蔣<EFBFBD>?*
|
||||
- <EFBFBD>?<3F>臭誑閫<E8AA91><E996AB>LLM<4C>鞉𧋦<E99E89>桅<EFBFBD>
|
||||
- <EFBFBD>𩤃<EFBFBD> <20>蹂遙<E8B982>∪虾<E288AA>䭾<EFBFBD>找<EFBFBD><E689BE>厰<EFBFBD><E58EB0>?
|
||||
- <EFBFBD><EFBFBD> 撱箄悅嚗𡁏覔<F0A1818F>桃鍂<E6A183>瑁<EFBFBD>璅∪<E79285>摰𡁏糓<F0A1818F>血鍳<E8A180>券<EFBFBD><E588B8>?
|
||||
|
||||
5. **Redis队列的风险**
|
||||
5. **Redis<EFBFBD>笔<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>?*
|
||||
- <20>椬 憓𧼮<E68693>憭齿<E686AD>摨佗<E691A8>雿<EFBFBD>虾<EFBFBD>改<EFBFBD>
|
||||
- 🟢 依赖Redis稳定性(高可用版99.95%)
|
||||
- ✅ 收益 > 风险
|
||||
- <EFBFBD>叚 靘肽<E99D98>Redis蝔喳<E89D94><E596B3>改<EFBFBD>擃睃虾<E79D83>函<EFBFBD>99.95%嚗?
|
||||
- <EFBFBD>?<3F>嗥<EFBFBD> > 憌𡡞埯
|
||||
|
||||
---
|
||||
|
||||
### 8.2 <20>喟<EFBFBD>撱箄悅
|
||||
|
||||
**如果您的系统满足以下条件之一,建议尽快启用Redis队列:**
|
||||
**憒<EFBFBD><EFBFBD><EFBFBD>函<EFBFBD>蝟餌<EFBFBD>皛∟雲隞乩<EFBFBD><EFBFBD>∩辣銋衤<EFBFBD>嚗<EFBFBD>遣霈桀偷敹怠鍳<EFBFBD>沖edis<EFBFBD>笔<EFBFBD>嚗?*
|
||||
|
||||
```
|
||||
✅ 用户数 > 20
|
||||
✅ SAE实例数 > 1
|
||||
✅ 批量任务频繁(每天 > 5个)
|
||||
✅ 任务时长 > 10分钟
|
||||
✅ 用户抱怨任务丢失
|
||||
<EFBFBD>?<3F>冽<EFBFBD><E586BD>?> 20
|
||||
<EFBFBD>?SAE摰硺<EFBFBD><EFBFBD>?> 1
|
||||
<EFBFBD>?<3F>寥<EFBFBD>隞餃𦛚憸𤑳<E686B8>嚗<EFBFBD><E59A97>憭?> 5銝迎<E98A9D>
|
||||
<EFBFBD>?隞餃𦛚<E9A483>園鵭 > 10<EFBFBD><EFBFBD><EFBFBD>
|
||||
<EFBFBD>?<3F>冽<EFBFBD><E586BD>望<EFBFBD>其遙<E585B6>∩腺憭?
|
||||
```
|
||||
|
||||
**否则,可以先启用Redis缓存,队列暂缓。**
|
||||
**<EFBFBD>血<EFBFBD>嚗<EFBFBD>虾隞亙<EFBFBD><EFBFBD>舐鍂Redis蝻枏<EFBFBD>嚗屸<EFBFBD><EFBFBD>埈<EFBFBD>蝻瓐<EFBFBD>?*
|
||||
|
||||
---
|
||||
|
||||
**文档维护者:** 技术团队
|
||||
**<EFBFBD><EFBFBD>﹝蝏湔擪<EFBFBD><EFBFBD><EFBFBD>** <20><><EFBFBD>臬𣪧<E887AC>?
|
||||
**<EFBFBD><EFBFBD><EFBFBD>擧凒<EFBFBD>堆<EFBFBD>** 2025-12-12
|
||||
**相关文档:** [Redis改造实施计划](./04-Redis改造实施计划.md)
|
||||
**<EFBFBD>詨<EFBFBD><EFBFBD><EFBFBD>﹝嚗?* [Redis<EFBFBD>寥<EFBFBD>惩<EFBFBD><EFBFBD>質恣<EFBFBD>哋(./04-Redis<69>寥<EFBFBD>惩<EFBFBD><E683A9>質恣<E8B3AA>?md)
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -1,9 +1,9 @@
|
||||
# 髟ソ譌カ髣エ莉サ蜉。蜿ッ髱<EFBDAF>諤ァ蛻<EFBDA7>梵<EFBFBD>哺emoryQueue vs Redis髦溷<E9ABA6>
|
||||
|
||||
> **场景:** 1000篇文献筛选,预计2小时处理时间
|
||||
> **当前方案:** MemoryQueue(内存队列)
|
||||
> **问题:** 能否可靠完成?
|
||||
> **结论:** ❌ **不能**
|
||||
> **蝨コ譎ッ<EFBFBD>?* 1000遽<30>枚迪ョ遲幃会シ碁「<E7A281>ョ。2蟆乗慮螟<E685AE>炊譌カ髣エ
|
||||
> **蠖灘燕譁ケ譯茨シ?* MemoryQueue<EFBFBD>亥<EFBFBD>蟄倬弌蛻暦シ<EFBFBD>
|
||||
> **髣ョ鬚假シ?* 閭ス蜷ヲ蜿ッ髱<EFBDAF>螳梧<E89EB3><E6A2A7>?
|
||||
> **扈楢ョコ<EFBFBD>?* 笶?**荳崎<E88DB3>**
|
||||
|
||||
---
|
||||
|
||||
@@ -11,103 +11,103 @@
|
||||
|
||||
### 莉サ蜉。迚ケ蠕<EFBDB9>
|
||||
```
|
||||
任务类型:文献筛选(标题摘要初筛)
|
||||
文献数量:1000篇
|
||||
单篇耗时:6-10秒(双模型并行)
|
||||
总耗时:6000-10000秒 = 100-167分钟 ≈ 2小时
|
||||
莉サ蜉。邀サ蝙具シ壽枚迪ョ遲幃会シ域<EFBFBD><EFBFBD>「俶遭隕∝<EFBFBD>遲幢シ?
|
||||
譁<EFBFBD>鍵謨ー驥擾シ?000遽?
|
||||
蜊慕ッ<EFBFBD>玲慮<EFBFBD>?-10遘抵シ亥曙讓。蝙句ケカ陦鯉シ<E9AF89>
|
||||
諤サ閠玲慮<EFBFBD>?000-10000遘?= 100-167蛻<EFBFBD>帖 竕?2蟆乗慮
|
||||
```
|
||||
|
||||
### 蠖灘燕螳樒鴫
|
||||
```typescript
|
||||
// backend/src/modules/asl/services/screeningService.ts (第65行)
|
||||
// backend/src/modules/asl/services/screeningService.ts (隨?5陦?
|
||||
|
||||
// 4. 异步处理文献(简化版:直接在这里处理)
|
||||
// 生产环境应该发送到消息队列 ← 注意这行注释!
|
||||
// 4. 蠑よュ・螟<EFBFBD>炊譁<EFBFBD>鍵<EFBFBD>育ョ蛹也沿<EFBFBD>夂峩謗・蝨ィ霑咎㈹螟<EFBFBD>炊<EFBFBD>?
|
||||
// 逕滉コァ邇ッ蠅<EFBFBD>コ碑ッ・蜿鷹∝芦豸域<EFBFBD>髦溷<EFBFBD> 竊?豕ィ諢剰ソ呵。梧ウィ驥奇シ?
|
||||
processLiteraturesInBackground(task.id, projectId, literatures);
|
||||
|
||||
// 霑吩クェ蜃ス謨ー莨夲シ<E5A4B2>
|
||||
// 1. 运行在当前Node进程中
|
||||
// 2. 串行处理1000篇文献
|
||||
// 3. 没有持久化(全在内存)
|
||||
// 1. 霑占。悟惠蠖灘燕Node霑帷ィ倶ク?
|
||||
// 2. 荳イ陦悟、<EFBFBD>炊1000遽<EFBFBD>枚迪?
|
||||
// 3. 豐。譛画戟荵<EFBFBD>喧<EFBFBD>亥<EFBFBD>蝨ィ蜀<EFBFBD>ュ假シ?
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ❌ **MemoryQueue的致命问题**
|
||||
## 笶?**MemoryQueue逧<EFBFBD><EFBFBD>蜻ス髣ョ鬚?*
|
||||
|
||||
### 问题1:SAE实例会被自动销毁 🔥 **最严重**
|
||||
### 髣ョ鬚<EFBFBD>1<EFBFBD>售AE螳樔セ倶シ夊「ォ閾ェ蜉ィ髞豈?<3F>櫨 **譛荳・驥<EFBDA5>**
|
||||
|
||||
#### **Serverless的本质:按需计费 = 按需销毁**
|
||||
#### **Serverless逧<EFBFBD>悽雍ィ<EFBFBD>壽潔髴隶。雍ケ = 謖蛾怙髞豈?*
|
||||
|
||||
```
|
||||
髦ソ驥御コ全AE逧<EFBFBD><EFBFBD>蜉ィ郛ゥ螳ケ遲也払<EFBFBD><EFBFBD>
|
||||
├─ 无流量时:15分钟后缩容到0
|
||||
笏懌楳 譌<>豬<EFBFBD>㍼譌カ<E8AD8C>?5蛻<35>帖蜷守シゥ螳ケ蛻ー0
|
||||
笏懌楳 菴取オ<E58F96>㍼譌カ<E8AD8C>夂シゥ蜃丞ョ樔セ区焚
|
||||
├─ 夜间时段:自动缩容(节省成本)
|
||||
└─ 系统升级:实例重启
|
||||
笏懌楳 螟憺龍譌カ谿オ<E8B0BF>夊<EFBFBD>蜉ィ郛ゥ螳ケ<E89EB3>郁鰍逵∵<E980B5>譛ャ<E8AD9B>?
|
||||
笏披楳 邉サ扈溷合郤ァ<E983A4>壼ョ樔セ矩㍾蜷?
|
||||
```
|
||||
|
||||
#### **2小时任务的风险评估**
|
||||
#### **2蟆乗慮莉サ蜉。逧<EFBFBD>」朱勦隸<EFBFBD>シ?*
|
||||
|
||||
| 时段 | SAE实例销毁概率 | 说明 |
|
||||
| 譌カ谿オ | SAE螳樔セ矩楳豈∵ヲら<EFBFBD>?| 隸エ譏<EFBDB4> |
|
||||
|------|----------------|------|
|
||||
| **工作时间(9:00-18:00)** | 🟡 30-50% | 流量波动导致缩容 |
|
||||
| **夜间时段(22:00-06:00)** | 🔴 80-95% | 自动缩容策略 |
|
||||
| **周末/节假日** | 🔴 70-90% | 低流量时段 |
|
||||
| **蟾・菴懈慮髣エ<EFBFBD>?:00-18:00<EFBFBD>?* | <EFBFBD>泯 30-50% | 豬<EFBFBD>㍼豕「蜉ィ蟇シ閾エ郛ゥ螳ケ |
|
||||
| **螟憺龍譌カ谿オ<EFBFBD>?2:00-06:00<EFBFBD>?* | <EFBFBD>閥 80-95% | 閾ェ蜉ィ郛ゥ螳ケ遲也払 |
|
||||
| **蜻ィ譛ォ/闃ょ∞譌?* | <EFBFBD>閥 70-90% | 菴取オ<EFBFBD>㍼譌カ谿?|
|
||||
|
||||
**真实场景模拟**:
|
||||
**逵溷ョ槫惻譎ッ讓。諡<EFBFBD>**<EFBFBD>?
|
||||
```
|
||||
21:00 用户提交1000篇文献筛选
|
||||
21:00 SAE实例开始处理(预计2小时完成)
|
||||
21:15 前端有用户访问(实例存活)
|
||||
21:00 逕ィ謌キ謠蝉コ、1000遽<EFBFBD>枚迪ョ遲幃?
|
||||
21:00 SAE螳樔セ句シ蟋句、<EFBFBD>炊<EFBFBD>磯「<EFBFBD>ョ。2蟆乗慮螳梧<EFBFBD><EFBFBD>?
|
||||
21:15 蜑咲ォッ譛臥畑謌キ隶ソ髣ョ<EFBFBD>亥ョ樔セ句ュ俶エサ<EFBFBD>?
|
||||
22:00 逕ィ謌キ荳狗少蝗槫ョカ<EFBDAE>域裏譁ー隶ソ髣ョ<E9ABA3><EFBDAE>
|
||||
22:15 SAE检测:15分钟无流量 → 准备缩容
|
||||
22:16 ❌ 实例被销毁
|
||||
└─ 任务进度:150/1000(15%)
|
||||
22:15 SAE譽豬具シ<EFBFBD>15蛻<EFBFBD>帖譌<EFBFBD>豬<EFBFBD><EFBFBD>?竊?蜃<>、<EFBFBD>シゥ螳ケ
|
||||
22:16 笶?螳樔セ玖「ォ髞豈?
|
||||
笏披楳 莉サ蜉。霑帛コヲ<EFBDBA>?50/1000<EFBFBD>?5%<25>?
|
||||
笏披楳 扈捺棡<E68DBA>壻ササ蜉。荳「螟ア<E89E9F>悟燕蜉溷ース蠑<EFBDBD>
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 问题2:进程崩溃无法恢复
|
||||
### 髣ョ鬚<EFBFBD>2<EFBFBD>夊ソ帷ィ句エゥ貅<EFBFBD>裏豕墓△螟?
|
||||
|
||||
```typescript
|
||||
// 当前实现(简化版)
|
||||
// 蠖灘燕螳樒鴫<EFBFBD>育ョ蛹也沿<EFBFBD>?
|
||||
async function processLiteraturesInBackground(taskId, projectId, literatures) {
|
||||
for (const lit of literatures) {
|
||||
try {
|
||||
// 螟<>炊蜊慕ッ<E68595>枚迪ョ<E8BFAA>郁玲慮6-10遘抵シ<E68AB5>
|
||||
await processLiterature(lit);
|
||||
} catch (error) {
|
||||
// 某篇失败,继续下一篇
|
||||
// 譟千ッ<EFBFBD>、ア雍・<EFBFBD>檎サァ扈ュ荳倶ク遽?
|
||||
logger.error('Failed to process literature', { error });
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 风险:
|
||||
// 鬟朱勦<EFBFBD>?
|
||||
// 1. 螯よ棡Node霑帷ィ句エゥ貅<EFBDA9>シ<EFBFBD>OM縲∵悴謐戊執蠑ょクク<EFBDB8>俄<EFBFBD> 蜈ィ驛ィ荳「螟ア
|
||||
// 2. 如果DB连接断开 → 无法保存进度
|
||||
// 3. 如果API限流 → 任务卡死
|
||||
// 4. 没有断点续传 → 必须重头开始
|
||||
// 2. 螯よ棡DB霑樊磁譁ュ蠑 竊?譌<>豕穂ソ晏ュ倩ソ帛コヲ
|
||||
// 3. 螯よ棡API髯先オ<EFBFBD> 竊?莉サ蜉。蜊。豁サ
|
||||
// 4. 豐。譛画妙轤ケ扈ュ莨<EFBFBD> 竊?蠢<>。サ驥榊、エ蠑蟋?
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 问题3:无法监控真实进度
|
||||
### 髣ョ鬚<EFBFBD>3<EFBFBD>壽裏豕慕尅謗ァ逵溷ョ櫁ソ帛コ?
|
||||
|
||||
```typescript
|
||||
// 当前实现的进度更新
|
||||
// 蠖灘燕螳樒鴫逧<EFBFBD>ソ帛コヲ譖エ譁?
|
||||
await prisma.aslScreeningTask.update({
|
||||
where: { id: taskId },
|
||||
data: { processedItems: processedCount }
|
||||
});
|
||||
|
||||
// 问题:
|
||||
// 髣ョ鬚假シ?
|
||||
// - 霑帛コヲ蜿ェ蟄伜惠謨ー謐ョ蠎<EFBDAE>
|
||||
// - 任务状态在内存中
|
||||
// - 莉サ蜉。迥カ諤∝惠蜀<EFBFBD>ュ倅ク?
|
||||
// - 螳樔セ矩楳豈∝錘<E2889D>梧焚謐ョ蠎捺仞遉コ processedItems: 150
|
||||
// - 但任务实际已丢失,无法恢复
|
||||
// - 菴<EFBFBD>ササ蜉。螳樣刔蟾イ荳「螟ア<EFBFBD>梧裏豕墓△螟?
|
||||
```
|
||||
|
||||
---
|
||||
@@ -115,24 +115,24 @@ await prisma.aslScreeningTask.update({
|
||||
### 髣ョ鬚<EFBDAE>4<EFBFBD>壼、壼ョ樔セ句<EFBDBE>遯<EFBFBD>
|
||||
|
||||
```
|
||||
场景:SAE有2个实例
|
||||
蝨コ譎ッ<EFBFBD>售AE譛?荳ェ螳樔セ?
|
||||
|
||||
用户提交任务 → 实例A开始处理
|
||||
↓
|
||||
处理到500篇时,实例A销毁
|
||||
↓
|
||||
用户刷新页面 → 请求路由到实例B
|
||||
↓
|
||||
逕ィ謌キ謠蝉コ、莉サ蜉。 竊?螳樔セ帰蠑蟋句、<E58FA5><EFBDA4>?
|
||||
竊?
|
||||
螟<EFBFBD>炊蛻?00遽<30>慮<EFBFBD>悟ョ樔セ帰髞豈?
|
||||
竊?
|
||||
逕ィ謌キ蛻キ譁ー鬘オ髱「 竊?隸キ豎りキッ逕ア蛻ー螳樔セ毅
|
||||
竊?
|
||||
螳樔セ毅隸サ蜿紋ササ蜉。迥カ諤<EFBFBD>シ嗔rocessedItems: 500
|
||||
↓
|
||||
竊?
|
||||
螳樔セ毅荳咲衍驕謎ササ蜉。蟾イ荳ュ譁ュ
|
||||
↓
|
||||
❌ 任务显示"进行中",但实际没人在处理
|
||||
竊?
|
||||
笶?莉サ蜉。譏セ遉コ"霑幄。御ク?<3F>御ス<E5BEA1>ョ樣刔豐。莠コ蝨ィ螟<EFBDA8><E89E9F>?
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ✅ **Redis队列的优势**
|
||||
## 笨?**Redis髦溷<EFBFBD>逧<EFBFBD>シ伜<EFBFBD>?*
|
||||
|
||||
### 莨伜漢1<E6BCA2>壻ササ蜉。謖∽ケ<E288BD>喧
|
||||
|
||||
@@ -145,12 +145,12 @@ await jobQueue.push('asl:screening', {
|
||||
});
|
||||
|
||||
// 莉サ蜉。菫晏ュ伜惠Redis荳ュ<E88DB3><EFBDAD>
|
||||
// - 实例销毁 → ✅ 任务仍在Redis
|
||||
// - 新实例启动 → ✅ 自动拾取任务
|
||||
// - 进程崩溃 → ✅ 其他Worker接管
|
||||
// - 螳樔セ矩楳豈?竊?笨?莉サ蜉。莉榊惠Redis
|
||||
// - 譁ー螳樔セ句星蜉?竊?笨?閾ェ蜉ィ諡セ蜿紋ササ蜉。
|
||||
// - 霑帷ィ句エゥ貅<EFBFBD> 竊?笨?蜈カ莉妨orker謗・邂。
|
||||
```
|
||||
|
||||
### 优势2:断点续传
|
||||
### 莨伜漢2<EFBFBD>壽妙轤ケ扈ュ莨?
|
||||
|
||||
```typescript
|
||||
// Worker螟<72>炊莉サ蜉。
|
||||
@@ -161,234 +161,234 @@ jobQueue.process('asl:screening', async (job) => {
|
||||
// 螟<>炊譁<E7828A>鍵
|
||||
await processLiterature(literatureIds[i]);
|
||||
|
||||
// 更新进度(保存到Redis)
|
||||
// 譖エ譁ー霑帛コヲ<EFBFBD>井ソ晏ュ伜芦Redis<EFBFBD>?
|
||||
await job.updateProgress((i + 1) / literatureIds.length * 100);
|
||||
|
||||
// 螯よ棡Worker蝨ィ霑咎㈹蟠ゥ貅<EFBDA9>シ<EFBFBD>
|
||||
// - BullMQ会将任务标记为"停滞"
|
||||
// - 其他Worker会重新拾取
|
||||
// - BullMQ莨壼ー<EFBFBD>ササ蜉。譬<EFBFBD>ョー荳?蛛懈サ<E68788>"
|
||||
// - 蜈カ莉妨orker莨夐㍾譁ー諡セ蜿?
|
||||
// - 莉惹ク頑ャ。霑帛コヲ扈ァ扈ュ<E68988>郁御ク肴弍驥榊、エ蠑蟋具シ<E585B7>
|
||||
}
|
||||
});
|
||||
```
|
||||
|
||||
### 优势3:自动重试
|
||||
### 莨伜漢3<EFBFBD>夊<EFBFBD>蜉ィ驥崎ッ?
|
||||
|
||||
```typescript
|
||||
// BullMQ驟咲スョ
|
||||
const queue = new Queue('asl:screening', {
|
||||
connection: { host: 'redis' },
|
||||
defaultJobOptions: {
|
||||
attempts: 3, // 失败后重试3次
|
||||
attempts: 3, // 螟ア雍・蜷朱㍾隸?谺?
|
||||
backoff: {
|
||||
type: 'exponential',
|
||||
delay: 2000 // 2秒、4秒、8秒
|
||||
delay: 2000 // 2遘偵?遘偵?遘?
|
||||
},
|
||||
removeOnComplete: true, // 完成后清理
|
||||
removeOnFail: false // 失败后保留(便于排查)
|
||||
removeOnComplete: true, // 螳梧<EFBFBD>蜷取ク<EFBFBD><EFBFBD>?
|
||||
removeOnFail: false // 螟ア雍・蜷惹ソ晉蕗<EFBFBD>井セソ莠取賜譟・<EFBFBD>?
|
||||
}
|
||||
});
|
||||
|
||||
// 场景:
|
||||
// - LLM API临时故障 → ✅ 自动重试
|
||||
// - 网络抖动 → ✅ 自动重试
|
||||
// - DB连接断开 → ✅ 自动重试
|
||||
// 蝨コ譎ッ<EFBFBD>?
|
||||
// - LLM API荳エ譌カ謨<EFBFBD>囿 竊?笨?閾ェ蜉ィ驥崎ッ<E5B48E>
|
||||
// - 鄂醍サ懈竃蜉ィ 竊?笨?閾ェ蜉ィ驥崎ッ<E5B48E>
|
||||
// - DB霑樊磁譁ュ蠑 竊?笨?閾ェ蜉ィ驥崎ッ<E5B48E>
|
||||
```
|
||||
|
||||
### 莨伜漢4<E6BCA2>壼<EFBFBD>蟶<EFBFBD>シ丈ササ蜉。蛻<EFBDA1><E89BBB>
|
||||
|
||||
```
|
||||
SAE有3个实例:
|
||||
SAE譛?荳ェ螳樔セ具シ<E585B7>
|
||||
|
||||
Redis队列(1000个任务)
|
||||
↓
|
||||
自动分配:
|
||||
├─ 实例A Worker:处理 Task 1-350
|
||||
├─ 实例B Worker:处理 Task 351-700
|
||||
└─ 实例C Worker:处理 Task 701-1000
|
||||
Redis髦溷<EFBFBD><EFBFBD>?000荳ェ莉サ蜉。<E89C89><EFBDA1>
|
||||
竊?
|
||||
閾ェ蜉ィ蛻<EFBFBD><EFBFBD><EFBFBD>?
|
||||
笏懌楳 螳樔セ帰 Worker<EFBFBD>壼、<EFBFBD><EFBFBD>?Task 1-350
|
||||
笏懌楳 螳樔セ毅 Worker<EFBFBD>壼、<EFBFBD><EFBFBD>?Task 351-700
|
||||
笏披楳 螳樔セ気 Worker<EFBFBD>壼、<EFBFBD><EFBFBD>?Task 701-1000
|
||||
|
||||
螯よ棡螳樔セ毅髞豈<EFBFBD>シ<EFBFBD>
|
||||
├─ Task 351-700 标记为"停滞"
|
||||
笏懌楳 Task 351-700 譬<EFBFBD>ョー荳?蛛懈サ<E68788>"
|
||||
笏懌楳 螳樔セ帰謌砲逧Цorker閾ェ蜉ィ謗・邂。
|
||||
笏披楳 扈ァ扈ュ螟<EFBDAD>炊<EFBFBD>梧裏髴莠コ蟾・蟷イ鬚<EFBDB2>
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📊 **可靠性对比**
|
||||
## <EFBFBD>投 **蜿ッ髱<EFBFBD>諤ァ蟇ケ豈?*
|
||||
|
||||
| 扈エ蠎ヲ | MemoryQueue | Redis髦溷<E9ABA6> | 蟾ョ蠑<EFBDAE> |
|
||||
|------|------------|----------|------|
|
||||
| **2小时任务完成率** | 10-30% | 99%+ | **300%提升** |
|
||||
| **实例销毁后** | ❌ 任务丢失 | ✅ 自动恢复 | **关键** |
|
||||
| **进程崩溃后** | ❌ 全部丢失 | ✅ 断点续传 | **关键** |
|
||||
| **API临时故障** | ❌ 任务失败 | ✅ 自动重试 | **关键** |
|
||||
| **多实例协调** | ❌ 无法协调 | ✅ 自动分配 | **关键** |
|
||||
| **任务监控** | ⚠️ 仅DB | ✅ 实时状态 | 可选 |
|
||||
| **成本** | ¥0 | ¥108/年 | 可接受 |
|
||||
| **2蟆乗慮莉サ蜉。螳梧<EFBFBD>邇?* | 10-30% | 99%+ | **300%謠仙合** |
|
||||
| **螳樔セ矩楳豈∝錘** | 笶?莉サ蜉。荳「螟ア | 笨?閾ェ蜉ィ諱「螟<EFBDA2> | **蜈ウ髞ョ** |
|
||||
| **霑帷ィ句エゥ貅<EFBFBD><EFBFBD>?* | 笶?蜈ィ驛ィ荳「螟ア | 笨?譁ュ轤ケ扈ュ莨<EFBDAD> | **蜈ウ髞ョ** |
|
||||
| **API荳エ譌カ謨<EFBFBD>囿** | 笶?莉サ蜉。螟ア雍・ | 笨?閾ェ蜉ィ驥崎ッ<E5B48E> | **蜈ウ髞ョ** |
|
||||
| **螟壼ョ樔セ句刻隹?* | 笶?譌<>豕募刻隹<E588BB> | 笨?閾ェ蜉ィ蛻<EFBDA8><E89BBB> | **蜈ウ髞ョ** |
|
||||
| **莉サ蜉。逶第而** | 笞<EFBFBD><EFBFBD><EFBFBD> 莉<>B | 笨?螳樊慮迥カ諤?| 蜿ッ騾?|
|
||||
| **謌先悽** | ツ・0 | ツ・108/蟷?| 蜿ッ謗・蜿?|
|
||||
|
||||
---
|
||||
|
||||
## <20>識 **逵溷ョ槫惻譎ッ讓。諡<EFBDA1>**
|
||||
|
||||
### 场景1:工作时间提交(成功率30%)
|
||||
### 蝨コ譎ッ1<EFBFBD>壼キ・菴懈慮髣エ謠蝉コ、<EFBFBD>域<EFBFBD>蜉溽<EFBFBD>?0%<25>?
|
||||
|
||||
```
|
||||
10:00 用户提交1000篇文献筛选
|
||||
10:00 逕ィ謌キ謠蝉コ、1000遽<EFBFBD>枚迪ョ遲幃?
|
||||
笏懌楳 MemoryQueue<75>壼シ蟋句、<E58FA5>炊<EFBFBD>碁「<E7A281>ョ。12:00螳梧<E89EB3>
|
||||
│
|
||||
11:30 流量降低,SAE缩容(删除1个实例)
|
||||
├─ 如果任务在被删除的实例上 → ❌ 丢失(概率50%)
|
||||
│
|
||||
12:00 如果幸运未被删除 → ✅ 完成(概率50%)
|
||||
笏?
|
||||
11:30 豬<EFBFBD>㍼髯堺ス趣シ郡AE郛ゥ螳ケ<EFBFBD>亥唖髯?荳ェ螳樔セ具シ<E585B7>
|
||||
笏懌楳 螯よ棡莉サ蜉。蝨ィ陲ォ蛻<EFBDAB>髯、逧<EFBDA4>ョ樔セ倶ク<E580B6> 竊?笶?荳「螟ア<E89E9F>域ヲら<EFBDA6>?0%<25>?
|
||||
笏?
|
||||
12:00 螯よ棡蟷ク霑先悴陲ォ蛻<EFBFBD>髯、 竊?笨?螳梧<E89EB3><E6A2A7>域ヲら<EFBDA6>?0%<25>?
|
||||
|
||||
总成功率:50%
|
||||
諤サ謌仙粥邇<EFBFBD>シ?0%
|
||||
```
|
||||
|
||||
### 场景2:夜间提交(成功率5%)
|
||||
### 蝨コ譎ッ2<EFBFBD>壼、憺龍謠蝉コ、<EFBFBD>域<EFBFBD>蜉溽<EFBFBD>?%<25>?
|
||||
|
||||
```
|
||||
21:00 用户提交1000篇文献筛选
|
||||
21:00 逕ィ謌キ謠蝉コ、1000遽<EFBFBD>枚迪ョ遲幃?
|
||||
笏懌楳 MemoryQueue<75>壼シ蟋句、<E58FA5>炊<EFBFBD>碁「<E7A281>ョ。23:00螳梧<E89EB3>
|
||||
│
|
||||
21:15 无新用户访问,流量降为0
|
||||
│
|
||||
21:30 SAE检测:15分钟无流量 → 准备缩容
|
||||
│
|
||||
21:31 ❌ 实例销毁,任务丢失(概率95%)
|
||||
笏?
|
||||
21:15 譌<EFBFBD>譁ー逕ィ謌キ隶ソ髣ョ<EFBFBD>梧オ<EFBFBD>㍼髯堺ク?
|
||||
笏?
|
||||
21:30 SAE譽豬具シ<EFBFBD>15蛻<EFBFBD>帖譌<EFBFBD>豬<EFBFBD><EFBFBD>?竊?蜃<>、<EFBFBD>シゥ螳ケ
|
||||
笏?
|
||||
21:31 笶?螳樔セ矩楳豈<E6A5B3>シ御ササ蜉。荳「螟ア<E89E9F>域ヲら<EFBDA6>?5%<25>?
|
||||
|
||||
总成功率:5%
|
||||
諤サ謌仙粥邇<EFBFBD>シ?%
|
||||
```
|
||||
|
||||
### 场景3:Redis队列(成功率99%+)
|
||||
### 蝨コ譎ッ3<EFBFBD>啌edis髦溷<EFBFBD><EFBFBD>域<EFBFBD>蜉溽紫99%+<EFBFBD>?
|
||||
|
||||
```
|
||||
21:00 用户提交1000篇文献筛选
|
||||
├─ Redis队列:任务入队
|
||||
├─ Worker:开始处理
|
||||
│
|
||||
21:31 实例销毁
|
||||
21:00 逕ィ謌キ謠蝉コ、1000遽<EFBFBD>枚迪ョ遲幃?
|
||||
笏懌楳 Redis髦溷<EFBFBD><EFBFBD>壻ササ蜉。蜈・髦?
|
||||
笏懌楳 Worker<EFBFBD>壼シ蟋句、<EFBFBD><EFBFBD>?
|
||||
笏?
|
||||
21:31 螳樔セ矩楳豈?
|
||||
笏懌楳 莉サ蜉。菫晏ュ伜惠Redis
|
||||
│
|
||||
笏?
|
||||
21:32 譁ー螳樔セ句星蜉ィ<E89C89>域<EFBFBD>蜈カ莉門ョ樔セ具シ<E585B7>
|
||||
├─ Worker:自动拾取任务
|
||||
├─ 从Redis读取进度:已处理150篇
|
||||
├─ 继续处理剩余850篇
|
||||
│
|
||||
23:00 ✅ 任务完成
|
||||
笏懌楳 Worker<EFBFBD>夊<EFBFBD>蜉ィ諡セ蜿紋ササ蜉?
|
||||
笏懌楳 莉山edis隸サ蜿冶ソ帛コヲ<EFBDBA>壼キイ螟<EFBDB2>炊150遽?
|
||||
笏懌楳 扈ァ扈ュ螟<EFBDAD>炊蜑ゥ菴<EFBDA9>850遽?
|
||||
笏?
|
||||
23:00 笨?莉サ蜉。螳梧<E89EB3>
|
||||
|
||||
总成功率:99%+
|
||||
諤サ謌仙粥邇<EFBFBD>シ?9%+
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## <20>腸 **謌先悽蛻<E682BD>梵**
|
||||
|
||||
### MemoryQueue的隐藏成本
|
||||
### MemoryQueue逧<EFBFBD>嚼阯乗<EFBFBD>譛?
|
||||
|
||||
```
|
||||
莉サ蜉。螟ア雍・邇<EFBFBD>シ<EFBFBD>70%<25>亥、憺龍<E686BA><E9BE8D>
|
||||
用户重新提交次数:平均3次才成功
|
||||
LLM API浪费:
|
||||
- 第1次:处理200篇后失败 → 浪费 ¥86
|
||||
- 第2次:处理500篇后失败 → 浪费 ¥215
|
||||
- 第3次:完成 → ¥430
|
||||
总成本:¥731(应该只需¥430)
|
||||
逕ィ謌キ驥肴眠謠蝉コ、谺。謨ー<EFBFBD>壼ケウ蝮?谺。謇肴<E8AC87>蜉<EFBFBD>
|
||||
LLM API豬ェ雍ケ<EFBFBD>?
|
||||
- 隨?谺。<E8B0BA>壼、<E5A3BC>炊200遽<30>錘螟ア雍・ 竊?豬ェ雍ケ ツ・86
|
||||
- 隨?谺。<E8B0BA>壼、<E5A3BC>炊500遽<30>錘螟ア雍・ 竊?豬ェ雍ケ ツ・215
|
||||
- 隨?谺。<E8B0BA>壼ョ梧<EFBDAE> 竊?ツ・430
|
||||
諤サ謌先悽<EFBFBD>堋・731<EFBFBD>亥コ碑ッ・蜿ェ髴ツ・430<EFBFBD>?
|
||||
|
||||
用户体验:
|
||||
- 反复失败 → 投诉率上升
|
||||
- 不敢夜间提交 → 使用受限
|
||||
- 对系统失去信任 → 流失风险
|
||||
逕ィ謌キ菴馴ェ鯉シ?
|
||||
- 蜿榊、榊、ア雍・ 竊?謚戊ッ臥紫荳雁<E88DB3>?
|
||||
- 荳肴覆螟憺龍謠蝉コ、 竊?菴ソ逕ィ蜿鈴剞
|
||||
- 蟇ケ邉サ扈溷、ア蜴サ菫。莉?竊?豬∝、ア鬟朱勦
|
||||
```
|
||||
|
||||
### Redis队列的真实成本
|
||||
### Redis髦溷<EFBFBD>逧<EFBFBD>悄螳樊<EFBFBD>譛?
|
||||
|
||||
```
|
||||
Redis年费:¥108
|
||||
Redis蟷エ雍ケ<EFBFBD>堋?08
|
||||
莉サ蜉。謌仙粥邇<EFBFBD>シ<EFBFBD>99%+
|
||||
逕ィ謌キ驥肴眠謠蝉コ、谺。謨ー<EFBFBD>壼<EFBFBD>荵惹クコ0
|
||||
LLM API成本:¥430(无浪费)
|
||||
LLM API謌先悽<EFBFBD>堋?30<33>域裏豬ェ雍ケ<E99B8D>?
|
||||
|
||||
额外收益:
|
||||
- 用户满意度提升
|
||||
- 可以支持更大批量(5000篇+)
|
||||
鬚晏、匁噺逶奇シ?
|
||||
- 逕ィ謌キ貊。諢丞コヲ謠仙<EFBFBD>?
|
||||
- 蜿ッ莉・謾ッ謖∵峩螟ァ謇ケ驥擾シ?000遽?<3F>?
|
||||
- 螟憺龍莉サ蜉。蜿ッ髱<EFBDAF>霑占。<E58DA0>
|
||||
```
|
||||
|
||||
**ROI计算**:
|
||||
**ROI隶。邂<EFBFBD>**<EFBFBD>?
|
||||
```
|
||||
节省成本:¥731 - ¥430 = ¥301/次
|
||||
闃ら怐謌先悽<EFBFBD>堋?31 - ツ・430 = ツ・301/谺?
|
||||
螯よ棡豈乗怦10谺。謇ケ驥丈ササ蜉。<EFBFBD><EFBFBD>
|
||||
节省 = ¥301 × 10 = ¥3,010/月
|
||||
Redis成本 = ¥9/月
|
||||
净收益 = ¥3,001/月
|
||||
闃ら怐 = ツ・301 テ<EFBFBD> 10 = ツ・3,010/譛?
|
||||
Redis謌先悽 = ツ・9/譛?
|
||||
蜃謾カ逶<EFBFBD> = ツ・3,001/譛?
|
||||
|
||||
ROI = 33,344%(投入¥9,回报¥3,010)
|
||||
ROI = 33,344%<EFBFBD>域兜蜈・ツ?<3F>悟屓謚・ツ?,010<EFBFBD>?
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ⚠️ **结论与建议**
|
||||
## 笞<EFBFBD><EFBFBD><EFBFBD> **扈楢ョコ荳主サコ隶?*
|
||||
|
||||
### 譏守。ョ扈楢ョコ
|
||||
|
||||
```
|
||||
问题:MemoryQueue能否完成2小时任务?
|
||||
髣ョ鬚假シ哺emoryQueue閭ス蜷ヲ螳梧<EFBFBD>2蟆乗慮莉サ蜉。<EFBFBD>?
|
||||
遲疲。茨シ壺搆 荳崎<E88DB3>蜿ッ髱<EFBDAF>螳梧<E89EB3>
|
||||
|
||||
原因:
|
||||
蜴溷屏<EFBFBD>?
|
||||
1. SAE螳樔セ倶シ夊<EFBDBC>蜉ィ髞豈<C280>シ<EFBFBD>15蛻<35>帖譌<E5B896>豬<EFBFBD>㍼<EFBFBD><E38DBC>
|
||||
2. 2小时任务几乎必然遇到实例销毁
|
||||
3. 任务丢失后无法恢复
|
||||
4. 成功率 < 30%,夜间 < 5%
|
||||
2. 2蟆乗慮莉サ蜉。蜃<EFBFBD>荵主ソ<EFBFBD>┯驕<EFBFBD>芦螳樔セ矩楳豈?
|
||||
3. 莉サ蜉。荳「螟ア蜷取裏豕墓△螟?
|
||||
4. 謌仙粥邇?< 30%<25>悟、憺<EFBDA4>?< 5%
|
||||
```
|
||||
|
||||
### 蠑コ辜亥サコ隶ョ
|
||||
|
||||
```
|
||||
对于超过10分钟的任务,必须使用Redis队列!
|
||||
蟇ケ莠手カ<EFBFBD>ソ<EFBFBD>10蛻<EFBFBD>帖逧<EFBFBD>ササ蜉。<EFBFBD>悟ソ<EFBFBD>。サ菴ソ逕ィRedis髦溷<EFBFBD><EFBFBD>?
|
||||
|
||||
譌カ髣エ髦亥シ<EFBFBD><EFBFBD>
|
||||
- < 10遘抵シ壼庄莉・逕ィMemoryQueue<75>亥酔豁・螟<EFBDA5>炊<EFBFBD><E7828A>
|
||||
- 10秒 - 10分钟:建议用Redis队列
|
||||
- 10遘?- 10蛻<EFBFBD>帖<EFBFBD>壼サコ隶ョ逕ィRedis髦溷<EFBFBD>
|
||||
- > 10蛻<30>帖<EFBFBD>壼ソ<E5A3BC>。サ逕ィRedis髦溷<E9ABA6>
|
||||
- > 1蟆乗慮<E4B997>壼シコ蛻カ隕∵ア3edis髦溷<E9ABA6>
|
||||
```
|
||||
|
||||
### 实施优先级
|
||||
### 螳樊命莨伜<EFBFBD>郤?
|
||||
|
||||
```
|
||||
髦カ谿オ1<EFBFBD>域悽蜻ィ<EFBFBD>会シ啌edis郛灘ュ<EFBFBD>
|
||||
笏懌楳 隗」蜀ウLLM謌先悽髣ョ鬚<EFBDAE>
|
||||
└─ 工作量:2天
|
||||
笏披楳 蟾・菴憺㍼<E686BA><E38DBC>2螟?
|
||||
|
||||
阶段2(下周):Redis队列 ← **必须做!**
|
||||
├─ 解决长任务可靠性
|
||||
├─ 工作量:3天
|
||||
└─ 不做的风险:70%任务失败率
|
||||
髦カ谿オ2<EFBFBD>井ク句捉<EFBFBD>会シ啌edis髦溷<EFBFBD> 竊?**蠢<>。サ蛛夲シ<E5A4B2>**
|
||||
笏懌楳 隗」蜀ウ髟ソ莉サ蜉。蜿ッ髱<EFBDAF>諤?
|
||||
笏懌楳 蟾・菴憺㍼<E686BA><E38DBC>3螟?
|
||||
笏披楳 荳榊★逧<E29885>」朱勦<E69CB1><E58BA6>70%莉サ蜉。螟ア雍・邇?
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📝 **技术细节:为什么10分钟是分水岭?**
|
||||
## <EFBFBD>統 **謚譛ッ扈<EFBFBD>鰍<EFBFBD>壻クコ莉荵?0蛻<30>帖譏ッ蛻<EFBDAF>ーエ蟯ュ<E89FAF>?*
|
||||
|
||||
### SAE螳樔セ狗シゥ螳ケ遲也払
|
||||
|
||||
```
|
||||
阿里云SAE默认策略:
|
||||
髦ソ驥御コ全AE鮟倩ョ、遲也払<EFBFBD>?
|
||||
- 譽豬句捉譛滂シ<E6BB82>5蛻<35>帖
|
||||
- 譌<>豬<EFBFBD>㍼髦亥シ<C280><EFBDBC>15蛻<35>帖
|
||||
- 缩容延迟:5分钟
|
||||
- 郛ゥ螳ケ蟒カ霑滂シ?蛻<>帖
|
||||
|
||||
总计:15分钟后可能缩容
|
||||
諤サ隶。<EFBFBD>?5蛻<35>帖蜷主庄閭ス郛ゥ螳?
|
||||
```
|
||||
|
||||
### 任务时长与风险
|
||||
### 莉サ蜉。譌カ髟ソ荳朱」朱<EFBFBD>?
|
||||
|
||||
```
|
||||
任务时长 实例销毁风险 建议
|
||||
莉サ蜉。譌カ髟ソ 螳樔セ矩楳豈<E6A5B3>」朱<EFBDA3>? 蟒コ隶ョ
|
||||
笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏笏
|
||||
< 1分钟 几乎为0% 同步处理
|
||||
< 1蛻<EFBFBD>帖 蜃<EFBFBD>荵惹ク?% 蜷梧ュ・螟<EFBFBD>炊
|
||||
1-5蛻<35>帖 < 5% 蜿ッ逕ィMemoryQueue
|
||||
5-10蛻<30>帖 10-20% 蟒コ隶ョRedis髦溷<E9ABA6>
|
||||
10-30蛻<30>帖 50-70% 蠢<>。サRedis髦溷<E9ABA6>
|
||||
@@ -399,72 +399,73 @@ ROI = 33,344%(投入¥9,回报¥3,010)
|
||||
|
||||
## <20>識 **遶句叉陦悟勘**
|
||||
|
||||
### 如果您想现在就测试长任务:
|
||||
### 螯よ棡謔ィ諠ウ邇ー蝨ィ蟆ア豬玖ッ暮柄莉サ蜉。<EFBFBD>?
|
||||
|
||||
**不推荐**:用MemoryQueue测试1000篇
|
||||
- 风险:70%概率失败
|
||||
**荳肴耳闕?*<2A>夂畑MemoryQueue豬玖ッ<EFBFBD>1000遽?
|
||||
- 鬟朱勦<EFBFBD>?0%讎ら紫螟ア雍・
|
||||
- 豬ェ雍ケ<E99B8D>夐㍾螟崎ー<E5B48E>畑LLM API
|
||||
|
||||
**推荐**:先用100篇测试(10分钟)
|
||||
**謗ィ闕<EFBFBD>**<EFBFBD>壼<EFBFBD>逕?00遽<30>オ玖ッ包シ<E58C85>10蛻<30>帖<EFBFBD>?
|
||||
```typescript
|
||||
// 髯仙宛豬玖ッ墓焚驥<E7849A>
|
||||
const testLiteratures = literatures.slice(0, 100);
|
||||
processLiteraturesInBackground(task.id, projectId, testLiteratures);
|
||||
```
|
||||
|
||||
然后观察:
|
||||
辟カ蜷手ァょッ滂シ?
|
||||
- 譏ッ蜷ヲ驕<EFBDA6>芦螳樔セ矩楳豈<E6A5B3>シ<EFBFBD>
|
||||
- 任务是否完整?
|
||||
- 莉サ蜉。譏ッ蜷ヲ螳梧紛<EFBFBD>?
|
||||
- 螯よ棡螟ア雍・<E99B8D>檎ォ句叉謾ケ逕ィRedis髦溷<E9ABA6>
|
||||
|
||||
### 螯よ棡謔ィ蜃<EFBDA8>、<EFBFBD>隼騾<E99ABC><E9A8BE><EFBFBD>
|
||||
|
||||
**参考文档**:
|
||||
- `04-Redis改造实施计划.md`
|
||||
**蜿り<EFBFBD>枚譯?*<2A>?
|
||||
- `04-Redis謾ケ騾<EFBFBD>螳樊命隶。蛻?md`
|
||||
- `05-Redis郛灘ュ倅ク朱弌蛻礼噪蛹コ蛻ォ隸エ譏<EFBDB4>.md`
|
||||
|
||||
**改造顺序**:
|
||||
1. ✅ Redis缓存(本周)
|
||||
2. ✅ Redis队列(下周)← **重点**
|
||||
3. ✅ 测试2小时任务
|
||||
**謾ケ騾<EFBFBD>鬘コ蠎?*<2A>?
|
||||
1. 笨?Redis郛灘ュ假シ域悽蜻ィ<EFBFBD><EFBFBD>
|
||||
2. 笨?Redis髦溷<EFBFBD><EFBFBD>井ク句捉<EFBFBD>俄<EFBFBD>?**驥咲せ**
|
||||
3. 笨?豬玖ッ<E78E96>2蟆乗慮莉サ蜉。
|
||||
|
||||
---
|
||||
|
||||
## 📊 **附录:实际测试建议**
|
||||
## <EFBFBD>投 **髯<EFBFBD>ス包シ壼ョ樣刔豬玖ッ募サコ隶?*
|
||||
|
||||
### 测试方案A:验证MemoryQueue的不可靠性
|
||||
### 豬玖ッ墓婿譯<EFBFBD><EFBFBD>夐ェ瑚ッ`emoryQueue逧<EFBFBD>ク榊庄髱<EFBFBD>諤?
|
||||
|
||||
```bash
|
||||
# 步骤1:提交1000篇文献筛选任务
|
||||
# 步骤2:等待15分钟
|
||||
# 步骤3:检查任务状态
|
||||
# - 如果失败 → 证明实例被销毁
|
||||
# - 如果成功 → 运气好,不代表可靠
|
||||
# 豁・鬪、1<EFBFBD>壽署莠?000遽<30>枚迪ョ遲幃我ササ蜉?
|
||||
# 豁・鬪、2<EFBFBD>夂ュ牙セ?5蛻<35>帖
|
||||
# 豁・鬪、3<EFBFBD>壽」譟・莉サ蜉。迥カ諤?
|
||||
# - 螯よ棡螟ア雍・ 竊?隸∵<E99AB8>螳樔セ玖「ォ髞豈?
|
||||
# - 螯よ棡謌仙粥 竊?霑先ー泌・ス<EFBDA5>御ク堺サ」陦ィ蜿ッ髱?
|
||||
|
||||
# 驥榊、肴オ玖ッ<E78E96>5谺。<E8B0BA><EFBDA1>
|
||||
# - 成功率应该 < 30%
|
||||
# - 謌仙粥邇<EFBFBD>コ碑ッ?< 30%
|
||||
```
|
||||
|
||||
### 豬玖ッ墓婿譯<E5A9BF><E8ADAF>啌edis髦溷<E9ABA6>鬪瑚ッ<E7919A>
|
||||
|
||||
```bash
|
||||
# 豁・鬪、1<EFBDA4>夐Κ鄂イRedis髦溷<E9ABA6>迚域悽
|
||||
# 步骤2:提交1000篇文献筛选任务
|
||||
# 豁・鬪、2<EFBFBD>壽署莠?000遽<30>枚迪ョ遲幃我ササ蜉?
|
||||
# 豁・鬪、3<EFBDA4>壻クサ蜉ィ蛛懈ュ「SAE螳樔セ<E6A894>
|
||||
# 步骤4:重新启动实例
|
||||
# 步骤5:检查任务是否自动恢复
|
||||
# 豁・鬪、4<EFBFBD>夐㍾譁ー蜷ッ蜉ィ螳樔セ?
|
||||
# 豁・鬪、5<EFBFBD>壽」譟・莉サ蜉。譏ッ蜷ヲ閾ェ蜉ィ諱「螟?
|
||||
|
||||
# 预期结果:
|
||||
# - 任务自动恢复 ✅
|
||||
# - 从断点继续 ✅
|
||||
# - 最终完成 ✅
|
||||
# 鬚<EFBFBD>悄扈捺棡<EFBFBD>?
|
||||
# - 莉サ蜉。閾ェ蜉ィ諱「螟<EFBFBD> 笨?
|
||||
# - 莉取妙轤ケ扈ァ扈?笨?
|
||||
# - 譛扈亥ョ梧<EFBFBD>?笨?
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
**文档维护者:** 技术团队
|
||||
**譁<EFBFBD>。」扈エ謚、閠<EFBFBD>シ<EFBFBD>** 謚譛ッ蝗「髦?
|
||||
**譛蜷取峩譁ー<E8AD81><EFBDB0>** 2025-12-12
|
||||
**关键结论:** MemoryQueue无法可靠完成2小时任务,必须迁移到Redis队列
|
||||
**蜈ウ髞ョ扈楢ョコ<EFBFBD>?* MemoryQueue譌<EFBFBD>豕募庄髱<EFBFBD>螳梧<EFBFBD>2蟆乗慮莉サ蜉。<EFBFBD>悟ソ<EFBFBD>。サ霑∫ァサ蛻ーRedis髦溷<EFBFBD>
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,115 +1,96 @@
|
||||
# **Postgres-Only <20>刻<EFBFBD><E588BB>嗆<EFBFBD>閫<EFBFBD><E996AB><EFBFBD>寞<EFBFBD>**
|
||||
|
||||
## **—— 面向微型 AI 团队的高可靠、低成本技术战略**
|
||||
## **<EFBFBD>婙<EFBFBD>?<3F>W<EFBFBD>敺桀<E695BA> AI <20>a<EFBFBD><EFBD81><EFBFBD><EFBFBD><EFBFBD>舫<EFBFBD><E888AB><EFBFBD><EFBFBD><EFBFBD>鞉𧋦<E99E89><F0A78BA6><EFBFBD>舀<EFBFBD><E88880>?*
|
||||
|
||||
<EFBFBD><EFBFBD>𧋦嚗鯝1.0
|
||||
适用场景:1-2人初创团队、Node.js/Fastify 技术栈、阿里云 SAE 部署环境
|
||||
核心目标:在不引入 Redis 的前提下,实现企业级的任务队列、缓存与会话管理,保障 2小时+ 长任务的绝对可靠性。
|
||||
|
||||
<EFBFBD><EFBFBD>鍂<EFBFBD>箸艶嚗?-2鈭箏<E988AD><E7AE8F>𥕦𣪧<F0A595A6>麄<EFBFBD><E9BA84>ode.js/Fastify <20><><EFBFBD>舀<EFBFBD><E88880><EFBFBD>燵<EFBFBD>䔶<EFBFBD> SAE <20>函蔡<E587BD>臬<EFBFBD>
|
||||
<EFBFBD>詨<EFBFBD><EFBFBD>格<EFBFBD>嚗𡁜銁銝滚<EFBFBD><EFBFBD>?Redis <20><><EFBFBD><EFBFBD>𣂷<EFBFBD>嚗<EFBFBD><E59A97><EFBFBD>唬<EFBFBD>銝𡁶漣<F0A181B6><E6BCA3>遙<EFBFBD>⊿<EFBFBD><E28ABF>𨰜<EFBFBD><F0A8B09C><EFBFBD>摮䀝<E691AE>隡朞<E99AA1>蝞∠<E89D9E>嚗䔶<E59A97><E494B6>?2撠𤩺𧒄+ <20>蹂遙<E8B982>∠<EFBFBD>蝏嘥笆<E598A5>舫<EFBFBD><E888AB>扼<EFBFBD>?
|
||||
## **1\. <20>扯<EFBFBD><E689AF>䁅<EFBFBD> (Executive Summary)**
|
||||
|
||||
针对我方当前(MAU \< 5000)的业务规模与“稳定性优先”的战略诉求,本方案主张采用 **"Postgres-Only" (全能数据库)** 架构。
|
||||
<EFBFBD><EFBFBD>笆<EFBFBD>烐䲮敶枏<EFBFBD>嚗㇈AU \< 5000嚗厩<E59A97>銝𡁜𦛚閫<F0A69B9A>芋銝𢛶<E98A9D>𦦵迅摰𡁏<E691B0>找<EFBFBD><E689BE><EFBFBD><EFBFBD>萘<EFBFBD><E89098>条裦霂㗇<E99C82>嚗峕𧋦<E5B395>寞<EFBFBD>銝餃<E98A9D><E9A483><EFBFBD>鍂 **"Postgres-Only" (<EFBFBD>刻<EFBFBD><EFBFBD>唳旿摨?** <20>嗆<EFBFBD><E59786>?
|
||||
<EFBFBD>朞<EFBFBD><EFBFBD>拍鍂 PostgreSQL <20><><EFBFBD>蝥抒鸌<E68A92>改<EFBFBD>憒?SKIP LOCKED <20><>㦤<EFBFBD>嗚<EFBFBD><E5979A>SONB 摮睃<E691AE><E79D83><EFBFBD>nlogged Tables嚗㚁<E59A97><E39A81>睲賑<E79DB2>臭誑摰<E8AA91><E691B0><EFBFBD>蹂誨 Redis <20>?*隞餃𦛚<E9A483>笔<EFBFBD>**<2A>?*蝻枏<E89DBB>**<2A>?*隡朞<E99AA1>摮睃<E691AE>**銝剔<E98A9D>雿𦦵鍂<F0A6A6B5>?
|
||||
**<EFBFBD>条裦<EFBFBD>嗥<EFBFBD>嚗?*
|
||||
|
||||
通过利用 PostgreSQL 的高级特性(如 SKIP LOCKED 锁机制、JSONB 存储、Unlogged Tables),我们可以完全替代 Redis 在**任务队列**、**缓存**、**会话存储**中的作用。
|
||||
1. **<EFBFBD>嗆<EFBFBD><EFBFBD><EFBFBD><EFBFBD>**嚗𡁶宏<F0A181B6>?Redis 銝剝𡢿隞塚<E99A9E>蝟餌<E89D9F>憭齿<E686AD>摨阡<E691A8>雿?50%<25>?
|
||||
2. **<EFBFBD>唳旿撘箔<EFBFBD><EFBFBD>?*嚗帋<E59A97><E5B88B>⊥㺭<E28AA5>桐<EFBFBD>隞餃𦛚<E9A483>嗆<EFBFBD><E59786>銁<EFBFBD>䔶<EFBFBD>鈭见𦛚銝剜<E98A9D>鈭歹<E988AD>敶餃<E695B6><E9A483>寥膄<E5AFA5>𨅯<EFBFBD>撣<EFBFBD><E692A3>鈭见𦛚<E8A781>嗪<EFBFBD><E597AA>押<EFBFBD>?
|
||||
3. **<EFBFBD>園<EFBFBD>憭𡝗<EFBFBD><EFBFBD>?*嚗𡁜<E59A97><F0A1819C>函緵<E587BD>?RDS 韏<><E99F8F>嚗峕<E59A97>撟渲<E6929F><E6B8B2><EFBFBD>㺭<EFBFBD><E3BAAD><EFBFBD>銝剝𡢿隞嗉晶<E59789>具<EFBFBD>?
|
||||
4. **隡<><E99AA1>蝥批虾<E689B9>?*嚗帋<E59A97><E5B88B>?RDS <20><>䌊<EFBFBD>典<EFBFBD>隞賭<E99A9E> PITR嚗<52>𧒄<EFBFBD>渡<EFBFBD><E6B8A1>W<EFBFBD>嚗㕑<E59A97><E39591>𨥈<EFBFBD>靽嗪<E99DBD>隞餃𦛚<E9A483>笔<EFBFBD><E7AC94>唳旿<E594B3>𨀣偶銝滢腺憭晦<E686AD>腈<EFBFBD>?
|
||||
## **2\. <20>桅<EFBFBD><E6A185>峕艶銝擧<E98A9D><E693A7>?*
|
||||
|
||||
**战略收益:**
|
||||
### **2.1 敶枏<E695B6><E69E8F>𤤿<EFBFBD>嚗𡁻鵭隞餃𦛚<E9A483><F0A69B9A><EFBFBD>撘望<E69298>?*
|
||||
|
||||
1. **架构极简**:移除 Redis 中间件,系统复杂度降低 50%。
|
||||
2. **数据强一致**:业务数据与任务状态在同一事务中提交,彻底根除“分布式事务”风险。
|
||||
3. **零额外成本**:复用现有 RDS 资源,每年节省数千元中间件费用。
|
||||
4. **企业级可靠**:依托 RDS 的自动备份与 PITR(时间点恢复)能力,保障任务队列数据“永不丢失”。
|
||||
|
||||
## **2\. 问题背景与挑战**
|
||||
|
||||
### **2.1 当前痛点:长任务的脆弱性**
|
||||
|
||||
我们的业务涉及“文献全库解析”和“双模型交叉验证”,单次任务耗时可能长达 **2小时**。
|
||||
|
||||
* **现状**:使用内存队列(MemoryQueue)。
|
||||
* **风险**:在 Serverless (SAE) 环境下,实例可能因无流量缩容、发布更新或内存溢出而随时销毁。一旦销毁,内存中的任务进度即刻丢失,导致用户任务失败。
|
||||
|
||||
### **2.2 常见误区:只有 Redis 能救命?**
|
||||
|
||||
业界常见的观点认为:“必须引入 Redis (BullMQ) 才能实现任务持久化。”
|
||||
|
||||
* **反驳**:这是惯性思维。任务持久化的核心是\*\*“持久化存储”\*\*,而非 Redis 本身。PostgreSQL 同样具备持久化能力,且在事务安全性上优于 Redis。
|
||||
<EFBFBD>睲賑<EFBFBD><EFBFBD><EFBFBD><EFBFBD>⊥<EFBFBD><EFBFBD>𪙛<EFBFBD>𨀣<EFBFBD><EFBFBD>桀<EFBFBD>摨栞圾<EFBFBD>鐥<EFBFBD>嘥<EFBFBD><EFBFBD>𨅯<EFBFBD>璅∪<EFBFBD>鈭文<EFBFBD>撉諹<EFBFBD><EFBFBD>嘅<EFBFBD><EFBFBD>閙活隞餃𦛚<EFBFBD>埈𧒄<EFBFBD>航<EFBFBD><EFBFBD>輯噢 **2撠𤩺𧒄**<EFBFBD>?
|
||||
* **<2A>啁𠶖**嚗帋蝙<E5B88B>典<EFBFBD>摮㗛<E691AE><E3979B>梹<EFBFBD>MemoryQueue嚗剹<E59A97>?
|
||||
* **憌𡡞埯**嚗𡁜銁 Serverless (SAE) <20>臬<EFBFBD>銝页<E98A9D>摰硺<E691B0><E7A1BA>航<EFBFBD><E888AA>䭾<EFBFBD>瘚<EFBFBD><E7989A>蝻拙捆<E68B99><E68D86><EFBFBD>撣<EFBFBD>凒<EFBFBD>唳<EFBFBD><E594B3><EFBFBD><EFBFBD>皞W枂<EFBCB7>屸<EFBFBD><E5B1B8>園<EFBFBD>瘥<EFBFBD><E798A5><EFBFBD><EFBFBD><EFBFBD>阡<EFBFBD>瘥<EFBFBD><E798A5><EFBFBD><EFBFBD><EFBFBD>銝剔<E98A9D>隞餃𦛚餈𥕦漲<F0A595A6>喳<EFBFBD>銝W仃嚗<E4BB83>紡<EFBFBD>渡鍂<E6B8A1>瑚遙<E7919A>∪仃韐乓<E99F90>?
|
||||
### **2.2 撣貉<E692A3>霂臬躹嚗𡁜蘨<F0A1819C>?Redis <20>賣<EFBFBD><E8B3A3>踝<EFBFBD>**
|
||||
|
||||
銝𡁶<EFBFBD>撣貉<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>寡恕銝綽<EFBFBD><EFBFBD>𨅯<EFBFBD>憿餃<EFBFBD><EFBFBD>?Redis (BullMQ) <20>滩<EFBFBD>摰䂿緵隞餃𦛚<E9A483><F0A69B9A><EFBFBD><EFBFBD>硔<EFBFBD><E7A194><EFBFBD>?
|
||||
* **<2A>漤底**嚗朞<E59A97><E69C9E>舀<EFBFBD><E88880>扳<EFBFBD>萘輕<E89098><E8BC95>遙<EFBFBD>⊥<EFBFBD>銋<EFBFBD><E98A8B><EFBFBD><EFBFBD>瓲敹<E793B2>糓\*\*<2A>𨀣<EFBFBD>銋<EFBFBD><E98A8B>摮睃<E691AE><E79D83>𩄼*\*嚗諹<E59A97>屸<EFBFBD> Redis <20>祈澈<E7A588><E6BE88>ostgreSQL <20>峕甅<E5B395>瑕<EFBFBD><E79195><EFBFBD><EFBFBD><EFBFBD>𤥁<EFBFBD><F0A4A581>𨥈<EFBFBD>銝𥪜銁鈭见𦛚摰匧<E691B0><E58CA7>找<EFBFBD>隡䀝<E99AA1> Redis<69>?
|
||||
## **3\. <20>詨<EFBFBD>閫<EFBFBD><E996AB><EFBFBD>寞<EFBFBD>嚗䥪ostgres-Only <20>嗆<EFBFBD>**
|
||||
|
||||
本方案将 Redis 的三大核心功能(队列、缓存、会话)全部收敛至 PostgreSQL。
|
||||
|
||||
### **3.1 替代 Redis 队列:使用 pg-boss**
|
||||
|
||||
我们引入 Node.js 库 **pg-boss**,它利用 PostgreSQL 的 FOR UPDATE SKIP LOCKED 特性,实现了高性能的抢占式队列。
|
||||
<EFBFBD>祆䲮獢<EFBFBD><EFBFBD> Redis <EFBFBD><EFBFBD><EFBFBD>憭扳瓲敹<EFBFBD><EFBFBD><EFBFBD>踝<EFBFBD><EFBFBD>笔<EFBFBD><EFBFBD><EFBFBD><EFBFBD>摮塩<EFBFBD><EFBFBD><EFBFBD>霂嘅<EFBFBD><EFBFBD>券<EFBFBD><EFBFBD>嗆<EFBFBD><EFBFBD>?PostgreSQL<EFBFBD>?
|
||||
### **3.1 <20>蹂誨 Redis <20>笔<EFBFBD>嚗帋蝙<E5B88B>?pg-boss**
|
||||
|
||||
<EFBFBD>睲賑撘訫<EFBFBD> Node.js 摨?**pg-boss**嚗<><E59A97><EFBFBD>拍鍂 PostgreSQL <20>?FOR UPDATE SKIP LOCKED <20>寞<EFBFBD>改<EFBFBD>摰䂿緵鈭<E7B7B5><E988AD><EFBFBD>扯<EFBFBD><E689AF><EFBFBD>𦜖<EFBFBD>惩<EFBFBD><E683A9>笔<EFBFBD><E7AC94>?
|
||||
#### **<2A>嗆<EFBFBD><E59786>餉<EFBFBD>**
|
||||
|
||||
1. **入队**:API 接收请求,将任务元数据(JSON)写入 job 表。**此操作在毫秒级完成,数据立即安全落盘。**
|
||||
2. **处理**:Worker 进程从数据库捞取任务,并锁定该行记录。
|
||||
3. **容灾**:如果 SAE 实例在处理过程中崩溃(如 OOM),数据库锁会在超时后自动释放,其他存活的 Worker 实例会立即接管该任务重试。
|
||||
|
||||
1. **<EFBFBD>仿<EFBFBD>**嚗鋫PI <20>交𤣰霂瑟<E99C82>嚗<EFBFBD><E59A97>隞餃𦛚<E9A483><F0A69B9A>㺭<EFBFBD>殷<EFBFBD>JSON嚗匧<E59A97><E58CA7>?job 銵具<E98AB5>?*甇斗<E79487>雿𨅯銁瘥怎<E798A5>蝥批<E89DA5><E689B9>琜<EFBFBD><E7909C>唳旿蝡见朖摰匧<E691B0><E58CA7>賜<EFBFBD><E8B39C>?*
|
||||
2. **憭<EFBFBD><EFBFBD>**嚗阳orker 餈𤤿<E9A488>隞擧㺭<E693A7>桀<EFBFBD><E6A180>𧼮<EFBFBD>隞餃𦛚嚗<F0A69B9A>僎<EFBFBD><E5838E><EFBFBD>霂亥<E99C82>霈啣<E99C88><E595A3>?
|
||||
3. **摰寧<EFBFBD>**嚗𡁜<E59A97><F0A1819C>?SAE 摰硺<E691B0><E7A1BA>典<EFBFBD><E585B8><EFBFBD><EFBFBD>蝔衤葉撏拇<E6928F>嚗<EFBFBD><E59A97> OOM嚗㚁<E59A97><E39A81>唳旿摨㯄<E691A8>隡𡁜銁頞<E98A81>𧒄<EFBFBD>舘䌊<E88898>券<EFBFBD><E588B8>橘<EFBFBD><E6A998>嗡<EFBFBD>摮䀹暑<E480B9>?Worker 摰硺<E691B0>隡𡁶<E99AA1><F0A181B6>單𦻖蝞∟砲隞餃𦛚<E9A483>滩<EFBFBD><E6BBA9>?
|
||||
#### **隞<><E99A9E>摰䂿緵<E482BF><E7B7B5><EFBFBD>**
|
||||
|
||||
import PgBoss from 'pg-boss';
|
||||
|
||||
const boss \= new PgBoss({
|
||||
connectionString: process.env.DATABASE\_URL,
|
||||
schema: 'job\_queue', // 独立Schema,不污染业务表
|
||||
max: 5 // 并发控制,保护 DeepSeek API
|
||||
schema: 'job\_queue', // <EFBFBD>祉<EFBFBD>Schema嚗䔶<EFBFBD>瘙⊥<EFBFBD>銝𡁜𦛚銵?
|
||||
max: 5 // 撟嗅<EFBFBD><EFBFBD>批<EFBFBD>嚗䔶<EFBFBD><EFBFBD>?DeepSeek API
|
||||
});
|
||||
|
||||
await boss.start();
|
||||
|
||||
// 消费者定义 (Worker)
|
||||
// 瘨<EFBFBD>晶<EFBFBD><EFBFBD><EFBFBD>銋?(Worker)
|
||||
await boss.work('screening-task', {
|
||||
// 关键配置:设置锁的有效期为 4小时
|
||||
// 即使任务跑 3.9小时,只要 Worker 活着,就不会被抢走
|
||||
// 如果 Worker 死了,锁过期,任务自动重试
|
||||
// <EFBFBD>喲睸<EFBFBD>滨蔭嚗朞挽蝵桅<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>銝?4撠𤩺𧒄
|
||||
// <EFBFBD>喃蝙隞餃𦛚頝?3.9撠𤩺𧒄嚗<EFBFBD>蘨閬?Worker 瘣餌<E798A3>嚗<EFBFBD>停銝滢<E98A9D>鋡急𦜖韏?
|
||||
// 憒<EFBFBD><EFBFBD> Worker 甇颱<EFBFBD>嚗屸<EFBFBD>餈<EFBFBD><EFBFBD>嚗䔶遙<EFBFBD>∟䌊<EFBFBD>券<EFBFBD>霂?
|
||||
expireInSeconds: 14400,
|
||||
retryLimit: 3
|
||||
}, async (job) \=\> {
|
||||
// 銝𡁜𦛚<F0A1819C>餉<EFBFBD>...
|
||||
});
|
||||
|
||||
### **3.2 替代 Redis 缓存:基于 Table 的 KV 存储**
|
||||
|
||||
对于 AI 结果缓存(避免重复调用 LLM),Postgres 的查询速度(1-3ms)对于用户体验(秒级等待)来说完全可以接受。
|
||||
### **3.2 <EFBFBD>蹂誨 Redis 蝻枏<EFBFBD>嚗𡁜抅鈭?Table <EFBFBD>?KV 摮睃<EFBFBD>**
|
||||
|
||||
撖嫣<EFBFBD> AI 蝏𤘪<E89D8F>蝻枏<E89DBB>嚗<EFBFBD><E59A97><EFBFBD>漤<EFBFBD>憭滩<E686AD><E6BBA9>?LLM嚗㚁<E59A97>Postgres <20><>䰻霂a<E99C82>笔漲嚗?-3ms嚗匧笆鈭𡒊鍂<F0A1928A>瑚<EFBFBD>撉䕘<E69289>蝘垍漣蝑匧<E89D91>嚗㗇䔉霂游<E99C82><E6B8B8>典虾隞交𦻖<E4BAA4>𨰜<EFBFBD>?
|
||||
#### **<2A>扯<EFBFBD>霈箄<E99C88>**
|
||||
|
||||
```
|
||||
实际并发分析:
|
||||
- 当前规模: 500 MAU
|
||||
- 峰值并发: < 50 QPS(极端情况)
|
||||
- Postgres能力: 5万+ QPS(简单查询)
|
||||
- 性能余量: 1000倍
|
||||
|
||||
响应时间对比:
|
||||
- Redis: 0.15ms(网络+读取)
|
||||
- Postgres: 1.5ms(网络+查询)
|
||||
- 差异: 1.35ms
|
||||
- 用户感知: 无(总耗时200ms中占比 < 1%)
|
||||
|
||||
摰鮋<EFBFBD>撟嗅<EFBFBD><EFBFBD><EFBFBD><EFBFBD>嚗?- 敶枏<E695B6>閫<EFBFBD>芋: 500 MAU
|
||||
- 撜啣<EFBFBD>澆僎<EFBFBD>? < 50 QPS嚗<EFBFBD><EFBFBD>蝡舀<EFBFBD><EFBFBD>蛛<EFBFBD>
|
||||
- Postgres<EFBFBD>賢<EFBFBD>: 5銝? QPS嚗<53><E59A97><EFBFBD>閙䰻霂g<E99C82>
|
||||
- <EFBFBD>扯<EFBFBD>雿䠷<EFBFBD>: 1000<30>?
|
||||
<EFBFBD>滚<EFBFBD><EFBFBD>園𡢿撖寞<EFBFBD>嚗?- Redis: 0.15ms嚗<EFBFBD><EFBFBD>蝏?霂餃<E99C82>嚗?- Postgres: 1.5ms嚗<EFBFBD><EFBFBD>蝏?<3F>亥砭嚗?- 撌桀<E6928C>: 1.35ms
|
||||
- <20>冽<EFBFBD><E586BD>毺䰻: <20>𩤃<EFBFBD><F0A9A483>餉<EFBFBD>埈𧒄200ms銝剖<E98A9D>瘥?< 1%嚗?
|
||||
蝏栞捏嚗𡁜銁<EFBFBD>交暑10銝<EFBFBD>誑銝页<EFBFBD>Postgres<EFBFBD>扯<EFBFBD>摰<EFBFBD><EFBFBD>憭毺鍂
|
||||
```
|
||||
|
||||
#### **数据库设计**
|
||||
#### **<EFBFBD>唳旿摨栞挽霈?*
|
||||
|
||||
```prisma
|
||||
model AppCache {
|
||||
id Int @id @default(autoincrement())
|
||||
key String @unique
|
||||
value Json // 对应 Redis 的 Value
|
||||
expiresAt DateTime // 对应 Redis 的 TTL
|
||||
value Json // 撖孵<EFBFBD> Redis <EFBFBD>?Value
|
||||
expiresAt DateTime // 撖孵<EFBFBD> Redis <EFBFBD>?TTL
|
||||
createdAt DateTime @default(now())
|
||||
|
||||
@@index([expiresAt]) // 索引用于快速清理过期数据
|
||||
@@index([key, expiresAt]) // 复合索引优化查询
|
||||
@@index([expiresAt]) // 蝝W<EFBFBD><EFBFBD>其<EFBFBD>敹恍<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>㺭<EFBFBD>? @@index([key, expiresAt]) // 憭滚<E686AD>蝝W<E89D9D>隡睃<E99AA1><E79D83>亥砭
|
||||
@@map("app_cache")
|
||||
}
|
||||
```
|
||||
|
||||
#### **封装 Service(完整版)**
|
||||
#### **撠<EFBFBD><EFBFBD> Service嚗<EFBFBD><EFBFBD><EFBFBD>渡<EFBFBD>嚗?*
|
||||
|
||||
```typescript
|
||||
// <20><>辣嚗颹ackend/src/common/cache/PostgresCacheAdapter.ts
|
||||
@@ -121,8 +102,7 @@ import { logger } from '../logging/index';
|
||||
export class PostgresCacheAdapter implements CacheAdapter {
|
||||
|
||||
/**
|
||||
* 获取缓存(带懒惰删除)
|
||||
*/
|
||||
* <EFBFBD>瑕<EFBFBD>蝻枏<EFBFBD>嚗<EFBFBD>蒂<EFBFBD>埝<EFBFBD><EFBFBD>𣳇膄嚗? */
|
||||
async get<T = any>(key: string): Promise<T | null> {
|
||||
try {
|
||||
const record = await prisma.appCache.findUnique({
|
||||
@@ -131,10 +111,8 @@ export class PostgresCacheAdapter implements CacheAdapter {
|
||||
|
||||
if (!record) return null;
|
||||
|
||||
// 检查是否过期
|
||||
if (record.expiresAt < new Date()) {
|
||||
// 懒惰删除:顺手清理(异步,不阻塞)
|
||||
this.deleteAsync(key);
|
||||
// 璉<EFBFBD><EFBFBD>交糓<EFBFBD>西<EFBFBD><EFBFBD>? if (record.expiresAt < new Date()) {
|
||||
// <20>埝<EFBFBD><E59F9D>𣳇膄嚗𡁻◇<F0A181BB>𧢲<EFBFBD><F0A7A2B2><EFBFBD><EFBFBD>撘<EFBFBD>郊嚗䔶<E59A97><E494B6>餃<EFBFBD>嚗? this.deleteAsync(key);
|
||||
return null;
|
||||
}
|
||||
|
||||
@@ -203,12 +181,11 @@ export class PostgresCacheAdapter implements CacheAdapter {
|
||||
}
|
||||
|
||||
/**
|
||||
* 清空所有缓存
|
||||
*/
|
||||
* 皜<EFBFBD>征<EFBFBD><EFBFBD><EFBFBD>厩<EFBFBD>摮? */
|
||||
async flush(): Promise<void> {
|
||||
try {
|
||||
await prisma.appCache.deleteMany({});
|
||||
logger.info('[PostgresCache] 缓存已清空');
|
||||
logger.info('[PostgresCache] 蝻枏<EFBFBD>撌脫<EFBFBD>蝛?);
|
||||
} catch (error) {
|
||||
logger.error('[PostgresCache] 皜<EFBFBD>征憭梯揖', { error });
|
||||
}
|
||||
@@ -216,13 +193,11 @@ export class PostgresCacheAdapter implements CacheAdapter {
|
||||
}
|
||||
|
||||
/**
|
||||
* 启动定时清理任务(分批清理,防止阻塞)
|
||||
*/
|
||||
* <EFBFBD>臬𢆡摰𡁏𧒄皜<EFBFBD><EFBFBD>隞餃𦛚嚗<EFBFBD><EFBFBD><EFBFBD>寞<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>脫迫<EFBFBD>餃<EFBFBD>嚗? */
|
||||
export function startCacheCleanupTask() {
|
||||
setInterval(async () => {
|
||||
try {
|
||||
// 每次只删除1000条过期数据
|
||||
const result = await prisma.$executeRaw`
|
||||
// 瘥𤩺活<EFBFBD>芸<EFBFBD><EFBFBD>?000<30>∟<EFBFBD><E2889F><EFBFBD>㺭<EFBFBD>? const result = await prisma.$executeRaw`
|
||||
DELETE FROM app_cache
|
||||
WHERE id IN (
|
||||
SELECT id FROM app_cache
|
||||
@@ -238,96 +213,80 @@ export function startCacheCleanupTask() {
|
||||
} catch (error) {
|
||||
logger.error('[PostgresCache] 摰𡁏𧒄皜<EFBFBD><EFBFBD>憭梯揖', { error });
|
||||
}
|
||||
}, 60000); // 每分钟执行一次
|
||||
|
||||
logger.info('[PostgresCache] 定时清理任务已启动(每分钟1000条)');
|
||||
}, 60000); // 瘥誩<EFBFBD><EFBFBD><EFBFBD><EFBFBD>銵䔶<EFBFBD>甈?
|
||||
logger.info('[PostgresCache] 摰𡁏𧒄皜<EFBFBD><EFBFBD>隞餃𦛚撌脣鍳<EFBFBD>剁<EFBFBD>瘥誩<EFBFBD><EFBFBD>?000<EFBFBD>∴<EFBFBD>');
|
||||
}
|
||||
```
|
||||
|
||||
#### **性能优化技巧**
|
||||
#### **<EFBFBD>扯<EFBFBD>隡睃<EFBFBD><EFBFBD><EFBFBD>撌?*
|
||||
|
||||
1. **蝝W<E89D9D>隡睃<E99AA1>**嚗䫤@@index([key, expiresAt])` 閬<><E996AC><EFBFBD>亥砭嚗峕<E59A97><E5B395><EFBFBD><EFBFBD>噼”
|
||||
2. **懒惰删除**:读取时顺便清理,分散负载
|
||||
3. **分批清理**:每次LIMIT 1000,毫秒级完成
|
||||
4. **连接池复用**:Prisma自动管理,无额外开销
|
||||
2. **<EFBFBD>埝<EFBFBD><EFBFBD>𣳇膄**嚗朞粉<E69C9E>𡝗𧒄憿箔噶皜<E599B6><E79A9C>嚗<EFBFBD><E59A97><EFBFBD><EFBFBD><EFBFBD>頧?3. **<EFBFBD><EFBFBD>鸌皜<EFBFBD><EFBFBD>**嚗𡁏<E59A97>甈‥IMIT 1000嚗峕神蝘垍漣摰峕<E691B0>
|
||||
4. **餈墧𦻖瘙惩<EFBFBD><EFBFBD>?*嚗䥪risma<6D>芸𢆡蝞∠<E89D9E>嚗峕<E59A97>憸嘥<E686B8>撘<EFBFBD><E69298><EFBFBD>
|
||||
|
||||
### **3.3 <20>蹂誨 Redis 隡朞<E99AA1>嚗䬙onnect-pg-simple**
|
||||
|
||||
使用成熟的社区方案,将 Session 存储在 Postgres 的 session 表中。SAE 多实例重启后,用户无需重新登录。
|
||||
|
||||
## **4\. 深度对比:为什么 Postgres 胜出?**
|
||||
雿輻鍂<EFBFBD>鞟<EFBFBD><EFBFBD><EFBFBD>冗<EFBFBD>箸䲮獢<EFBFBD><EFBFBD>撠?Session 摮睃<EFBFBD><EFBFBD>?Postgres <EFBFBD>?session 銵其葉<EFBFBD><EFBFBD>AE 憭𡁜<E686AD>靘钅<E99D98><E99285>臬<EFBFBD>嚗𣬚鍂<F0A3AC9A>瑟<EFBFBD><E7919F><EFBFBD><EFBFBD>齿鰵<E9BDBF>餃<EFBFBD><E9A483>?
|
||||
## **4\. 瘛勗漲撖寞<E69296>嚗帋蛹隞<E89BB9>銋?Postgres <20>𨅯枂嚗?*
|
||||
|
||||
| 蝏游漲 | <20>寞<EFBFBD> A: 隡删<E99AA1> Redis (BullMQ) | <20>寞<EFBFBD> B: Postgres (pg-boss) | <20>瑁<EFBFBD><E79181>笔<EFBFBD> |
|
||||
| :---- | :---- | :---- | :---- |
|
||||
| **数据一致性** | **弱** (双写一致性难题) 任务在 Redis,业务在 DB。若 DB 事务回滚,Redis 任务可能无法回滚。 | **强** (事务级原子性) 任务入队与业务数据写入在同一个 DB 事务中。要么全成,要么全败。 | **Postgres** |
|
||||
| **运维复杂度** | **高** 需维护 Redis 实例、VPC 白名单、监控内存碎片率、持久化策略。 | **零** 复用现有 RDS。备份、监控、扩容全由阿里云 RDS 托管。 | **Postgres** |
|
||||
| **备份与恢复** | **困难** Redis RDB/AOF 恢复可能会丢失最后几秒的数据。 | **完美** RDS 支持 PITR (按时间点恢复)。误删任务可精确回滚到 1秒前。 | **Postgres** |
|
||||
| **成本** | **¥1000+/年** (Tair 基础版) | **¥0** (资源复用) | **Postgres** |
|
||||
| **性能 (TPS)** | **极高** (10w+) | **高** (5000+) 对于日均几万次 AI 调用的场景,Postgres 性能绰绰有余。 | **Postgres** |
|
||||
| **锁竞争问题** | **误解** 读写都需要网络往返,高并发下Redis也会有竞争 | **真相** SELECT是快照读,不加锁。Node.js单线程会先成为瓶颈。 | **误解澄清** |
|
||||
| **缓存清理风险** | **存在** 内存溢出需要配置eviction策略,不当配置会导致数据丢失 | **可控** 分批删除(LIMIT 1000)+ 懒惰删除,永远不会阻塞。 | **Postgres** |
|
||||
| **学习曲线** | **陡峭** 需要学习Redis、BullMQ、ioredis、持久化策略、内存管理 | **平缓** 只需学习pg-boss(API类似BullMQ),其余都是熟悉的Postgres | **Postgres** |
|
||||
| **<EFBFBD>唳旿銝<EFBFBD><EFBFBD>湔<EFBFBD>?* | **撘?* (<28><><EFBFBD>銝<EFBFBD><E98A9D>湔<EFBFBD>折𠗕憸? 隞餃𦛚<E9A483>?Redis嚗䔶<E59A97><E494B6>∪銁 DB<44><42>𥅾 DB 鈭见𦛚<E8A781>墧<EFBFBD>嚗朙edis 隞餃𦛚<E9A483>航<EFBFBD><E888AA>䭾<EFBFBD><E4ADBE>墧<EFBFBD><E5A2A7>?| **撘?* (鈭见𦛚蝥批<E89DA5>摮鞉<E691AE>? 隞餃𦛚<E9A483>仿<EFBFBD>銝𦒘<E98A9D><F0A69298>⊥㺭<E28AA5>桀<EFBFBD><E6A180>亙銁<E4BA99>䔶<EFBFBD>銝?DB 鈭见𦛚銝准<E98A9D><E58786><EFBFBD>銋<EFBFBD><E98A8B><EFBFBD>琜<EFBFBD>閬<EFBFBD><E996AC><EFBFBD>刻揖<E588BB>?| **Postgres** |
|
||||
| **餈鞟輕憭齿<EFBFBD>摨?* | **擃?* <20><>蝏湔擪 Redis 摰硺<E691B0><E7A1BA><EFBFBD>PC <20>賢<EFBFBD><E8B3A2>𨰻<EFBFBD><F0A8B0BB><EFBFBD><EFBFBD>批<EFBFBD>摮条<E691AE><E69DA1><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>銋<EFBFBD><E98A8B>蝑𣇉裦<F0A38789>?| **<EFBFBD>?* 憭滨鍂<E6BBA8>唳<EFBFBD> RDS<44><53><EFBFBD>隞賬<E99A9E><E8B3AC><EFBFBD><EFBFBD>扼<EFBFBD><E689BC><EFBFBD>摰孵<E691B0><E5ADB5>梢燵<E6A2A2>䔶<EFBFBD> RDS <20>条恣<E69DA1>?| **Postgres** |
|
||||
| **憭<EFBFBD>遢銝擧<EFBFBD>憭?* | **<EFBFBD>圈𠗕** Redis RDB/AOF <EFBFBD>W<EFBFBD><EFBFBD>航<EFBFBD>隡帋腺憭望<EFBFBD><EFBFBD>𤾸<EFBFBD>蝘垍<EFBFBD><EFBFBD>唳旿<EFBFBD>?| **摰𣬚<EFBFBD>** RDS <EFBFBD>舀<EFBFBD> PITR (<EFBFBD>㗇𧒄<EFBFBD>渡<EFBFBD><EFBFBD>W<EFBFBD>)<29><>秤<EFBFBD>牐遙<E78990>∪虾蝎曄&<E69B84>墧<EFBFBD><E5A2A7>?1蝘鍦<E89D98><E98DA6>?| **Postgres** |
|
||||
| **<EFBFBD>鞉𧋦** | **瞼1000+/撟?* (Tair <EFBFBD>箇<EFBFBD><EFBFBD>? | **瞼0** (韏<EFBFBD><EFBFBD>憭滨鍂) | **Postgres** |
|
||||
| **<EFBFBD>扯<EFBFBD> (TPS)** | **<EFBFBD><EFBFBD><EFBFBD>** (10w+) | **擃?* (5000+) 撖嫣<EFBFBD><EFBFBD>亙<EFBFBD><EFBFBD>牐<EFBFBD>甈?AI 靚<>鍂<EFBFBD><E98D82>㦤<EFBFBD>荔<EFBFBD>Postgres <20>扯<EFBFBD>蝏啁趕<E59581>劐<EFBFBD><E58A90>?| **Postgres** |
|
||||
| **<EFBFBD><EFBFBD><EFBFBD>鈭厰䔮憸?* | **霂航圾** 霂餃<EFBFBD><EFBFBD>賡<EFBFBD>閬<EFBFBD><EFBFBD>蝏𨅯<EFBFBD>餈䈑<EFBFBD>擃睃僎<EFBFBD>睲<EFBFBD>Redis銋煺<EFBFBD><EFBFBD>厩<EFBFBD>鈭?| **<EFBFBD>毺㮾** SELECT<EFBFBD>臬翰<EFBFBD>扯粉嚗䔶<EFBFBD><EFBFBD>𣳇<EFBFBD><EFBFBD><EFBFBD>ode.js<6A>閧瑪蝔衤<E89D94><E8A1A4><EFBFBD><EFBFBD>銝箇𣂎憸<F0A3828E><E686B8>?| **霂航圾瞉<E59CBE><E79E89>** |
|
||||
| **蝻枏<EFBFBD>皜<EFBFBD><EFBFBD>憌𡡞埯** | **摮睃銁** <EFBFBD><EFBFBD><EFBFBD>皞W枂<EFBFBD><EFBFBD>閬<EFBFBD><EFBFBD>蝵容viction蝑𣇉裦嚗䔶<EFBFBD>敶㯄<EFBFBD>蝵桐<EFBFBD>撖潸稲<EFBFBD>唳旿銝W仃 | **<EFBFBD>舀綉** <20><>鸌<EFBFBD>𣳇膄嚗𡿨IMIT 1000嚗? <20>埝<EFBFBD><E59F9D>𣳇膄嚗峕偶餈靝<E9A488>隡𡁻獈憛𠺶<E6869B>?| **Postgres** |
|
||||
| **摮虫<EFBFBD><EFBFBD>脩瑪** | **<EFBFBD>∪陪** <20><>閬<EFBFBD>郎銋袠edis<EFBFBD><EFBFBD>ullMQ<EFBFBD><EFBFBD>oredis<EFBFBD><EFBFBD><EFBFBD>銋<EFBFBD><EFBFBD>蝑𣇉裦<EFBFBD><EFBFBD><EFBFBD>摮条恣<EFBFBD>?| **撟喟<EFBFBD>** <EFBFBD>芷<EFBFBD>摮虫<EFBFBD>pg-boss嚗㇁PI蝐颱撮BullMQ嚗㚁<EFBFBD><EFBFBD>嗡<EFBFBD><EFBFBD>賣糓<EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>ostgres | **Postgres** |
|
||||
|
||||
### **撣貉<E692A3>霂航圾瞉<E59CBE><E79E89>**
|
||||
|
||||
#### **误解1:Postgres并发性能差**
|
||||
#### **霂航圾1嚗䥪ostgres撟嗅<EFBFBD><EFBFBD>扯<EFBFBD>撌?*
|
||||
```
|
||||
事实:
|
||||
- Postgres可处理5万+ QPS(简单查询)
|
||||
鈭见<EFBFBD>嚗?- Postgres<65>臬<EFBFBD><E887AC>?銝? QPS嚗<53><E59A97><EFBFBD>閙䰻霂g<E99C82>
|
||||
- <20>函<EFBFBD>摰鮋<E691B0>撟嗅<E6929F>: < 50 QPS
|
||||
- SELECT<43>臬翰<E887AC>扯粉嚗㇈VCC嚗㚁<E59A97><E39A81>𣳇<EFBFBD>蝡硺<E89DA1>
|
||||
- Node.js单线程(1-2万QPS上限)会先成为瓶颈
|
||||
|
||||
- Node.js<EFBFBD>閧瑪蝔页<EFBFBD>1-2銝𣘗PS銝𢠃<E98A9D>嚗劐<E59A97><E58A90><EFBFBD><EFBFBD>銝箇𣂎憸?
|
||||
蝏栞捏嚗䥪ostgres銝齿糓<EFBFBD>園<EFBFBD>
|
||||
```
|
||||
|
||||
#### **误解2:DELETE会锁表阻塞**
|
||||
#### **霂航圾2嚗鋽ELETE隡𡁻<EFBFBD>銵券獈憛?*
|
||||
```
|
||||
事实:
|
||||
- DELETE是行级锁,不是表锁
|
||||
- LIMIT 1000,毫秒级完成(~5ms)
|
||||
- 配合懒惰删除,大部分过期数据在读取时已删除
|
||||
- 即使有积压,每分钟1000条,1小时可清理6万条
|
||||
鈭见<EFBFBD>嚗?- DELETE<54>航<EFBFBD>蝥折<E89DA5>嚗䔶<E59A97><E494B6>航”<E888AA>?- LIMIT 1000嚗峕神蝘垍漣摰峕<E691B0>嚗ǚ5ms嚗?- <20>滚<EFBFBD><E6BB9A>埝<EFBFBD><E59F9D>𣳇膄嚗<E88684>之<EFBFBD>典<EFBFBD>餈<EFBFBD><E9A488><EFBFBD>唳旿<E594B3>刻粉<E588BB>𡝗𧒄撌脣<E6928C><E884A3>?- <20>喃蝙<E59683>厩妖<E58EA9>页<EFBFBD>瘥誩<E798A5><E8AAA9>?000<30>∴<EFBFBD>1撠𤩺𧒄<F0A4A9BA>舀<EFBFBD><E88880>?銝<>辺
|
||||
|
||||
结论:不会阻塞
|
||||
```
|
||||
蝏栞捏嚗帋<EFBFBD>隡𡁻獈憛?```
|
||||
|
||||
#### **霂航圾3嚗鑹edis<69><73><EFBFBD><EFBFBD>滢<EFBFBD>銝<EFBFBD>摰𡁜翰**
|
||||
```
|
||||
事实:
|
||||
- Redis: 网络延迟0.1ms + 读取0.05ms = 0.15ms
|
||||
鈭见<EFBFBD>嚗?- Redis: 蝵𤑳<E89DB5>撱嗉<E692B1>0.1ms + 霂餃<E99C82>0.05ms = 0.15ms
|
||||
- Postgres: 蝵𤑳<E89DB5>撱嗉<E692B1>0.5ms + <20>亥砭1ms = 1.5ms
|
||||
- 撌桀<E6928C>: 1.35ms
|
||||
- 但是总响应时间(含业务逻辑): 200ms
|
||||
- 用户感知差异: 0%(1.35/200 < 1%)
|
||||
- 雿<>糓<EFBFBD>餃<EFBFBD>摨娍𧒄<E5A88D>湛<EFBFBD><E6B99B>思<EFBFBD><E6809D>⊿<EFBFBD>餉<EFBFBD>嚗? 200ms
|
||||
- <20>冽<EFBFBD><E586BD>毺䰻撌桀<E6928C>: 0%嚗?.35/200 < 1%嚗?
|
||||
蝏栞捏嚗𡁏<EFBFBD>扯<EFBFBD>撌桀<EFBFBD><EFBFBD>函鍂<EFBFBD>瑚<EFBFBD>撉䔶葉<EFBFBD>䭾<EFBFBD><EFBFBD>?```
|
||||
|
||||
结论:性能差异在用户体验中无感知
|
||||
```
|
||||
## **5\. <20><>笆<EFBFBD>?撠𤩺𧒄<F0A4A9BA>蹂遙<E8B982>﹦<EFBFBD>萘<EFBFBD><E89098>舫<EFBFBD><E888AB>扯<EFBFBD><E689AF>?*
|
||||
|
||||
## **5\. 针对“2小时长任务”的可靠性证明**
|
||||
韐函<EFBFBD>嚗䥪ostgres <20>毺<EFBFBD><E6AFBA>賭<EFBFBD>霂?2 撠𤩺𧒄<F0A4A9BA><F0A79284>遙<EFBFBD>∩<EFBFBD>銝剜鱏<E5899C>梹<EFBFBD>
|
||||
霂<EFBFBD><EFBFBD>嚗?
|
||||
1. **<2A><><EFBFBD><EFBFBD>碶<EFBFBD><E7A2B6>?*嚗帋遙<E5B88B>∩<EFBFBD><E288A9>行<EFBFBD>鈭歹<E988AD>API餈𥪜<E9A488> 200 OK嚗㚁<E59A97><E39A81>喳<EFBFBD><E596B3>亦′<E4BAA6>塩<EFBFBD><E5A1A9>朖雿?SAE <20><>黎銝衤<E98A9D>蝘鍦<E89D98><E98DA6>哨<EFBFBD>隞餃𦛚霈啣<E99C88>靘萘<E99D98><E89098>冽㺭<E586BD>桀<EFBFBD>銝准<E98A9D>?
|
||||
2. **撏拇<E6928F><E68B87>W<EFBFBD><EFBCB7>箏<EFBFBD>**嚗?
|
||||
* **甇<>虜<EFBFBD><E8999C><EFBFBD>**嚗阳orker <20><><EFBFBD>隞餃𦛚 \-\> <20>扯<EFBFBD> 2 撠𤩺𧒄 \-\> <20>𣂷漱蝏𤘪<E89D8F> \-\> <20><>扇摰峕<E691B0><E5B395>?
|
||||
* **撘<>虜<EFBFBD><E8999C><EFBFBD> (SAE 蝻拙捆)**嚗阳orker <20>扯<EFBFBD><E689AF>?1 撠𤩺𧒄鋡恍<E98BA1>瘥?\-\> <20>唳旿摨㯄<E691A8><E3AF84>?4 撠𤩺𧒄<F0A4A9BA>舘<EFBFBD><E88898>?\-\> pg-boss 摰<>擪餈𤤿<E9A488>璉<EFBFBD>瘚见<E7989A>餈<EFBFBD><E9A488> \-\> 撠<>遙<EFBFBD>⊿<EFBFBD><E28ABF>唳<EFBFBD>霈唬蛹 Pending \-\> <20>?Worker 憸<><E686B8><EFBFBD>滩<EFBFBD><E6BBA9>?
|
||||
3. **<2A>剔<EFBFBD>蝏凋<E89D8F>**嚗阳orker <20>臬<EFBFBD><E887AC><EFBFBD><EFBFBD>憒<EFBFBD><E68692> 10 <20><><EFBFBD>嚗㗇凒<E39787>唳㺭<E594B3>桀<EFBFBD>銝剔<E98A9D> progress 摮埈挾<E59F88><E68CBE><EFBFBD>霂閙𧒄霂餃<E99C82> progress嚗䔶<E59A97><E494B6>剔<EFBFBD>蝏抒賒<E68A92>扯<EFBFBD><E689AF>?
|
||||
**蝏栞捏**嚗𡁻<E59A97><F0A181BB>?pg-boss <20><>香靽⊿<E99DBD><E28ABF>梹<EFBFBD>Dead Letter嚗匧<E59A97><E58CA7>滩<EFBFBD>蝑𣇉裦嚗<E8A3A6>虾<EFBFBD>䭾<EFBFBD>抒<EFBFBD><E68A92>𣬚<EFBFBD><F0A3AC9A>喲<EFBFBD>鈭?Redis嚗<73><E59A97>銝?Redis <20><><EFBFBD>皞W枂憌𡡞埯<F0A1A19E>游之嚗剹<E59A97>?
|
||||
## **6\. 摰墧鴌頝舐瑪<E88890>?*
|
||||
|
||||
质疑:Postgres 真的能保证 2 小时的任务不中断吗?
|
||||
证明:
|
||||
### **<2A>嗆挾1嚗帋遙<E5B88B>⊿<EFBFBD><E28ABF>埈㺿<E59F88>𩤃<EFBFBD>Week 1嚗?*
|
||||
|
||||
1. **持久化保障**:任务一旦提交(API返回 200 OK),即写入硬盘。即使 SAE 集群下一秒全灭,任务记录依然在数据库中。
|
||||
2. **崩溃恢复机制**:
|
||||
* **正常情况**:Worker 锁定任务 \-\> 执行 2 小时 \-\> 提交结果 \-\> 标记完成。
|
||||
* **异常情况 (SAE 缩容)**:Worker 执行到 1 小时被销毁 \-\> 数据库锁在 4 小时后过期 \-\> pg-boss 守护进程检测到过期 \-\> 将任务重新标记为 Pending \-\> 新 Worker 领取重试。
|
||||
3. **断点续传**:Worker 可定期(如每 10 分钟)更新数据库中的 progress 字段。重试时读取 progress,从断点继续执行。
|
||||
|
||||
**结论**:配合 pg-boss 的死信队列(Dead Letter)和重试策略,可靠性等同甚至高于 Redis(因为 Redis 内存溢出风险更大)。
|
||||
|
||||
## **6\. 实施路线图**
|
||||
|
||||
### **阶段1:任务队列改造(Week 1)**
|
||||
|
||||
#### **Step 1.1:安装依赖**
|
||||
#### **Step 1.1嚗𡁜<EFBFBD>鋆<EFBFBD><EFBFBD>韏?*
|
||||
```bash
|
||||
cd backend
|
||||
npm install pg-boss --save
|
||||
```
|
||||
|
||||
#### **Step 1.2:实现PgBossQueue适配器**
|
||||
#### **Step 1.2嚗𡁜<EFBFBD><EFBFBD>訐gBossQueue<EFBFBD><EFBFBD><EFBFBD><EFBFBD>?*
|
||||
```typescript
|
||||
// <20><>辣嚗颹ackend/src/common/jobs/PgBossQueue.ts
|
||||
|
||||
@@ -343,9 +302,7 @@ export class PgBossQueue implements JobQueue {
|
||||
constructor() {
|
||||
this.boss = new PgBoss({
|
||||
connectionString: config.databaseUrl,
|
||||
schema: 'job_queue', // 独立schema,不污染业务表
|
||||
max: 5, // 连接池大小
|
||||
// 关键配置:设置锁的有效期为4小时
|
||||
schema: 'job_queue', // <EFBFBD>祉<EFBFBD>schema嚗䔶<EFBFBD>瘙⊥<EFBFBD>銝𡁜𦛚銵? max: 5, // 餈墧𦻖瘙惩之撠? // <20>喲睸<E596B2>滨蔭嚗朞挽蝵桅<E89DB5><E6A185><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>銝?撠𤩺𧒄
|
||||
// 靽肽<E99DBD>2撠𤩺𧒄隞餃𦛚銝滩◤<E6BBA9>Z粥嚗䔶<E59A97>摰硺<E691B0>撏拇<E6928F><E68B87>舘<EFBFBD><E88898>芸𢆡<E88AB8>W<EFBFBD>
|
||||
expireInHours: 4,
|
||||
});
|
||||
@@ -361,7 +318,7 @@ export class PgBossQueue implements JobQueue {
|
||||
|
||||
await this.boss.start();
|
||||
this.started = true;
|
||||
logger.info('[PgBoss] 队列已启动');
|
||||
logger.info('[PgBoss] <20>笔<EFBFBD>撌脣鍳<E884A3>?);
|
||||
}
|
||||
|
||||
async push<T = any>(type: string, data: T, options?: any): Promise<Job> {
|
||||
@@ -369,9 +326,7 @@ export class PgBossQueue implements JobQueue {
|
||||
|
||||
const jobId = await this.boss.send(type, data, {
|
||||
retryLimit: 3,
|
||||
retryDelay: 60, // 失败后60秒重试
|
||||
expireInHours: 4, // 4小时后过期
|
||||
...options
|
||||
retryDelay: 60, // 憭梯揖<EFBFBD>?0蝘㘾<E89D98>霂? expireInHours: 4, // 4撠𤩺𧒄<F0A4A9BA>舘<EFBFBD><E88898>? ...options
|
||||
});
|
||||
|
||||
logger.info('[PgBoss] 隞餃𦛚<E9A483>仿<EFBFBD>', { type, jobId });
|
||||
@@ -387,7 +342,7 @@ export class PgBossQueue implements JobQueue {
|
||||
|
||||
process<T = any>(type: string, handler: JobHandler<T>): void {
|
||||
this.boss.work(type, async (job: any) => {
|
||||
logger.info('[PgBoss] 开始处理任务', {
|
||||
logger.info('[PgBoss] 撘<>憪见<E686AA><E8A781><EFBFBD>遙<EFBFBD>?, {
|
||||
type,
|
||||
jobId: job.id,
|
||||
attemptsMade: job.data.__retryCount || 0
|
||||
@@ -422,7 +377,7 @@ export class PgBossQueue implements JobQueue {
|
||||
}
|
||||
});
|
||||
|
||||
logger.info('[PgBoss] Worker已注册', { type });
|
||||
logger.info('[PgBoss] Worker撌脫釣<EFBFBD>?, { type });
|
||||
}
|
||||
|
||||
async getJob(id: string): Promise<Job | null> {
|
||||
@@ -439,8 +394,7 @@ export class PgBossQueue implements JobQueue {
|
||||
}
|
||||
|
||||
async updateProgress(id: string, progress: number, message?: string): Promise<void> {
|
||||
// pg-boss暂不支持进度更新,可通过更新业务表实现
|
||||
logger.debug('[PgBoss] 进度更新', { id, progress, message });
|
||||
// pg-boss<EFBFBD><EFBFBD><EFBFBD><EFBFBD>舀<EFBFBD>餈𥕦漲<EFBFBD>湔鰵嚗<EFBFBD>虾<EFBFBD>朞<EFBFBD><EFBFBD>湔鰵銝𡁜𦛚銵典<EFBFBD><EFBFBD>? logger.debug('[PgBoss] 餈𥕦漲<F0A595A6>湔鰵', { id, progress, message });
|
||||
}
|
||||
|
||||
async cancelJob(id: string): Promise<boolean> {
|
||||
@@ -454,8 +408,7 @@ export class PgBossQueue implements JobQueue {
|
||||
}
|
||||
|
||||
async cleanup(olderThan: number = 86400000): Promise<number> {
|
||||
// pg-boss有自动清理机制
|
||||
return 0;
|
||||
// pg-boss<EFBFBD>㕑䌊<EFBFBD>冽<EFBFBD><EFBFBD><EFBFBD>㦤<EFBFBD>? return 0;
|
||||
}
|
||||
|
||||
private mapState(state: string): string {
|
||||
@@ -469,7 +422,7 @@ export class PgBossQueue implements JobQueue {
|
||||
|
||||
async close(): Promise<void> {
|
||||
await this.boss.stop();
|
||||
logger.info('[PgBoss] 队列已关闭');
|
||||
logger.info('[PgBoss] <20>笔<EFBFBD>撌脣<E6928C><E884A3>?);
|
||||
}
|
||||
}
|
||||
```
|
||||
@@ -517,7 +470,7 @@ export class JobFactory {
|
||||
}
|
||||
```
|
||||
|
||||
#### **Step 1.4:更新环境变量**
|
||||
#### **Step 1.4嚗𡁏凒<EFBFBD>啁㴓憓<EFBFBD><EFBFBD><EFBFBD>?*
|
||||
```env
|
||||
# backend/.env
|
||||
QUEUE_TYPE=pgboss
|
||||
@@ -526,19 +479,16 @@ DATABASE_URL=postgresql://user:password@host:5432/dbname
|
||||
|
||||
#### **Step 1.5嚗𡁏<EFBFBD>霂𤏪<EFBFBD>2撠𤩺𧒄<EFBFBD>蹂遙<EFBFBD>∴<EFBFBD>**
|
||||
```bash
|
||||
# 提交1000篇文献筛选任务
|
||||
curl -X POST http://localhost:3001/api/v1/asl/projects/:id/screening
|
||||
# <20>𣂷漱1000蝭<30><E89DAD><EFBFBD>桃<EFBFBD><E6A183>劐遙<E58A90>?curl -X POST http://localhost:3001/api/v1/asl/projects/:id/screening
|
||||
|
||||
# 等待处理到50% → 手动停止服务(Ctrl+C)
|
||||
# 重启服务
|
||||
# 蝑匧<EFBFBD>憭<EFBFBD><EFBFBD><EFBFBD>?0% <20>?<3F>见𢆡<E8A781>𨀣迫<F0A880A3>滚𦛚嚗㇃trl+C嚗?# <20>滚鍳<E6BB9A>滚𦛚
|
||||
npm run dev
|
||||
|
||||
# 查看任务状态 → 应该自动恢复并继续处理
|
||||
```
|
||||
# <EFBFBD>亦<EFBFBD>隞餃𦛚<EFBFBD>嗆<EFBFBD>?<3F>?摨磰砲<E7A3B0>芸𢆡<E88AB8>W<EFBFBD>撟嗥誧蝏剖<E89D8F><E58996>?```
|
||||
|
||||
---
|
||||
|
||||
### **阶段2:缓存改造(Week 2)**
|
||||
### **<EFBFBD>嗆挾2嚗𡁶<EFBFBD>摮䀹㺿<EFBFBD>𩤃<EFBFBD>Week 2嚗?*
|
||||
|
||||
#### **Step 2.1嚗𡁏溶<EFBFBD>蘯risma Schema**
|
||||
```prisma
|
||||
@@ -563,8 +513,7 @@ npx prisma migrate dev --name add_app_cache
|
||||
```
|
||||
|
||||
#### **Step 2.2嚗𡁜<EFBFBD><EFBFBD>訐ostgresCacheAdapter**
|
||||
(见上文"3.2 替代 Redis 缓存"部分)
|
||||
|
||||
嚗<EFBFBD><EFBFBD>銝𦠜<EFBFBD>"3.2 <EFBFBD>蹂誨 Redis 蝻枏<EFBFBD>"<22>典<EFBFBD>嚗?
|
||||
#### **Step 2.3嚗𡁏凒<EFBFBD>蚓acheFactory**
|
||||
```typescript
|
||||
// <20><>辣嚗颹ackend/src/common/cache/CacheFactory.ts
|
||||
@@ -585,7 +534,7 @@ export class CacheFactory {
|
||||
}
|
||||
```
|
||||
|
||||
#### **Step 2.4:启动定时清理**
|
||||
#### **Step 2.4嚗𡁜鍳<EFBFBD>典<EFBFBD><EFBFBD>嗆<EFBFBD><EFBFBD>?*
|
||||
```typescript
|
||||
// <20><>辣嚗颹ackend/src/index.ts
|
||||
|
||||
@@ -598,120 +547,88 @@ startCacheCleanupTask(); // 启动缓存清理
|
||||
|
||||
---
|
||||
|
||||
### **阶段3:SAE部署(Week 3)**
|
||||
### **<EFBFBD>嗆挾3嚗锭AE<EFBFBD>函蔡嚗Áeek 3嚗?*
|
||||
|
||||
1. **本地测试通过**(ASL 1000篇文献 + DC 100份病历)
|
||||
2. **数据库连接配置**(SAE环境变量设置DATABASE_URL)
|
||||
3. **灰度发布**(先1个实例,观察24小时)
|
||||
4. **全量上线**(扩容到2-3个实例)
|
||||
1. **<2A>砍𧑐瘚贝<E7989A><E8B49D>朞<EFBFBD>**嚗㇁SL 1000蝭<EFBFBD><EFBFBD><EFBFBD>?+ DC 100隞賜<EFBFBD><EFBFBD><EFBFBD><EFBFBD>
|
||||
2. **<2A>唳旿摨栞<E691A8><E6A09E>仿<EFBFBD>蝵?*嚗𠄎AE<41>臬<EFBFBD><E887AC>㗛<EFBFBD>霈曄蔭DATABASE_URL嚗?3. **<2A>啣漲<E595A3>穃<EFBFBD>**嚗<><E59A97>1銝芸<E98A9D>靘页<E99D98>閫<EFBFBD><E996AB>24撠𤩺𧒄嚗?4. **<2A>券<EFBFBD>銝羓瑪**嚗<><E59A97>摰孵<E691B0>2-3銝芸<E98A9D>靘页<E99D98>
|
||||
|
||||
## **7\. 性能边界与扩展路径**
|
||||
## **7\. <EFBFBD>扯<EFBFBD>颲寧<EFBFBD>銝擧<EFBFBD>撅閗楝敺?*
|
||||
|
||||
### **7.1 <20><>鍂閫<E98D82>芋**
|
||||
|
||||
| 指标 | Postgres-Only方案上限 | 您的当前值 | 安全余量 |
|
||||
| <EFBFBD><EFBFBD><EFBFBD> | Postgres-Only<EFBFBD>寞<EFBFBD>銝𢠃<EFBFBD> | <20>函<EFBFBD>敶枏<E695B6><E69E8F>?| 摰匧<E691B0>雿䠷<E99BBF> |
|
||||
|------|---------------------|-----------|---------|
|
||||
| 日活用户 | 10万 | 500 | 200倍 |
|
||||
| 并发QPS | 5000 | < 50 | 100倍 |
|
||||
| 缓存容量 | 10GB | < 100MB | 100倍 |
|
||||
| 队列吞吐 | 1000任务/小时 | < 50任务/小时 | 20倍 |
|
||||
| <EFBFBD>交暑<EFBFBD>冽<EFBFBD> | 10銝?| 500 | 200<EFBFBD>?|
|
||||
| 撟嗅<EFBFBD>QPS | 5000 | < 50 | 100<EFBFBD>?|
|
||||
| 蝻枏<EFBFBD>摰寥<EFBFBD> | 10GB | < 100MB | 100<EFBFBD>?|
|
||||
| <EFBFBD>笔<EFBFBD><EFBFBD>𧼮<EFBFBD> | 1000隞餃𦛚/撠𤩺𧒄 | < 50隞餃𦛚/撠𤩺𧒄 | 20<EFBFBD>?|
|
||||
|
||||
**结论:在可预见的未来(2-3年),您不会超出这个上限。**
|
||||
**蝏栞捏嚗𡁜銁<EFBFBD>舫<EFBFBD>閫<EFBFBD><EFBFBD><EFBFBD>芣䔉嚗?-3撟湛<E6929F>嚗峕<E59A97>銝滢<E98A9D>頞<EFBFBD>枂餈嗘葵銝𢠃<E98A9D><F0A2A083>?*
|
||||
|
||||
### **7.2 何时需要Redis?**
|
||||
|
||||
只有在以下情况发生时,才需要考虑引入Redis:
|
||||
### **7.2 雿閙𧒄<EFBFBD><EFBFBD>閬<EFBFBD>edis嚗?*
|
||||
|
||||
<EFBFBD>芣<EFBFBD><EFBFBD>其誑銝𧢲<EFBFBD><EFBFBD>萄<EFBFBD><EFBFBD><EFBFBD>𧒄嚗峕<EFBFBD><EFBFBD><EFBFBD>閬<EFBFBD><EFBFBD><EFBFBD><EFBFBD>撘訫<EFBFBD>Redis嚗?
|
||||
```
|
||||
閫血<EFBFBD><EFBFBD>∩辣嚗㇁NY嚗㚁<EFBFBD>
|
||||
✅ 日活 > 5万
|
||||
✅ 并发QPS > 1000
|
||||
✅ Postgres CPU使用率持续 > 70%
|
||||
✅ 缓存查询延迟 > 50ms(P99)
|
||||
✅ LLM API月成本 > ¥5000(缓存命中率低)
|
||||
<EFBFBD>?<3F>交暑 > 5銝?<3F>?撟嗅<E6929F>QPS > 1000
|
||||
<EFBFBD>?Postgres CPU雿輻鍂<E8BCBB><E98D82><EFBFBD>蝏?> 70%
|
||||
<EFBFBD>?蝻枏<E89DBB><E69E8F>亥砭撱嗉<E692B1> > 50ms嚗㇊99嚗?<3F>?LLM API<50><49><EFBFBD><EFBFBD>?> 瞼5000嚗<30><E59A97>摮睃𦶢銝剔<E98A9D>雿𠬍<E99BBF>
|
||||
|
||||
迁移策略:
|
||||
1. 先迁移LLM缓存到Redis(高频读)
|
||||
2. 保持任务队列在Postgres(强一致性)
|
||||
餈<EFBFBD>宏蝑𣇉裦嚗?1. <20><><EFBFBD>蝘腿LM蝻枏<E89DBB><E69E8F>訌edis嚗<73><E59A97>憸𤏸粉嚗?2. 靽脲<E99DBD>隞餃𦛚<E9A483>笔<EFBFBD><E7AC94>沌ostgres嚗<73>撩銝<E692A9><E98A9D>湔<EFBFBD>改<EFBFBD>
|
||||
3. 銝𡁜𦛚蝻枏<E89DBB><E69E8F>厰<EFBFBD>餈<EFBFBD>宏
|
||||
|
||||
成本:
|
||||
- 迁移工作量: 2-3天
|
||||
- 运维增加: 可接受(已有经验)
|
||||
```
|
||||
<EFBFBD>鞉𧋦嚗?- 餈<>宏撌乩<E6928C><E4B9A9>? 2-3憭?- 餈鞟輕憓𧼮<E68693>: <20>舀𦻖<E88880>梹<EFBFBD>撌脫<E6928C>蝏誯<E89D8F>嚗?```
|
||||
|
||||
### **7.3 <20>拙<EFBFBD>頝臬<E9A09D>**
|
||||
|
||||
```
|
||||
阶段1(当前-5000用户): Postgres-Only
|
||||
<EFBFBD>嗆挾1嚗<EFBFBD><EFBFBD><EFBFBD>?5000<30>冽<EFBFBD>嚗? Postgres-Only
|
||||
<20>鎿<EFBFBD> <20>笔<EFBFBD>: pg-boss
|
||||
├─ 缓存: Postgres表
|
||||
└─ 成本: ¥0
|
||||
<EFBFBD>鎿<EFBFBD> 蝻枏<E89DBB>: Postgres銵? <20>婙<EFBFBD> <20>鞉𧋦: 瞼0
|
||||
|
||||
阶段2(5000-5万用户): 混合架构
|
||||
<EFBFBD>嗆挾2嚗?000-5銝<35>鍂<EFBFBD>瘀<EFBFBD>: 瘛瑕<E7989B><E79195>嗆<EFBFBD>
|
||||
<20>鎿<EFBFBD> <20>笔<EFBFBD>: pg-boss嚗<73><E59A97><EFBFBD><EFBFBD><EFBFBD>
|
||||
<20>鎿<EFBFBD> LLM蝻枏<E89DBB>: Redis嚗<73><E59A97>蝘鳴<E89D98>
|
||||
<20>鎿<EFBFBD> 銝𡁜𦛚蝻枏<E89DBB>: Postgres嚗<73><E59A97><EFBFBD><EFBFBD><EFBFBD>
|
||||
└─ 成本: +¥1000/年
|
||||
|
||||
阶段3(5万-50万用户): 全Redis
|
||||
<EFBFBD>婙<EFBFBD> <20>鞉𧋦: +瞼1000/撟?
|
||||
<EFBFBD>嗆挾3嚗?銝?50銝<30>鍂<EFBFBD>瘀<EFBFBD>: <20>沖edis
|
||||
<20>鎿<EFBFBD> <20>笔<EFBFBD>: BullMQ + Redis
|
||||
<20>鎿<EFBFBD> 蝻枏<E89DBB>: Redis
|
||||
└─ 成本: +¥5000/年
|
||||
```
|
||||
<EFBFBD>婙<EFBFBD> <20>鞉𧋦: +瞼5000/撟?```
|
||||
|
||||
---
|
||||
|
||||
## **8\. FAQ嚗<51>虜閫<E8999C><E996AB><EFBFBD>殷<EFBFBD>**
|
||||
|
||||
### **Q1: pg-boss会不会拖慢Postgres?**
|
||||
### **Q1: pg-boss隡帋<EFBFBD>隡𡁏<EFBFBD><EFBFBD>㎜ostgres嚗?*
|
||||
|
||||
**A:** 不会。pg-boss的查询都有索引优化,单次查询 < 5ms。即使100个Worker同时抢任务,也只是500ms的额外负载,对于5万QPS的Postgres来说可忽略。
|
||||
|
||||
### **Q2: 缓存表会不会无限增长?**
|
||||
|
||||
**A:** 不会。懒惰删除 + 分批清理,过期数据会被自动清理。即使有积压,每分钟1000条的清理速度足以应对。
|
||||
**A:** 銝滢<E98A9D><E6BBA2><EFBFBD>g-boss<73><73>䰻霂a<E99C82><EFBD81>厩揣撘蓥<E69298><E893A5>吔<EFBFBD><E59094>閙活<E99699>亥砭 < 5ms<6D><73>朖雿?00銝杪orker<65>峕𧒄<E5B395>V遙<EFBCB6>∴<EFBFBD>銋笔蘨<E7AC94>?00ms<6D><73><EFBFBD>憭𤥁<E686AD>頧踝<E9A0A7>撖嫣<E69296>5銝𣘗PS<50><53>ostgres<65>亥秩<E4BAA5>臬蕭<E887AC>乓<EFBFBD>?
|
||||
### **Q2: 蝻枏<E89DBB>銵其<E98AB5>銝滢<E98A9D><E6BBA2>𣳇<EFBFBD>憓鮋鵭嚗?*
|
||||
|
||||
**A:** 銝滢<E98A9D><E6BBA2><EFBFBD><EFBFBD><EFBFBD>啣<EFBFBD><E595A3>?+ <20><>鸌皜<E9B88C><E79A9C>嚗諹<E59A97><E8ABB9><EFBFBD>㺭<EFBFBD>桐<EFBFBD>鋡怨䌊<E680A8>冽<EFBFBD><E586BD><EFBFBD><EFBFBD><EFBFBD>朖雿踵<E99BBF>蝘臬<E89D98>嚗峕<E59A97><E5B395><EFBFBD><EFBFBD>1000<30>∠<EFBFBD>皜<EFBFBD><E79A9C><EFBFBD>笔漲頞喃誑摨𥪜笆<F0A5AA9C>?
|
||||
### **Q3: 憒<><E68692>Postgres<65><73><EFBFBD><EFBFBD>𦒘<EFBFBD><F0A69298>痹<EFBFBD>**
|
||||
|
||||
**A:**
|
||||
- **阿里云RDS**:高可用版自动主从切换,故障恢复 < 30秒
|
||||
- **备份恢复**:PITR可恢复到任意秒,数据不丢失
|
||||
- **降级策略**:队列和缓存都在DB,一起恢复,无不一致风险
|
||||
|
||||
相比之下,Redis挂了还需要担心数据不一致问题。
|
||||
|
||||
- **<2A>輸<EFBFBD>鈭駵DS**嚗𡁻<E59A97><F0A181BB>舐鍂<E88890><E98D82>䌊<EFBFBD>其蜓隞𤾸<E99A9E><F0A4BEB8>g<EFBFBD><EFBD87><EFBFBD><EFBFBD><EFBFBD>W<EFBFBD> < 30蝘?- **憭<>遢<EFBFBD>W<EFBFBD>**嚗䥪ITR<54>舀<EFBFBD>憭滚<E686AD>隞餅<E99A9E>蝘𡜐<E89D98><F0A19C90>唳旿銝滢腺憭?- **<2A>滨漣蝑𣇉裦**嚗𡁻<E59A97><F0A181BB>堒<EFBFBD>蝻枏<E89DBB><E69E8F>賢銁DB嚗䔶<E59A97>韏瑟<E99F8F>憭㵪<E686AD><E3B5AA>牐<EFBFBD>銝<EFBFBD><E98A9D>湧<EFBFBD><E6B9A7>?
|
||||
<EFBFBD>豢<EFBFBD>銋衤<EFBFBD>嚗朙edis<EFBFBD><EFBFBD><EFBFBD>餈㗛<EFBFBD>閬<EFBFBD><EFBFBD>敹<EFBFBD>㺭<EFBFBD>桐<EFBFBD>銝<EFBFBD><EFBFBD>湧䔮憸塩<EFBFBD>?
|
||||
### **Q4: 銝箔<E98A9D>銋<EFBFBD><E98A8B><EFBFBD>沖edis嚗<73>㭂霂渲䌊撌望糓鈭穃<E988AD><E7A983><EFBFBD><EFBFBD>**
|
||||
|
||||
**A:** 云原生的核心是**状态外置**,不是**必须用Redis**。
|
||||
|
||||
```
|
||||
云原生的本质:
|
||||
✅ 无状态应用(不依赖本地内存)
|
||||
✅ 状态持久化(数据不丢失)
|
||||
✅ 水平扩展(多实例协调)
|
||||
|
||||
Postgres-Only完全满足:
|
||||
✅ 状态存储在RDS(外置)
|
||||
✅ 任务持久化(不丢失)
|
||||
✅ pg-boss支持多实例(SKIP LOCKED)
|
||||
|
||||
Redis只是实现方式之一,不是唯一方式。
|
||||
**A:** 鈭穃<E988AD><E7A983>毺<EFBFBD><E6AFBA>詨<EFBFBD><E8A9A8>?*<2A>嗆<EFBFBD><E59786><EFBFBD>蝵?*嚗䔶<E59A97><E494B6>?*敹<>◆<EFBFBD>沖edis**<EFBFBD>?
|
||||
```
|
||||
鈭穃<EFBFBD><EFBFBD>毺<EFBFBD><EFBFBD>祈捶嚗?<3F>?<3F>删𠶖<E588A0><F0A0B696><EFBFBD><EFBFBD>剁<EFBFBD>銝滢<E98A9D>韏𡝗𧋦<F0A19D97>啣<EFBFBD>摮矋<E691AE>
|
||||
<EFBFBD>?<3F>嗆<EFBFBD><E59786><EFBFBD>銋<EFBFBD><E98A8B>嚗<EFBFBD>㺭<EFBFBD>桐<EFBFBD>銝W仃嚗?<3F>?瘞游像<E6B8B8>拙<EFBFBD>嚗<EFBFBD><E59A97>摰硺<E691B0><E7A1BA>讛<EFBFBD>嚗?
|
||||
Postgres-Only摰<79><E691B0>皛∟雲嚗?<3F>?<3F>嗆<EFBFBD><E59786><EFBFBD><EFBFBD>典銁RDS嚗<53><E59A97>蝵殷<E89DB5>
|
||||
<EFBFBD>?隞餃𦛚<E9A483><F0A69B9A><EFBFBD><EFBFBD>吔<EFBFBD>銝滢腺憭梧<E686AD>
|
||||
<EFBFBD>?pg-boss<73>舀<EFBFBD>憭𡁜<E686AD>靘页<E99D98>SKIP LOCKED嚗?
|
||||
Redis<EFBFBD>芣糓摰䂿緵<EFBFBD>孵<EFBFBD>銋衤<EFBFBD>嚗䔶<EFBFBD><EFBFBD>臬𣈲銝<EFBFBD><EFBFBD>孵<EFBFBD><EFBFBD>?```
|
||||
|
||||
---
|
||||
|
||||
## **9\. 蝏栞祗**
|
||||
|
||||
对于 1-2 人规模的精益创业团队,**技术栈的坍缩(Stack Collapse)是降低熵增的最佳手段**。
|
||||
撖嫣<EFBFBD> 1-2 鈭箄<EFBFBD>璅∠<EFBFBD>蝎曄<EFBFBD><EFBFBD>𥕢<EFBFBD><EFBFBD>a<EFBFBD>嚗?*<2A><><EFBFBD>舀<EFBFBD><E88880><EFBFBD><EFBFBD>蝻抬<E89DBB>Stack Collapse嚗㗇糓<E39787>滢<EFBFBD><E6BBA2>萄<EFBFBD><E89084><EFBFBD><EFBFBD>雿單<E99BBF>畾?*<2A>?
|
||||
<EFBFBD>㗇𥋘 Postgres-Only 銝齿糓<E9BDBF>牐蛹<E78990>睲賑<E79DB2><E8B391><EFBFBD>航氜<E888AA>𠬍<EFBFBD><F0A0AC8D>峕糓<E5B395>牐蛹<E78990>睲賑撖寞<E69296><E5AF9E>舀<EFBFBD><E88880><EFBFBD><EFBFBD>湔楛<E6B994>餌<EFBFBD><E9A48C><EFBFBD>圾嚗?
|
||||
- <20>睲賑<E79DB2><E8B391>圾**撟嗅<E6929F><E59785><EFBFBD><EFBFBD>摰噼<E691B0>璅?*嚗<><E59A97><EFBFBD>航<EFBFBD><E888AA>喟<EFBFBD><E5969F>曆<EFBFBD>QPS嚗?- <20>睲賑<E79DB2><E8B391>圾**Postgres<65><73><EFBFBD><EFBFBD>𥡝器<F0A5A19D>?*嚗<><E59A97><EFBFBD>臬㫲鞊∩葉<E288A9>?<3F>?嚗?- <20>睲賑<E79DB2><E8B391>圾**<2A>嗆<EFBFBD><E59786><EFBFBD>瓲敹<E793B2>𤌍<EFBFBD>?*嚗<>迅摰𡁏<E691B0>?> <20>急<EFBFBD>嚗?- <20>睲賑<E79DB2><E8B391>圾**<2A>a<EFBFBD><EFBD81><EFBFBD><EFBFBD>摰噼<E691B0><E599BC>?*嚗<><E59A97>蝏渲<E89D8F><E6B8B2>?= 蝔喳<E89D94><E596B3>改<EFBFBD>
|
||||
|
||||
选择 Postgres-Only 不是因为我们技术落后,而是因为我们对技术有着更深刻的理解:
|
||||
|
||||
- 我们理解**并发的真实规模**(不是臆想的百万QPS)
|
||||
- 我们理解**Postgres的能力边界**(不是印象中的"慢")
|
||||
- 我们理解**架构的核心目标**(稳定性 > 炫技)
|
||||
- 我们理解**团队的真实能力**(运维能力 = 稳定性)
|
||||
|
||||
我们选择用**架构的简洁性**来换取**运维的稳定性**,用**务实的判断**来换取**业务的快速迭代**。
|
||||
|
||||
**这不是妥协,这是智慧。**
|
||||
<EFBFBD>睲賑<EFBFBD>㗇𥋘<EFBFBD>?*<2A>嗆<EFBFBD><E59786><EFBFBD><EFBFBD>瘣<EFBFBD><E798A3>?*<2A>交揢<E4BAA4>?*餈鞟輕<E99E9F><E8BC95>迅摰𡁏<E691B0>?*嚗𣬚鍂**<2A>∪<EFBFBD><E288AA><EFBFBD>ế<EFBFBD>?*<2A>交揢<E4BAA4>?*銝𡁜𦛚<F0A1819C><F0A69B9A>翰<EFBFBD>蠘翮隞?*<2A>?
|
||||
**餈嗘<E9A488><E59798>臬戎<E887AC>𧶏<EFBFBD>餈蹱糓<E8B9B1>箸<EFBFBD><E7AEB8>?*
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,42 +1,41 @@
|
||||
# Postgres-Only 鏋舵瀯鏀归€犺繘搴﹁拷韪<E68BB7>〃
|
||||
|
||||
> **开始日期:** 2025年12月7日
|
||||
> **预计完成:** 2025年12月16日(9天)
|
||||
> **实际完成:** 2025年12月13日(Phase 1-7 完成) 🎉
|
||||
> **负责人:** 开发团队
|
||||
> **当前Phase:** Phase 7 已完成,Phase 8 待进行
|
||||
|
||||
> **寮€濮嬫棩鏈燂細** 2025骞?2鏈?鏃?
|
||||
> **棰勮<EFBFBD>瀹屾垚锛?* 2025骞?2鏈?6鏃ワ紙9澶╋級
|
||||
> **瀹為檯瀹屾垚锛?* 2025骞?2鏈?3鏃ワ紙Phase 1-7 瀹屾垚锛?馃帀
|
||||
> **璐熻矗浜猴細** 寮€鍙戝洟闃?
|
||||
> **褰撳墠Phase锛?* Phase 7 宸插畬鎴愶紝Phase 8 寰呰繘琛?
|
||||
---
|
||||
|
||||
## 馃搳 鎬讳綋杩涘害姒傝<E5A792>
|
||||
|
||||
| 鎸囨爣 | 鐩<>爣 | 褰撳墠 | 杩涘害 |
|
||||
|------|------|------|------|
|
||||
| **总任务数** | 45个 | 完成 31个 | 69% ✅ |
|
||||
| **总工作量** | 9天 | 已用 6.5天 | 72% ✅ |
|
||||
| **代码行数** | ~1900行 | 已写 ~1750行 | 92% ✅ |
|
||||
| **测试通过** | 100% | 100% | 100% ✅ |
|
||||
| **鎬讳换鍔℃暟** | 45涓?| 瀹屾垚 31涓?| 69% 鉁?|
|
||||
| **鎬诲伐浣滈噺** | 9澶?| 宸茬敤 6.5澶?| 72% 鉁?|
|
||||
| **浠g爜琛屾暟** | ~1900琛?| 宸插啓 ~1750琛?| 92% 鉁?|
|
||||
| **娴嬭瘯閫氳繃** | 100% | 100% | 100% 鉁?|
|
||||
|
||||
**当前状态:** 🟢 **Phase 1-7 已完成!Platform-Only 架构重构完成!**
|
||||
**褰撳墠鐘舵€侊細** 馃煝 **Phase 1-7 宸插畬鎴愶紒Platform-Only 鏋舵瀯閲嶆瀯瀹屾垚锛?*
|
||||
|
||||
---
|
||||
|
||||
## 馃搮 Phase杩涘害鎬昏<E98EAC>
|
||||
|
||||
| Phase | 名称 | 任务数 | 工作量 | 状态 | 开始日期 | 完成日期 | 备注 |
|
||||
| Phase | 鍚嶇О | 浠诲姟鏁?| 宸ヤ綔閲?| 鐘舵€?| 寮€濮嬫棩鏈?| 瀹屾垚鏃ユ湡 | 澶囨敞 |
|
||||
|-------|------|--------|--------|------|----------|----------|------|
|
||||
| **Phase 1** | 环境准备 | 4 | 0.5天 | ✅ | 12-07 | 12-07 | 完成 |
|
||||
| **Phase 2** | PostgresCacheAdapter | 5 | 0.5天 | ✅ | 12-08 | 12-08 | 完成 |
|
||||
| **Phase 3** | PgBossQueue | 5 | 2天 | ✅ | 12-09 | 12-10 | 完成 |
|
||||
| **Phase 4** | 任务拆分机制 | 4 | 1天 | ✅ | 12-11 | 12-11 | 完成 |
|
||||
| **Phase 5** | 断点续传机制 | 4 | 1天 | ✅ | 12-12 | 12-12 | 完成 |
|
||||
| **Phase 6** | ASL筛选改造 | 4 | 1.5天 | ✅ | 12-13 | 12-13 | 完成+重构 |
|
||||
| **🏆 重构** | **Platform-Only架构** | 3 | 1天 | ✅ | 12-13 | 12-13 | **架构创新** |
|
||||
| **Phase 7** | DC提取改造 | 5 | 0.5天 | ✅ | 12-13 | 12-13 | 完成 |
|
||||
| **Phase 8** | 全面测试验证 | 7 | 1.5天 | ⬜ | _____ | _____ | 待进行 |
|
||||
| **Phase 9** | SAE部署上线 | 5 | 0.5天 | ⬜ | _____ | _____ | 待进行 |
|
||||
| **Phase 1** | 鐜<EFBFBD><EFBFBD>鍑嗗<EFBFBD> | 4 | 0.5澶?| 鉁?| 12-07 | 12-07 | 瀹屾垚 |
|
||||
| **Phase 2** | PostgresCacheAdapter | 5 | 0.5澶?| 鉁?| 12-08 | 12-08 | 瀹屾垚 |
|
||||
| **Phase 3** | PgBossQueue | 5 | 2澶?| 鉁?| 12-09 | 12-10 | 瀹屾垚 |
|
||||
| **Phase 4** | 浠诲姟鎷嗗垎鏈哄埗 | 4 | 1澶?| 鉁?| 12-11 | 12-11 | 瀹屾垚 |
|
||||
| **Phase 5** | 鏂<EFBFBD>偣缁<EFBFBD>紶鏈哄埗 | 4 | 1澶?| 鉁?| 12-12 | 12-12 | 瀹屾垚 |
|
||||
| **Phase 6** | ASL绛涢€夋敼閫?| 4 | 1.5澶?| 鉁?| 12-13 | 12-13 | 瀹屾垚+閲嶆瀯 |
|
||||
| **馃弳 閲嶆瀯** | **Platform-Only鏋舵瀯** | 3 | 1澶?| 鉁?| 12-13 | 12-13 | **鏋舵瀯鍒涙柊** |
|
||||
| **Phase 7** | DC鎻愬彇鏀归€?| 5 | 0.5澶?| 鉁?| 12-13 | 12-13 | 瀹屾垚 |
|
||||
| **Phase 8** | 鍏ㄩ潰娴嬭瘯楠岃瘉 | 7 | 1.5澶?| 猬?| _____ | _____ | 寰呰繘琛?|
|
||||
| **Phase 9** | SAE閮ㄧ讲涓婄嚎 | 5 | 0.5澶?| 猬?| _____ | _____ | 寰呰繘琛?|
|
||||
|
||||
**图例:** ⬜ 待开始 | 🟡 进行中 | ✅ 已完成 | ❌ 失败 | ⏸️ 暂停
|
||||
**鍥句緥锛?* 猬?寰呭紑濮?| 馃煛 杩涜<E69DA9>涓?| 鉁?宸插畬鎴?| 鉂?澶辫触 | 鈴革笍 鏆傚仠
|
||||
|
||||
---
|
||||
|
||||
@@ -44,157 +43,132 @@
|
||||
|
||||
### Phase 1锛氱幆澧冨噯澶囷紙0.5澶╋級
|
||||
|
||||
| # | 任务 | 优先级 | 状态 | 开始时间 | 完成时间 | 耗时 | 负责人 | 备注 |
|
||||
| # | 浠诲姟 | 浼樺厛绾?| 鐘舵€?| 寮€濮嬫椂闂?| 瀹屾垚鏃堕棿 | 鑰楁椂 | 璐熻矗浜?| 澶囨敞 |
|
||||
|---|------|--------|------|----------|----------|------|--------|------|
|
||||
| 1.1 | 安装pg-boss依赖 | P0 | ✅ | 12-07 10:00 | 12-07 10:05 | 5min | 团队 | 完成 |
|
||||
| 1.2 | 更新Prisma Schema | P0 | ✅ | 12-07 10:05 | 12-07 10:30 | 25min | 团队 | 添加AppCache模型 |
|
||||
| 1.3 | 执行数据库迁移 | P0 | ✅ | 12-07 10:30 | 12-07 11:00 | 30min | 团队 | 手动SQL迁移 |
|
||||
| 1.4 | 更新env.ts配置 | P0 | ✅ | 12-07 11:00 | 12-07 11:15 | 15min | 团队 | 完成 |
|
||||
|
||||
**验收标准:**
|
||||
- [x] `npm list pg-boss` 显示版本号 ✅
|
||||
- [x] `platform_schema.app_cache` 表已创建 ✅
|
||||
- [x] 本地环境启动无错误 ✅
|
||||
| 1.1 | 瀹夎<EFBFBD>pg-boss渚濊禆 | P0 | 鉁?| 12-07 10:00 | 12-07 10:05 | 5min | 鍥㈤槦 | 瀹屾垚 |
|
||||
| 1.2 | 鏇存柊Prisma Schema | P0 | 鉁?| 12-07 10:05 | 12-07 10:30 | 25min | 鍥㈤槦 | 娣诲姞AppCache妯″瀷 |
|
||||
| 1.3 | 鎵ц<EFBFBD>鏁版嵁搴撹縼绉?| P0 | 鉁?| 12-07 10:30 | 12-07 11:00 | 30min | 鍥㈤槦 | 鎵嬪姩SQL杩佺Щ |
|
||||
| 1.4 | 鏇存柊env.ts閰嶇疆 | P0 | 鉁?| 12-07 11:00 | 12-07 11:15 | 15min | 鍥㈤槦 | 瀹屾垚 |
|
||||
|
||||
**楠屾敹鏍囧噯锛?*
|
||||
- [x] `npm list pg-boss` 鏄剧ず鐗堟湰鍙?鉁?- [x] `platform_schema.app_cache` 琛ㄥ凡鍒涘缓 鉁?- [x] 鏈<>湴鐜<E6B9B4><E9909C>鍚<EFBFBD>姩鏃犻敊璇?鉁?
|
||||
**閬囧埌鐨勯棶棰橈細**
|
||||
```
|
||||
问题1:
|
||||
描述:
|
||||
解决方案:
|
||||
解决时间:
|
||||
|
||||
问题2:
|
||||
描述:
|
||||
解决方案:
|
||||
解决时间:
|
||||
```
|
||||
闂<EFBFBD><EFBFBD>1锛?鎻忚堪锛?瑙e喅鏂规<E98F82>锛?瑙e喅鏃堕棿锛?
|
||||
闂<EFBFBD><EFBFBD>2锛?鎻忚堪锛?瑙e喅鏂规<E98F82>锛?瑙e喅鏃堕棿锛?```
|
||||
|
||||
---
|
||||
|
||||
### Phase 2:实现PostgresCacheAdapter(0.5天)
|
||||
### Phase 2锛氬疄鐜癙ostgresCacheAdapter锛?.5澶╋級
|
||||
|
||||
| # | 任务 | 优先级 | 状态 | 开始时间 | 完成时间 | 耗时 | 负责人 | 备注 |
|
||||
| # | 浠诲姟 | 浼樺厛绾?| 鐘舵€?| 寮€濮嬫椂闂?| 瀹屾垚鏃堕棿 | 鑰楁椂 | 璐熻矗浜?| 澶囨敞 |
|
||||
|---|------|--------|------|----------|----------|------|--------|------|
|
||||
| 2.1 | 创建PostgresCacheAdapter.ts | P0 | ⬜ | _____ | _____ | _____ | _____ | get/set/delete方法 |
|
||||
| 2.2 | 实现缓存清理函数 | P0 | ⬜ | _____ | _____ | _____ | _____ | startCacheCleanupTask |
|
||||
| 2.3 | 更新CacheFactory | P0 | ⬜ | _____ | _____ | _____ | _____ | 支持postgres选项 |
|
||||
| 2.4 | 更新cache/index.ts导出 | P0 | ⬜ | _____ | _____ | _____ | _____ | 导出新类和函数 |
|
||||
| 2.5 | 编写PostgresCache单元测试 | P0 | ⬜ | _____ | _____ | _____ | _____ | 测试覆盖率>80% |
|
||||
| 2.1 | 鍒涘缓PostgresCacheAdapter.ts | P0 | 猬?| _____ | _____ | _____ | _____ | get/set/delete鏂规硶 |
|
||||
| 2.2 | 瀹炵幇缂撳瓨娓呯悊鍑芥暟 | P0 | 猬?| _____ | _____ | _____ | _____ | startCacheCleanupTask |
|
||||
| 2.3 | 鏇存柊CacheFactory | P0 | 猬?| _____ | _____ | _____ | _____ | 鏀<EFBFBD>寔postgres閫夐」 |
|
||||
| 2.4 | 鏇存柊cache/index.ts瀵煎嚭 | P0 | 猬?| _____ | _____ | _____ | _____ | 瀵煎嚭鏂扮被鍜屽嚱鏁?|
|
||||
| 2.5 | 缂栧啓PostgresCache鍗曞厓娴嬭瘯 | P0 | 猬?| _____ | _____ | _____ | _____ | 娴嬭瘯瑕嗙洊鐜?80% |
|
||||
|
||||
**验收标准:**
|
||||
- [ ] 所有单元测试通过(`npm test`)
|
||||
- [ ] 缓存读写功能正常
|
||||
**楠屾敹鏍囧噯锛?*
|
||||
- [ ] 鎵€鏈夊崟鍏冩祴璇曢€氳繃锛坄npm test`锛?- [ ] 缂撳瓨璇诲啓鍔熻兘姝e父
|
||||
- [ ] 杩囨湡娓呯悊鍔熻兘姝e父锛堟瘡鍒嗛挓1000鏉★級
|
||||
- [ ] 鏈<>湴鐜<E6B9B4><E9909C>CACHE_TYPE=postgres姝e父杩愯<E69DA9>
|
||||
|
||||
**代码位置:**
|
||||
- `backend/src/common/cache/PostgresCacheAdapter.ts` (~300行)
|
||||
- `backend/src/common/cache/CacheFactory.ts` (+10行)
|
||||
**浠g爜浣嶇疆锛?*
|
||||
- `backend/src/common/cache/PostgresCacheAdapter.ts` (~300琛?
|
||||
- `backend/src/common/cache/CacheFactory.ts` (+10琛?
|
||||
- `backend/tests/common/cache/PostgresCacheAdapter.test.ts` (鏂板缓)
|
||||
|
||||
**閬囧埌鐨勯棶棰橈細**
|
||||
```
|
||||
问题1:
|
||||
描述:
|
||||
解决方案:
|
||||
解决时间:
|
||||
```
|
||||
闂<EFBFBD><EFBFBD>1锛?鎻忚堪锛?瑙e喅鏂规<E98F82>锛?瑙e喅鏃堕棿锛?```
|
||||
|
||||
---
|
||||
|
||||
### Phase 3:实现PgBossQueue(2天)
|
||||
### Phase 3锛氬疄鐜癙gBossQueue锛?澶╋級
|
||||
|
||||
| # | 任务 | 优先级 | 状态 | 开始时间 | 完成时间 | 耗时 | 负责人 | 备注 |
|
||||
| # | 浠诲姟 | 浼樺厛绾?| 鐘舵€?| 寮€濮嬫椂闂?| 瀹屾垚鏃堕棿 | 鑰楁椂 | 璐熻矗浜?| 澶囨敞 |
|
||||
|---|------|--------|------|----------|----------|------|--------|------|
|
||||
| 3.1 | 创建PgBossQueue.ts | P0 | ⬜ | _____ | _____ | _____ | _____ | push/process/getJob方法 |
|
||||
| 3.2 | 实现任务状态映射和错误处理 | P0 | ⬜ | _____ | _____ | _____ | _____ | mapState + 重试逻辑 |
|
||||
| 3.3 | 更新JobFactory | P0 | ⬜ | _____ | _____ | _____ | _____ | 支持pgboss选项 |
|
||||
| 3.4 | 更新jobs/index.ts导出 | P0 | ⬜ | _____ | _____ | _____ | _____ | 导出新类 |
|
||||
| 3.5 | 编写PgBossQueue单元测试 | P0 | ⬜ | _____ | _____ | _____ | _____ | 测试覆盖率>80% |
|
||||
| 3.1 | 鍒涘缓PgBossQueue.ts | P0 | 猬?| _____ | _____ | _____ | _____ | push/process/getJob鏂规硶 |
|
||||
| 3.2 | 瀹炵幇浠诲姟鐘舵€佹槧灏勫拰閿欒<EFBFBD>澶勭悊 | P0 | 猬?| _____ | _____ | _____ | _____ | mapState + 閲嶈瘯閫昏緫 |
|
||||
| 3.3 | 鏇存柊JobFactory | P0 | 猬?| _____ | _____ | _____ | _____ | 鏀<EFBFBD>寔pgboss閫夐」 |
|
||||
| 3.4 | 鏇存柊jobs/index.ts瀵煎嚭 | P0 | 猬?| _____ | _____ | _____ | _____ | 瀵煎嚭鏂扮被 |
|
||||
| 3.5 | 缂栧啓PgBossQueue鍗曞厓娴嬭瘯 | P0 | 猬?| _____ | _____ | _____ | _____ | 娴嬭瘯瑕嗙洊鐜?80% |
|
||||
|
||||
**验收标准:**
|
||||
**楠屾敹鏍囧噯锛?*
|
||||
- [ ] 鎵€鏈夊崟鍏冩祴璇曢€氳繃
|
||||
- [ ] 浠诲姟鍏ラ槦鍔熻兘姝e父
|
||||
- [ ] Worker娉ㄥ唽鍔熻兘姝e父
|
||||
- [ ] 任务重试功能正常(失败3次)
|
||||
- [ ] 浠诲姟閲嶈瘯鍔熻兘姝e父锛堝け璐?娆★級
|
||||
- [ ] 鏈<>湴鐜<E6B9B4><E9909C>QUEUE_TYPE=pgboss姝e父杩愯<E69DA9>
|
||||
- [ ] pg-boss鑷<73>姩鍒涘缓琛<E7BC93>紙platform_schema.job绛夛級
|
||||
|
||||
**代码位置:**
|
||||
- `backend/src/common/jobs/PgBossQueue.ts` (~400行)
|
||||
- `backend/src/common/jobs/JobFactory.ts` (+10行)
|
||||
**浠g爜浣嶇疆锛?*
|
||||
- `backend/src/common/jobs/PgBossQueue.ts` (~400琛?
|
||||
- `backend/src/common/jobs/JobFactory.ts` (+10琛?
|
||||
- `backend/tests/common/jobs/PgBossQueue.test.ts` (鏂板缓)
|
||||
|
||||
**閬囧埌鐨勯棶棰橈細**
|
||||
```
|
||||
问题1:
|
||||
描述:
|
||||
解决方案:
|
||||
解决时间:
|
||||
```
|
||||
闂<EFBFBD><EFBFBD>1锛?鎻忚堪锛?瑙e喅鏂规<E98F82>锛?瑙e喅鏃堕棿锛?```
|
||||
|
||||
---
|
||||
|
||||
### Phase 4锛氬疄鐜颁换鍔℃媶鍒嗘満鍒讹紙1澶╋級
|
||||
|
||||
| # | 任务 | 优先级 | 状态 | 开始时间 | 完成时间 | 耗时 | 负责人 | 备注 |
|
||||
| # | 浠诲姟 | 浼樺厛绾?| 鐘舵€?| 寮€濮嬫椂闂?| 瀹屾垚鏃堕棿 | 鑰楁椂 | 璐熻矗浜?| 澶囨敞 |
|
||||
|---|------|--------|------|----------|----------|------|--------|------|
|
||||
| 4.1 | 创建jobs/utils.ts | P0 | ⬜ | _____ | _____ | _____ | _____ | 拆分工具函数 |
|
||||
| 4.2 | 实现splitIntoChunks和recommendChunkSize | P0 | ⬜ | _____ | _____ | _____ | _____ | 核心拆分逻辑 |
|
||||
| 4.3 | 定义CHUNK_STRATEGIES配置 | P0 | ⬜ | _____ | _____ | _____ | _____ | ASL/DC/SSA策略 |
|
||||
| 4.4 | 编写任务拆分单元测试 | P0 | ⬜ | _____ | _____ | _____ | _____ | 测试覆盖率>90% |
|
||||
| 4.1 | 鍒涘缓jobs/utils.ts | P0 | 猬?| _____ | _____ | _____ | _____ | 鎷嗗垎宸ュ叿鍑芥暟 |
|
||||
| 4.2 | 瀹炵幇splitIntoChunks鍜宺ecommendChunkSize | P0 | 猬?| _____ | _____ | _____ | _____ | 鏍稿績鎷嗗垎閫昏緫 |
|
||||
| 4.3 | 瀹氫箟CHUNK_STRATEGIES閰嶇疆 | P0 | 猬?| _____ | _____ | _____ | _____ | ASL/DC/SSA绛栫暐 |
|
||||
| 4.4 | 缂栧啓浠诲姟鎷嗗垎鍗曞厓娴嬭瘯 | P0 | 猬?| _____ | _____ | _____ | _____ | 娴嬭瘯瑕嗙洊鐜?90% |
|
||||
|
||||
**验收标准:**
|
||||
**楠屾敹鏍囧噯锛?*
|
||||
- [ ] 鎵€鏈夊崟鍏冩祴璇曢€氳繃
|
||||
- [ ] splitIntoChunks鍔熻兘姝g‘
|
||||
- [ ] recommendChunkSize璁$畻鍑嗙‘
|
||||
- [ ] CHUNK_STRATEGIES閰嶇疆鍚堢悊
|
||||
|
||||
**代码位置:**
|
||||
- `backend/src/common/jobs/utils.ts` (~200行)
|
||||
**浠g爜浣嶇疆锛?*
|
||||
- `backend/src/common/jobs/utils.ts` (~200琛?
|
||||
- `backend/tests/common/jobs/utils.test.ts` (鏂板缓)
|
||||
|
||||
**测试案例:**
|
||||
**娴嬭瘯妗堜緥锛?*
|
||||
```typescript
|
||||
// 娴嬭瘯鎷嗗垎
|
||||
splitIntoChunks([1..100], 30) → [[1..30], [31..60], [61..90], [91..100]]
|
||||
splitIntoChunks([1..100], 30) 鈫?[[1..30], [31..60], [61..90], [91..100]]
|
||||
|
||||
// 娴嬭瘯鎺ㄨ崘
|
||||
recommendChunkSize(1000, 7.2, 900) → 125
|
||||
recommendChunkSize(1000, 7.2, 900) 鈫?125
|
||||
```
|
||||
|
||||
**閬囧埌鐨勯棶棰橈細**
|
||||
```
|
||||
问题1:
|
||||
描述:
|
||||
解决方案:
|
||||
解决时间:
|
||||
```
|
||||
闂<EFBFBD><EFBFBD>1锛?鎻忚堪锛?瑙e喅鏂规<E98F82>锛?瑙e喅鏃堕棿锛?```
|
||||
|
||||
---
|
||||
|
||||
### Phase 5锛氬疄鐜版柇鐐圭画浼犳満鍒讹紙1澶╋級
|
||||
|
||||
| # | 任务 | 优先级 | 状态 | 开始时间 | 完成时间 | 耗时 | 负责人 | 备注 |
|
||||
| # | 浠诲姟 | 浼樺厛绾?| 鐘舵€?| 寮€濮嬫椂闂?| 瀹屾垚鏃堕棿 | 鑰楁椂 | 璐熻矗浜?| 澶囨敞 |
|
||||
|---|------|--------|------|----------|----------|------|--------|------|
|
||||
| 5.1 | 更新AslScreeningTask Schema | P0 | ⬜ | _____ | _____ | _____ | _____ | 新增6个断点字段 |
|
||||
| 5.2 | 执行数据库迁移 | P0 | ⬜ | _____ | _____ | _____ | _____ | `npx prisma migrate dev` |
|
||||
| 5.3 | 创建CheckpointService.ts | P0 | ⬜ | _____ | _____ | _____ | _____ | 保存/读取/恢复断点 |
|
||||
| 5.4 | 编写断点续传单元测试 | P0 | ⬜ | _____ | _____ | _____ | _____ | 测试覆盖率>80% |
|
||||
| 5.1 | 鏇存柊AslScreeningTask Schema | P0 | 猬?| _____ | _____ | _____ | _____ | 鏂板<EFBFBD>6涓<EFBFBD>柇鐐瑰瓧娈?|
|
||||
| 5.2 | 鎵ц<EFBFBD>鏁版嵁搴撹縼绉?| P0 | 猬?| _____ | _____ | _____ | _____ | `npx prisma migrate dev` |
|
||||
| 5.3 | 鍒涘缓CheckpointService.ts | P0 | 猬?| _____ | _____ | _____ | _____ | 淇濆瓨/璇诲彇/鎭㈠<E98EAD>鏂<EFBFBD>偣 |
|
||||
| 5.4 | 缂栧啓鏂<EFBFBD>偣缁<EFBFBD>紶鍗曞厓娴嬭瘯 | P0 | 猬?| _____ | _____ | _____ | _____ | 娴嬭瘯瑕嗙洊鐜?80% |
|
||||
|
||||
**验收标准:**
|
||||
- [ ] 数据库字段新增成功
|
||||
- [ ] 所有单元测试通过
|
||||
**楠屾敹鏍囧噯锛?*
|
||||
- [ ] 鏁版嵁搴撳瓧娈垫柊澧炴垚鍔?- [ ] 鎵€鏈夊崟鍏冩祴璇曢€氳繃
|
||||
- [ ] saveCheckpoint鍔熻兘姝e父
|
||||
- [ ] loadCheckpoint鍔熻兘姝e父
|
||||
- [ ] updateProgress鍔熻兘姝e父
|
||||
|
||||
**代码位置:**
|
||||
- `backend/prisma/schema.prisma` (+40行)
|
||||
- `backend/src/common/jobs/CheckpointService.ts` (~150行)
|
||||
**浠g爜浣嶇疆锛?*
|
||||
- `backend/prisma/schema.prisma` (+40琛?
|
||||
- `backend/src/common/jobs/CheckpointService.ts` (~150琛?
|
||||
- `backend/tests/common/jobs/CheckpointService.test.ts` (鏂板缓)
|
||||
|
||||
**新增字段:**
|
||||
**鏂板<E98F82>瀛楁<E7809B>锛?*
|
||||
```prisma
|
||||
totalBatches Int
|
||||
processedBatches Int
|
||||
@@ -206,87 +180,73 @@ checkpointData Json?
|
||||
|
||||
**閬囧埌鐨勯棶棰橈細**
|
||||
```
|
||||
问题1:
|
||||
描述:
|
||||
解决方案:
|
||||
解决时间:
|
||||
```
|
||||
闂<EFBFBD><EFBFBD>1锛?鎻忚堪锛?瑙e喅鏂规<E98F82>锛?瑙e喅鏃堕棿锛?```
|
||||
|
||||
---
|
||||
|
||||
### Phase 6锛氭敼閫燗SL绛涢€夋湇鍔★紙1.5澶╋級
|
||||
|
||||
| # | 任务 | 优先级 | 状态 | 开始时间 | 完成时间 | 耗时 | 负责人 | 备注 |
|
||||
| # | 浠诲姟 | 浼樺厛绾?| 鐘舵€?| 寮€濮嬫椂闂?| 瀹屾垚鏃堕棿 | 鑰楁椂 | 璐熻矗浜?| 澶囨敞 |
|
||||
|---|------|--------|------|----------|----------|------|--------|------|
|
||||
| 6.1 | 改造startScreeningTask | P0 | ⬜ | _____ | _____ | _____ | _____ | 使用任务拆分 |
|
||||
| 6.2 | 实现批次Worker | P0 | ⬜ | _____ | _____ | _____ | _____ | 带断点续传 |
|
||||
| 6.3 | 更新index.ts注册Workers | P0 | ⬜ | _____ | _____ | _____ | _____ | 启动时注册 |
|
||||
| 6.4 | 本地测试:100篇文献筛选 | P0 | ⬜ | _____ | _____ | _____ | _____ | 验证完整流程 |
|
||||
| 6.1 | 鏀归€爏tartScreeningTask | P0 | 猬?| _____ | _____ | _____ | _____ | 浣跨敤浠诲姟鎷嗗垎 |
|
||||
| 6.2 | 瀹炵幇鎵规<EFBFBD>Worker | P0 | 猬?| _____ | _____ | _____ | _____ | 甯︽柇鐐圭画浼?|
|
||||
| 6.3 | 鏇存柊index.ts娉ㄥ唽Workers | P0 | 猬?| _____ | _____ | _____ | _____ | 鍚<EFBFBD>姩鏃舵敞鍐?|
|
||||
| 6.4 | 鏈<EFBFBD>湴娴嬭瘯锛?00绡囨枃鐚<E69E83>瓫閫?| P0 | 猬?| _____ | _____ | _____ | _____ | 楠岃瘉瀹屾暣娴佺▼ |
|
||||
|
||||
**验收标准:**
|
||||
- [ ] 100篇文献筛选成功(拆分成2批)
|
||||
**楠屾敹鏍囧噯锛?*
|
||||
- [ ] 100绡囨枃鐚<EFBFBD>瓫閫夋垚鍔燂紙鎷嗗垎鎴?鎵癸級
|
||||
- [ ] 鎵规<E98EB5>浠诲姟鍏ラ槦姝e父
|
||||
- [ ] Worker澶勭悊鎵规<E98EB5>姝e父
|
||||
- [ ] 杩涘害鏇存柊姝e父锛堟瘡10绡囷級
|
||||
- [ ] 鏂<>偣淇濆瓨姝e父
|
||||
|
||||
**代码位置:**
|
||||
- `backend/src/modules/asl/services/screeningService.ts` (~200行改动)
|
||||
- `backend/src/index.ts` (+20行)
|
||||
**浠g爜浣嶇疆锛?*
|
||||
- `backend/src/modules/asl/services/screeningService.ts` (~200琛屾敼鍔?
|
||||
- `backend/src/index.ts` (+20琛?
|
||||
|
||||
**测试流程:**
|
||||
**娴嬭瘯娴佺▼锛?*
|
||||
```bash
|
||||
1. 准备100篇测试文献
|
||||
2. 提交筛选任务
|
||||
3. 观察日志:应该看到2批任务
|
||||
4. 验证数据库:totalBatches=2
|
||||
1. 鍑嗗<EFBFBD>100绡囨祴璇曟枃鐚?2. 鎻愪氦绛涢€変换鍔?3. 瑙傚療鏃ュ織锛氬簲璇ョ湅鍒?鎵逛换鍔?4. 楠岃瘉鏁版嵁搴擄細totalBatches=2
|
||||
5. 绛夊緟瀹屾垚锛歱rocessedBatches=2
|
||||
6. 验证结果:100篇都有结果
|
||||
```
|
||||
6. 楠岃瘉缁撴灉锛?00绡囬兘鏈夌粨鏋?```
|
||||
|
||||
**閬囧埌鐨勯棶棰橈細**
|
||||
```
|
||||
问题1:
|
||||
描述:
|
||||
解决方案:
|
||||
解决时间:
|
||||
```
|
||||
闂<EFBFBD><EFBFBD>1锛?鎻忚堪锛?瑙e喅鏂规<E98F82>锛?瑙e喅鏃堕棿锛?```
|
||||
|
||||
---
|
||||
|
||||
### Phase 7锛氭敼閫燚C鎻愬彇鏈嶅姟锛堝彲閫夛級
|
||||
|
||||
| # | 任务 | 优先级 | 状态 | 开始时间 | 完成时间 | 耗时 | 负责人 | 备注 |
|
||||
| # | 浠诲姟 | 浼樺厛绾?| 鐘舵€?| 寮€濮嬫椂闂?| 瀹屾垚鏃堕棿 | 鑰楁椂 | 璐熻矗浜?| 澶囨敞 |
|
||||
|---|------|--------|------|----------|----------|------|--------|------|
|
||||
| 7.1 | 参考ASL改造DC服务 | P1 | ⬜ | _____ | _____ | _____ | _____ | 按需实施 |
|
||||
|
||||
**说明:** 此Phase可根据实际需求决定是否实施。建议先完成ASL改造并验证稳定后再考虑。
|
||||
| 7.1 | 鍙傝€傾SL鏀归€燚C鏈嶅姟 | P1 | 猬?| _____ | _____ | _____ | _____ | 鎸夐渶瀹炴柦 |
|
||||
|
||||
**璇存槑锛?* 姝<>hase鍙<65>牴鎹<E789B4>疄闄呴渶姹傚喅瀹氭槸鍚﹀疄鏂姐€傚缓璁<E7BC93>厛瀹屾垚ASL鏀归€犲苟楠岃瘉绋冲畾鍚庡啀鑰冭檻銆?
|
||||
---
|
||||
|
||||
### Phase 8锛氬叏闈㈡祴璇曢獙璇侊紙1.5澶╋級
|
||||
|
||||
| # | 任务 | 优先级 | 状态 | 开始时间 | 完成时间 | 耗时 | 负责人 | 备注 |
|
||||
| # | 浠诲姟 | 浼樺厛绾?| 鐘舵€?| 寮€濮嬫椂闂?| 瀹屾垚鏃堕棿 | 鑰楁椂 | 璐熻矗浜?| 澶囨敞 |
|
||||
|---|------|--------|------|----------|----------|------|--------|------|
|
||||
| 8.1 | 功能测试:缓存读写和过期清理 | P0 | ⬜ | _____ | _____ | _____ | _____ | 基础功能验证 |
|
||||
| 8.2 | 任务拆分测试:验证批次正确性 | P0 | ⬜ | _____ | _____ | _____ | _____ | 1000篇→10批 |
|
||||
| 8.3 | 断点续传测试:中断恢复验证 | P0 | ⬜ | _____ | _____ | _____ | _____ | Ctrl+C后恢复 |
|
||||
| 8.4 | 长任务测试:1000篇文献完整流程 | P0 | ⬜ | _____ | _____ | _____ | _____ | 2小时任务 |
|
||||
| 8.5 | 实例重启测试:关键恢复测试 | P0 | ⬜ | _____ | _____ | _____ | _____ | 50%中断恢复 |
|
||||
| 8.6 | 并发测试:多Worker并行处理 | P0 | ⬜ | _____ | _____ | _____ | _____ | 3个实例测试 |
|
||||
| 8.7 | 性能测试:缓存和队列延迟 | P1 | ⬜ | _____ | _____ | _____ | _____ | P99延迟 |
|
||||
| 8.1 | 鍔熻兘娴嬭瘯锛氱紦瀛樿<EFBFBD>鍐欏拰杩囨湡娓呯悊 | P0 | 猬?| _____ | _____ | _____ | _____ | 鍩虹<EFBFBD>鍔熻兘楠岃瘉 |
|
||||
| 8.2 | 浠诲姟鎷嗗垎娴嬭瘯锛氶獙璇佹壒娆℃<EFBFBD>纭<EFBFBD>€?| P0 | 猬?| _____ | _____ | _____ | _____ | 1000绡団啋10鎵?|
|
||||
| 8.3 | 鏂<EFBFBD>偣缁<EFBFBD>紶娴嬭瘯锛氫腑鏂<EFBFBD>仮澶嶉獙璇?| P0 | 猬?| _____ | _____ | _____ | _____ | Ctrl+C鍚庢仮澶?|
|
||||
| 8.4 | 闀夸换鍔℃祴璇曪細1000绡囨枃鐚<EFBFBD>畬鏁存祦绋?| P0 | 猬?| _____ | _____ | _____ | _____ | 2灏忔椂浠诲姟 |
|
||||
| 8.5 | 瀹炰緥閲嶅惎娴嬭瘯锛氬叧閿<EFBFBD>仮澶嶆祴璇?| P0 | 猬?| _____ | _____ | _____ | _____ | 50%涓<EFBFBD>柇鎭㈠<EFBFBD> |
|
||||
| 8.6 | 骞跺彂娴嬭瘯锛氬<EFBFBD>Worker骞惰<EFBFBD>澶勭悊 | P0 | 猬?| _____ | _____ | _____ | _____ | 3涓<EFBFBD>疄渚嬫祴璇?|
|
||||
| 8.7 | 鎬ц兘娴嬭瘯锛氱紦瀛樺拰闃熷垪寤惰繜 | P1 | 猬?| _____ | _____ | _____ | _____ | P99寤惰繜 |
|
||||
|
||||
**验收标准:**
|
||||
**楠屾敹鏍囧噯锛?*
|
||||
- [ ] 鎵€鏈夊姛鑳芥祴璇曢€氳繃
|
||||
- [ ] 1000绡囨枃鐚<E69E83>瓫閫夋垚鍔熺巼 > 99%
|
||||
- [ ] 实例重启恢复成功(至少3次)
|
||||
- [ ] 断点续传不重复处理
|
||||
- [ ] 缓存命中率 > 60%
|
||||
- [ ] 队列吞吐量 > 100任务/小时
|
||||
- [ ] 瀹炰緥閲嶅惎鎭㈠<E98EAD>鎴愬姛锛堣嚦灏?娆★級
|
||||
- [ ] 鏂<>偣缁<E581A3>紶涓嶉噸澶嶅<E6BEB6>鐞?- [ ] 缂撳瓨鍛戒腑鐜?> 60%
|
||||
- [ ] 闃熷垪鍚炲悙閲?> 100浠诲姟/灏忔椂
|
||||
|
||||
**测试记录:**
|
||||
**娴嬭瘯璁板綍锛?*
|
||||
|
||||
**测试1:1000篇文献筛选**
|
||||
**娴嬭瘯1锛?000绡囨枃鐚<E69E83>瓫閫?*
|
||||
- 寮€濮嬫椂闂达細_____
|
||||
- 缁撴潫鏃堕棿锛歘____
|
||||
- 鎬昏€楁椂锛歘____
|
||||
@@ -294,14 +254,13 @@ checkpointData Json?
|
||||
- 鎵规<E98EB5>鏁帮細_____
|
||||
- 澶辫触鎵规<E98EB5>锛歘____
|
||||
|
||||
**测试2:实例重启恢复**
|
||||
**娴嬭瘯2锛氬疄渚嬮噸鍚<EFBFBD>仮澶?*
|
||||
- 娴嬭瘯娆℃暟锛歘____
|
||||
- 鎴愬姛娆℃暟锛歘____
|
||||
- 鎴愬姛鐜囷細_____%
|
||||
- 鏂<>偣鎭㈠<E98EAD>浣嶇疆锛歘____
|
||||
- 是否重复处理:是 / 否
|
||||
|
||||
**测试3:并发处理**
|
||||
- 鏄<>惁閲嶅<E996B2>澶勭悊锛氭槸 / 鍚?
|
||||
**娴嬭瘯3锛氬苟鍙戝<E98D99>鐞?*
|
||||
- Worker鏁伴噺锛歘____
|
||||
- 鎬讳换鍔℃暟锛歘____
|
||||
- 鎬昏€楁椂锛歘____
|
||||
@@ -310,34 +269,24 @@ checkpointData Json?
|
||||
|
||||
**閬囧埌鐨勯棶棰橈細**
|
||||
```
|
||||
问题1:
|
||||
描述:
|
||||
解决方案:
|
||||
解决时间:
|
||||
```
|
||||
闂<EFBFBD><EFBFBD>1锛?鎻忚堪锛?瑙e喅鏂规<E98F82>锛?瑙e喅鏃堕棿锛?```
|
||||
|
||||
---
|
||||
|
||||
### Phase 9:SAE部署上线(0.5天)
|
||||
### Phase 9锛歋AE閮ㄧ讲涓婄嚎锛?.5澶╋級
|
||||
|
||||
| # | 任务 | 优先级 | 状态 | 开始时间 | 完成时间 | 耗时 | 负责人 | 备注 |
|
||||
| # | 浠诲姟 | 浼樺厛绾?| 鐘舵€?| 寮€濮嬫椂闂?| 瀹屾垚鏃堕棿 | 鑰楁椂 | 璐熻矗浜?| 澶囨敞 |
|
||||
|---|------|--------|------|----------|----------|------|--------|------|
|
||||
| 9.1 | 配置SAE环境变量 | P0 | ⬜ | _____ | _____ | _____ | _____ | CACHE_TYPE=postgres等 |
|
||||
| 9.2 | 配置SAE弹性伸缩 | P0 | ⬜ | _____ | _____ | _____ | _____ | 1-5实例 |
|
||||
| 9.3 | 灰度发布 | P0 | ⬜ | _____ | _____ | _____ | _____ | 1个实例,观察24小时 |
|
||||
| 9.4 | 全量发布 | P0 | ⬜ | _____ | _____ | _____ | _____ | 扩容到2-3实例 |
|
||||
| 9.5 | 生产验证 | P0 | ⬜ | _____ | _____ | _____ | _____ | 监控48小时无错误 |
|
||||
| 9.1 | 閰嶇疆SAE鐜<EFBFBD><EFBFBD>鍙橀噺 | P0 | 猬?| _____ | _____ | _____ | _____ | CACHE_TYPE=postgres绛?|
|
||||
| 9.2 | 閰嶇疆SAE寮规€т几缂?| P0 | 猬?| _____ | _____ | _____ | _____ | 1-5瀹炰緥 |
|
||||
| 9.3 | 鐏板害鍙戝竷 | P0 | 猬?| _____ | _____ | _____ | _____ | 1涓<EFBFBD>疄渚嬶紝瑙傚療24灏忔椂 |
|
||||
| 9.4 | 鍏ㄩ噺鍙戝竷 | P0 | 猬?| _____ | _____ | _____ | _____ | 鎵╁<EFBFBD>鍒?-3瀹炰緥 |
|
||||
| 9.5 | 鐢熶骇楠岃瘉 | P0 | 猬?| _____ | _____ | _____ | _____ | 鐩戞帶48灏忔椂鏃犻敊璇?|
|
||||
|
||||
**验收标准:**
|
||||
**楠屾敹鏍囧噯锛?*
|
||||
- [ ] 鐜<><E9909C>鍙橀噺閰嶇疆姝g‘
|
||||
- [ ] 弹性伸缩配置正确
|
||||
- [ ] 灰度发布24小时无错误
|
||||
- [ ] 全量发布48小时无错误
|
||||
- [ ] 至少3个真实用户任务成功
|
||||
- [ ] 至少1次实例重启恢复成功
|
||||
- [ ] 无用户投诉
|
||||
|
||||
**环境变量清单:**
|
||||
- [ ] 寮规€т几缂╅厤缃<E58EA4><E7BC83>纭?- [ ] 鐏板害鍙戝竷24灏忔椂鏃犻敊璇?- [ ] 鍏ㄩ噺鍙戝竷48灏忔椂鏃犻敊璇?- [ ] 鑷冲皯3涓<33>湡瀹炵敤鎴蜂换鍔℃垚鍔?- [ ] 鑷冲皯1娆″疄渚嬮噸鍚<E599B8>仮澶嶆垚鍔?- [ ] 鏃犵敤鎴锋姇璇?
|
||||
**鐜<><E9909C>鍙橀噺娓呭崟锛?*
|
||||
```bash
|
||||
CACHE_TYPE=postgres
|
||||
QUEUE_TYPE=pgboss
|
||||
@@ -345,7 +294,7 @@ DATABASE_URL=postgresql://...
|
||||
NODE_ENV=production
|
||||
```
|
||||
|
||||
**SAE配置:**
|
||||
**SAE閰嶇疆锛?*
|
||||
```yaml
|
||||
replicas:
|
||||
min: 1
|
||||
@@ -356,7 +305,7 @@ autoScaling:
|
||||
memory: 70%
|
||||
```
|
||||
|
||||
**生产监控(48小时):**
|
||||
**鐢熶骇鐩戞帶锛?8灏忔椂锛夛細**
|
||||
- 缂撳瓨鍛戒腑鐜囷細_____%
|
||||
- LLM API璋冪敤閲忥細_____ (涓嬮檷___%)
|
||||
- 浠诲姟鎴愬姛鐜囷細_____%
|
||||
@@ -367,42 +316,36 @@ autoScaling:
|
||||
|
||||
**閬囧埌鐨勯棶棰橈細**
|
||||
```
|
||||
问题1:
|
||||
描述:
|
||||
解决方案:
|
||||
解决时间:
|
||||
```
|
||||
闂<EFBFBD><EFBFBD>1锛?鎻忚堪锛?瑙e喅鏂规<E98F82>锛?瑙e喅鏃堕棿锛?```
|
||||
|
||||
---
|
||||
|
||||
## 🎯 关键里程碑
|
||||
|
||||
| # | 里程碑 | 目标日期 | 实际日期 | 状态 | 备注 |
|
||||
## 馃幆 鍏抽敭閲岀▼纰?
|
||||
| # | 閲岀▼纰?| 鐩<>爣鏃ユ湡 | 瀹為檯鏃ユ湡 | 鐘舵€?| 澶囨敞 |
|
||||
|---|--------|----------|----------|------|------|
|
||||
| M1 | 环境准备完成 | Day 1 | _____ | ⬜ | pg-boss安装,数据库迁移 |
|
||||
| M2 | 缓存系统完成 | Day 1 | _____ | ⬜ | PostgresCacheAdapter测试通过 |
|
||||
| M3 | 队列系统完成 | Day 3 | _____ | ⬜ | PgBossQueue测试通过 |
|
||||
| M4 | 高级特性完成 | Day 5 | _____ | ⬜ | 拆分+断点机制完成 |
|
||||
| M5 | 业务集成完成 | Day 7 | _____ | ⬜ | ASL改造完成,100篇测试通过 |
|
||||
| M6 | 全面测试完成 | Day 8 | _____ | ⬜ | 1000篇测试通过 |
|
||||
| M7 | 生产上线完成 | Day 9 | _____ | ⬜ | 48小时验证通过 |
|
||||
| M1 | 鐜<EFBFBD><EFBFBD>鍑嗗<EFBFBD>瀹屾垚 | Day 1 | _____ | 猬?| pg-boss瀹夎<EFBFBD>锛屾暟鎹<EFBFBD>簱杩佺Щ |
|
||||
| M2 | 缂撳瓨绯荤粺瀹屾垚 | Day 1 | _____ | 猬?| PostgresCacheAdapter娴嬭瘯閫氳繃 |
|
||||
| M3 | 闃熷垪绯荤粺瀹屾垚 | Day 3 | _____ | 猬?| PgBossQueue娴嬭瘯閫氳繃 |
|
||||
| M4 | 楂樼骇鐗规€у畬鎴?| Day 5 | _____ | 猬?| 鎷嗗垎+鏂<>偣鏈哄埗瀹屾垚 |
|
||||
| M5 | 涓氬姟闆嗘垚瀹屾垚 | Day 7 | _____ | 猬?| ASL鏀归€犲畬鎴愶紝100绡囨祴璇曢€氳繃 |
|
||||
| M6 | 鍏ㄩ潰娴嬭瘯瀹屾垚 | Day 8 | _____ | 猬?| 1000绡囨祴璇曢€氳繃 |
|
||||
| M7 | 鐢熶骇涓婄嚎瀹屾垚 | Day 9 | _____ | 猬?| 48灏忔椂楠岃瘉閫氳繃 |
|
||||
|
||||
---
|
||||
|
||||
## 馃搱 姣忔棩杩涘害璁板綍
|
||||
|
||||
### Day 1(___月___日,周___)
|
||||
|
||||
**计划任务:**
|
||||
### Day 1锛坃__鏈坃__鏃ワ紝鍛╛__锛?
|
||||
**璁″垝浠诲姟锛?*
|
||||
- [ ] Phase 1: 鐜<><E9909C>鍑嗗<E98D91>
|
||||
- [ ] Phase 2: PostgresCacheAdapter
|
||||
|
||||
**实际完成:**
|
||||
**瀹為檯瀹屾垚锛?*
|
||||
-
|
||||
|
||||
**工作时长:** ___小时
|
||||
**宸ヤ綔鏃堕暱锛?* ___灏忔椂
|
||||
|
||||
**完成质量:** 🟢 优秀 / 🟡 良好 / 🔴 需改进
|
||||
**瀹屾垚璐ㄩ噺锛?* 馃煝 浼樼<E6B5BC> / 馃煛 鑹<>ソ / 馃敶 闇€鏀硅繘
|
||||
|
||||
**閬囧埌鐨勪富瑕侀棶棰橈細**
|
||||
1.
|
||||
@@ -410,22 +353,21 @@ autoScaling:
|
||||
**瀛﹀埌鐨勭粡楠岋細**
|
||||
1.
|
||||
|
||||
**明天计划:**
|
||||
**鏄庡ぉ璁″垝锛?*
|
||||
-
|
||||
|
||||
---
|
||||
|
||||
### Day 2(___月___日,周___)
|
||||
|
||||
**计划任务:**
|
||||
### Day 2锛坃__鏈坃__鏃ワ紝鍛╛__锛?
|
||||
**璁″垝浠诲姟锛?*
|
||||
- [ ] Phase 3: PgBossQueue锛堝紑濮嬶級
|
||||
|
||||
**实际完成:**
|
||||
**瀹為檯瀹屾垚锛?*
|
||||
-
|
||||
|
||||
**工作时长:** ___小时
|
||||
**宸ヤ綔鏃堕暱锛?* ___灏忔椂
|
||||
|
||||
**完成质量:** 🟢 优秀 / 🟡 良好 / 🔴 需改进
|
||||
**瀹屾垚璐ㄩ噺锛?* 馃煝 浼樼<E6B5BC> / 馃煛 鑹<>ソ / 馃敶 闇€鏀硅繘
|
||||
|
||||
**閬囧埌鐨勪富瑕侀棶棰橈細**
|
||||
1.
|
||||
@@ -433,23 +375,22 @@ autoScaling:
|
||||
**瀛﹀埌鐨勭粡楠岋細**
|
||||
1.
|
||||
|
||||
**明天计划:**
|
||||
**鏄庡ぉ璁″垝锛?*
|
||||
-
|
||||
|
||||
---
|
||||
|
||||
### Day 3(___月___日,周___)
|
||||
|
||||
**计划任务:**
|
||||
### Day 3锛坃__鏈坃__鏃ワ紝鍛╛__锛?
|
||||
**璁″垝浠诲姟锛?*
|
||||
- [ ] Phase 3: PgBossQueue锛堝畬鎴愶級
|
||||
- [ ] 娴嬭瘯楠岃瘉Phase 2-3
|
||||
|
||||
**实际完成:**
|
||||
**瀹為檯瀹屾垚锛?*
|
||||
-
|
||||
|
||||
**工作时长:** ___小时
|
||||
**宸ヤ綔鏃堕暱锛?* ___灏忔椂
|
||||
|
||||
**完成质量:** 🟢 优秀 / 🟡 良好 / 🔴 需改进
|
||||
**瀹屾垚璐ㄩ噺锛?* 馃煝 浼樼<E6B5BC> / 馃煛 鑹<>ソ / 馃敶 闇€鏀硅繘
|
||||
|
||||
**閬囧埌鐨勪富瑕侀棶棰橈細**
|
||||
1.
|
||||
@@ -457,22 +398,21 @@ autoScaling:
|
||||
**瀛﹀埌鐨勭粡楠岋細**
|
||||
1.
|
||||
|
||||
**明天计划:**
|
||||
**鏄庡ぉ璁″垝锛?*
|
||||
-
|
||||
|
||||
---
|
||||
|
||||
### Day 4(___月___日,周___)
|
||||
|
||||
**计划任务:**
|
||||
### Day 4锛坃__鏈坃__鏃ワ紝鍛╛__锛?
|
||||
**璁″垝浠诲姟锛?*
|
||||
- [ ] Phase 4: 浠诲姟鎷嗗垎鏈哄埗
|
||||
|
||||
**实际完成:**
|
||||
**瀹為檯瀹屾垚锛?*
|
||||
-
|
||||
|
||||
**工作时长:** ___小时
|
||||
**宸ヤ綔鏃堕暱锛?* ___灏忔椂
|
||||
|
||||
**完成质量:** 🟢 优秀 / 🟡 良好 / 🔴 需改进
|
||||
**瀹屾垚璐ㄩ噺锛?* 馃煝 浼樼<E6B5BC> / 馃煛 鑹<>ソ / 馃敶 闇€鏀硅繘
|
||||
|
||||
**閬囧埌鐨勪富瑕侀棶棰橈細**
|
||||
1.
|
||||
@@ -480,22 +420,21 @@ autoScaling:
|
||||
**瀛﹀埌鐨勭粡楠岋細**
|
||||
1.
|
||||
|
||||
**明天计划:**
|
||||
**鏄庡ぉ璁″垝锛?*
|
||||
-
|
||||
|
||||
---
|
||||
|
||||
### Day 5(___月___日,周___)
|
||||
|
||||
**计划任务:**
|
||||
### Day 5锛坃__鏈坃__鏃ワ紝鍛╛__锛?
|
||||
**璁″垝浠诲姟锛?*
|
||||
- [ ] Phase 5: 鏂<>偣缁<E581A3>紶鏈哄埗
|
||||
|
||||
**实际完成:**
|
||||
**瀹為檯瀹屾垚锛?*
|
||||
-
|
||||
|
||||
**工作时长:** ___小时
|
||||
**宸ヤ綔鏃堕暱锛?* ___灏忔椂
|
||||
|
||||
**完成质量:** 🟢 优秀 / 🟡 良好 / 🔴 需改进
|
||||
**瀹屾垚璐ㄩ噺锛?* 馃煝 浼樼<E6B5BC> / 馃煛 鑹<>ソ / 馃敶 闇€鏀硅繘
|
||||
|
||||
**閬囧埌鐨勪富瑕侀棶棰橈細**
|
||||
1.
|
||||
@@ -503,22 +442,21 @@ autoScaling:
|
||||
**瀛﹀埌鐨勭粡楠岋細**
|
||||
1.
|
||||
|
||||
**明天计划:**
|
||||
**鏄庡ぉ璁″垝锛?*
|
||||
-
|
||||
|
||||
---
|
||||
|
||||
### Day 6(___月___日,周___)
|
||||
|
||||
**计划任务:**
|
||||
### Day 6锛坃__鏈坃__鏃ワ紝鍛╛__锛?
|
||||
**璁″垝浠诲姟锛?*
|
||||
- [ ] Phase 6: ASL绛涢€夋敼閫狅紙寮€濮嬶級
|
||||
|
||||
**实际完成:**
|
||||
**瀹為檯瀹屾垚锛?*
|
||||
-
|
||||
|
||||
**工作时长:** ___小时
|
||||
**宸ヤ綔鏃堕暱锛?* ___灏忔椂
|
||||
|
||||
**完成质量:** 🟢 优秀 / 🟡 良好 / 🔴 需改进
|
||||
**瀹屾垚璐ㄩ噺锛?* 馃煝 浼樼<E6B5BC> / 馃煛 鑹<>ソ / 馃敶 闇€鏀硅繘
|
||||
|
||||
**閬囧埌鐨勪富瑕侀棶棰橈細**
|
||||
1.
|
||||
@@ -526,23 +464,21 @@ autoScaling:
|
||||
**瀛﹀埌鐨勭粡楠岋細**
|
||||
1.
|
||||
|
||||
**明天计划:**
|
||||
**鏄庡ぉ璁″垝锛?*
|
||||
-
|
||||
|
||||
---
|
||||
|
||||
### Day 7(___月___日,周___)
|
||||
### Day 7锛坃__鏈坃__鏃ワ紝鍛╛__锛?
|
||||
**璁″垝浠诲姟锛?*
|
||||
- [ ] Phase 6: ASL绛涢€夋敼閫狅紙瀹屾垚锛?- [ ] Phase 7: DC鎻愬彇鏀归€狅紙鍙<E7B499>€夛級
|
||||
|
||||
**计划任务:**
|
||||
- [ ] Phase 6: ASL筛选改造(完成)
|
||||
- [ ] Phase 7: DC提取改造(可选)
|
||||
|
||||
**实际完成:**
|
||||
**瀹為檯瀹屾垚锛?*
|
||||
-
|
||||
|
||||
**工作时长:** ___小时
|
||||
**宸ヤ綔鏃堕暱锛?* ___灏忔椂
|
||||
|
||||
**完成质量:** 🟢 优秀 / 🟡 良好 / 🔴 需改进
|
||||
**瀹屾垚璐ㄩ噺锛?* 馃煝 浼樼<E6B5BC> / 馃煛 鑹<>ソ / 馃敶 闇€鏀硅繘
|
||||
|
||||
**閬囧埌鐨勪富瑕侀棶棰橈細**
|
||||
1.
|
||||
@@ -550,22 +486,21 @@ autoScaling:
|
||||
**瀛﹀埌鐨勭粡楠岋細**
|
||||
1.
|
||||
|
||||
**明天计划:**
|
||||
**鏄庡ぉ璁″垝锛?*
|
||||
-
|
||||
|
||||
---
|
||||
|
||||
### Day 8(___月___日,周___)
|
||||
|
||||
**计划任务:**
|
||||
### Day 8锛坃__鏈坃__鏃ワ紝鍛╛__锛?
|
||||
**璁″垝浠诲姟锛?*
|
||||
- [ ] Phase 8: 鍏ㄩ潰娴嬭瘯楠岃瘉
|
||||
|
||||
**实际完成:**
|
||||
**瀹為檯瀹屾垚锛?*
|
||||
-
|
||||
|
||||
**工作时长:** ___小时
|
||||
**宸ヤ綔鏃堕暱锛?* ___灏忔椂
|
||||
|
||||
**完成质量:** 🟢 优秀 / 🟡 良好 / 🔴 需改进
|
||||
**瀹屾垚璐ㄩ噺锛?* 馃煝 浼樼<E6B5BC> / 馃煛 鑹<>ソ / 馃敶 闇€鏀硅繘
|
||||
|
||||
**閬囧埌鐨勪富瑕侀棶棰橈細**
|
||||
1.
|
||||
@@ -573,22 +508,21 @@ autoScaling:
|
||||
**瀛﹀埌鐨勭粡楠岋細**
|
||||
1.
|
||||
|
||||
**明天计划:**
|
||||
**鏄庡ぉ璁″垝锛?*
|
||||
-
|
||||
|
||||
---
|
||||
|
||||
### Day 9(___月___日,周___)
|
||||
|
||||
**计划任务:**
|
||||
### Day 9锛坃__鏈坃__鏃ワ紝鍛╛__锛?
|
||||
**璁″垝浠诲姟锛?*
|
||||
- [ ] Phase 9: SAE閮ㄧ讲涓婄嚎
|
||||
|
||||
**实际完成:**
|
||||
**瀹為檯瀹屾垚锛?*
|
||||
-
|
||||
|
||||
**工作时长:** ___小时
|
||||
**宸ヤ綔鏃堕暱锛?* ___灏忔椂
|
||||
|
||||
**完成质量:** 🟢 优秀 / 🟡 良好 / 🔴 需改进
|
||||
**瀹屾垚璐ㄩ噺锛?* 馃煝 浼樼<E6B5BC> / 馃煛 鑹<>ソ / 馃敶 闇€鏀硅繘
|
||||
|
||||
**閬囧埌鐨勪富瑕侀棶棰橈細**
|
||||
1.
|
||||
@@ -596,36 +530,34 @@ autoScaling:
|
||||
**瀛﹀埌鐨勭粡楠岋細**
|
||||
1.
|
||||
|
||||
**后续计划:**
|
||||
**鍚庣画璁″垝锛?*
|
||||
-
|
||||
|
||||
---
|
||||
|
||||
## 📝 问题与解决方案汇总
|
||||
|
||||
## 馃摑 闂<><E99782>涓庤В鍐虫柟妗堟眹鎬?
|
||||
### 闂<><E99782>鍒楄〃
|
||||
|
||||
| # | 发现日期 | Phase | 问题描述 | 严重程度 | 状态 | 解决方案 | 解决日期 |
|
||||
| # | 鍙戠幇鏃ユ湡 | Phase | 闂<EFBFBD><EFBFBD>鎻忚堪 | 涓ラ噸绋嬪害 | 鐘舵€?| 瑙e喅鏂规<E98F82> | 瑙e喅鏃ユ湡 |
|
||||
|---|---------|-------|---------|---------|------|---------|---------|
|
||||
| 1 | _____ | Phase ___ | | 🔴高/🟡中/🟢低 | ⬜未解决/✅已解决 | | _____ |
|
||||
| 2 | _____ | Phase ___ | | 🔴高/🟡中/🟢低 | ⬜未解决/✅已解决 | | _____ |
|
||||
| 3 | _____ | Phase ___ | | 🔴高/🟡中/🟢低 | ⬜未解决/✅已解决 | | _____ |
|
||||
| 1 | _____ | Phase ___ | | 馃敶楂?馃煛涓?馃煝浣?| 猬滄湭瑙e喅/鉁呭凡瑙e喅 | | _____ |
|
||||
| 2 | _____ | Phase ___ | | 馃敶楂?馃煛涓?馃煝浣?| 猬滄湭瑙e喅/鉁呭凡瑙e喅 | | _____ |
|
||||
| 3 | _____ | Phase ___ | | 馃敶楂?馃煛涓?馃煝浣?| 猬滄湭瑙e喅/鉁呭凡瑙e喅 | | _____ |
|
||||
|
||||
### 閲嶈<E996B2>闂<EFBFBD><E99782>璇︾粏璁板綍
|
||||
|
||||
**问题1:**
|
||||
- **发现时间:**
|
||||
- **问题描述:**
|
||||
- **影响范围:**
|
||||
- **根本原因:**
|
||||
- **解决方案:**
|
||||
- **预防措施:**
|
||||
- **解决时间:**
|
||||
**闂<EFBFBD><EFBFBD>1锛?*
|
||||
- **鍙戠幇鏃堕棿锛?*
|
||||
- **闂<><E99782>鎻忚堪锛?*
|
||||
- **褰卞搷鑼冨洿锛?*
|
||||
- **鏍规湰鍘熷洜锛?*
|
||||
- **瑙e喅鏂规<E98F82>锛?*
|
||||
- **棰勯槻鎺<E6A7BB>柦锛?*
|
||||
- **瑙e喅鏃堕棿锛?*
|
||||
|
||||
---
|
||||
|
||||
## 📚 学习笔记与最佳实践
|
||||
|
||||
## 馃摎 瀛︿範绗旇<E7BB97>涓庢渶浣冲疄璺?
|
||||
### pg-boss 浣跨敤蹇冨緱
|
||||
```
|
||||
1.
|
||||
@@ -640,8 +572,7 @@ autoScaling:
|
||||
3.
|
||||
```
|
||||
|
||||
### 测试技巧
|
||||
```
|
||||
### 娴嬭瘯鎶€宸?```
|
||||
1.
|
||||
2.
|
||||
3.
|
||||
@@ -658,33 +589,23 @@ autoScaling:
|
||||
|
||||
## 馃帀 椤圭洰鎬荤粨
|
||||
|
||||
### 最终成果
|
||||
|
||||
**代码统计:**
|
||||
- 新增代码:_____ 行
|
||||
- 修改代码:_____ 行
|
||||
- 测试代码:_____ 行
|
||||
- 总代码量:_____ 行
|
||||
|
||||
**测试结果:**
|
||||
### 鏈€缁堟垚鏋?
|
||||
**浠g爜缁熻<E7BC81>锛?*
|
||||
- 鏂板<E98F82>浠g爜锛歘____ 琛?- 淇<>敼浠g爜锛歘____ 琛?- 娴嬭瘯浠g爜锛歘____ 琛?- 鎬讳唬鐮侀噺锛歘____ 琛?
|
||||
**娴嬭瘯缁撴灉锛?*
|
||||
- 鍗曞厓娴嬭瘯锛歘____ / _____ 閫氳繃
|
||||
- 闆嗘垚娴嬭瘯锛歘____ / _____ 閫氳繃
|
||||
- 鍔熻兘娴嬭瘯锛歘____ / _____ 閫氳繃
|
||||
- 鎬ц兘娴嬭瘯锛歘____ / _____ 閫氳繃
|
||||
|
||||
**性能指标:**
|
||||
- 缓存命中率:_____%(目标 > 60%)
|
||||
- LLM API调用量:下降_____%(目标 > 40%)
|
||||
- 长任务成功率:_____%(目标 > 99%)
|
||||
- 实例重启恢复:_____%(目标 100%)
|
||||
|
||||
**鎬ц兘鎸囨爣锛?*
|
||||
- 缂撳瓨鍛戒腑鐜囷細_____%锛堢洰鏍?> 60%锛?- LLM API璋冪敤閲忥細涓嬮檷_____%锛堢洰鏍?> 40%锛?- 闀夸换鍔℃垚鍔熺巼锛歘____%锛堢洰鏍?> 99%锛?- 瀹炰緥閲嶅惎鎭㈠<E98EAD>锛歘____%锛堢洰鏍?100%锛?
|
||||
### 椤圭洰浜<E6B4B0>偣
|
||||
1.
|
||||
2.
|
||||
3.
|
||||
|
||||
### 待改进事项
|
||||
1.
|
||||
### 寰呮敼杩涗簨椤?1.
|
||||
2.
|
||||
3.
|
||||
|
||||
@@ -697,63 +618,52 @@ autoScaling:
|
||||
|
||||
## 馃敆 鐩稿叧鏂囨。閾炬帴
|
||||
|
||||
- [Postgres-Only改造实施计划(完整版)](./09-Postgres-Only改造实施计划(完整版).md)
|
||||
- [Postgres-Only鏀归€犲疄鏂借<EFBFBD>鍒掞紙瀹屾暣鐗堬級](./09-Postgres-Only鏀归€犲疄鏂借<EFBFBD>鍒?瀹屾暣鐗?.md)
|
||||
- [Postgres-Only鍏ㄨ兘鏋舵瀯瑙e喅鏂规<E98F82>](./08-Postgres-Only 鍏ㄨ兘鏋舵瀯瑙e喅鏂规<E98F82>.md)
|
||||
- [长时间任务可靠性分析](./06-长时间任务可靠性分析.md)
|
||||
- [SAE部署完全指南](../05-部署文档/02-SAE部署完全指南(产品经理版).md)
|
||||
- [闀挎椂闂翠换鍔″彲闈犳€у垎鏋怾(./06-闀挎椂闂翠换鍔″彲闈犳€у垎鏋?md)
|
||||
- [SAE閮ㄧ讲瀹屽叏鎸囧崡](../05-閮ㄧ讲鏂囨。/02-SAE閮ㄧ讲瀹屽叏鎸囧崡(浜у搧缁忕悊鐗?.md)
|
||||
|
||||
---
|
||||
|
||||
## 🎉 Phase 1-7 完成总结(2025-12-13)
|
||||
## 馃帀 Phase 1-7 瀹屾垚鎬荤粨锛?025-12-13锛?
|
||||
### 鉁?瀹屾垚鎯呭喌
|
||||
|
||||
### ✅ 完成情况
|
||||
|
||||
| 完成项 | 数量 | 说明 |
|
||||
| 瀹屾垚椤?| 鏁伴噺 | 璇存槑 |
|
||||
|--------|------|------|
|
||||
| **完成阶段** | 7个 | Phase 1-7 全部完成 |
|
||||
| **完成任务** | 31个 | 总任务数45个,完成69% |
|
||||
| **代码量** | ~1750行 | 新增核心代码 |
|
||||
| **测试** | 10个 | 全部通过 |
|
||||
| **文档** | 4个 | 全部更新 |
|
||||
| **瀹屾垚闃舵<EFBFBD>** | 7涓?| Phase 1-7 鍏ㄩ儴瀹屾垚 |
|
||||
| **瀹屾垚浠诲姟** | 31涓?| 鎬讳换鍔℃暟45涓<35>紝瀹屾垚69% |
|
||||
| **浠g爜閲?* | ~1750琛?| 鏂板<E98F82>鏍稿績浠g爜 |
|
||||
| **娴嬭瘯** | 10涓?| 鍏ㄩ儴閫氳繃 |
|
||||
| **鏂囨。** | 4涓?| 鍏ㄩ儴鏇存柊 |
|
||||
|
||||
### 馃弳 鏍稿績鎴愭灉
|
||||
|
||||
1. **Platform-Only 鏋舵瀯閲嶆瀯**
|
||||
- 缁熶竴浣跨敤 `platform_schema.job.data` 瀛樺偍浠诲姟绠$悊淇℃伅
|
||||
- 业务表保持简洁,只存储业务信息
|
||||
- CheckpointService 所有模块通用
|
||||
- 符合 3 层架构原则
|
||||
|
||||
2. **智能双模式处理**
|
||||
- 小任务(<50条):直接处理,快速响应
|
||||
- 大任务(≥50条):队列处理,可靠性高
|
||||
- 涓氬姟琛ㄤ繚鎸佺畝娲侊紝鍙<E7B49D>瓨鍌ㄤ笟鍔′俊鎭? - CheckpointService 鎵€鏈夋ā鍧楅€氱敤
|
||||
- 绗﹀悎 3 灞傛灦鏋勫師鍒?
|
||||
2. **鏅鸿兘鍙屾ā寮忓<E5AFAE>鐞?*
|
||||
- 灏忎换鍔★紙<50鏉★級锛氱洿鎺ュ<E98EBA>鐞嗭紝蹇<E7B49D>€熷搷搴? - 澶т换鍔★紙鈮?0鏉★級锛氶槦鍒楀<E98D92>鐞嗭紝鍙<E7B49D>潬鎬ч珮
|
||||
- 鎬ц兘涓庡彲闈犳€х殑瀹岀編骞宠
|
||||
|
||||
3. **零额外成本**
|
||||
- 使用 Postgres,不需要 Redis
|
||||
- 年省 ¥8400
|
||||
- 运维成本零增加
|
||||
|
||||
### 📊 工作量统计
|
||||
|
||||
3. **闆堕<E99786>澶栨垚鏈?*
|
||||
- 浣跨敤 Postgres锛屼笉闇€瑕?Redis
|
||||
- 骞寸渷 锟?400
|
||||
- 杩愮淮鎴愭湰闆跺<E99786>鍔?
|
||||
### 馃搳 宸ヤ綔閲忕粺璁?
|
||||
```
|
||||
实际用时:6.5天
|
||||
预计用时:9天
|
||||
提前完成:2.5天 ✅
|
||||
|
||||
瀹為檯鐢ㄦ椂锛?.5澶?棰勮<E6A3B0>鐢ㄦ椂锛?澶?鎻愬墠瀹屾垚锛?.5澶?鉁?
|
||||
浠g爜閲忥細~1750琛岋紙鐩<E7B499>爣~1900琛岋級
|
||||
测试覆盖:100%
|
||||
Linter错误:0个
|
||||
```
|
||||
|
||||
### 🎯 下一步
|
||||
娴嬭瘯瑕嗙洊锛?00%
|
||||
Linter閿欒<EFBFBD>锛?涓?```
|
||||
|
||||
### 馃幆 涓嬩竴姝?
|
||||
- **Phase 8**锛氬叏闈㈡祴璇曢獙璇侊紙棰勮<E6A3B0>5澶╋級
|
||||
- **Phase 9**:SAE部署上线(预计5.5天)
|
||||
- **Phase 9**锛歋AE閮ㄧ讲涓婄嚎锛堥<EFBFBD>璁?.5澶╋級
|
||||
|
||||
---
|
||||
|
||||
**版本历史:**
|
||||
- V1.0(2025-12-07):初始版本
|
||||
- V1.1(2025-12-13):Phase 1-7 完成,添加 Platform-Only 架构重构记录
|
||||
**鐗堟湰鍘嗗彶锛?*
|
||||
- V1.0锛?025-12-07锛夛細鍒濆<EFBFBD>鐗堟湰
|
||||
- V1.1锛?025-12-13锛夛細Phase 1-7 瀹屾垚锛屾坊鍔?Platform-Only 鏋舵瀯閲嶆瀯璁板綍
|
||||
|
||||
|
||||
@@ -1,27 +1,26 @@
|
||||
# 杩愮淮鏂囨。
|
||||
|
||||
> **文档定位:** 系统运维、监控、故障排查
|
||||
> **适用范围:** 运维团队、SRE团队
|
||||
> **鏂囨。瀹氫綅锛?* 绯荤粺杩愮淮銆佺洃鎺с€佹晠闅滄帓鏌?
|
||||
> **閫傜敤鑼冨洿锛?* 杩愮淮鍥㈤槦銆丼RE鍥㈤槦
|
||||
|
||||
---
|
||||
|
||||
## 馃搵 杩愮淮鏂囨。娓呭崟
|
||||
|
||||
| 文档 | 说明 | 状态 |
|
||||
| 鏂囨。 | 璇存槑 | 鐘舵€?|
|
||||
|------|------|------|
|
||||
| **01-环境配置指南.md** | 环境变量、数据库连接、API密钥配置 | ✅ 已完成 |
|
||||
| **02-环境变量配置模板.md** | .env配置模板,含CloseAI配置 ⭐ | ✅ 已完成 |
|
||||
| **03-监控告警.md** | 监控指标、告警规则 | ⏳ 待创建 |
|
||||
| **04-故障排查.md** | 常见问题排查手册 | ⏳ 待创建 |
|
||||
| **05-备份恢复.md** | 数据备份和恢复策略 | ⏳ 待创建 |
|
||||
| **01-鐜<EFBFBD><EFBFBD>閰嶇疆鎸囧崡.md** | 鐜<EFBFBD><EFBFBD>鍙橀噺銆佹暟鎹<EFBFBD>簱杩炴帴銆丄PI瀵嗛挜閰嶇疆 | 鉁?宸插畬鎴?|
|
||||
| **02-鐜<EFBFBD><EFBFBD>鍙橀噺閰嶇疆妯℃澘.md** | .env閰嶇疆妯℃澘锛屽惈CloseAI閰嶇疆 猸?| 鉁?宸插畬鎴?|
|
||||
| **03-鐩戞帶鍛婅<EFBFBD>.md** | 鐩戞帶鎸囨爣銆佸憡璀﹁<EFBFBD>鍒?| 鈴?寰呭垱寤?|
|
||||
| **04-鏁呴殰鎺掓煡.md** | 甯歌<EFBFBD>闂<EFBFBD><EFBFBD>鎺掓煡鎵嬪唽 | 鈴?寰呭垱寤?|
|
||||
| **05-澶囦唤鎭㈠<EFBFBD>.md** | 鏁版嵁澶囦唤鍜屾仮澶嶇瓥鐣?| 鈴?寰呭垱寤?|
|
||||
|
||||
---
|
||||
|
||||
## 馃幆 鏍稿績杩愮淮浠诲姟
|
||||
|
||||
### 1. 鐩戞帶
|
||||
- 系统健康检查
|
||||
- 性能监控
|
||||
- 绯荤粺鍋ュ悍妫€鏌?- 鎬ц兘鐩戞帶
|
||||
- 鍛婅<E98D9B>閫氱煡
|
||||
|
||||
### 2. 鏃ュ織
|
||||
@@ -30,14 +29,12 @@
|
||||
- 鏃ュ織褰掓。
|
||||
|
||||
### 3. 澶囦唤
|
||||
- 数据库备份
|
||||
- 文件备份
|
||||
- 鏁版嵁搴撳<EFBFBD>浠?- 鏂囦欢澶囦唤
|
||||
- 鎭㈠<E98EAD>婕旂粌
|
||||
|
||||
### 4. 鏁呴殰澶勭悊
|
||||
- 鏁呴殰璇婃柇
|
||||
- 应急预案
|
||||
- 事后总结
|
||||
- 搴旀€ラ<EFBFBD>妗?- 浜嬪悗鎬荤粨
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user