This commit is contained in:
@@ -70,10 +70,13 @@
|
||||
└───────────────┘ └──────┬───────┘ └───────────────┘
|
||||
│
|
||||
┌───────────────────▼─────────────────────────┐
|
||||
│ app/migrate.py（封装层） │
|
||||
│ run_migrations(url): 自动 stamp / upgrade │
|
||||
│ app/migrate.py │
|
||||
│ 启动 / boot: verify_schema_is_current() 只读 │
|
||||
│ └─ 与 head 不一致 → fail-close，拒绝启动 │
|
||||
│ 命令 / CLI `python -m app.migrate`（幂等）: │
|
||||
│ └─ 空库建库 / 认领老库 / upgrade（见 §3） │
|
||||
└───────────────────┬─────────────────────────┘
|
||||
│ command.upgrade / stamp
|
||||
│ command.upgrade / stamp（仅迁移命令 / migration command only）
|
||||
┌───────────────────▼─────────────────────────┐
|
||||
│ Alembic (alembic.ini + migrations/) │
|
||||
│ V1 baseline → V2(app_settings) → … │
|
||||
@@ -112,32 +115,60 @@ All databases converge to the same `head`. The `V1 baseline` must equal **today'
|
||||
> `stamp` 只向 `alembic_version` 写一条版本记录，**不执行任何 DDL、不修改数据**。这是安全认领已有库的关键。
|
||||
> `stamp` only writes a row into `alembic_version`; it runs **no DDL and touches no data**. This is the key to safely adopting an existing DB.
|
||||
|
||||
### 3.3 自动认领逻辑 / Auto-adoption (in `app/migrate.py`)
|
||||
### 3.3 运行时机：校验与迁移分离 / Migrations Run Separately from Startup
|
||||
|
||||
`init_db()` 启动时调用 `run_migrations(url)`，内部用 SQLAlchemy inspector 判断：
|
||||
At startup `init_db()` calls `run_migrations(url)`, which inspects the DB:
|
||||
**关键决策：迁移不在应用启动时发生。** 启动只做**只读校验**，迁移由一个独立、显式的命令/步骤执行。
|
||||
**Key decision: migrations do not happen at app startup.** Startup only **verifies** (read-only); migrating is an explicit, separate step.
|
||||
|
||||
- **启动校验（fail-close）/ Startup check (fail-closed)：** `app/db.py::init_db()` 调 `app/migrate.py::verify_schema_is_current(url)`，比较 DB 当前 revision 与 `head`：
|
||||
- 一致 → 正常启动 / match → start normally。
|
||||
- 不一致（含空库、未认领的老库）→ **fail-close**：输出清晰日志、拒绝提供服务、提示先跑迁移步骤；**不执行任何 DDL、不碰数据**。
|
||||
Mismatch (incl. empty or un-adopted DBs) → **fail closed**: clear log, refuse to serve, no DDL, no data change.
|
||||
- **迁移命令 / The migration command：** 独立、显式、**幂等**的 `python -m app.migrate`（逻辑在 `app/migrate.py`）。已在 `head` 则空操作并退出 0，便于每次部署都安全重跑。
|
||||
A separate, explicit, **idempotent** `python -m app.migrate`. No-op (exit 0) when already at `head`, so it is safe to re-run on every deploy.
|
||||
- 退休手写列同步 / Retire the hand-rolled sync：`_sync_sqlite_image_columns()` 删除，schema 由 Alembic 单一接管。
|
||||
`_sync_sqlite_image_columns()` is removed; Alembic is the sole owner of schema.
|
||||
|
||||
为什么 / Why：避免"启动副作用式迁移"、避免多实例并发迁移竞态；当 code 与 DB 不一致时，**宁可不启动也不带病运行**。
|
||||
Avoids surprise startup migrations and concurrent-migration races; on a code/DB mismatch it refuses to run rather than run wrong.
|
||||
|
||||
### 3.4 迁移命令的三种情况 / The Migration Command's Three Cases
|
||||
|
||||
`python -m app.migrate` 用 SQLAlchemy inspector 判定，分三种：
|
||||
`python -m app.migrate` inspects the DB and branches three ways:
|
||||
|
||||
| 库的状态 / DB state | 动作 / Action |
|
||||
| --- | --- |
|
||||
| 有 `alembic_version` / has `alembic_version` | `upgrade head` |
|
||||
| 无 `alembic_version` 但有 `boxes` 表（＝老生产库）/ no `alembic_version` but `boxes` exists | `stamp V1` → `upgrade head` |
|
||||
| 全空 / empty | `upgrade head`（从 V1 建起 / build from V1） |
|
||||
| **空库 / empty** | `upgrade head`（建库并升到最新 / create & upgrade to head） |
|
||||
| **老库且与 baseline 一致 / existing, matches baseline（2a）** | `stamp V1` → `upgrade head`（认领后升级 / adopt then upgrade） |
|
||||
| **老库但与 baseline 不一致 / existing, mismatched（2b）** | **fail-close，不做任何改动 / fail closed, no changes** |
|
||||
|
||||
这样**生产机重新部署零手动迁移命令**，老数据安全。
|
||||
So redeploying the production box needs **zero manual migration commands**; existing data is safe.
|
||||
> **一致性比对的基准是 baseline(V1)，不是 head。** 未认领的老库结构停在 V1（不含 `app_settings` 等后续内容），若拿 head 去比会把合法老库误判为不一致。
|
||||
> The match is compared against the **baseline (V1)**, not `head` — an un-adopted DB sits at V1 and would wrongly look "mismatched" if compared against head.
|
||||
>
|
||||
> ⚠️ SQLite 的 autogenerate 比对存在假阳性（类型亲和、索引命名等），可能让 2b 误 fail。实现上需用**容忍性比对**或允许**人工确认覆盖**（见 §3.6 验证）。
|
||||
> SQLite autogenerate has false positives; 2b should use a tolerant comparison or allow a documented manual override (see §3.6).
|
||||
|
||||
### 3.4 封装层 / The Wrapper (`app/migrate.py`)
|
||||
### 3.5 部署形态：Compose db-migration 闸门 / Deployment Shape: a Compose Gate（未来 / future）
|
||||
|
||||
应用其余部分只调用 `run_migrations(database_url)`，不直接接触 Alembic API。封装内部：
|
||||
The rest of the app only calls `run_migrations(database_url)`; Alembic stays encapsulated. Internally it:
|
||||
意图：用一个一次性 `db-migration` 服务跑迁移命令，**成功才放行 App**。本轮可先只交付命令本身，Compose 接线随后。
|
||||
Intent: a one-shot `db-migration` service runs the command and **the app starts only on its success**. The command ships this round; the Compose wiring can follow.
|
||||
|
||||
- 以编程方式构造 Alembic `Config`（`script_location` 指向打包进镜像的 `migrations/`，`sqlalchemy.url` 用传入的 URL）。
|
||||
Builds an Alembic `Config` programmatically (`script_location` → bundled `migrations/`, `sqlalchemy.url` → the passed URL).
|
||||
- 调 `command.stamp(...)` / `command.upgrade(...)`。
|
||||
- 由 `init_db()` 调用，取代原来的 `create_all()` + `_sync_sqlite_image_columns()`（后者删除）。
|
||||
Called by `init_db()`, replacing `create_all()` + `_sync_sqlite_image_columns()` (the latter is removed).
|
||||
```yaml
|
||||
services:
|
||||
db-migration:
|
||||
image: <same image>
|
||||
command: python -m app.migrate # 成功 exit 0；2b/失败 exit ≠0
|
||||
web:
|
||||
depends_on:
|
||||
db-migration:
|
||||
condition: service_completed_successfully
|
||||
```
|
||||
|
||||
### 3.5 Alembic 配置要点 / Alembic config notes
|
||||
迁移失败（含 2b 不一致）→ App 永不启动。
|
||||
A failed migration (incl. a 2b mismatch) → the app never starts.
|
||||
|
||||
### 3.6 Alembic 配置要点 / Alembic config notes
|
||||
|
||||
- `migrations/env.py`：`target_metadata = Base.metadata`；DB URL 从 `get_settings().database_url` 动态读取（不写死在 `alembic.ini`）；对 SQLite 设 `render_as_batch=True`（便于未来改列/删列走 batch 模式）。
|
||||
`target_metadata = Base.metadata`; URL read dynamically from settings; `render_as_batch=True` for SQLite.
|
||||
@@ -267,10 +298,12 @@ AI off/unconfigured → no button (or a hint to `/settings`); on failure → a f
|
||||
|
||||
## 8. 测试策略 / Testing Strategy
|
||||
|
||||
- **迁移在测试中真实执行 / Migrations run in tests：** 临时 SQLite 上 `upgrade head`，schema 来自迁移本身——单一事实来源，且为迁移提供覆盖。
|
||||
`upgrade head` on a tmp SQLite; schema comes from migrations — single source of truth plus migration coverage.
|
||||
- **认领逻辑测试 / Adoption test：** 构造一个"有 `boxes` 数据但无 `alembic_version`"的库，跑 `run_migrations`，断言数据保留且版本到达 head。
|
||||
Build a "has `boxes` data, no `alembic_version`" DB, run `run_migrations`, assert data preserved and version at head.
|
||||
- **迁移在测试中真实执行 / Migrations run in tests：** fixture 先在临时 SQLite 上跑迁移命令（建库 → `upgrade head`），再 `create_app()`（启动校验随之通过）。schema 来自迁移本身——单一事实来源 + 迁移覆盖。
|
||||
The fixture runs the migration command on a tmp SQLite first, then `create_app()` (whose startup check then passes).
|
||||
- **认领逻辑测试 / Adoption test（2a）：** 构造"有 `boxes` 数据但无 `alembic_version`"的库 → 跑迁移命令 → 断言数据保留、版本到达 head。
|
||||
Build a "has `boxes` data, no `alembic_version`" DB → run the migration command → assert data preserved and version at head.
|
||||
- **fail-close 测试 / Fail-closed tests：** ① DB 未到 head 时 `create_app()` 启动应 fail-close；② 2b 不一致时迁移命令应 fail-close 且不改动。
|
||||
① `create_app()` fails closed when the DB is not at head; ② the migration command fails closed (and changes nothing) on a 2b mismatch.
|
||||
- **LLM 全程 mock / Mock the LLM：** 打桩 `expand_query` / `test_connection`（或底层 httpx），CI 不联网。
|
||||
- **新增用例 / New cases：** 配置增删改 + Key 脱敏；测试连接（mock）；AI 搜索扩展命中；各降级路径（未配置/失败）。
|
||||
|
||||
@@ -293,7 +326,7 @@ AI off/unconfigured → no button (or a hint to `/settings`); on failure → a f
|
||||
| --- | --- | --- |
|
||||
| D1 | 先引入 Alembic 再做功能 / Alembic before features | 配置表与未来列都依赖可靠迁移；退休手写列同步。 |
|
||||
| D2 | V1 baseline 严格等于现状，新东西放 V2+ / baseline = current schema only | 使 `stamp` 认领老库为真、安全。 |
|
||||
| D3 | 自动 stamp/upgrade / auto-adopt | 生产机零手动迁移；契合自托管"开箱即用"。 |
|
||||
| D3 | 迁移与启动分离：启动只校验 + fail-close，迁移走独立幂等命令（`python -m app.migrate`）/ 未来 Compose `db-migration` 闸门 / migrations separated from startup | 避免启动副作用式迁移与并发竞态；schema 不一致宁可不启动也不带病运行；迁移成功才放行 App。 |
|
||||
| D4 | 配置用 KV 表 / KV settings table | 后续配置项多，避免反复给已有表加列。 |
|
||||
| D5 | API Key 明文落库 / plaintext key | 业主威胁模型下可接受；备份至自有 OneDrive。 |
|
||||
| D6 | 复用 httpx，手搓 OpenAI 调用 / reuse httpx | 不引入 `openai` SDK，依赖最小。 |
|
||||
|
||||
Reference in New Issue
Block a user