Add Alembic migration foundation
test / pytest (push) Successful in 1m34s

This commit is contained in:
2026-06-01 16:02:43 +02:00
parent c42cc2ddb6
commit 8b8bd9f38f
17 changed files with 1459 additions and 101 deletions
+42 -24
View File
@@ -8,8 +8,8 @@
## 目标 / Goal
引入 Alembic 并**安全接管现有生产库**,schema 一点不改,所有现有测试保持绿。
Introduce Alembic and **safely adopt the existing prod DB**, with zero schema change; all existing tests stay green.
引入 Alembic 并**安全接管现有生产库**,schema 一点不改,所有现有测试保持绿。**迁移与应用启动分离**:启动只做只读校验 + fail-close,实际迁移由独立、幂等命令 `python -m app.migrate` 执行。
Introduce Alembic and **safely adopt the existing prod DB** with zero schema change; all tests stay green. **Migration is separated from startup**: boot only verifies (read-only) and fails closed; the actual migrating is done by a separate idempotent command `python -m app.migrate`.
---
@@ -17,8 +17,8 @@ Introduce Alembic and **safely adopt the existing prod DB**, with zero schema ch
- **当前没有 Alembic。** 唯一的"迁移"是 `app/db.py::_sync_sqlite_image_columns()`(启动时缺图片列就 `ALTER TABLE ADD COLUMN`)。
No Alembic today; the only "migration" is the hand-rolled image-column sync in `app/db.py`.
- `app/db.py::init_db()` 在 FastAPI lifespan 启动时被 `create_app()` 调用,现在执行 `Base.metadata.create_all()` + `_sync_sqlite_image_columns()`。相关符号:`Base``engine``SessionLocal``configure_database()`
`init_db()` runs at lifespan startup and currently does `create_all()` + the image-column sync.
- `app/db.py::init_db()` 在 FastAPI lifespan 启动时被 `create_app()` 调用,现在执行 `Base.metadata.create_all()` + `_sync_sqlite_image_columns()`**本步把它改成只读校验**(不再在启动时建表/迁移)。相关符号:`Base``engine``SessionLocal``configure_database()`
`init_db()` runs at lifespan startup and currently does `create_all()` + the image-column sync. **This step turns it into a read-only check** (no table creation/migration at boot).
- `tests/conftest.py``client` fixture`configure_database(tmp_url)``create_app()`(触发 `init_db`)。每个测试用临时 SQLite,互不污染。
- models 在 `app/models.py``Box` / `Item` / `SubItem` 三张表;每张含 `image_blob`(BLOB) / `image_mime_type` / `image_width` / `image_height`,以及 `created_at` / `updated_at`
- DB URL 来自 `app/config.py::get_settings().database_url`(默认 `sqlite:///./data/app.db`)。
@@ -29,10 +29,11 @@ Introduce Alembic and **safely adopt the existing prod DB**, with zero schema ch
- 所有数据库最终收敛到同一个 `head`。All DBs converge to the same `head`.
- **V1 baseline 必须严格等于"今天的真实 schema"**(三张表 + 现有图片列 + 索引),**不多一列**。新东西放后续 revision。
The V1 baseline must equal **today's actual schema exactly** — nothing more.
- 老库:`stamp V1`(只写版本号,**不建表、不碰数据**)→ `upgrade head`
Existing DB: `stamp V1` (writes only the version row, **no DDL, no data change**) `upgrade head`.
- 新库:跑 `V1`(真正建表)`upgrade head`
Fresh DB: run `V1` (creates tables) → `upgrade head`.
- 以下动作**由迁移命令执行,不在应用启动时** / done by the **migration command**, not at boot
- 老库且与 baseline 一致:`stamp V1`(只写版本号,**不建表、不碰数据**)`upgrade head`
Existing DB matching baseline: `stamp V1` (no DDL, no data change) `upgrade head`.
- 老库但与 baseline 不一致:**fail-close,不做任何改动**。Mismatched existing DB → fail closed.
- 新库:跑 `V1`(真正建表)→ `upgrade head`。Fresh DB: run `V1``upgrade head`.
---
@@ -48,39 +49,54 @@ Introduce Alembic and **safely adopt the existing prod DB**, with zero schema ch
Author V1 by autogenerating against an **empty** DB.
- [ ] **验证 baseline**:对一份**生产库副本**跑 `alembic check`,确认**无差异**(印证可安全 `stamp`;SQLite 偶有类型亲和/索引命名假差异,人眼复核)。
Verify with `alembic check` against a **copy of the prod DB** → expect no diff.
- [ ] 新增封装 `app/migrate.py`导出 `run_migrations(database_url: str)`
- 编程方式构造 Alembic `Config``script_location` 指向打包进镜像的 `migrations/``sqlalchemy.url` = 传入 URL
- 用 SQLAlchemy inspector 实现自动认领:
- `alembic_version``command.upgrade(cfg, "head")`
- `alembic_version` 但有 `boxes` 表 → `command.stamp(cfg, "<V1 rev>")``command.upgrade(cfg, "head")`
- 全空 → `command.upgrade(cfg, "head")`
- [ ]`app/db.py::init_db()`:改为调 `run_migrations(resolved_url)`**删除** `_sync_sqlite_image_columns()`Alembic 接管后冗余)。保留 `configure_database()` / engine 装配逻辑
`init_db()` calls `run_migrations(...)`; **remove** `_sync_sqlite_image_columns()`.
- [ ] 新增 `app/migrate.py`承担两个职责 / two responsibilities
- **(A) 迁移命令入口 `python -m app.migrate`(幂等 / idempotent**编程方式构造 Alembic `Config``script_location` 打包进镜像的 `migrations/``sqlalchemy.url` = 解析出的 URL,用 SQLAlchemy inspector 分情况:
- 空库 / empty → `command.upgrade(cfg, "head")`
- 老库且与 **baseline(V1)** 一致 → `command.stamp(cfg, "<V1 rev>")``command.upgrade(cfg, "head")`
- 老库但与 baseline 不一致 → **fail-close**:非零退出 + 清晰日志 + **不做任何改动**
- 已在 `head` → 空操作、退出 0
- `<V1 rev>`**baseline 这个具体 revision**`down_revision=None` 的那条),不是 `head`
- "与 baseline 一致"的判定**对照 baseline(V1) 的预期 schema**(不是 head);SQLite 假差异需容忍或允许人工确认覆盖。
- **(B) 启动校验 `verify_schema_is_current(url)`(只读 / read-only**:比较 DB 当前 revision 与 `head`;不一致返回失败/抛错,**绝不改动 DB**。
- [ ]`app/db.py::init_db()`:改为调 `verify_schema_is_current(resolved_url)` —— **一致才放行;不一致 fail-close**(清晰日志,提示先跑 `python -m app.migrate`)。不再在启动时建表/迁移。**删除** `_sync_sqlite_image_columns()`。保留 `configure_database()` / engine 装配。
`init_db()` now only verifies and **fails closed** on mismatch (pointing the user to `python -m app.migrate`); remove `_sync_sqlite_image_columns()`.
- [ ] `tests/conftest.py`fixture 改为**先跑迁移命令**把临时库带到 `head`,再 `create_app()`(这样启动校验通过)。
Fixture runs the migration first, then `create_app()`.
- [ ] `Dockerfile`:加 `COPY alembic.ini .``COPY migrations ./migrations`(否则容器内无迁移脚本)。
- [ ] CI(可选 / optional):`.github/workflows/test.yml` 加一步 `alembic check`,防止 model 与迁移漂移。
- [ ] Compose `db-migration` 闸门(可后续 / can be deferred):加一个一次性服务跑 `python -m app.migrate``web``depends_on: condition: service_completed_successfully` 等它成功(见设计 §3.5)。
Add a one-shot `db-migration` service gating `web` (design §3.5); may be deferred.
---
## 涉及文件 / Files
`requirements.txt``alembic.ini`(新)、`migrations/**`(新)、`app/migrate.py`(新)、`app/db.py``Dockerfile``tests/`(可选)`.github/workflows/test.yml`
`requirements.txt``alembic.ini`(新)、`migrations/**`(新)、`app/migrate.py`(新)、`app/db.py``tests/conftest.py``Dockerfile`、(可选)`.github/workflows/test.yml`、(可后续)`docker-compose.yml`
---
## 测试 / Tests
- [ ] 现有 ~83 个测试全绿(它们经 `init_db` 现在改走迁移建表)。
All existing ~83 tests pass (schema now built via migrations through `init_db`).
- [ ] 新增**认领老库**用例:构造一个"有 `boxes` 数据、无 `alembic_version`"的库(可先用 `create_all` 造),调 `run_migrations`断言数据保留、`alembic_version` 到达 `head`、未重复建表报错。
New adoption test: a "has `boxes` data, no `alembic_version`" DB → after `run_migrations`, data preserved and version at `head`.
- [ ] 新增**全新库**用例:空 URL → `run_migrations`三张表存在、版本到 `head`
- [ ] 现有 ~83 个测试全绿(fixture 先跑迁移、再起 App,启动校验通过)。
All existing ~83 tests pass (fixture migrates first, then starts the app).
- [ ] **认领老库2a**:构造"有 `boxes` 数据、无 `alembic_version`"的库(可先用 `create_all` 造)→ 跑迁移命令 → 断言数据保留、版本到达 `head`、未重复建表报错。
Adoption (2a): migrate an un-stamped populated DB → data preserved, version at `head`.
- [ ] **全新库**:空 URL → 跑迁移命令 → 三张表存在、版本到 `head`
Fresh DB: empty URL → migrate → tables exist, version at `head`.
- [ ] **fail-close(启动)**DB 未到 `head``create_app()` / `init_db()` 启动应 fail-close(抛错/拒绝服务)、不改动 DB。
Startup fails closed when the DB is not at `head`; DB unchanged.
- [ ] **fail-close2b**:构造与 baseline 不一致的老库 → 跑迁移命令 → 断言非零退出、DB 不变。
Migration command fails closed on a 2b mismatch; DB unchanged.
---
## 验收 / Acceptance
- 全新库:从 V1 建表,应用正常起。Fresh DB builds from V1; app starts.
- 模拟老库:自动 `stamp` + `upgrade`**数据无损**。Existing-like DB auto-adopts; data intact.
- 迁移命令:空库建到 `head`;老库一致则认领并到 `head`;老库不一致则 **fail-close 不改动**;已在 `head` 则幂等空操作。
Migration command: empty→head; matching existing→adopt+head; mismatch→fail closed; already-at-head→no-op.
- 启动校验:DB 未到 `head` 时**拒绝启动**并输出清晰日志;到 `head` 才正常起。
Startup refuses to boot (clear log) unless the DB is at `head`.
- 模拟老库认领后**数据无损**。Adopted existing-like DB keeps data intact.
- 全部测试绿;schema 与本步骤前**逐列一致**(本步不改业务 schema)。
All tests green; schema identical to before (no business-schema change).
@@ -90,6 +106,8 @@ Introduce Alembic and **safely adopt the existing prod DB**, with zero schema ch
- **baseline 与现状有偏差 → `stamp` 失真。** 缓解:`alembic check` 对生产副本校验 + 人眼复核 SQLite 假差异。
Baseline drift → `alembic check` against a prod copy + manual eyeball.
- **2b 一致性比对假阳性 → 合法老库被误 fail-close。** 缓解:比对基准用 baseline(V1) 而非 head;容忍已知 SQLite 噪声,或提供"人工确认覆盖"的开关。
2b false positives wrongly fail a legit DB → compare against baseline (not head); tolerate known SQLite noise or offer a manual-confirm override.
- **容器内找不到迁移脚本。** 缓解:确认 `Dockerfile``COPY` `alembic.ini``migrations/``script_location` 用绝对/相对镜像 WORKDIR(`/app`) 正确解析。
Migrations missing in image → ensure they're `COPY`-ed and `script_location` resolves under `/app`.