17 Commits

Author SHA1 Message Date
tliu93 1e0b235cef Merge pull request 'Feature/m1 db consolidation' (#7) from feature/m1-db-consolidation into main
pytest / test (push) Has been cancelled
docker-image / build-and-push (push) Has been cancelled
Reviewed-on: #7
2026-06-12 20:33:34 +02:00
tliu93 a337b06c94 M1-rework: rename leftover pk_cols param in reconciliation test stubs
pytest / test (push) Successful in 48s
pytest / test (pull_request) Failing after 10m49s
Round-2 audit nit (review-notes/M1-full-review-2.md): two _always_fail
monkeypatch stubs still named their (ignored) third positional parameter
pk_cols after _reconcile switched to a full columns list. Rename to columns
for consistency. Test-only, no behavior change.

pytest 97 passed; ruff clean.
2026-06-12 19:11:13 +02:00
tliu93 1cbe6c46d2 M1-rework: harden legacy-migration reconciliation to full-row equality
Audit finding (review-notes/M1-full-review-1.md, FINDING 1): _reconcile only
checked primary-key presence, so a source row skipped by INSERT OR IGNORE due
to a value difference against a pre-existing same-PK target row would
false-pass. Compare ALL columns with SQLite's NULL-safe IS operator instead,
so reconciliation is a true full-row guarantee (idempotent re-runs still pass
because the rows match column-for-column). Add tests for the value-mismatch
abort and for idempotency under full-row reconciliation. Remove the now-unused
pk_cols parameter.

pytest 97 passed; ruff clean (pre-existing only); data-safety grep still empty.
2026-06-12 19:05:56 +02:00
tliu93 2f634006d2 M1-T07: align docs to single-DB reality and re-export OpenAPI
Rewrite README (single app.db + one alembic_app chain, legacy data moved
once via scripts.migrate_legacy_data, accurate test list) and remove the
Grafana Provisioning section. Update architecture-overview to the unified
data layer (one Base, app-DB engine with WAL) and retire the
alembic_location / alembic_poo sections. Mark M1 done in the roadmap.
Re-export openapi/, which catches the spec up to the already-existing
/config/smtp/test and /public-ip/check endpoints (purely additive; M1's
DB-session dependency swap produced no schema change).

pytest 95 passed; ruff clean (pre-existing only); OpenAPI export idempotent.
2026-06-12 17:13:28 +02:00
tliu93 dc624bb7e5 M1-T06: remove Grafana from compose and delete provisioning
Drop the grafana service and the homeautomation_grafana_storage volume
declaration from docker-compose.yml (migration and app services unchanged),
and delete the grafana/ provisioning + dashboards directory. Visualization
moves to the M2 React frontend; the named volume's actual data is retired
manually as an ops step, never by automation.

docker compose config -q passes; pytest 95 passed; ruff clean.
2026-06-12 17:02:49 +02:00
tliu93 af8c602988 M1-T05: drop location/poo database config from Settings and tests
Remove the dead location_database_url / poo_database_url fields and the
location_sqlite_path / poo_sqlite_path computed properties from Settings;
drop them from the config-page payload and from .env.example. Update the
test hardcodes (test_config, test_public_ip, test_smtp) and reduce the
conftest test_database_urls fixture to the single app DB. The one-time
migration script keeps reading legacy URLs from env/CLI, independent of
Settings.

pytest 95 passed; ruff clean (pre-existing only).
2026-06-12 16:57:54 +02:00
tliu93 0d898e09f2 M1-T04: converge startup chain onto the single app DB
run_all_migrations() now adopts/initializes only the app DB and returns
{'app': ...}. app/main.py drops the location/poo readiness checks
(ensure_location_db_ready / ensure_poo_db_ready) and their imports;
ensure_runtime_dirs only provisions the app DB path; lifespan still
fail-closes on a missing/unmanaged app DB. Delete the retired
location/poo adopt scripts and the alembic_location / alembic_poo
chains. Update tests to single-DB expectations and drop the obsolete
location/poo adoption + readiness tests.

pytest 95 passed; ruff clean (pre-existing only); a fresh app DB
initialized via scripts.run_migrations contains location + poo_records.
2026-06-12 16:50:05 +02:00
tliu93 3d3c2bcc57 M1-T03: unify data layer, models, deps and routes onto single app DB
Collapse the three data layers into one. app/db.py now exposes a single
Base, a cached engine bound to app_database_url with SQLite WAL enabled, and
get_engine/get_session_local/reset_db_caches/get_db_session. Delete
app/auth_db.py, app/poo_db.py and app/models/base.py. All models (auth,
config, public_ip, location, poo) inherit the one Base and register on a
single metadata. Dependencies converge to a single get_db; all routes use it.

Also update the alembic env.py files (app/location/poo) and tests that
imported the removed modules so the suite stays green, and drop the obsolete
test_legacy_style_location_db test whose flow (app reading a separate location
DB) no longer exists. Location/poo Alembic chains, adopt scripts and adoption
tests remain for M1-T04; config fields remain for M1-T05.

pytest 109 passed; ruff clean (pre-existing only); WAL verified; single
Base.metadata holds all seven tables.
2026-06-12 16:35:07 +02:00
tliu93 bc8dd062d5 M1-T02: add idempotent legacy data migration script
scripts/migrate_legacy_data.py copies rows from the legacy locationRecorder.db
/ pooRecorder.db into the unified app DB's location / poo_records tables using
ATTACH + INSERT OR IGNORE (idempotent via PK-conflict skip; explicit columns,
never SELECT *). After copy it reconciles every source row against the target
and raises / exits non-zero on any shortfall. Missing legacy files are a safe
no-op (skipped); --dry-run writes nothing. Not part of the Alembic chain; run
manually once at cut-over. Never deletes or overwrites any file.

Validated end-to-end on copies of the real production DBs: dry-run reported
75103 location + 874 poo rows and wrote nothing; the real run copied all rows
with reconciliation passing; a second run copied 0 (idempotent).
2026-06-12 16:13:55 +02:00
tliu93 427a491380 M1-T01: add app-chain revision creating location + poo_records tables
Add alembic_app revision 20260611_06_merge_location_poo_tables that builds
empty location and poo_records tables (REAL float columns, matching the
production schema and the adopt scripts' EXPECTED_*_TABLE_INFO constants).
Update APP_BASELINE_REVISION to the new head. Schema-only; data migration
is handled separately by scripts/migrate_legacy_data.py (M1-T02).
2026-06-12 16:02:46 +02:00
tliu93 b359bbe3bf docs: add next-phase roadmap, milestone design docs, and CLAUDE.md
pytest / test (push) Successful in 54s
- roadmap.md: M1 (DB consolidation) -> M2 (React SPA) -> M3 (token/mobile)
- docs/design/: agent-pipeline design docs with atomic tasks for M1-M3
- CLAUDE.md: workflow, doc map, commit conventions, review-notes briefing flow
- .gitignore: ignore local review-notes/
2026-06-12 15:37:17 +02:00
tliu93 636bb2b80b Merge pull request 'add get public and storage feature' (#6) from feature/public_ip into main
pytest / test (push) Successful in 53s
docker-image / build-and-push (push) Successful in 3m59s
Reviewed-on: #6
2026-04-29 13:16:58 +02:00
tliu93 eda49489e0 update reademe and docs
pytest / test (push) Successful in 56s
pytest / test (pull_request) Successful in 59s
2026-04-29 13:07:59 +02:00
tliu93 779e160b95 add ip change notification and refine sender display
pytest / test (push) Successful in 57s
pytest / test (pull_request) Successful in 54s
2026-04-29 13:03:12 +02:00
tliu93 3ea3498e58 add smtp module and testing 2026-04-29 12:11:10 +02:00
tliu93 5a420bd37b add get public and storage feature 2026-04-29 11:45:49 +02:00
tliu93 a24e402d47 add grafana provisioning
pytest / test (push) Successful in 46s
2026-04-23 00:12:51 +02:00
69 changed files with 3762 additions and 1707 deletions
-2
View File
@@ -4,8 +4,6 @@ APP_NAME=Home Automation Backend (Python)
APP_ENV=production
APP_HOSTNAME=home-automation.example.com
APP_DATABASE_URL=sqlite:////app/data/app.db
LOCATION_DATABASE_URL=sqlite:////app/data/locationRecorder.db
POO_DATABASE_URL=sqlite:////app/data/pooRecorder.db
AUTH_BOOTSTRAP_USERNAME=admin
AUTH_BOOTSTRAP_PASSWORD=change-me
+1
View File
@@ -5,3 +5,4 @@
__pycache__/
*.pyc
data/
review-notes/
+137
View File
@@ -0,0 +1,137 @@
# CLAUDE.md — Home Automation Backend
本文件每次会话自动加载。它定义本项目的**工作流程、文档位置、commit 规范**。请在动手前先读完。
## 项目速览
- 个人用 home-automation 后端:**FastAPI + SQLite + SQLAlchemy + Alembic**,服务端模板(JinjaM2 将换成 React SPA)。
- 单 admin 鉴权(Argon2 + server-side session cookie),runtime config 落 `app_config` 表。
- 模块:public IPv4 monitor、SMTP 通知、location recorder、poo recorder、Home Assistant in/out、TickTick OAuth。
- 已发布 `v1.0.3`。下一阶段方向:**M1 单库化 → M2 React 前端 → M3 token/移动端(远期,M2 后再说)**。
- **当前现实**:在 M1 完成前仍是**三个独立 SQLite 库**app / location / poo),三套 DeclarativeBase、三条 Alembic 链。不要假设已经单库——以代码现状为准。
- 明确不做:Notion 模块。
## 文档地图与「开工前必读」
文档都在 `docs/`
| 路径 | 作用 |
| --- | --- |
| `docs/roadmap.md` | 全局规划与里程碑总览 |
| `docs/design/README.md` | **协作契约**:任务卡格式、原子任务定义、校验闸门、数据安全红线 |
| `docs/design/m1-db-consolidation.md` | M1 原子任务(含真实代码现状盘点 + 人工 runbook) |
| `docs/design/m2-frontend-v2.md` | M2 原子任务 + API 契约 + 前端校验闸门 |
| `docs/design/m3-token-mobile.md` | M3(远期,暂缓) |
| `docs/*.md`auth / public-ip-monitor / location-recorder …) | 各模块说明,按需读 |
**开工时读取顺序**
1. `docs/design/README.md`(每轮都读,它是流程与验收的共同契约)。
2. 本轮对应的 milestone 文档(如 `docs/design/m1-db-consolidation.md`),定位要做的任务卡。
3. 任务卡 `Files` 列出的源文件 + 该模块的 `docs/*.md`(按需)。
4. `docs/roadmap.md` 仅在需要全局视角时读。
## 工作流程
### 实现模式(由用户的提示词决定)
- **默认逐步**:给一个 milestone 文档,按其中原子任务**一步一步**实现。
- **(a) 只实现一步**:用户说"只实现一步 / 这一个任务"时,**只做那一个任务卡**,跑完校验闸门后停下,等用户确认,不要顺手往下做。
- **(b) 完成整个 milestone**:仅当用户在提示词里**显式要求启用 sub-agent 并指定模型**时,才用指定模型起 implementer sub-agent,按任务依赖顺序跑完整条链。
- **Sub-agent 纪律**:只在用户显式要求时才 spawn sub-agent;单步/小改动在主线内联完成。起 sub-agent 时用用户**指定的模型**Agent 工具的 `model` 覆盖)。
### 角色(Orchestrator → Implementer → Reviewer
- 我(主线)= **Orchestrator**:挑依赖已满足的下一个任务、派发、转述结果、维护任务 `Status`
- **Implementer**(便宜模型,用户指定):一次一个任务,严格按任务卡,不扩范围。
- **Reviewer**(强模型,用户指定):实现完成后起 Reviewer sub-agent,按任务卡 `Acceptance criteria` + `Reviewer checklist` 复核、**独立重跑校验闸门**,驱动 implementer 返工直到本轮 PASS。
### 校验闸门(每个任务结束都要全绿)
根目录、激活 `.venv` 后:
```bash
pytest # 权威闸门(CI 跑的就是它)
ruff check . # line-length=100
python scripts/export_openapi.py && git diff --exit-code openapi/ # 改了路由/schema 才需要,且产物须入库
```
前端任务(M2)在 `frontend/` 下另跑 `npm run lint && npm run typecheck && npm run test && npm run build`(详见 m2 文档 §8)。
**不过闸门就不算完成**,不得跳过、不得留红给下一轮。
## 每轮简报(`review-notes/`
每轮工作都要在 `review-notes/` 下产出**中文简报**。该目录**已在 `.gitignore` 忽略**,纯本地、不入库——它是 agent 之间和与人之间的交接载体,不是仓库产物。
- **实现 / 返工简报**:每轮实现完成后(无论首次实现还是返工),写一份。文件名建议 `<task-id>-impl-<n>.md` / `<task-id>-rework-<n>.md`(如 `M1-T03-impl-1.md``M1-T03-rework-1.md`)。至少包含:
1. **本轮修改的具体内容**(改了哪些文件、做了什么、为什么)。
2. **自动化测试结果**`pytest` / `ruff` / 前端闸门的实际输出或结论,通过/失败逐项写清)。
3. **若需人工 walkthrough**:写明具体步骤(怎么启动、点哪里、预期看到什么);若无需人工验证,明确写"无需人工 walkthrough"。
- **review 简报**:每轮 review 后写一份,文件名建议 `<task-id>-review-<n>.md`(如 `M1-T03-review-1.md`)。至少包含:评审结论(`PASS` 或带编号的返工清单)、对照任务卡 `Acceptance criteria` + `Reviewer checklist` 的逐条核对、reviewer 独立重跑校验闸门的结果。
**用途**:① reviewer 审核时参考对应的实现简报;② implementer 返工时参考对应的 review 简报;③ 人类(用户)通读这些简报确认有无问题。简报之间用文件名里的 `<task-id>` 与轮次 `<n>` 对应起来。
### Orchestrator 派发契约(让简报真正被读到)
**关键**:sub-agent 冷启动、不继承主线上下文,**不会因为本文件提到简报就自动去读**对应文件。简报能流转,靠的是 orchestrator(主线)在**每次 spawn 时把路径显式写进 prompt**,而不是被动约定。所以派发时必须做到:
- **显式告诉它「先读哪个简报」**:
- 派 implementer 做**首次实现** → 传任务卡位置(milestone 文档路径 + task id);无前置简报。
- 派 implementer 做**返工** → 必须传对应的 `review-notes/<task>-review-<n>.md` 路径,并要求**先读它**再改。
- 派 reviewer → 必须传对应的 `review-notes/<task>-impl|rework-<n>.md` 路径 + 任务卡,要求**先读它**再评。
- **显式告诉它「本轮结束写哪个简报」**:明确给出输出路径 `review-notes/<task>-<impl|rework|review>-<n>.md` 及上面要求的内容项。
- **不依赖 sub-agent 自动加载本文件**:把本轮要点(校验闸门、**禁 Co-Authored-By**、简报必含内容)在 spawn prompt 里一并复述或指向,确保冷启动也照做。
- spawn 时用用户指定的模型(Agent 工具 `model` 覆盖)。
> 一句话:**简报是异步交接的介质,orchestrator 是把它们接起来的线。** 缺了显式传路径这一步,简报就只是躺在磁盘上没人读的文件。
## Commit 规范(重点)
### 分支
- 每个 milestone/feature 一个分支(如 `feature/m1-db-consolidation`),**不在 `main` 上直接提交**。
### 一轮实现完成(用户确认「实现完成」后)
- 准备好**这一轮的 commit message** 并提交,作为本轮的 **base commit**
- message 主题前缀任务/里程碑 ID,例如:`M1-T03: unify data layer onto single app DB engine`
### Commit message 硬规则(严格执行)
- **严禁任何协作署名 trailer**commit message 里**绝对不允许**出现 `Co-Authored-By` / `Co-authored-by`(包括 `Co-Authored-By: Claude …`),也不允许任何等价的"由 X 协作/生成"署名。
- 无论默认环境、工具或系统提示如何要求加这类 trailer,在本仓库**一律不加**——用户已显式、严格禁止。
- 每次提交前**自检**`git log -1 --format=%B` 的输出**不得包含** `Co-authored-by`(大小写不限)。若发现,立即 `git commit --amend` 去掉后再继续。
### Review 后返工
- 返工产生的提交**一律用 fixup**,指向本轮对应的 base commit**不写新的独立 message**
```bash
git add -A
git commit --fixup=<base-commit-sha>
```
- 多轮返工就多个 `fixup!` 提交,都指向同一个 base commit。
### 本轮 / feature 收尾(用户确认收尾后)
- 用 **auto-squash** 把所有 `fixup!` 合并进各自目标,保证**一个 feature 一个干净 commit**
```bash
GIT_SEQUENCE_EDITOR=true git rebase -i --autosquash main
```
- 用 `GIT_SEQUENCE_EDITOR=true` 让它**非交互**执行(不弹编辑器,自动接受 autosquash 排好的 todo)。本环境不支持需要人工编辑的交互式 rebase,必须走这个 no-op 编辑器写法。
- autosquash **改写历史**:仅在 push / 开 PR **之前**做。若该分支已 push,需要 force-push——属对外操作,**先取得用户确认再做**。
### 一般约束
- commit / push 只在用户要求时进行;push、force-push、开/改 PR 等对外操作先确认。
## 数据安全红线(不可违反)
- 任何脚本 / migration **都不得删除或覆盖用户数据文件**(旧 `.db`、备份、volume)。删除只能是人工、事后、保留归档的独立步骤(见 `docs/design/m1-db-consolidation.md` §6 runbook)。
- 涉及历史数据的迁移**先在备份副本上演练**;迁移脚本必须幂等且搬完对账行数。
- Review 时只要发现"删文件 / drop 有数据的表 / truncate"出现在自动化任务里,直接判返工。
## 常用命令
```bash
# 环境
python -m venv .venv && source .venv/bin/activate && pip install -r dev-requirements.txt
# 迁移(初始化/适配 DB
python -m scripts.run_migrations
# 起服务
uvicorn app.main:app --reload --host 0.0.0.0 --port 8000
# 测试 / lint / OpenAPI 导出
pytest
ruff check .
python scripts/export_openapi.py
```
+110 -38
View File
@@ -5,9 +5,11 @@
当前系统已经包含:
- FastAPI Web 应用与服务端模板页面
- SQLite + SQLAlchemy + Alembic 的库结构
- SQLite + SQLAlchemy + Alembic 的库结构
- username/password + server-side session 鉴权
- runtime config 页面与 app DB 持久化
- public IPv4 monitor、历史持久化与定时检查
- SMTP 配置、测试发信与 public IPv4 changed 邮件通知
- location recorder
- poo recorder
- Home Assistant inbound / outbound integration
@@ -21,40 +23,32 @@
## 当前配置现实
当前系统仍然是三个独立的 SQLite 数据库文件,而不是单一数据库
当前系统使用单一 SQLite 数据库文件`app.db`),所有数据表都在其中
- `app` 级共享数据使用自己的 DB 文件
- `location` 模块使用自己的 DB 文件
- `poo` 模块使用自己的 DB 文件
- auth(单个 admin 用户、server-side session
- runtime config 持久化(`app_config` 表)
- public IPv4 当前状态与变化历史
- location 记录(`location` 表)
- poo 记录(`poo_records` 表)
当前阶段明确不借这次重构把这些 DB 合并。配置层已经显式反映这一点
配置层只保留一个数据库环境变量
- `APP_DATABASE_URL`
- `LOCATION_DATABASE_URL`
- `POO_DATABASE_URL`
目前 auth、`location``poo` 都已经接到各自独立的数据库文件。
`app.db` 不会在应用启动时自动创建,需要先运行:
其中 `app` 级共享 DB 当前主要用于:
```bash
python -m scripts.run_migrations
```
- 单个 admin 用户
- server-side session
- runtime config 持久化
这部分现在也使用 Alembic 管理:
- `app db` 不会在应用启动时自动创建
- 需要先运行 `python scripts/app_db_adopt.py`
- 这个脚本会创建新 DB 并建好 schema
该命令会通过 Alembic 将 `app.db` 初始化或升级到最新 head(含 `location` / `poo_records` 表)。
## 当前目录
主要目录如下:
- `app/`: FastAPI 应用代码
- `alembic_app/`: App DB 的 Alembic migration 环境
- `alembic_location/`: Location DB 的 Alembic migration 环境
- `alembic_poo/`: Poo DB 的 Alembic migration 环境
- `alembic_app/`: App DB 的 Alembic migration 环境(同时管理 `location` / `poo_records` 表)
- `tests/`: pytest 测试
- `docs/`: 当前系统说明文档
- `scripts/`: 辅助脚本,例如 OpenAPI 导出
@@ -125,24 +119,22 @@ uvicorn app.main:app --reload --host 0.0.0.0 --port 8000
## 数据库与 Alembic
当前默认使用 SQLite,并区分三个数据库文件:
当前使用单一 SQLite 数据库文件:
- App DB`sqlite:///./data/app.db`
- Location DB`sqlite:///./data/locationRecorder.db`
- Poo DB`sqlite:///./data/pooRecorder.db`
- 数据目录:`./data/`
初始化 migration 环境后,可继续添加模型并生成迁移
所有模型(auth / config / public_ip / location / poo)共用同一个 `Base`,均通过单一 Alembic 链管理
当前 `app``location` `poo` 都已经有各自独立的 Alembic 链路。
- App Alembic 环境:`alembic_app.ini` + `alembic_app/`
- Location Alembic 环境:`alembic_location.ini` + `alembic_location/`
- Poo Alembic 环境:`alembic_poo.ini` + `alembic_poo/`
- Alembic 环境:`alembic_app.ini` + `alembic_app/`
- 统一 migration job`python -m scripts.run_migrations`
- App DB 初始化:`python scripts/app_db_adopt.py`
- Location DB 接管 / 初始化:`python scripts/location_db_adopt.py`
- Poo DB 接管 / 初始化:`python scripts/poo_db_adopt.py`
- App DB 接管 / 初始化:`python scripts/app_db_adopt.py`
历史 location / poo 数据(旧版本遗留的独立 DB 文件)已通过以下脚本一次性迁移至 `app.db`(幂等,不删除旧文件):
```bash
python -m scripts.migrate_legacy_data
```
## 基础鉴权
@@ -194,11 +186,84 @@ uvicorn app.main:app --reload --host 0.0.0.0 --port 8000
这意味着:
- location / poo / app DB 地址仍然属于 bootstrap 范畴
- app DB 地址`APP_DATABASE_URL`仍然属于 bootstrap 范畴
- 运行时可编辑配置主要通过 `app_config` 表持久化
- token / secret 这类运行时必须可取回的配置,目前允许明文存储在 config 表中
- 登录密码仍然单独使用 Argon2 哈希,不走 config 表明文存储
当前已经接入 config 页面的运行时配置包括:
- 基础系统配置
- auth cookie 相关配置
- SMTP 基础配置
- TickTick OAuth 配置
- Home Assistant 配置
其中 SMTP password 与其他 secret 字段一致:
- 页面不明文回显
- 留空提交时保留旧值
- 用于测试发信与自动通知时不会写入响应
## Public IPv4 Monitor
当前系统已经提供最小可用的 public IPv4 monitor
- 使用单一 provider 检查当前公网 IPv4
- 将状态与变化历史持久化到 app DB
- 提供受保护的手动检查入口:`GET /public-ip/check`
- 启动时注册 APScheduler job,默认每 4 小时检查一次
当前 app DB 中与此功能相关的新表:
- `public_ip_state`
- `public_ip_history`
状态语义如下:
- `first_seen`:首次发现当前公网 IPv4
- `unchanged`:与上次状态一致
- `changed`:公网 IPv4 发生变化
- `error`:provider 请求失败或返回无效值
## SMTP 与邮件通知
当前系统已经提供最小可用的 SMTP 能力:
- SMTP 配置可在 `/config` 页面填写并保存到 `app_config`
- 可通过 config 页面发送测试邮件
- 邮件 `From` 头支持显示名,例如 `Home Automation <sender@example.com>`
当前 SMTP 配置项包括:
- `SMTP_ENABLED`
- `SMTP_HOST`
- `SMTP_PORT`
- `SMTP_USERNAME`
- `SMTP_PASSWORD`
- `SMTP_FROM_NAME`
- `SMTP_FROM_ADDRESS`
- `SMTP_TO_ADDRESS`
- `SMTP_USE_STARTTLS`
当前 public IPv4 monitor 已与 SMTP sender 接通,但只处理一个很小的通知场景:
- 当 public IPv4 check 结果为 `changed` 时,自动发送一封英文纯文本邮件
以下情况不会发邮件:
- `first_seen`
- `unchanged`
- `error`
当前通知邮件内容固定,不提供模板系统,正文会包含:
- previous IP
- current IP
- detected time
手动测试时,如果需要再次模拟一次 IP 变化,可以临时修改 `public_ip_state.current_ipv4` 为一个保留测试地址,然后再次调用 `GET /public-ip/check`
## OpenAPI
可使用下面的脚本重新导出当前 API 定义:
@@ -286,9 +351,16 @@ pytest
当前测试包含:
- app 基本启动测试
- `/status` endpoint 测试
- 登录 / session 基础流程测试
- app 启动与 `/status` 检查
- 登录 / session / 鉴权流程
- runtime config 读写
- public IPv4 monitor
- SMTP 配置与测试发信
- location / poo recorder 端点
- Home Assistant inbound 集成
- TickTick OAuth
- 部署与迁移(`run_migrations`
- legacy 数据迁移脚本(`migrate_legacy_data`
## OpenAPI 导出
+5 -2
View File
@@ -3,10 +3,13 @@ from logging.config import fileConfig
from alembic import context
from sqlalchemy import engine_from_config, pool
from app.auth_db import AuthBase
from app.config import get_settings
from app.db import Base
from app.models.config import AppConfigEntry # noqa: F401
from app.models.auth import AuthSession, AuthUser # noqa: F401
from app.models.public_ip import PublicIPHistory, PublicIPState # noqa: F401
from app.models.location import Location # noqa: F401
from app.models.poo import PooRecord # noqa: F401
config = context.config
@@ -18,7 +21,7 @@ configured_url = config.get_main_option("sqlalchemy.url")
if not configured_url or configured_url == "sqlite:///./data/app.db":
config.set_main_option("sqlalchemy.url", settings.app_database_url)
target_metadata = AuthBase.metadata
target_metadata = Base.metadata
def run_migrations_offline() -> None:
@@ -0,0 +1,55 @@
"""public ip monitor tables
Revision ID: 20260429_05_public_ip_monitor
Revises: 20260420_04_app_config_table
Create Date: 2026-04-29 00:00:01.000000
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
revision: str = "20260429_05_public_ip_monitor"
down_revision: Union[str, None] = "20260420_04_app_config_table"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.create_table(
"public_ip_history",
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
sa.Column("ipv4", sa.String(length=45), nullable=False),
sa.Column("observed_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("change_type", sa.String(length=32), nullable=False),
sa.Column("provider", sa.String(length=64), nullable=True),
sa.PrimaryKeyConstraint("id"),
)
op.create_index(
"ix_public_ip_history_observed_at",
"public_ip_history",
["observed_at"],
unique=False,
)
op.create_table(
"public_ip_state",
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
sa.Column("current_ipv4", sa.String(length=45), nullable=False),
sa.Column("previous_ipv4", sa.String(length=45), nullable=True),
sa.Column("first_seen_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("last_checked_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("last_changed_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("last_check_status", sa.String(length=32), nullable=False),
sa.Column("last_check_error", sa.String(length=255), nullable=True),
sa.Column("last_provider", sa.String(length=64), nullable=True),
sa.PrimaryKeyConstraint("id"),
)
def downgrade() -> None:
op.drop_table("public_ip_state")
op.drop_index("ix_public_ip_history_observed_at", table_name="public_ip_history")
op.drop_table("public_ip_history")
@@ -0,0 +1,43 @@
"""merge location and poo_records tables into app chain
Revision ID: 20260611_06_merge_location_poo_tables
Revises: 20260429_05_public_ip_monitor
Create Date: 2026-06-11 00:00:01.000000
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
revision: str = "20260611_06_merge_location_poo_tables"
down_revision: Union[str, None] = "20260429_05_public_ip_monitor"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.create_table(
"location",
sa.Column("person", sa.Text(), nullable=False),
sa.Column("datetime", sa.Text(), nullable=False),
sa.Column("latitude", sa.REAL(), nullable=False),
sa.Column("longitude", sa.REAL(), nullable=False),
sa.Column("altitude", sa.REAL(), nullable=True),
sa.PrimaryKeyConstraint("person", "datetime"),
)
op.create_table(
"poo_records",
sa.Column("timestamp", sa.Text(), nullable=False),
sa.Column("status", sa.Text(), nullable=False),
sa.Column("latitude", sa.REAL(), nullable=False),
sa.Column("longitude", sa.REAL(), nullable=False),
sa.PrimaryKeyConstraint("timestamp"),
)
def downgrade() -> None:
op.drop_table("poo_records")
op.drop_table("location")
-37
View File
@@ -1,37 +0,0 @@
[alembic]
script_location = alembic_location
prepend_sys_path = .
path_separator = os
sqlalchemy.url = sqlite:///./data/locationRecorder.db
[loggers]
keys = root,sqlalchemy,alembic
[handlers]
keys = console
[formatters]
keys = generic
[logger_root]
level = WARN
handlers = console
[logger_sqlalchemy]
level = WARN
handlers =
qualname = sqlalchemy.engine
[logger_alembic]
level = INFO
handlers =
qualname = alembic
[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic
[formatter_generic]
format = %(levelname)-5.5s [%(name)s] %(message)s
-2
View File
@@ -1,2 +0,0 @@
This directory contains the Alembic migration environment for the Python rewrite skeleton.
-48
View File
@@ -1,48 +0,0 @@
from logging.config import fileConfig
from alembic import context
from sqlalchemy import engine_from_config, pool
from app.config import get_settings
from app.models import Location # noqa: F401
from app.models.base import Base
config = context.config
if config.config_file_name is not None:
fileConfig(config.config_file_name)
settings = get_settings()
configured_url = config.get_main_option("sqlalchemy.url")
if not configured_url or configured_url == "sqlite:///./data/locationRecorder.db":
config.set_main_option("sqlalchemy.url", settings.location_database_url)
target_metadata = Base.metadata
def run_migrations_offline() -> None:
url = config.get_main_option("sqlalchemy.url")
context.configure(url=url, target_metadata=target_metadata, literal_binds=True)
with context.begin_transaction():
context.run_migrations()
def run_migrations_online() -> None:
connectable = engine_from_config(
config.get_section(config.config_ini_section, {}),
prefix="sqlalchemy.",
poolclass=pool.NullPool,
)
with connectable.connect() as connection:
context.configure(connection=connection, target_metadata=target_metadata)
with context.begin_transaction():
context.run_migrations()
if context.is_offline_mode():
run_migrations_offline()
else:
run_migrations_online()
-26
View File
@@ -1,26 +0,0 @@
"""${message}
Revision ID: ${up_revision}
Revises: ${down_revision | comma,n}
Create Date: ${create_date}
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision: str = ${repr(up_revision)}
down_revision: Union[str, None] = ${repr(down_revision)}
branch_labels: Union[str, Sequence[str], None] = ${repr(branch_labels)}
depends_on: Union[str, Sequence[str], None] = ${repr(depends_on)}
def upgrade() -> None:
${upgrades if upgrades else "pass"}
def downgrade() -> None:
${downgrades if downgrades else "pass"}
-1
View File
@@ -1 +0,0 @@
@@ -1,33 +0,0 @@
"""location baseline
Revision ID: 20260419_01_location_baseline
Revises:
Create Date: 2026-04-19 00:00:00.000000
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
revision: str = "20260419_01_location_baseline"
down_revision: Union[str, None] = None
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.create_table(
"location",
sa.Column("person", sa.Text(), nullable=False),
sa.Column("datetime", sa.Text(), nullable=False),
sa.Column("latitude", sa.Float(), nullable=False),
sa.Column("longitude", sa.Float(), nullable=False),
sa.Column("altitude", sa.Float(), nullable=True),
sa.PrimaryKeyConstraint("person", "datetime"),
)
def downgrade() -> None:
op.drop_table("location")
-37
View File
@@ -1,37 +0,0 @@
[alembic]
script_location = alembic_poo
prepend_sys_path = .
path_separator = os
sqlalchemy.url = sqlite:///./data/pooRecorder.db
[loggers]
keys = root,sqlalchemy,alembic
[handlers]
keys = console
[formatters]
keys = generic
[logger_root]
level = WARN
handlers = console
[logger_sqlalchemy]
level = WARN
handlers =
qualname = sqlalchemy.engine
[logger_alembic]
level = INFO
handlers = console
qualname = alembic
[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic
[formatter_generic]
format = %(levelname)-5.5s [%(name)s] %(message)s
-48
View File
@@ -1,48 +0,0 @@
from logging.config import fileConfig
from alembic import context
from sqlalchemy import engine_from_config, pool
from app.config import get_settings
from app.models.poo import PooRecord # noqa: F401
from app.poo_db import PooBase
config = context.config
if config.config_file_name is not None:
fileConfig(config.config_file_name)
settings = get_settings()
configured_url = config.get_main_option("sqlalchemy.url")
if not configured_url or configured_url == "sqlite:///./data/pooRecorder.db":
config.set_main_option("sqlalchemy.url", settings.poo_database_url)
target_metadata = PooBase.metadata
def run_migrations_offline() -> None:
url = config.get_main_option("sqlalchemy.url")
context.configure(url=url, target_metadata=target_metadata, literal_binds=True)
with context.begin_transaction():
context.run_migrations()
def run_migrations_online() -> None:
connectable = engine_from_config(
config.get_section(config.config_ini_section, {}),
prefix="sqlalchemy.",
poolclass=pool.NullPool,
)
with connectable.connect() as connection:
context.configure(connection=connection, target_metadata=target_metadata)
with context.begin_transaction():
context.run_migrations()
if context.is_offline_mode():
run_migrations_offline()
else:
run_migrations_online()
@@ -1,32 +0,0 @@
"""poo baseline
Revision ID: 20260420_01_poo_baseline
Revises:
Create Date: 2026-04-20 00:00:00.000000
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
revision: str = "20260420_01_poo_baseline"
down_revision: Union[str, None] = None
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.create_table(
"poo_records",
sa.Column("timestamp", sa.Text(), nullable=False),
sa.Column("status", sa.Text(), nullable=False),
sa.Column("latitude", sa.Float(), nullable=False),
sa.Column("longitude", sa.Float(), nullable=False),
sa.PrimaryKeyConstraint("timestamp"),
)
def downgrade() -> None:
op.drop_table("poo_records")
+4 -4
View File
@@ -7,7 +7,7 @@ from fastapi.templating import Jinja2Templates
from sqlalchemy.orm import Session
from app.config import Settings
from app.dependencies import get_app_settings, get_auth_db, get_current_auth_session
from app.dependencies import get_app_settings, get_db, get_current_auth_session
from app.services.auth import (
AuthenticatedSession,
authenticate_user,
@@ -57,7 +57,7 @@ def login_submit(
username: str = Form(),
password: str = Form(),
csrf_token: str = Form(),
session: Session = Depends(get_auth_db),
session: Session = Depends(get_db),
settings: Settings = Depends(get_app_settings),
) -> Response:
cookie_csrf_token = request.cookies.get(LOGIN_CSRF_COOKIE_NAME)
@@ -102,7 +102,7 @@ def change_password_submit(
new_password: str = Form(),
confirm_password: str = Form(),
csrf_token: str = Form(),
session: Session = Depends(get_auth_db),
session: Session = Depends(get_db),
settings: Settings = Depends(get_app_settings),
current_auth: AuthenticatedSession | None = Depends(get_current_auth_session),
) -> Response:
@@ -151,7 +151,7 @@ def change_password_submit(
def logout(
request: Request,
csrf_token: str = Form(),
session: Session = Depends(get_auth_db),
session: Session = Depends(get_db),
settings: Settings = Depends(get_app_settings),
current_auth: AuthenticatedSession | None = Depends(get_current_auth_session),
) -> RedirectResponse:
+1 -3
View File
@@ -11,7 +11,6 @@ from app.dependencies import (
get_app_settings,
get_db,
get_homeassistant_client,
get_poo_db,
get_ticktick_client,
)
from app.integrations.homeassistant import (
@@ -36,7 +35,6 @@ INTERNAL_SERVER_ERROR_MESSAGE = "internal server error"
async def publish_from_homeassistant(
request: Request,
db: Session = Depends(get_db),
poo_db: Session = Depends(get_poo_db),
settings: Settings = Depends(get_app_settings),
homeassistant_client: HomeAssistantClient = Depends(get_homeassistant_client),
ticktick_client: TickTickClient = Depends(get_ticktick_client),
@@ -49,7 +47,7 @@ async def publish_from_homeassistant(
db,
envelope,
ticktick_client=ticktick_client,
poo_session=poo_db,
poo_session=db,
settings=settings,
homeassistant_client=homeassistant_client,
)
+138 -49
View File
@@ -6,7 +6,7 @@ from fastapi.responses import HTMLResponse, RedirectResponse, Response
from fastapi.templating import Jinja2Templates
from app.config import Settings, get_settings
from app.dependencies import get_app_settings, get_auth_db, get_current_auth_session
from app.dependencies import get_app_settings, get_db, get_current_auth_session
from app.services.auth import AuthenticatedSession
from app.services.config_page import (
ConfigSaveError,
@@ -14,6 +14,7 @@ from app.services.config_page import (
is_ticktick_oauth_ready,
save_config_updates,
)
from app.services.email import EmailConfigurationError, EmailDeliveryError, is_smtp_ready, send_smtp_test_email
from sqlalchemy.orm import Session
templates = Jinja2Templates(directory=str(Path(__file__).resolve().parents[2] / "templates"))
@@ -33,6 +34,49 @@ def _ticktick_oauth_notice(status_value: str | None) -> tuple[str | None, str |
return None, None
def _smtp_test_notice(status_value: str | None) -> tuple[str | None, str | None]:
if status_value == "success":
return "SMTP test email sent successfully.", None
if status_value == "config-error":
return None, "SMTP test failed. Check required SMTP settings before sending a test email."
if status_value == "failed":
return None, "SMTP test failed. Check saved SMTP settings and server reachability."
return None, None
def _build_config_context(
*,
auth_db_session: Session,
settings: Settings,
current_auth: AuthenticatedSession,
config_saved: bool,
config_error: str | None,
password_change_error: str | None,
ticktick_oauth_notice: str | None,
ticktick_oauth_error: str | None,
smtp_test_notice: str | None,
smtp_test_error: str | None,
) -> dict[str, object]:
return {
"app_name": settings.app_name,
"app_env": settings.app_env,
"current_username": current_auth.user.username,
"csrf_token": current_auth.session.csrf_token,
"force_password_change": current_auth.user.force_password_change,
"password_change_error": password_change_error,
"config_error": config_error,
"config_saved": config_saved,
"config_sections": build_config_sections(auth_db_session, settings),
"ticktick_oauth_ready": is_ticktick_oauth_ready(settings),
"ticktick_redirect_uri": settings.ticktick_redirect_uri,
"ticktick_oauth_notice": ticktick_oauth_notice,
"ticktick_oauth_error": ticktick_oauth_error,
"smtp_test_ready": is_smtp_ready(settings),
"smtp_test_notice": smtp_test_notice,
"smtp_test_error": smtp_test_error,
}
@router.get("/", response_class=HTMLResponse)
def home(
request: Request,
@@ -56,7 +100,7 @@ def admin_redirect(
@router.get("/config", response_class=HTMLResponse)
def config_page(
request: Request,
auth_db_session: Session = Depends(get_auth_db),
auth_db_session: Session = Depends(get_db),
settings: Settings = Depends(get_app_settings),
current_auth: AuthenticatedSession | None = Depends(get_current_auth_session),
) -> Response:
@@ -66,29 +110,26 @@ def config_page(
ticktick_oauth_notice, ticktick_oauth_error = _ticktick_oauth_notice(
request.query_params.get("ticktick_oauth")
)
context = {
"app_name": settings.app_name,
"app_env": settings.app_env,
"current_username": current_auth.user.username,
"csrf_token": current_auth.session.csrf_token,
"force_password_change": current_auth.user.force_password_change,
"password_change_error": None,
"config_error": None,
"config_saved": request.query_params.get("saved") == "1",
"config_sections": build_config_sections(auth_db_session, settings),
"ticktick_oauth_ready": is_ticktick_oauth_ready(settings),
"ticktick_redirect_uri": settings.ticktick_redirect_uri,
"ticktick_oauth_notice": ticktick_oauth_notice,
"ticktick_oauth_error": ticktick_oauth_error,
}
smtp_test_notice, smtp_test_error = _smtp_test_notice(request.query_params.get("smtp_test"))
context = _build_config_context(
auth_db_session=auth_db_session,
settings=settings,
current_auth=current_auth,
config_saved=request.query_params.get("saved") == "1",
config_error=None,
password_change_error=None,
ticktick_oauth_notice=ticktick_oauth_notice,
ticktick_oauth_error=ticktick_oauth_error,
smtp_test_notice=smtp_test_notice,
smtp_test_error=smtp_test_error,
)
return templates.TemplateResponse(request, "config.html", context)
@router.post("/config", response_class=HTMLResponse)
async def config_submit(
request: Request,
auth_db_session: Session = Depends(get_auth_db),
auth_db_session: Session = Depends(get_db),
settings: Settings = Depends(get_app_settings),
current_auth: AuthenticatedSession | None = Depends(get_current_auth_session),
) -> Response:
@@ -99,21 +140,18 @@ async def config_submit(
csrf_token = form.get("csrf_token")
if csrf_token != current_auth.session.csrf_token:
logger.warning("Rejected config update due to CSRF validation failure")
context = {
"app_name": settings.app_name,
"app_env": settings.app_env,
"current_username": current_auth.user.username,
"csrf_token": current_auth.session.csrf_token,
"force_password_change": current_auth.user.force_password_change,
"password_change_error": None,
"config_error": "invalid config update request",
"config_saved": False,
"config_sections": build_config_sections(auth_db_session, settings),
"ticktick_oauth_ready": is_ticktick_oauth_ready(settings),
"ticktick_redirect_uri": settings.ticktick_redirect_uri,
"ticktick_oauth_notice": None,
"ticktick_oauth_error": None,
}
context = _build_config_context(
auth_db_session=auth_db_session,
settings=settings,
current_auth=current_auth,
config_saved=False,
config_error="invalid config update request",
password_change_error=None,
ticktick_oauth_notice=None,
ticktick_oauth_error=None,
smtp_test_notice=None,
smtp_test_error=None,
)
return templates.TemplateResponse(
request,
"config.html",
@@ -126,21 +164,18 @@ async def config_submit(
except ConfigSaveError:
logger.warning("Rejected config update due to invalid submitted values")
refreshed_settings = get_settings()
context = {
"app_name": refreshed_settings.app_name,
"app_env": refreshed_settings.app_env,
"current_username": current_auth.user.username,
"csrf_token": current_auth.session.csrf_token,
"force_password_change": current_auth.user.force_password_change,
"password_change_error": None,
"config_error": "invalid config submission",
"config_saved": False,
"config_sections": build_config_sections(auth_db_session, refreshed_settings),
"ticktick_oauth_ready": is_ticktick_oauth_ready(refreshed_settings),
"ticktick_redirect_uri": refreshed_settings.ticktick_redirect_uri,
"ticktick_oauth_notice": None,
"ticktick_oauth_error": None,
}
context = _build_config_context(
auth_db_session=auth_db_session,
settings=refreshed_settings,
current_auth=current_auth,
config_saved=False,
config_error="invalid config submission",
password_change_error=None,
ticktick_oauth_notice=None,
ticktick_oauth_error=None,
smtp_test_notice=None,
smtp_test_error=None,
)
return templates.TemplateResponse(
request,
"config.html",
@@ -149,3 +184,57 @@ async def config_submit(
)
return RedirectResponse(url="/config?saved=1", status_code=status.HTTP_303_SEE_OTHER)
@router.post("/config/smtp/test", response_class=HTMLResponse)
async def smtp_test_submit(
request: Request,
auth_db_session: Session = Depends(get_db),
settings: Settings = Depends(get_app_settings),
current_auth: AuthenticatedSession | None = Depends(get_current_auth_session),
) -> Response:
if current_auth is None:
return RedirectResponse(url="/login", status_code=status.HTTP_303_SEE_OTHER)
form = await request.form()
csrf_token = form.get("csrf_token")
if csrf_token != current_auth.session.csrf_token:
logger.warning("Rejected SMTP test due to CSRF validation failure")
context = _build_config_context(
auth_db_session=auth_db_session,
settings=settings,
current_auth=current_auth,
config_saved=False,
config_error=None,
password_change_error=None,
ticktick_oauth_notice=None,
ticktick_oauth_error=None,
smtp_test_notice=None,
smtp_test_error="invalid SMTP test request",
)
return templates.TemplateResponse(
request,
"config.html",
context,
status_code=status.HTTP_400_BAD_REQUEST,
)
try:
send_smtp_test_email(settings)
except EmailConfigurationError as exc:
logger.warning("SMTP test email rejected due to configuration: %s", exc)
return RedirectResponse(
url="/config?smtp_test=config-error",
status_code=status.HTTP_303_SEE_OTHER,
)
except EmailDeliveryError as exc:
logger.warning("SMTP test email failed: %s", exc)
return RedirectResponse(
url="/config?smtp_test=failed",
status_code=status.HTTP_303_SEE_OTHER,
)
return RedirectResponse(
url="/config?smtp_test=success",
status_code=status.HTTP_303_SEE_OTHER,
)
+3 -3
View File
@@ -7,7 +7,7 @@ from pydantic import ValidationError
from sqlalchemy.orm import Session
from app.config import Settings
from app.dependencies import get_app_settings, get_homeassistant_client, get_poo_db
from app.dependencies import get_app_settings, get_homeassistant_client, get_db
from app.integrations.homeassistant import HomeAssistantClient
from app.schemas.poo import PooRecordRequest
from app.services.poo import publish_latest_poo_status, record_poo
@@ -21,7 +21,7 @@ INTERNAL_SERVER_ERROR_MESSAGE = "internal server error"
@router.post("/poo/record")
async def create_poo_record(
request: Request,
db: Session = Depends(get_poo_db),
db: Session = Depends(get_db),
settings: Settings = Depends(get_app_settings),
homeassistant_client: HomeAssistantClient = Depends(get_homeassistant_client),
) -> Response:
@@ -56,7 +56,7 @@ async def create_poo_record(
@router.get("/poo/latest")
def notify_latest_poo(
db: Session = Depends(get_poo_db),
db: Session = Depends(get_db),
settings: Settings = Depends(get_app_settings),
homeassistant_client: HomeAssistantClient = Depends(get_homeassistant_client),
) -> Response:
+26
View File
@@ -0,0 +1,26 @@
from fastapi import APIRouter, Depends, HTTPException, status
from sqlalchemy.orm import Session
from app.dependencies import get_db, get_current_auth_session
from app.schemas.public_ip import PublicIPCheckResponse
from app.config import get_settings
from app.services.auth import AuthenticatedSession
from app.services.public_ip import check_public_ipv4_and_notify
router = APIRouter(tags=["public-ip"])
@router.get("/public-ip/check", response_model=PublicIPCheckResponse)
def run_public_ip_check(
session: Session = Depends(get_db),
current_auth: AuthenticatedSession | None = Depends(get_current_auth_session),
) -> PublicIPCheckResponse:
if current_auth is None:
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="authentication required")
result = check_public_ipv4_and_notify(session, bootstrap_settings=get_settings())
return PublicIPCheckResponse(
status=result.status,
checked_at=result.checked_at,
changed=result.changed,
)
+2 -2
View File
@@ -7,7 +7,7 @@ from sqlalchemy.orm import Session
from app.config import Settings
from app.dependencies import (
get_app_settings,
get_auth_db,
get_db,
get_current_auth_session,
get_ticktick_client,
)
@@ -39,7 +39,7 @@ def start_ticktick_auth(
@router.get("/ticktick/auth/code")
def handle_ticktick_auth_code(
request: Request,
auth_db_session: Session = Depends(get_auth_db),
auth_db_session: Session = Depends(get_db),
settings: Settings = Depends(get_app_settings),
ticktick_client: TickTickClient = Depends(get_ticktick_client),
) -> Response:
-53
View File
@@ -1,53 +0,0 @@
from collections.abc import Generator
from functools import lru_cache
from sqlalchemy import create_engine
from sqlalchemy.orm import DeclarativeBase, Session, sessionmaker
from app.config import get_settings
class AuthBase(DeclarativeBase):
pass
def _build_connect_args(database_url: str) -> dict[str, object]:
connect_args: dict[str, object] = {}
if database_url.startswith("sqlite"):
connect_args["check_same_thread"] = False
return connect_args
@lru_cache
def _get_auth_engine(database_url: str):
return create_engine(database_url, connect_args=_build_connect_args(database_url))
@lru_cache
def _get_auth_session_local(database_url: str):
engine = _get_auth_engine(database_url)
return sessionmaker(bind=engine, autoflush=False, autocommit=False, class_=Session)
def get_auth_engine():
settings = get_settings()
return _get_auth_engine(settings.app_database_url)
def get_auth_session_local():
settings = get_settings()
return _get_auth_session_local(settings.app_database_url)
def reset_auth_db_caches() -> None:
_get_auth_session_local.cache_clear()
_get_auth_engine.cache_clear()
def get_auth_db_session() -> Generator[Session, None, None]:
session_local = get_auth_session_local()
session = session_local()
try:
yield session
finally:
session.close()
+9 -13
View File
@@ -12,9 +12,6 @@ class Settings(BaseSettings):
app_hostname: str = "localhost:8000"
app_database_url: str = "sqlite:///./data/app.db"
location_database_url: str = "sqlite:///./data/locationRecorder.db"
poo_database_url: str = "sqlite:///./data/pooRecorder.db"
ticktick_client_id: str = ""
ticktick_client_secret: str = ""
ticktick_token: str = ""
@@ -23,6 +20,15 @@ class Settings(BaseSettings):
home_assistant_auth_token: str = ""
home_assistant_timeout_seconds: float = 1.0
home_assistant_action_task_project_id: str = ""
smtp_enabled: bool = False
smtp_host: str = ""
smtp_port: int = 587
smtp_username: str = ""
smtp_password: str = ""
smtp_from_name: str = ""
smtp_from_address: str = ""
smtp_to_address: str = ""
smtp_use_starttls: bool = True
poo_webhook_id: str = ""
poo_sensor_entity_name: str = "sensor.test_poo_status"
poo_sensor_friendly_name: str = "Poo Status"
@@ -68,21 +74,11 @@ class Settings(BaseSettings):
raw_path = database_url[len(prefix) :]
return Path(raw_path)
@computed_field
@property
def location_sqlite_path(self) -> Path | None:
return self._sqlite_path_from_url(self.location_database_url)
@computed_field
@property
def app_sqlite_path(self) -> Path | None:
return self._sqlite_path_from_url(self.app_database_url)
@computed_field
@property
def poo_sqlite_path(self) -> Path | None:
return self._sqlite_path_from_url(self.poo_database_url)
@computed_field
@property
def auth_cookie_secure(self) -> bool:
+41 -8
View File
@@ -1,6 +1,8 @@
from collections.abc import Generator
from functools import lru_cache
from sqlalchemy import create_engine
from sqlalchemy import create_engine, event
from sqlalchemy.engine import Engine
from sqlalchemy.orm import DeclarativeBase, Session, sessionmaker
from app.config import get_settings
@@ -10,18 +12,49 @@ class Base(DeclarativeBase):
pass
settings = get_settings()
def _build_connect_args(database_url: str) -> dict[str, object]:
connect_args: dict[str, object] = {}
if database_url.startswith("sqlite"):
connect_args["check_same_thread"] = False
return connect_args
connect_args: dict[str, object] = {}
if settings.location_database_url.startswith("sqlite"):
connect_args["check_same_thread"] = False
engine = create_engine(settings.location_database_url, connect_args=connect_args)
SessionLocal = sessionmaker(bind=engine, autoflush=False, autocommit=False, class_=Session)
@lru_cache
def _get_engine(database_url: str) -> Engine:
engine = create_engine(database_url, connect_args=_build_connect_args(database_url))
if database_url.startswith("sqlite"):
@event.listens_for(engine, "connect")
def _enable_sqlite_wal(dbapi_connection, _connection_record):
cursor = dbapi_connection.cursor()
cursor.execute("PRAGMA journal_mode=WAL")
cursor.close()
return engine
@lru_cache
def _get_session_local(database_url: str) -> sessionmaker:
engine = _get_engine(database_url)
return sessionmaker(bind=engine, autoflush=False, autocommit=False, class_=Session)
def get_engine() -> Engine:
return _get_engine(get_settings().app_database_url)
def get_session_local() -> sessionmaker:
return _get_session_local(get_settings().app_database_url)
def reset_db_caches() -> None:
_get_session_local.cache_clear()
_get_engine.cache_clear()
def get_db_session() -> Generator[Session, None, None]:
session = SessionLocal()
session_local = get_session_local()
session = session_local()
try:
yield session
finally:
+3 -13
View File
@@ -3,30 +3,20 @@ from collections.abc import Generator
from fastapi import Depends, Request
from sqlalchemy.orm import Session
from app.auth_db import get_auth_db_session
from app.config import Settings, get_settings
from app.db import get_db_session
from app.integrations.homeassistant import HomeAssistantClient
from app.integrations.ticktick import TickTickClient
from app.poo_db import get_poo_db_session
from app.services.auth import AuthenticatedSession, get_authenticated_session
from app.services.config_page import build_runtime_settings
def get_auth_db() -> Generator[Session, None, None]:
yield from get_auth_db_session()
def get_app_settings(session: Session = Depends(get_auth_db)) -> Settings:
return build_runtime_settings(session, get_settings())
def get_db() -> Generator[Session, None, None]:
yield from get_db_session()
def get_poo_db() -> Generator[Session, None, None]:
yield from get_poo_db_session()
def get_app_settings(session: Session = Depends(get_db)) -> Settings:
return build_runtime_settings(session, get_settings())
def get_homeassistant_client(settings: Settings = Depends(get_app_settings)) -> HomeAssistantClient:
@@ -39,7 +29,7 @@ def get_ticktick_client(settings: Settings = Depends(get_app_settings)) -> TickT
def get_current_auth_session(
request: Request,
session: Session = Depends(get_auth_db),
session: Session = Depends(get_db),
settings: Settings = Depends(get_app_settings),
) -> AuthenticatedSession | None:
raw_token = request.cookies.get(settings.auth_session_cookie_name)
+29 -31
View File
@@ -3,26 +3,37 @@ from pathlib import Path
from fastapi import FastAPI
from fastapi.staticfiles import StaticFiles
from apscheduler.schedulers.background import BackgroundScheduler
from apscheduler.triggers.interval import IntervalTrigger
from sqlalchemy.orm import Session
from app import models # noqa: F401
from app.api.routes.auth import router as auth_router
from app.api.routes import pages, status
import app.auth_db as auth_db
from app.db import get_session_local
from app.api.routes.homeassistant import router as homeassistant_router
from app.api.routes.location import router as location_router
from app.api.routes.poo import router as poo_router
from app.api.routes.public_ip import router as public_ip_router
from app.api.routes.ticktick import router as ticktick_router
from app.config import get_settings
from app.services.auth import AuthBootstrapError, initialize_auth_schema
from app.services.config_page import seed_missing_config_from_bootstrap, sync_app_hostname_from_bootstrap
from app.services.public_ip import check_public_ipv4_and_notify
from scripts.app_db_adopt import AppDatabaseAdoptionError, validate_app_runtime_db
from scripts.location_db_adopt import LocationDatabaseAdoptionError, validate_location_runtime_db
from scripts.poo_db_adopt import PooDatabaseAdoptionError, validate_poo_runtime_db
def _run_scheduled_public_ip_check() -> None:
session_local = get_session_local()
session: Session = session_local()
try:
check_public_ipv4_and_notify(session, bootstrap_settings=get_settings())
finally:
session.close()
def ensure_auth_db_ready() -> None:
session_local = auth_db.get_auth_session_local()
session_local = get_session_local()
session: Session = session_local()
try:
validate_app_runtime_db(get_settings().app_database_url)
@@ -37,42 +48,28 @@ def ensure_auth_db_ready() -> None:
session.close()
def ensure_location_db_ready() -> None:
settings = get_settings()
if settings.location_sqlite_path is None:
return
try:
validate_location_runtime_db(settings.location_database_url)
except LocationDatabaseAdoptionError as exc:
raise RuntimeError(str(exc)) from exc
def ensure_poo_db_ready() -> None:
settings = get_settings()
if settings.poo_sqlite_path is None:
return
try:
validate_poo_runtime_db(settings.poo_database_url)
except PooDatabaseAdoptionError as exc:
raise RuntimeError(str(exc)) from exc
def ensure_runtime_dirs() -> None:
settings = get_settings()
for path in (settings.app_sqlite_path, settings.location_sqlite_path, settings.poo_sqlite_path):
if path is not None:
path.parent.mkdir(parents=True, exist_ok=True)
if settings.app_sqlite_path is not None:
settings.app_sqlite_path.parent.mkdir(parents=True, exist_ok=True)
@asynccontextmanager
async def lifespan(_: FastAPI):
ensure_runtime_dirs()
ensure_auth_db_ready()
ensure_location_db_ready()
ensure_poo_db_ready()
scheduler = BackgroundScheduler(timezone="UTC")
scheduler.add_job(
_run_scheduled_public_ip_check,
trigger=IntervalTrigger(hours=4),
id="public-ip-check",
replace_existing=True,
max_instances=1,
coalesce=True,
)
scheduler.start()
yield
scheduler.shutdown(wait=False)
def create_app() -> FastAPI:
@@ -97,6 +94,7 @@ def create_app() -> FastAPI:
app.include_router(homeassistant_router)
app.include_router(location_router)
app.include_router(poo_router)
app.include_router(public_ip_router)
app.include_router(ticktick_router)
return app
+11 -1
View File
@@ -3,5 +3,15 @@
from app.models.auth import AuthSession, AuthUser
from app.models.config import AppConfigEntry
from app.models.location import Location
from app.models.poo import PooRecord
from app.models.public_ip import PublicIPHistory, PublicIPState
__all__ = ["AppConfigEntry", "AuthSession", "AuthUser", "Location"]
__all__ = [
"AppConfigEntry",
"AuthSession",
"AuthUser",
"Location",
"PooRecord",
"PublicIPHistory",
"PublicIPState",
]
+3 -3
View File
@@ -3,10 +3,10 @@ from datetime import datetime
from sqlalchemy import Boolean, DateTime, ForeignKey, Integer, String
from sqlalchemy.orm import Mapped, mapped_column, relationship
from app.auth_db import AuthBase
from app.db import Base
class AuthUser(AuthBase):
class AuthUser(Base):
__tablename__ = "auth_users"
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
@@ -19,7 +19,7 @@ class AuthUser(AuthBase):
sessions: Mapped[list["AuthSession"]] = relationship(back_populates="user")
class AuthSession(AuthBase):
class AuthSession(Base):
__tablename__ = "auth_sessions"
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
-4
View File
@@ -1,4 +0,0 @@
from app.db import Base
__all__ = ["Base"]
+2 -2
View File
@@ -3,10 +3,10 @@ from datetime import datetime
from sqlalchemy import DateTime, Integer, String
from sqlalchemy.orm import Mapped, mapped_column
from app.auth_db import AuthBase
from app.db import Base
class AppConfigEntry(AuthBase):
class AppConfigEntry(Base):
__tablename__ = "app_config"
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
+2 -2
View File
@@ -1,10 +1,10 @@
from sqlalchemy import Float, String
from sqlalchemy.orm import Mapped, mapped_column
from app.poo_db import PooBase
from app.db import Base
class PooRecord(PooBase):
class PooRecord(Base):
__tablename__ = "poo_records"
timestamp: Mapped[str] = mapped_column(String, primary_key=True)
+30
View File
@@ -0,0 +1,30 @@
from datetime import datetime
from sqlalchemy import DateTime, Integer, String
from sqlalchemy.orm import Mapped, mapped_column
from app.db import Base
class PublicIPState(Base):
__tablename__ = "public_ip_state"
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
current_ipv4: Mapped[str] = mapped_column(String(45), nullable=False)
previous_ipv4: Mapped[str | None] = mapped_column(String(45), nullable=True)
first_seen_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
last_checked_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
last_changed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
last_check_status: Mapped[str] = mapped_column(String(32), nullable=False)
last_check_error: Mapped[str | None] = mapped_column(String(255), nullable=True)
last_provider: Mapped[str | None] = mapped_column(String(64), nullable=True)
class PublicIPHistory(Base):
__tablename__ = "public_ip_history"
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
ipv4: Mapped[str] = mapped_column(String(45), nullable=False)
observed_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
change_type: Mapped[str] = mapped_column(String(32), nullable=False)
provider: Mapped[str | None] = mapped_column(String(64), nullable=True)
-28
View File
@@ -1,28 +0,0 @@
from collections.abc import Generator
from sqlalchemy import create_engine
from sqlalchemy.orm import DeclarativeBase, Session, sessionmaker
from app.config import get_settings
class PooBase(DeclarativeBase):
pass
settings = get_settings()
connect_args: dict[str, object] = {}
if settings.poo_database_url.startswith("sqlite"):
connect_args["check_same_thread"] = False
poo_engine = create_engine(settings.poo_database_url, connect_args=connect_args)
PooSessionLocal = sessionmaker(bind=poo_engine, autoflush=False, autocommit=False, class_=Session)
def get_poo_db_session() -> Generator[Session, None, None]:
session = PooSessionLocal()
try:
yield session
finally:
session.close()
+13
View File
@@ -0,0 +1,13 @@
from datetime import datetime
from typing import Literal
from pydantic import BaseModel
PublicIPCheckStatus = Literal["first_seen", "unchanged", "changed", "error"]
class PublicIPCheckResponse(BaseModel):
status: PublicIPCheckStatus
checked_at: datetime
changed: bool
+22 -6
View File
@@ -7,7 +7,7 @@ from typing import Any
from sqlalchemy import select
from sqlalchemy.orm import Session
from app.auth_db import reset_auth_db_caches
from app.db import reset_db_caches
from app.config import Settings, get_settings
from app.models.config import AppConfigEntry
@@ -27,6 +27,15 @@ CONFIG_FIELDS: tuple[ConfigField, ...] = (
ConfigField("System", "APP_ENV", "app_env", "App Env"),
ConfigField("System", "APP_DEBUG", "app_debug", "App Debug"),
ConfigField("System", "APP_HOSTNAME", "app_hostname", "App Hostname"),
ConfigField("SMTP", "SMTP_ENABLED", "smtp_enabled", "SMTP Enabled"),
ConfigField("SMTP", "SMTP_HOST", "smtp_host", "SMTP Host"),
ConfigField("SMTP", "SMTP_PORT", "smtp_port", "SMTP Port"),
ConfigField("SMTP", "SMTP_USERNAME", "smtp_username", "SMTP Username"),
ConfigField("SMTP", "SMTP_PASSWORD", "smtp_password", "SMTP Password", secret=True),
ConfigField("SMTP", "SMTP_FROM_NAME", "smtp_from_name", "SMTP From Name"),
ConfigField("SMTP", "SMTP_FROM_ADDRESS", "smtp_from_address", "SMTP From Address"),
ConfigField("SMTP", "SMTP_TO_ADDRESS", "smtp_to_address", "SMTP To Address"),
ConfigField("SMTP", "SMTP_USE_STARTTLS", "smtp_use_starttls", "SMTP Use STARTTLS"),
ConfigField(
"Authentication",
"AUTH_SESSION_COOKIE_NAME",
@@ -118,7 +127,7 @@ def sync_app_hostname_from_bootstrap(session: Session, bootstrap_settings: Setti
current_values["APP_HOSTNAME"] = bootstrap_hostname
_persist_config_values(session, current_values)
get_settings.cache_clear()
reset_auth_db_caches()
reset_db_caches()
def build_runtime_settings(session: Session, bootstrap_settings: Settings) -> Settings:
@@ -175,7 +184,7 @@ def save_config_updates(session: Session, form_data: dict[str, str], bootstrap_s
_validate_config_values(merged_values, bootstrap_settings)
_persist_config_values(session, merged_values)
get_settings.cache_clear()
reset_auth_db_caches()
reset_db_caches()
def save_config_value(
@@ -190,7 +199,7 @@ def save_config_value(
_validate_config_values(current_values, bootstrap_settings)
_persist_config_values(session, current_values)
get_settings.cache_clear()
reset_auth_db_caches()
reset_db_caches()
def is_ticktick_oauth_ready(settings: Settings) -> bool:
@@ -251,8 +260,6 @@ def _settings_payload(settings: Settings) -> dict[str, Any]:
"app_debug": settings.app_debug,
"app_hostname": settings.app_hostname,
"app_database_url": settings.app_database_url,
"location_database_url": settings.location_database_url,
"poo_database_url": settings.poo_database_url,
"ticktick_client_id": settings.ticktick_client_id,
"ticktick_client_secret": settings.ticktick_client_secret,
"ticktick_token": settings.ticktick_token,
@@ -260,6 +267,15 @@ def _settings_payload(settings: Settings) -> dict[str, Any]:
"home_assistant_auth_token": settings.home_assistant_auth_token,
"home_assistant_timeout_seconds": settings.home_assistant_timeout_seconds,
"home_assistant_action_task_project_id": settings.home_assistant_action_task_project_id,
"smtp_enabled": settings.smtp_enabled,
"smtp_host": settings.smtp_host,
"smtp_port": settings.smtp_port,
"smtp_username": settings.smtp_username,
"smtp_password": settings.smtp_password,
"smtp_from_name": settings.smtp_from_name,
"smtp_from_address": settings.smtp_from_address,
"smtp_to_address": settings.smtp_to_address,
"smtp_use_starttls": settings.smtp_use_starttls,
"poo_webhook_id": settings.poo_webhook_id,
"poo_sensor_entity_name": settings.poo_sensor_entity_name,
"poo_sensor_friendly_name": settings.poo_sensor_friendly_name,
+149
View File
@@ -0,0 +1,149 @@
from __future__ import annotations
from dataclasses import dataclass
from datetime import UTC, datetime
from email.message import EmailMessage
from email.utils import formataddr
import smtplib
from app.config import Settings
class EmailConfigurationError(ValueError):
"""Raised when SMTP settings are incomplete or disabled."""
class EmailDeliveryError(RuntimeError):
"""Raised when sending email fails."""
@dataclass(frozen=True, slots=True)
class SMTPConfig:
host: str
port: int
username: str
password: str
from_name: str
from_address: str
to_address: str
use_starttls: bool
def get_smtp_config(settings: Settings, *, require_enabled: bool = True) -> SMTPConfig:
if require_enabled and not settings.smtp_enabled:
raise EmailConfigurationError("SMTP is disabled")
if not settings.smtp_host:
raise EmailConfigurationError("SMTP host is required")
if settings.smtp_port <= 0:
raise EmailConfigurationError("SMTP port must be greater than zero")
if not settings.smtp_from_address:
raise EmailConfigurationError("SMTP from address is required")
if not settings.smtp_to_address:
raise EmailConfigurationError("SMTP to address is required")
return SMTPConfig(
host=settings.smtp_host,
port=settings.smtp_port,
username=settings.smtp_username,
password=settings.smtp_password,
from_name=settings.smtp_from_name,
from_address=settings.smtp_from_address,
to_address=settings.smtp_to_address,
use_starttls=settings.smtp_use_starttls,
)
def is_smtp_ready(settings: Settings) -> bool:
try:
get_smtp_config(settings, require_enabled=False)
except EmailConfigurationError:
return False
return True
def send_plaintext_email(
settings: Settings,
*,
subject: str,
body: str,
recipient: str | None = None,
require_enabled: bool = True,
) -> None:
smtp_config = get_smtp_config(settings, require_enabled=require_enabled)
message = EmailMessage()
message["Subject"] = subject
message["From"] = _build_from_header(smtp_config)
message["To"] = recipient or smtp_config.to_address
message.set_content(body)
try:
with smtplib.SMTP(smtp_config.host, smtp_config.port, timeout=10) as smtp:
smtp.ehlo()
if smtp_config.use_starttls:
smtp.starttls()
smtp.ehlo()
if smtp_config.username:
smtp.login(smtp_config.username, smtp_config.password)
smtp.send_message(
message,
from_addr=smtp_config.from_address,
to_addrs=[recipient or smtp_config.to_address],
)
except (OSError, smtplib.SMTPException) as exc:
error_message = _sanitize_error_message(str(exc), smtp_config.password)
raise EmailDeliveryError(error_message or "SMTP delivery failed") from exc
def send_smtp_test_email(settings: Settings) -> None:
send_plaintext_email(
settings,
subject="Home Automation SMTP Test",
body="This is a test email from Home Automation SMTP settings.",
require_enabled=False,
)
def send_public_ip_changed_email(
settings: Settings,
*,
previous_ipv4: str,
current_ipv4: str,
detected_at: datetime,
) -> None:
send_plaintext_email(
settings,
subject="Public IP changed",
body=(
"Your public IPv4 address has changed.\n\n"
f"Previous IP: {previous_ipv4}\n"
f"Current IP: {current_ipv4}\n"
f"Detected at: {_format_utc_timestamp(detected_at)}\n\n"
"If you use Namecheap API trusted IP restrictions, you may need to "
"update the trusted IP manually.\n"
),
)
def _sanitize_error_message(message: str, password: str) -> str:
sanitized = message
if password:
sanitized = sanitized.replace(password, "[redacted]")
return sanitized
def _format_utc_timestamp(value: datetime) -> str:
if value.tzinfo is None:
normalized = value.replace(tzinfo=UTC)
else:
normalized = value.astimezone(UTC)
return normalized.strftime("%Y-%m-%d %H:%M:%S UTC")
def _build_from_header(smtp_config: SMTPConfig) -> str:
if smtp_config.from_name:
return formataddr((smtp_config.from_name, smtp_config.from_address))
return smtp_config.from_address
+191
View File
@@ -0,0 +1,191 @@
from __future__ import annotations
import ipaddress
import logging
from dataclasses import dataclass
from datetime import UTC, datetime
from typing import Callable, Literal
import httpx
from sqlalchemy import select
from sqlalchemy.orm import Session
from app.config import Settings
from app.models.public_ip import PublicIPHistory, PublicIPState
from app.services.config_page import build_runtime_settings
from app.services.email import EmailConfigurationError, EmailDeliveryError, send_public_ip_changed_email
logger = logging.getLogger(__name__)
PUBLIC_IP_PROVIDER_NAME = "ipify"
PUBLIC_IP_PROVIDER_URL = "https://api.ipify.org"
PUBLIC_IP_PROVIDER_TIMEOUT_SECONDS = 5.0
PublicIPResultStatus = Literal["first_seen", "unchanged", "changed", "error"]
PublicIPv4Fetcher = Callable[[], str]
class PublicIPCheckError(RuntimeError):
"""Raised when the public IPv4 provider cannot return a valid IPv4."""
@dataclass(slots=True)
class PublicIPCheckResult:
status: PublicIPResultStatus
checked_at: datetime
changed: bool
previous_ipv4: str | None = None
current_ipv4: str | None = None
def check_public_ipv4(
session: Session,
*,
fetch_public_ipv4: PublicIPv4Fetcher | None = None,
provider_name: str = PUBLIC_IP_PROVIDER_NAME,
) -> PublicIPCheckResult:
checked_at = _utc_now()
state = session.scalar(select(PublicIPState).where(PublicIPState.id == 1).limit(1))
try:
raw_ipv4 = (fetch_public_ipv4 or fetch_public_ipv4_from_provider)()
current_ipv4 = _validate_ipv4(raw_ipv4)
except PublicIPCheckError as exc:
logger.warning("Public IPv4 check failed: %s", exc)
if state is not None:
state.last_checked_at = checked_at
state.last_check_status = "error"
state.last_check_error = str(exc)
state.last_provider = provider_name
session.commit()
return PublicIPCheckResult(status="error", checked_at=checked_at, changed=False)
if state is None:
state = PublicIPState(
id=1,
current_ipv4=current_ipv4,
previous_ipv4=None,
first_seen_at=checked_at,
last_checked_at=checked_at,
last_changed_at=None,
last_check_status="first_seen",
last_check_error=None,
last_provider=provider_name,
)
session.add(state)
session.add(
PublicIPHistory(
ipv4=current_ipv4,
observed_at=checked_at,
change_type="first_seen",
provider=provider_name,
)
)
session.commit()
return PublicIPCheckResult(
status="first_seen",
checked_at=checked_at,
changed=False,
current_ipv4=current_ipv4,
)
if state.current_ipv4 == current_ipv4:
state.last_checked_at = checked_at
state.last_check_status = "unchanged"
state.last_check_error = None
state.last_provider = provider_name
session.commit()
return PublicIPCheckResult(
status="unchanged",
checked_at=checked_at,
changed=False,
current_ipv4=current_ipv4,
)
previous_ipv4 = state.current_ipv4
state.previous_ipv4 = previous_ipv4
state.current_ipv4 = current_ipv4
state.last_checked_at = checked_at
state.last_changed_at = checked_at
state.last_check_status = "changed"
state.last_check_error = None
state.last_provider = provider_name
session.add(
PublicIPHistory(
ipv4=current_ipv4,
observed_at=checked_at,
change_type="changed",
provider=provider_name,
)
)
session.commit()
return PublicIPCheckResult(
status="changed",
checked_at=checked_at,
changed=True,
previous_ipv4=previous_ipv4,
current_ipv4=current_ipv4,
)
def check_public_ipv4_and_notify(
session: Session,
*,
bootstrap_settings: Settings,
fetch_public_ipv4: PublicIPv4Fetcher | None = None,
provider_name: str = PUBLIC_IP_PROVIDER_NAME,
) -> PublicIPCheckResult:
result = check_public_ipv4(
session,
fetch_public_ipv4=fetch_public_ipv4,
provider_name=provider_name,
)
if result.status != "changed" or result.previous_ipv4 is None or result.current_ipv4 is None:
return result
runtime_settings = build_runtime_settings(session, bootstrap_settings)
try:
send_public_ip_changed_email(
runtime_settings,
previous_ipv4=result.previous_ipv4,
current_ipv4=result.current_ipv4,
detected_at=result.checked_at,
)
except (EmailConfigurationError, EmailDeliveryError) as exc:
logger.warning("Public IPv4 change notification failed: %s", exc)
return result
def fetch_public_ipv4_from_provider() -> str:
try:
response = httpx.get(
PUBLIC_IP_PROVIDER_URL,
params={"format": "text"},
timeout=PUBLIC_IP_PROVIDER_TIMEOUT_SECONDS,
)
response.raise_for_status()
except httpx.HTTPError as exc:
raise PublicIPCheckError(f"provider request failed: {exc}") from exc
return response.text.strip()
def _validate_ipv4(raw_value: str) -> str:
if not raw_value:
raise PublicIPCheckError("provider returned an empty response")
try:
parsed = ipaddress.ip_address(raw_value)
except ValueError as exc:
raise PublicIPCheckError("provider returned an invalid IPv4 value") from exc
if parsed.version != 4:
raise PublicIPCheckError("provider returned a non-IPv4 value")
return str(parsed)
def _utc_now() -> datetime:
return datetime.now(UTC)
+22
View File
@@ -33,6 +33,14 @@
<div class="notice">{{ ticktick_oauth_notice }}</div>
{% endif %}
{% if smtp_test_error %}
<div class="alert">{{ smtp_test_error }}</div>
{% endif %}
{% if smtp_test_notice %}
<div class="notice">{{ smtp_test_notice }}</div>
{% endif %}
<div class="meta single-column">
<div>
<dt>当前用户</dt>
@@ -102,6 +110,20 @@
{% endif %}
</div>
{% endif %}
{% if section.name == "SMTP" %}
<div class="integration-action-row">
<div>
<p class="integration-action-title">SMTP Test Email</p>
<p class="integration-action-copy">Save the SMTP settings first, then send a simple plaintext test email to the configured recipient.</p>
</div>
{% if smtp_test_ready %}
<button type="submit" formaction="/config/smtp/test" formmethod="post">Send SMTP Test</button>
{% else %}
<span class="button-link disabled" aria-disabled="true">Send SMTP Test</span>
{% endif %}
</div>
{% endif %}
</fieldset>
{% endfor %}
+13 -7
View File
@@ -8,15 +8,17 @@ alembic==1.18.4
# via -r requirements.in
annotated-types==0.7.0
# via pydantic
argon2-cffi==25.1.0
# via -r requirements.in
argon2-cffi-bindings==25.1.0
# via argon2-cffi
anyio==4.13.0
# via
# httpx
# starlette
# watchfiles
apscheduler==3.11.2
# via -r requirements.in
argon2-cffi==25.1.0
# via -r requirements.in
argon2-cffi-bindings==25.1.0
# via argon2-cffi
build==1.4.3
# via pip-tools
certifi==2026.2.25
@@ -42,7 +44,9 @@ httpcore==1.0.9
httptools==0.7.1
# via uvicorn
httpx==0.28.1
# via -r dev-requirements.in
# via
# -r dev-requirements.in
# -r requirements.in
idna==3.11
# via
# anyio
@@ -66,6 +70,8 @@ pip-tools==7.5.3
# via -r dev-requirements.in
pluggy==1.6.0
# via pytest
pycparser==2.23
# via cffi
pydantic==2.13.2
# via
# fastapi
@@ -88,8 +94,6 @@ python-dotenv==1.2.2
# uvicorn
python-multipart==0.0.26
# via -r requirements.in
pycparser==2.23
# via cffi
pyyaml==6.0.3
# via
# -r requirements.in
@@ -112,6 +116,8 @@ typing-inspection==0.4.2
# via
# pydantic
# pydantic-settings
tzlocal==5.3.1
# via apscheduler
uvicorn[standard]==0.44.0
# via -r requirements.in
uvloop==0.22.1
-16
View File
@@ -25,19 +25,3 @@ services:
- ./data:/app/data
- ./.env:/app/.env:ro
grafana:
image: grafana/grafana:latest
container_name: home-automation-grafana
depends_on:
- app
restart: unless-stopped
ports:
- "10.238.75.70:8882:3000"
environment:
GF_PLUGINS_PREINSTALL: frser-sqlite-datasource
volumes:
- ./data:/data/home-automation:ro
- homeautomation_grafana_storage:/var/lib/grafana
volumes:
homeautomation_grafana_storage:
+7 -13
View File
@@ -23,25 +23,26 @@
- 基础路由注册
- `config.py`
- 环境变量驱动的 settings
- `auth_db.py`
- app 级共享 auth 数据库
- `db.py`
- SQLAlchemy engine / session / Base
- 统一数据层:一个 `Base`、一个绑定 `app_database_url` 的 cached engineSQLite WAL)、`get_engine` / `get_session_local` / `reset_db_caches` / `get_db_session`
- `dependencies.py`
- 通用依赖注入
- `api/`
- HTTP routes
- 当前已迁入 `/login``/logout``/admin`
- 当前已迁入 `GET /public-ip/check`
- 当前已迁入 `POST /homeassistant/publish` 第一版入口
- 当前已迁入 `POST /poo/record``GET /poo/latest`
- `models/`
- SQLAlchemy models
- 当前 `auth``location``poo` 使用各自独立的数据库 base
- 所有模型(auth / config / public_ip / location / poo)共用同一个 `Base`,均落在单一 `app.db`
- `schemas/`
- Pydantic schemas
- `services/`
- 业务服务层
- 当前已迁入 config page 的 DB 持久化逻辑
- 当前已迁入 public IPv4 检查、状态持久化与变化通知逻辑
- 当前已迁入 SMTP 发信与测试发信逻辑
- `integrations/`
- 外部系统适配层
- 当前已迁入 Home Assistant outbound adapter
@@ -50,17 +51,9 @@
- `static/`
- 极简静态资源
### `alembic_location/`
Location DB 的 migration 基础设施。
### `alembic_app/`
App DB 的 migration 基础设施
### `alembic_poo/`
Poo DB 的 migration 基础设施。
App DB 的唯一 Alembic migration 链,同时管理 `location` / `poo_records` 表。M1 将三个独立 DB 合并进 `app.db` 后,`alembic_location/``alembic_poo/` 已退役,全部由此链统一管理
### `tests/`
@@ -80,6 +73,7 @@ pytest 测试目录。后续可以在这里自然扩展:
- 当前数据库继续使用 SQLite
- 当前不引入前后端分离
- 当前不设计 Notion 模块
- 当前通知能力仍保持极小范围,不引入独立通知中心或多渠道抽象
## 关于 Notion
+127
View File
@@ -0,0 +1,127 @@
# 设计文档与多模型协作约定
本目录把 `docs/roadmap.md` 里的三个里程碑展开成**可被 coding agent 流水线执行**的详细设计文档。
- [`m1-db-consolidation.md`](./m1-db-consolidation.md) — 单库化地基
- [`m2-frontend-v2.md`](./m2-frontend-v2.md) — React SPA 前端 v2
- [`m3-token-mobile.md`](./m3-token-mobile.md) — token 鉴权与移动端(远期)
本文件定义**所有任务共用的格式与协作规则**,三个里程碑文档不再重复这些约定。
---
## 1. 协作模型:Orchestrator → Implementer → Reviewer
设计目标是让三个不同档位的模型分工协作:
| 角色 | 模型档位 | 职责 |
| --- | --- | --- |
| **Orchestrator(编排者)** | 强模型 | 读里程碑文档,挑出依赖已满足的下一个原子任务,派发给 implementer;收到 reviewer 的 PASS 后推进下一个任务 |
| **Implementer(实现者)** | 便宜模型 | 一次只做**一个**原子任务,严格按任务卡执行,跑完本地校验后回报 |
| **Reviewer(评审者)** | 强模型 | 对照验收标准 + Reviewer checklist 独立复核、独立跑校验闸门,返回 `PASS` 或一份编号返工清单 |
### 循环
```
Orchestrator 选任务 T(其 Depends 全部为 done
Implementer 实现 T ──► 跑校验闸门 ──► 回报 diff + 校验输出
Reviewer 复核 T
├── PASS ─────────► Orchestrator 标记 T 为 done,进入下一个任务
└── REWORK[1..n] ─► 退回 Implementer,按编号逐条修,直到 PASS
```
### 角色边界(重要)
- **Implementer 不得扩大范围**:只能改任务卡 `Files` 里列出的文件;超出范围的问题要在回报里以 `OUT-OF-SCOPE:` 标注,交给 Orchestrator 决定是否新开任务,而不是顺手改掉。
- **Reviewer 必须独立重跑校验闸门**,不能只信 implementer 的回报。
- **Reviewer 的返工清单必须可执行、带编号**(`REWORK 1: ...`),不写主观感受。
- 一个任务**不通过校验闸门就不算完成**,Orchestrator 不得跳过。
---
## 2. 原子任务的定义
一个"原子任务"必须同时满足:
1. **单一关注点**:一个任务只解决一件事(一次 schema 变更、一个端点、一个模块迁移……)。
2. **PR 大小**:理想 diff < ~200 行(结构性 sweep 任务可放宽,但应在卡上标 `[structural]` 并优先派给较强 implementer)。
3. **边界处可绿**:任务完成时,整个仓库通过校验闸门(见下)。**一个任务"拥有"它所改代码对应的测试**——如果改动会让某些现有测试失败,修这些测试就属于这个任务的范围,不允许留红给下一个任务。
4. **可独立验收**:验收标准是客观、可机械检查的断言,不依赖人的主观判断。
5. **依赖显式**:通过 `Depends` 字段声明前置任务。没有声明依赖的任务,Orchestrator 可并行派发。
---
## 3. 任务卡格式
每个任务在里程碑文档中以如下结构出现。Implementer 和 Reviewer 都只需要任务卡 + 本约定文件,**不需要读其它任务**即可工作。
```markdown
### M{n}-T{nn} — <标题> [structural?]
- **Status**: `todo` | `in-progress` | `in-review` | `done`
- **Depends**: M{n}-T{nn}, …(或 `none`
- **Context**: 1–2 句,为什么要做这个、它在里程碑里的位置。
**Files**(精确到路径,标注动作)
- `create path/to/new_file.py`
- `modify path/to/existing.py`
- `delete path/to/old.py`
**Steps**(便宜模型可直接照做的有序步骤)
1.
2.
**Out of scope / 不要碰**
- …(明确列出容易被误改的相邻区域,约束便宜模型漂移)
**Acceptance criteria**(客观、可勾选;Reviewer 逐条核)
- [ ]
- [ ] 校验闸门全绿(见 §4
**Reviewer checklist**(除验收标准外,强模型重点看的点)
-
```
---
## 4. 校验闸门(每个任务结束都要全绿)
在仓库根目录、激活 `.venv` 后执行:
```bash
# 1) 单元 / 集成测试(CI 同款,权威闸门)
pytest
# 2) Lintpyproject 已配置 ruffline-length=100
ruff check .
# 3) 若本任务改动了任何 HTTP 路由 / schema:重导出 OpenAPI 并确认已提交
python scripts/export_openapi.py
git diff --exit-code openapi/ # 必须无未提交差异
```
- `pytest` 是**权威闸门**`.github/workflows/pytest.yml` 跑的就是它,任何任务都不得让它变红。
- 改了路由 / Pydantic schema 的任务,`openapi/openapi.json``openapi/openapi.yaml` 必须在同一任务里重新生成并提交(`openapi/` 纳入版本控制)。
- 前端(M2/M3)相关任务的前端侧闸门(lint / typecheck / build)在对应里程碑文档里单独定义。
---
## 5. 提交与集成约定
- 每个任务一个 commit,message 前缀任务 ID,例如:`M1-T03: unify data layer onto single app DB engine`
- 一个里程碑在一个 feature 分支上推进(如 `feature/m1-db-consolidation`),按任务依赖顺序合并。
- 任务卡里的 `Status` 字段由 Orchestrator 维护,作为流水线的单一进度源。
- 涉及**不可逆 / 数据破坏**的步骤(删旧 DB 文件、删 Grafana volume 等)一律不进自动化任务,只在文档里标为人工步骤(见 M1 的"人工操作"小节)。
---
## 6. 数据安全红线(贯穿所有里程碑,不可违反)
1. **任何脚本 / migration 都不得删除或覆盖用户数据文件**(旧 `.db`、备份、volume)。删除只能是人工、事后、确认无误的独立步骤。
2. 涉及历史数据的迁移**先在备份副本上演练**,再对真实库执行。
3. 数据迁移脚本必须**幂等**且**搬完对账行数**,对不上立即中止并非零退出。
4. 破坏性 Reviewer 一票否决:只要任务里出现"删文件 / drop 有数据的表 / truncate"Reviewer 直接 REWORK,要求改为人工步骤。
+348
View File
@@ -0,0 +1,348 @@
# M1 — 单库化地基(DB Consolidation
> 阅读前提:先读 [`README.md`](./README.md)(协作模型、任务卡格式、校验闸门、数据安全红线)。本文档只展开 M1 的现状、目标与原子任务。
## 1. 目标
把 location、poo 两个独立 SQLite 库合并进 `app.db`,收敛成**单库 + 单 engine + 单 DeclarativeBase + 单 Alembic 链**,清理项目早期散落的数据层代码,并移除 Grafana。历史数据零丢失。
## 2. 现状(实现者可据此工作,不必重新通读全仓库)
**三套数据层(散落点)**
- `app/db.py``Base` + `engine`/`SessionLocal`/`get_db_session` —— 实际绑定 `settings.location_database_url`(即 location 库,命名有误导性)。
- `app/models/base.py`:仅 `from app.db import Base` 转出。
- `app/poo_db.py``PooBase` + `poo_engine`/`PooSessionLocal`/`get_poo_db_session` —— 绑定 `poo_database_url`
- `app/auth_db.py``AuthBase` + 带 `lru_cache``_get_auth_engine` / `get_auth_session_local` / `reset_auth_db_caches` / `get_auth_db_session` —— 绑定 `app_database_url`(真正的 app 库)。
**模型与归属**
- `app/models/auth.py``AuthUser``AuthSession``AuthBase` → app 库)
- `app/models/config.py``AppConfigEntry``AuthBase` → app 库)
- `app/models/public_ip.py``PublicIPState``PublicIPHistory``AuthBase` → app 库)
- `app/models/location.py``Location``Base` → location 库),表 `location`PK(`person`,`datetime`)`latitude`/`longitude` NOT NULL`altitude` nullable
- `app/models/poo.py``PooRecord``PooBase` → poo 库),表 `poo_records`PK(`timestamp`)`status`/`latitude`/`longitude` NOT NULL
- `app/models/__init__.py`:导出除 `PooRecord` 外的模型(`PooRecord` 单独存在)
**三条 Alembic 链**
- `alembic_app.ini` + `alembic_app/``env.py``AuthBase.metadata`),head = `20260429_05_public_ip_monitor`
- `alembic_location.ini` + `alembic_location/`head = `20260419_01_location_baseline`
- `alembic_poo.ini` + `alembic_poo/`head = `20260420_01_poo_baseline`
**adoption / 启动链路**
- `scripts/app_db_adopt.py`(常量 `APP_BASELINE_REVISION = "20260429_05_public_ip_monitor"`
- `scripts/location_db_adopt.py``scripts/poo_db_adopt.py`(含 legacy 校验:`EXPECTED_USER_VERSION`、表结构断言)
- `scripts/run_migrations.py`:依次调用三个 adopt 函数,返回 `{"app","location","poo"}`
- `app/main.py` lifespan`ensure_runtime_dirs`app/location/poo 三路径)、`ensure_auth_db_ready``ensure_location_db_ready``ensure_poo_db_ready`,再起 APScheduler 每 4h 检查 public IP
**依赖与路由**
- `app/dependencies.py``get_auth_db`app session)、`get_db`location session)、`get_poo_db`poo session)、`get_app_settings``get_current_auth_session``get_homeassistant_client``get_ticktick_client`
- `app/api/routes/location.py``POST /location/record`,依赖 `get_db`**无鉴权**
- `app/api/routes/poo.py``POST /poo/record``GET /poo/latest`,依赖 `get_poo_db`**无鉴权**
- `app/api/routes/homeassistant.py`:同时用 `get_db`location)和 `get_poo_db`
**config**
- `app/config.py``app_database_url` / `location_database_url` / `poo_database_url` 三字段 + computed `app_sqlite_path` / `location_sqlite_path` / `poo_sqlite_path`
- `app/services/config_page.py``build_runtime_settings` 用到 `reset_auth_db_caches`;配置页 sections 暴露 `location_database_url` / `poo_database_url`(约 263264 行)
**测试耦合点(M1 必然要改)**
- `tests/conftest.py``test_database_urls` 设三套环境变量;`ready_location_database` / `ready_poo_database` / `auth_database` / `location_client`monkeypatch `app_db.engine`/`SessionLocal`/ `poo_client`monkeypatch `poo_db.poo_engine`/`PooSessionLocal`
- `tests/test_location.py` / `tests/test_poo.py`:用上述 client + 各自 adopt 脚本的 adoption 测试
- `tests/test_deployment.py`:断言 `run_all_migrations()` 返回 `{app,location,poo}` 三库各自 revision;断言 entrypoint 不含 `*_db_adopt`
- `tests/test_homeassistant_inbound.py`monkeypatch `app.poo_db`
- `tests/test_config.py` / `tests/test_public_ip.py` / `tests/test_smtp.py`:硬编码三套 URL / 路径
- `reset_auth_db_caches``conftest``test_app``test_auth``test_deployment``test_ticktick` 引用
## 3. 目标架构(M1 完成态)
**单数据层 `app/db.py`**
```python
class Base(DeclarativeBase): ...
# 绑定 settings.app_database_url 的 cached engine;建连时启用 WALPRAGMA journal_mode=WAL
def get_engine() -> Engine: ...
def get_session_local() -> sessionmaker: ...
def reset_db_caches() -> None: ...
def get_db_session() -> Generator[Session, None, None]: ...
```
- 所有模型(auth / config / public_ip / location / poo)都继承这一个 `Base`
- 删除 `app/auth_db.py``app/poo_db.py``app/models/base.py`
- 依赖收敛为**单一** `get_db`app session);移除 `get_poo_db`、旧 `get_auth_db`
- 一条 Alembic 链(`alembic_app`),`location` / `poo_records` 成为其管理对象;删除 `alembic_location*` / `alembic_poo*`
- `config.py` 只保留 `app_database_url`;移除 location/poo 的 url 与 path。
- `docker-compose.yml` 去掉 grafana service;删除 `grafana/`
- 数据搬迁由 `scripts/migrate_legacy_data.py` 一次性完成(不进 Alembic 链)。
## 4. 任务依赖图
```
T01 (app 链建 location+poo 空表)
├─► T02 (数据搬迁脚本) # 逻辑上需要新表存在
└─► T03 [structural] (统一数据层/模型/依赖/路由)
└─► T04 (lifespan + run_migrations 收敛, 删 adopt 脚本)
└─► T05 (config 去 location/poo url + 配置页 + 测试硬编码)
T06 (删 Grafana) # 独立, 可并行
T07 (文档 + OpenAPI 重导出) # 收尾, 依赖 T03/T04/T05
```
`T01``T06` 无前置可先开;`T02` 依赖 `T01``T03` 依赖 `T01``T04`/`T05` 依赖 `T03``T07` 最后。
---
## 5. 原子任务
### M1-T01 — app 链新增 revision:建 `location` + `poo_records` 空表 `[schema]`
- **Status**: `todo`
- **Depends**: none
- **Context**: 让 app 库的 Alembic 链能建出这两张表,schema 与旧库**完全一致**。本任务只动 schema,不搬数据、不移模型。
**Files**
- `create alembic_app/versions/20260611_06_merge_location_poo_tables.py`
- `modify scripts/app_db_adopt.py`(更新 `APP_BASELINE_REVISION`
**Steps**
1. 新 revision`revision = "20260611_06_merge_location_poo_tables"``down_revision = "20260429_05_public_ip_monitor"`
2. `upgrade()``op.create_table` 手写建 `location``poo_records`,列/约束严格照抄现有 baseline(`location`: person TEXT, datetime TEXT, latitude REAL NOT NULL, longitude REAL NOT NULL, altitude REAL nullable, PK(person,datetime)`poo_records`: timestamp TEXT, status TEXT, latitude REAL NOT NULL, longitude REAL NOT NULL, PK(timestamp))。
3. `downgrade()``op.drop_table("poo_records")` + `op.drop_table("location")`
4.`scripts/app_db_adopt.py``APP_BASELINE_REVISION` 更新为新 head。
**Out of scope / 不要碰**
- 不要把 `Location` / `PooRecord` 模型改到 app Base(那是 T03)。
- 不要触碰 `alembic_location*` / `alembic_poo*`T03/T04 删)。
- 不要在本 revision 里写任何数据拷贝。
**Acceptance criteria**
- [ ] 在一个全新临时 app 库上 `command.upgrade(alembic_app head)` 后,`sqlite_master``location``poo_records`、且与旧 baseline 表结构一致(`PRAGMA table_info` 对齐)。
- [ ] `downgrade -1` 能干净回滚这两张表。
- [ ] `APP_BASELINE_REVISION == "20260611_06_merge_location_poo_tables"`
- [ ] 校验闸门全绿(`pytest``test_deployment` 对 app head 的断言仍通过,因为它用的是常量)。
**Reviewer checklist**
- 表结构与旧 baseline **逐列逐约束**一致(类型 TEXT/REAL、nullable、PK 顺序)。
- `down_revision` 正确指向旧 head,链上只有一个 head。
---
### M1-T02 — 数据搬迁脚本 `scripts/migrate_legacy_data.py`
- **Status**: `todo`
- **Depends**: M1-T01
- **Context**: 把旧 `locationRecorder.db` / `pooRecorder.db` 的行幂等拷进 app 库的新表,搬完对账。**不进 Alembic 链**,人工运行一次。
**Files**
- `create scripts/migrate_legacy_data.py`
- `create tests/test_migrate_legacy_data.py`
**Steps**
1. 入口 `migrate_legacy_data(app_url, location_url, poo_url, *, dry_run=False) -> dict`CLI 默认从 env 读三个 url(即便 location/poo url 已从 `Settings` 移除,本脚本可直接读环境变量或接受 `--location-db`/`--poo-db` 参数,保持自包含)。
2. 对每个旧库:若文件不存在 → 该表 `skipped`(**不报错**,保证 CI / 全新部署可安全 no-op)。
3. 拷贝用 SQLite `ATTACH DATABASE '<old>' AS legacy` + `INSERT OR IGNORE INTO main.<table> SELECT <显式列> FROM legacy.<table>`(显式列名,禁用 `SELECT *`)。`INSERT OR IGNORE` 保证幂等(PK 冲突跳过)。
4. 搬完对账:对每张表比对 `源行数``目标行数中来自源的部分`;目标行数 < 源行数则 `raise` 并以非零码退出。
5. `dry_run` 模式只读统计、不写入。
6. 打印每表结果:`{location: {source, copied, skipped, final}, poo_records: {...}}`
**Out of scope / 不要碰**
- **绝不** `os.remove` / 覆盖任何旧文件(数据安全红线)。
- 不修改 Alembic 链,不在 app 启动链路里调用本脚本。
- 不改 `config.py`
**Acceptance criteria**
- [ ] 单测:给定含 N 行的临时旧库 + 已 upgrade 的临时 app 库,运行后 app 库对应表有 N 行;**再运行一次**仍是 N 行(幂等)。
- [ ] 单测:旧库文件不存在时该表返回 `skipped`,不抛异常,app 库该表保持为空。
- [ ] 单测:构造"目标缺行"场景,断言对账失败抛错且退出码非零。
- [ ] 脚本中不出现任何文件删除/覆盖调用(`grep -nE "os\.remove|unlink|shutil|truncate|DROP TABLE" scripts/migrate_legacy_data.py` 为空)。
- [ ] 校验闸门全绿。
**Reviewer checklist**
- 幂等机制确实是 PK 冲突安全(`INSERT OR IGNORE` 或等价 upsert),不是靠"先清空目标"。
- 对账逻辑会在丢行时**真的中止**(非零退出),不是只打印 warning。
- 列名显式,与两表 schema 完全对应。
---
### M1-T03 — 统一数据层、模型、依赖、路由到单库 `[structural]`
- **Status**: `todo`
- **Depends**: M1-T01
- **Context**: M1 的核心 sweep。把三套 engine/Base/session 收敛成 `app/db.py` 一套(绑 app 库、开 WAL),所有模型挂到同一个 `Base`,依赖收敛为单一 `get_db`,所有路由改用它。**本任务必须原子落地**——删除旧模块会同时打断所有 importer,无法分多次保持绿色。Orchestrator 可按下方 Steps 的自然分段派给较强 implementer。
**Files**
- `modify app/db.py`(改写为统一数据层:`Base` + 绑 `app_database_url` 的 cached engine + WAL + `get_session_local` + `reset_db_caches` + `get_db_session`
- `delete app/auth_db.py`
- `delete app/poo_db.py`
- `delete app/models/base.py`
- `modify app/models/location.py``from app.db import Base`
- `modify app/models/poo.py`(改继承统一 `Base`import 改 `app.db`
- `modify app/models/auth.py``app/models/config.py``app/models/public_ip.py``AuthBase` → 统一 `Base`
- `modify app/models/__init__.py`(补导出 `PooRecord`,保证 `from app import models` 注册所有表到同一 metadata)
- `modify app/dependencies.py`(单一 `get_db`;删 `get_poo_db``get_app_settings`/`get_current_auth_session` 改用 `get_db`
- `modify app/api/routes/auth.py``pages.py``public_ip.py``ticktick.py``get_auth_db``get_db`
- `modify app/api/routes/location.py``poo.py``homeassistant.py`location/poo session 改用 `get_db`;删 `get_poo_db` 引用)
- `modify app/services/config_page.py``reset_auth_db_caches``reset_db_caches`
- `modify app/main.py``import app.auth_db as auth_db` → 统一层;`get_auth_session_local``get_session_local`
- `modify tests/conftest.py``tests/test_app.py``tests/test_auth.py``tests/test_ticktick.py``tests/test_homeassistant_inbound.py``tests/test_location.py``tests/test_poo.py`import sweep + 把 location/poo client 改成写 app 库的统一 session;移除对 `app.poo_db`/`app.db`(location) monkeypatch 的依赖)
**Steps**
1. 改写 `app/db.py``Base(DeclarativeBase)`;沿用 `auth_db.py` 的 cached-engine + reset 模式但绑 `app_database_url`;为 sqlite 连接注册 `PRAGMA journal_mode=WAL`(用 `event.listens_for(engine, "connect")` 或建连后执行)。导出 `get_engine`/`get_session_local`/`reset_db_caches`/`get_db_session`
2. 模型 sweep:所有 `from app.auth_db import AuthBase` / `from app.poo_db import PooBase` / `from app.db import Base` 统一成 `from app.db import Base`;类继承统一 `Base``app/models/__init__.py` 增加 `from app.models.poo import PooRecord` 并补进 `__all__`
3.`app/auth_db.py``app/poo_db.py``app/models/base.py`
4. 依赖 sweep`app/dependencies.py` 留单一 `get_db`yield 统一 session),删 `get_poo_db``get_app_settings``get_current_auth_session``Depends(get_auth_db)``Depends(get_db)`
5. 路由 sweep:所有 `Depends(get_auth_db)``Depends(get_poo_db)``Depends(get_db)` 统一为 `Depends(get_db)`(变量名 `auth_db_session`/`poo_db`/`db` 可保留,不强制改)。
6. `app/services/config_page.py``reset_auth_db_caches``reset_db_caches`
7. `app/main.py`:把 `_run_scheduled_public_ip_check` / `ensure_auth_db_ready` 里的 `auth_db.get_auth_session_local()` 换成统一 `get_session_local()`。(lifespan 里 location/poo 的 ready 检查留到 T04 删。)
8. 测试 sweep`reset_auth_db_caches``reset_db_caches`6 个文件);conftest 的 `location_client`/`poo_client` 改成"写入统一 app session 即可"的形式(不再 monkeypatch 已删除的 `app.poo_db`/location `app.db`);`test_homeassistant_inbound` 同理。
**Out of scope / 不要碰**
- 不删 `scripts/location_db_adopt.py` / `scripts/poo_db_adopt.py`,不改 lifespan 的 location/poo ready 调用(那是 T04,避免与本任务交叉冲突)。
- 不动 `config.py` 的字段(T05)。
- 不改业务逻辑(service 内部算法、HA 集成行为保持不变)。
**Acceptance criteria**
- [ ] `grep -rnE "auth_db|poo_db|PooBase|AuthBase|get_auth_db|get_poo_db|reset_auth_db_caches|app\.models\.base" app | grep -v __pycache__` 结果为空。
- [ ] `app/db.py` 的 engine 绑定 `app_database_url`sqlite 下 `PRAGMA journal_mode` 实测为 `wal`
- [ ] 所有模型 `Base.metadata.tables` 同时包含 auth/config/public_ip/location/poo_records 五类表。
- [ ] `pytest` 全绿(含 location/poo/homeassistant_inbound 测试在单库下通过)。
- [ ] `ruff check .` 无新增告警。
**Reviewer checklist**
- WAL 真的生效(实际连接 `PRAGMA journal_mode` 返回 `wal`),不是只写了注释。
- location/poo 的读写在单库下行为不变(端点仍返回 200、行落库)。
- 没有遗留指向已删模块的死 import;没有把业务逻辑顺手改了。
- `get_db` 现在产出的是 app 库 session(不是旧 location 库)。
---
### M1-T04 — 收敛启动链路:lifespan + run_migrations,删除 location/poo adopt 脚本
- **Status**: `todo`
- **Depends**: M1-T03
- **Context**: 单库后只需保证 app 库就绪;location/poo 的 adoption 链路整条退役。
**Files**
- `modify app/main.py``ensure_runtime_dirs` 只建 app 路径;删 `ensure_location_db_ready`/`ensure_poo_db_ready` 及其调用与 import
- `modify scripts/run_migrations.py`(只 `adopt_or_initialize_app_db`,返回 `{"app": ...}`
- `delete scripts/location_db_adopt.py`
- `delete scripts/poo_db_adopt.py`
- `delete alembic_location.ini``alembic_location/`(含 env.py、versions
- `delete alembic_poo.ini``alembic_poo/`
- `modify tests/test_deployment.py``run_all_migrations` 期望值改为单 `{"app": ...}`;删/改 legacy location/poo 迁移断言;保留"app DB 不存在则 fail-closed"用例)
- `modify tests/test_location.py``tests/test_poo.py`(删除针对已删 adopt 脚本的 adoption 测试;保留端点行为测试)
- `modify tests/conftest.py`(删 `_make_alembic_config`/`_make_poo_alembic_config`/`ready_location_database`/`ready_poo_database` 等已无意义的 fixture
**Steps**
1. `app/main.py`:移除 `from scripts.location_db_adopt ...` / `poo_db_adopt` import;删两个 `ensure_*_db_ready` 函数及 lifespan 中调用;`ensure_runtime_dirs` 只处理 `settings.app_sqlite_path`
2. `scripts/run_migrations.py``run_all_migrations` 只返回 app 一项。
3. 删除两套 adopt 脚本与两套 alembic 环境/ini。
4. 测试:把 `test_migration_runner_*` 改成单库口径;删掉引用已删脚本常量(`LOCATION_BASELINE_REVISION` 等)的用例。
**Out of scope / 不要碰**
- 不动 `scripts/app_db_adopt.py` 的核心逻辑(仅 T01 已更新其常量)。
- 不动数据搬迁脚本(T02)。
**Acceptance criteria**
- [ ] `grep -rnE "location_db_adopt|poo_db_adopt|alembic_location|alembic_poo" app scripts tests | grep -v __pycache__` 为空。
- [ ] 仓库不再有 `alembic_location*` / `alembic_poo*` 文件。
- [ ] `python -m scripts.run_migrations` 在全新临时 app 库上成功初始化(含 location/poo_records 表)。
- [ ] 校验闸门全绿。
**Reviewer checklist**
- lifespan 仍对 app 库 fail-closed(缺库时明确报错),未弱化启动安全。
- 没有残留对已删 alembic 环境的引用(包括 `.ini` 路径字符串)。
---
### M1-T05 — config 去除 location/poo URL 与路径,清理配置页与测试硬编码
- **Status**: `todo`
- **Depends**: M1-T03
- **Context**: 配置层只剩 `app_database_url`,运行时不再有 location/poo 库概念。
**Files**
- `modify app/config.py`(删 `location_database_url`/`poo_database_url` 字段与 `location_sqlite_path`/`poo_sqlite_path` computed 属性)
- `modify app/services/config_page.py`(配置页 sections 移除 `location_database_url`/`poo_database_url` 展示项)
- `modify .env.example`(移除两行 legacy DB URL;保留 `APP_DATABASE_URL`
- `modify tests/test_config.py`(删对两个 URL/路径的断言)
- `modify tests/test_public_ip.py``tests/test_smtp.py`(构造 `Settings` 时去掉 location/poo url 入参)
- `modify tests/conftest.py``test_database_urls` 不再 set `LOCATION_DATABASE_URL`/`POO_DATABASE_URL`
**Out of scope / 不要碰**
- 不动 `migrate_legacy_data.py`(它自带读旧库路径的能力,与 `Settings` 解耦)。
- 不改其它配置项(SMTP / TickTick / HA 等)。
**Acceptance criteria**
- [ ] `grep -rnE "location_database_url|poo_database_url|location_sqlite_path|poo_sqlite_path" app tests | grep -v __pycache__` 为空。
- [ ] 配置页渲染不再出现 location/poo DB URL 字段。
- [ ] 校验闸门全绿。
**Reviewer checklist**
- 没有别的代码还假设 `Settings` 上存在这两个属性(运行期不会 AttributeError)。
---
### M1-T06 — 移除 Grafana
- **Status**: `todo`
- **Depends**: none(可与 T01 并行)
- **Context**: 可视化将由 M2 的 React 承担;Grafana 直接删除,不再 re-point。
**Files**
- `modify docker-compose.yml`(删 `grafana` service 及其 `depends_on`/挂载;删顶层 `volumes.homeautomation_grafana_storage`
- `delete grafana/``provisioning/``dashboards/` 全部)
- `modify tests/test_deployment.py`(若有针对 grafana service 的断言则同步移除)
- `modify README.md`(删"Grafana Provisioning"整节——也可并入 T07,二选一,避免重复改同段)
**Out of scope / 不要碰**
- **不在脚本里删除** named volume `homeautomation_grafana_storage` 的实际数据卷——这是人工 ops 步骤(见 §6),compose 里移除声明即可。
- 不动 app/migration service。
**Acceptance criteria**
- [ ] `docker-compose.yml` 不再含 `grafana``homeautomation_grafana_storage`
- [ ] 仓库不再有 `grafana/` 目录。
- [ ] `docker compose config` 能成功解析(语法有效)。
- [ ] 校验闸门全绿。
**Reviewer checklist**
- 没有遗留对 `./grafana/...` 挂载路径的引用。
- 没有顺手删 `./data` 卷或改动 app service 端口/卷。
---
### M1-T07 — 文档与 OpenAPI 收尾
- **Status**: `todo`
- **Depends**: M1-T03, M1-T04, M1-T05
- **Context**: 让文档反映单库现实,并把"前后端不分离 / 三库不合并 / Grafana"约束在 architecture 文档中正式退役。
**Files**
- `modify README.md`(三库 → 单库;删 location/poo DB 初始化与 adopt 说明;更新"运行测试"段落使其与实际测试一致)
- `modify docs/architecture-overview.md`(退役"三库不合并"location/poo Alembic 链合并说明)
- `modify docs/roadmap.md`(勾掉 M1 范围项)
- `run python scripts/export_openapi.py` 并提交 `openapi/` 变更(location/poo 路由依赖在 T03 改过,schema 可能变化)
**Acceptance criteria**
- [ ] README / architecture 不再描述 location/poo 独立库与 adopt 脚本。
- [ ] `python scripts/export_openapi.py``git diff --exit-code openapi/` 无未提交差异。
- [ ] 校验闸门全绿。
**Reviewer checklist**
- 文档无残留的旧命令(`location_db_adopt.py` 等)。
- OpenAPI 已重导出且入库。
---
## 6. 人工操作 runbook(生产切换,不进自动化任务)
按数据安全红线,下列步骤由人执行,**不**写进 implementer 任务:
1. **备份**:停服前复制 `data/app.db``data/locationRecorder.db``data/pooRecorder.db` 到带时间戳的归档目录。
2. **演练**:把上述备份恢复到 scratch 目录,先在副本上跑完整流程(升级 + `migrate_legacy_data.py --dry-run` 再实跑),核对行数。
3. **部署新镜像**:新镜像的 migration job 会把 app 库升级到新 head,建出空的 `location` / `poo_records`
4. **搬数据**:在生产机运行 `python scripts/migrate_legacy_data.py`(指向归档前的旧库),核对对账输出。
5. **验证**app 起来后确认 location/poo 端点与历史查询正常、行数与旧库一致。
6. **(事后,确认无误再做)撤旧库**:归档旧 `.db` 文件、删除 `homeautomation_grafana_storage` 卷。**这一步人工、可回退地保留归档,永不在脚本中自动执行。**
## 7. 里程碑完成定义(Definition of Done
- 运行期只存在 `app.db` 一个库、一个 engine、一个 `Base`、一条 Alembic 链。
- `grep` 不到任何 `auth_db` / `poo_db` / location 独立库 / adopt 脚本 / grafana 的残留引用。
- 旧库历史数据已通过 `migrate_legacy_data.py` 搬入且对账通过。
- `pytest``ruff check .``export_openapi` 全绿且 `openapi/` 已入库。
- README / architecture / roadmap 反映单库现实。
+232
View File
@@ -0,0 +1,232 @@
# M2 — 前端 v2React SPA
> 阅读前提:先读 [`README.md`](./README.md)。M2 依赖 M1 完成(单库 + 干净的数据层 + API 建立在合并后的 schema 上)。
## 1. 目标
**React SPA** 取代现有 Jinja 页面,由 FastAPI **同源**托管(同一容器、同一 origin)。一步合并 roadmap 的"前端重写"与"前端做厚":配置界面 + 数据可视化(热力图 / 地图,接管 Grafana)+ 记录的按需展示与小幅增删改。
> **元目标(agentic 实验)**:这是用 agent 写 React 的试水,全程尽量不读代码。因此本里程碑**强约束 OpenAPI → 类型化 TS client 作为契约护栏**:后端 API 先稳,前端永远对着强类型契约写,便宜模型不易跑偏,reviewer 也有客观依据。
## 2. 现状(M1 完成后)
- 页面仍是服务端 Jinja`app/api/routes/pages.py``GET/POST /config``/``/admin``POST /config/smtp/test`+ `app/templates/``base/config/home/login.html``styles.css`)。
- 鉴权:`get_current_auth_session`(读 `auth_session_cookie_name` cookie),server-side session + 每会话 `csrf_token` 内嵌在表单。
- `app/main.py``app.mount("/static", StaticFiles(...))`
- 配置读写逻辑在 `app/services/config_page.py``build_config_sections` / `save_config_updates` / `build_runtime_settings`)。
- 业务数据:单库中的 `location``poo_records``public_ip_state``public_ip_history`
## 3. 目标架构
### 3.1 后端:JSON API + SPA 托管
- 所有数据交互走 **JSON API**,统一前缀 `/api`(SPA 是客户端渲染,必须有 API——这与"同源/同容器"无关)。
- FastAPI 既挂 `/api/*`,又挂 SPA 静态产物,并对非 `/api`、非静态资源的路径**回退到 `index.html`**(支持前端路由 deep-link)。
- Jinja 页面在 SPA 达到功能对齐后移除。
### 3.2 鉴权:复用 session cookie + SPA 版 CSRF
- 继续用现有 **HttpOnly session cookie**(同源自动携带),M2 **不引入 token**token 属 M3)。
- CSRF:新增 `GET /api/session` 返回当前用户 + 该会话的 `csrf_token`SPA 在所有写请求(POST/PUT/PATCH/DELETE)放 `X-CSRF-Token` header,后端校验其与 session 内 `csrf_token` 一致。等价于把现有表单 CSRF 平移到 header。
- 浏览器面向的所有新端点一律 session 保护;**裸 ingestion 端点(设备调用的 `POST /location/record``POST /poo/record`)维持现状到 M3**。
### 3.3 前端工程
- `frontend/`**Vite + React + TypeScript**。
- API client:由后端 `openapi/openapi.json` **自动生成** TS 类型与请求函数(如 `openapi-typescript` + 轻量 fetch 封装,或同类工具)。生成物入库或在 build 时生成(见 T06 决策)。
- 可视化:地图 + 热力图(location 轨迹 / poo 点位)。建议 **MapLibre GL 或 Leaflet + heatmap 插件**(最终选型见 §5 决策)。
- 状态/数据请求:轻量即可(如 TanStack Query),不引入重型框架。
### 3.4 构建与部署
- 多阶段 `Dockerfile`node 阶段 `npm ci && npm run build` → 把 `frontend/dist` 拷进 python 镜像的静态目录;运行镜像不带 node。
- compose 仍是单 app 容器(同源)。
## 4. API 契约(M2 要落地的端点)
> 全部 `/api` 前缀、session 保护、JSON 进出。具体 schema 在各任务里用 Pydantic 定义,并经 `export_openapi.py` 固化。
| 分组 | 端点 | 用途 |
| --- | --- | --- |
| 会话 | `GET /api/session` | 返回当前用户 + csrf_token;未登录 401 |
| 会话 | `POST /api/auth/login` | 账号密码登录,下发 session cookie |
| 会话 | `POST /api/auth/logout` | 注销 |
| 会话 | `POST /api/auth/password` | 改密(沿用现有强制改密语义)|
| 配置 | `GET /api/config` | 返回配置 sectionssecret 不回显)|
| 配置 | `PUT /api/config` | 保存配置(留空保留旧 secret 语义不变)|
| 配置 | `POST /api/config/smtp/test` | 触发测试发信 |
| 数据 | `GET /api/locations` | location 记录查询(时间范围/分页,供地图/热力图)|
| 数据 | `GET /api/poo` | poo 记录列表(分页)|
| 数据 | `GET /api/public-ip` | 当前状态 + 变化历史 |
| CRUD | `PATCH /api/locations/{person}/{datetime}` | 修正单条 location |
| CRUD | `DELETE /api/locations/{person}/{datetime}` | 删除单条 location |
| CRUD | `PATCH /api/poo/{timestamp}` | 修正单条 poo |
| CRUD | `DELETE /api/poo/{timestamp}` | 删除单条 poo |
> 记录 CRUD 依赖现有 PK 作行标识(location PK=`person+datetime`poo PK=`timestamp`)。路径参数需对 `datetime`/`timestamp` 做 URL 编码处理。
## 5. 需先拍板的决策(Orchestrator 在派 T06 前确认)
1. **地图/热力图库**MapLibre GL(矢量、现代)vs Leaflet(简单、生态大)。推荐 Leaflet + `leaflet.heat`(试水门槛低)。
2. **OpenAPI client 生成物**:入库(确定性、便于 review)vs build 时生成(仓库干净)。推荐**入库**,并加一个 `npm run codegen` + CI 校验"生成物与 openapi 同步"。
3. **CSRF 落地**header `X-CSRF-Token` + `GET /api/session` 下发(推荐)vs 双提交 cookie。
4. **是否保留少量 Jinja**:建议 SPA 对齐后**全量移除** `templates/`,只留 SPA。
> 这些可用 1 个轻量"决策任务"或直接由 Orchestrator 在本节记录选择,再开 T06。
## 6. 任务依赖图
```
后端 API(可与前端 scaffold 并行)
M2-T01 config API
M2-T02 session/auth API ─┐
M2-T03 data read API ├─► 都产出 OpenAPI 契约
M2-T04 record CRUD API │
M2-T05 smtp/action API ─┘
│ (openapi 稳定后)
M2-T06 前端 scaffold + codegen ──► M2-T07 auth UI
├─► M2-T08 config UI
├─► M2-T09 可视化 UI
└─► M2-T10 records 管理 UI
M2-T11 FastAPI 托管 SPA + 移除 Jinja(依赖 T07T10 达到对齐)
M2-T12 多阶段 Dockerfile + CI/compose
M2-T13 文档 + OpenAPI 收尾
```
---
## 7. 原子任务(任务卡)
> 后端任务沿用 M1 的校验闸门(`pytest` / `ruff` / `export_openapi`)。前端任务的闸门见 §8。
### M2-T01 — config JSON API
- **Status**: `todo` · **Depends**: noneM1 完成后)
- **Context**: 把 `config_page` 的读写能力暴露成 JSON,复用现有 service,不重写业务逻辑。
- **Files**: `create app/api/routes/api/config.py``create app/schemas/config.py``modify app/main.py`(注册路由);`create tests/test_api_config.py`
- **Steps**: 用 `build_config_sections`/`save_config_updates` 包出 `GET/PUT /api/config`session 保护;secret 不回显、留空保留旧值语义照搬。
- **Acceptance**:
- [ ] 未登录访问 `GET /api/config` 返回 401。
- [ ] 登录后 `GET` 返回 sectionssecret 字段被遮罩。
- [ ] `PUT` 留空 secret 时保留旧值;非法值返回 4xx 且不写库。
- [ ] 校验闸门全绿(含 `openapi/` 重导出入库)。
- **Reviewer**: 复用了 service 而非复制逻辑;CSRF 校验存在;secret 不泄漏到响应或 OpenAPI 示例。
### M2-T02 — session / auth JSON API
- **Status**: `todo` · **Depends**: none
- **Context**: 给 SPA 提供登录/注销/会话探测 + CSRF 下发。
- **Files**: `create app/api/routes/api/session.py``app/schemas/session.py``modify app/main.py``create tests/test_api_session.py`
- **Steps**: `GET /api/session`401 或 user+csrf)、`POST /api/auth/login``POST /api/auth/logout``POST /api/auth/password`,复用 `app/services/auth.py`
- **Acceptance**:
- [ ] 正确账号密码登录后置下 HttpOnly session cookie`GET /api/session` 返回 user + csrf_token。
- [ ] 错误凭据 401,不下发 cookie。
- [ ] 写端点缺 `X-CSRF-Token` 或不匹配 → 403。
- [ ] 强制改密语义与现有一致。
- [ ] 校验闸门全绿。
- **Reviewer**: cookie 仍 HttpOnly、`Secure` 跟随 `app_env``SameSite=Lax`;密码仍 Argon2,不明文。
### M2-T03 — 数据读取 APIlocations / poo / public-ip
- **Status**: `todo` · **Depends**: none
- **Files**: `create app/api/routes/api/data.py``app/schemas/data.py``modify app/main.py``create tests/test_api_data.py`
- **Steps**: `GET /api/locations`(时间范围 + 分页)、`GET /api/poo`(分页)、`GET /api/public-ip`state + history);session 保护;查询参数有上限防全表导出。
- **Acceptance**:
- [ ] 分页/时间范围参数生效且有上限;越权未登录 401。
- [ ] 返回 schema 经 OpenAPI 固化。
- [ ] 校验闸门全绿。
- **Reviewer**: 查询走索引/PK,无 N+1;时间过滤边界正确。
### M2-T04 — 记录 CRUD API(修正 / 删除)
- **Status**: `todo` · **Depends**: M2-T03
- **Files**: `modify app/api/routes/api/data.py``app/services/location.py``app/services/poo.py``create tests/test_api_record_crud.py`
- **Steps**: `PATCH`/`DELETE` locationPK person+datetime)与 pooPK timestamp);session + CSRF 保护;PK 路径参数 URL 解码;删除是**硬删单行**(不是清表)。
- **Acceptance**:
- [ ] PATCH 改单行字段、DELETE 删单行,行数变化精确为 1。
- [ ] 不存在的 PK → 404。
- [ ] 缺 CSRF → 403。
- [ ] 没有任何"批量删/清表"路径。
- [ ] 校验闸门全绿。
- **Reviewer**: 删除限定单 PK;编辑校验输入;ingestion 裸端点未被顺手加保护或改动。
### M2-T05 — SMTP 测试 / 动作类 JSON API
- **Status**: `todo` · **Depends**: M2-T01
- **Files**: `modify app/api/routes/api/config.py``modify tests/test_api_config.py`
- **Steps**: `POST /api/config/smtp/test` 复用 `send_smtp_test_email`,返回结构化结果(success / config-error / failed)。
- **Acceptance**:
- [ ] 三种结果都有明确 JSON 状态码/字段;session + CSRF 保护。
- [ ] 校验闸门全绿。
### M2-T06 — 前端 scaffold + OpenAPI codegen `[structural]`
- **Status**: `todo` · **Depends**: M2-T01..T05OpenAPI 已稳定)
- **Context**: 建 `frontend/` 工程与类型化 client 流水线,这是后续所有前端任务的地基。
- **Files**: `create frontend/`Vite+React+TS 脚手架、`package.json``tsconfig.json`、eslint、vitest、`.gitignore`)、`frontend/src/api/`codegen 产物 + fetch 封装,自动注入 `X-CSRF-Token`)、`frontend/README.md``npm run codegen` 脚本
- **Steps**: 初始化 Vite React-TS;接 `openapi/openapi.json` 生成类型;写一个最小 App 壳 + 受保护路由骨架;fetch 封装统一带 cookie、写请求注入 CSRF header、401 跳登录。
- **Acceptance**:
- [ ] `npm ci && npm run build` 成功产出 `frontend/dist`
- [ ] `npm run lint``npm run typecheck``npm run test` 全绿(哪怕只有 1 个 smoke 测试)。
- [ ] `npm run codegen` 生成物与当前 `openapi/openapi.json` 一致(CI 可校验)。
- **Reviewer**: client 全部基于生成类型;CSRF/cookie/401 处理在统一封装层;无手写、与契约不符的请求类型。
### M2-T07 — 鉴权 UI(登录 / 会话引导 / 改密)
- **Status**: `todo` · **Depends**: M2-T06
- **Acceptance**: 登录成功进受保护区;未登录访问受保护路由跳登录;强制改密流程可走完;`build/lint/typecheck/test` 全绿。
### M2-T08 — 配置 UI(取代 Jinja config 页)
- **Status**: `todo` · **Depends**: M2-T06
- **Acceptance**: 能读/存所有现有配置 section;secret 不回显、留空保留;SMTP 测试按钮反映三态;前端闸门全绿。
### M2-T09 — 数据可视化 UI(地图 + 热力图)
- **Status**: `todo` · **Depends**: M2-T06(数据来自 T03
- **Context**: 接管 Grafana 原职责:location 轨迹/热力图、poo 点位。
- **Acceptance**: 地图渲染 location/poo 点;热力图层可切换;时间范围筛选生效;前端闸门全绿。
### M2-T10 — 记录管理 UI(按需展示 + 增删改)
- **Status**: `todo` · **Depends**: M2-T06CRUD 来自 T04
- **Acceptance**: 列表分页展示 poo/location;可编辑、可删除单条并即时刷新;删除有二次确认;前端闸门全绿。
### M2-T11 — FastAPI 托管 SPA + 移除 Jinja
- **Status**: `todo` · **Depends**: M2-T07, T08, T09, T10
- **Files**: `modify app/main.py`(挂载 SPA 静态目录 + 非 `/api` 路径回退 `index.html`);`delete app/templates/``app/api/routes/pages.py`(功能对齐后);`modify tests`(移除 Jinja 页面测试,新增 SPA fallback 测试)
- **Acceptance**:
- [ ] `/config` 等路径返回 SPA`index.html`),`/api/*` 不被 fallback 吞掉,`/static`/资源正常。
- [ ] 旧 Jinja 模板与 pages 路由移除后 `pytest` 全绿。
- [ ] 校验闸门全绿(含 OpenAPI 重导出)。
- **Reviewer**: fallback 不拦截 `/api``/docs``/openapi.json`、静态资源;未登录访问 API 仍 401(不是被 SPA 壳吞掉)。
### M2-T12 — 多阶段 Dockerfile + CI/compose
- **Status**: `todo` · **Depends**: M2-T11
- **Files**: `modify Dockerfile`node build 阶段 → 拷 `dist` 进 python 镜像);`modify .github/workflows/*`(加前端 build/lint/typecheck);`modify tests/test_deployment.py`(镜像断言更新)
- **Acceptance**:
- [ ] 镜像构建成功且运行镜像不含 node 运行时。
- [ ] CI 跑前端闸门 + 后端 `pytest`
- [ ] 校验闸门全绿。
### M2-T13 — 文档 + OpenAPI 收尾
- **Status**: `todo` · **Depends**: M2-T12
- **Acceptance**: README 增"前端 v2"段(开发/构建说明);architecture 退役"不前后端分离"约束;roadmap 勾选 M2`openapi/` 已同步入库。
---
## 8. 前端校验闸门(前端任务每次结束都要全绿)
`frontend/` 下:
```bash
npm ci
npm run codegen # 生成类型化 client;产物须与 openapi/openapi.json 同步
npm run lint
npm run typecheck
npm run test
npm run build # 必须产出 dist
```
- 后端若同任务改了路由/schema,仍需根目录 `python scripts/export_openapi.py` 并提交 `openapi/`
- "codegen 产物与 OpenAPI 同步"应在 CI 校验(生成后 `git diff --exit-code`)。
## 9. 里程碑完成定义(DoD
- 访问应用得到 React SPA;配置、可视化、记录增删改都在 SPA 内完成。
- 所有浏览器交互走 `/api` JSON 端点,session + CSRF 保护;ingestion 裸端点维持现状(留给 M3)。
- Jinja `templates/` 与 pages 路由移除;FastAPI 同源托管 SPA。
- 多阶段镜像构建通过;CI 含前端闸门。
- 后端 `pytest`/`ruff`/`export_openapi` + 前端 `build/lint/typecheck/test` 全绿。
+109
View File
@@ -0,0 +1,109 @@
# M3 — Token 鉴权与移动端(远期试水)
> 阅读前提:先读 [`README.md`](./README.md)。M3 依赖 M2(已有 `/api` JSON 契约与 session 鉴权)。
>
> **定位**:远期、低投入、探索性。React Native 部分主要是"没做过、试试水"。范围**可能收缩**——其中**token 鉴权 + ingestion 端点收口**是有持久价值的安全改进,应优先;RN app 是加分项。Orchestrator 可只取前半。
## 1. 目标
1. 引入 **bearer token 鉴权**,让非浏览器客户端(移动端、设备脚本)能安全访问。
2. 把 M2 暂时维持裸奔的 **ingestion 端点**`POST /location/record``POST /poo/record`)收口到 token 鉴权下。
3. 做一个 **React Native** 移动端,用类 OAuth 流程拿 token 后消费现有 `/api`
## 2. 现状(M2 完成后)
- `/api/*` 走 session cookie + `X-CSRF-Token`
- `app/services/auth.py` 有 server-side session`auth_sessions` 表,token_hash 存储)。
- `POST /location/record``POST /poo/record` 仍**无鉴权**(设备/脚本裸调用)。
## 3. 目标架构
### 3.1 Token 模型
- 新表 `auth_tokens``id``user_id``token_hash`(仅存哈希)、`label`(设备名)、`created_at``expires_at`(可空=长期)、`revoked_at`
- bearer 校验:`Authorization: Bearer <token>` → 哈希比对 `auth_tokens` → 命中且未撤销未过期则认定身份。
### 3.2 类 OAuth 签发流程(无第三方的 Authorization Code 简化版)
1. 移动端在**内置浏览器**打开 `/authorize?...`
2. 用户账号密码登录(走现有 session),页面展示"授权此设备"。
3. 批准后服务端生成**一次性 authorization code**,重定向到 app 深链 `homeautomation://callback?code=...`
4. app 用 code 调 `POST /api/auth/token` 换取 bearer token 并存本地。
> 简化兜底:批准页直接展示一次性 token 由 app 捕获。优先实现重定向 + code 交换的正规版。
### 3.3 统一鉴权依赖
- `/api` 的数据/CRUD 端点接受**session cookie 或 bearer**两者之一(同一套端点同时服务 Web 与移动端)。
- ingestion 端点(location/poo record)改为**要求 bearer**。
### 3.4 React Native
- **Expo + TypeScript**,复用 M2 的 OpenAPI 类型化 client(共享契约)。
- 内置浏览器走 §3.2 流程拿 token;之后所有请求带 `Authorization: Bearer`
## 4. 迁移注意(重要)
- ingestion 端点一旦要求 bearer,**现有调用方(HA/设备脚本)必须先配置 token**,否则记录会中断。
- 上线顺序:先签发 token 能力(T01–T02)→ 给现有设备配 token → 再对 ingestion 端点强制 bearer(T03),避免断流。可设一个过渡开关或灰度。
## 5. 任务依赖图
```
M3-T01 token 模型 + 迁移
└─► M3-T02 签发流程(authorize + code 交换)
└─► M3-T03 统一鉴权依赖 + ingestion 端点收口(含过渡开关)
├─► M3-T04 Web 端 token 管理 UI(列出/撤销设备)
└─► M3-T05 React Native app(试水)
└─► M3-T06 文档收尾
```
## 6. 原子任务(任务卡)
### M3-T01 — token 数据模型 + Alembic 迁移
- **Status**: `todo` · **Depends**: noneM2 完成后)
- **Files**: `create app/models/token.py``alembic_app/versions/<new>_auth_tokens.py``modify app/models/__init__.py``create tests/test_token_model.py`
- **Acceptance**:
- [ ] 迁移在全新库 upgrade 后建出 `auth_tokens` 表;downgrade 可回滚。
- [ ] token 仅以哈希存储(与 `auth_sessions` 同等强度),明文不入库。
- [ ] 校验闸门全绿。
- **Reviewer**: 哈希算法/长度与现有 session token 一致;`expires_at` 可空语义明确。
### M3-T02 — 签发流程:authorize 页 + code 交换端点
- **Status**: `todo` · **Depends**: M3-T01
- **Files**: `create app/api/routes/api/token.py``app/schemas/token.py`;前端 `/authorize` 页(M2 SPA 内);`create tests/test_api_token.py`
- **Acceptance**:
- [ ] 登录用户在 `/authorize` 批准后得到一次性 code`POST /api/auth/token` 用 code 换取 bearercode 一次性且短时效。
- [ ] 未登录访问 `/authorize` 跳登录;无效/过期 code 换取失败。
- [ ] 返回的 bearer 仅此一次明文出现,库中只存哈希。
- [ ] 校验闸门全绿。
- **Reviewer**: code 一次性、绑定用户、短 TTL;深链 redirect 白名单校验,防开放重定向。
### M3-T03 — 统一鉴权依赖 + ingestion 端点收口
- **Status**: `todo` · **Depends**: M3-T02
- **Files**: `modify app/dependencies.py`(新增"cookie 或 bearer"统一身份依赖);`modify app/api/routes/location.py``poo.py`(要求 bearer,带过渡开关);`modify tests`
- **Acceptance**:
- [ ] `/api` 数据/CRUD 端点用合法 bearer 可访问(等价于 session)。
- [ ] ingestion 端点:带合法 bearer 通过,缺/错 token 在强制模式下 401;过渡开关可临时放行(默认关)。
- [ ] 撤销的 token 立即失效。
- [ ] 校验闸门全绿。
- **Reviewer**: 过渡开关默认安全(强制);bearer 与 session 两路鉴权不产生绕过;ingestion 行为变更有测试覆盖。
### M3-T04 — Web 端 token 管理 UI
- **Status**: `todo` · **Depends**: M3-T03
- **Acceptance**: 在 SPA 内可列出已签发设备 token(label/创建时间/最近使用)、可撤销;撤销后该 token 立即失效;前端闸门全绿。
### M3-T05 — React Native app(试水)
- **Status**: `todo` · **Depends**: M3-T03 · `[experimental]`
- **Files**: `create mobile/`Expo + TS,复用 OpenAPI 类型化 client
- **Acceptance**:
- [ ] 内置浏览器走签发流程拿到 token 并安全存储(Keychain/Keystore)。
- [ ] 至少跑通:登录拿 token → 拉取一类数据展示 → 记一条 ingestion。
- [ ] `npm run lint`/`typecheck`/`build`(或 Expo 等价) 全绿。
- **Reviewer**: token 存安全存储而非明文;client 基于共享 OpenAPI 类型。
### M3-T06 — 文档收尾
- **Status**: `todo` · **Depends**: M3-T05
- **Acceptance**: README/architecture 增 token 鉴权与移动端说明;roadmap 勾选 M3;`openapi/` 同步。
## 7. 里程碑完成定义(DoD
- 存在 bearer token 鉴权与签发流程;token 仅哈希存储、可撤销。
- ingestion 端点已收口到 bearer(过渡完成后强制)。
- `/api` 同时支持 session 与 bearer。
- (加分)React Native app 能拿 token 并消费 `/api`
- 后端 + 前端 + 移动端各自校验闸门全绿,`openapi/` 入库。
> 提醒:本里程碑探索性强,T05 可作为独立试水随时叫停,不影响 T01–T04 带来的安全收口价值。
+126
View File
@@ -0,0 +1,126 @@
# Public IPv4 Monitor 与邮件通知
本文档说明当前 public IPv4 monitor 与 SMTP 邮件通知能力的职责边界和运行方式。
## 当前范围
当前实现只覆盖一个很小的通知能力:
- 定期或手动检查当前公网 IPv4
- 将当前状态和变化历史持久化到 app DB
- 仅在公网 IPv4 发生变化时发送一封英文纯文本邮件
当前明确不包含:
- Namecheap API 自动更新
- IPv6 检查
- 错误告警邮件
- 重复提醒 / 升级告警
- Telegram / Slack / Discord 通知
- 完整通知中心或模板系统
## 数据存储
当前数据全部进入 app DB。
相关表:
- `public_ip_state`
- 保存当前状态
- 逻辑上通常只有一行
- `public_ip_history`
- 保存首次发现和变化历史
当前不会把 public IP 状态放进 `app_config`
## 检查结果语义
一次检查会返回以下四种结果之一:
- `first_seen`
- `unchanged`
- `changed`
- `error`
行为约束:
- `first_seen`:写入当前 IP 和首条 history,但不发通知邮件
- `unchanged`:只更新时间和状态,不写 history,不发邮件
- `changed`:更新 `previous_ipv4` / `current_ipv4` / `last_changed_at`,写入 history,并发送邮件
- `error`:保留已有有效 IP,不写伪 history,也不发邮件
## 手动检查与定时检查
手动检查入口:
- `GET /public-ip/check`
约束:
- 需要现有鉴权
- 响应不暴露 IP 本身
- 只返回非敏感检查结果
定时检查:
- 应用启动时注册 APScheduler job
- 默认每 4 小时执行一次
- 与手动检查复用同一套 public IP check + notify 逻辑
## SMTP 通知
当前通知发信复用现有 SMTP sender。
依赖的配置项:
- `SMTP_ENABLED`
- `SMTP_HOST`
- `SMTP_PORT`
- `SMTP_USERNAME`
- `SMTP_PASSWORD`
- `SMTP_FROM_NAME`
- `SMTP_FROM_ADDRESS`
- `SMTP_TO_ADDRESS`
- `SMTP_USE_STARTTLS`
其中:
- `SMTP_FROM_NAME` 用于邮件头显示名
- `From` 头会渲染成 `Name <mail@domain>`
- SMTP envelope sender 仍然使用纯邮箱地址,保持兼容性
## 通知触发条件
只有在 `changed` 时发邮件。
不会发邮件的情况:
- `first_seen`
- `unchanged`
- `error`
这使得同一 IP 状态不会被重复通知,因为在首次变更之后,后续重复检查会变成 `unchanged`
## 邮件内容
当前邮件标题固定为:
- `Public IP changed`
正文为英文纯文本,至少包含:
- previous IP
- current IP
- detected time
当前正文还会附带一句 Namecheap trusted IP 的人工更新提示。
## 失败处理
当前通知发送是“尽力而为”的附加动作:
- public IP 状态持久化先完成
- 邮件发送失败不会回滚 public IP 状态
- 失败只记录 warning 日志
这样可以避免通知链路反过来影响主检查流程。
+148
View File
@@ -0,0 +1,148 @@
# Roadmap
本文档记录 `home-automation``v1.0.3` 之后的下一阶段规划。这一阶段不是小修补,而是几次较大的结构性改动:单库化、前端重写、以及远期的移动端试水。
> 每个里程碑的**可执行原子任务**展开在 [`docs/design/`](./design/README.md)M1 [`m1-db-consolidation.md`](./design/m1-db-consolidation.md)、M2 [`m2-frontend-v2.md`](./design/m2-frontend-v2.md)、M3 [`m3-token-mobile.md`](./design/m3-token-mobile.md)。这些文档为 Orchestrator→Implementer→Reviewer 的多模型流水线设计。
## 当前基线(v1.0.3
- FastAPI + 服务端 Jinja 模板页面(目前只有 `/login``/config`
- 三个独立 SQLite 库:
- App DB`sqlite:///./data/app.db`
- Location DB`sqlite:///./data/locationRecorder.db`
- Poo DB`sqlite:///./data/pooRecorder.db`
- 三条独立 Alembic 链:`alembic_app/``alembic_location/``alembic_poo/`
- 单 admin 鉴权(Argon2 + server-side session cookie
- Public IPv4 monitor、SMTP 通知、Location / Poo recorder、Home Assistant in/out、TickTick OAuth
- 数据可视化目前由 Grafana provisioning 承担(仅 location / poo dashboard
- 已有 OpenAPI 导出脚本:`scripts/export_openapi.py`
## 本阶段正式退役的架构约束
`docs/architecture-overview.md` 里有几条当时刻意写死的约束,这一阶段明确退役:
- **“不引入前后端分离”** → 退役。本阶段改为 React SPA(仍由 FastAPI 同源托管,但渲染移到客户端)。
- **“三个独立 DB 不合并”** → 退役。本阶段把 location / poo 合并进 `app.db`
- **Grafana 作为可视化方案** → 退役。可视化由 React 前端自己承担(热力图、地图等)。
保持不变的约束:
- 继续使用 **SQLite**,本阶段不上 Postgres。
- 不引入 Notion。
## 里程碑总览
| 里程碑 | 主题 | 一句话 |
| --- | --- | --- |
| **M1** ✅ | 单库化地基 | 把三库合并成单一 `app.db`,清理散落数据层,删掉 Grafana |
| **M2** | 前端 v2 | React SPA 取代 Jinja,承载 config + 可视化 + 记录增删改 |
| **M3** | 开放与移动端(远期试水) | token 鉴权 + React Native 移动端 |
排序原则:**先清地基,再在干净结构上盖楼。** M2 的新 API 和 React 必须建立在合并后的单库之上,否则就是在准备推倒的旧数据层上盖新楼、之后回头返工。
---
## M1 — 单库化地基(✅ 已完成)
### 目标
把 location / poo 两个独立库合并进 `app.db`,借机清理项目早期散落各处的数据访问代码,并移除 Grafana。
### 范围
- **Alembic 收敛为单链(app 链)**location / poo 的表此后纳入 app 链管理;`alembic_location/``alembic_poo/` 退出活跃使用(保留在 git 历史)。
- **新建表(schema only**:在 app 链上加一条 upgrade revision,把原来两个旧库里的表**原样**建到 `app.db` 中。Alembic **不需要知道任何旧数据**——它只负责把 app DB 往上升一个版本、建出这两张新表。
- **数据搬迁交给独立脚本**`scripts/migrate_legacy_data.py`(见下方“迁移策略”),手动跑一次。
- **配置层收敛**:去掉 `LOCATION_DATABASE_URL` / `POO_DATABASE_URL`,统一到 `APP_DATABASE_URL`
- **开启 SQLite WAL**:单文件 + Web + APScheduler 并发写入,开 WAL 更稳。
- **删除 Grafana**:移除 compose 中的 grafana service、`grafana/provisioning/``grafana/dashboards/`。直接删除,不再 re-point datasource。
- **更新文档**README、architecture-overview 同步反映单库现实。
### 注意
- **可视化空窗可接受**:M1 删掉 Grafana 后、到 M2 React 可视化落地之前会有一段没有可视化面板的时间。已确认可以接受。
- **历史数据是第一优先级,绝不能丢**(见“数据安全原则”)。
---
## 迁移策略(M1 核心)
职责拆分得很清楚:**Alembic 管 schema,脚本管数据。**
### Alembic revision(只建结构)
- 一条 app 链上的 upgrade revision,建出与旧库**完全相同**的表结构。
- 确定性、与环境无关:在生产机、CI、全新部署上都一样地建空表,不依赖任何旧文件是否存在。
- 本步**只原样挪表,不顺手改 schema**。任何表结构清理留到之后一条单独的 migration 去做——不可替代的历史数据,一次只承担一种风险。
### 数据搬迁脚本(`scripts/migrate_legacy_data.py`
- 把旧 `locationRecorder.db` / `pooRecorder.db` 里的行,拷进 `app.db` 的新表(SQLite `ATTACH DATABASE` 或单独连接均可)。
- **幂等**:重复运行不会重复插入。
- **搬完对账**:逐表核对源 / 目标行数,对不上就报错中止。
- 只在生产机上**手动跑一次**,不进 Alembic 永久链路(避免把一次性历史搬迁焊死进每次全新建库都要跑的链路里)。
### 旧库的“撤掉”
- “撤掉旧库” = ① 配置不再指向它们 + ② 文件**归档保留**。
- **绝不**在任何脚本 / migration 里 `os.remove` 旧文件——那不可逆,且踩数据安全红线。
- 真正的删除是**人工、最后、确认无误之后**单独的一步。
---
## 数据安全原则
历史数据(location / poo 记录)是这个项目里最不可替代的东西,迁移期间一律按以下原则:
1. **迁移前先归档**旧 `.db` 文件一份。
2. **先在副本上演练**:把每日备份恢复到一个 scratch 目录,在副本上跑完整迁移、核对行数无误,再对真实库动手。
3. **脚本幂等 + 行数对账**,对不上立即中止。
4. **旧文件只读归档、绝不自动删除**,删除是事后人工动作。
---
## M2 — 前端 v2React SPA
### 目标
用 React SPA 取代现有 Jinja 页面,由 FastAPI 同源托管(同一容器、同一 origin)。这一步合并了“前端重写为 React”和“前端做厚”两件原本分开的事——它们本质是同一坨活。
> 备注:React 是一次 agentic programming 试水。之前只手写过 Vue、没手写过 React,这一轮想全程靠 agent、尽量不读代码地把它做出来。OpenAPI 导出 → 生成类型化 TS client 作为 agent 的契约护栏,正好服务这个目标。
### 范围
- **React SPA**FastAPI 挂载打包后的静态产物(同源,省掉 CORS)。
- **Config 界面**:取代现有 Jinja config 页。
- **数据可视化**:热力图、地图等,接管原先 Grafana 干的事。
- **按需展示 DB 数据**(例如 poo 记录)。
- **记录的小幅增删改**:用于修正不准确的记录。
### 后端配套
- **补一套 JSON API**:SPA 是客户端渲染,需要后端提供 config 读写、数据查询、记录 CRUD 等 JSON 端点。(同源不等于不需要 API——API 是“客户端怎么拿数据”,与文件托管在哪无关。)
- **鉴权**:浏览器面向的新端点(含记录 CRUD)复用现有 session cookie 保护。
- **类型化 client**:用 `scripts/export_openapi.py` 的输出生成 TS client。
### 鉴权边界(与 M3 衔接)
- 现在那个“裸 API 记小狗日志”的 ingestion 端点(设备 / 脚本调用,非浏览器)**维持现状到 M3**。
- M2 新增的、浏览器调用的 CRUD 端点,用 session 保护即可,本步不引入 token。
---
## M3 — 开放与移动端(远期试水)
### 目标
引入 token 鉴权并做一个 React Native 移动端。**明确是很远期、低投入的试水**——先把 React 前端做出来,之后才会碰移动端,且主要是想试试没做过的 React / React Native。
### 范围
- **OAuth-lite token 签发**:移动端在内置浏览器里用账号密码登录,走一遍类 OAuth 流程,服务端签发一个 bearer token 给 app 存起来使用。(本质是没有第三方的 Authorization Code 简化版。)
- **React Native 移动端**:试水性质。
- **给 ingestion 端点上 token**:把 M2 暂时维持裸奔的设备端点收口到 token 鉴权下。
### 为什么放最后
- 移动端是这一阶段最远期、最不确定的部分。
- token 主要是移动端的前置条件;Web 端 React 用现有 session cookie 即可,不需要为它提前引入 token。
+72
View File
@@ -249,6 +249,27 @@
}
}
},
"/config/smtp/test": {
"post": {
"tags": [
"pages"
],
"summary": "Smtp Test Submit",
"operationId": "smtp_test_submit_config_smtp_test_post",
"responses": {
"200": {
"description": "Successful Response",
"content": {
"text/html": {
"schema": {
"type": "string"
}
}
}
}
}
}
},
"/homeassistant/publish": {
"post": {
"tags": [
@@ -325,6 +346,27 @@
}
}
},
"/public-ip/check": {
"get": {
"tags": [
"public-ip"
],
"summary": "Run Public Ip Check",
"operationId": "run_public_ip_check_public_ip_check_get",
"responses": {
"200": {
"description": "Successful Response",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/PublicIPCheckResponse"
}
}
}
}
}
}
},
"/ticktick/auth/start": {
"get": {
"tags": [
@@ -443,6 +485,36 @@
"type": "object",
"title": "HTTPValidationError"
},
"PublicIPCheckResponse": {
"properties": {
"status": {
"type": "string",
"enum": [
"first_seen",
"unchanged",
"changed",
"error"
],
"title": "Status"
},
"checked_at": {
"type": "string",
"format": "date-time",
"title": "Checked At"
},
"changed": {
"type": "boolean",
"title": "Changed"
}
},
"type": "object",
"required": [
"status",
"checked_at",
"changed"
],
"title": "PublicIPCheckResponse"
},
"StatusResponse": {
"properties": {
"status": {
+49
View File
@@ -155,6 +155,19 @@ paths:
text/html:
schema:
type: string
/config/smtp/test:
post:
tags:
- pages
summary: Smtp Test Submit
operationId: smtp_test_submit_config_smtp_test_post
responses:
'200':
description: Successful Response
content:
text/html:
schema:
type: string
/homeassistant/publish:
post:
tags:
@@ -203,6 +216,19 @@ paths:
content:
application/json:
schema: {}
/public-ip/check:
get:
tags:
- public-ip
summary: Run Public Ip Check
operationId: run_public_ip_check_public_ip_check_get
responses:
'200':
description: Successful Response
content:
application/json:
schema:
$ref: '#/components/schemas/PublicIPCheckResponse'
/ticktick/auth/start:
get:
tags:
@@ -285,6 +311,29 @@ components:
title: Detail
type: object
title: HTTPValidationError
PublicIPCheckResponse:
properties:
status:
type: string
enum:
- first_seen
- unchanged
- changed
- error
title: Status
checked_at:
type: string
format: date-time
title: Checked At
changed:
type: boolean
title: Changed
type: object
required:
- status
- checked_at
- changed
title: PublicIPCheckResponse
StatusResponse:
properties:
status:
+2
View File
@@ -1,6 +1,8 @@
alembic>=1.14,<2.0
apscheduler>=3.10,<4.0
argon2-cffi>=25.1,<26.0
fastapi>=0.115,<0.116
httpx>=0.28,<1.0
jinja2>=3.1,<4.0
pydantic-settings>=2.6,<3.0
python-multipart>=0.0.12,<1.0
+24 -7
View File
@@ -8,14 +8,21 @@ alembic==1.18.4
# via -r requirements.in
annotated-types==0.7.0
# via pydantic
anyio==4.13.0
# via
# httpx
# starlette
# watchfiles
apscheduler==3.11.2
# via -r requirements.in
argon2-cffi==25.1.0
# via -r requirements.in
argon2-cffi-bindings==25.1.0
# via argon2-cffi
anyio==4.13.0
certifi==2026.4.22
# via
# starlette
# watchfiles
# httpcore
# httpx
cffi==2.0.0
# via argon2-cffi-bindings
click==8.3.2
@@ -25,11 +32,19 @@ fastapi==0.115.14
greenlet==3.4.0
# via sqlalchemy
h11==0.16.0
# via uvicorn
# via
# httpcore
# uvicorn
httpcore==1.0.9
# via httpx
httptools==0.7.1
# via uvicorn
httpx==0.28.1
# via -r requirements.in
idna==3.11
# via anyio
# via
# anyio
# httpx
jinja2==3.1.6
# via -r requirements.in
mako==1.3.11
@@ -38,6 +53,8 @@ markupsafe==3.0.3
# via
# jinja2
# mako
pycparser==2.23
# via cffi
pydantic==2.13.2
# via
# fastapi
@@ -52,8 +69,6 @@ python-dotenv==1.2.2
# uvicorn
python-multipart==0.0.26
# via -r requirements.in
pycparser==2.23
# via cffi
pyyaml==6.0.3
# via
# -r requirements.in
@@ -76,6 +91,8 @@ typing-inspection==0.4.2
# via
# pydantic
# pydantic-settings
tzlocal==5.3.1
# via apscheduler
uvicorn[standard]==0.44.0
# via -r requirements.in
uvloop==0.22.1
+1 -1
View File
@@ -15,7 +15,7 @@ if str(PROJECT_ROOT) not in sys.path:
from app.config import get_settings
APP_BASELINE_REVISION = "20260420_04_app_config_table"
APP_BASELINE_REVISION = "20260611_06_merge_location_poo_tables"
class AppDatabaseAdoptionError(RuntimeError):
-205
View File
@@ -1,205 +0,0 @@
from __future__ import annotations
import sqlite3
import sys
from pathlib import Path
from alembic import command
from alembic.config import Config
from alembic.script import ScriptDirectory
from alembic.util.exc import CommandError
PROJECT_ROOT = Path(__file__).resolve().parents[1]
if str(PROJECT_ROOT) not in sys.path:
sys.path.insert(0, str(PROJECT_ROOT))
from app.config import get_settings
LOCATION_BASELINE_REVISION = "20260419_01_location_baseline"
EXPECTED_USER_VERSION = 2
EXPECTED_LOCATION_TABLE_INFO = [
(0, "person", "TEXT", 1, None, 1),
(1, "datetime", "TEXT", 1, None, 2),
(2, "latitude", "REAL", 1, None, 0),
(3, "longitude", "REAL", 1, None, 0),
(4, "altitude", "REAL", 0, None, 0),
]
class LocationDatabaseAdoptionError(RuntimeError):
"""Raised when a legacy location database does not match the expected baseline."""
def _database_path_from_url(database_url: str) -> Path:
prefix = "sqlite:///"
if not database_url.startswith(prefix):
raise LocationDatabaseAdoptionError(
f"Only sqlite URLs are supported for location DB adoption, got: {database_url}"
)
return Path(database_url[len(prefix) :])
def _make_alembic_config(database_url: str) -> Config:
config = Config("alembic_location.ini")
config.set_main_option("sqlalchemy.url", database_url)
return config
def _expected_head_revision(alembic_config: Config) -> str:
script = ScriptDirectory.from_config(alembic_config)
heads = script.get_heads()
if len(heads) != 1:
raise LocationDatabaseAdoptionError(
f"Expected exactly one Alembic head for location DB, got {len(heads)}"
)
return heads[0]
def _is_known_revision(alembic_config: Config, revision: str) -> bool:
script = ScriptDirectory.from_config(alembic_config)
try:
return script.get_revision(revision) is not None
except CommandError:
return False
def _location_table_exists(database_path: Path) -> bool:
conn = sqlite3.connect(database_path)
try:
row = conn.execute(
"SELECT 1 FROM sqlite_master WHERE type = 'table' AND name = 'location'"
).fetchone()
return row is not None
finally:
conn.close()
def _alembic_version_table_exists(database_path: Path) -> bool:
conn = sqlite3.connect(database_path)
try:
row = conn.execute(
"SELECT 1 FROM sqlite_master WHERE type = 'table' AND name = 'alembic_version'"
).fetchone()
return row is not None
finally:
conn.close()
def _fetch_alembic_revision(database_path: Path) -> str:
conn = sqlite3.connect(database_path)
try:
row = conn.execute("SELECT version_num FROM alembic_version").fetchone()
if row is None:
raise LocationDatabaseAdoptionError(
"Alembic version table exists but contains no revision"
)
return row[0]
finally:
conn.close()
def _fetch_location_table_info(database_path: Path) -> list[tuple]:
conn = sqlite3.connect(database_path)
try:
return list(conn.execute("PRAGMA table_info(location)"))
finally:
conn.close()
def _fetch_user_version(database_path: Path) -> int:
conn = sqlite3.connect(database_path)
try:
return conn.execute("PRAGMA user_version").fetchone()[0]
finally:
conn.close()
def validate_legacy_location_db(database_url: str) -> None:
database_path = _database_path_from_url(database_url)
if not database_path.exists():
raise LocationDatabaseAdoptionError(f"Location DB file does not exist: {database_path}")
if not _location_table_exists(database_path):
raise LocationDatabaseAdoptionError("Expected table 'location' was not found in the DB")
table_info = _fetch_location_table_info(database_path)
if table_info != EXPECTED_LOCATION_TABLE_INFO:
raise LocationDatabaseAdoptionError(
"Location table schema does not match the expected baseline schema"
)
user_version = _fetch_user_version(database_path)
if user_version != EXPECTED_USER_VERSION:
raise LocationDatabaseAdoptionError(
f"Expected PRAGMA user_version = {EXPECTED_USER_VERSION}, got {user_version}"
)
def validate_location_runtime_db(database_url: str) -> None:
database_path = _database_path_from_url(database_url)
alembic_config = _make_alembic_config(database_url)
expected_revision = _expected_head_revision(alembic_config)
if not database_path.exists():
raise LocationDatabaseAdoptionError(
"Location DB file was not found. Run 'python scripts/location_db_adopt.py' "
"first to initialize or adopt the location DB before starting the app."
)
if not _alembic_version_table_exists(database_path):
raise LocationDatabaseAdoptionError(
"Location DB exists but is not yet Alembic-managed. Run "
"'python scripts/location_db_adopt.py' first to adopt the legacy DB "
"before starting the app."
)
current_revision = _fetch_alembic_revision(database_path)
if current_revision != expected_revision:
raise LocationDatabaseAdoptionError(
"Location DB revision mismatch. Refusing to start the app: "
f"expected {expected_revision}, got {current_revision}"
)
def adopt_or_initialize_location_db(database_url: str) -> str:
database_path = _database_path_from_url(database_url)
alembic_config = _make_alembic_config(database_url)
expected_revision = _expected_head_revision(alembic_config)
if database_path.exists():
if _alembic_version_table_exists(database_path):
current_revision = _fetch_alembic_revision(database_path)
if current_revision == expected_revision:
return "already_managed"
if not _is_known_revision(alembic_config, current_revision):
raise LocationDatabaseAdoptionError(
"Location DB is already Alembic-managed but revision does not match "
f"a known migration revision: got {current_revision}"
)
command.upgrade(alembic_config, "head")
return "upgraded"
validate_legacy_location_db(database_url)
command.stamp(alembic_config, LOCATION_BASELINE_REVISION)
if LOCATION_BASELINE_REVISION != expected_revision:
command.upgrade(alembic_config, "head")
return "upgraded"
return "adopted"
database_path.parent.mkdir(parents=True, exist_ok=True)
command.upgrade(alembic_config, "head")
return "initialized"
def main() -> None:
settings = get_settings()
result = adopt_or_initialize_location_db(settings.location_database_url)
if result == "initialized":
print("Initialized a new location DB via Alembic upgrade head.")
elif result == "already_managed":
print("Location DB is already Alembic-managed at the expected baseline revision.")
else:
print("Validated legacy location DB and stamped Alembic baseline successfully.")
if __name__ == "__main__":
main()
+267
View File
@@ -0,0 +1,267 @@
"""One-time idempotent data migration: copy rows from legacy locationRecorder.db /
pooRecorder.db into the unified app DB's location / poo_records tables.
NOT part of the Alembic chain. Run manually, once, during production cut-over:
python -m scripts.migrate_legacy_data \\
--app-db sqlite:///./data/app.db \\
--location-db sqlite:///./data/locationRecorder.db \\
--poo-db sqlite:///./data/pooRecorder.db
Or rely on environment variables:
APP_DATABASE_URL, LOCATION_DATABASE_URL, POO_DATABASE_URL
Add --dry-run to preview row counts without writing anything.
Return value of migrate_legacy_data(): a dict shaped like:
{
"location": {"source": N, "copied": C, "skipped": bool, "final": F},
"poo_records": {"source": N, "copied": C, "skipped": bool, "final": F},
}
where:
source - rows in the legacy DB (0 when skipped)
copied - rows inserted by this run (0 when dry_run or skipped)
skipped - True when the legacy file was absent
final - rows present in the app table after the run (0 when dry_run)
"""
from __future__ import annotations
import argparse
import os
import sqlite3
import sys
from pathlib import Path
# ---------------------------------------------------------------------------
# Internal helpers
# ---------------------------------------------------------------------------
def _sqlite_path_from_url(url: str) -> Path:
"""Extract the filesystem path from a sqlite:///... URL.
If *url* does not start with 'sqlite:///', it is treated as a plain path.
"""
prefix = "sqlite:///"
if url.startswith(prefix):
return Path(url[len(prefix):])
return Path(url)
def _reconcile(
conn: sqlite3.Connection,
table: str,
columns: list[str],
source_count: int,
) -> int:
"""Verify every legacy source row is present in the main (app) table.
Matches on ALL columns using SQLite's NULL-safe IS operator so that nullable
columns (e.g. altitude) compare correctly. A row that was silently skipped
by INSERT OR IGNORE due to a value difference will NOT satisfy this predicate
even if its primary key is present in the target.
Returns the count of source rows whose full-row data is present in main.
Raises RuntimeError if any rows are missing or differ in value.
"""
join_cond = " AND ".join(f"m.{col} IS l.{col}" for col in columns)
sql = (
f"SELECT COUNT(*) FROM legacy.{table} l "
f"WHERE EXISTS (SELECT 1 FROM main.{table} m WHERE {join_cond})"
)
(present,) = conn.execute(sql).fetchone()
if present < source_count:
missing = source_count - present
raise RuntimeError(
f"Reconciliation failed for table '{table}': "
f"{missing} of {source_count} source rows are missing or differing in the app DB."
)
return present
# ---------------------------------------------------------------------------
# Public API
# ---------------------------------------------------------------------------
def migrate_legacy_data(
app_url: str,
location_url: str | None,
poo_url: str | None,
*,
dry_run: bool = False,
) -> dict:
"""Copy rows from legacy DBs into the app DB's location / poo_records tables.
Parameters
----------
app_url: sqlite:///... URL (or plain path) for the unified app DB.
location_url: sqlite:///... URL (or plain path) for the legacy location DB,
or None to skip that table.
poo_url: sqlite:///... URL (or plain path) for the legacy poo DB,
or None to skip that table.
dry_run: When True, gather counts only; perform no writes.
Returns a dict with per-table stats (see module docstring).
Raises RuntimeError on reconciliation failure (non-zero rows missing).
"""
app_path = _sqlite_path_from_url(app_url)
results: dict[str, dict] = {}
# --- location table ---
results["location"] = _migrate_table(
app_path=app_path,
legacy_url=location_url,
table="location",
columns=["person", "datetime", "latitude", "longitude", "altitude"],
dry_run=dry_run,
)
# --- poo_records table ---
results["poo_records"] = _migrate_table(
app_path=app_path,
legacy_url=poo_url,
table="poo_records",
columns=["timestamp", "status", "latitude", "longitude"],
dry_run=dry_run,
)
return results
def _migrate_table(
*,
app_path: Path,
legacy_url: str | None,
table: str,
columns: list[str],
dry_run: bool,
) -> dict:
"""Migrate a single table from a legacy DB into the app DB.
Returns a per-table stats dict.
"""
# If the caller passed None → treat as absent
if legacy_url is None:
return {"source": 0, "copied": 0, "skipped": True, "final": 0}
legacy_path = _sqlite_path_from_url(legacy_url)
# If the file doesn't exist → safe no-op
if not legacy_path.exists():
return {"source": 0, "copied": 0, "skipped": True, "final": 0}
col_list = ", ".join(columns)
conn = sqlite3.connect(app_path)
try:
conn.execute("ATTACH DATABASE ? AS legacy", (str(legacy_path),))
# Count source rows
(source_count,) = conn.execute(f"SELECT COUNT(*) FROM legacy.{table}").fetchone()
if dry_run:
conn.execute("DETACH DATABASE legacy")
return {
"source": source_count,
"copied": 0,
"skipped": False,
"final": 0,
}
# Count rows already in the target before this run
(before_count,) = conn.execute(f"SELECT COUNT(*) FROM main.{table}").fetchone()
# Idempotent insert — PK conflict → skip
conn.execute(
f"INSERT OR IGNORE INTO main.{table} ({col_list}) "
f"SELECT {col_list} FROM legacy.{table}"
)
conn.commit()
# Count rows now
(after_count,) = conn.execute(f"SELECT COUNT(*) FROM main.{table}").fetchone()
copied = after_count - before_count
# Reconciliation: every source row must be present with matching values
_reconcile(conn, table, columns, source_count)
conn.execute("DETACH DATABASE legacy")
finally:
conn.close()
return {
"source": source_count,
"copied": copied,
"skipped": False,
"final": after_count,
}
# ---------------------------------------------------------------------------
# CLI entry point
# ---------------------------------------------------------------------------
def main() -> None:
parser = argparse.ArgumentParser(
description="Migrate legacy location/poo data into the unified app DB."
)
parser.add_argument(
"--app-db",
default=os.environ.get("APP_DATABASE_URL"),
help="sqlite:///... URL or path for the app DB "
"(default: $APP_DATABASE_URL)",
)
parser.add_argument(
"--location-db",
default=os.environ.get("LOCATION_DATABASE_URL"),
help="sqlite:///... URL or path for the legacy location DB "
"(default: $LOCATION_DATABASE_URL). Omit to skip location table.",
)
parser.add_argument(
"--poo-db",
default=os.environ.get("POO_DATABASE_URL"),
help="sqlite:///... URL or path for the legacy poo DB "
"(default: $POO_DATABASE_URL). Omit to skip poo_records table.",
)
parser.add_argument(
"--dry-run",
action="store_true",
help="Report counts only; do not write any rows.",
)
args = parser.parse_args()
if not args.app_db:
parser.error(
"App DB not specified. Pass --app-db or set APP_DATABASE_URL."
)
try:
results = migrate_legacy_data(
app_url=args.app_db,
location_url=args.location_db,
poo_url=args.poo_db,
dry_run=args.dry_run,
)
except RuntimeError as exc:
print(f"ERROR: {exc}", file=sys.stderr)
sys.exit(1)
prefix = "[DRY RUN] " if args.dry_run else ""
print(f"{prefix}Migration results:")
for table_name, stats in results.items():
if stats["skipped"]:
print(f" {table_name}: SKIPPED (legacy file absent or not provided)")
else:
print(
f" {table_name}: source={stats['source']}, "
f"copied={stats['copied']}, final={stats['final']}"
)
if __name__ == "__main__":
main()
-200
View File
@@ -1,200 +0,0 @@
from __future__ import annotations
import sqlite3
import sys
from pathlib import Path
from alembic import command
from alembic.config import Config
from alembic.script import ScriptDirectory
from alembic.util.exc import CommandError
PROJECT_ROOT = Path(__file__).resolve().parents[1]
if str(PROJECT_ROOT) not in sys.path:
sys.path.insert(0, str(PROJECT_ROOT))
from app.config import get_settings
POO_BASELINE_REVISION = "20260420_01_poo_baseline"
EXPECTED_USER_VERSION = 1
EXPECTED_POO_TABLE_INFO = [
(0, "timestamp", "TEXT", 1, None, 1),
(1, "status", "TEXT", 1, None, 0),
(2, "latitude", "REAL", 1, None, 0),
(3, "longitude", "REAL", 1, None, 0),
]
class PooDatabaseAdoptionError(RuntimeError):
"""Raised when a legacy poo database does not match the expected baseline."""
def _database_path_from_url(database_url: str) -> Path:
prefix = "sqlite:///"
if not database_url.startswith(prefix):
raise PooDatabaseAdoptionError(
f"Only sqlite URLs are supported for poo DB adoption, got: {database_url}"
)
return Path(database_url[len(prefix) :])
def _make_alembic_config(database_url: str) -> Config:
config = Config("alembic_poo.ini")
config.set_main_option("sqlalchemy.url", database_url)
return config
def _expected_head_revision(alembic_config: Config) -> str:
script = ScriptDirectory.from_config(alembic_config)
heads = script.get_heads()
if len(heads) != 1:
raise PooDatabaseAdoptionError(
f"Expected exactly one Alembic head for poo DB, got {len(heads)}"
)
return heads[0]
def _is_known_revision(alembic_config: Config, revision: str) -> bool:
script = ScriptDirectory.from_config(alembic_config)
try:
return script.get_revision(revision) is not None
except CommandError:
return False
def _poo_table_exists(database_path: Path) -> bool:
conn = sqlite3.connect(database_path)
try:
row = conn.execute(
"SELECT 1 FROM sqlite_master WHERE type = 'table' AND name = 'poo_records'"
).fetchone()
return row is not None
finally:
conn.close()
def _alembic_version_table_exists(database_path: Path) -> bool:
conn = sqlite3.connect(database_path)
try:
row = conn.execute(
"SELECT 1 FROM sqlite_master WHERE type = 'table' AND name = 'alembic_version'"
).fetchone()
return row is not None
finally:
conn.close()
def _fetch_alembic_revision(database_path: Path) -> str:
conn = sqlite3.connect(database_path)
try:
row = conn.execute("SELECT version_num FROM alembic_version").fetchone()
if row is None:
raise PooDatabaseAdoptionError("Alembic version table exists but contains no revision")
return row[0]
finally:
conn.close()
def _fetch_poo_table_info(database_path: Path) -> list[tuple]:
conn = sqlite3.connect(database_path)
try:
return list(conn.execute("PRAGMA table_info(poo_records)"))
finally:
conn.close()
def _fetch_user_version(database_path: Path) -> int:
conn = sqlite3.connect(database_path)
try:
return conn.execute("PRAGMA user_version").fetchone()[0]
finally:
conn.close()
def validate_legacy_poo_db(database_url: str) -> None:
database_path = _database_path_from_url(database_url)
if not database_path.exists():
raise PooDatabaseAdoptionError(f"Poo DB file does not exist: {database_path}")
if not _poo_table_exists(database_path):
raise PooDatabaseAdoptionError("Expected table 'poo_records' was not found in the DB")
table_info = _fetch_poo_table_info(database_path)
if table_info != EXPECTED_POO_TABLE_INFO:
raise PooDatabaseAdoptionError("Poo table schema does not match the expected baseline")
user_version = _fetch_user_version(database_path)
if user_version != EXPECTED_USER_VERSION:
raise PooDatabaseAdoptionError(
f"Expected PRAGMA user_version = {EXPECTED_USER_VERSION}, got {user_version}"
)
def validate_poo_runtime_db(database_url: str) -> None:
database_path = _database_path_from_url(database_url)
alembic_config = _make_alembic_config(database_url)
expected_revision = _expected_head_revision(alembic_config)
if not database_path.exists():
raise PooDatabaseAdoptionError(
"Poo DB file was not found. Run 'python scripts/poo_db_adopt.py' first to "
"initialize or adopt the poo DB before starting the app."
)
if not _alembic_version_table_exists(database_path):
raise PooDatabaseAdoptionError(
"Poo DB exists but is not yet Alembic-managed. Run "
"'python scripts/poo_db_adopt.py' first to adopt the legacy DB "
"before starting the app."
)
current_revision = _fetch_alembic_revision(database_path)
if current_revision != expected_revision:
raise PooDatabaseAdoptionError(
"Poo DB revision mismatch. Refusing to start the app: "
f"expected {expected_revision}, got {current_revision}"
)
def adopt_or_initialize_poo_db(database_url: str) -> str:
database_path = _database_path_from_url(database_url)
alembic_config = _make_alembic_config(database_url)
expected_revision = _expected_head_revision(alembic_config)
if database_path.exists():
if _alembic_version_table_exists(database_path):
current_revision = _fetch_alembic_revision(database_path)
if current_revision == expected_revision:
return "already_managed"
if not _is_known_revision(alembic_config, current_revision):
raise PooDatabaseAdoptionError(
"Poo DB is already Alembic-managed but revision does not match "
f"a known migration revision: got {current_revision}"
)
command.upgrade(alembic_config, "head")
return "upgraded"
validate_legacy_poo_db(database_url)
command.stamp(alembic_config, POO_BASELINE_REVISION)
if POO_BASELINE_REVISION != expected_revision:
command.upgrade(alembic_config, "head")
return "upgraded"
return "adopted"
database_path.parent.mkdir(parents=True, exist_ok=True)
command.upgrade(alembic_config, "head")
return "initialized"
def main() -> None:
settings = get_settings()
result = adopt_or_initialize_poo_db(settings.poo_database_url)
if result == "initialized":
print("Initialized a new poo DB via Alembic upgrade head.")
elif result == "already_managed":
print("Poo DB is already Alembic-managed at the expected baseline revision.")
else:
print("Validated legacy poo DB and stamped Alembic baseline successfully.")
if __name__ == "__main__":
main()
-4
View File
@@ -2,16 +2,12 @@ from __future__ import annotations
from app.config import get_settings
from scripts.app_db_adopt import adopt_or_initialize_app_db
from scripts.location_db_adopt import adopt_or_initialize_location_db
from scripts.poo_db_adopt import adopt_or_initialize_poo_db
def run_all_migrations() -> dict[str, str]:
settings = get_settings()
return {
"app": adopt_or_initialize_app_db(settings.app_database_url),
"location": adopt_or_initialize_location_db(settings.location_database_url),
"poo": adopt_or_initialize_poo_db(settings.poo_database_url),
}
+12 -74
View File
@@ -5,10 +5,8 @@ from alembic import command
from alembic.config import Config
from fastapi.testclient import TestClient
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from app.auth_db import reset_auth_db_caches
import app.db as app_db
from app.db import reset_db_caches
from app.config import get_settings
from app.main import create_app
@@ -19,74 +17,40 @@ def _make_app_alembic_config(database_url: str) -> Config:
return config
def _make_alembic_config(database_url: str) -> Config:
config = Config("alembic_location.ini")
config.set_main_option("sqlalchemy.url", database_url)
return config
def _make_poo_alembic_config(database_url: str) -> Config:
config = Config("alembic_poo.ini")
config.set_main_option("sqlalchemy.url", database_url)
return config
@pytest.fixture
def test_database_urls(tmp_path: Path, monkeypatch: pytest.MonkeyPatch):
app_database_path = tmp_path / "app_test.db"
location_database_path = tmp_path / "location_test.db"
poo_database_path = tmp_path / "poo_placeholder.db"
app_database_url = f"sqlite:///{app_database_path}"
location_database_url = f"sqlite:///{location_database_path}"
poo_database_url = f"sqlite:///{poo_database_path}"
monkeypatch.setenv("APP_DATABASE_URL", app_database_url)
monkeypatch.setenv("LOCATION_DATABASE_URL", location_database_url)
monkeypatch.setenv("POO_DATABASE_URL", poo_database_url)
monkeypatch.setenv("AUTH_BOOTSTRAP_USERNAME", "admin")
monkeypatch.setenv("AUTH_BOOTSTRAP_PASSWORD", "test-password")
monkeypatch.setenv("AUTH_COOKIE_SECURE_OVERRIDE", "false")
get_settings.cache_clear()
reset_auth_db_caches()
reset_db_caches()
try:
yield {
"app_path": app_database_path,
"app_url": app_database_url,
"location_path": location_database_path,
"location_url": location_database_url,
"poo_path": poo_database_path,
"poo_url": poo_database_url,
}
finally:
get_settings.cache_clear()
reset_auth_db_caches()
@pytest.fixture
def ready_location_database(test_database_urls):
command.upgrade(_make_alembic_config(test_database_urls["location_url"]), "head")
return test_database_urls
@pytest.fixture
def ready_poo_database(test_database_urls):
command.upgrade(_make_poo_alembic_config(test_database_urls["poo_url"]), "head")
return test_database_urls
reset_db_caches()
@pytest.fixture
def auth_database(test_database_urls, monkeypatch: pytest.MonkeyPatch):
database_url = test_database_urls["app_url"]
command.upgrade(_make_app_alembic_config(database_url), "head")
reset_auth_db_caches()
reset_db_caches()
yield test_database_urls
reset_auth_db_caches()
reset_db_caches()
@pytest.fixture
def app(ready_location_database, ready_poo_database, auth_database):
def app(auth_database):
yield create_app()
@@ -97,46 +61,20 @@ def client(app):
@pytest.fixture
def location_client(
ready_location_database,
ready_poo_database,
auth_database,
monkeypatch: pytest.MonkeyPatch,
):
database_url = ready_location_database["location_url"]
engine = create_engine(database_url, connect_args={"check_same_thread": False})
session_local = sessionmaker(bind=engine, autoflush=False, autocommit=False)
monkeypatch.setattr(app_db, "engine", engine)
monkeypatch.setattr(app_db, "SessionLocal", session_local)
def location_client(auth_database):
app_url = auth_database["app_url"]
engine = create_engine(app_url, connect_args={"check_same_thread": False})
fastapi_app = create_app()
with TestClient(fastapi_app) as client:
yield client, engine
engine.dispose()
@pytest.fixture
def poo_client(
ready_location_database,
ready_poo_database,
auth_database,
monkeypatch: pytest.MonkeyPatch,
):
database_url = ready_poo_database["poo_url"]
engine = create_engine(database_url, connect_args={"check_same_thread": False})
session_local = sessionmaker(bind=engine, autoflush=False, autocommit=False)
import app.poo_db as poo_db
monkeypatch.setattr(poo_db, "poo_engine", engine)
monkeypatch.setattr(poo_db, "PooSessionLocal", session_local)
def poo_client(auth_database):
app_url = auth_database["app_url"]
engine = create_engine(app_url, connect_args={"check_same_thread": False})
fastapi_app = create_app()
with TestClient(fastapi_app) as client:
yield client, engine
engine.dispose()
+8 -121
View File
@@ -5,11 +5,11 @@ import pytest
from alembic import command
from fastapi.testclient import TestClient
from app.auth_db import reset_auth_db_caches
from app.db import reset_db_caches
from app.config import get_settings
from app.main import create_app
from scripts.app_db_adopt import APP_BASELINE_REVISION, adopt_or_initialize_app_db
from tests.conftest import _make_alembic_config, _make_app_alembic_config, _make_poo_alembic_config
from tests.conftest import _make_app_alembic_config
async def _run_lifespan(app) -> None:
@@ -38,18 +38,12 @@ def test_status_endpoint(client: TestClient) -> None:
def test_app_start_fails_when_app_db_missing(tmp_path, monkeypatch: pytest.MonkeyPatch) -> None:
missing_app_path = tmp_path / "missing_app.db"
poo_database_path = tmp_path / "poo_ready.db"
location_database_path = tmp_path / "location_ready.db"
command.upgrade(_make_poo_alembic_config(f"sqlite:///{poo_database_path}"), "head")
command.upgrade(_make_alembic_config(f"sqlite:///{location_database_path}"), "head")
monkeypatch.setenv("APP_DATABASE_URL", f"sqlite:///{missing_app_path}")
monkeypatch.setenv("AUTH_BOOTSTRAP_USERNAME", "admin")
monkeypatch.setenv("AUTH_BOOTSTRAP_PASSWORD", "test-password")
monkeypatch.setenv("LOCATION_DATABASE_URL", f"sqlite:///{location_database_path}")
monkeypatch.setenv("POO_DATABASE_URL", f"sqlite:///{poo_database_path}")
get_settings.cache_clear()
reset_auth_db_caches()
reset_db_caches()
app = create_app()
with pytest.raises(RuntimeError, match="Run 'python scripts/app_db_adopt.py' first"):
@@ -58,7 +52,7 @@ def test_app_start_fails_when_app_db_missing(tmp_path, monkeypatch: pytest.Monke
assert not missing_app_path.exists()
get_settings.cache_clear()
reset_auth_db_caches()
reset_db_caches()
def test_app_db_adoption_initializes_new_database(tmp_path) -> None:
@@ -86,10 +80,6 @@ def test_app_start_seeds_missing_config_from_env_without_overwriting_existing_va
tmp_path, monkeypatch: pytest.MonkeyPatch
) -> None:
app_database_url = _prepare_app_db(tmp_path)
location_database_path = tmp_path / "location_ready.db"
poo_database_path = tmp_path / "poo_ready.db"
command.upgrade(_make_alembic_config(f"sqlite:///{location_database_path}"), "head")
command.upgrade(_make_poo_alembic_config(f"sqlite:///{poo_database_path}"), "head")
app_database_path = tmp_path / "app_ready.db"
conn = sqlite3.connect(app_database_path)
@@ -105,10 +95,8 @@ def test_app_start_seeds_missing_config_from_env_without_overwriting_existing_va
monkeypatch.setenv("AUTH_BOOTSTRAP_PASSWORD", "test-password")
monkeypatch.setenv("APP_NAME", "Bootstrap Name")
monkeypatch.setenv("HOME_ASSISTANT_BASE_URL", "http://bootstrap-ha.local:8123")
monkeypatch.setenv("LOCATION_DATABASE_URL", f"sqlite:///{location_database_path}")
monkeypatch.setenv("POO_DATABASE_URL", f"sqlite:///{poo_database_path}")
get_settings.cache_clear()
reset_auth_db_caches()
reset_db_caches()
app = create_app()
anyio.run(_run_lifespan, app)
@@ -124,17 +112,13 @@ def test_app_start_seeds_missing_config_from_env_without_overwriting_existing_va
assert rows["AUTH_SESSION_COOKIE_NAME"] == "home_automation_session"
get_settings.cache_clear()
reset_auth_db_caches()
reset_db_caches()
def test_app_start_syncs_app_hostname_from_env_even_when_db_has_old_value(
tmp_path, monkeypatch: pytest.MonkeyPatch
) -> None:
app_database_url = _prepare_app_db(tmp_path)
location_database_path = tmp_path / "location_ready.db"
poo_database_path = tmp_path / "poo_ready.db"
command.upgrade(_make_alembic_config(f"sqlite:///{location_database_path}"), "head")
command.upgrade(_make_poo_alembic_config(f"sqlite:///{poo_database_path}"), "head")
app_database_path = tmp_path / "app_ready.db"
conn = sqlite3.connect(app_database_path)
@@ -149,10 +133,8 @@ def test_app_start_syncs_app_hostname_from_env_even_when_db_has_old_value(
monkeypatch.setenv("AUTH_BOOTSTRAP_USERNAME", "admin")
monkeypatch.setenv("AUTH_BOOTSTRAP_PASSWORD", "test-password")
monkeypatch.setenv("APP_HOSTNAME", "new.example.com")
monkeypatch.setenv("LOCATION_DATABASE_URL", f"sqlite:///{location_database_path}")
monkeypatch.setenv("POO_DATABASE_URL", f"sqlite:///{poo_database_path}")
get_settings.cache_clear()
reset_auth_db_caches()
reset_db_caches()
app = create_app()
anyio.run(_run_lifespan, app)
@@ -166,99 +148,4 @@ def test_app_start_syncs_app_hostname_from_env_even_when_db_has_old_value(
assert rows["APP_HOSTNAME"] == "new.example.com"
get_settings.cache_clear()
reset_auth_db_caches()
def test_app_start_fails_when_location_db_missing(
tmp_path, monkeypatch: pytest.MonkeyPatch
) -> None:
app_database_url = _prepare_app_db(tmp_path)
monkeypatch.setenv("APP_DATABASE_URL", app_database_url)
monkeypatch.setenv("AUTH_BOOTSTRAP_USERNAME", "admin")
monkeypatch.setenv("AUTH_BOOTSTRAP_PASSWORD", "test-password")
poo_database_path = tmp_path / "poo_ready.db"
command.upgrade(_make_poo_alembic_config(f"sqlite:///{poo_database_path}"), "head")
monkeypatch.setenv("LOCATION_DATABASE_URL", f"sqlite:///{tmp_path / 'missing.db'}")
monkeypatch.setenv("POO_DATABASE_URL", f"sqlite:///{poo_database_path}")
get_settings.cache_clear()
reset_auth_db_caches()
app = create_app()
with pytest.raises(RuntimeError, match="Run 'python scripts/location_db_adopt.py' first"):
anyio.run(_run_lifespan, app)
get_settings.cache_clear()
reset_auth_db_caches()
def test_app_start_fails_when_location_db_exists_but_is_not_adopted(
tmp_path, monkeypatch: pytest.MonkeyPatch
) -> None:
app_database_url = _prepare_app_db(tmp_path)
monkeypatch.setenv("APP_DATABASE_URL", app_database_url)
monkeypatch.setenv("AUTH_BOOTSTRAP_USERNAME", "admin")
monkeypatch.setenv("AUTH_BOOTSTRAP_PASSWORD", "test-password")
poo_database_path = tmp_path / "poo_ready.db"
command.upgrade(_make_poo_alembic_config(f"sqlite:///{poo_database_path}"), "head")
database_path = tmp_path / "legacy_only.db"
conn = sqlite3.connect(database_path)
conn.execute(
"""
CREATE TABLE location (
person TEXT NOT NULL,
datetime TEXT NOT NULL,
latitude REAL NOT NULL,
longitude REAL NOT NULL,
altitude REAL,
PRIMARY KEY (person, datetime)
)
"""
)
conn.execute("PRAGMA user_version = 2")
conn.commit()
conn.close()
monkeypatch.setenv("LOCATION_DATABASE_URL", f"sqlite:///{database_path}")
monkeypatch.setenv("POO_DATABASE_URL", f"sqlite:///{poo_database_path}")
get_settings.cache_clear()
reset_auth_db_caches()
app = create_app()
with pytest.raises(RuntimeError, match="is not yet Alembic-managed"):
anyio.run(_run_lifespan, app)
get_settings.cache_clear()
reset_auth_db_caches()
def test_app_start_fails_when_location_db_revision_mismatches(
tmp_path, monkeypatch: pytest.MonkeyPatch
) -> None:
app_database_url = _prepare_app_db(tmp_path)
monkeypatch.setenv("APP_DATABASE_URL", app_database_url)
monkeypatch.setenv("AUTH_BOOTSTRAP_USERNAME", "admin")
monkeypatch.setenv("AUTH_BOOTSTRAP_PASSWORD", "test-password")
poo_database_path = tmp_path / "poo_ready.db"
command.upgrade(_make_poo_alembic_config(f"sqlite:///{poo_database_path}"), "head")
database_path = tmp_path / "wrong_revision.db"
command.upgrade(_make_alembic_config(f"sqlite:///{database_path}"), "head")
conn = sqlite3.connect(database_path)
conn.execute("UPDATE alembic_version SET version_num = 'wrong_revision'")
conn.commit()
conn.close()
monkeypatch.setenv("LOCATION_DATABASE_URL", f"sqlite:///{database_path}")
monkeypatch.setenv("POO_DATABASE_URL", f"sqlite:///{poo_database_path}")
get_settings.cache_clear()
reset_auth_db_caches()
app = create_app()
with pytest.raises(RuntimeError, match="Location DB revision mismatch"):
anyio.run(_run_lifespan, app)
get_settings.cache_clear()
reset_auth_db_caches()
reset_db_caches()
+2 -5
View File
@@ -4,7 +4,7 @@ from pathlib import Path
from fastapi.testclient import TestClient
from app.auth_db import reset_auth_db_caches
from app.db import reset_db_caches
from app.config import get_settings
from app.main import create_app
@@ -194,9 +194,6 @@ def test_config_page_update_persists_to_database(
def test_config_page_shows_ticktick_oauth_link_when_ticktick_is_configured(
test_database_urls,
ready_location_database,
ready_poo_database,
auth_database,
monkeypatch,
) -> None:
@@ -205,7 +202,7 @@ def test_config_page_shows_ticktick_oauth_link_when_ticktick_is_configured(
monkeypatch.setenv("TICKTICK_CLIENT_ID", "ticktick-client-id")
monkeypatch.setenv("TICKTICK_CLIENT_SECRET", "ticktick-client-secret")
get_settings.cache_clear()
reset_auth_db_caches()
reset_db_caches()
with TestClient(create_app()) as client:
login_page = client.get("/login")
+25 -9
View File
@@ -1,10 +1,8 @@
from app.config import Settings
def test_settings_support_two_independent_database_urls(monkeypatch) -> None:
def test_settings_load_core_fields_from_env(monkeypatch) -> None:
monkeypatch.setenv("APP_DATABASE_URL", "sqlite:///./data/app.db")
monkeypatch.setenv("LOCATION_DATABASE_URL", "sqlite:///./data/locationRecorder.db")
monkeypatch.setenv("POO_DATABASE_URL", "sqlite:///./data/pooRecorder.db")
monkeypatch.setenv("APP_HOSTNAME", "home.example.com")
monkeypatch.setenv("POO_WEBHOOK_ID", "poo-hook")
monkeypatch.setenv("POO_SENSOR_ENTITY_NAME", "sensor.test_poo_status")
@@ -21,8 +19,6 @@ def test_settings_support_two_independent_database_urls(monkeypatch) -> None:
settings = Settings(_env_file=None)
assert settings.app_database_url == "sqlite:///./data/app.db"
assert settings.location_database_url == "sqlite:///./data/locationRecorder.db"
assert settings.poo_database_url == "sqlite:///./data/pooRecorder.db"
assert settings.poo_webhook_id == "poo-hook"
assert settings.poo_sensor_entity_name == "sensor.test_poo_status"
assert settings.poo_sensor_friendly_name == "Poo Status"
@@ -36,12 +32,8 @@ def test_settings_support_two_independent_database_urls(monkeypatch) -> None:
assert settings.auth_bootstrap_password == "secret"
assert settings.auth_session_cookie_name == "auth_cookie"
assert settings.auth_session_ttl_hours == 8
assert settings.location_sqlite_path is not None
assert settings.location_sqlite_path.name == "locationRecorder.db"
assert settings.app_sqlite_path is not None
assert settings.app_sqlite_path.name == "app.db"
assert settings.poo_sqlite_path is not None
assert settings.poo_sqlite_path.name == "pooRecorder.db"
assert settings.auth_cookie_secure is True
@@ -53,3 +45,27 @@ def test_settings_derive_development_ticktick_redirect_uri(monkeypatch) -> None:
assert settings.app_base_url == "http://localhost:11001"
assert settings.ticktick_redirect_uri == "http://localhost:11001/ticktick/auth/code"
def test_settings_support_smtp_fields(monkeypatch) -> None:
monkeypatch.setenv("SMTP_ENABLED", "true")
monkeypatch.setenv("SMTP_HOST", "smtp.example.com")
monkeypatch.setenv("SMTP_PORT", "2525")
monkeypatch.setenv("SMTP_USERNAME", "smtp-user")
monkeypatch.setenv("SMTP_PASSWORD", "smtp-password")
monkeypatch.setenv("SMTP_FROM_NAME", "Home Automation")
monkeypatch.setenv("SMTP_FROM_ADDRESS", "sender@example.com")
monkeypatch.setenv("SMTP_TO_ADDRESS", "recipient@example.com")
monkeypatch.setenv("SMTP_USE_STARTTLS", "false")
settings = Settings(_env_file=None)
assert settings.smtp_enabled is True
assert settings.smtp_host == "smtp.example.com"
assert settings.smtp_port == 2525
assert settings.smtp_username == "smtp-user"
assert settings.smtp_password == "smtp-password"
assert settings.smtp_from_name == "Home Automation"
assert settings.smtp_from_address == "sender@example.com"
assert settings.smtp_to_address == "recipient@example.com"
assert settings.smtp_use_starttls is False
+9 -113
View File
@@ -4,18 +4,12 @@ import sqlite3
import anyio
import pytest
import yaml
from alembic import command
from app.auth_db import reset_auth_db_caches
from app.db import reset_db_caches
from app.config import get_settings
from app.main import create_app
from scripts.app_db_adopt import APP_BASELINE_REVISION
from scripts.location_db_adopt import EXPECTED_USER_VERSION as LOCATION_USER_VERSION
from scripts.location_db_adopt import LOCATION_BASELINE_REVISION
from scripts.poo_db_adopt import EXPECTED_USER_VERSION as POO_USER_VERSION
from scripts.poo_db_adopt import POO_BASELINE_REVISION
from scripts.run_migrations import run_all_migrations
from tests.conftest import _make_alembic_config, _make_poo_alembic_config
PROJECT_ROOT = Path(__file__).resolve().parents[1]
@@ -31,73 +25,20 @@ async def _run_lifespan(app) -> None:
def _configure_database_env(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> dict[str, Path | str]:
app_path = tmp_path / "app.db"
location_path = tmp_path / "location.db"
poo_path = tmp_path / "poo.db"
monkeypatch.setenv("APP_DATABASE_URL", f"sqlite:///{app_path}")
monkeypatch.setenv("LOCATION_DATABASE_URL", f"sqlite:///{location_path}")
monkeypatch.setenv("POO_DATABASE_URL", f"sqlite:///{poo_path}")
monkeypatch.setenv("AUTH_BOOTSTRAP_USERNAME", "admin")
monkeypatch.setenv("AUTH_BOOTSTRAP_PASSWORD", "test-password")
monkeypatch.setenv("AUTH_COOKIE_SECURE_OVERRIDE", "false")
get_settings.cache_clear()
reset_auth_db_caches()
reset_db_caches()
return {
"app_path": app_path,
"app_url": f"sqlite:///{app_path}",
"location_path": location_path,
"location_url": f"sqlite:///{location_path}",
"poo_path": poo_path,
"poo_url": f"sqlite:///{poo_path}",
}
def _create_legacy_location_db(database_path: Path) -> None:
conn = sqlite3.connect(database_path)
conn.execute(
"""
CREATE TABLE location (
person TEXT NOT NULL,
datetime TEXT NOT NULL,
latitude REAL NOT NULL,
longitude REAL NOT NULL,
altitude REAL,
PRIMARY KEY (person, datetime)
)
"""
)
conn.execute(
"INSERT INTO location (person, datetime, latitude, longitude, altitude) VALUES (?, ?, ?, ?, ?)",
("alice", "2026-04-22T10:00:00Z", 1.23, 4.56, 7.89),
)
conn.execute(f"PRAGMA user_version = {LOCATION_USER_VERSION}")
conn.commit()
conn.close()
def _create_legacy_poo_db(database_path: Path) -> None:
conn = sqlite3.connect(database_path)
conn.execute(
"""
CREATE TABLE poo_records (
timestamp TEXT NOT NULL,
status TEXT NOT NULL,
latitude REAL NOT NULL,
longitude REAL NOT NULL,
PRIMARY KEY (timestamp)
)
"""
)
conn.execute(
"INSERT INTO poo_records (timestamp, status, latitude, longitude) VALUES (?, ?, ?, ?)",
("2026-04-22T11:00:00Z", "complete", 9.87, 6.54),
)
conn.execute(f"PRAGMA user_version = {POO_USER_VERSION}")
conn.commit()
conn.close()
def test_compose_uses_migration_job_before_app() -> None:
compose = _read_yaml("docker-compose.yml")
override = _read_yaml("docker-compose.override.yml")
@@ -131,12 +72,8 @@ def test_migration_runner_initializes_and_is_idempotent(
first_run = run_all_migrations()
second_run = run_all_migrations()
assert first_run == {"app": "initialized", "location": "initialized", "poo": "initialized"}
assert second_run == {
"app": "already_managed",
"location": "already_managed",
"poo": "already_managed",
}
assert first_run == {"app": "initialized"}
assert second_run == {"app": "already_managed"}
conn = sqlite3.connect(database_urls["app_path"])
try:
@@ -150,51 +87,12 @@ def test_migration_runner_initializes_and_is_idempotent(
finally:
conn.close()
assert {"auth_users", "auth_sessions", "app_config", "alembic_version"} <= tables
conn = sqlite3.connect(database_urls["location_path"])
try:
assert conn.execute("SELECT version_num FROM alembic_version").fetchone()[0] == LOCATION_BASELINE_REVISION
finally:
conn.close()
conn = sqlite3.connect(database_urls["poo_path"])
try:
assert conn.execute("SELECT version_num FROM alembic_version").fetchone()[0] == POO_BASELINE_REVISION
finally:
conn.close()
assert {
"auth_users", "auth_sessions", "app_config", "alembic_version", "location", "poo_records"
} <= tables
get_settings.cache_clear()
reset_auth_db_caches()
def test_migration_runner_adopts_legacy_sqlite_without_data_loss(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
database_urls = _configure_database_env(tmp_path, monkeypatch)
_create_legacy_location_db(database_urls["location_path"])
_create_legacy_poo_db(database_urls["poo_path"])
results = run_all_migrations()
assert results == {"app": "initialized", "location": "adopted", "poo": "adopted"}
conn = sqlite3.connect(database_urls["location_path"])
try:
assert conn.execute("SELECT version_num FROM alembic_version").fetchone()[0] == LOCATION_BASELINE_REVISION
assert conn.execute("SELECT COUNT(*) FROM location").fetchone()[0] == 1
finally:
conn.close()
conn = sqlite3.connect(database_urls["poo_path"])
try:
assert conn.execute("SELECT version_num FROM alembic_version").fetchone()[0] == POO_BASELINE_REVISION
assert conn.execute("SELECT COUNT(*) FROM poo_records").fetchone()[0] == 1
finally:
conn.close()
get_settings.cache_clear()
reset_auth_db_caches()
reset_db_caches()
def test_app_startup_still_fails_closed_without_running_adoption(
@@ -202,8 +100,6 @@ def test_app_startup_still_fails_closed_without_running_adoption(
) -> None:
database_urls = _configure_database_env(tmp_path, monkeypatch)
missing_app_path = database_urls["app_path"]
command.upgrade(_make_alembic_config(database_urls["location_url"]), "head")
command.upgrade(_make_poo_alembic_config(database_urls["poo_url"]), "head")
app = create_app()
with pytest.raises(RuntimeError, match="Run 'python scripts/app_db_adopt.py' first"):
@@ -212,4 +108,4 @@ def test_app_startup_still_fails_closed_without_running_adoption(
assert not Path(missing_app_path).exists()
get_settings.cache_clear()
reset_auth_db_caches()
reset_db_caches()
+10 -63
View File
@@ -1,7 +1,5 @@
from sqlalchemy import text
import app.db as app_db
import app.poo_db as poo_db
from app.config import Settings, get_settings
from app.dependencies import get_app_settings, get_homeassistant_client
from app.main import create_app
@@ -158,45 +156,25 @@ def test_homeassistant_publish_rejects_invalid_ticktick_content(location_client)
def test_homeassistant_publish_poo_get_latest_publishes_latest_status(
ready_location_database,
ready_poo_database,
auth_database,
monkeypatch,
) -> None:
location_engine = app_db.create_engine(
ready_location_database["location_url"],
connect_args={"check_same_thread": False},
)
location_session_local = app_db.sessionmaker(
bind=location_engine,
autoflush=False,
autocommit=False,
)
poo_engine = poo_db.create_engine(
ready_poo_database["poo_url"],
connect_args={"check_same_thread": False},
)
poo_session_local = poo_db.sessionmaker(
bind=poo_engine,
autoflush=False,
autocommit=False,
)
from fastapi.testclient import TestClient
from sqlalchemy import create_engine
app_url = auth_database["app_url"]
engine = create_engine(app_url, connect_args={"check_same_thread": False})
fake_ha = _FakeHomeAssistantClient()
settings = Settings(
poo_sensor_entity_name="sensor.test_poo_status",
poo_sensor_friendly_name="Poo Status",
)
monkeypatch.setattr(app_db, "engine", location_engine)
monkeypatch.setattr(app_db, "SessionLocal", location_session_local)
monkeypatch.setattr(poo_db, "poo_engine", poo_engine)
monkeypatch.setattr(poo_db, "PooSessionLocal", poo_session_local)
test_app = create_app()
test_app.dependency_overrides[get_homeassistant_client] = lambda: fake_ha
test_app.dependency_overrides[get_app_settings] = lambda: settings
with poo_engine.begin() as conn:
with engine.begin() as conn:
conn.execute(
text(
"INSERT INTO poo_records (timestamp, status, latitude, longitude) "
@@ -211,8 +189,6 @@ def test_homeassistant_publish_poo_get_latest_publishes_latest_status(
)
try:
from fastapi.testclient import TestClient
with TestClient(test_app) as client:
response = client.post(
"/homeassistant/publish",
@@ -233,52 +209,25 @@ def test_homeassistant_publish_poo_get_latest_publishes_latest_status(
finally:
test_app.dependency_overrides.clear()
get_settings.cache_clear()
location_engine.dispose()
poo_engine.dispose()
engine.dispose()
def test_homeassistant_publish_returns_internal_error_for_unknown_poo_action(
ready_location_database,
ready_poo_database,
auth_database,
monkeypatch,
) -> None:
location_engine = app_db.create_engine(
ready_location_database["location_url"],
connect_args={"check_same_thread": False},
)
location_session_local = app_db.sessionmaker(
bind=location_engine,
autoflush=False,
autocommit=False,
)
poo_engine = poo_db.create_engine(
ready_poo_database["poo_url"],
connect_args={"check_same_thread": False},
)
poo_session_local = poo_db.sessionmaker(
bind=poo_engine,
autoflush=False,
autocommit=False,
)
from fastapi.testclient import TestClient
fake_ha = _FakeHomeAssistantClient()
settings = Settings(
poo_sensor_entity_name="sensor.test_poo_status",
poo_sensor_friendly_name="Poo Status",
)
monkeypatch.setattr(app_db, "engine", location_engine)
monkeypatch.setattr(app_db, "SessionLocal", location_session_local)
monkeypatch.setattr(poo_db, "poo_engine", poo_engine)
monkeypatch.setattr(poo_db, "PooSessionLocal", poo_session_local)
test_app = create_app()
test_app.dependency_overrides[get_homeassistant_client] = lambda: fake_ha
test_app.dependency_overrides[get_app_settings] = lambda: settings
try:
from fastapi.testclient import TestClient
with TestClient(test_app) as client:
response = client.post(
"/homeassistant/publish",
@@ -295,8 +244,6 @@ def test_homeassistant_publish_returns_internal_error_for_unknown_poo_action(
finally:
test_app.dependency_overrides.clear()
get_settings.cache_clear()
location_engine.dispose()
poo_engine.dispose()
def test_homeassistant_publish_returns_not_implemented_for_unknown_location_action(
+1 -214
View File
@@ -1,28 +1,8 @@
from datetime import datetime
from pathlib import Path
import sqlite3
import pytest
from alembic import command
from alembic.config import Config
from sqlalchemy import create_engine, text
from sqlalchemy.orm import sessionmaker
from sqlalchemy import text
import app.db as app_db
from app.main import create_app
from scripts.location_db_adopt import (
EXPECTED_USER_VERSION,
LOCATION_BASELINE_REVISION,
LocationDatabaseAdoptionError,
adopt_or_initialize_location_db,
)
from tests.conftest import _make_app_alembic_config, _make_poo_alembic_config
def _make_alembic_config(database_url: str) -> Config:
config = Config("alembic_location.ini")
config.set_main_option("sqlalchemy.url", database_url)
return config
def test_location_record_endpoint_writes_row(location_client) -> None:
client, engine = location_client
@@ -197,196 +177,3 @@ def test_location_record_endpoint_defaults_invalid_altitude_to_zero(location_cli
assert row.altitude == pytest.approx(0.0)
def test_legacy_style_location_db_can_be_stamped_and_adopted(
test_database_urls, monkeypatch: pytest.MonkeyPatch
) -> None:
app_database_url = test_database_urls["app_url"]
database_path = test_database_urls["location_path"]
database_url = test_database_urls["location_url"]
poo_database_url = test_database_urls["poo_url"]
conn = sqlite3.connect(database_path)
conn.execute(
"""
CREATE TABLE location (
person TEXT NOT NULL,
datetime TEXT NOT NULL,
latitude REAL NOT NULL,
longitude REAL NOT NULL,
altitude REAL,
PRIMARY KEY (person, datetime)
)
"""
)
conn.execute("PRAGMA user_version = 2")
conn.commit()
conn.close()
command.upgrade(_make_app_alembic_config(app_database_url), "head")
command.stamp(_make_alembic_config(database_url), LOCATION_BASELINE_REVISION)
command.upgrade(_make_poo_alembic_config(poo_database_url), "head")
engine = create_engine(database_url, connect_args={"check_same_thread": False})
session_local = sessionmaker(bind=engine, autoflush=False, autocommit=False)
monkeypatch.setattr(app_db, "engine", engine)
monkeypatch.setattr(app_db, "SessionLocal", session_local)
from fastapi.testclient import TestClient
fastapi_app = create_app()
with TestClient(fastapi_app) as client:
response = client.post(
"/location/record",
json={
"person": "legacy-user",
"latitude": "12.3",
"longitude": "45.6",
"altitude": "7.8",
},
)
assert response.status_code == 200
with engine.connect() as db_conn:
revision = db_conn.execute(text("SELECT version_num FROM alembic_version")).scalar_one()
row_count = db_conn.execute(text("SELECT COUNT(*) FROM location")).scalar_one()
assert revision == LOCATION_BASELINE_REVISION
assert row_count == 1
engine.dispose()
def test_location_db_adoption_initializes_new_db(tmp_path: Path) -> None:
database_path = tmp_path / "new_location.db"
result = adopt_or_initialize_location_db(f"sqlite:///{database_path}")
assert result == "initialized"
assert database_path.exists()
conn = sqlite3.connect(database_path)
try:
revision = conn.execute("SELECT version_num FROM alembic_version").fetchone()[0]
location_table = conn.execute(
"SELECT name FROM sqlite_master WHERE type = 'table' AND name = 'location'"
).fetchone()
finally:
conn.close()
assert revision == LOCATION_BASELINE_REVISION
assert location_table is not None
def test_location_db_adoption_validates_and_stamps_legacy_db(tmp_path: Path) -> None:
database_path = tmp_path / "legacy_location.db"
conn = sqlite3.connect(database_path)
conn.execute(
"""
CREATE TABLE location (
person TEXT NOT NULL,
datetime TEXT NOT NULL,
latitude REAL NOT NULL,
longitude REAL NOT NULL,
altitude REAL,
PRIMARY KEY (person, datetime)
)
"""
)
conn.execute(f"PRAGMA user_version = {EXPECTED_USER_VERSION}")
conn.commit()
conn.close()
result = adopt_or_initialize_location_db(f"sqlite:///{database_path}")
assert result == "adopted"
conn = sqlite3.connect(database_path)
try:
revision = conn.execute("SELECT version_num FROM alembic_version").fetchone()[0]
finally:
conn.close()
assert revision == LOCATION_BASELINE_REVISION
def test_location_db_adoption_accepts_already_managed_matching_revision(
tmp_path: Path,
) -> None:
database_path = tmp_path / "managed_location.db"
command.upgrade(_make_alembic_config(f"sqlite:///{database_path}"), "head")
result = adopt_or_initialize_location_db(f"sqlite:///{database_path}")
assert result == "already_managed"
def test_location_db_adoption_fails_closed_on_alembic_revision_mismatch(
tmp_path: Path,
) -> None:
database_path = tmp_path / "wrong_revision.db"
conn = sqlite3.connect(database_path)
conn.execute(
"""
CREATE TABLE location (
person TEXT NOT NULL,
datetime TEXT NOT NULL,
latitude REAL NOT NULL,
longitude REAL NOT NULL,
altitude REAL,
PRIMARY KEY (person, datetime)
)
"""
)
conn.execute("CREATE TABLE alembic_version (version_num VARCHAR(32) NOT NULL)")
conn.execute("INSERT INTO alembic_version (version_num) VALUES ('wrong_revision')")
conn.execute(f"PRAGMA user_version = {EXPECTED_USER_VERSION}")
conn.commit()
conn.close()
with pytest.raises(LocationDatabaseAdoptionError, match="known migration revision"):
adopt_or_initialize_location_db(f"sqlite:///{database_path}")
def test_location_db_adoption_fails_closed_on_schema_mismatch(tmp_path: Path) -> None:
database_path = tmp_path / "bad_schema.db"
conn = sqlite3.connect(database_path)
conn.execute(
"""
CREATE TABLE location (
person TEXT NOT NULL,
datetime TEXT NOT NULL,
latitude REAL NOT NULL,
longitude REAL NOT NULL,
PRIMARY KEY (person, datetime)
)
"""
)
conn.execute(f"PRAGMA user_version = {EXPECTED_USER_VERSION}")
conn.commit()
conn.close()
with pytest.raises(LocationDatabaseAdoptionError, match="schema does not match"):
adopt_or_initialize_location_db(f"sqlite:///{database_path}")
def test_location_db_adoption_fails_closed_on_user_version_mismatch(tmp_path: Path) -> None:
database_path = tmp_path / "bad_user_version.db"
conn = sqlite3.connect(database_path)
conn.execute(
"""
CREATE TABLE location (
person TEXT NOT NULL,
datetime TEXT NOT NULL,
latitude REAL NOT NULL,
longitude REAL NOT NULL,
altitude REAL,
PRIMARY KEY (person, datetime)
)
"""
)
conn.execute("PRAGMA user_version = 999")
conn.commit()
conn.close()
with pytest.raises(LocationDatabaseAdoptionError, match="Expected PRAGMA user_version"):
adopt_or_initialize_location_db(f"sqlite:///{database_path}")
+460
View File
@@ -0,0 +1,460 @@
"""Tests for scripts/migrate_legacy_data.py (M1-T02).
Uses pytest tmp_path for all temp files. The app DB is brought to head via
alembic_app.ini (the same approach used by conftest._make_app_alembic_config),
so it has the location and poo_records tables created in T01.
Legacy DBs are built by hand with real columns matching the legacy baseline schema.
"""
from __future__ import annotations
import sqlite3
import sys
from pathlib import Path
import pytest
from alembic import command
from alembic.config import Config
from scripts.migrate_legacy_data import (
_reconcile,
_sqlite_path_from_url,
migrate_legacy_data,
)
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _make_app_alembic_config(database_url: str) -> Config:
cfg = Config("alembic_app.ini")
cfg.set_main_option("sqlalchemy.url", database_url)
return cfg
def _upgraded_app_db(tmp_path: Path, name: str = "app_test.db") -> tuple[Path, str]:
"""Create and upgrade an app DB to head; return (path, url)."""
db_path = tmp_path / name
db_url = f"sqlite:///{db_path}"
command.upgrade(_make_app_alembic_config(db_url), "head")
return db_path, db_url
def _make_legacy_location_db(db_path: Path, rows: list[tuple]) -> None:
"""Create a legacy location DB and insert given rows.
Each row is a tuple: (person, datetime, latitude, longitude, altitude).
altitude may be None.
"""
conn = sqlite3.connect(db_path)
conn.execute(
"""
CREATE TABLE location (
person TEXT NOT NULL,
datetime TEXT NOT NULL,
latitude REAL NOT NULL,
longitude REAL NOT NULL,
altitude REAL,
PRIMARY KEY (person, datetime)
)
"""
)
conn.executemany(
"INSERT INTO location (person, datetime, latitude, longitude, altitude) "
"VALUES (?, ?, ?, ?, ?)",
rows,
)
conn.commit()
conn.close()
def _make_legacy_poo_db(db_path: Path, rows: list[tuple]) -> None:
"""Create a legacy poo DB and insert given rows.
Each row is a tuple: (timestamp, status, latitude, longitude).
"""
conn = sqlite3.connect(db_path)
conn.execute(
"""
CREATE TABLE poo_records (
timestamp TEXT NOT NULL,
status TEXT NOT NULL,
latitude REAL NOT NULL,
longitude REAL NOT NULL,
PRIMARY KEY (timestamp)
)
"""
)
conn.executemany(
"INSERT INTO poo_records (timestamp, status, latitude, longitude) "
"VALUES (?, ?, ?, ?)",
rows,
)
conn.commit()
conn.close()
def _count_rows(db_path: Path, table: str) -> int:
conn = sqlite3.connect(db_path)
try:
(count,) = conn.execute(f"SELECT COUNT(*) FROM {table}").fetchone()
return count
finally:
conn.close()
# ---------------------------------------------------------------------------
# Sample data
# ---------------------------------------------------------------------------
LOCATION_ROWS = [
("alice", "2026-01-01T10:00:00Z", 1.23, 4.56, 7.89),
("bob", "2026-01-02T10:00:00Z", 2.34, 5.67, None),
("alice", "2026-01-03T10:00:00Z", 3.45, 6.78, 9.01),
]
POO_ROWS = [
("2026-01-01T08:00:00Z", "complete", 10.0, 20.0),
("2026-01-02T08:00:00Z", "urgent", 11.0, 21.0),
]
# ---------------------------------------------------------------------------
# Test 1: Idempotency
# ---------------------------------------------------------------------------
def test_location_migration_is_idempotent(tmp_path: Path) -> None:
"""N source rows → app table has N rows; run again → still N rows."""
app_path, app_url = _upgraded_app_db(tmp_path)
legacy_path = tmp_path / "locationRecorder.db"
_make_legacy_location_db(legacy_path, LOCATION_ROWS)
legacy_url = f"sqlite:///{legacy_path}"
# First run
result1 = migrate_legacy_data(app_url, legacy_url, None)
assert result1["location"]["source"] == len(LOCATION_ROWS)
assert result1["location"]["copied"] == len(LOCATION_ROWS)
assert result1["location"]["skipped"] is False
assert result1["location"]["final"] == len(LOCATION_ROWS)
assert _count_rows(app_path, "location") == len(LOCATION_ROWS)
# Second run — idempotent, no dupes, no error
result2 = migrate_legacy_data(app_url, legacy_url, None)
assert result2["location"]["source"] == len(LOCATION_ROWS)
assert result2["location"]["copied"] == 0 # nothing new
assert result2["location"]["skipped"] is False
assert result2["location"]["final"] == len(LOCATION_ROWS)
assert _count_rows(app_path, "location") == len(LOCATION_ROWS)
def test_poo_migration_is_idempotent(tmp_path: Path) -> None:
"""N poo source rows → app table has N rows; run again → still N rows."""
app_path, app_url = _upgraded_app_db(tmp_path)
legacy_path = tmp_path / "pooRecorder.db"
_make_legacy_poo_db(legacy_path, POO_ROWS)
legacy_url = f"sqlite:///{legacy_path}"
result1 = migrate_legacy_data(app_url, None, legacy_url)
assert result1["poo_records"]["source"] == len(POO_ROWS)
assert result1["poo_records"]["copied"] == len(POO_ROWS)
assert result1["poo_records"]["skipped"] is False
assert result1["poo_records"]["final"] == len(POO_ROWS)
assert _count_rows(app_path, "poo_records") == len(POO_ROWS)
result2 = migrate_legacy_data(app_url, None, legacy_url)
assert result2["poo_records"]["copied"] == 0
assert result2["poo_records"]["final"] == len(POO_ROWS)
assert _count_rows(app_path, "poo_records") == len(POO_ROWS)
def test_both_tables_migration_is_idempotent(tmp_path: Path) -> None:
"""Migrating both tables at once is idempotent."""
app_path, app_url = _upgraded_app_db(tmp_path)
loc_path = tmp_path / "locationRecorder.db"
_make_legacy_location_db(loc_path, LOCATION_ROWS)
loc_url = f"sqlite:///{loc_path}"
poo_path = tmp_path / "pooRecorder.db"
_make_legacy_poo_db(poo_path, POO_ROWS)
poo_url = f"sqlite:///{poo_path}"
result1 = migrate_legacy_data(app_url, loc_url, poo_url)
assert result1["location"]["final"] == len(LOCATION_ROWS)
assert result1["poo_records"]["final"] == len(POO_ROWS)
result2 = migrate_legacy_data(app_url, loc_url, poo_url)
assert result2["location"]["copied"] == 0
assert result2["poo_records"]["copied"] == 0
assert _count_rows(app_path, "location") == len(LOCATION_ROWS)
assert _count_rows(app_path, "poo_records") == len(POO_ROWS)
# ---------------------------------------------------------------------------
# Test 2: Missing legacy file
# ---------------------------------------------------------------------------
def test_missing_location_file_is_skipped(tmp_path: Path) -> None:
"""Absent location DB → table result is skipped, no exception, app table empty."""
app_path, app_url = _upgraded_app_db(tmp_path)
nonexistent = f"sqlite:///{tmp_path / 'does_not_exist_location.db'}"
result = migrate_legacy_data(app_url, nonexistent, None)
assert result["location"]["skipped"] is True
assert result["location"]["source"] == 0
assert result["location"]["copied"] == 0
assert _count_rows(app_path, "location") == 0
def test_missing_poo_file_is_skipped(tmp_path: Path) -> None:
"""Absent poo DB → table result is skipped, no exception, app table empty."""
app_path, app_url = _upgraded_app_db(tmp_path)
nonexistent = f"sqlite:///{tmp_path / 'does_not_exist_poo.db'}"
result = migrate_legacy_data(app_url, None, nonexistent)
assert result["poo_records"]["skipped"] is True
assert result["poo_records"]["source"] == 0
assert result["poo_records"]["copied"] == 0
assert _count_rows(app_path, "poo_records") == 0
def test_none_location_url_is_skipped(tmp_path: Path) -> None:
"""Passing None for location_url → skipped, no exception."""
_, app_url = _upgraded_app_db(tmp_path)
result = migrate_legacy_data(app_url, None, None)
assert result["location"]["skipped"] is True
assert result["poo_records"]["skipped"] is True
# ---------------------------------------------------------------------------
# Test 3: Reconciliation failure
# ---------------------------------------------------------------------------
def test_reconcile_raises_on_missing_rows(tmp_path: Path) -> None:
"""_reconcile() raises RuntimeError when source rows are missing from target."""
# Build an app DB and a legacy DB with 3 rows
app_path, app_url = _upgraded_app_db(tmp_path)
legacy_path = tmp_path / "locationRecorder.db"
_make_legacy_location_db(legacy_path, LOCATION_ROWS)
# Only insert 1 row into the app DB manually (simulate partial migration)
conn = sqlite3.connect(app_path)
conn.execute(
"INSERT INTO location (person, datetime, latitude, longitude, altitude) "
"VALUES (?, ?, ?, ?, ?)",
LOCATION_ROWS[0],
)
conn.commit()
# ATTACH legacy to run _reconcile
conn.execute(f"ATTACH DATABASE '{legacy_path}' AS legacy")
with pytest.raises(RuntimeError, match="Reconciliation failed"):
_reconcile(
conn,
table="location",
columns=["person", "datetime", "latitude", "longitude", "altitude"],
source_count=len(LOCATION_ROWS),
)
conn.execute("DETACH DATABASE legacy")
conn.close()
def test_migrate_reconciliation_failure_raises(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""If a row goes missing after INSERT, migrate_legacy_data raises RuntimeError."""
import scripts.migrate_legacy_data as mod
app_path, app_url = _upgraded_app_db(tmp_path)
legacy_path = tmp_path / "locationRecorder.db"
_make_legacy_location_db(legacy_path, LOCATION_ROWS)
legacy_url = f"sqlite:///{legacy_path}"
def _always_fail(conn, table, columns, source_count):
# Simulate a scenario where reconciliation finds rows missing
raise RuntimeError(
f"Reconciliation failed for table '{table}': "
f"1 of {source_count} source rows are missing from the app DB."
)
monkeypatch.setattr(mod, "_reconcile", _always_fail)
with pytest.raises(RuntimeError, match="Reconciliation failed"):
migrate_legacy_data(app_url, legacy_url, None)
def test_cli_exits_nonzero_on_reconciliation_failure(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
"""CLI main() exits non-zero when reconciliation raises."""
import scripts.migrate_legacy_data as mod
app_path, app_url = _upgraded_app_db(tmp_path)
legacy_path = tmp_path / "locationRecorder.db"
_make_legacy_location_db(legacy_path, LOCATION_ROWS)
legacy_url = f"sqlite:///{legacy_path}"
# Patch _reconcile to always raise
def _always_fail(conn, table, columns, source_count):
raise RuntimeError(
f"Reconciliation failed for table '{table}': 1 row missing."
)
monkeypatch.setattr(mod, "_reconcile", _always_fail)
# Patch sys.argv so main() picks up the right args
monkeypatch.setattr(
sys,
"argv",
[
"migrate_legacy_data",
"--app-db", app_url,
"--location-db", legacy_url,
],
)
with pytest.raises(SystemExit) as exc_info:
mod.main()
assert exc_info.value.code != 0
def test_reconcile_catches_value_mismatch_not_just_pk(tmp_path: Path) -> None:
"""Full-row reconciliation catches value mismatch that PK-only check would miss.
Scenario: the app DB is PRE-POPULATED with a row that shares the same PK as
a legacy source row but has DIFFERENT non-PK column values. INSERT OR IGNORE
skips the source row (PK conflict), so the target retains the stale data.
The old PK-only reconciliation would have incorrectly reported success.
The new full-row reconciliation must detect the mismatch and raise.
"""
app_path, app_url = _upgraded_app_db(tmp_path)
# Legacy source has a row: person="alice", datetime="2026-01-01T10:00:00Z",
# latitude=1.23, longitude=4.56, altitude=7.89
legacy_path = tmp_path / "locationRecorder.db"
_make_legacy_location_db(legacy_path, [("alice", "2026-01-01T10:00:00Z", 1.23, 4.56, 7.89)])
legacy_url = f"sqlite:///{legacy_path}"
# App DB is pre-populated with the SAME PK but DIFFERENT non-PK values
# (latitude/longitude/altitude all differ from the source row)
conn = sqlite3.connect(app_path)
conn.execute(
"INSERT INTO location (person, datetime, latitude, longitude, altitude) "
"VALUES (?, ?, ?, ?, ?)",
("alice", "2026-01-01T10:00:00Z", 99.0, 99.0, 99.0),
)
conn.commit()
conn.close()
# migrate_legacy_data must raise: the source row's data is NOT in the target
# (INSERT OR IGNORE skipped it because of PK conflict, retaining the 99.0 values)
with pytest.raises(RuntimeError, match="Reconciliation failed"):
migrate_legacy_data(app_url, legacy_url, None)
def test_full_row_reconciliation_idempotent_on_identical_data(tmp_path: Path) -> None:
"""Second run on already-migrated data still reconciles cleanly.
When the target already holds identical rows (from the first run), the full-row
IS predicate matches every column and reconciliation passes (no raise).
"""
app_path, app_url = _upgraded_app_db(tmp_path)
legacy_path = tmp_path / "locationRecorder.db"
_make_legacy_location_db(legacy_path, LOCATION_ROWS)
legacy_url = f"sqlite:///{legacy_path}"
# First run: migrate all rows
result1 = migrate_legacy_data(app_url, legacy_url, None)
assert result1["location"]["copied"] == len(LOCATION_ROWS)
# Second run: rows already present, INSERT OR IGNORE skips all, full-row
# reconciliation must still pass because values are identical
result2 = migrate_legacy_data(app_url, legacy_url, None)
assert result2["location"]["copied"] == 0
assert result2["location"]["final"] == len(LOCATION_ROWS)
# No exception raised — idempotency holds under full-row reconciliation
# ---------------------------------------------------------------------------
# Test 4: dry_run
# ---------------------------------------------------------------------------
def test_dry_run_does_not_write_location_rows(tmp_path: Path) -> None:
"""dry_run=True reports source counts but writes nothing."""
app_path, app_url = _upgraded_app_db(tmp_path)
legacy_path = tmp_path / "locationRecorder.db"
_make_legacy_location_db(legacy_path, LOCATION_ROWS)
legacy_url = f"sqlite:///{legacy_path}"
result = migrate_legacy_data(app_url, legacy_url, None, dry_run=True)
assert result["location"]["source"] == len(LOCATION_ROWS)
assert result["location"]["copied"] == 0
assert result["location"]["skipped"] is False
# dry_run returns final=0 (no actual query on app side)
assert result["location"]["final"] == 0
# App table must still be empty
assert _count_rows(app_path, "location") == 0
def test_dry_run_does_not_write_poo_rows(tmp_path: Path) -> None:
"""dry_run=True for poo_records: source reported, nothing written."""
app_path, app_url = _upgraded_app_db(tmp_path)
legacy_path = tmp_path / "pooRecorder.db"
_make_legacy_poo_db(legacy_path, POO_ROWS)
legacy_url = f"sqlite:///{legacy_path}"
result = migrate_legacy_data(app_url, None, legacy_url, dry_run=True)
assert result["poo_records"]["source"] == len(POO_ROWS)
assert result["poo_records"]["copied"] == 0
assert result["poo_records"]["skipped"] is False
assert result["poo_records"]["final"] == 0
assert _count_rows(app_path, "poo_records") == 0
def test_dry_run_both_tables(tmp_path: Path) -> None:
"""dry_run=True for both tables: both reported, nothing written."""
app_path, app_url = _upgraded_app_db(tmp_path)
loc_path = tmp_path / "locationRecorder.db"
_make_legacy_location_db(loc_path, LOCATION_ROWS)
loc_url = f"sqlite:///{loc_path}"
poo_path = tmp_path / "pooRecorder.db"
_make_legacy_poo_db(poo_path, POO_ROWS)
poo_url = f"sqlite:///{poo_path}"
result = migrate_legacy_data(app_url, loc_url, poo_url, dry_run=True)
assert result["location"]["source"] == len(LOCATION_ROWS)
assert result["location"]["copied"] == 0
assert result["poo_records"]["source"] == len(POO_ROWS)
assert result["poo_records"]["copied"] == 0
assert _count_rows(app_path, "location") == 0
assert _count_rows(app_path, "poo_records") == 0
# ---------------------------------------------------------------------------
# Test: _sqlite_path_from_url helper
# ---------------------------------------------------------------------------
def test_sqlite_path_from_url_parses_url() -> None:
path = _sqlite_path_from_url("sqlite:///./data/app.db")
# Path normalises './' away, but the tail should remain
assert path == Path("data/app.db")
def test_sqlite_path_from_url_treats_plain_path_as_path() -> None:
path = _sqlite_path_from_url("/tmp/some.db")
assert str(path) == "/tmp/some.db"
-102
View File
@@ -1,17 +1,8 @@
from pathlib import Path
import sqlite3
import pytest
from sqlalchemy import text
from app.config import Settings, get_settings
from app.dependencies import get_app_settings, get_homeassistant_client
from scripts.poo_db_adopt import (
EXPECTED_USER_VERSION,
POO_BASELINE_REVISION,
PooDatabaseAdoptionError,
adopt_or_initialize_poo_db,
)
class _FakeHomeAssistantClient:
@@ -153,96 +144,3 @@ def test_poo_latest_endpoint_returns_ok_when_no_record_exists(poo_client_with_ov
assert response.text == ""
def test_poo_db_adoption_initializes_new_db(tmp_path: Path) -> None:
database_path = tmp_path / "new_poo.db"
result = adopt_or_initialize_poo_db(f"sqlite:///{database_path}")
assert result == "initialized"
assert database_path.exists()
conn = sqlite3.connect(database_path)
try:
revision = conn.execute("SELECT version_num FROM alembic_version").fetchone()[0]
poo_table = conn.execute(
"SELECT name FROM sqlite_master WHERE type = 'table' AND name = 'poo_records'"
).fetchone()
finally:
conn.close()
assert revision == POO_BASELINE_REVISION
assert poo_table is not None
def test_poo_db_adoption_validates_and_stamps_legacy_db(tmp_path: Path) -> None:
database_path = tmp_path / "legacy_poo.db"
conn = sqlite3.connect(database_path)
conn.execute(
"""
CREATE TABLE poo_records (
timestamp TEXT NOT NULL,
status TEXT NOT NULL,
latitude REAL NOT NULL,
longitude REAL NOT NULL,
PRIMARY KEY (timestamp)
)
"""
)
conn.execute(f"PRAGMA user_version = {EXPECTED_USER_VERSION}")
conn.commit()
conn.close()
result = adopt_or_initialize_poo_db(f"sqlite:///{database_path}")
assert result == "adopted"
conn = sqlite3.connect(database_path)
try:
revision = conn.execute("SELECT version_num FROM alembic_version").fetchone()[0]
finally:
conn.close()
assert revision == POO_BASELINE_REVISION
def test_poo_db_adoption_fails_closed_on_schema_mismatch(tmp_path: Path) -> None:
database_path = tmp_path / "bad_poo_schema.db"
conn = sqlite3.connect(database_path)
conn.execute(
"""
CREATE TABLE poo_records (
timestamp TEXT NOT NULL,
status TEXT NOT NULL,
latitude REAL NOT NULL,
PRIMARY KEY (timestamp)
)
"""
)
conn.execute(f"PRAGMA user_version = {EXPECTED_USER_VERSION}")
conn.commit()
conn.close()
with pytest.raises(PooDatabaseAdoptionError, match="schema does not match"):
adopt_or_initialize_poo_db(f"sqlite:///{database_path}")
def test_poo_db_adoption_fails_closed_on_user_version_mismatch(tmp_path: Path) -> None:
database_path = tmp_path / "bad_poo_user_version.db"
conn = sqlite3.connect(database_path)
conn.execute(
"""
CREATE TABLE poo_records (
timestamp TEXT NOT NULL,
status TEXT NOT NULL,
latitude REAL NOT NULL,
longitude REAL NOT NULL,
PRIMARY KEY (timestamp)
)
"""
)
conn.execute("PRAGMA user_version = 999")
conn.commit()
conn.close()
with pytest.raises(PooDatabaseAdoptionError, match="Expected PRAGMA user_version"):
adopt_or_initialize_poo_db(f"sqlite:///{database_path}")
+256
View File
@@ -0,0 +1,256 @@
from datetime import UTC, datetime
import re
import sqlite3
from fastapi.testclient import TestClient
from sqlalchemy import create_engine
from sqlalchemy.orm import Session, sessionmaker
from app.config import Settings
from app.services.email import EmailDeliveryError
from app.services.public_ip import PublicIPCheckResult, check_public_ipv4, check_public_ipv4_and_notify
def _make_session(database_url: str) -> Session:
engine = create_engine(database_url, connect_args={"check_same_thread": False})
session_local = sessionmaker(bind=engine, autoflush=False, autocommit=False, class_=Session)
return session_local()
def _extract_csrf_token(html: str) -> str:
match = re.search(r'name="csrf_token" value="([^"]+)"', html)
assert match is not None
return match.group(1)
def _login(client: TestClient) -> None:
login_page = client.get("/login")
csrf_token = _extract_csrf_token(login_page.text)
response = client.post(
"/login",
data={
"username": "admin",
"password": "test-password",
"csrf_token": csrf_token,
},
follow_redirects=False,
)
assert response.status_code == 303
def test_public_ip_first_seen_persists_state_and_history(auth_database) -> None:
session = _make_session(auth_database["app_url"])
try:
result = check_public_ipv4(session, fetch_public_ipv4=lambda: "203.0.113.10")
finally:
session.close()
assert result.status == "first_seen"
assert result.changed is False
conn = sqlite3.connect(auth_database["app_path"])
try:
state = conn.execute(
"SELECT current_ipv4, previous_ipv4, last_check_status, last_check_error, last_provider FROM public_ip_state"
).fetchone()
history = conn.execute(
"SELECT ipv4, change_type, provider FROM public_ip_history ORDER BY id"
).fetchall()
finally:
conn.close()
assert state == ("203.0.113.10", None, "first_seen", None, "ipify")
assert history == [("203.0.113.10", "first_seen", "ipify")]
def test_public_ip_unchanged_updates_state_without_adding_history(auth_database) -> None:
session = _make_session(auth_database["app_url"])
try:
first_result = check_public_ipv4(session, fetch_public_ipv4=lambda: "203.0.113.10")
unchanged_result = check_public_ipv4(session, fetch_public_ipv4=lambda: "203.0.113.10")
finally:
session.close()
assert first_result.status == "first_seen"
assert unchanged_result.status == "unchanged"
assert unchanged_result.changed is False
conn = sqlite3.connect(auth_database["app_path"])
try:
state = conn.execute(
"SELECT current_ipv4, previous_ipv4, last_check_status FROM public_ip_state"
).fetchone()
history_count = conn.execute("SELECT COUNT(*) FROM public_ip_history").fetchone()[0]
finally:
conn.close()
assert state == ("203.0.113.10", None, "unchanged")
assert history_count == 1
def test_public_ip_changed_updates_state_and_adds_history(auth_database) -> None:
session = _make_session(auth_database["app_url"])
try:
check_public_ipv4(session, fetch_public_ipv4=lambda: "203.0.113.10")
result = check_public_ipv4(session, fetch_public_ipv4=lambda: "198.51.100.25")
finally:
session.close()
assert result.status == "changed"
assert result.changed is True
conn = sqlite3.connect(auth_database["app_path"])
try:
state = conn.execute(
"SELECT current_ipv4, previous_ipv4, last_check_status, last_changed_at FROM public_ip_state"
).fetchone()
history = conn.execute(
"SELECT ipv4, change_type FROM public_ip_history ORDER BY id"
).fetchall()
finally:
conn.close()
assert state[0:3] == ("198.51.100.25", "203.0.113.10", "changed")
assert state[3] is not None
assert history == [("203.0.113.10", "first_seen"), ("198.51.100.25", "changed")]
def test_public_ip_error_keeps_existing_ip_and_does_not_add_history(auth_database) -> None:
session = _make_session(auth_database["app_url"])
try:
check_public_ipv4(session, fetch_public_ipv4=lambda: "203.0.113.10")
result = check_public_ipv4(session, fetch_public_ipv4=lambda: "not-an-ip")
finally:
session.close()
assert result.status == "error"
assert result.changed is False
conn = sqlite3.connect(auth_database["app_path"])
try:
state = conn.execute(
"SELECT current_ipv4, previous_ipv4, last_check_status, last_check_error FROM public_ip_state"
).fetchone()
history_count = conn.execute("SELECT COUNT(*) FROM public_ip_history").fetchone()[0]
finally:
conn.close()
assert state[0:3] == ("203.0.113.10", None, "error")
assert state[3] is not None
assert history_count == 1
def test_public_ip_check_endpoint_requires_authentication(client: TestClient) -> None:
response = client.get("/public-ip/check")
assert response.status_code == 401
assert response.json() == {"detail": "authentication required"}
def test_public_ip_check_endpoint_hides_ip_values(client: TestClient, monkeypatch) -> None:
from app.api.routes import public_ip as public_ip_route
fixed_checked_at = datetime(2026, 4, 29, 12, 0, tzinfo=UTC)
monkeypatch.setattr(
public_ip_route,
"check_public_ipv4_and_notify",
lambda session, bootstrap_settings: PublicIPCheckResult(
status="changed",
checked_at=fixed_checked_at,
changed=True,
),
)
_login(client)
response = client.get("/public-ip/check")
assert response.status_code == 200
assert response.json() == {
"status": "changed",
"checked_at": "2026-04-29T12:00:00Z",
"changed": True,
}
assert "current_ipv4" not in response.text
assert "previous_ipv4" not in response.text
assert "203.0.113.10" not in response.text
def _notification_settings() -> Settings:
return Settings(
_env_file=None,
app_env="development",
app_hostname="localhost:8000",
app_database_url="sqlite:///./data/app.db",
auth_bootstrap_username="admin",
auth_bootstrap_password="secret-password",
smtp_enabled=True,
smtp_host="smtp.example.com",
smtp_port=587,
smtp_username="smtp-user",
smtp_password="super-secret-password",
smtp_from_address="sender@example.com",
smtp_to_address="recipient@example.com",
smtp_use_starttls=True,
)
def test_public_ip_notification_sends_only_when_changed(auth_database, monkeypatch) -> None:
session = _make_session(auth_database["app_url"])
sent = []
monkeypatch.setattr(
"app.services.public_ip.send_public_ip_changed_email",
lambda settings, *, previous_ipv4, current_ipv4, detected_at: sent.append(
(previous_ipv4, current_ipv4, detected_at)
),
)
try:
first_seen = check_public_ipv4_and_notify(
session,
bootstrap_settings=_notification_settings(),
fetch_public_ipv4=lambda: "203.0.113.10",
)
unchanged = check_public_ipv4_and_notify(
session,
bootstrap_settings=_notification_settings(),
fetch_public_ipv4=lambda: "203.0.113.10",
)
changed = check_public_ipv4_and_notify(
session,
bootstrap_settings=_notification_settings(),
fetch_public_ipv4=lambda: "198.51.100.25",
)
finally:
session.close()
assert first_seen.status == "first_seen"
assert unchanged.status == "unchanged"
assert changed.status == "changed"
assert len(sent) == 1
assert sent[0][0] == "203.0.113.10"
assert sent[0][1] == "198.51.100.25"
assert sent[0][2] == changed.checked_at
def test_public_ip_notification_failure_does_not_break_changed_result(auth_database, monkeypatch) -> None:
session = _make_session(auth_database["app_url"])
monkeypatch.setattr(
"app.services.public_ip.send_public_ip_changed_email",
lambda settings, *, previous_ipv4, current_ipv4, detected_at: (_ for _ in ()).throw(
EmailDeliveryError("smtp down")
),
)
try:
check_public_ipv4(session, fetch_public_ipv4=lambda: "203.0.113.10")
result = check_public_ipv4_and_notify(
session,
bootstrap_settings=_notification_settings(),
fetch_public_ipv4=lambda: "198.51.100.25",
)
finally:
session.close()
assert result.status == "changed"
assert result.changed is True
assert result.previous_ipv4 == "203.0.113.10"
assert result.current_ipv4 == "198.51.100.25"
+395
View File
@@ -0,0 +1,395 @@
import re
import sqlite3
import smtplib
from fastapi.testclient import TestClient
from app.config import Settings
from app.services.email import (
EmailDeliveryError,
get_smtp_config,
is_smtp_ready,
send_public_ip_changed_email,
send_smtp_test_email,
)
def _extract_csrf_token(html: str) -> str:
match = re.search(r'name="csrf_token" value="([^"]+)"', html)
assert match is not None
return match.group(1)
def _login(client: TestClient) -> None:
login_page = client.get("/login")
csrf_token = _extract_csrf_token(login_page.text)
response = client.post(
"/login",
data={
"username": "admin",
"password": "test-password",
"csrf_token": csrf_token,
},
follow_redirects=False,
)
assert response.status_code == 303
def _smtp_settings(**overrides) -> Settings:
payload = {
"app_env": "development",
"app_hostname": "localhost:8000",
"app_database_url": "sqlite:///./data/app.db",
"auth_bootstrap_username": "admin",
"auth_bootstrap_password": "secret-password",
"smtp_enabled": True,
"smtp_host": "smtp.example.com",
"smtp_port": 587,
"smtp_username": "smtp-user",
"smtp_password": "super-secret-password",
"smtp_from_name": "Home Automation",
"smtp_from_address": "sender@example.com",
"smtp_to_address": "recipient@example.com",
"smtp_use_starttls": True,
}
payload.update(overrides)
return Settings(_env_file=None, **payload)
def test_get_smtp_config_reads_runtime_values() -> None:
settings = _smtp_settings(smtp_port=2525, smtp_use_starttls=False)
smtp_config = get_smtp_config(settings)
assert smtp_config.host == "smtp.example.com"
assert smtp_config.port == 2525
assert smtp_config.username == "smtp-user"
assert smtp_config.password == "super-secret-password"
assert smtp_config.from_name == "Home Automation"
assert smtp_config.from_address == "sender@example.com"
assert smtp_config.to_address == "recipient@example.com"
assert smtp_config.use_starttls is False
def test_smtp_test_readiness_does_not_require_smtp_enabled() -> None:
settings = _smtp_settings(smtp_enabled=False)
assert is_smtp_ready(settings) is True
def test_send_smtp_test_email_success(monkeypatch) -> None:
sent = {}
class FakeSMTP:
def __init__(self, host, port, timeout):
sent["host"] = host
sent["port"] = port
sent["timeout"] = timeout
def __enter__(self):
return self
def __exit__(self, exc_type, exc, tb):
return None
def ehlo(self):
sent["ehlo"] = sent.get("ehlo", 0) + 1
def starttls(self):
sent["starttls"] = True
def login(self, username, password):
sent["username"] = username
sent["password"] = password
def send_message(self, message, from_addr=None, to_addrs=None):
sent["subject"] = message["Subject"]
sent["from"] = message["From"]
sent["to"] = message["To"]
sent["body"] = message.get_content()
sent["envelope_from"] = from_addr
sent["envelope_to"] = to_addrs
monkeypatch.setattr("app.services.email.smtplib.SMTP", FakeSMTP)
send_smtp_test_email(_smtp_settings())
assert sent["host"] == "smtp.example.com"
assert sent["port"] == 587
assert sent["timeout"] == 10
assert sent["starttls"] is True
assert sent["username"] == "smtp-user"
assert sent["password"] == "super-secret-password"
assert sent["subject"] == "Home Automation SMTP Test"
assert sent["from"] == "Home Automation <sender@example.com>"
assert sent["to"] == "recipient@example.com"
assert sent["envelope_from"] == "sender@example.com"
assert sent["envelope_to"] == ["recipient@example.com"]
assert "This is a test email" in sent["body"]
def test_send_smtp_test_email_does_not_require_smtp_enabled(monkeypatch) -> None:
sent = {}
class FakeSMTP:
def __init__(self, host, port, timeout):
sent["host"] = host
def __enter__(self):
return self
def __exit__(self, exc_type, exc, tb):
return None
def ehlo(self):
return None
def starttls(self):
return None
def login(self, username, password):
return None
def send_message(self, message, from_addr=None, to_addrs=None):
sent["subject"] = message["Subject"]
sent["from"] = message["From"]
sent["envelope_from"] = from_addr
monkeypatch.setattr("app.services.email.smtplib.SMTP", FakeSMTP)
send_smtp_test_email(_smtp_settings(smtp_enabled=False))
assert sent["host"] == "smtp.example.com"
assert sent["subject"] == "Home Automation SMTP Test"
assert sent["from"] == "Home Automation <sender@example.com>"
assert sent["envelope_from"] == "sender@example.com"
def test_send_smtp_test_email_failure_sanitizes_password(monkeypatch) -> None:
class FakeSMTP:
def __init__(self, host, port, timeout):
pass
def __enter__(self):
return self
def __exit__(self, exc_type, exc, tb):
return None
def ehlo(self):
return None
def starttls(self):
raise smtplib.SMTPException("authentication failed for super-secret-password")
monkeypatch.setattr("app.services.email.smtplib.SMTP", FakeSMTP)
try:
send_smtp_test_email(_smtp_settings())
assert False, "expected EmailDeliveryError"
except EmailDeliveryError as exc:
assert "super-secret-password" not in str(exc)
assert "[redacted]" in str(exc)
def test_send_public_ip_changed_email_contains_expected_english_content(monkeypatch) -> None:
sent = {}
class FakeSMTP:
def __init__(self, host, port, timeout):
sent["host"] = host
def __enter__(self):
return self
def __exit__(self, exc_type, exc, tb):
return None
def ehlo(self):
return None
def starttls(self):
return None
def login(self, username, password):
return None
def send_message(self, message, from_addr=None, to_addrs=None):
sent["subject"] = message["Subject"]
sent["body"] = message.get_content()
sent["from"] = message["From"]
sent["envelope_from"] = from_addr
monkeypatch.setattr("app.services.email.smtplib.SMTP", FakeSMTP)
send_public_ip_changed_email(
_smtp_settings(),
previous_ipv4="203.0.113.10",
current_ipv4="198.51.100.25",
detected_at=__import__("datetime").datetime(2026, 4, 29, 10, 0, tzinfo=__import__("datetime").UTC),
)
assert sent["subject"] == "Public IP changed"
assert sent["from"] == "Home Automation <sender@example.com>"
assert sent["envelope_from"] == "sender@example.com"
assert "Your public IPv4 address has changed." in sent["body"]
assert "Previous IP: 203.0.113.10" in sent["body"]
assert "Current IP: 198.51.100.25" in sent["body"]
assert "Detected at: 2026-04-29 10:00:00 UTC" in sent["body"]
assert "update the trusted IP manually" in sent["body"]
def test_config_update_does_not_clear_existing_smtp_password(
client: TestClient, test_database_urls
) -> None:
_login(client)
config_page = client.get("/config")
config_csrf_token = _extract_csrf_token(config_page.text)
response = client.post(
"/config",
data={
"csrf_token": config_csrf_token,
"APP_NAME": "SMTP Config Test",
"APP_ENV": "development",
"APP_DEBUG": "true",
"APP_HOSTNAME": "localhost:8000",
"SMTP_ENABLED": "true",
"SMTP_HOST": "smtp.example.com",
"SMTP_PORT": "587",
"SMTP_USERNAME": "smtp-user",
"SMTP_PASSWORD": "persist-me",
"SMTP_FROM_ADDRESS": "sender@example.com",
"SMTP_TO_ADDRESS": "recipient@example.com",
"SMTP_USE_STARTTLS": "true",
"AUTH_SESSION_COOKIE_NAME": "home_automation_session",
"AUTH_SESSION_TTL_HOURS": "12",
"AUTH_COOKIE_SECURE_OVERRIDE": "false",
"POO_WEBHOOK_ID": "",
"POO_SENSOR_ENTITY_NAME": "sensor.test_poo_status",
"POO_SENSOR_FRIENDLY_NAME": "Poo Status",
"TICKTICK_CLIENT_ID": "",
"TICKTICK_CLIENT_SECRET": "",
"TICKTICK_TOKEN": "",
"HOME_ASSISTANT_BASE_URL": "",
"HOME_ASSISTANT_AUTH_TOKEN": "",
"HOME_ASSISTANT_TIMEOUT_SECONDS": "1.0",
"HOME_ASSISTANT_ACTION_TASK_PROJECT_ID": "",
},
follow_redirects=False,
)
assert response.status_code == 303
config_page = client.get("/config")
config_csrf_token = _extract_csrf_token(config_page.text)
response = client.post(
"/config",
data={
"csrf_token": config_csrf_token,
"APP_NAME": "SMTP Config Updated",
"APP_ENV": "development",
"APP_DEBUG": "true",
"APP_HOSTNAME": "localhost:8000",
"SMTP_ENABLED": "true",
"SMTP_HOST": "smtp.example.com",
"SMTP_PORT": "587",
"SMTP_USERNAME": "smtp-user",
"SMTP_PASSWORD": "",
"SMTP_FROM_ADDRESS": "sender@example.com",
"SMTP_TO_ADDRESS": "recipient@example.com",
"SMTP_USE_STARTTLS": "true",
"AUTH_SESSION_COOKIE_NAME": "home_automation_session",
"AUTH_SESSION_TTL_HOURS": "12",
"AUTH_COOKIE_SECURE_OVERRIDE": "false",
"POO_WEBHOOK_ID": "",
"POO_SENSOR_ENTITY_NAME": "sensor.test_poo_status",
"POO_SENSOR_FRIENDLY_NAME": "Poo Status",
"TICKTICK_CLIENT_ID": "",
"TICKTICK_CLIENT_SECRET": "",
"TICKTICK_TOKEN": "",
"HOME_ASSISTANT_BASE_URL": "",
"HOME_ASSISTANT_AUTH_TOKEN": "",
"HOME_ASSISTANT_TIMEOUT_SECONDS": "1.0",
"HOME_ASSISTANT_ACTION_TASK_PROJECT_ID": "",
},
follow_redirects=False,
)
assert response.status_code == 303
conn = sqlite3.connect(test_database_urls["app_path"])
try:
rows = dict(conn.execute("SELECT key, value FROM app_config").fetchall())
finally:
conn.close()
assert rows["SMTP_PASSWORD"] == "persist-me"
assert rows["APP_NAME"] == "SMTP Config Updated"
def test_smtp_test_endpoint_requires_authentication(client: TestClient) -> None:
response = client.post("/config/smtp/test", data={"csrf_token": "ignored"}, follow_redirects=False)
assert response.status_code == 303
assert response.headers["location"] == "/login"
def test_smtp_test_endpoint_success_and_failure_do_not_expose_password(
client: TestClient, monkeypatch
) -> None:
from app.api.routes import pages
_login(client)
config_page = client.get("/config")
csrf_token = _extract_csrf_token(config_page.text)
monkeypatch.setattr(pages, "send_smtp_test_email", lambda settings: None)
response = client.post("/config/smtp/test", data={"csrf_token": csrf_token}, follow_redirects=False)
assert response.status_code == 303
assert response.headers["location"] == "/config?smtp_test=success"
follow_up = client.get(response.headers["location"])
assert follow_up.status_code == 200
assert "SMTP test email sent successfully." in follow_up.text
assert "super-secret-password" not in follow_up.text
monkeypatch.setattr(
pages,
"send_smtp_test_email",
lambda settings: (_ for _ in ()).throw(EmailDeliveryError("smtp auth failed for [redacted]")),
)
response = client.post("/config/smtp/test", data={"csrf_token": csrf_token}, follow_redirects=False)
assert response.status_code == 303
assert response.headers["location"] == "/config?smtp_test=failed"
follow_up = client.get(response.headers["location"])
assert follow_up.status_code == 200
assert "SMTP test failed. Check saved SMTP settings and server reachability." in follow_up.text
assert "super-secret-password" not in follow_up.text
def test_config_page_renders_smtp_test_button_with_formaction(
client: TestClient, test_database_urls
) -> None:
_login(client)
conn = sqlite3.connect(test_database_urls["app_path"])
try:
conn.executemany(
"INSERT INTO app_config (key, value, updated_at) VALUES (?, ?, CURRENT_TIMESTAMP) "
"ON CONFLICT(key) DO UPDATE SET value=excluded.value, updated_at=excluded.updated_at",
[
("SMTP_ENABLED", "true"),
("SMTP_HOST", "smtp.example.com"),
("SMTP_PORT", "587"),
("SMTP_FROM_ADDRESS", "sender@example.com"),
("SMTP_TO_ADDRESS", "recipient@example.com"),
],
)
conn.commit()
finally:
conn.close()
response = client.get("/config")
assert response.status_code == 200
assert 'formaction="/config/smtp/test"' in response.text
+7 -22
View File
@@ -6,7 +6,7 @@ from urllib.parse import parse_qs, urlparse
import pytest
from fastapi.testclient import TestClient
from app.auth_db import reset_auth_db_caches
from app.db import reset_db_caches
from app.config import Settings, get_settings
from app.integrations.ticktick import (
AUTH_SCOPE,
@@ -209,9 +209,6 @@ def test_create_task_posts_expected_payload(monkeypatch: pytest.MonkeyPatch) ->
def test_homeassistant_publish_creates_ticktick_action_task(
test_database_urls,
ready_location_database,
ready_poo_database,
auth_database,
monkeypatch: pytest.MonkeyPatch,
) -> None:
@@ -221,7 +218,7 @@ def test_homeassistant_publish_creates_ticktick_action_task(
monkeypatch.setenv("TICKTICK_TOKEN", "ticktick-access-token")
monkeypatch.setenv("HOME_ASSISTANT_ACTION_TASK_PROJECT_ID", "project-123")
get_settings.cache_clear()
reset_auth_db_caches()
reset_db_caches()
captured = {"calls": []}
@@ -255,9 +252,6 @@ def test_homeassistant_publish_creates_ticktick_action_task(
def test_ticktick_auth_start_redirects_authenticated_user(
test_database_urls,
ready_location_database,
ready_poo_database,
auth_database,
monkeypatch: pytest.MonkeyPatch,
) -> None:
@@ -265,7 +259,7 @@ def test_ticktick_auth_start_redirects_authenticated_user(
monkeypatch.setenv("TICKTICK_CLIENT_ID", "ticktick-client-id")
monkeypatch.setenv("TICKTICK_CLIENT_SECRET", "ticktick-client-secret")
get_settings.cache_clear()
reset_auth_db_caches()
reset_db_caches()
monkeypatch.setattr("app.integrations.ticktick.secrets.token_hex", lambda _: "state-redirect")
with TestClient(create_app()) as client:
@@ -291,9 +285,6 @@ def test_ticktick_auth_start_redirects_authenticated_user(
def test_ticktick_auth_callback_persists_token(
test_database_urls,
ready_location_database,
ready_poo_database,
auth_database,
monkeypatch: pytest.MonkeyPatch,
) -> None:
@@ -301,7 +292,7 @@ def test_ticktick_auth_callback_persists_token(
monkeypatch.setenv("TICKTICK_CLIENT_ID", "ticktick-client-id")
monkeypatch.setenv("TICKTICK_CLIENT_SECRET", "ticktick-client-secret")
get_settings.cache_clear()
reset_auth_db_caches()
reset_db_caches()
default_auth_state_store.pending_state = "callback-state"
def fake_urlopen(req, timeout):
@@ -318,7 +309,7 @@ def test_ticktick_auth_callback_persists_token(
assert response.status_code == 303
assert response.headers["location"] == "/config?ticktick_oauth=success"
conn = sqlite3.connect(test_database_urls["app_path"])
conn = sqlite3.connect(auth_database["app_path"])
try:
row = conn.execute(
"SELECT value FROM app_config WHERE key = ?",
@@ -332,9 +323,6 @@ def test_ticktick_auth_callback_persists_token(
def test_ticktick_auth_callback_redirects_on_invalid_state(
test_database_urls,
ready_location_database,
ready_poo_database,
auth_database,
monkeypatch: pytest.MonkeyPatch,
) -> None:
@@ -342,7 +330,7 @@ def test_ticktick_auth_callback_redirects_on_invalid_state(
monkeypatch.setenv("TICKTICK_CLIENT_ID", "ticktick-client-id")
monkeypatch.setenv("TICKTICK_CLIENT_SECRET", "ticktick-client-secret")
get_settings.cache_clear()
reset_auth_db_caches()
reset_db_caches()
default_auth_state_store.pending_state = "expected-state"
with TestClient(create_app()) as client:
@@ -356,9 +344,6 @@ def test_ticktick_auth_callback_redirects_on_invalid_state(
def test_ticktick_auth_callback_redirects_when_token_exchange_fails(
test_database_urls,
ready_location_database,
ready_poo_database,
auth_database,
monkeypatch: pytest.MonkeyPatch,
) -> None:
@@ -366,7 +351,7 @@ def test_ticktick_auth_callback_redirects_when_token_exchange_fails(
monkeypatch.setenv("TICKTICK_CLIENT_ID", "ticktick-client-id")
monkeypatch.setenv("TICKTICK_CLIENT_SECRET", "ticktick-client-secret")
get_settings.cache_clear()
reset_auth_db_caches()
reset_db_caches()
default_auth_state_store.pending_state = "callback-state"
def fake_urlopen(req, timeout):