Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
6e197d7808 | ||
|
|
018f13d73d | ||
|
|
8180082f90 | ||
|
|
b472f91f19 | ||
|
|
c9ce05d95a | ||
|
|
33ca3da593 | ||
|
|
d5623b9fcb | ||
|
|
b1b6a309cb | ||
|
|
8dc3f71aaf | ||
|
|
2be4f78f8a | ||
|
|
1a958cb102 | ||
|
|
c851bad829 | ||
|
|
09abe05f66 | ||
|
|
0924e8df52 | ||
|
|
72ac7e7300 | ||
|
|
18083822ea | ||
|
|
3beeb5a461 | ||
|
|
631b14e2ec | ||
|
|
231c340ea6 | ||
|
|
c24b6684cc | ||
|
|
d9e82038dc | ||
|
|
e59c192097 | ||
|
|
2a47dab272 | ||
|
|
9db7f63274 | ||
|
|
ebf96de4f1 | ||
|
|
5b9d60e80a | ||
|
|
963e43e3e4 | ||
|
|
39c11ae606 | ||
|
|
3eec701448 | ||
|
|
489e5b596a | ||
|
|
567ddb9779 | ||
|
|
0fb51d338c | ||
|
|
b812d5ac46 | ||
|
|
a9458394f2 | ||
|
|
4884a19e3d | ||
|
|
afe653bafa | ||
|
|
25a08c47a4 | ||
|
|
ffc693e995 | ||
|
|
5855fff451 | ||
|
|
1ea2f659e0 | ||
|
|
2e125dbd53 | ||
|
|
28486a83c7 | ||
|
|
a78401c2ef | ||
|
|
009856a50d | ||
|
|
43c2ddce1a | ||
|
|
d7f04aee8c | ||
|
|
8dbb59a3b7 | ||
|
|
2992bbb0ef | ||
|
|
22faeb45bb | ||
|
|
c37dfacfc7 | ||
|
|
16b050d821 | ||
|
|
d3d914b117 | ||
|
|
fc4af857e3 | ||
|
|
0958d9a2e9 |
@@ -7,8 +7,14 @@ APP_DATABASE_URL=sqlite:////app/data/app.db
|
|||||||
AUTH_BOOTSTRAP_USERNAME=admin
|
AUTH_BOOTSTRAP_USERNAME=admin
|
||||||
AUTH_BOOTSTRAP_PASSWORD=change-me
|
AUTH_BOOTSTRAP_PASSWORD=change-me
|
||||||
|
|
||||||
|
# Required by Docker Compose for the WarmteLink serial device. Set these only in
|
||||||
|
# your local .env; use a stable /dev/serial/by-id path and its numeric host GID.
|
||||||
|
# WARMTELINK_DEVICE_PATH=/dev/serial/by-id/<stable-by-id-name>
|
||||||
|
# WARMTELINK_SERIAL_GID=<host-serial-gid>
|
||||||
|
|
||||||
# Optional: runtime overrides.
|
# Optional: runtime overrides.
|
||||||
# Leave these commented out to use the application's built-in defaults.
|
# Leave these commented out to use the application's built-in defaults.
|
||||||
|
# TZ=Europe/Amsterdam
|
||||||
# APP_DEBUG=
|
# APP_DEBUG=
|
||||||
# AUTH_SESSION_COOKIE_NAME=
|
# AUTH_SESSION_COOKIE_NAME=
|
||||||
# AUTH_SESSION_TTL_HOURS=
|
# AUTH_SESSION_TTL_HOURS=
|
||||||
@@ -40,6 +46,8 @@ MQTT_BROKER_PORT=1883
|
|||||||
MQTT_USERNAME=
|
MQTT_USERNAME=
|
||||||
MQTT_PASSWORD=
|
MQTT_PASSWORD=
|
||||||
MQTT_TLS_ENABLED=false
|
MQTT_TLS_ENABLED=false
|
||||||
|
# MQTT_CLIENT_ID must be a non-empty ASCII slug; use a distinct value per deployment.
|
||||||
|
MQTT_CLIENT_ID=home-automation
|
||||||
|
|
||||||
# Optional: Home Assistant MQTT Discovery.
|
# Optional: Home Assistant MQTT Discovery.
|
||||||
# Requires MQTT_ENABLED=true and a running MQTT broker.
|
# Requires MQTT_ENABLED=true and a running MQTT broker.
|
||||||
|
|||||||
@@ -0,0 +1,208 @@
|
|||||||
|
# AGENTS.md — Home Automation Backend
|
||||||
|
|
||||||
|
本文件是本仓库 coding agent 指引的 **single source of truth**;`CLAUDE.md` 通过符号链接指向本文件。它定义本项目的**工作流程、文档位置、commit 规范**。支持对应项目指引的 agent 在动手前应完整读取本文件。
|
||||||
|
|
||||||
|
## 项目速览
|
||||||
|
|
||||||
|
- 个人用 home-automation 应用:**FastAPI + React SPA + SQLite + SQLAlchemy + Alembic**,前后端同源托管。
|
||||||
|
- 单 admin 鉴权(Argon2 + server-side session cookie),runtime config 落 `app_config` 表。
|
||||||
|
- 模块:public IPv4 monitor、SMTP 通知、location / poo recorder、Home Assistant in/out、TickTick OAuth、Modbus / DSMR 能耗采集、MQTT / HA Discovery、动态电价与电费计算。
|
||||||
|
- 已发布 `v1.5.1`。M1、M2、M4-M7 已完成;M3 token / 移动端仍为远期方向。
|
||||||
|
- **当前现实**:已收敛为单一 `app.db`、一套 DeclarativeBase 和一条 Alembic 链;只有历史数据迁移 runbook 会读取旧 location / poo 数据库。
|
||||||
|
- 明确不做:Notion 模块。
|
||||||
|
|
||||||
|
## 文档地图与「开工前必读」
|
||||||
|
|
||||||
|
文档都在 `docs/`:
|
||||||
|
|
||||||
|
| 路径 | 作用 |
|
||||||
|
| --- | --- |
|
||||||
|
| `docs/roadmap.md` | 全局规划与里程碑总览 |
|
||||||
|
| `docs/design/README.md` | **协作契约**:任务卡格式、原子任务定义、校验闸门、数据安全红线 |
|
||||||
|
| `docs/design/m1-db-consolidation.md` | M1 原子任务(含真实代码现状盘点 + 人工 runbook) |
|
||||||
|
| `docs/design/m2-frontend-v2.md` | M2 原子任务 + API 契约 + 前端校验闸门 |
|
||||||
|
| `docs/design/m3-token-mobile.md` | M3(远期,暂缓) |
|
||||||
|
| `docs/design/m4-login-hardening.md` | M4 登录加固(已完成) |
|
||||||
|
| `docs/design/m5-iot-energy.md` | M5 IoT / 能耗采集(已完成) |
|
||||||
|
| `docs/design/m6-tibber-dynamic-energy.md` | M6 动态电价、DSMR 与电费计算(已完成) |
|
||||||
|
| `docs/design/m7-meter-epochs-archival.md` | M7 电表生命周期 / 换表归档(已完成) |
|
||||||
|
| `docs/*.md`(auth / public-ip-monitor / location-recorder …) | 各模块说明,按需读 |
|
||||||
|
|
||||||
|
**开工时读取顺序**:
|
||||||
|
1. `docs/design/README.md`(每轮都读,它是流程与验收的共同契约)。
|
||||||
|
2. 本轮对应的 milestone 文档(如 `docs/design/m1-db-consolidation.md`),定位要做的任务卡。
|
||||||
|
3. 任务卡 `Files` 列出的源文件 + 该模块的 `docs/*.md`(按需)。
|
||||||
|
4. `docs/roadmap.md` 仅在需要全局视角时读。
|
||||||
|
|
||||||
|
## 工作流程
|
||||||
|
|
||||||
|
### 实现模式(由用户的提示词决定)
|
||||||
|
|
||||||
|
- **默认逐步**:给一个 milestone 文档,按其中原子任务**一步一步**实现。
|
||||||
|
- **(a) 只实现一步**:用户说"只实现一步 / 这一个任务"时,**只做那一个任务卡**,跑完校验闸门后停下,等用户确认,不要顺手往下做。
|
||||||
|
- **(b) 完成整个 milestone**:仅当用户在提示词里**显式要求启用 sub-agent**时,才起 implementer / reviewer / fixer sub-agent(按下方**『默认能力档位』**选择模型,用户人工指定则覆盖),按任务依赖顺序跑完整条链。
|
||||||
|
- **Sub-agent 纪律**:只在用户显式要求时才 spawn sub-agent;单步/小改动在主线内联完成。当前 harness 支持独立 sub-agent 时,使用其原生机制按下方**『默认能力档位』**派发;不支持时不得假装已创建 sub-agent,应明确说明限制,并仅在用户允许 fallback 时由主 agent 继续。
|
||||||
|
|
||||||
|
### 默认能力档位(实现模式 sub-agent;可被人工指定覆盖)
|
||||||
|
|
||||||
|
起 implementer / reviewer / fixer sub-agent 时,**默认**按下列能力档位选择当前 harness 支持的模型,无需用户每次人工指定:
|
||||||
|
|
||||||
|
| 角色 | 通用模型要求 | 推理档位 | Harness 示例(非强制) |
|
||||||
|
| --- | --- | --- | --- |
|
||||||
|
| **Implementer** | 平衡型代码实现模型 | `medium` 或等效档位 | Claude Code:Sonnet;Codex/OpenAI:GPT-5.6 Terra (`gpt-5.6-terra`) |
|
||||||
|
| **Fixer**(返工) | 平衡型代码实现模型 | `medium` 或等效档位 | Claude Code:Sonnet;Codex/OpenAI:GPT-5.6 Terra (`gpt-5.6-terra`) |
|
||||||
|
| **Reviewer** | 当前 harness 支持的最强通用推理 / 代码模型 | `extra-high` / `xhigh` 或等效档位 | Claude Code:Opus;Codex/OpenAI:GPT-5.6 Sol (`gpt-5.6-sol`) |
|
||||||
|
|
||||||
|
- **示例非强制**:示例模型只表示当前推荐映射,不构成跨 harness 的硬性模型 ID;当前 harness 不支持时,选择最符合「通用模型要求」的可用模型。
|
||||||
|
- **选择优先级**:用户显式指定 > 当前 harness 的原生角色配置 > 上表的 harness 示例 > 按通用模型要求自动选择。
|
||||||
|
- **推理档位说明**:若 harness 提供独立的 reasoning-effort 设置,按上表设置;若不提供,在 spawn prompt 中明确 implementer/fixer 按平衡深度思考,reviewer 按对抗性外部审计强度复核。
|
||||||
|
|
||||||
|
### 角色(Orchestrator → Implementer → Reviewer → Fixer)
|
||||||
|
|
||||||
|
- 我(主线)= **Orchestrator**:挑依赖已满足的下一个任务、派发、转述结果、维护任务 `Status`。
|
||||||
|
- **Implementer**(平衡型代码实现模型,medium 或等效档位):一次一个任务,严格按任务卡,不扩范围。
|
||||||
|
- **Reviewer**(最强通用推理 / 代码模型,extra-high / xhigh 或等效档位):实现完成后起 Reviewer sub-agent,按任务卡 `Acceptance criteria` + `Reviewer checklist` 复核、**独立重跑校验闸门**,驱动返工直到本轮 PASS。
|
||||||
|
- **Fixer**(平衡型代码实现模型,medium 或等效档位):按 reviewer 的编号返工清单返工;**每轮返工起一个干净的 Fixer**(与首次实现的 Implementer 分开冷启动),先读对应 `review-notes/<task>-review-<n>.md` 再改。
|
||||||
|
|
||||||
|
#### Reviewer 盲审纪律(M1 教训)
|
||||||
|
|
||||||
|
M1 里 review **从未触发过一次 rework**,根因是 orchestrator 把自己的结论 / 辩护喂给了 reviewer,造成 context bleed、review 沦为橡皮图章。所以:
|
||||||
|
|
||||||
|
- reviewer 必须**使用全新、独立的 sub-agent / thread 冷启动,并最小化喂料**——spawn prompt 只给:① 任务卡(`Acceptance criteria` + `Reviewer checklist`)、② 对应的 `review-notes/<task>-impl|rework-<n>.md` 路径、③ 要审的 diff / commit 范围。
|
||||||
|
- **不要**在 prompt 里塞 orchestrator 自己的判断、"我觉得没问题"、对实现选择的辩护,或上一轮 reviewer 的倾向性结论。让它**独立得出结论、独立重跑校验闸门**。
|
||||||
|
- 事后另起的整库**独立盲审**(如对抗复审)同理:使用全新独立的 agent / thread、最小上下文,把它当"**外部审计**"而非"确认自己没错"。
|
||||||
|
|
||||||
|
### 校验闸门(每个任务结束都要全绿)
|
||||||
|
|
||||||
|
根目录、激活 `.venv` 后:
|
||||||
|
```bash
|
||||||
|
pytest # 权威闸门(CI 跑的就是它)
|
||||||
|
ruff check . # line-length=100
|
||||||
|
python scripts/export_openapi.py && git diff --exit-code openapi/ # 改了路由/schema 才需要,且产物须入库
|
||||||
|
```
|
||||||
|
前端任务(M2)在 `frontend/` 下另跑 `npm run lint && npm run typecheck && npm run test && npm run build`(详见 m2 文档 §8)。
|
||||||
|
**不过闸门就不算完成**,不得跳过、不得留红给下一轮。
|
||||||
|
|
||||||
|
**Repo-meta 例外**:纯文档、agent 指引、符号链接等不影响可执行代码、构建与 API 契约的变更,可在用户明确同意时跳过代码闸门。仍须完成针对性校验(如链接目标、文件类型、diff 与 Git 状态),并在结果中明确记录未运行哪些闸门。
|
||||||
|
|
||||||
|
#### API 契约同步:`openapi/` 与 `schema.d.ts` 是**两步**(v1.4.0 后教训)
|
||||||
|
|
||||||
|
**只跑 `export_openapi.py` 不够。** 前端的 `frontend/src/api/schema.d.ts` 是由 `openapi/openapi.json` 二次生成的,CI(`.github/workflows/frontend.yml` 的 *Check codegen is in sync*)会重跑 codegen 并 `git diff --exit-code src/api/schema.d.ts`。漏了第二步 → 本地闸门全绿、远端 CI 红。真出过:`d07a083` 改了 `/api/energy/prices` 的 docstring 并同步了 `openapi.json`,但没重跑 codegen,只差一行注释就把 CI 挂了。
|
||||||
|
|
||||||
|
所以**只要动了路由 / Pydantic schema / 路由 docstring**(docstring 也会进 OpenAPI description!),两步都要跑,两个产物都要入库:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1) 后端契约
|
||||||
|
python scripts/export_openapi.py && git diff --exit-code openapi/
|
||||||
|
# 2) 前端类型(在 frontend/ 下)
|
||||||
|
npm run codegen && git diff --exit-code src/api/schema.d.ts
|
||||||
|
```
|
||||||
|
|
||||||
|
- 判据:`git diff --exit-code openapi/` 有输出 → **必然**还要跑一次 `npm run codegen`。
|
||||||
|
- 反过来也成立:`schema.d.ts` 不要手改,它是生成物。
|
||||||
|
- Reviewer 审"动了路由 / schema / 路由 docstring"类任务时,把**这两个产物是否都已重新生成并入库**当作 acceptance 的一部分。
|
||||||
|
|
||||||
|
### 构建上下文完整性(M1 Dockerfile 教训)
|
||||||
|
|
||||||
|
`docker build` **不在 pytest/ruff 闸门里**——M1 删了 `alembic_location/poo` 后忘了同步 `Dockerfile` 的 `COPY`,单元闸门全绿却把坏掉的镜像构建一路漏到 release tag。所以:
|
||||||
|
|
||||||
|
- 任务**删除 / 移动 / 重命名文件或目录**时,必须 grep 构建清单是否还在引用它们:`Dockerfile`(尤其 `COPY` 源)、`docker/`、`*.ini`、CI workflow、`requirements*.txt` 等。
|
||||||
|
- 已有回归测试 `tests/test_deployment.py::test_dockerfile_copy_sources_exist` 守"Dockerfile `COPY` 源必须存在于构建上下文";新增 / 改动 `COPY` 时确保它仍覆盖得到。
|
||||||
|
- Reviewer 审"删 / 移文件"类任务时,**必须顺带核对构建清单引用**,把它当 acceptance 的一部分。
|
||||||
|
|
||||||
|
## 每轮简报(`review-notes/`)
|
||||||
|
|
||||||
|
由 milestone 任务卡驱动的每轮实现、返工或 review,都要在 `review-notes/` 下产出**中文简报**。该目录**已在 `.gitignore` 忽略**,纯本地、不入库——它是 agent 之间和与人之间的交接载体,不是仓库产物。纯讨论、只读分析与不进入正式任务链的 repo-meta 变更无需产出简报,除非用户明确要求。
|
||||||
|
|
||||||
|
- **实现 / 返工简报**:每轮实现完成后(无论首次实现还是返工),写一份。文件名建议 `<task-id>-impl-<n>.md` / `<task-id>-rework-<n>.md`(如 `M1-T03-impl-1.md`、`M1-T03-rework-1.md`)。至少包含:
|
||||||
|
1. **本轮修改的具体内容**(改了哪些文件、做了什么、为什么)。
|
||||||
|
2. **自动化测试结果**(`pytest` / `ruff` / 前端闸门的实际输出或结论,通过/失败逐项写清)。
|
||||||
|
3. **若需人工 walkthrough**:写明具体步骤(怎么启动、点哪里、预期看到什么);若无需人工验证,明确写"无需人工 walkthrough"。
|
||||||
|
- **review 简报**:每轮 review 后写一份,文件名建议 `<task-id>-review-<n>.md`(如 `M1-T03-review-1.md`)。至少包含:评审结论(`PASS` 或带编号的返工清单)、对照任务卡 `Acceptance criteria` + `Reviewer checklist` 的逐条核对、reviewer 独立重跑校验闸门的结果。
|
||||||
|
|
||||||
|
**用途**:① reviewer 审核时参考对应的实现简报;② implementer 返工时参考对应的 review 简报;③ 人类(用户)通读这些简报确认有无问题。简报之间用文件名里的 `<task-id>` 与轮次 `<n>` 对应起来。
|
||||||
|
|
||||||
|
### Orchestrator 派发契约(让简报真正被读到)
|
||||||
|
|
||||||
|
**关键**:sub-agent 冷启动、不继承主线上下文,**不会因为本文件提到简报就自动去读**对应文件。简报能流转,靠的是 orchestrator(主线)在**每次 spawn 时把路径显式写进 prompt**,而不是被动约定。所以派发时必须做到:
|
||||||
|
|
||||||
|
- **显式告诉它「先读哪个简报」**:
|
||||||
|
- 派 implementer 做**首次实现** → 传任务卡位置(milestone 文档路径 + task id);无前置简报。
|
||||||
|
- 派 implementer 做**返工** → 必须传对应的 `review-notes/<task>-review-<n>.md` 路径,并要求**先读它**再改。
|
||||||
|
- 派 reviewer → 必须传对应的 `review-notes/<task>-impl|rework-<n>.md` 路径 + 任务卡,要求**先读它**再评。
|
||||||
|
- **显式告诉它「本轮结束写哪个简报」**:明确给出输出路径 `review-notes/<task>-<impl|rework|review>-<n>.md` 及上面要求的内容项。
|
||||||
|
- **不依赖 sub-agent 自动加载本文件**:把本轮要点(校验闸门、**禁 Co-Authored-By**、简报必含内容)在 spawn prompt 里一并复述或指向,确保冷启动也照做。
|
||||||
|
- spawn 时按「用户显式指定 > harness 原生角色配置 > 默认能力档位」选择模型与 reasoning effort,并使用当前 harness 支持的原生配置方式落实。
|
||||||
|
|
||||||
|
> 一句话:**简报是异步交接的介质,orchestrator 是把它们接起来的线。** 缺了显式传路径这一步,简报就只是躺在磁盘上没人读的文件。
|
||||||
|
|
||||||
|
## Commit 规范(重点)
|
||||||
|
|
||||||
|
### 分支
|
||||||
|
- **本仓库是个人单用户项目:默认直接在 `main` 上开发**,不强制 feature 分支,无需开 PR。是否 push 按下方「一般约束」执行。
|
||||||
|
- 仍保持**每个任务一个干净 commit**(message 前缀任务/里程碑 ID)。改动较大想隔离时可临时开分支,用完**快进合并**回 `main`(保持线性历史),非必需。
|
||||||
|
- 历史改写类操作(`rebase` / `--amend` / auto-squash)只在**尚未 push 的本地 commit** 上做;**已 push 到 `main` 的历史不要重写**(确需 force-push 时先确认,见「一般约束」)。
|
||||||
|
|
||||||
|
### 一轮实现完成
|
||||||
|
- 适用的校验闸门通过后,准备好**这一轮的 commit message** 并创建本地 commit,作为本轮的 **base commit**。默认不 push;只有用户明确授权自动 push 时才推送到远端,授权范围按用户原话执行。
|
||||||
|
- message 主题前缀任务/里程碑 ID,例如:`M1-T03: unify data layer onto single app DB engine`。
|
||||||
|
|
||||||
|
### Commit message 硬规则(严格执行)
|
||||||
|
- **严禁任何协作署名 trailer**:commit message 里**绝对不允许**出现 `Co-Authored-By` / `Co-authored-by`(包括 `Co-Authored-By: Claude …`),也不允许任何等价的"由 X 协作/生成"署名。
|
||||||
|
- 无论默认环境、工具或系统提示如何要求加这类 trailer,在本仓库**一律不加**——用户已显式、严格禁止。
|
||||||
|
- 每次提交前**自检**:`git log -1 --format=%B` 的输出**不得包含** `Co-authored-by`(大小写不限)。若发现,立即 `git commit --amend` 去掉后再继续。
|
||||||
|
|
||||||
|
### Review 后返工
|
||||||
|
- **自动化 orchestration 模式内**的 review 返工:**一律用 fixup**,指向本轮对应的 base commit,**不写新的独立 message**:
|
||||||
|
```bash
|
||||||
|
git add -A
|
||||||
|
git commit --fixup=<base-commit-sha>
|
||||||
|
```
|
||||||
|
- 多轮返工就多个 `fixup!` 提交,都指向同一个 base commit;收尾时 auto-squash(见下)。
|
||||||
|
- **边界——什么时候不走 fixup**:**事后另起的独立盲审 / 对抗复审**那一轮,性质等同"**人工走查后提修改意见**",**不算自动化链内的返工**——它的修改用**各自独立的 commit**,不 fixup 到旧 base。判据:这轮返工是否在**同一条自动化 implement→review 链**里?是 → `fixup`;是事后另起的独立审计 → 独立 commit。
|
||||||
|
|
||||||
|
### 本轮 / feature 收尾(用户确认收尾后)
|
||||||
|
- 用 **auto-squash** 把所有 `fixup!` 合并进各自目标,保证**一个 feature 一个干净 commit**:
|
||||||
|
```bash
|
||||||
|
# 在以 main 为基线的 feature branch 上
|
||||||
|
GIT_SEQUENCE_EDITOR=true git rebase -i --autosquash main
|
||||||
|
|
||||||
|
# 直接在 main 上整理尚未 push 的本地提交
|
||||||
|
GIT_SEQUENCE_EDITOR=true git rebase -i --autosquash origin/main
|
||||||
|
```
|
||||||
|
- 执行前确认选定的基线位于 base commit 之前,以便 base commit 与对应 `fixup!` 都进入 rebase 范围。用 `GIT_SEQUENCE_EDITOR=true` 让它**非交互**执行(不弹编辑器,自动接受 autosquash 排好的 todo)。
|
||||||
|
- autosquash **改写历史**:仅在 push / 开 PR **之前**做。若该分支已 push,需要 force-push——属对外操作,**先取得用户确认再做**。
|
||||||
|
|
||||||
|
### 一般约束
|
||||||
|
- **个人单用户仓库:默认直接在 `main` 上开发并创建本地 commit**。默认不 push;只有用户明确授权自动 push 时才推送到远端。
|
||||||
|
- 始终需要**单独、明确授权**的操作:**force-push / 改写已推送历史**,以及**打 tag**(会触发镜像 CI / 对外发布;且打 tag 前须按下方「发版前置走查」真跑一次 `docker build`)。
|
||||||
|
|
||||||
|
## 发版前置走查(打 tag 前必做)
|
||||||
|
|
||||||
|
单元闸门绿 ≠ 真的能跑、能构建、能用。M1 出过"绿了但 docker 构建坏了"的事故,所以**打版本 tag(触发镜像 CI)之前**,除了 `pytest` / `ruff` 全绿,还要:
|
||||||
|
|
||||||
|
- **真起 app**:迁移(`python -m scripts.run_migrations`)→ `uvicorn app.main:app ...`,确认能正常启动、关键路由不 500。
|
||||||
|
- **真跑镜像构建**:本地 `docker build`(多阶段就跑完整条),确认构建通过、`COPY` 源都在。
|
||||||
|
- **关键功能人工瞄一眼**:尤其前端 / 可视化类(M2 的热力图、首页地图)——自动闸门判断不了"渲染对不对、UX 顺不顺",这部分**靠看跑起来的 app,不靠读代码**。
|
||||||
|
- 上述任一不过 → **不打 tag**。tag 一旦 push 会触发 docker 镜像 CI / 对外发布,属对外操作,**先确认**。
|
||||||
|
|
||||||
|
## 数据安全红线(不可违反)
|
||||||
|
|
||||||
|
- 任何脚本 / migration **都不得删除或覆盖用户数据文件**(旧 `.db`、备份、volume)。删除只能是人工、事后、保留归档的独立步骤(见 `docs/design/m1-db-consolidation.md` §6 runbook)。
|
||||||
|
- 涉及历史数据的迁移**先在备份副本上演练**;迁移脚本必须幂等且搬完对账行数。
|
||||||
|
- Review 时只要发现"删文件 / drop 有数据的表 / truncate"出现在自动化任务里,直接判返工。
|
||||||
|
|
||||||
|
## 常用命令
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 环境
|
||||||
|
python -m venv .venv && source .venv/bin/activate && pip install -r dev-requirements.txt
|
||||||
|
# 迁移(初始化/适配 DB)
|
||||||
|
python -m scripts.run_migrations
|
||||||
|
# 起服务
|
||||||
|
uvicorn app.main:app --reload --host 0.0.0.0 --port 8000
|
||||||
|
# 测试 / lint / OpenAPI 导出
|
||||||
|
pytest
|
||||||
|
ruff check .
|
||||||
|
python scripts/export_openapi.py
|
||||||
|
```
|
||||||
@@ -1,198 +0,0 @@
|
|||||||
# CLAUDE.md — Home Automation Backend
|
|
||||||
|
|
||||||
本文件每次会话自动加载。它定义本项目的**工作流程、文档位置、commit 规范**。请在动手前先读完。
|
|
||||||
|
|
||||||
## 项目速览
|
|
||||||
|
|
||||||
- 个人用 home-automation 后端:**FastAPI + SQLite + SQLAlchemy + Alembic**,服务端模板(Jinja,M2 将换成 React SPA)。
|
|
||||||
- 单 admin 鉴权(Argon2 + server-side session cookie),runtime config 落 `app_config` 表。
|
|
||||||
- 模块:public IPv4 monitor、SMTP 通知、location recorder、poo recorder、Home Assistant in/out、TickTick OAuth。
|
|
||||||
- 已发布 `v1.0.3`。下一阶段方向:**M1 单库化 → M2 React 前端 → M3 token/移动端(远期,M2 后再说)**。
|
|
||||||
- **当前现实**:在 M1 完成前仍是**三个独立 SQLite 库**(app / location / poo),三套 DeclarativeBase、三条 Alembic 链。不要假设已经单库——以代码现状为准。
|
|
||||||
- 明确不做:Notion 模块。
|
|
||||||
|
|
||||||
## 文档地图与「开工前必读」
|
|
||||||
|
|
||||||
文档都在 `docs/`:
|
|
||||||
|
|
||||||
| 路径 | 作用 |
|
|
||||||
| --- | --- |
|
|
||||||
| `docs/roadmap.md` | 全局规划与里程碑总览 |
|
|
||||||
| `docs/design/README.md` | **协作契约**:任务卡格式、原子任务定义、校验闸门、数据安全红线 |
|
|
||||||
| `docs/design/m1-db-consolidation.md` | M1 原子任务(含真实代码现状盘点 + 人工 runbook) |
|
|
||||||
| `docs/design/m2-frontend-v2.md` | M2 原子任务 + API 契约 + 前端校验闸门 |
|
|
||||||
| `docs/design/m3-token-mobile.md` | M3(远期,暂缓) |
|
|
||||||
| `docs/*.md`(auth / public-ip-monitor / location-recorder …) | 各模块说明,按需读 |
|
|
||||||
|
|
||||||
**开工时读取顺序**:
|
|
||||||
1. `docs/design/README.md`(每轮都读,它是流程与验收的共同契约)。
|
|
||||||
2. 本轮对应的 milestone 文档(如 `docs/design/m1-db-consolidation.md`),定位要做的任务卡。
|
|
||||||
3. 任务卡 `Files` 列出的源文件 + 该模块的 `docs/*.md`(按需)。
|
|
||||||
4. `docs/roadmap.md` 仅在需要全局视角时读。
|
|
||||||
|
|
||||||
## 工作流程
|
|
||||||
|
|
||||||
### 实现模式(由用户的提示词决定)
|
|
||||||
|
|
||||||
- **默认逐步**:给一个 milestone 文档,按其中原子任务**一步一步**实现。
|
|
||||||
- **(a) 只实现一步**:用户说"只实现一步 / 这一个任务"时,**只做那一个任务卡**,跑完校验闸门后停下,等用户确认,不要顺手往下做。
|
|
||||||
- **(b) 完成整个 milestone**:仅当用户在提示词里**显式要求启用 sub-agent**时,才起 implementer / reviewer / fixer sub-agent(模型用下方**『默认模型档位』**,用户人工指定则覆盖),按任务依赖顺序跑完整条链。
|
|
||||||
- **Sub-agent 纪律**:只在用户显式要求时才 spawn sub-agent;单步/小改动在主线内联完成。起 sub-agent 时按下方**『默认模型档位』**选择模型(用户人工指定则以人工指定为准),用 Agent 工具的 `model` 字段落实。
|
|
||||||
|
|
||||||
### 默认模型档位(实现模式 sub-agent;可被人工指定覆盖)
|
|
||||||
|
|
||||||
起 implementer / reviewer / fixer sub-agent 时,**默认**用下列模型档位,无需用户每次人工指定:
|
|
||||||
|
|
||||||
| 角色 | 默认模型 | 推理档位 |
|
|
||||||
| --- | --- | --- |
|
|
||||||
| **Implementer** | **Sonnet** | high reasoning effort |
|
|
||||||
| **Fixer**(返工) | **Sonnet** | high reasoning effort |
|
|
||||||
| **Reviewer** | **Opus** | extra-high reasoning effort |
|
|
||||||
|
|
||||||
- **人工指定覆盖**:若用户在提示词里**显式指定了其他模型**(针对任一角色),则**以用户人工指定为准**,覆盖上述默认。
|
|
||||||
- 用 Agent 工具的 `model` 字段落实模型选择;该字段当前仅支持 `sonnet` / `opus` / `haiku` / `fable`。
|
|
||||||
- **推理档位说明**:Agent 工具未暴露独立的 reasoning-effort 旋钮,"high / extra-high reasoning" 通过 spawn prompt 里的显式指令传达(要求 implementer/fixer 动手前充分推理边界条件;要求 reviewer 以对抗性外部审计心态最高强度复核)。若所在 harness 提供真正的 effort 设置,则一并按此档位设置。
|
|
||||||
|
|
||||||
### 角色(Orchestrator → Implementer → Reviewer → Fixer)
|
|
||||||
|
|
||||||
- 我(主线)= **Orchestrator**:挑依赖已满足的下一个任务、派发、转述结果、维护任务 `Status`。
|
|
||||||
- **Implementer**(默认 **Sonnet**,high reasoning;见上方『默认模型档位』):一次一个任务,严格按任务卡,不扩范围。
|
|
||||||
- **Reviewer**(默认 **Opus**,extra-high reasoning):实现完成后起 Reviewer sub-agent,按任务卡 `Acceptance criteria` + `Reviewer checklist` 复核、**独立重跑校验闸门**,驱动返工直到本轮 PASS。
|
|
||||||
- **Fixer**(默认 **Sonnet**,high reasoning):按 reviewer 的编号返工清单返工;**每轮返工起一个干净的 Fixer**(与首次实现的 Implementer 分开冷启动),先读对应 `review-notes/<task>-review-<n>.md` 再改。
|
|
||||||
|
|
||||||
#### Reviewer 盲审纪律(M1 教训)
|
|
||||||
|
|
||||||
M1 里 review **从未触发过一次 rework**,根因是 orchestrator 把自己的结论 / 辩护喂给了 reviewer,造成 context bleed、review 沦为橡皮图章。所以:
|
|
||||||
|
|
||||||
- reviewer 必须**冷启动(Clear-Agent)、最小化喂料**——spawn prompt 只给:① 任务卡(`Acceptance criteria` + `Reviewer checklist`)、② 对应的 `review-notes/<task>-impl|rework-<n>.md` 路径、③ 要审的 diff / commit 范围。
|
|
||||||
- **不要**在 prompt 里塞 orchestrator 自己的判断、"我觉得没问题"、对实现选择的辩护,或上一轮 reviewer 的倾向性结论。让它**独立得出结论、独立重跑校验闸门**。
|
|
||||||
- 事后另起的整库**独立盲审**(如对抗复审)同理:Clear-Agent、最小上下文,把它当"**外部审计**"而非"确认自己没错"。
|
|
||||||
|
|
||||||
### 校验闸门(每个任务结束都要全绿)
|
|
||||||
|
|
||||||
根目录、激活 `.venv` 后:
|
|
||||||
```bash
|
|
||||||
pytest # 权威闸门(CI 跑的就是它)
|
|
||||||
ruff check . # line-length=100
|
|
||||||
python scripts/export_openapi.py && git diff --exit-code openapi/ # 改了路由/schema 才需要,且产物须入库
|
|
||||||
```
|
|
||||||
前端任务(M2)在 `frontend/` 下另跑 `npm run lint && npm run typecheck && npm run test && npm run build`(详见 m2 文档 §8)。
|
|
||||||
**不过闸门就不算完成**,不得跳过、不得留红给下一轮。
|
|
||||||
|
|
||||||
#### API 契约同步:`openapi/` 与 `schema.d.ts` 是**两步**(v1.4.0 后教训)
|
|
||||||
|
|
||||||
**只跑 `export_openapi.py` 不够。** 前端的 `frontend/src/api/schema.d.ts` 是由 `openapi/openapi.json` 二次生成的,CI(`.github/workflows/frontend.yml` 的 *Check codegen is in sync*)会重跑 codegen 并 `git diff --exit-code src/api/schema.d.ts`。漏了第二步 → 本地闸门全绿、远端 CI 红。真出过:`d07a083` 改了 `/api/energy/prices` 的 docstring 并同步了 `openapi.json`,但没重跑 codegen,只差一行注释就把 CI 挂了。
|
|
||||||
|
|
||||||
所以**只要动了路由 / Pydantic schema / 路由 docstring**(docstring 也会进 OpenAPI description!),两步都要跑,两个产物都要入库:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# 1) 后端契约
|
|
||||||
python scripts/export_openapi.py && git diff --exit-code openapi/
|
|
||||||
# 2) 前端类型(在 frontend/ 下)
|
|
||||||
npm run codegen && git diff --exit-code src/api/schema.d.ts
|
|
||||||
```
|
|
||||||
|
|
||||||
- 判据:`git diff --exit-code openapi/` 有输出 → **必然**还要跑一次 `npm run codegen`。
|
|
||||||
- 反过来也成立:`schema.d.ts` 不要手改,它是生成物。
|
|
||||||
- Reviewer 审"动了路由 / schema / 路由 docstring"类任务时,把**这两个产物是否都已重新生成并入库**当作 acceptance 的一部分。
|
|
||||||
|
|
||||||
### 构建上下文完整性(M1 Dockerfile 教训)
|
|
||||||
|
|
||||||
`docker build` **不在 pytest/ruff 闸门里**——M1 删了 `alembic_location/poo` 后忘了同步 `Dockerfile` 的 `COPY`,单元闸门全绿却把坏掉的镜像构建一路漏到 release tag。所以:
|
|
||||||
|
|
||||||
- 任务**删除 / 移动 / 重命名文件或目录**时,必须 grep 构建清单是否还在引用它们:`Dockerfile`(尤其 `COPY` 源)、`docker/`、`*.ini`、CI workflow、`requirements*.txt` 等。
|
|
||||||
- 已有回归测试 `tests/test_deployment.py::test_dockerfile_copy_sources_exist` 守"Dockerfile `COPY` 源必须存在于构建上下文";新增 / 改动 `COPY` 时确保它仍覆盖得到。
|
|
||||||
- Reviewer 审"删 / 移文件"类任务时,**必须顺带核对构建清单引用**,把它当 acceptance 的一部分。
|
|
||||||
|
|
||||||
## 每轮简报(`review-notes/`)
|
|
||||||
|
|
||||||
每轮工作都要在 `review-notes/` 下产出**中文简报**。该目录**已在 `.gitignore` 忽略**,纯本地、不入库——它是 agent 之间和与人之间的交接载体,不是仓库产物。
|
|
||||||
|
|
||||||
- **实现 / 返工简报**:每轮实现完成后(无论首次实现还是返工),写一份。文件名建议 `<task-id>-impl-<n>.md` / `<task-id>-rework-<n>.md`(如 `M1-T03-impl-1.md`、`M1-T03-rework-1.md`)。至少包含:
|
|
||||||
1. **本轮修改的具体内容**(改了哪些文件、做了什么、为什么)。
|
|
||||||
2. **自动化测试结果**(`pytest` / `ruff` / 前端闸门的实际输出或结论,通过/失败逐项写清)。
|
|
||||||
3. **若需人工 walkthrough**:写明具体步骤(怎么启动、点哪里、预期看到什么);若无需人工验证,明确写"无需人工 walkthrough"。
|
|
||||||
- **review 简报**:每轮 review 后写一份,文件名建议 `<task-id>-review-<n>.md`(如 `M1-T03-review-1.md`)。至少包含:评审结论(`PASS` 或带编号的返工清单)、对照任务卡 `Acceptance criteria` + `Reviewer checklist` 的逐条核对、reviewer 独立重跑校验闸门的结果。
|
|
||||||
|
|
||||||
**用途**:① reviewer 审核时参考对应的实现简报;② implementer 返工时参考对应的 review 简报;③ 人类(用户)通读这些简报确认有无问题。简报之间用文件名里的 `<task-id>` 与轮次 `<n>` 对应起来。
|
|
||||||
|
|
||||||
### Orchestrator 派发契约(让简报真正被读到)
|
|
||||||
|
|
||||||
**关键**:sub-agent 冷启动、不继承主线上下文,**不会因为本文件提到简报就自动去读**对应文件。简报能流转,靠的是 orchestrator(主线)在**每次 spawn 时把路径显式写进 prompt**,而不是被动约定。所以派发时必须做到:
|
|
||||||
|
|
||||||
- **显式告诉它「先读哪个简报」**:
|
|
||||||
- 派 implementer 做**首次实现** → 传任务卡位置(milestone 文档路径 + task id);无前置简报。
|
|
||||||
- 派 implementer 做**返工** → 必须传对应的 `review-notes/<task>-review-<n>.md` 路径,并要求**先读它**再改。
|
|
||||||
- 派 reviewer → 必须传对应的 `review-notes/<task>-impl|rework-<n>.md` 路径 + 任务卡,要求**先读它**再评。
|
|
||||||
- **显式告诉它「本轮结束写哪个简报」**:明确给出输出路径 `review-notes/<task>-<impl|rework|review>-<n>.md` 及上面要求的内容项。
|
|
||||||
- **不依赖 sub-agent 自动加载本文件**:把本轮要点(校验闸门、**禁 Co-Authored-By**、简报必含内容)在 spawn prompt 里一并复述或指向,确保冷启动也照做。
|
|
||||||
- spawn 时用用户指定的模型(Agent 工具 `model` 覆盖)。
|
|
||||||
|
|
||||||
> 一句话:**简报是异步交接的介质,orchestrator 是把它们接起来的线。** 缺了显式传路径这一步,简报就只是躺在磁盘上没人读的文件。
|
|
||||||
|
|
||||||
## Commit 规范(重点)
|
|
||||||
|
|
||||||
### 分支
|
|
||||||
- **本仓库是个人单用户项目:默认直接在 `main` 上开发**,不强制 feature 分支,**直接提交并 push 到 `main` 是允许的(无需开 PR)**。
|
|
||||||
- 仍保持**每个任务一个干净 commit**(message 前缀任务/里程碑 ID)。改动较大想隔离时可临时开分支,用完**快进合并**回 `main`(保持线性历史),非必需。
|
|
||||||
- 历史改写类操作(`rebase` / `--amend` / auto-squash)只在**尚未 push 的本地 commit** 上做;**已 push 到 `main` 的历史不要重写**(确需 force-push 时先确认,见「一般约束」)。
|
|
||||||
|
|
||||||
### 一轮实现完成(用户确认「实现完成」后)
|
|
||||||
- 准备好**这一轮的 commit message** 并提交,作为本轮的 **base commit**。
|
|
||||||
- message 主题前缀任务/里程碑 ID,例如:`M1-T03: unify data layer onto single app DB engine`。
|
|
||||||
|
|
||||||
### Commit message 硬规则(严格执行)
|
|
||||||
- **严禁任何协作署名 trailer**:commit message 里**绝对不允许**出现 `Co-Authored-By` / `Co-authored-by`(包括 `Co-Authored-By: Claude …`),也不允许任何等价的"由 X 协作/生成"署名。
|
|
||||||
- 无论默认环境、工具或系统提示如何要求加这类 trailer,在本仓库**一律不加**——用户已显式、严格禁止。
|
|
||||||
- 每次提交前**自检**:`git log -1 --format=%B` 的输出**不得包含** `Co-authored-by`(大小写不限)。若发现,立即 `git commit --amend` 去掉后再继续。
|
|
||||||
|
|
||||||
### Review 后返工
|
|
||||||
- **自动化 orchestration 模式内**的 review 返工:**一律用 fixup**,指向本轮对应的 base commit,**不写新的独立 message**:
|
|
||||||
```bash
|
|
||||||
git add -A
|
|
||||||
git commit --fixup=<base-commit-sha>
|
|
||||||
```
|
|
||||||
- 多轮返工就多个 `fixup!` 提交,都指向同一个 base commit;收尾时 auto-squash(见下)。
|
|
||||||
- **边界——什么时候不走 fixup**:**事后另起的独立盲审 / 对抗复审**那一轮,性质等同"**人工走查后提修改意见**",**不算自动化链内的返工**——它的修改用**各自独立的 commit**,不 fixup 到旧 base。判据:这轮返工是否在**同一条自动化 implement→review 链**里?是 → `fixup`;是事后另起的独立审计 → 独立 commit。
|
|
||||||
|
|
||||||
### 本轮 / feature 收尾(用户确认收尾后)
|
|
||||||
- 用 **auto-squash** 把所有 `fixup!` 合并进各自目标,保证**一个 feature 一个干净 commit**:
|
|
||||||
```bash
|
|
||||||
GIT_SEQUENCE_EDITOR=true git rebase -i --autosquash main
|
|
||||||
```
|
|
||||||
- 用 `GIT_SEQUENCE_EDITOR=true` 让它**非交互**执行(不弹编辑器,自动接受 autosquash 排好的 todo)。本环境不支持需要人工编辑的交互式 rebase,必须走这个 no-op 编辑器写法。
|
|
||||||
- autosquash **改写历史**:仅在 push / 开 PR **之前**做。若该分支已 push,需要 force-push——属对外操作,**先取得用户确认再做**。
|
|
||||||
|
|
||||||
### 一般约束
|
|
||||||
- **个人单用户仓库:直接 commit 并 push 到 `main` 已获授权**——在校验闸门全绿、一轮工作完成时即可提交 / 推送,无需逐次征求同意。
|
|
||||||
- 仍需**先取得用户确认**的操作:**force-push / 改写已推送历史**,以及**打 tag**(会触发镜像 CI / 对外发布;且打 tag 前须按下方「发版前置走查」真跑一次 `docker build`)。
|
|
||||||
|
|
||||||
## 发版前置走查(打 tag 前必做)
|
|
||||||
|
|
||||||
单元闸门绿 ≠ 真的能跑、能构建、能用。M1 出过"绿了但 docker 构建坏了"的事故,所以**打版本 tag(触发镜像 CI)之前**,除了 `pytest` / `ruff` 全绿,还要:
|
|
||||||
|
|
||||||
- **真起 app**:迁移(`python -m scripts.run_migrations`)→ `uvicorn app.main:app ...`,确认能正常启动、关键路由不 500。
|
|
||||||
- **真跑镜像构建**:本地 `docker build`(多阶段就跑完整条),确认构建通过、`COPY` 源都在。
|
|
||||||
- **关键功能人工瞄一眼**:尤其前端 / 可视化类(M2 的热力图、首页地图)——自动闸门判断不了"渲染对不对、UX 顺不顺",这部分**靠看跑起来的 app,不靠读代码**。
|
|
||||||
- 上述任一不过 → **不打 tag**。tag 一旦 push 会触发 docker 镜像 CI / 对外发布,属对外操作,**先确认**。
|
|
||||||
|
|
||||||
## 数据安全红线(不可违反)
|
|
||||||
|
|
||||||
- 任何脚本 / migration **都不得删除或覆盖用户数据文件**(旧 `.db`、备份、volume)。删除只能是人工、事后、保留归档的独立步骤(见 `docs/design/m1-db-consolidation.md` §6 runbook)。
|
|
||||||
- 涉及历史数据的迁移**先在备份副本上演练**;迁移脚本必须幂等且搬完对账行数。
|
|
||||||
- Review 时只要发现"删文件 / drop 有数据的表 / truncate"出现在自动化任务里,直接判返工。
|
|
||||||
|
|
||||||
## 常用命令
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# 环境
|
|
||||||
python -m venv .venv && source .venv/bin/activate && pip install -r dev-requirements.txt
|
|
||||||
# 迁移(初始化/适配 DB)
|
|
||||||
python -m scripts.run_migrations
|
|
||||||
# 起服务
|
|
||||||
uvicorn app.main:app --reload --host 0.0.0.0 --port 8000
|
|
||||||
# 测试 / lint / OpenAPI 导出
|
|
||||||
pytest
|
|
||||||
ruff check .
|
|
||||||
python scripts/export_openapi.py
|
|
||||||
```
|
|
||||||
@@ -21,6 +21,8 @@
|
|||||||
- **通用电价合同层**:YAML profile 定合同结构(manual 固定/双费率 / tibber 动态电价);`EnergyContract`+`EnergyContractVersion` 存 UI 可填的数值,改价加新版本旧版本保留;price strategy 按 kind 出价
|
- **通用电价合同层**:YAML profile 定合同结构(manual 固定/双费率 / tibber 动态电价);`EnergyContract`+`EnergyContractVersion` 存 UI 可填的数值,改价加新版本旧版本保留;price strategy 按 kind 出价
|
||||||
- **实时买卖电费计算**:每 15 分钟按寄存器差值(`_1`=dal/低、`_2`=normal/高)× 买/卖价算计量电费,快照不可变;日/月/年汇总加固定费减 heffingskorting
|
- **实时买卖电费计算**:每 15 分钟按寄存器差值(`_1`=dal/低、`_2`=normal/高)× 买/卖价算计量电费,快照不可变;日/月/年汇总加固定费减 heffingskorting
|
||||||
- **反哺 Home Assistant Energy**:当前买/卖价 + 累计买电支出/卖电收入(`total_increasing`)发成 HA 实体,可直接挂 HA Energy 仪表盘
|
- **反哺 Home Assistant Energy**:当前买/卖价 + 累计买电支出/卖电收入(`total_increasing`)发成 HA 实体,可直接挂 HA Energy 仪表盘
|
||||||
|
- **多数据源 Meter 与 WarmteLink**:DSMR MQTT 与只读 WarmteLink P1 serial source 统一为 Source → Channel → Binding → Meter;WarmteLink 提供 heating `GJ` 与 hot-water `m³` 的 Decimal history、质量与重连
|
||||||
|
- **热力合同与成本**:electricity / thermal scope 可各有一个 active 合同;热力按 15 分钟账本计算 variable、fixed 与 all-in 成本,并可按需暴露给 HA
|
||||||
- pytest 测试与 OpenAPI 导出脚本
|
- pytest 测试与 OpenAPI 导出脚本
|
||||||
- Docker / Compose 部署入口
|
- Docker / Compose 部署入口
|
||||||
|
|
||||||
@@ -44,6 +46,8 @@
|
|||||||
- 电价合同(`energy_contract` 表)与版本(`energy_contract_version` 表,values JSON)
|
- 电价合同(`energy_contract` 表)与版本(`energy_contract_version` 表,values JSON)
|
||||||
- Tibber 15 分钟电价缓存(`tibber_price` 表,不可变)
|
- Tibber 15 分钟电价缓存(`tibber_price` 表,不可变)
|
||||||
- 每 15 分钟计量电费(`energy_cost_period` 表,快照价,不可变)
|
- 每 15 分钟计量电费(`energy_cost_period` 表,快照价,不可变)
|
||||||
|
- meter source、channel 与 binding(`meter_source`、`meter_source_channel`、`meter_source_binding`)
|
||||||
|
- WarmteLink scalar 历史(`warmtelink_reading`)与热力 15 分钟成本账本(`meter_cost_period`)
|
||||||
|
|
||||||
配置层只保留一个数据库环境变量:
|
配置层只保留一个数据库环境变量:
|
||||||
|
|
||||||
@@ -55,7 +59,7 @@
|
|||||||
python -m scripts.run_migrations
|
python -m scripts.run_migrations
|
||||||
```
|
```
|
||||||
|
|
||||||
该命令会通过 Alembic 将 `app.db` 初始化或升级到最新 head(含全部表,包括 M5 新增的 `modbus_device`、`modbus_reading`、`exposed_entity_toggle`,以及 M6 新增的 `dsmr_reading`、`energy_contract`、`energy_contract_version`、`tibber_price`、`energy_cost_period`)。
|
该命令会通过 Alembic 将 `app.db` 初始化或升级到最新 head(包括 Modbus、DSMR、Source/Channel/Binding、WarmteLink、electricity/thermal 合同与成本账本)。
|
||||||
|
|
||||||
## 当前目录
|
## 当前目录
|
||||||
|
|
||||||
@@ -63,7 +67,7 @@ python -m scripts.run_migrations
|
|||||||
|
|
||||||
- `app/`: FastAPI 应用代码(包含 JSON API、业务服务、数据模型)
|
- `app/`: FastAPI 应用代码(包含 JSON API、业务服务、数据模型)
|
||||||
- `frontend/`: React SPA 前端(Vite + React + TypeScript + Mantine)
|
- `frontend/`: React SPA 前端(Vite + React + TypeScript + Mantine)
|
||||||
- `alembic_app/`: App DB 的 Alembic migration 环境(管理所有表,含 M5 新增的 `modbus_device`、`modbus_reading`、`exposed_entity_toggle`,以及 M6 新增的 `dsmr_reading`、`energy_contract`、`energy_contract_version`、`tibber_price`、`energy_cost_period`)
|
- `alembic_app/`: App DB 的唯一 Alembic migration 环境(管理所有 app 表,包括 Modbus、DSMR、Meter source、WarmteLink、合同与成本账本)
|
||||||
- `tests/`: pytest 测试
|
- `tests/`: pytest 测试
|
||||||
- `docs/`: 当前系统说明文档
|
- `docs/`: 当前系统说明文档
|
||||||
- `scripts/`: 辅助脚本,例如 OpenAPI 导出
|
- `scripts/`: 辅助脚本,例如 OpenAPI 导出
|
||||||
@@ -551,6 +555,20 @@ python scripts/export_openapi.py
|
|||||||
- `docker-compose.dev.yml`:本地开发显式叠加层——追加 `build: .`、独立 project /
|
- `docker-compose.dev.yml`:本地开发显式叠加层——追加 `build: .`、独立 project /
|
||||||
容器名(`-dev` 后缀)、暴露 8001,并把 DB 指向挂载的 `./data` 副本,可与生产栈在同一台机器上并存
|
容器名(`-dev` 后缀)、暴露 8001,并把 DB 指向挂载的 `./data` 副本,可与生产栈在同一台机器上并存
|
||||||
|
|
||||||
|
WarmteLink serial access is configured directly by both Compose combinations. Before starting either
|
||||||
|
one, set these host-specific values in your uncommitted local `.env` (use a stable `/dev/serial/by-id/...`
|
||||||
|
path, never a transient `/dev/ttyUSB*` name):
|
||||||
|
|
||||||
|
```dotenv
|
||||||
|
WARMTELINK_DEVICE_PATH=/dev/serial/by-id/<stable-by-id-name>
|
||||||
|
WARMTELINK_SERIAL_GID=<host-serial-gid>
|
||||||
|
```
|
||||||
|
|
||||||
|
Only `app` receives the device as `/dev/warmtelink:rw` and the serial group; `migration` does not.
|
||||||
|
The app remains non-root, non-privileged, and has no added capabilities. One physical serial port may
|
||||||
|
have only one owner: stop the app before running the Pre-M8 P1 probe. Never remove `./data`, databases,
|
||||||
|
or volumes while changing this configuration.
|
||||||
|
|
||||||
本地开发启动方式(显式叠加 dev 层):
|
本地开发启动方式(显式叠加 dev 层):
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
|||||||
@@ -20,6 +20,7 @@ from app.models.energy import ( # noqa: F401
|
|||||||
TibberPrice,
|
TibberPrice,
|
||||||
EnergyCostPeriod,
|
EnergyCostPeriod,
|
||||||
)
|
)
|
||||||
|
from app.models.meter_source import MeterSource, MeterSourceBinding, MeterSourceChannel # noqa: F401
|
||||||
|
|
||||||
config = context.config
|
config = context.config
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,107 @@
|
|||||||
|
"""add protocol-agnostic meter source, channel, and binding tables
|
||||||
|
|
||||||
|
Revision ID: 20260822_15_meter_sources
|
||||||
|
Revises: 20260625_14_meter_uuid
|
||||||
|
Create Date: 2026-08-22 00:00:00.000000
|
||||||
|
|
||||||
|
This revision is additive on upgrade. It deliberately does not backfill
|
||||||
|
existing DSMR data; that adoption is a later, separately audited migration.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from typing import Sequence, Union
|
||||||
|
|
||||||
|
import sqlalchemy as sa
|
||||||
|
from alembic import op
|
||||||
|
|
||||||
|
|
||||||
|
revision: str = "20260822_15_meter_sources"
|
||||||
|
down_revision: Union[str, None] = "20260625_14_meter_uuid"
|
||||||
|
branch_labels: Union[str, Sequence[str], None] = None
|
||||||
|
depends_on: Union[str, Sequence[str], None] = None
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
op.create_table(
|
||||||
|
"meter_source",
|
||||||
|
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
|
||||||
|
sa.Column("uuid", sa.String(length=36), nullable=False),
|
||||||
|
sa.Column("name", sa.String(length=255), nullable=False),
|
||||||
|
sa.Column("kind", sa.String(length=64), nullable=False),
|
||||||
|
sa.Column("enabled", sa.Boolean(), nullable=False),
|
||||||
|
sa.Column("config", sa.JSON(), nullable=False),
|
||||||
|
sa.Column("status", sa.String(length=32), nullable=False),
|
||||||
|
sa.Column("last_seen_at", sa.DateTime(timezone=True), nullable=True),
|
||||||
|
sa.Column("last_error", sa.String(length=1024), nullable=True),
|
||||||
|
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.PrimaryKeyConstraint("id"),
|
||||||
|
sa.UniqueConstraint("uuid", name="uq_meter_source_uuid"),
|
||||||
|
)
|
||||||
|
op.create_index("ix_meter_source_kind_enabled", "meter_source", ["kind", "enabled"])
|
||||||
|
|
||||||
|
op.create_table(
|
||||||
|
"meter_source_channel",
|
||||||
|
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
|
||||||
|
sa.Column("uuid", sa.String(length=36), nullable=False),
|
||||||
|
sa.Column("source_id", sa.Integer(), nullable=False),
|
||||||
|
sa.Column("channel_key", sa.String(length=128), nullable=False),
|
||||||
|
sa.Column("label", sa.String(length=255), nullable=False),
|
||||||
|
sa.Column("suggested_commodity", sa.String(length=32), nullable=True),
|
||||||
|
sa.Column("unit", sa.String(length=32), nullable=False),
|
||||||
|
sa.Column("device_type", sa.String(length=64), nullable=True),
|
||||||
|
sa.Column("fingerprint", sa.String(length=64), nullable=True),
|
||||||
|
sa.Column("latest_value", sa.Numeric(precision=20, scale=6), nullable=True),
|
||||||
|
sa.Column("latest_at", sa.DateTime(timezone=True), nullable=True),
|
||||||
|
sa.Column("latest_quality", sa.String(length=32), nullable=True),
|
||||||
|
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.ForeignKeyConstraint(["source_id"], ["meter_source.id"], ondelete="RESTRICT"),
|
||||||
|
sa.PrimaryKeyConstraint("id"),
|
||||||
|
sa.UniqueConstraint("uuid", name="uq_meter_source_channel_uuid"),
|
||||||
|
sa.UniqueConstraint("source_id", "channel_key", name="uq_meter_source_channel_source_key"),
|
||||||
|
)
|
||||||
|
op.create_index("ix_meter_source_channel_source_id", "meter_source_channel", ["source_id"])
|
||||||
|
|
||||||
|
op.create_table(
|
||||||
|
"meter_source_binding",
|
||||||
|
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
|
||||||
|
sa.Column("uuid", sa.String(length=36), nullable=False),
|
||||||
|
sa.Column("meter_id", sa.Integer(), nullable=False),
|
||||||
|
sa.Column("channel_id", sa.Integer(), nullable=False),
|
||||||
|
sa.Column("started_at", sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.Column("ended_at", sa.DateTime(timezone=True), nullable=True),
|
||||||
|
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.ForeignKeyConstraint(["meter_id"], ["meter.id"], ondelete="RESTRICT"),
|
||||||
|
sa.ForeignKeyConstraint(
|
||||||
|
["channel_id"], ["meter_source_channel.id"], ondelete="RESTRICT"
|
||||||
|
),
|
||||||
|
sa.PrimaryKeyConstraint("id"),
|
||||||
|
sa.UniqueConstraint("uuid", name="uq_meter_source_binding_uuid"),
|
||||||
|
)
|
||||||
|
op.create_index("ix_meter_source_binding_meter_id", "meter_source_binding", ["meter_id"])
|
||||||
|
op.create_index("ix_meter_source_binding_channel_id", "meter_source_binding", ["channel_id"])
|
||||||
|
|
||||||
|
with op.batch_alter_table("energy_cost_period", schema=None) as batch_op:
|
||||||
|
batch_op.add_column(sa.Column("source_binding_id", sa.Integer(), nullable=True))
|
||||||
|
batch_op.create_foreign_key(
|
||||||
|
"fk_energy_cost_period_source_binding_id",
|
||||||
|
"meter_source_binding",
|
||||||
|
["source_binding_id"],
|
||||||
|
["id"],
|
||||||
|
ondelete="RESTRICT",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
with op.batch_alter_table("energy_cost_period", schema=None) as batch_op:
|
||||||
|
batch_op.drop_constraint("fk_energy_cost_period_source_binding_id", type_="foreignkey")
|
||||||
|
batch_op.drop_column("source_binding_id")
|
||||||
|
|
||||||
|
op.drop_index("ix_meter_source_binding_channel_id", table_name="meter_source_binding")
|
||||||
|
op.drop_index("ix_meter_source_binding_meter_id", table_name="meter_source_binding")
|
||||||
|
op.drop_table("meter_source_binding")
|
||||||
|
op.drop_index("ix_meter_source_channel_source_id", table_name="meter_source_channel")
|
||||||
|
op.drop_table("meter_source_channel")
|
||||||
|
op.drop_index("ix_meter_source_kind_enabled", table_name="meter_source")
|
||||||
|
op.drop_table("meter_source")
|
||||||
@@ -0,0 +1,269 @@
|
|||||||
|
"""adopt historical DSMR rows into the source and binding model
|
||||||
|
|
||||||
|
Revision ID: 20260822_16_dsmr_source_adoption
|
||||||
|
Revises: 20260822_15_meter_sources
|
||||||
|
Create Date: 2026-08-22 00:00:00.000000
|
||||||
|
|
||||||
|
The upgrade is deliberately data-preserving: it creates one migration-owned
|
||||||
|
DSMR source/channel, moves the telegram identifier to ``telegram_id``, and
|
||||||
|
audits every reading and cost row before committing. Old ``app_config`` rows,
|
||||||
|
payload JSON, and cost snapshots are never deleted or rewritten.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import uuid
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from typing import Sequence, Union
|
||||||
|
|
||||||
|
import sqlalchemy as sa
|
||||||
|
from alembic import op
|
||||||
|
|
||||||
|
|
||||||
|
revision: str = "20260822_16_dsmr_source_adoption"
|
||||||
|
down_revision: Union[str, None] = "20260822_15_meter_sources"
|
||||||
|
branch_labels: Union[str, Sequence[str], None] = None
|
||||||
|
depends_on: Union[str, Sequence[str], None] = None
|
||||||
|
|
||||||
|
|
||||||
|
def _as_bool(value: str | None) -> bool:
|
||||||
|
return value is not None and value.strip().lower() in {"1", "true", "yes", "on"}
|
||||||
|
|
||||||
|
|
||||||
|
def _iso_now() -> str:
|
||||||
|
return datetime.now(tz=timezone.utc).replace(tzinfo=None).isoformat(sep=" ")
|
||||||
|
|
||||||
|
|
||||||
|
def _config(connection: sa.Connection) -> dict[str, object]:
|
||||||
|
rows = connection.execute(sa.text("SELECT key, value FROM app_config")).all()
|
||||||
|
values = {str(key): str(value) for key, value in rows}
|
||||||
|
# An unconfigured historical DSMR installation needs a disabled identity,
|
||||||
|
# not guessed connection details. Preserve every legacy value we model
|
||||||
|
# when any legacy DSMR/MQTT configuration was explicitly present.
|
||||||
|
legacy_keys = {
|
||||||
|
"MQTT_BROKER_HOST", "MQTT_BROKER_PORT", "MQTT_USERNAME", "MQTT_PASSWORD",
|
||||||
|
"MQTT_TLS_ENABLED", "DSMR_MQTT_TOPIC", "DSMR_TARIFF_TOPIC", "DSMR_SAMPLE_INTERVAL_S",
|
||||||
|
}
|
||||||
|
if not legacy_keys & values.keys():
|
||||||
|
return {}
|
||||||
|
return {
|
||||||
|
"broker_host": values.get("MQTT_BROKER_HOST", ""),
|
||||||
|
"broker_port": int(values.get("MQTT_BROKER_PORT", "1883")),
|
||||||
|
"username": values.get("MQTT_USERNAME", ""),
|
||||||
|
"password": values.get("MQTT_PASSWORD", ""),
|
||||||
|
"tls_enabled": _as_bool(values.get("MQTT_TLS_ENABLED")),
|
||||||
|
"topic": values.get("DSMR_MQTT_TOPIC", "dsmr/json"),
|
||||||
|
"tariff_topic": values.get("DSMR_TARIFF_TOPIC", "dsmr/meter-stats/electricity_tariff"),
|
||||||
|
"sample_interval_s": int(values.get("DSMR_SAMPLE_INTERVAL_S", "10")),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _count(connection: sa.Connection, table: str) -> int:
|
||||||
|
return int(connection.execute(sa.text(f"SELECT COUNT(*) FROM {table}")).scalar_one())
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
connection = op.get_bind()
|
||||||
|
readings_before = _count(connection, "dsmr_reading")
|
||||||
|
costs_before = _count(connection, "energy_cost_period")
|
||||||
|
sources_before = _count(connection, "meter_source")
|
||||||
|
channels_before = _count(connection, "meter_source_channel")
|
||||||
|
bindings_before = _count(connection, "meter_source_binding")
|
||||||
|
now = _iso_now()
|
||||||
|
|
||||||
|
# A source exists even without historical configuration/readings. It stays
|
||||||
|
# disabled unless the old explicit DSMR switch was enabled, so no broker or
|
||||||
|
# topic is guessed at runtime.
|
||||||
|
config = _config(connection)
|
||||||
|
source_result = connection.execute(
|
||||||
|
sa.text(
|
||||||
|
"INSERT INTO meter_source "
|
||||||
|
"(uuid, name, kind, enabled, config, status, last_seen_at, last_error, created_at, updated_at) "
|
||||||
|
"VALUES (:uuid, :name, 'dsmr_mqtt', :enabled, :config, 'unknown', NULL, NULL, :now, :now)"
|
||||||
|
),
|
||||||
|
{
|
||||||
|
"uuid": str(uuid.uuid4()),
|
||||||
|
"name": "Migrated DSMR source",
|
||||||
|
"enabled": _as_bool(
|
||||||
|
connection.execute(
|
||||||
|
sa.text("SELECT value FROM app_config WHERE key = 'DSMR_INGEST_ENABLED'")
|
||||||
|
).scalar_one_or_none()
|
||||||
|
),
|
||||||
|
"config": __import__("json").dumps(config),
|
||||||
|
"now": now,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
source_id = source_result.lastrowid
|
||||||
|
if source_id is None:
|
||||||
|
raise RuntimeError("DSMR source adoption failed to create a source")
|
||||||
|
channel_result = connection.execute(
|
||||||
|
sa.text(
|
||||||
|
"INSERT INTO meter_source_channel "
|
||||||
|
"(uuid, source_id, channel_key, label, suggested_commodity, unit, device_type, fingerprint, "
|
||||||
|
"latest_value, latest_at, latest_quality, created_at, updated_at) "
|
||||||
|
"VALUES (:uuid, :source_id, 'electricity-total', 'DSMR electricity total', 'electricity', "
|
||||||
|
"'kWh', NULL, NULL, NULL, NULL, NULL, :now, :now)"
|
||||||
|
),
|
||||||
|
{"uuid": str(uuid.uuid4()), "source_id": source_id, "now": now},
|
||||||
|
)
|
||||||
|
channel_id = channel_result.lastrowid
|
||||||
|
if channel_id is None:
|
||||||
|
raise RuntimeError("DSMR source adoption failed to create an electricity channel")
|
||||||
|
if _count(connection, "meter_source") != sources_before + 1:
|
||||||
|
raise RuntimeError("DSMR source adoption source row-count audit failed")
|
||||||
|
if _count(connection, "meter_source_channel") != channels_before + 1:
|
||||||
|
raise RuntimeError("DSMR source adoption channel row-count audit failed")
|
||||||
|
|
||||||
|
# Rename/add while nullable, back-fill all rows, then make the FK non-null
|
||||||
|
# and replace the legacy timestamp-only uniqueness in a SQLite batch rebuild.
|
||||||
|
with op.batch_alter_table("dsmr_reading", schema=None) as batch_op:
|
||||||
|
batch_op.alter_column("source_id", new_column_name="telegram_id")
|
||||||
|
batch_op.add_column(sa.Column("meter_source_id", sa.Integer(), nullable=True))
|
||||||
|
connection.execute(
|
||||||
|
sa.text("UPDATE dsmr_reading SET meter_source_id = :source_id WHERE meter_source_id IS NULL"),
|
||||||
|
{"source_id": source_id},
|
||||||
|
)
|
||||||
|
with op.batch_alter_table("dsmr_reading", schema=None) as batch_op:
|
||||||
|
batch_op.drop_constraint("uq_dsmr_reading_recorded_at", type_="unique")
|
||||||
|
batch_op.alter_column("meter_source_id", existing_type=sa.Integer(), nullable=False)
|
||||||
|
batch_op.create_foreign_key(
|
||||||
|
"fk_dsmr_reading_meter_source_id", "meter_source", ["meter_source_id"], ["id"],
|
||||||
|
ondelete="RESTRICT",
|
||||||
|
)
|
||||||
|
batch_op.create_unique_constraint(
|
||||||
|
"uq_dsmr_reading_source_recorded_at", ["meter_source_id", "recorded_at"]
|
||||||
|
)
|
||||||
|
batch_op.create_index("ix_dsmr_reading_meter_source_id", ["meter_source_id"])
|
||||||
|
adopted_readings = int(
|
||||||
|
connection.execute(
|
||||||
|
sa.text("SELECT COUNT(*) FROM dsmr_reading WHERE meter_source_id = :source_id"),
|
||||||
|
{"source_id": source_id},
|
||||||
|
).scalar_one()
|
||||||
|
)
|
||||||
|
if adopted_readings != readings_before:
|
||||||
|
raise RuntimeError("DSMR source adoption reading source audit failed")
|
||||||
|
|
||||||
|
# Bind each electricity meter only where it overlaps the actual DSMR data.
|
||||||
|
data_window = connection.execute(
|
||||||
|
sa.text("SELECT MIN(recorded_at), MAX(recorded_at) FROM dsmr_reading")
|
||||||
|
).one()
|
||||||
|
expected_binding_count = 0
|
||||||
|
if data_window[0] is not None:
|
||||||
|
meters = connection.execute(
|
||||||
|
sa.text(
|
||||||
|
"SELECT id, started_at, ended_at FROM meter WHERE commodity = 'electricity' "
|
||||||
|
"ORDER BY started_at, id"
|
||||||
|
)
|
||||||
|
).all()
|
||||||
|
for meter_id, started_at, ended_at in meters:
|
||||||
|
# Intersect [meter start, meter end) with the inclusive historical
|
||||||
|
# samples. A closed boundary at the final sample remains valid for
|
||||||
|
# the preceding interval; an empty intersection gets no fake binding.
|
||||||
|
if started_at > data_window[1] or (ended_at is not None and ended_at <= data_window[0]):
|
||||||
|
continue
|
||||||
|
expected_binding_count += 1
|
||||||
|
binding_start = max(started_at, data_window[0])
|
||||||
|
binding_end = ended_at
|
||||||
|
connection.execute(
|
||||||
|
sa.text(
|
||||||
|
"INSERT INTO meter_source_binding "
|
||||||
|
"(uuid, meter_id, channel_id, started_at, ended_at, created_at, updated_at) "
|
||||||
|
"VALUES (:uuid, :meter_id, :channel_id, :started_at, :ended_at, :now, :now)"
|
||||||
|
),
|
||||||
|
{
|
||||||
|
"uuid": str(uuid.uuid4()), "meter_id": meter_id, "channel_id": channel_id,
|
||||||
|
"started_at": binding_start, "ended_at": binding_end, "now": now,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
# A cost period may be linked only if exactly one binding covers both its
|
||||||
|
# start and end. Historical boundary/unknown rows remain auditable but are
|
||||||
|
# explicitly degraded instead of being silently attributed to a current meter.
|
||||||
|
periods = connection.execute(
|
||||||
|
sa.text("SELECT id, meter_id, period_start, degraded FROM energy_cost_period")
|
||||||
|
).all()
|
||||||
|
resolvable_normal_periods: dict[int, int] = {}
|
||||||
|
unresolved_period_ids: set[int] = set()
|
||||||
|
for period_id, meter_id, period_start, degraded_before in periods:
|
||||||
|
candidates = []
|
||||||
|
if meter_id is not None:
|
||||||
|
candidates = connection.execute(
|
||||||
|
sa.text(
|
||||||
|
"SELECT id FROM meter_source_binding "
|
||||||
|
"WHERE meter_id = :meter_id AND started_at <= :start "
|
||||||
|
"AND (ended_at IS NULL OR julianday(ended_at) > julianday(:start, '+15 minutes'))"
|
||||||
|
),
|
||||||
|
{"meter_id": meter_id, "start": period_start},
|
||||||
|
).all()
|
||||||
|
if len(candidates) == 1:
|
||||||
|
if not degraded_before:
|
||||||
|
resolvable_normal_periods[period_id] = candidates[0][0]
|
||||||
|
connection.execute(
|
||||||
|
sa.text("UPDATE energy_cost_period SET source_binding_id = :binding_id WHERE id = :id"),
|
||||||
|
{"binding_id": candidates[0][0], "id": period_id},
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
unresolved_period_ids.add(period_id)
|
||||||
|
connection.execute(
|
||||||
|
sa.text(
|
||||||
|
"UPDATE energy_cost_period SET degraded = 1, source_binding_id = NULL WHERE id = :id"
|
||||||
|
),
|
||||||
|
{"id": period_id},
|
||||||
|
)
|
||||||
|
|
||||||
|
readings_after = _count(connection, "dsmr_reading")
|
||||||
|
costs_after = _count(connection, "energy_cost_period")
|
||||||
|
if readings_after != readings_before or costs_after != costs_before:
|
||||||
|
raise RuntimeError("DSMR source adoption row-count audit failed")
|
||||||
|
if _count(connection, "meter_source_binding") != bindings_before + expected_binding_count:
|
||||||
|
raise RuntimeError("DSMR source adoption binding row-count audit failed")
|
||||||
|
if int(
|
||||||
|
connection.execute(
|
||||||
|
sa.text("SELECT COUNT(*) FROM meter_source_binding WHERE channel_id = :channel_id"),
|
||||||
|
{"channel_id": channel_id},
|
||||||
|
).scalar_one()
|
||||||
|
) != expected_binding_count:
|
||||||
|
raise RuntimeError("DSMR source adoption binding channel audit failed")
|
||||||
|
for period_id, binding_id in resolvable_normal_periods.items():
|
||||||
|
bound, degraded = connection.execute(
|
||||||
|
sa.text("SELECT source_binding_id, degraded FROM energy_cost_period WHERE id = :id"),
|
||||||
|
{"id": period_id},
|
||||||
|
).one()
|
||||||
|
if bound != binding_id or degraded:
|
||||||
|
raise RuntimeError("DSMR source adoption resolvable cost audit failed")
|
||||||
|
if unresolved_period_ids:
|
||||||
|
unresolved_count = int(
|
||||||
|
connection.execute(
|
||||||
|
sa.text(
|
||||||
|
"SELECT COUNT(*) FROM energy_cost_period "
|
||||||
|
"WHERE id IN :period_ids AND (degraded != 1 OR source_binding_id IS NOT NULL)"
|
||||||
|
).bindparams(sa.bindparam("period_ids", expanding=True)),
|
||||||
|
{"period_ids": list(unresolved_period_ids)},
|
||||||
|
).scalar_one()
|
||||||
|
)
|
||||||
|
if unresolved_count:
|
||||||
|
raise RuntimeError("DSMR source adoption unresolved cost audit failed")
|
||||||
|
orphan_rows = connection.execute(sa.text("PRAGMA foreign_key_check")).all()
|
||||||
|
if orphan_rows:
|
||||||
|
raise RuntimeError("DSMR source adoption foreign-key audit failed")
|
||||||
|
normal_unbound = int(
|
||||||
|
connection.execute(
|
||||||
|
sa.text("SELECT COUNT(*) FROM energy_cost_period WHERE degraded = 0 AND source_binding_id IS NULL")
|
||||||
|
).scalar_one()
|
||||||
|
)
|
||||||
|
if normal_unbound:
|
||||||
|
raise RuntimeError(f"DSMR source adoption left {normal_unbound} normal cost period(s) unbound")
|
||||||
|
if _count(connection, "meter_source") < 1:
|
||||||
|
raise RuntimeError("DSMR source adoption source audit failed")
|
||||||
|
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
# Schema-only downgrade for isolated test databases. It intentionally does
|
||||||
|
# not delete migration-created source/channel/binding rows.
|
||||||
|
with op.batch_alter_table("dsmr_reading", schema=None) as batch_op:
|
||||||
|
batch_op.drop_index("ix_dsmr_reading_meter_source_id")
|
||||||
|
batch_op.drop_constraint("uq_dsmr_reading_source_recorded_at", type_="unique")
|
||||||
|
batch_op.drop_constraint("fk_dsmr_reading_meter_source_id", type_="foreignkey")
|
||||||
|
batch_op.drop_column("meter_source_id")
|
||||||
|
batch_op.alter_column("telegram_id", new_column_name="source_id")
|
||||||
|
batch_op.create_unique_constraint("uq_dsmr_reading_recorded_at", ["recorded_at"])
|
||||||
@@ -0,0 +1,52 @@
|
|||||||
|
"""add normalized WarmteLink scalar reading history
|
||||||
|
|
||||||
|
Revision ID: 20260822_17_warmtelink_readings
|
||||||
|
Revises: 20260822_16_dsmr_source_adoption
|
||||||
|
Create Date: 2026-08-22 00:00:00.000000
|
||||||
|
|
||||||
|
The upgrade is additive: existing business rows are neither changed nor
|
||||||
|
removed. The downgrade is schema-only and is exercised only on isolated test
|
||||||
|
databases.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from typing import Sequence, Union
|
||||||
|
|
||||||
|
import sqlalchemy as sa
|
||||||
|
from alembic import op
|
||||||
|
|
||||||
|
|
||||||
|
revision: str = "20260822_17_warmtelink_readings"
|
||||||
|
down_revision: Union[str, None] = "20260822_16_dsmr_source_adoption"
|
||||||
|
branch_labels: Union[str, Sequence[str], None] = None
|
||||||
|
depends_on: Union[str, Sequence[str], None] = None
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
op.create_table(
|
||||||
|
"warmtelink_reading",
|
||||||
|
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
|
||||||
|
sa.Column("channel_id", sa.Integer(), nullable=False),
|
||||||
|
sa.Column("recorded_at", sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.Column("received_at", sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.Column("value", sa.Numeric(precision=15, scale=3), nullable=False),
|
||||||
|
sa.Column("unit", sa.String(length=32), nullable=False),
|
||||||
|
sa.Column("quality", sa.String(length=32), nullable=False),
|
||||||
|
sa.Column("equipment_fingerprint", sa.String(length=64), nullable=False),
|
||||||
|
sa.CheckConstraint(
|
||||||
|
"quality IN ('valid', 'invalid', 'unverifiable')",
|
||||||
|
name="ck_warmtelink_reading_quality",
|
||||||
|
),
|
||||||
|
sa.ForeignKeyConstraint(
|
||||||
|
["channel_id"], ["meter_source_channel.id"], ondelete="RESTRICT"
|
||||||
|
),
|
||||||
|
sa.PrimaryKeyConstraint("id"),
|
||||||
|
sa.UniqueConstraint(
|
||||||
|
"channel_id", "recorded_at", name="uq_warmtelink_reading_channel_recorded_at"
|
||||||
|
),
|
||||||
|
)
|
||||||
|
op.create_index("ix_warmtelink_reading_recorded_at", "warmtelink_reading", ["recorded_at"])
|
||||||
|
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
op.drop_index("ix_warmtelink_reading_recorded_at", table_name="warmtelink_reading")
|
||||||
|
op.drop_table("warmtelink_reading")
|
||||||
@@ -0,0 +1,107 @@
|
|||||||
|
"""add a billing scope to energy contracts
|
||||||
|
|
||||||
|
Revision ID: 20260822_18_contract_scopes
|
||||||
|
Revises: 20260822_17_warmtelink_readings
|
||||||
|
Create Date: 2026-08-22 00:00:00.000000
|
||||||
|
|
||||||
|
The upgrade preserves every existing contract, version and cost row. Existing
|
||||||
|
contracts predate scopes and therefore deterministically belong to electricity.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from typing import Sequence, Union
|
||||||
|
|
||||||
|
import sqlalchemy as sa
|
||||||
|
from alembic import op
|
||||||
|
|
||||||
|
|
||||||
|
revision: str = "20260822_18_contract_scopes"
|
||||||
|
down_revision: Union[str, None] = "20260822_17_warmtelink_readings"
|
||||||
|
branch_labels: Union[str, Sequence[str], None] = None
|
||||||
|
depends_on: Union[str, Sequence[str], None] = None
|
||||||
|
|
||||||
|
|
||||||
|
def _count(connection: sa.Connection, table: str) -> int:
|
||||||
|
return int(connection.execute(sa.text(f"SELECT COUNT(*) FROM {table}")).scalar_one())
|
||||||
|
|
||||||
|
|
||||||
|
def _orphan_count(connection: sa.Connection) -> int:
|
||||||
|
version_orphans = connection.execute(
|
||||||
|
sa.text(
|
||||||
|
"SELECT COUNT(*) FROM energy_contract_version v "
|
||||||
|
"LEFT JOIN energy_contract c ON c.id = v.contract_id WHERE c.id IS NULL"
|
||||||
|
)
|
||||||
|
).scalar_one()
|
||||||
|
cost_orphans = connection.execute(
|
||||||
|
sa.text(
|
||||||
|
"SELECT COUNT(*) FROM energy_cost_period p "
|
||||||
|
"LEFT JOIN energy_contract_version v ON v.id = p.contract_version_id "
|
||||||
|
"WHERE p.contract_version_id IS NOT NULL AND v.id IS NULL"
|
||||||
|
)
|
||||||
|
).scalar_one()
|
||||||
|
return int(version_orphans) + int(cost_orphans)
|
||||||
|
|
||||||
|
|
||||||
|
def _audit_scope_upgrade(connection: sa.Connection, before: dict[str, int], orphan_before: int) -> None:
|
||||||
|
after = {table: _count(connection, table) for table in before}
|
||||||
|
if after != before:
|
||||||
|
raise RuntimeError("contract scope migration row-count audit failed")
|
||||||
|
if _orphan_count(connection) != orphan_before:
|
||||||
|
raise RuntimeError("contract scope migration FK audit failed")
|
||||||
|
invalid_scope_count = connection.execute(
|
||||||
|
sa.text("SELECT COUNT(*) FROM energy_contract WHERE scope IS NULL OR scope != 'electricity'")
|
||||||
|
).scalar_one()
|
||||||
|
if invalid_scope_count:
|
||||||
|
raise RuntimeError("contract scope migration backfill audit failed")
|
||||||
|
|
||||||
|
# Kept on Alembic's Config attributes rather than an environment switch so
|
||||||
|
# isolated migration tests can deterministically exercise the rollback
|
||||||
|
# boundary without changing production behavior.
|
||||||
|
failure_injector = op.get_context().config.attributes.get("m8_t12_post_ddl_audit_failure")
|
||||||
|
if callable(failure_injector):
|
||||||
|
failure_injector()
|
||||||
|
|
||||||
|
|
||||||
|
def _apply_scope_schema() -> None:
|
||||||
|
# SQLite batch mode reconstructs the table. The server default gives every
|
||||||
|
# historical row its deterministic value during reconstruction.
|
||||||
|
with op.batch_alter_table("energy_contract", schema=None) as batch_op:
|
||||||
|
batch_op.add_column(
|
||||||
|
sa.Column("scope", sa.String(length=32), nullable=False, server_default="electricity")
|
||||||
|
)
|
||||||
|
batch_op.create_index("ix_energy_contract_scope", ["scope"])
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
connection = op.get_bind()
|
||||||
|
before = {
|
||||||
|
table: _count(connection, table)
|
||||||
|
for table in ("energy_contract", "energy_contract_version", "energy_cost_period")
|
||||||
|
}
|
||||||
|
orphan_before = _orphan_count(connection)
|
||||||
|
|
||||||
|
if connection.dialect.name != "sqlite":
|
||||||
|
_apply_scope_schema()
|
||||||
|
_audit_scope_upgrade(connection, before, orphan_before)
|
||||||
|
return
|
||||||
|
|
||||||
|
# Alembic marks SQLite batch DDL as non-transactional. SQLite itself can
|
||||||
|
# nevertheless atomically roll back CREATE/COPY/DROP/RENAME when an
|
||||||
|
# explicit transaction owns the complete batch operation. Keep the audit
|
||||||
|
# inside that boundary so a failed audit cannot strand a revision-17 DB
|
||||||
|
# with a revision-18 table shape.
|
||||||
|
connection.exec_driver_sql("BEGIN IMMEDIATE")
|
||||||
|
try:
|
||||||
|
_apply_scope_schema()
|
||||||
|
_audit_scope_upgrade(connection, before, orphan_before)
|
||||||
|
except BaseException:
|
||||||
|
connection.exec_driver_sql("ROLLBACK")
|
||||||
|
raise
|
||||||
|
else:
|
||||||
|
connection.exec_driver_sql("COMMIT")
|
||||||
|
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
# Schema reversibility is only exercised against isolated temporary test DBs.
|
||||||
|
with op.batch_alter_table("energy_contract", schema=None) as batch_op:
|
||||||
|
batch_op.drop_index("ix_energy_contract_scope")
|
||||||
|
batch_op.drop_column("scope")
|
||||||
@@ -0,0 +1,95 @@
|
|||||||
|
"""add generic commodity-scoped meter cost periods
|
||||||
|
|
||||||
|
Revision ID: 20260822_19_meter_cost_periods
|
||||||
|
Revises: 20260822_18_contract_scopes
|
||||||
|
Create Date: 2026-08-22 00:00:00.000000
|
||||||
|
|
||||||
|
This additive migration creates a separate audit ledger for non-electricity
|
||||||
|
meter costs. It deliberately does not alter, migrate, or delete rows from the
|
||||||
|
existing electricity-only energy_cost_period table.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from typing import Sequence, Union
|
||||||
|
|
||||||
|
import sqlalchemy as sa
|
||||||
|
from alembic import op
|
||||||
|
|
||||||
|
|
||||||
|
revision: str = "20260822_19_meter_cost_periods"
|
||||||
|
down_revision: Union[str, None] = "20260822_18_contract_scopes"
|
||||||
|
branch_labels: Union[str, Sequence[str], None] = None
|
||||||
|
depends_on: Union[str, Sequence[str], None] = None
|
||||||
|
|
||||||
|
|
||||||
|
class ExactDecimal(sa.TypeDecorator):
|
||||||
|
"""Use SQLite text storage while retaining Numeric semantics elsewhere."""
|
||||||
|
|
||||||
|
impl = sa.Numeric
|
||||||
|
cache_ok = True
|
||||||
|
|
||||||
|
def __init__(self, precision: int, scale: int) -> None:
|
||||||
|
self.precision = precision
|
||||||
|
self.scale = scale
|
||||||
|
super().__init__(precision=precision, scale=scale)
|
||||||
|
|
||||||
|
def load_dialect_impl(self, dialect):
|
||||||
|
if dialect.name == "sqlite":
|
||||||
|
return dialect.type_descriptor(sa.String(self.precision + 2))
|
||||||
|
return dialect.type_descriptor(sa.Numeric(self.precision, self.scale, asdecimal=True))
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
op.create_table(
|
||||||
|
"meter_cost_period",
|
||||||
|
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
|
||||||
|
sa.Column("commodity", sa.String(length=32), nullable=False),
|
||||||
|
sa.Column("period_start", sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.Column("period_end", sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.Column("meter_id", sa.Integer(), nullable=True),
|
||||||
|
sa.Column("source_binding_id", sa.Integer(), nullable=True),
|
||||||
|
sa.Column("contract_version_id", sa.Integer(), nullable=True),
|
||||||
|
# SQLite NUMERIC coercion binds Decimal values as binary floats. Store
|
||||||
|
# fixed-width decimal text there, while retaining Numeric semantics on
|
||||||
|
# other supported dialects.
|
||||||
|
sa.Column("quantity", ExactDecimal(15, 6), nullable=False),
|
||||||
|
sa.Column("cost", ExactDecimal(15, 9), nullable=False),
|
||||||
|
sa.Column("currency", sa.String(length=8), nullable=False),
|
||||||
|
sa.Column("cost_breakdown", sa.JSON(), nullable=False),
|
||||||
|
sa.Column("pricing_snapshot", sa.JSON(), nullable=False),
|
||||||
|
sa.Column("quality", sa.String(length=32), nullable=False),
|
||||||
|
sa.Column("degraded", sa.Boolean(), nullable=False, server_default=sa.false()),
|
||||||
|
sa.Column("degraded_reason", sa.String(length=255), nullable=True),
|
||||||
|
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
|
||||||
|
sa.ForeignKeyConstraint(["meter_id"], ["meter.id"], ondelete="RESTRICT"),
|
||||||
|
sa.ForeignKeyConstraint(
|
||||||
|
["source_binding_id"], ["meter_source_binding.id"], ondelete="RESTRICT"
|
||||||
|
),
|
||||||
|
sa.ForeignKeyConstraint(
|
||||||
|
["contract_version_id"], ["energy_contract_version.id"], ondelete="RESTRICT"
|
||||||
|
),
|
||||||
|
sa.CheckConstraint(
|
||||||
|
"degraded OR (meter_id IS NOT NULL AND source_binding_id IS NOT NULL "
|
||||||
|
"AND contract_version_id IS NOT NULL)",
|
||||||
|
name="ck_meter_cost_period_normal_audit_links",
|
||||||
|
),
|
||||||
|
sa.CheckConstraint("period_end > period_start", name="ck_meter_cost_period_positive_interval"),
|
||||||
|
sa.CheckConstraint(
|
||||||
|
"NOT degraded OR (degraded_reason IS NOT NULL AND length(trim(degraded_reason)) > 0)",
|
||||||
|
name="ck_meter_cost_period_degraded_reason",
|
||||||
|
),
|
||||||
|
sa.PrimaryKeyConstraint("id"),
|
||||||
|
sa.UniqueConstraint("commodity", "period_start", name="uq_meter_cost_period_commodity_start"),
|
||||||
|
)
|
||||||
|
op.create_index(
|
||||||
|
"ix_meter_cost_period_commodity_start", "meter_cost_period", ["commodity", "period_start"]
|
||||||
|
)
|
||||||
|
op.create_index(
|
||||||
|
"ix_meter_cost_period_source_binding_id", "meter_cost_period", ["source_binding_id"]
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
op.drop_index("ix_meter_cost_period_source_binding_id", table_name="meter_cost_period")
|
||||||
|
op.drop_index("ix_meter_cost_period_commodity_start", table_name="meter_cost_period")
|
||||||
|
op.drop_table("meter_cost_period")
|
||||||
@@ -9,7 +9,7 @@ from sqlalchemy.orm import Session
|
|||||||
from app.api.routes.api.deps import require_csrf, require_session
|
from app.api.routes.api.deps import require_csrf, require_session
|
||||||
from app.config import Settings, get_settings
|
from app.config import Settings, get_settings
|
||||||
from app.dependencies import get_app_settings, get_db
|
from app.dependencies import get_app_settings, get_db
|
||||||
from app.integrations.mqtt import MQTT_SETTINGS_KEYS, mqtt_manager
|
from app.integrations.mqtt import MQTT_SETTINGS_KEYS, mqtt_manager, mqtt_test_client_id
|
||||||
from app.schemas.config import (
|
from app.schemas.config import (
|
||||||
ConfigField,
|
ConfigField,
|
||||||
ConfigResponse,
|
ConfigResponse,
|
||||||
@@ -22,6 +22,7 @@ from app.schemas.config import (
|
|||||||
from app.services.auth import AuthenticatedSession
|
from app.services.auth import AuthenticatedSession
|
||||||
from app.services.config_page import ConfigSaveError, build_config_sections, save_config_updates
|
from app.services.config_page import ConfigSaveError, build_config_sections, save_config_updates
|
||||||
from app.services.email import EmailConfigurationError, EmailDeliveryError, send_smtp_test_email
|
from app.services.email import EmailConfigurationError, EmailDeliveryError, send_smtp_test_email
|
||||||
|
from app.services.tibber_prices import active_tibber_contract_exists, trigger_tibber_refresh
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
@@ -65,6 +66,10 @@ def put_config(
|
|||||||
# Detect whether any MQTT-related key is being submitted (non-secret change
|
# Detect whether any MQTT-related key is being submitted (non-secret change
|
||||||
# or non-blank secret change) so we know to reconnect after saving.
|
# or non-blank secret change) so we know to reconnect after saving.
|
||||||
mqtt_keys_submitted = any(k.lower() in MQTT_SETTINGS_KEYS for k in body.updates)
|
mqtt_keys_submitted = any(k.lower() in MQTT_SETTINGS_KEYS for k in body.updates)
|
||||||
|
tibber_values_before = (
|
||||||
|
settings.tibber_api_token,
|
||||||
|
settings.tibber_home_id,
|
||||||
|
)
|
||||||
|
|
||||||
try:
|
try:
|
||||||
save_config_updates(db, body.updates, settings)
|
save_config_updates(db, body.updates, settings)
|
||||||
@@ -92,6 +97,13 @@ def put_config(
|
|||||||
from app.services.dsmr_ingest import apply_dsmr_subscription
|
from app.services.dsmr_ingest import apply_dsmr_subscription
|
||||||
apply_dsmr_subscription(refreshed_settings)
|
apply_dsmr_subscription(refreshed_settings)
|
||||||
|
|
||||||
|
tibber_values_changed = tibber_values_before != (
|
||||||
|
refreshed_settings.tibber_api_token,
|
||||||
|
refreshed_settings.tibber_home_id,
|
||||||
|
)
|
||||||
|
if tibber_values_changed and active_tibber_contract_exists(db):
|
||||||
|
trigger_tibber_refresh()
|
||||||
|
|
||||||
sections_raw = build_config_sections(db, refreshed_settings)
|
sections_raw = build_config_sections(db, refreshed_settings)
|
||||||
return ConfigUpdateResponse(sections=_sections_from_raw(sections_raw))
|
return ConfigUpdateResponse(sections=_sections_from_raw(sections_raw))
|
||||||
|
|
||||||
@@ -233,7 +245,7 @@ def _run_mqtt_test(settings: Settings) -> None:
|
|||||||
|
|
||||||
client = mqtt.Client(
|
client = mqtt.Client(
|
||||||
callback_api_version=mqtt.CallbackAPIVersion.VERSION2,
|
callback_api_version=mqtt.CallbackAPIVersion.VERSION2,
|
||||||
client_id="home-automation-test",
|
client_id=mqtt_test_client_id(settings.mqtt_client_id),
|
||||||
)
|
)
|
||||||
|
|
||||||
def _on_connect(
|
def _on_connect(
|
||||||
|
|||||||
@@ -44,7 +44,7 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import logging
|
import logging
|
||||||
from datetime import UTC, datetime, timedelta
|
from datetime import UTC, datetime, timedelta
|
||||||
from typing import Any
|
from typing import Any, Literal
|
||||||
|
|
||||||
from fastapi import APIRouter, Depends, Query, status
|
from fastapi import APIRouter, Depends, Query, status
|
||||||
from fastapi.responses import JSONResponse
|
from fastapi.responses import JSONResponse
|
||||||
@@ -133,13 +133,23 @@ def _manual_tariff_from_values(values: dict[str, Any]) -> ManualTariffSchema:
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _electricity_prices_response(response: PricesResponse) -> JSONResponse:
|
||||||
|
"""Preserve the exact pre-scope electricity response body."""
|
||||||
|
return JSONResponse(
|
||||||
|
content=response.model_dump(
|
||||||
|
mode="json", include={"kind", "currency", "points", "tariff"}
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# GET /api/energy/prices
|
# GET /api/energy/prices
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
@router.get("/prices", response_model=PricesResponse)
|
@router.get("/prices", response_model=PricesResponse, response_model_exclude_none=True)
|
||||||
def get_prices(
|
def get_prices(
|
||||||
|
scope: Literal["electricity", "thermal"] = Query("electricity"),
|
||||||
start: datetime | None = Query(
|
start: datetime | None = Query(
|
||||||
default=None,
|
default=None,
|
||||||
description="Inclusive start of the time window (ISO 8601). "
|
description="Inclusive start of the time window (ISO 8601). "
|
||||||
@@ -186,6 +196,17 @@ def get_prices(
|
|||||||
"""
|
"""
|
||||||
now = datetime.now(UTC)
|
now = datetime.now(UTC)
|
||||||
|
|
||||||
|
if scope == "thermal":
|
||||||
|
version = active_contract_version_at(db, now, scope="thermal")
|
||||||
|
if version is None:
|
||||||
|
return PricesResponse(kind=None, currency="EUR", points=[], tariff=None)
|
||||||
|
return PricesResponse(
|
||||||
|
kind="district_heating", currency=version.contract.currency,
|
||||||
|
contract_version_id=version.id, effective_from=_as_utc(version.effective_from),
|
||||||
|
effective_to=_as_utc(version.effective_to) if version.effective_to else None,
|
||||||
|
values=version.values, points=[], tariff=None,
|
||||||
|
)
|
||||||
|
|
||||||
# Default window: today + tomorrow.
|
# Default window: today + tomorrow.
|
||||||
if start is None:
|
if start is None:
|
||||||
start = now.replace(hour=0, minute=0, second=0, microsecond=0)
|
start = now.replace(hour=0, minute=0, second=0, microsecond=0)
|
||||||
@@ -199,12 +220,12 @@ def get_prices(
|
|||||||
version = active_contract_version_at(db, start_utc)
|
version = active_contract_version_at(db, start_utc)
|
||||||
|
|
||||||
if version is None:
|
if version is None:
|
||||||
return PricesResponse(
|
return _electricity_prices_response(PricesResponse(
|
||||||
kind=None,
|
kind=None,
|
||||||
currency="EUR",
|
currency="EUR",
|
||||||
points=[],
|
points=[],
|
||||||
tariff=None,
|
tariff=None,
|
||||||
)
|
))
|
||||||
|
|
||||||
contract = version.contract
|
contract = version.contract
|
||||||
currency = contract.currency
|
currency = contract.currency
|
||||||
@@ -247,30 +268,30 @@ def get_prices(
|
|||||||
)
|
)
|
||||||
)
|
)
|
||||||
|
|
||||||
return PricesResponse(
|
return _electricity_prices_response(PricesResponse(
|
||||||
kind="tibber",
|
kind="tibber",
|
||||||
currency=currency,
|
currency=currency,
|
||||||
points=points,
|
points=points,
|
||||||
tariff=None,
|
tariff=None,
|
||||||
)
|
))
|
||||||
|
|
||||||
elif contract.kind == "manual":
|
elif contract.kind == "manual":
|
||||||
tariff = _manual_tariff_from_values(version.values or {})
|
tariff = _manual_tariff_from_values(version.values or {})
|
||||||
return PricesResponse(
|
return _electricity_prices_response(PricesResponse(
|
||||||
kind="manual",
|
kind="manual",
|
||||||
currency=currency,
|
currency=currency,
|
||||||
points=[],
|
points=[],
|
||||||
tariff=tariff,
|
tariff=tariff,
|
||||||
)
|
))
|
||||||
|
|
||||||
else:
|
else:
|
||||||
# Unknown kind — return empty response gracefully.
|
# Unknown kind — return empty response gracefully.
|
||||||
return PricesResponse(
|
return _electricity_prices_response(PricesResponse(
|
||||||
kind=contract.kind,
|
kind=contract.kind,
|
||||||
currency=currency,
|
currency=currency,
|
||||||
points=[],
|
points=[],
|
||||||
tariff=None,
|
tariff=None,
|
||||||
)
|
))
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
|
|||||||
@@ -46,7 +46,9 @@ from app.schemas.energy_contract import (
|
|||||||
from app.services.auth import AuthenticatedSession
|
from app.services.auth import AuthenticatedSession
|
||||||
from app.services import timezone as _tz_mod
|
from app.services import timezone as _tz_mod
|
||||||
from app.services.contracts import (
|
from app.services.contracts import (
|
||||||
|
CONTRACT_KIND_SCOPES,
|
||||||
ContractVersionError,
|
ContractVersionError,
|
||||||
|
ContractScopeError,
|
||||||
activate_contract,
|
activate_contract,
|
||||||
add_version,
|
add_version,
|
||||||
create_contract,
|
create_contract,
|
||||||
@@ -54,6 +56,7 @@ from app.services.contracts import (
|
|||||||
get_contract_or_none,
|
get_contract_or_none,
|
||||||
list_contracts,
|
list_contracts,
|
||||||
)
|
)
|
||||||
|
from app.services.tibber_prices import trigger_tibber_refresh
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
@@ -98,6 +101,7 @@ def _contract_detail(db: Session, contract) -> ContractDetailResponse:
|
|||||||
id=contract.id,
|
id=contract.id,
|
||||||
name=contract.name,
|
name=contract.name,
|
||||||
kind=contract.kind,
|
kind=contract.kind,
|
||||||
|
scope=contract.scope,
|
||||||
active=contract.active,
|
active=contract.active,
|
||||||
currency=contract.currency,
|
currency=contract.currency,
|
||||||
created_at=contract.created_at,
|
created_at=contract.created_at,
|
||||||
@@ -163,16 +167,22 @@ def get_profiles(
|
|||||||
|
|
||||||
@router.get("/contracts", response_model=ContractListResponse)
|
@router.get("/contracts", response_model=ContractListResponse)
|
||||||
def list_energy_contracts(
|
def list_energy_contracts(
|
||||||
|
scope: str = "electricity",
|
||||||
db: Session = Depends(get_db),
|
db: Session = Depends(get_db),
|
||||||
_auth: AuthenticatedSession = Depends(require_session),
|
_auth: AuthenticatedSession = Depends(require_session),
|
||||||
) -> ContractListResponse:
|
) -> ContractListResponse:
|
||||||
"""List all energy contracts with their active status.
|
"""List all energy contracts with their active status.
|
||||||
|
|
||||||
Returns a flat list (no embedded version history); use
|
Scope defaults to ``electricity`` for old clients. Returns a flat list (no embedded version history); use
|
||||||
GET /api/energy/contracts/{id} to fetch the full version history for a
|
GET /api/energy/contracts/{id} to fetch the full version history for a
|
||||||
specific contract.
|
specific contract.
|
||||||
"""
|
"""
|
||||||
contracts = list_contracts(db)
|
if scope not in set(CONTRACT_KIND_SCOPES.values()):
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
|
||||||
|
detail=f"Unknown energy contract scope: {scope!r}",
|
||||||
|
)
|
||||||
|
contracts = list_contracts(db, scope=scope)
|
||||||
items = [ContractResponse.model_validate(c) for c in contracts]
|
items = [ContractResponse.model_validate(c) for c in contracts]
|
||||||
return ContractListResponse(items=items, total=len(items))
|
return ContractListResponse(items=items, total=len(items))
|
||||||
|
|
||||||
@@ -208,10 +218,11 @@ def create_energy_contract(
|
|||||||
name=body.name,
|
name=body.name,
|
||||||
kind=body.kind,
|
kind=body.kind,
|
||||||
currency=body.currency,
|
currency=body.currency,
|
||||||
|
scope=body.scope,
|
||||||
values=body.values,
|
values=body.values,
|
||||||
effective_from=effective_from,
|
effective_from=effective_from,
|
||||||
)
|
)
|
||||||
except (ProfileNotFoundError, ProfileValidationError) as exc:
|
except (ProfileNotFoundError, ProfileValidationError, ContractScopeError) as exc:
|
||||||
_raise_422_for_profile_error(exc)
|
_raise_422_for_profile_error(exc)
|
||||||
|
|
||||||
db.commit()
|
db.commit()
|
||||||
@@ -256,13 +267,14 @@ def patch_energy_contract(
|
|||||||
"""Partially update a contract: rename or change activation status.
|
"""Partially update a contract: rename or change activation status.
|
||||||
|
|
||||||
- ``name``: updates the human-readable label.
|
- ``name``: updates the human-readable label.
|
||||||
- ``active=true``: activates this contract (all others are deactivated).
|
- ``active=true``: activates this contract (same-scope contracts are deactivated).
|
||||||
- ``active=false``: deactivates this contract (no effect on others).
|
- ``active=false``: deactivates this contract (no effect on others).
|
||||||
|
|
||||||
At most one contract may be active at any time; the service layer enforces
|
At most one contract may be active per scope; the service layer enforces
|
||||||
mutual exclusion.
|
scope-local mutual exclusion.
|
||||||
"""
|
"""
|
||||||
contract = _get_contract_or_404(db, contract_id)
|
contract = _get_contract_or_404(db, contract_id)
|
||||||
|
was_active = contract.active
|
||||||
|
|
||||||
if body.name is not None:
|
if body.name is not None:
|
||||||
contract.name = body.name
|
contract.name = body.name
|
||||||
@@ -275,6 +287,8 @@ def patch_energy_contract(
|
|||||||
|
|
||||||
db.commit()
|
db.commit()
|
||||||
db.refresh(contract)
|
db.refresh(contract)
|
||||||
|
if body.active is True and not was_active and contract.kind == "tibber":
|
||||||
|
trigger_tibber_refresh()
|
||||||
return _contract_detail(db, contract)
|
return _contract_detail(db, contract)
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,150 @@
|
|||||||
|
"""Authenticated API for the thermal 15-minute cost ledger."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import UTC, datetime, timedelta
|
||||||
|
from decimal import Decimal
|
||||||
|
from typing import Literal
|
||||||
|
|
||||||
|
from fastapi import APIRouter, Depends, HTTPException, Query, status
|
||||||
|
from sqlalchemy import func, select
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from app.api.routes.api.deps import require_csrf, require_session
|
||||||
|
from app.dependencies import get_db
|
||||||
|
from app.models.energy import MeterCostPeriod
|
||||||
|
from app.schemas.meter_cost import (
|
||||||
|
MeterCostPeriodSchema,
|
||||||
|
MeterCostRecomputeResponse,
|
||||||
|
MeterCostsResponse,
|
||||||
|
ThermalCostSummaryResponse,
|
||||||
|
)
|
||||||
|
from app.services.auth import AuthenticatedSession
|
||||||
|
from app.services.meter_cost import recompute_range, summarize
|
||||||
|
from app.services.timezone import local_midnight_utc, local_now
|
||||||
|
|
||||||
|
router = APIRouter(prefix="/api/energy/meter-costs", tags=["api-energy"])
|
||||||
|
|
||||||
|
_LIMIT_MAX = 5000
|
||||||
|
_RECOMPUTE_MAX_DAYS = 31
|
||||||
|
_QUARTER = timedelta(minutes=15)
|
||||||
|
|
||||||
|
|
||||||
|
def _utc(value: datetime) -> datetime:
|
||||||
|
return value.replace(tzinfo=UTC) if value.tzinfo is None else value.astimezone(UTC)
|
||||||
|
|
||||||
|
|
||||||
|
def _decimal_strings(value: object) -> object:
|
||||||
|
if isinstance(value, dict):
|
||||||
|
return {str(key): _decimal_strings(item) for key, item in value.items()}
|
||||||
|
if isinstance(value, Decimal):
|
||||||
|
return format(value, "f")
|
||||||
|
return str(value) if isinstance(value, (int, float)) else value
|
||||||
|
|
||||||
|
|
||||||
|
def _row_schema(row: MeterCostPeriod) -> MeterCostPeriodSchema:
|
||||||
|
return MeterCostPeriodSchema(
|
||||||
|
commodity=row.commodity,
|
||||||
|
period_start=_utc(row.period_start), period_end=_utc(row.period_end),
|
||||||
|
meter_id=row.meter_id, source_binding_id=row.source_binding_id,
|
||||||
|
contract_version_id=row.contract_version_id, quantity=format(row.quantity, "f"),
|
||||||
|
cost=format(row.cost, "f"), currency=row.currency,
|
||||||
|
cost_breakdown=_decimal_strings(row.cost_breakdown),
|
||||||
|
pricing_snapshot=_decimal_strings(row.pricing_snapshot), quality=row.quality,
|
||||||
|
degraded=row.degraded, degraded_reason=row.degraded_reason,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("", response_model=MeterCostsResponse)
|
||||||
|
def get_meter_costs(
|
||||||
|
scope: Literal["thermal"] = Query("thermal"),
|
||||||
|
commodity: Literal["heating", "hot_water"] | None = None,
|
||||||
|
start: datetime | None = None,
|
||||||
|
end: datetime | None = None,
|
||||||
|
limit: int = Query(500, ge=1, le=_LIMIT_MAX),
|
||||||
|
offset: int = Query(0, ge=0),
|
||||||
|
db: Session = Depends(get_db),
|
||||||
|
_auth: AuthenticatedSession = Depends(require_session),
|
||||||
|
) -> MeterCostsResponse:
|
||||||
|
"""List thermal rows in a half-open time window with stable pagination."""
|
||||||
|
del scope
|
||||||
|
if start is not None and end is not None and _utc(end) <= _utc(start):
|
||||||
|
raise HTTPException(status.HTTP_422_UNPROCESSABLE_ENTITY, "'end' must be after 'start'.")
|
||||||
|
clauses = []
|
||||||
|
if commodity is not None:
|
||||||
|
clauses.append(MeterCostPeriod.commodity == commodity)
|
||||||
|
if start is not None:
|
||||||
|
clauses.append(MeterCostPeriod.period_start >= _utc(start))
|
||||||
|
if end is not None:
|
||||||
|
clauses.append(MeterCostPeriod.period_start < _utc(end))
|
||||||
|
total = db.scalar(select(func.count()).select_from(MeterCostPeriod).where(*clauses)) or 0
|
||||||
|
rows = db.execute(
|
||||||
|
select(MeterCostPeriod).where(*clauses).order_by(MeterCostPeriod.period_start, MeterCostPeriod.id)
|
||||||
|
.offset(offset).limit(limit)
|
||||||
|
).scalars().all()
|
||||||
|
return MeterCostsResponse(items=[_row_schema(row) for row in rows], total=total)
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/summary", response_model=ThermalCostSummaryResponse)
|
||||||
|
def get_meter_cost_summary(
|
||||||
|
scope: Literal["thermal"] = Query("thermal"),
|
||||||
|
start: datetime | None = None,
|
||||||
|
end: datetime | None = None,
|
||||||
|
db: Session = Depends(get_db),
|
||||||
|
_auth: AuthenticatedSession = Depends(require_session),
|
||||||
|
) -> ThermalCostSummaryResponse:
|
||||||
|
"""Summarize thermal variable and once-per-contract daily fixed costs."""
|
||||||
|
del scope
|
||||||
|
if start is None or end is None:
|
||||||
|
today = local_now().date()
|
||||||
|
start = start or local_midnight_utc(today)
|
||||||
|
end = end or local_midnight_utc(today + timedelta(days=1))
|
||||||
|
start, end = _utc(start), _utc(end)
|
||||||
|
if end <= start:
|
||||||
|
raise HTTPException(status.HTTP_422_UNPROCESSABLE_ENTITY, "'end' must be after 'start'.")
|
||||||
|
result = summarize(db, start, end)
|
||||||
|
breakdown = result["breakdown"]
|
||||||
|
fixed_breakdown = result["fixed_breakdown"]
|
||||||
|
fixed = result["fixed_cost"]
|
||||||
|
return ThermalCostSummaryResponse(
|
||||||
|
currency=result["currency"], heating=format(breakdown["heating"], "f"),
|
||||||
|
hot_water_heating=format(breakdown["hot_water_heating"], "f"),
|
||||||
|
hot_water=format(breakdown["hot_water"], "f"), hot_water_tax=format(breakdown["hot_water_tax"], "f"),
|
||||||
|
variable_subtotal=format(result["variable_cost"], "f"),
|
||||||
|
fixed_breakdown={key: format(value, "f") for key, value in fixed_breakdown.items()},
|
||||||
|
fixed_subtotal=format(fixed, "f"),
|
||||||
|
all_in=format(result["total_cost"], "f"), period_count=result["period_count"],
|
||||||
|
degraded_count=result["degraded_count"],
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/recompute", response_model=MeterCostRecomputeResponse)
|
||||||
|
def post_meter_cost_recompute(
|
||||||
|
scope: Literal["thermal"] = Query("thermal"),
|
||||||
|
start: datetime = Query(...), end: datetime = Query(...),
|
||||||
|
db: Session = Depends(get_db), _auth: AuthenticatedSession = Depends(require_session),
|
||||||
|
_csrf: None = Depends(require_csrf),
|
||||||
|
) -> MeterCostRecomputeResponse:
|
||||||
|
"""Atomically overwrite closed, UTC-quarter thermal rows in a bounded window."""
|
||||||
|
del scope
|
||||||
|
start, end = _utc(start), _utc(end)
|
||||||
|
if end <= start or end - start > timedelta(days=_RECOMPUTE_MAX_DAYS):
|
||||||
|
raise HTTPException(status.HTTP_422_UNPROCESSABLE_ENTITY, "invalid or overlarge recompute window")
|
||||||
|
if start.minute % 15 or start.second or start.microsecond or end.minute % 15 or end.second or end.microsecond:
|
||||||
|
raise HTTPException(status.HTTP_422_UNPROCESSABLE_ENTITY, "start and end must align to UTC quarters")
|
||||||
|
if end > datetime.now(UTC):
|
||||||
|
raise HTTPException(status.HTTP_422_UNPROCESSABLE_ENTITY, "recompute window must be closed")
|
||||||
|
try:
|
||||||
|
processed = recompute_range(db, start, end, commit=False)
|
||||||
|
# Ensure pending upserts participate in this transaction before the
|
||||||
|
# counts are read; a flush/query failure must still roll everything back.
|
||||||
|
db.flush()
|
||||||
|
rows = db.execute(select(MeterCostPeriod.degraded).where(
|
||||||
|
MeterCostPeriod.period_start >= start, MeterCostPeriod.period_start < end
|
||||||
|
)).scalars().all()
|
||||||
|
degraded = sum(bool(value) for value in rows)
|
||||||
|
db.commit()
|
||||||
|
except Exception:
|
||||||
|
db.rollback()
|
||||||
|
raise
|
||||||
|
return MeterCostRecomputeResponse(processed=processed, normal=len(rows) - degraded, degraded=degraded)
|
||||||
@@ -0,0 +1,414 @@
|
|||||||
|
"""Authenticated HTTP contract for meter sources, channels, and bindings."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import UTC, datetime
|
||||||
|
|
||||||
|
from fastapi import APIRouter, Depends, HTTPException, Query, Response, status
|
||||||
|
from sqlalchemy import select
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from app.api.routes.api.deps import require_csrf, require_session
|
||||||
|
from app.config import get_settings
|
||||||
|
from app.dependencies import get_db
|
||||||
|
from app.integrations.meter_sources import SourceProfileError, list_source_profiles, sanitize_source_config
|
||||||
|
from app.models.energy import DsmrReading, Meter
|
||||||
|
from app.models.meter_source import MeterSource, MeterSourceBinding, MeterSourceChannel, WarmteLinkReading
|
||||||
|
from app.schemas.meter_source import (
|
||||||
|
BindingCreate, BindingListResponse, BindingPatch, BindingResponse, ChannelBindingSummaryResponse,
|
||||||
|
BindingTransferRequest, BindingTransferResponse,
|
||||||
|
ChannelReadingResponse,
|
||||||
|
ChannelReadingsResponse, CommoditiesResponse, CommodityResponse, DiscoverResponse,
|
||||||
|
DiscoverChannelResponse,
|
||||||
|
MeterSourceChannelListResponse, MeterSourceChannelResponse, MeterSourceCreate,
|
||||||
|
MeterSourceListResponse, MeterSourcePatch, MeterSourceResponse, SourceConfigFieldResponse,
|
||||||
|
SourceProfileResponse, SourceProfilesResponse,
|
||||||
|
)
|
||||||
|
from app.services.auth import AuthenticatedSession
|
||||||
|
from app.services.config_page import build_runtime_settings
|
||||||
|
from app.services.dsmr_ingest import apply_dsmr_subscription
|
||||||
|
from app.services.meter_sources import (
|
||||||
|
BindingNotFoundError, ChannelNotFoundError, MeterNotFoundError,
|
||||||
|
MeterSourceError, SourceDeleteRestrictedError, SourceNotFoundError, create_binding,
|
||||||
|
create_source, delete_source, list_bindings, list_sources, transfer_binding, update_binding, update_source,
|
||||||
|
)
|
||||||
|
from app.services.energy_cost import recompute_range as electricity_recompute_range
|
||||||
|
from app.services import timezone as _tz_mod
|
||||||
|
from app.services.warmtelink_worker import warmtelink_worker_manager
|
||||||
|
|
||||||
|
router = APIRouter(prefix="/api/energy", tags=["api-energy-meter-sources"])
|
||||||
|
|
||||||
|
|
||||||
|
def _reconcile_runtimes_after_commit(db: Session) -> None:
|
||||||
|
"""Best-effort runtime convergence after a durable source CRUD commit."""
|
||||||
|
try:
|
||||||
|
warmtelink_worker_manager.reconcile()
|
||||||
|
except Exception:
|
||||||
|
# The manager records individual source failures itself. Do not turn a
|
||||||
|
# successful durable create/update/delete into a misleading HTTP 500.
|
||||||
|
pass
|
||||||
|
try:
|
||||||
|
apply_dsmr_subscription(build_runtime_settings(db, get_settings()))
|
||||||
|
except Exception:
|
||||||
|
# DSMR owns independent source clients. Its failure must neither undo
|
||||||
|
# durable CRUD nor prevent the WarmteLink manager from converging.
|
||||||
|
pass
|
||||||
|
finally:
|
||||||
|
# DSMR health callbacks use short independent sessions. Make a CRUD
|
||||||
|
# response observe any durable status change they just committed.
|
||||||
|
db.expire_all()
|
||||||
|
|
||||||
|
|
||||||
|
def _as_utc(value: datetime) -> datetime:
|
||||||
|
if value.tzinfo is None:
|
||||||
|
return value.replace(tzinfo=_tz_mod.local_tz()).astimezone(UTC)
|
||||||
|
return value.astimezone(UTC)
|
||||||
|
|
||||||
|
|
||||||
|
def _source_or_404(db: Session, uuid: str) -> MeterSource:
|
||||||
|
source = db.execute(select(MeterSource).where(MeterSource.uuid == uuid)).scalar_one_or_none()
|
||||||
|
if source is None:
|
||||||
|
raise HTTPException(status_code=404, detail="Meter source not found.")
|
||||||
|
return source
|
||||||
|
|
||||||
|
|
||||||
|
def _channel_or_404(db: Session, source: MeterSource, uuid: str) -> MeterSourceChannel:
|
||||||
|
channel = db.execute(
|
||||||
|
select(MeterSourceChannel).where(
|
||||||
|
MeterSourceChannel.uuid == uuid, MeterSourceChannel.source_id == source.id
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if channel is None:
|
||||||
|
raise HTTPException(status_code=404, detail="Meter source channel not found.")
|
||||||
|
return channel
|
||||||
|
|
||||||
|
|
||||||
|
def _source_response(source: MeterSource) -> MeterSourceResponse:
|
||||||
|
return MeterSourceResponse(
|
||||||
|
uuid=source.uuid, name=source.name, kind=source.kind, enabled=source.enabled,
|
||||||
|
config=sanitize_source_config(source.kind, source.config), status=source.status,
|
||||||
|
last_seen_at=source.last_seen_at, last_error=source.last_error,
|
||||||
|
created_at=source.created_at, updated_at=source.updated_at,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def binding_response(binding: MeterSourceBinding) -> BindingResponse:
|
||||||
|
return BindingResponse(
|
||||||
|
uuid=binding.uuid, meter_id=binding.meter_id, source_channel_uuid=binding.channel.uuid,
|
||||||
|
source_uuid=binding.channel.source.uuid, started_at=binding.started_at, ended_at=binding.ended_at,
|
||||||
|
created_at=binding.created_at, updated_at=binding.updated_at,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _binding_error(exc: MeterSourceError) -> HTTPException:
|
||||||
|
if isinstance(exc, (SourceNotFoundError, ChannelNotFoundError, MeterNotFoundError, BindingNotFoundError)):
|
||||||
|
return HTTPException(status_code=404, detail=str(exc))
|
||||||
|
return HTTPException(status_code=422, detail=str(exc))
|
||||||
|
|
||||||
|
|
||||||
|
def _recompute_binding_commodity(db: Session, commodity: str, start: datetime) -> None:
|
||||||
|
end = datetime.now(UTC)
|
||||||
|
if start >= end:
|
||||||
|
return
|
||||||
|
if commodity == "electricity":
|
||||||
|
electricity_recompute_range(db, start, end, commit=False, strict=True)
|
||||||
|
else:
|
||||||
|
from app.services.meter_cost import recompute_range
|
||||||
|
recompute_range(db, start, end, commit=False)
|
||||||
|
|
||||||
|
|
||||||
|
def _republish_after_commit(db: Session) -> None:
|
||||||
|
try:
|
||||||
|
from app.services.ha_discovery import publish_discovery
|
||||||
|
publish_discovery(db)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/source-profiles", response_model=SourceProfilesResponse)
|
||||||
|
def source_profiles(_auth: AuthenticatedSession = Depends(require_session)) -> SourceProfilesResponse:
|
||||||
|
"""Return profile metadata; default secrets are never populated with stored values."""
|
||||||
|
profiles = []
|
||||||
|
for profile in list_source_profiles():
|
||||||
|
fields = [
|
||||||
|
SourceConfigFieldResponse(name=f.name, value_type=f.value_type.__name__, default=f.default,
|
||||||
|
required=f.required, secret=f.secret)
|
||||||
|
for f in profile.fields
|
||||||
|
]
|
||||||
|
profiles.append(SourceProfileResponse(
|
||||||
|
kind=profile.kind, fields=fields,
|
||||||
|
defaults={f.name: f.default for f in profile.fields if not f.required},
|
||||||
|
capabilities=sorted(profile.capabilities), allowed_units=sorted(profile.allowed_units),
|
||||||
|
))
|
||||||
|
return SourceProfilesResponse(items=profiles)
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/commodities", response_model=CommoditiesResponse)
|
||||||
|
def commodities(_auth: AuthenticatedSession = Depends(require_session)) -> CommoditiesResponse:
|
||||||
|
return CommoditiesResponse(items=[
|
||||||
|
CommodityResponse(key="electricity", unit="kWh", capabilities=["meter", "binding", "cost"]),
|
||||||
|
CommodityResponse(key="heating", unit="GJ", capabilities=["meter", "binding"]),
|
||||||
|
CommodityResponse(key="hot_water", unit="m³", capabilities=["meter", "binding"]),
|
||||||
|
])
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/sources", response_model=MeterSourceListResponse)
|
||||||
|
def get_sources(db: Session = Depends(get_db), _auth: AuthenticatedSession = Depends(require_session)) -> MeterSourceListResponse:
|
||||||
|
items = [_source_response(source) for source in list_sources(db)]
|
||||||
|
return MeterSourceListResponse(items=items, total=len(items))
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/sources", response_model=MeterSourceResponse, status_code=status.HTTP_201_CREATED)
|
||||||
|
def post_source(body: MeterSourceCreate, db: Session = Depends(get_db),
|
||||||
|
_auth: AuthenticatedSession = Depends(require_session), _csrf: None = Depends(require_csrf)) -> MeterSourceResponse:
|
||||||
|
try:
|
||||||
|
source = create_source(db, name=body.name, kind=body.kind, config=body.config, enabled=body.enabled)
|
||||||
|
db.commit()
|
||||||
|
_reconcile_runtimes_after_commit(db)
|
||||||
|
return _source_response(_source_or_404(db, source.uuid))
|
||||||
|
except (SourceProfileError, MeterSourceError) as exc:
|
||||||
|
db.rollback()
|
||||||
|
raise HTTPException(status_code=422, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/sources/{source_uuid}", response_model=MeterSourceResponse)
|
||||||
|
def get_source_detail(source_uuid: str, db: Session = Depends(get_db),
|
||||||
|
_auth: AuthenticatedSession = Depends(require_session)) -> MeterSourceResponse:
|
||||||
|
return _source_response(_source_or_404(db, source_uuid))
|
||||||
|
|
||||||
|
|
||||||
|
@router.patch("/sources/{source_uuid}", response_model=MeterSourceResponse)
|
||||||
|
def patch_source(source_uuid: str, body: MeterSourcePatch, db: Session = Depends(get_db),
|
||||||
|
_auth: AuthenticatedSession = Depends(require_session), _csrf: None = Depends(require_csrf)) -> MeterSourceResponse:
|
||||||
|
source = _source_or_404(db, source_uuid)
|
||||||
|
try:
|
||||||
|
updated = update_source(db, source.id, name=body.name, enabled=body.enabled, config_patch=body.config)
|
||||||
|
db.commit()
|
||||||
|
_reconcile_runtimes_after_commit(db)
|
||||||
|
return _source_response(_source_or_404(db, updated.uuid))
|
||||||
|
except (SourceProfileError, MeterSourceError) as exc:
|
||||||
|
db.rollback()
|
||||||
|
raise _binding_error(exc) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@router.delete(
|
||||||
|
"/sources/{source_uuid}", status_code=status.HTTP_204_NO_CONTENT, response_model=None
|
||||||
|
)
|
||||||
|
def remove_source(source_uuid: str, db: Session = Depends(get_db),
|
||||||
|
_auth: AuthenticatedSession = Depends(require_session), _csrf: None = Depends(require_csrf)) -> None:
|
||||||
|
source = _source_or_404(db, source_uuid)
|
||||||
|
# DSMR readings are not a relationship on MeterSource to avoid loading a large history.
|
||||||
|
if db.execute(select(DsmrReading.id).where(DsmrReading.meter_source_id == source.id).limit(1)).scalar() is not None:
|
||||||
|
raise HTTPException(status_code=409, detail="Meter source has dependent readings.")
|
||||||
|
try:
|
||||||
|
delete_source(db, source.id)
|
||||||
|
db.commit()
|
||||||
|
_reconcile_runtimes_after_commit(db)
|
||||||
|
return Response(status_code=status.HTTP_204_NO_CONTENT)
|
||||||
|
except SourceDeleteRestrictedError as exc:
|
||||||
|
db.rollback()
|
||||||
|
raise HTTPException(status_code=409, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/sources/{source_uuid}/discover", response_model=DiscoverResponse)
|
||||||
|
def discover_source(source_uuid: str, db: Session = Depends(get_db),
|
||||||
|
_auth: AuthenticatedSession = Depends(require_session), _csrf: None = Depends(require_csrf)) -> DiscoverResponse:
|
||||||
|
source = _source_or_404(db, source_uuid)
|
||||||
|
if source.kind == "warmtelink_serial":
|
||||||
|
if not source.enabled:
|
||||||
|
return DiscoverResponse(
|
||||||
|
requested=False, supported=True, status="error",
|
||||||
|
detail="The WarmteLink source is disabled.", channels=_discover_channels(db, source),
|
||||||
|
)
|
||||||
|
# This merely schedules lifecycle convergence. It never opens a serial
|
||||||
|
# descriptor or waits for a frame in the request thread; the one managed
|
||||||
|
# worker remains the sole owner of serial I/O and can keep reconnecting.
|
||||||
|
request = warmtelink_worker_manager.request_discovery(source.id)
|
||||||
|
if request.completed.is_set():
|
||||||
|
# A worker may have accepted a frame during the bounded wait.
|
||||||
|
# Refresh only durable accepted metadata, never candidates/raw data.
|
||||||
|
db.expire_all()
|
||||||
|
source = _source_or_404(db, source_uuid)
|
||||||
|
return DiscoverResponse(
|
||||||
|
requested=request.status != "error", supported=True, status=request.status,
|
||||||
|
request_id=request.request_id or None, detail=request.detail,
|
||||||
|
channels=_discover_channels(db, source),
|
||||||
|
)
|
||||||
|
return DiscoverResponse(requested=False, supported=True, status="managed_by_runtime",
|
||||||
|
detail="This source is discovered by its runtime subscription; no connection was opened.")
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/sources/{source_uuid}/channels", response_model=MeterSourceChannelListResponse)
|
||||||
|
def source_channels(source_uuid: str, db: Session = Depends(get_db),
|
||||||
|
_auth: AuthenticatedSession = Depends(require_session)) -> MeterSourceChannelListResponse:
|
||||||
|
source = _source_or_404(db, source_uuid)
|
||||||
|
channels = db.execute(select(MeterSourceChannel).where(MeterSourceChannel.source_id == source.id)).scalars().all()
|
||||||
|
items = []
|
||||||
|
for channel in channels:
|
||||||
|
bindings = list_bindings(db, channel_id=channel.id)
|
||||||
|
meter_ids = [binding.meter_id for binding in bindings]
|
||||||
|
items.append(MeterSourceChannelResponse(
|
||||||
|
uuid=channel.uuid, label=channel.label, suggested_commodity=channel.suggested_commodity,
|
||||||
|
unit=channel.unit, device_type=channel.device_type, latest_value=channel.latest_value,
|
||||||
|
latest_at=channel.latest_at, latest_quality=channel.latest_quality, binding_count=len(bindings),
|
||||||
|
bound_meter_ids=meter_ids,
|
||||||
|
binding_summary=ChannelBindingSummaryResponse(count=len(bindings), meter_ids=meter_ids),
|
||||||
|
))
|
||||||
|
return MeterSourceChannelListResponse(items=items, total=len(items), source_status=source.status)
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/sources/{source_uuid}/channels/{channel_uuid}/readings", response_model=ChannelReadingsResponse)
|
||||||
|
def channel_readings(source_uuid: str, channel_uuid: str, limit: int = Query(default=100, ge=1, le=1000),
|
||||||
|
from_: datetime | None = Query(default=None, alias="from"),
|
||||||
|
to: datetime | None = Query(default=None), db: Session = Depends(get_db),
|
||||||
|
_auth: AuthenticatedSession = Depends(require_session)) -> ChannelReadingsResponse:
|
||||||
|
source = _source_or_404(db, source_uuid)
|
||||||
|
channel = _channel_or_404(db, source, channel_uuid)
|
||||||
|
if from_ is not None and to is not None and _as_utc(from_) >= _as_utc(to):
|
||||||
|
raise HTTPException(status_code=422, detail="'from' must be earlier than 'to'.")
|
||||||
|
if source.kind == "warmtelink_serial":
|
||||||
|
statement = select(WarmteLinkReading).where(WarmteLinkReading.channel_id == channel.id)
|
||||||
|
model = WarmteLinkReading
|
||||||
|
else:
|
||||||
|
# DSMR remains a source-level protocol history. Its channel is the
|
||||||
|
# public electricity identity, while payload/telegram diagnostics stay
|
||||||
|
# private to ingestion and the legacy latest endpoint.
|
||||||
|
statement = select(DsmrReading).where(DsmrReading.meter_source_id == source.id)
|
||||||
|
model = DsmrReading
|
||||||
|
if from_ is not None:
|
||||||
|
statement = statement.where(model.recorded_at >= _as_utc(from_))
|
||||||
|
if to is not None:
|
||||||
|
statement = statement.where(model.recorded_at < _as_utc(to))
|
||||||
|
rows = list(db.execute(statement.order_by(model.recorded_at.asc()).limit(limit)).scalars())
|
||||||
|
return ChannelReadingsResponse(
|
||||||
|
items=[ChannelReadingResponse(
|
||||||
|
recorded_at=row.recorded_at,
|
||||||
|
value=getattr(row, "value", None), quality=getattr(row, "quality", None),
|
||||||
|
) for row in rows],
|
||||||
|
total=len(rows),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _discover_channels(db: Session, source: MeterSource) -> list[DiscoverChannelResponse]:
|
||||||
|
"""Return only public, accepted channel metadata for discover responses."""
|
||||||
|
return [
|
||||||
|
DiscoverChannelResponse(
|
||||||
|
uuid=channel.uuid, label=channel.label, unit=channel.unit,
|
||||||
|
latest_value=channel.latest_value, latest_at=channel.latest_at,
|
||||||
|
latest_quality=channel.latest_quality,
|
||||||
|
)
|
||||||
|
for channel in db.execute(
|
||||||
|
select(MeterSourceChannel).where(MeterSourceChannel.source_id == source.id)
|
||||||
|
).scalars()
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/meters/{meter_id}/bindings", response_model=BindingListResponse)
|
||||||
|
def meter_bindings(meter_id: int, db: Session = Depends(get_db),
|
||||||
|
_auth: AuthenticatedSession = Depends(require_session)) -> BindingListResponse:
|
||||||
|
if db.get(Meter, meter_id) is None:
|
||||||
|
raise HTTPException(status_code=404, detail="Meter not found.")
|
||||||
|
items = [binding_response(binding) for binding in list_bindings(db, meter_id=meter_id)]
|
||||||
|
return BindingListResponse(items=items, total=len(items))
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/meters/{meter_id}/bindings", response_model=BindingResponse, status_code=status.HTTP_201_CREATED)
|
||||||
|
def post_meter_binding(meter_id: int, body: BindingCreate, db: Session = Depends(get_db),
|
||||||
|
_auth: AuthenticatedSession = Depends(require_session), _csrf: None = Depends(require_csrf)) -> BindingResponse:
|
||||||
|
channel = db.execute(select(MeterSourceChannel).where(MeterSourceChannel.uuid == body.source_channel_uuid)).scalar_one_or_none()
|
||||||
|
if channel is None:
|
||||||
|
raise HTTPException(status_code=404, detail="Meter source channel not found.")
|
||||||
|
try:
|
||||||
|
binding = create_binding(db, meter_id=meter_id, channel_id=channel.id, started_at=_as_utc(body.started_at),
|
||||||
|
ended_at=_as_utc(body.ended_at) if body.ended_at else None)
|
||||||
|
meter = db.get(Meter, meter_id)
|
||||||
|
db.flush()
|
||||||
|
_recompute_binding_commodity(db, meter.commodity, _as_utc(body.started_at))
|
||||||
|
db.commit()
|
||||||
|
db.refresh(binding)
|
||||||
|
_republish_after_commit(db)
|
||||||
|
return binding_response(binding)
|
||||||
|
except MeterSourceError as exc:
|
||||||
|
db.rollback()
|
||||||
|
raise _binding_error(exc) from exc
|
||||||
|
except Exception:
|
||||||
|
db.rollback()
|
||||||
|
raise
|
||||||
|
|
||||||
|
|
||||||
|
@router.patch("/bindings/{binding_uuid}", response_model=BindingResponse)
|
||||||
|
def patch_binding(binding_uuid: str, body: BindingPatch, db: Session = Depends(get_db),
|
||||||
|
_auth: AuthenticatedSession = Depends(require_session), _csrf: None = Depends(require_csrf)) -> BindingResponse:
|
||||||
|
binding = db.execute(select(MeterSourceBinding).where(MeterSourceBinding.uuid == binding_uuid)).scalar_one_or_none()
|
||||||
|
if binding is None:
|
||||||
|
raise HTTPException(status_code=404, detail="Meter source binding not found.")
|
||||||
|
try:
|
||||||
|
# ``ended_at`` has three meaningful states in the service layer: omitted
|
||||||
|
# keeps the existing boundary, null reopens the interval, and a datetime
|
||||||
|
# changes the exclusive end. Do not collapse omitted into null here.
|
||||||
|
changes: dict[str, datetime | None] = {}
|
||||||
|
if "started_at" in body.model_fields_set:
|
||||||
|
changes["started_at"] = _as_utc(body.started_at) if body.started_at is not None else None
|
||||||
|
if "ended_at" in body.model_fields_set:
|
||||||
|
changes["ended_at"] = _as_utc(body.ended_at) if body.ended_at is not None else None
|
||||||
|
old_started_at = _as_utc(binding.started_at)
|
||||||
|
old_ended_at = _as_utc(binding.ended_at) if binding.ended_at is not None else None
|
||||||
|
updated = update_binding(db, binding.id, **changes)
|
||||||
|
meter = db.get(Meter, updated.meter_id)
|
||||||
|
if "started_at" in changes:
|
||||||
|
earliest = min(old_started_at, _as_utc(updated.started_at))
|
||||||
|
elif "ended_at" in changes:
|
||||||
|
new_ended_at = _as_utc(updated.ended_at) if updated.ended_at is not None else None
|
||||||
|
changed_ends = [value for value in (old_ended_at, new_ended_at) if value is not None]
|
||||||
|
earliest = min(changed_ends) if changed_ends else old_started_at
|
||||||
|
else:
|
||||||
|
earliest = old_started_at
|
||||||
|
db.flush()
|
||||||
|
_recompute_binding_commodity(db, meter.commodity, earliest)
|
||||||
|
db.commit()
|
||||||
|
db.refresh(updated)
|
||||||
|
_republish_after_commit(db)
|
||||||
|
return binding_response(updated)
|
||||||
|
except MeterSourceError as exc:
|
||||||
|
db.rollback()
|
||||||
|
raise _binding_error(exc) from exc
|
||||||
|
except Exception:
|
||||||
|
db.rollback()
|
||||||
|
raise
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/meters/{meter_id}/bindings/transfer", response_model=BindingTransferResponse)
|
||||||
|
def post_binding_transfer(
|
||||||
|
meter_id: int, body: BindingTransferRequest, db: Session = Depends(get_db),
|
||||||
|
_auth: AuthenticatedSession = Depends(require_session), _csrf: None = Depends(require_csrf),
|
||||||
|
) -> BindingTransferResponse:
|
||||||
|
source = db.execute(select(MeterSourceBinding).where(
|
||||||
|
MeterSourceBinding.uuid == body.from_binding_uuid
|
||||||
|
)).scalar_one_or_none()
|
||||||
|
channel = db.execute(select(MeterSourceChannel).where(
|
||||||
|
MeterSourceChannel.uuid == body.to_source_channel_uuid
|
||||||
|
)).scalar_one_or_none()
|
||||||
|
if source is None or channel is None:
|
||||||
|
raise HTTPException(status_code=404, detail="Meter source binding or channel not found.")
|
||||||
|
effective_at = _as_utc(body.effective_at)
|
||||||
|
try:
|
||||||
|
closed, created = transfer_binding(db, target_meter_id=meter_id, from_binding_id=source.id,
|
||||||
|
to_channel_id=channel.id, effective_at=effective_at)
|
||||||
|
meter = db.get(Meter, meter_id)
|
||||||
|
if closed.meter_id == meter.id:
|
||||||
|
earliest = effective_at
|
||||||
|
else:
|
||||||
|
earliest = min(_as_utc(closed.ended_at), effective_at)
|
||||||
|
db.flush()
|
||||||
|
_recompute_binding_commodity(db, meter.commodity, earliest)
|
||||||
|
db.commit()
|
||||||
|
db.refresh(closed)
|
||||||
|
db.refresh(created)
|
||||||
|
except MeterSourceError as exc:
|
||||||
|
db.rollback()
|
||||||
|
raise _binding_error(exc) from exc
|
||||||
|
except Exception:
|
||||||
|
db.rollback()
|
||||||
|
raise
|
||||||
|
_republish_after_commit(db)
|
||||||
|
return BindingTransferResponse(closed_binding=binding_response(closed), created_binding=binding_response(created))
|
||||||
+137
-19
@@ -48,16 +48,27 @@ from typing import Optional
|
|||||||
|
|
||||||
from fastapi import APIRouter, Depends, HTTPException, status
|
from fastapi import APIRouter, Depends, HTTPException, status
|
||||||
from sqlalchemy.orm import Session
|
from sqlalchemy.orm import Session
|
||||||
|
from sqlalchemy import select
|
||||||
|
|
||||||
from app.api.routes.api.deps import require_csrf, require_session
|
from app.api.routes.api.deps import require_csrf, require_session
|
||||||
from app.dependencies import get_db
|
from app.dependencies import get_db
|
||||||
from app.models.energy import Meter
|
from app.models.energy import Meter
|
||||||
|
from app.models.meter_source import MeterSourceChannel
|
||||||
from app.schemas.meter import (
|
from app.schemas.meter import (
|
||||||
|
MeterCloseRequest,
|
||||||
MeterDeclareRequest,
|
MeterDeclareRequest,
|
||||||
|
MeterBindingSummary,
|
||||||
MeterListResponse,
|
MeterListResponse,
|
||||||
MeterPatchRequest,
|
MeterPatchRequest,
|
||||||
MeterResponse,
|
MeterResponse,
|
||||||
)
|
)
|
||||||
|
from app.services.meter_sources import (
|
||||||
|
ChannelNotFoundError,
|
||||||
|
MeterSourceError,
|
||||||
|
create_binding,
|
||||||
|
create_binding_for_meter_swap,
|
||||||
|
close_open_bindings_for_meter,
|
||||||
|
)
|
||||||
from app.services import timezone as _tz_mod
|
from app.services import timezone as _tz_mod
|
||||||
from app.services.auth import AuthenticatedSession
|
from app.services.auth import AuthenticatedSession
|
||||||
from app.services.energy_cost import recompute_range
|
from app.services.energy_cost import recompute_range
|
||||||
@@ -65,6 +76,7 @@ from app.services.meters import (
|
|||||||
MeterIntervalError,
|
MeterIntervalError,
|
||||||
MeterOverlapError,
|
MeterOverlapError,
|
||||||
declare_meter,
|
declare_meter,
|
||||||
|
close_meter,
|
||||||
list_meters,
|
list_meters,
|
||||||
update_meter,
|
update_meter,
|
||||||
)
|
)
|
||||||
@@ -142,7 +154,7 @@ def _trigger_recompute(db: Session, start: datetime, label: str) -> int:
|
|||||||
# started_at is in the future — nothing to recompute.
|
# started_at is in the future — nothing to recompute.
|
||||||
logger.info("%s: started_at (%s) is in the future, skipping recompute.", label, start)
|
logger.info("%s: started_at (%s) is in the future, skipping recompute.", label, start)
|
||||||
return 0
|
return 0
|
||||||
n = recompute_range(db, start, end)
|
n = recompute_range(db, start, end, commit=False, strict=True)
|
||||||
logger.info(
|
logger.info(
|
||||||
"%s: recomputed %d period(s) in window [%s, %s).",
|
"%s: recomputed %d period(s) in window [%s, %s).",
|
||||||
label,
|
label,
|
||||||
@@ -153,6 +165,33 @@ def _trigger_recompute(db: Session, start: datetime, label: str) -> int:
|
|||||||
return n
|
return n
|
||||||
|
|
||||||
|
|
||||||
|
def _recompute_commodity(db: Session, commodity: str, start: datetime, label: str) -> int:
|
||||||
|
if commodity == "electricity":
|
||||||
|
return _trigger_recompute(db, start, label)
|
||||||
|
from app.services.meter_cost import recompute_range as thermal_recompute_range
|
||||||
|
|
||||||
|
end = datetime.now(UTC)
|
||||||
|
if start >= end:
|
||||||
|
return 0
|
||||||
|
return thermal_recompute_range(db, start, end, commit=False)
|
||||||
|
|
||||||
|
|
||||||
|
def _meter_response(meter: Meter) -> MeterResponse:
|
||||||
|
"""Serialize meter plus binding summaries without exposing source config."""
|
||||||
|
response = MeterResponse.model_validate(meter)
|
||||||
|
response.bindings = [
|
||||||
|
MeterBindingSummary(
|
||||||
|
uuid=binding.uuid,
|
||||||
|
source_channel_uuid=binding.channel.uuid,
|
||||||
|
source_uuid=binding.channel.source.uuid,
|
||||||
|
started_at=binding.started_at,
|
||||||
|
ended_at=binding.ended_at,
|
||||||
|
)
|
||||||
|
for binding in meter.source_bindings
|
||||||
|
]
|
||||||
|
return response
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# GET /api/energy/meters
|
# GET /api/energy/meters
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
@@ -170,7 +209,7 @@ def list_energy_meters(
|
|||||||
has the latest ``started_at``.
|
has the latest ``started_at``.
|
||||||
"""
|
"""
|
||||||
meters = list_meters(db)
|
meters = list_meters(db)
|
||||||
items = [MeterResponse.model_validate(m) for m in meters]
|
items = [_meter_response(m) for m in meters]
|
||||||
return MeterListResponse(items=items, total=len(items))
|
return MeterListResponse(items=items, total=len(items))
|
||||||
|
|
||||||
|
|
||||||
@@ -211,6 +250,9 @@ def declare_energy_meter(
|
|||||||
started_at_utc = _localize_started_at(body.started_at)
|
started_at_utc = _localize_started_at(body.started_at)
|
||||||
|
|
||||||
try:
|
try:
|
||||||
|
old_meter = db.execute(
|
||||||
|
select(Meter).where(Meter.commodity == body.commodity, Meter.ended_at.is_(None))
|
||||||
|
).scalar_one_or_none()
|
||||||
new_meter = declare_meter(
|
new_meter = declare_meter(
|
||||||
db,
|
db,
|
||||||
label=body.label,
|
label=body.label,
|
||||||
@@ -219,20 +261,62 @@ def declare_energy_meter(
|
|||||||
commodity=body.commodity,
|
commodity=body.commodity,
|
||||||
note=body.note,
|
note=body.note,
|
||||||
)
|
)
|
||||||
except MeterOverlapError as exc:
|
db.flush() # assign PK before an optional binding and recompute
|
||||||
raise HTTPException(
|
# A closed predecessor must never retain an open interval. For a
|
||||||
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
|
# meter swap with no selected channel we can safely hand off exactly
|
||||||
detail=str(exc),
|
# one compatible open channel; ambiguity is fail-closed.
|
||||||
|
auto_channel = None
|
||||||
|
if body.source_channel_uuid is None and old_meter is not None and body.reason.value == "meter_swap":
|
||||||
|
candidates = [b for b in old_meter.source_bindings if b.ended_at is None and b.channel.unit == {"electricity": "kWh", "heating": "GJ", "hot_water": "m³"}.get(body.commodity)]
|
||||||
|
if len(candidates) > 1:
|
||||||
|
raise MeterSourceError("Meter swap has ambiguous open bindings; select a channel explicitly.")
|
||||||
|
if len(candidates) == 1:
|
||||||
|
auto_channel = candidates[0].channel
|
||||||
|
if body.source_channel_uuid is not None:
|
||||||
|
channel = db.execute(
|
||||||
|
select(MeterSourceChannel).where(MeterSourceChannel.uuid == body.source_channel_uuid)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if channel is None:
|
||||||
|
raise ChannelNotFoundError("Meter source channel was not found.")
|
||||||
|
if body.reason.value == "meter_swap":
|
||||||
|
create_binding_for_meter_swap(
|
||||||
|
db,
|
||||||
|
old_meter_id=old_meter.id if old_meter is not None else None,
|
||||||
|
new_meter_id=new_meter.id,
|
||||||
|
channel_id=channel.id,
|
||||||
|
started_at=started_at_utc,
|
||||||
)
|
)
|
||||||
|
else:
|
||||||
|
create_binding(
|
||||||
|
db,
|
||||||
|
meter_id=new_meter.id,
|
||||||
|
channel_id=channel.id,
|
||||||
|
started_at=started_at_utc,
|
||||||
|
)
|
||||||
|
elif auto_channel is not None:
|
||||||
|
create_binding_for_meter_swap(db, old_meter_id=old_meter.id, new_meter_id=new_meter.id,
|
||||||
|
channel_id=auto_channel.id, started_at=started_at_utc)
|
||||||
|
if old_meter is not None:
|
||||||
|
close_open_bindings_for_meter(db, old_meter.id, ended_at=started_at_utc)
|
||||||
|
|
||||||
db.flush() # assign PK before recompute (recompute uses session, needs meter in DB)
|
# Keep recompute in this transaction: a failure must not leave a new
|
||||||
|
# meter, its predecessor, or either binding at a half-applied boundary.
|
||||||
# Retroactive recompute: re-judge attribution from the new boundary onward.
|
|
||||||
now = datetime.now(UTC)
|
now = datetime.now(UTC)
|
||||||
if started_at_utc < now:
|
if started_at_utc < now:
|
||||||
_trigger_recompute(db, started_at_utc, "POST /api/energy/meters")
|
db.flush()
|
||||||
|
_recompute_commodity(db, body.commodity, started_at_utc, "POST /api/energy/meters")
|
||||||
db.commit()
|
db.commit()
|
||||||
|
except (MeterIntervalError, MeterOverlapError, MeterSourceError) as exc:
|
||||||
|
db.rollback()
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=(status.HTTP_404_NOT_FOUND if isinstance(exc, ChannelNotFoundError)
|
||||||
|
else status.HTTP_422_UNPROCESSABLE_ENTITY),
|
||||||
|
detail=str(exc),
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
db.rollback()
|
||||||
|
raise
|
||||||
|
|
||||||
db.refresh(new_meter)
|
db.refresh(new_meter)
|
||||||
|
|
||||||
# Trigger HA discovery re-publish so the new active meter's energy-cost
|
# Trigger HA discovery re-publish so the new active meter's energy-cost
|
||||||
@@ -247,7 +331,31 @@ def declare_energy_meter(
|
|||||||
new_meter.label,
|
new_meter.label,
|
||||||
started_at_utc.isoformat(),
|
started_at_utc.isoformat(),
|
||||||
)
|
)
|
||||||
return MeterResponse.model_validate(new_meter)
|
return _meter_response(new_meter)
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/meters/{meter_id}/close", response_model=MeterResponse)
|
||||||
|
def close_energy_meter(
|
||||||
|
meter_id: int, body: MeterCloseRequest, db: Session = Depends(get_db),
|
||||||
|
_auth: AuthenticatedSession = Depends(require_session), _csrf: None = Depends(require_csrf),
|
||||||
|
) -> MeterResponse:
|
||||||
|
meter = _get_meter_or_404(db, meter_id)
|
||||||
|
boundary = _localize_started_at(body.ended_at)
|
||||||
|
try:
|
||||||
|
close_meter(db, meter, ended_at=boundary)
|
||||||
|
close_open_bindings_for_meter(db, meter.id, ended_at=boundary)
|
||||||
|
db.flush()
|
||||||
|
_recompute_commodity(db, meter.commodity, boundary, f"POST /api/energy/meters/{meter_id}/close")
|
||||||
|
db.commit()
|
||||||
|
except (MeterIntervalError, MeterSourceError) as exc:
|
||||||
|
db.rollback()
|
||||||
|
raise HTTPException(status_code=422, detail=str(exc)) from exc
|
||||||
|
except Exception:
|
||||||
|
db.rollback()
|
||||||
|
raise
|
||||||
|
db.refresh(meter)
|
||||||
|
_trigger_discovery_republish(db)
|
||||||
|
return _meter_response(meter)
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
@@ -298,11 +406,6 @@ def patch_energy_meter(
|
|||||||
note=body.note,
|
note=body.note,
|
||||||
started_at=new_started_at_utc,
|
started_at=new_started_at_utc,
|
||||||
)
|
)
|
||||||
except MeterIntervalError as exc:
|
|
||||||
raise HTTPException(
|
|
||||||
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
|
|
||||||
detail=str(exc),
|
|
||||||
)
|
|
||||||
|
|
||||||
# Retroactive recompute if started_at was changed.
|
# Retroactive recompute if started_at was changed.
|
||||||
if new_started_at_utc is not None and old_started_at is not None:
|
if new_started_at_utc is not None and old_started_at is not None:
|
||||||
@@ -312,9 +415,24 @@ def patch_energy_meter(
|
|||||||
# Window = [min(old, new), now) — covers all periods whose attribution
|
# Window = [min(old, new), now) — covers all periods whose attribution
|
||||||
# may have changed due to the boundary shift in either direction.
|
# may have changed due to the boundary shift in either direction.
|
||||||
window_start = min(old_started_at, new_started_at_utc)
|
window_start = min(old_started_at, new_started_at_utc)
|
||||||
_trigger_recompute(db, window_start, f"PATCH /api/energy/meters/{meter_id}")
|
db.flush()
|
||||||
|
_recompute_commodity(
|
||||||
|
db,
|
||||||
|
meter.commodity,
|
||||||
|
window_start,
|
||||||
|
f"PATCH /api/energy/meters/{meter_id}",
|
||||||
|
)
|
||||||
|
|
||||||
db.commit()
|
db.commit()
|
||||||
|
except MeterIntervalError as exc:
|
||||||
|
db.rollback()
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
|
||||||
|
detail=str(exc),
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
db.rollback()
|
||||||
|
raise
|
||||||
db.refresh(meter)
|
db.refresh(meter)
|
||||||
|
|
||||||
# Trigger HA discovery re-publish so label renames on the active meter
|
# Trigger HA discovery re-publish so label renames on the active meter
|
||||||
@@ -328,4 +446,4 @@ def patch_energy_meter(
|
|||||||
meter.label,
|
meter.label,
|
||||||
meter.started_at,
|
meter.started_at,
|
||||||
)
|
)
|
||||||
return MeterResponse.model_validate(meter)
|
return _meter_response(meter)
|
||||||
|
|||||||
+14
-1
@@ -1,7 +1,8 @@
|
|||||||
from functools import lru_cache
|
from functools import lru_cache
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
import re
|
||||||
|
|
||||||
from pydantic import computed_field
|
from pydantic import computed_field, field_validator
|
||||||
from pydantic_settings import BaseSettings, SettingsConfigDict
|
from pydantic_settings import BaseSettings, SettingsConfigDict
|
||||||
|
|
||||||
|
|
||||||
@@ -53,6 +54,7 @@ class Settings(BaseSettings):
|
|||||||
mqtt_username: str = ""
|
mqtt_username: str = ""
|
||||||
mqtt_password: str = ""
|
mqtt_password: str = ""
|
||||||
mqtt_tls_enabled: bool = False
|
mqtt_tls_enabled: bool = False
|
||||||
|
mqtt_client_id: str = "home-automation"
|
||||||
|
|
||||||
# Home Assistant MQTT Discovery (T08 wires into CONFIG_FIELDS/UI; T11 does publishing).
|
# Home Assistant MQTT Discovery (T08 wires into CONFIG_FIELDS/UI; T11 does publishing).
|
||||||
ha_discovery_enabled: bool = False
|
ha_discovery_enabled: bool = False
|
||||||
@@ -81,6 +83,17 @@ class Settings(BaseSettings):
|
|||||||
extra="ignore",
|
extra="ignore",
|
||||||
)
|
)
|
||||||
|
|
||||||
|
@field_validator("mqtt_client_id", mode="before")
|
||||||
|
@classmethod
|
||||||
|
def validate_mqtt_client_id(cls, value: object) -> str:
|
||||||
|
"""Normalize a broker-safe base client identity used by every MQTT client."""
|
||||||
|
if not isinstance(value, str):
|
||||||
|
raise ValueError("MQTT client ID must be a string")
|
||||||
|
normalized = value.strip()
|
||||||
|
if not re.fullmatch(r"[A-Za-z0-9][A-Za-z0-9_-]{0,63}", normalized):
|
||||||
|
raise ValueError("MQTT client ID must be a non-empty ASCII slug")
|
||||||
|
return normalized
|
||||||
|
|
||||||
@computed_field
|
@computed_field
|
||||||
@property
|
@property
|
||||||
def is_development(self) -> bool:
|
def is_development(self) -> bool:
|
||||||
|
|||||||
+372
-13
@@ -27,7 +27,7 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import logging
|
import logging
|
||||||
from dataclasses import dataclass, field
|
from dataclasses import dataclass, field
|
||||||
from datetime import timedelta
|
from datetime import UTC, datetime, timedelta
|
||||||
from typing import Any, Callable, Optional, Protocol
|
from typing import Any, Callable, Optional, Protocol
|
||||||
|
|
||||||
from sqlalchemy.orm import Session
|
from sqlalchemy.orm import Session
|
||||||
@@ -45,12 +45,15 @@ class DeviceInfo:
|
|||||||
"""Grouping metadata that maps an entity to a logical HA device.
|
"""Grouping metadata that maps an entity to a logical HA device.
|
||||||
|
|
||||||
``identifiers`` corresponds to ``device.identifiers`` in HA Discovery
|
``identifiers`` corresponds to ``device.identifiers`` in HA Discovery
|
||||||
config — a stable tuple used to group entities under one device card.
|
config. HA merges device cards if *any* identifier overlaps, so every
|
||||||
|
logical device deliberately has exactly one, globally namespaced value.
|
||||||
|
``identity`` is an independent internal seed for MQTT topics and unique
|
||||||
|
ids; it must never be inferred from the HA identifier list.
|
||||||
``name`` is the human-readable device name (friendly_name).
|
``name`` is the human-readable device name (friendly_name).
|
||||||
"""
|
"""
|
||||||
|
|
||||||
identifiers: tuple[str, ...]
|
identifiers: tuple[str, ...]
|
||||||
"""Stable identifiers for HA device grouping (e.g. ``("modbus", "<uuid>")``).
|
"""Stable HA identifiers, normally a one-item tuple.
|
||||||
|
|
||||||
These must not change over the device lifetime; they anchor the HA device
|
These must not change over the device lifetime; they anchor the HA device
|
||||||
card even when friendly_name changes.
|
card even when friendly_name changes.
|
||||||
@@ -59,6 +62,14 @@ class DeviceInfo:
|
|||||||
name: str
|
name: str
|
||||||
"""Human-readable device name (may change; triggers HA discovery re-publish)."""
|
"""Human-readable device name (may change; triggers HA discovery re-publish)."""
|
||||||
|
|
||||||
|
identity: Optional[str] = None
|
||||||
|
"""Stable internal topic/unique-id identity, separate from HA identifiers.
|
||||||
|
|
||||||
|
The fallback preserves compatibility for third-party providers still using
|
||||||
|
the historic ``("kind", "uuid")`` construction while first-party providers
|
||||||
|
use this field explicitly.
|
||||||
|
"""
|
||||||
|
|
||||||
provides_availability: bool = True
|
provides_availability: bool = True
|
||||||
"""Whether this device publishes an availability ("online"/"offline") heartbeat.
|
"""Whether this device publishes an availability ("online"/"offline") heartbeat.
|
||||||
|
|
||||||
@@ -74,6 +85,22 @@ class DeviceInfo:
|
|||||||
their state is being published.
|
their state is being published.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
|
availability_id: Optional[str] = None
|
||||||
|
"""Stable id used for the shared availability topic, when different from
|
||||||
|
this HA device's identity. A Meter is identified by its own UUID, while
|
||||||
|
its liveness comes from the source/channel feeding it.
|
||||||
|
"""
|
||||||
|
|
||||||
|
availability_getter: Optional[Callable[["Session"], bool]] = field(
|
||||||
|
default=None, repr=False
|
||||||
|
)
|
||||||
|
"""Return whether the source behind this device is currently usable."""
|
||||||
|
|
||||||
|
@property
|
||||||
|
def internal_identity(self) -> str:
|
||||||
|
"""Return the topic/unique-id seed without exposing HA grouping details."""
|
||||||
|
return self.identity or self.identifiers[-1]
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
@dataclass
|
||||||
class ExposableEntity:
|
class ExposableEntity:
|
||||||
@@ -265,8 +292,9 @@ def _modbus_provider(session: Session) -> list[ExposableEntity]:
|
|||||||
|
|
||||||
for device in devices:
|
for device in devices:
|
||||||
device_info = DeviceInfo(
|
device_info = DeviceInfo(
|
||||||
identifiers=("modbus", device.uuid),
|
identifiers=(f"home-automation:modbus:{device.uuid}",),
|
||||||
name=device.friendly_name,
|
name=device.friendly_name,
|
||||||
|
identity=device.uuid,
|
||||||
)
|
)
|
||||||
|
|
||||||
# Load the profile to get metric metadata.
|
# Load the profile to get metric metadata.
|
||||||
@@ -396,9 +424,10 @@ def _energy_cost_provider(session: Session) -> list[ExposableEntity]:
|
|||||||
|
|
||||||
HA device identity (换表 → 新 sensor)
|
HA device identity (换表 → 新 sensor)
|
||||||
--------------------------------------
|
--------------------------------------
|
||||||
``identifiers[1]`` is set to the active meter's **uuid** (not the fixed
|
The independent internal identity is set to the active meter's **uuid**.
|
||||||
string ``"energy-cost"``). ``ha_discovery.py`` uses ``identifiers[1]`` as
|
``ha_discovery.py`` uses that internal identity as the MQTT node_id and as
|
||||||
the MQTT node_id and as part of the ``unique_id`` for every entity.
|
part of the ``unique_id`` for every entity; HA receives one separate,
|
||||||
|
namespaced device identifier.
|
||||||
Declaring a new active electricity meter produces a new uuid → new node_id /
|
Declaring a new active electricity meter produces a new uuid → new node_id /
|
||||||
unique_id → HA creates a brand-new sensor, cleanly isolating post-swap data.
|
unique_id → HA creates a brand-new sensor, cleanly isolating post-swap data.
|
||||||
|
|
||||||
@@ -449,9 +478,9 @@ def _energy_cost_provider(session: Session) -> list[ExposableEntity]:
|
|||||||
|
|
||||||
DeviceInfo identifiers
|
DeviceInfo identifiers
|
||||||
----------------------
|
----------------------
|
||||||
**Two-element tuple** ``("energy-cost", meter.uuid)`` so that
|
One-element namespaced tuple ``("home-automation:energy-cost:<uuid>",)``.
|
||||||
``ha_discovery.py``'s ``entity.device.identifiers[1]`` resolves to the
|
The meter uuid used for MQTT topics and unique IDs is carried independently
|
||||||
meter uuid (used as the MQTT node_id and unique_id seed throughout).
|
in ``DeviceInfo.identity``.
|
||||||
"""
|
"""
|
||||||
from app.models.energy import EnergyCostPeriod, Meter # local import to avoid circular
|
from app.models.energy import EnergyCostPeriod, Meter # local import to avoid circular
|
||||||
from sqlalchemy import select
|
from sqlalchemy import select
|
||||||
@@ -488,14 +517,15 @@ def _energy_cost_provider(session: Session) -> list[ExposableEntity]:
|
|||||||
currency = latest_period.currency
|
currency = latest_period.currency
|
||||||
|
|
||||||
# --- Shared DeviceInfo anchored to the active meter's uuid ---
|
# --- Shared DeviceInfo anchored to the active meter's uuid ---
|
||||||
# identifiers[1] = meter.uuid drives the MQTT node_id and unique_id in
|
# The internal identity is meter.uuid, while the HA identifier is a single
|
||||||
# ha_discovery.py. Swapping the meter produces a new uuid → new HA sensor.
|
# namespaced value. Swapping the meter produces a new HA sensor/card.
|
||||||
# provides_availability=False: the energy-cost device has only sensors and no
|
# provides_availability=False: the energy-cost device has only sensors and no
|
||||||
# online/offline heartbeat, so its entities must be "always available" in HA.
|
# online/offline heartbeat, so its entities must be "always available" in HA.
|
||||||
# (Otherwise HA shows them unavailable despite state being published.)
|
# (Otherwise HA shows them unavailable despite state being published.)
|
||||||
device_info = DeviceInfo(
|
device_info = DeviceInfo(
|
||||||
identifiers=("energy-cost", active_meter.uuid),
|
identifiers=(f"home-automation:energy-cost:{active_meter.uuid}",),
|
||||||
name=active_meter.label,
|
name=active_meter.label,
|
||||||
|
identity=active_meter.uuid,
|
||||||
provides_availability=False,
|
provides_availability=False,
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -877,3 +907,332 @@ def _energy_cost_provider(session: Session) -> list[ExposableEntity]:
|
|||||||
|
|
||||||
# Register the energy cost provider at module load time.
|
# Register the energy cost provider at module load time.
|
||||||
register_provider(_energy_cost_provider)
|
register_provider(_energy_cost_provider)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# M8 source / meter / thermal-cost provider
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
_SOURCE_STALE_AFTER = timedelta(minutes=5)
|
||||||
|
|
||||||
|
|
||||||
|
def _utc_now() -> datetime:
|
||||||
|
"""Small clock seam for live-value bounds and deterministic tests."""
|
||||||
|
return datetime.now(UTC)
|
||||||
|
|
||||||
|
|
||||||
|
def _as_utc(value: datetime) -> datetime:
|
||||||
|
return value.replace(tzinfo=UTC) if value.tzinfo is None else value.astimezone(UTC)
|
||||||
|
|
||||||
|
|
||||||
|
def _source_is_online(source: Any, channel: Any | None = None) -> bool:
|
||||||
|
"""Do not turn an old cumulative value into a plausible live HA state."""
|
||||||
|
now = _utc_now()
|
||||||
|
if not source.enabled or source.status != "online" or source.last_seen_at is None:
|
||||||
|
return False
|
||||||
|
source_age = now - _as_utc(source.last_seen_at)
|
||||||
|
if not timedelta(0) <= source_age <= _SOURCE_STALE_AFTER:
|
||||||
|
return False
|
||||||
|
if channel is None:
|
||||||
|
return True
|
||||||
|
if channel.latest_at is None or channel.latest_quality not in {"valid", "unverifiable"}:
|
||||||
|
return False
|
||||||
|
channel_age = now - _as_utc(channel.latest_at)
|
||||||
|
return timedelta(0) <= channel_age <= _SOURCE_STALE_AFTER
|
||||||
|
|
||||||
|
|
||||||
|
def _dsmr_latest(session: Session, source_id: int, *, start: datetime | None = None,
|
||||||
|
end: datetime | None = None, not_after: datetime | None = None) -> Any:
|
||||||
|
"""Latest DSMR row in the source's (optionally bounded) cumulative domain."""
|
||||||
|
from app.models.energy import DsmrReading
|
||||||
|
from sqlalchemy import select
|
||||||
|
|
||||||
|
query = select(DsmrReading).where(DsmrReading.meter_source_id == source_id)
|
||||||
|
if start is not None:
|
||||||
|
query = query.where(DsmrReading.recorded_at >= start)
|
||||||
|
if end is not None:
|
||||||
|
query = query.where(DsmrReading.recorded_at < end)
|
||||||
|
if not_after is not None:
|
||||||
|
query = query.where(DsmrReading.recorded_at <= not_after)
|
||||||
|
return session.execute(query.order_by(DsmrReading.recorded_at.desc()).limit(1)).scalar_one_or_none()
|
||||||
|
|
||||||
|
|
||||||
|
def _dsmr_total(reading: Any) -> Any:
|
||||||
|
"""Return imported electricity total from a real DSMR telegram, or None."""
|
||||||
|
from decimal import Decimal, InvalidOperation
|
||||||
|
|
||||||
|
try:
|
||||||
|
payload = reading.payload or {}
|
||||||
|
return Decimal(str(payload["electricity_delivered_1"])) + Decimal(
|
||||||
|
str(payload["electricity_delivered_2"])
|
||||||
|
)
|
||||||
|
except (InvalidOperation, KeyError, TypeError, ValueError):
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _m8_energy_provider(session: Session) -> list[ExposableEntity]:
|
||||||
|
"""Expose accepted source snapshots and current M8 meters.
|
||||||
|
|
||||||
|
The provider intentionally reads the thermal service's public ``summarize``
|
||||||
|
result for money. Keeping formulas in ``meter_cost`` prevents HA from
|
||||||
|
becoming a second, subtly different billing implementation.
|
||||||
|
"""
|
||||||
|
from app.models.energy import Meter
|
||||||
|
from app.models.meter_source import MeterSource, MeterSourceBinding, MeterSourceChannel
|
||||||
|
from sqlalchemy import select
|
||||||
|
|
||||||
|
sources = session.execute(select(MeterSource)).scalars().all()
|
||||||
|
entities: list[ExposableEntity] = []
|
||||||
|
|
||||||
|
for source in sources:
|
||||||
|
source_info = DeviceInfo(
|
||||||
|
identifiers=(f"home-automation:meter-source:{source.uuid}",), name=source.name,
|
||||||
|
identity=source.uuid,
|
||||||
|
availability_id=source.uuid,
|
||||||
|
availability_getter=lambda sess, source_id=source.id: _source_online_by_id(sess, source_id),
|
||||||
|
)
|
||||||
|
entities.append(ExposableEntity(
|
||||||
|
key=f"source.{source.uuid}.online", component="binary_sensor", device=source_info,
|
||||||
|
device_class="connectivity", unit="", name=f"{source.name} Online",
|
||||||
|
value_getter=lambda sess, source_id=source.id: "ON" if _source_online_by_id(sess, source_id) else "OFF",
|
||||||
|
))
|
||||||
|
|
||||||
|
active_meters = session.execute(
|
||||||
|
select(Meter).where(
|
||||||
|
Meter.ended_at.is_(None), Meter.commodity.in_(("electricity", "heating", "hot_water"))
|
||||||
|
)
|
||||||
|
).scalars().all()
|
||||||
|
active_by_commodity = {meter.commodity: meter for meter in active_meters}
|
||||||
|
for meter in active_meters:
|
||||||
|
bound = session.execute(
|
||||||
|
select(MeterSourceBinding, MeterSourceChannel, MeterSource)
|
||||||
|
.join(MeterSourceChannel, MeterSourceChannel.id == MeterSourceBinding.channel_id)
|
||||||
|
.join(MeterSource, MeterSource.id == MeterSourceChannel.source_id)
|
||||||
|
.where(MeterSourceBinding.meter_id == meter.id, MeterSourceBinding.ended_at.is_(None))
|
||||||
|
).one_or_none()
|
||||||
|
if bound is None:
|
||||||
|
continue
|
||||||
|
binding, channel, source = bound
|
||||||
|
info = DeviceInfo(
|
||||||
|
identifiers=(f"home-automation:meter:{meter.uuid}",), name=meter.label,
|
||||||
|
identity=meter.uuid,
|
||||||
|
# Keep this opaque and Meter-anchored. In particular, do not use a
|
||||||
|
# source UUID here: two channels of one source can be independently
|
||||||
|
# stale/invalid and must not overwrite each other's availability.
|
||||||
|
availability_id=f"meter-availability-{meter.uuid}",
|
||||||
|
availability_getter=lambda sess, source_id=source.id, channel_id=channel.id:
|
||||||
|
_bound_channel_online(sess, source_id, channel_id),
|
||||||
|
)
|
||||||
|
if meter.commodity == "electricity":
|
||||||
|
unit, device_class = "kWh", "energy"
|
||||||
|
elif meter.commodity == "heating":
|
||||||
|
unit, device_class = "GJ", "energy"
|
||||||
|
else:
|
||||||
|
unit, device_class = "m³", "water"
|
||||||
|
for suffix, getter in (
|
||||||
|
("total", _meter_total_getter(binding.id, source.id, channel.id)),
|
||||||
|
("today", _meter_today_getter(binding.id, source.id, channel.id)),
|
||||||
|
):
|
||||||
|
entities.append(ExposableEntity(
|
||||||
|
key=f"meter.{meter.uuid}.{suffix}", component="sensor", device=info,
|
||||||
|
device_class=device_class, unit=unit,
|
||||||
|
name=f"{meter.label} {suffix.title()}", value_getter=getter,
|
||||||
|
state_class="total_increasing",
|
||||||
|
))
|
||||||
|
|
||||||
|
heating, hot_water = active_by_commodity.get("heating"), active_by_commodity.get("hot_water")
|
||||||
|
if heating is not None and hot_water is not None:
|
||||||
|
# Keep the v1.6.1 canonical identity/key seed stable. Discovery topic
|
||||||
|
# segments are sanitised independently by ha_discovery, so this dot is
|
||||||
|
# never emitted in a topic while existing toggle rows remain usable.
|
||||||
|
identity = ".".join(sorted((heating.uuid, hot_water.uuid)))
|
||||||
|
currency = _thermal_currency(session)
|
||||||
|
cost_info = DeviceInfo(
|
||||||
|
identifiers=(f"home-automation:thermal-cost:{identity}",), name="Thermal Energy Cost",
|
||||||
|
identity=identity,
|
||||||
|
provides_availability=False,
|
||||||
|
)
|
||||||
|
labels = {
|
||||||
|
"heating": "Heating", "hot_water_heating": "Hot Water Heating", "water": "Water",
|
||||||
|
"water_tax": "Water Tax", "hot_water_total": "Hot Water",
|
||||||
|
"fixed": "Fixed", "all_in": "All-in",
|
||||||
|
}
|
||||||
|
for suffix, window in (("total", None), ("today", "today")):
|
||||||
|
for metric, label in labels.items():
|
||||||
|
entities.append(ExposableEntity(
|
||||||
|
key=f"thermal_cost.{identity}.{metric}_{suffix}", component="sensor", device=cost_info,
|
||||||
|
device_class="monetary", unit=currency, name=f"Thermal {label} {suffix.title()}",
|
||||||
|
value_getter=_thermal_cost_getter(metric, window),
|
||||||
|
state_class="total" if suffix == "total" else "total_increasing",
|
||||||
|
))
|
||||||
|
return entities
|
||||||
|
|
||||||
|
|
||||||
|
def _source_online_by_id(session: Session, source_id: int) -> bool:
|
||||||
|
from app.models.meter_source import MeterSource
|
||||||
|
source = session.get(MeterSource, source_id)
|
||||||
|
if source is not None and source.kind == "dsmr_mqtt":
|
||||||
|
now = _utc_now()
|
||||||
|
# Inspect the actual latest telegram before calculating freshness: a
|
||||||
|
# clock-skewed future telegram must not make an older one look live.
|
||||||
|
latest = _dsmr_latest(session, source_id)
|
||||||
|
if not source.enabled or latest is None:
|
||||||
|
return False
|
||||||
|
age = now - _as_utc(latest.recorded_at)
|
||||||
|
return timedelta(0) <= age <= _SOURCE_STALE_AFTER
|
||||||
|
return source is not None and _source_is_online(source)
|
||||||
|
|
||||||
|
|
||||||
|
def _bound_channel_online(session: Session, source_id: int, channel_id: int) -> bool:
|
||||||
|
from app.models.meter_source import MeterSource, MeterSourceChannel
|
||||||
|
source, channel = session.get(MeterSource, source_id), session.get(MeterSourceChannel, channel_id)
|
||||||
|
if source is not None and source.kind == "dsmr_mqtt":
|
||||||
|
return _source_online_by_id(session, source_id)
|
||||||
|
return source is not None and channel is not None and _source_is_online(source, channel)
|
||||||
|
|
||||||
|
|
||||||
|
def _meter_total_getter(binding_id: int, source_id: int, channel_id: int) -> Callable[[Session], Any]:
|
||||||
|
def _getter(session: Session) -> Any:
|
||||||
|
from app.models.energy import Meter
|
||||||
|
from app.models.meter_source import MeterSource, MeterSourceBinding, MeterSourceChannel
|
||||||
|
if not _bound_channel_online(session, source_id, channel_id):
|
||||||
|
return None
|
||||||
|
binding = session.get(MeterSourceBinding, binding_id)
|
||||||
|
if (
|
||||||
|
binding is None or binding.ended_at is not None or binding.channel_id != channel_id
|
||||||
|
or (meter := session.get(Meter, binding.meter_id)) is None or meter.ended_at is not None
|
||||||
|
):
|
||||||
|
return None
|
||||||
|
source = session.get(MeterSource, source_id)
|
||||||
|
channel = session.get(MeterSourceChannel, channel_id)
|
||||||
|
if source is None or channel is None:
|
||||||
|
return None
|
||||||
|
start = max(_as_utc(meter.started_at), _as_utc(binding.started_at))
|
||||||
|
# Both windows are half-open. The current records have no end, but
|
||||||
|
# retaining this form makes a future close fail safely.
|
||||||
|
end = min(
|
||||||
|
(value for value in (_as_utc(meter.ended_at) if meter.ended_at else None,
|
||||||
|
_as_utc(binding.ended_at) if binding.ended_at else None) if value is not None),
|
||||||
|
default=None,
|
||||||
|
)
|
||||||
|
now = _utc_now()
|
||||||
|
if source.kind == "dsmr_mqtt":
|
||||||
|
latest = _dsmr_latest(session, source_id, start=start, end=end, not_after=now)
|
||||||
|
if latest is None:
|
||||||
|
return None
|
||||||
|
return _dsmr_total(latest)
|
||||||
|
if channel.latest_at is None or channel.latest_quality not in {"valid", "unverifiable"}:
|
||||||
|
return None
|
||||||
|
latest_at = _as_utc(channel.latest_at)
|
||||||
|
if latest_at < start or latest_at > now or (end is not None and latest_at >= end):
|
||||||
|
return None
|
||||||
|
return channel.latest_value
|
||||||
|
return _getter
|
||||||
|
|
||||||
|
|
||||||
|
def _meter_today_getter(binding_id: int, source_id: int, channel_id: int) -> Callable[[Session], Any]:
|
||||||
|
def _getter(session: Session) -> Any:
|
||||||
|
from app.models.energy import Meter
|
||||||
|
from app.models.meter_source import MeterSource, MeterSourceBinding, WarmteLinkReading
|
||||||
|
from app.services import timezone as tz
|
||||||
|
from sqlalchemy import select
|
||||||
|
binding = session.get(MeterSourceBinding, binding_id)
|
||||||
|
if binding is None or binding.ended_at is not None or binding.channel_id != channel_id:
|
||||||
|
return None
|
||||||
|
meter = session.get(Meter, binding.meter_id)
|
||||||
|
if meter is None or meter.ended_at is not None:
|
||||||
|
return None
|
||||||
|
now = _utc_now()
|
||||||
|
day = (tz.local_now() - _TODAY_RESET_GRACE).date()
|
||||||
|
start = max(tz.local_midnight_utc(day), _as_utc(meter.started_at), _as_utc(binding.started_at))
|
||||||
|
bounds = [tz.local_midnight_utc(day + timedelta(days=1))]
|
||||||
|
bounds.extend(value for value in (
|
||||||
|
_as_utc(meter.ended_at) if meter.ended_at else None,
|
||||||
|
_as_utc(binding.ended_at) if binding.ended_at else None,
|
||||||
|
) if value is not None)
|
||||||
|
end = min(bounds)
|
||||||
|
source = session.get(MeterSource, source_id)
|
||||||
|
if source is None or not source.enabled:
|
||||||
|
return None
|
||||||
|
if source.kind != "dsmr_mqtt" and source.status != "online":
|
||||||
|
return None
|
||||||
|
if source.kind == "dsmr_mqtt":
|
||||||
|
first = _dsmr_latest(session, source_id, start=start, end=end, not_after=now)
|
||||||
|
if first is None:
|
||||||
|
return None
|
||||||
|
from app.models.energy import DsmrReading
|
||||||
|
from sqlalchemy import select as dsmr_select
|
||||||
|
rows = session.execute(dsmr_select(DsmrReading).where(
|
||||||
|
DsmrReading.meter_source_id == source_id, DsmrReading.recorded_at >= start,
|
||||||
|
DsmrReading.recorded_at < end,
|
||||||
|
DsmrReading.recorded_at <= now,
|
||||||
|
).order_by(DsmrReading.recorded_at)).scalars().all()
|
||||||
|
if len(rows) < 2:
|
||||||
|
return None
|
||||||
|
first_total, last_total = _dsmr_total(rows[0]), _dsmr_total(rows[-1])
|
||||||
|
if first_total is None or last_total is None:
|
||||||
|
return None
|
||||||
|
value = last_total - first_total
|
||||||
|
return value if value >= 0 else None
|
||||||
|
query = select(WarmteLinkReading).where(
|
||||||
|
WarmteLinkReading.channel_id == channel_id,
|
||||||
|
WarmteLinkReading.recorded_at >= start,
|
||||||
|
WarmteLinkReading.quality.in_(("valid", "unverifiable")),
|
||||||
|
)
|
||||||
|
if end is not None:
|
||||||
|
query = query.where(WarmteLinkReading.recorded_at < end)
|
||||||
|
query = query.where(WarmteLinkReading.recorded_at <= now)
|
||||||
|
readings = session.execute(query.order_by(WarmteLinkReading.recorded_at)).scalars().all()
|
||||||
|
if len(readings) < 2:
|
||||||
|
return None
|
||||||
|
value = readings[-1].value - readings[0].value
|
||||||
|
return value if value >= 0 else None
|
||||||
|
return _getter
|
||||||
|
|
||||||
|
|
||||||
|
def _thermal_currency(session: Session) -> str:
|
||||||
|
from app.services.contracts import active_contract_versions
|
||||||
|
versions = active_contract_versions(session, scope="thermal")
|
||||||
|
return versions[-1].contract.currency if versions else "EUR"
|
||||||
|
|
||||||
|
|
||||||
|
def _thermal_cost_getter(metric: str, window: str | None) -> Callable[[Session], Any]:
|
||||||
|
def _getter(session: Session) -> Any:
|
||||||
|
from app.services import timezone as tz
|
||||||
|
from app.services.meter_cost import summarize
|
||||||
|
now = _utc_now()
|
||||||
|
from app.models.energy import Meter
|
||||||
|
meters = session.query(Meter).filter(
|
||||||
|
Meter.commodity.in_(("heating", "hot_water")), Meter.ended_at.is_(None)
|
||||||
|
).all()
|
||||||
|
if len(meters) != 2:
|
||||||
|
return None
|
||||||
|
epoch_start = max(_as_utc(m.started_at) for m in meters)
|
||||||
|
if window == "today":
|
||||||
|
day = (tz.local_now() - _TODAY_RESET_GRACE).date()
|
||||||
|
start = max(tz.local_midnight_utc(day), epoch_start)
|
||||||
|
# During the reset grace ``day`` is yesterday, whose local midnight
|
||||||
|
# remains the cap; otherwise do not summarize readings from later today.
|
||||||
|
end = min(tz.local_midnight_utc(day + timedelta(days=1)), now)
|
||||||
|
else:
|
||||||
|
# A combined thermal identity begins when its newest constituent
|
||||||
|
# meter epoch begins; including pre-swap rows would mix identities.
|
||||||
|
start, end = epoch_start, now
|
||||||
|
result = summarize(session, start, end, now=now)
|
||||||
|
if result["period_count"] == 0 and result["fixed_cost"] == 0:
|
||||||
|
return None
|
||||||
|
if metric == "fixed":
|
||||||
|
return result["fixed_cost"]
|
||||||
|
if metric == "all_in":
|
||||||
|
return result["total_cost"]
|
||||||
|
if metric == "water":
|
||||||
|
return result["breakdown"]["hot_water"]
|
||||||
|
if metric == "water_tax":
|
||||||
|
return result["breakdown"]["hot_water_tax"]
|
||||||
|
if metric == "hot_water_total":
|
||||||
|
return result["breakdown"]["hot_water_heating"] + result["breakdown"]["hot_water"]
|
||||||
|
return result["breakdown"][metric]
|
||||||
|
return _getter
|
||||||
|
|
||||||
|
|
||||||
|
register_provider(_m8_energy_provider)
|
||||||
|
|||||||
@@ -2,10 +2,14 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import json
|
import json
|
||||||
import logging
|
import logging
|
||||||
|
from time import monotonic
|
||||||
from dataclasses import dataclass, field
|
from dataclasses import dataclass, field
|
||||||
from typing import Any
|
from typing import Any
|
||||||
from urllib import error, parse, request
|
from urllib import error, parse, request
|
||||||
|
|
||||||
|
from websockets.exceptions import WebSocketException
|
||||||
|
from websockets.sync.client import connect
|
||||||
|
|
||||||
from app.config import Settings
|
from app.config import Settings
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
@@ -57,6 +61,81 @@ class HomeAssistantClient:
|
|||||||
|
|
||||||
self._post_json(f"/api/webhook/{webhook_id}", body, operation="trigger_webhook")
|
self._post_json(f"/api/webhook/{webhook_id}", body, operation="trigger_webhook")
|
||||||
|
|
||||||
|
def discovery_registry_bindings(self, unique_ids: set[str]) -> dict[str, set[str]]:
|
||||||
|
"""Return HA device identifiers currently bound to MQTT unique IDs.
|
||||||
|
|
||||||
|
This is deliberately a read-only WebSocket query. MQTT only confirms
|
||||||
|
broker receipt; the entity/device registries are the authoritative HA
|
||||||
|
observation that a discovery unload/re-add was actually processed.
|
||||||
|
"""
|
||||||
|
self._require_config()
|
||||||
|
if not unique_ids:
|
||||||
|
return {}
|
||||||
|
try:
|
||||||
|
deadline = monotonic() + self.timeout_seconds
|
||||||
|
with connect(self._websocket_url(), open_timeout=self.timeout_seconds,
|
||||||
|
close_timeout=self.timeout_seconds) as websocket:
|
||||||
|
greeting = json.loads(self._websocket_recv(websocket, deadline))
|
||||||
|
if greeting.get("type") != "auth_required":
|
||||||
|
raise HomeAssistantRequestError("Unexpected Home Assistant WebSocket greeting")
|
||||||
|
websocket.send(json.dumps({"type": "auth", "access_token": self.settings.home_assistant_auth_token}))
|
||||||
|
auth = json.loads(self._websocket_recv(websocket, deadline))
|
||||||
|
if auth.get("type") != "auth_ok":
|
||||||
|
raise HomeAssistantRequestError("Home Assistant WebSocket authentication failed")
|
||||||
|
entities = self._websocket_command(websocket, 1, "config/entity_registry/list", deadline)
|
||||||
|
devices = self._websocket_command(websocket, 2, "config/device_registry/list", deadline)
|
||||||
|
except (OSError, WebSocketException, TimeoutError, ValueError, KeyError, TypeError) as exc:
|
||||||
|
raise HomeAssistantRequestError("Home Assistant registry query failed") from exc
|
||||||
|
|
||||||
|
devices_by_id = {
|
||||||
|
device["id"]: {
|
||||||
|
identifier[1]
|
||||||
|
for identifier in device.get("identifiers", [])
|
||||||
|
if (
|
||||||
|
isinstance(identifier, (list, tuple))
|
||||||
|
and len(identifier) == 2
|
||||||
|
and identifier[0] == "mqtt"
|
||||||
|
and isinstance(identifier[1], str)
|
||||||
|
and identifier[1]
|
||||||
|
)
|
||||||
|
}
|
||||||
|
for device in devices
|
||||||
|
if isinstance(device, dict) and isinstance(device.get("id"), str)
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
entity["unique_id"]: devices_by_id.get(entity.get("device_id"), set())
|
||||||
|
for entity in entities
|
||||||
|
if entity.get("platform") == "mqtt" and entity.get("unique_id") in unique_ids
|
||||||
|
}
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _websocket_recv(websocket: Any, deadline: float) -> str:
|
||||||
|
remaining = deadline - monotonic()
|
||||||
|
if remaining <= 0:
|
||||||
|
raise TimeoutError("Home Assistant WebSocket registry query timed out")
|
||||||
|
return websocket.recv(timeout=remaining)
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def _websocket_command(
|
||||||
|
cls, websocket: Any, message_id: int, command: str, deadline: float
|
||||||
|
) -> list[dict[str, Any]]:
|
||||||
|
websocket.send(json.dumps({"id": message_id, "type": command}))
|
||||||
|
while True:
|
||||||
|
response = json.loads(cls._websocket_recv(websocket, deadline))
|
||||||
|
if response.get("id") != message_id:
|
||||||
|
continue
|
||||||
|
if not response.get("success"):
|
||||||
|
raise HomeAssistantRequestError(f"Home Assistant WebSocket {command} failed")
|
||||||
|
return response.get("result", [])
|
||||||
|
|
||||||
|
def _websocket_url(self) -> str:
|
||||||
|
parsed = parse.urlsplit(self.settings.home_assistant_base_url)
|
||||||
|
if parsed.scheme not in {"http", "https"} or not parsed.netloc:
|
||||||
|
raise HomeAssistantConfigError("HOME_ASSISTANT_BASE_URL must be an HTTP(S) URL")
|
||||||
|
scheme = "wss" if parsed.scheme == "https" else "ws"
|
||||||
|
path = f"{parsed.path.rstrip('/')}/api/websocket"
|
||||||
|
return parse.urlunsplit((scheme, parsed.netloc, path, "", ""))
|
||||||
|
|
||||||
def _require_config(self) -> None:
|
def _require_config(self) -> None:
|
||||||
if self.is_configured():
|
if self.is_configured():
|
||||||
return
|
return
|
||||||
|
|||||||
@@ -0,0 +1,173 @@
|
|||||||
|
"""Registry and configuration helpers for meter-source integrations.
|
||||||
|
|
||||||
|
The registry is deliberately I/O-free. Workers and HTTP handlers use these
|
||||||
|
helpers to share one config contract without opening a broker or serial port.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
|
||||||
|
SECRET_MASK = ""
|
||||||
|
|
||||||
|
|
||||||
|
class SourceProfileError(ValueError):
|
||||||
|
"""Raised when a source kind or its configuration is invalid."""
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class SourceConfigField:
|
||||||
|
"""One source configuration field and its public metadata."""
|
||||||
|
|
||||||
|
name: str
|
||||||
|
value_type: type
|
||||||
|
default: Any = None
|
||||||
|
required: bool = False
|
||||||
|
secret: bool = False
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class MeterSourceProfile:
|
||||||
|
"""A supported source kind's config, capabilities, and channel units."""
|
||||||
|
|
||||||
|
kind: str
|
||||||
|
fields: tuple[SourceConfigField, ...]
|
||||||
|
capabilities: frozenset[str]
|
||||||
|
allowed_units: frozenset[str]
|
||||||
|
|
||||||
|
|
||||||
|
DSMR_MQTT_PROFILE = MeterSourceProfile(
|
||||||
|
kind="dsmr_mqtt",
|
||||||
|
fields=(
|
||||||
|
SourceConfigField("broker_host", str, default=""),
|
||||||
|
SourceConfigField("broker_port", int, default=1883),
|
||||||
|
SourceConfigField("username", str, default="", secret=True),
|
||||||
|
SourceConfigField("password", str, default="", secret=True),
|
||||||
|
SourceConfigField("tls_enabled", bool, default=False),
|
||||||
|
SourceConfigField("topic", str, default="dsmr/json"),
|
||||||
|
SourceConfigField("tariff_topic", str, default="dsmr/meter-stats/electricity_tariff"),
|
||||||
|
SourceConfigField("sample_interval_s", int, default=10),
|
||||||
|
),
|
||||||
|
capabilities=frozenset({"discover", "mqtt_subscribe", "tariff"}),
|
||||||
|
allowed_units=frozenset({"kWh"}),
|
||||||
|
)
|
||||||
|
|
||||||
|
WARMTELINK_SERIAL_PROFILE = MeterSourceProfile(
|
||||||
|
kind="warmtelink_serial",
|
||||||
|
fields=(
|
||||||
|
SourceConfigField("path", str, required=True),
|
||||||
|
SourceConfigField("baudrate", int, default=115200),
|
||||||
|
SourceConfigField("data_bits", int, default=7),
|
||||||
|
SourceConfigField("parity", str, default="N"),
|
||||||
|
SourceConfigField("stop_bits", int, default=1),
|
||||||
|
),
|
||||||
|
capabilities=frozenset({"discover", "read_only_serial"}),
|
||||||
|
allowed_units=frozenset({"GJ", "m³"}),
|
||||||
|
)
|
||||||
|
|
||||||
|
SOURCE_PROFILES: dict[str, MeterSourceProfile] = {
|
||||||
|
DSMR_MQTT_PROFILE.kind: DSMR_MQTT_PROFILE,
|
||||||
|
WARMTELINK_SERIAL_PROFILE.kind: WARMTELINK_SERIAL_PROFILE,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def get_source_profile(kind: str) -> MeterSourceProfile:
|
||||||
|
"""Return the profile for *kind*, or raise a stable validation error."""
|
||||||
|
try:
|
||||||
|
return SOURCE_PROFILES[kind]
|
||||||
|
except KeyError as exc:
|
||||||
|
raise SourceProfileError(f"Unsupported meter source kind: {kind!r}") from exc
|
||||||
|
|
||||||
|
|
||||||
|
def list_source_profiles() -> list[MeterSourceProfile]:
|
||||||
|
"""Return profiles in deterministic kind order for a future API/UI."""
|
||||||
|
return [SOURCE_PROFILES[kind] for kind in sorted(SOURCE_PROFILES)]
|
||||||
|
|
||||||
|
|
||||||
|
def _check_type(field: SourceConfigField, value: Any) -> None:
|
||||||
|
# bool is a subclass of int; accept it only for explicitly boolean fields.
|
||||||
|
if type(value) is not field.value_type:
|
||||||
|
raise SourceProfileError(
|
||||||
|
f"Config field {field.name!r} must be a {field.value_type.__name__}."
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _validate_field_value(kind: str, field: SourceConfigField, value: Any) -> None:
|
||||||
|
_check_type(field, value)
|
||||||
|
if field.name == "path" and not value.startswith("/dev/"):
|
||||||
|
raise SourceProfileError("warmtelink_serial config path must start with '/dev/'.")
|
||||||
|
if field.name in {"broker_port", "baudrate"} and value <= 0:
|
||||||
|
raise SourceProfileError(f"Config field {field.name!r} must be greater than zero.")
|
||||||
|
if field.name == "sample_interval_s" and value < 0:
|
||||||
|
raise SourceProfileError("Config field 'sample_interval_s' must not be negative.")
|
||||||
|
if field.name == "data_bits" and value != 7:
|
||||||
|
raise SourceProfileError("warmtelink_serial data_bits must be 7.")
|
||||||
|
if field.name == "parity" and value != "N":
|
||||||
|
raise SourceProfileError("warmtelink_serial parity must be 'N'.")
|
||||||
|
if field.name == "stop_bits" and value != 1:
|
||||||
|
raise SourceProfileError("warmtelink_serial stop_bits must be 1.")
|
||||||
|
|
||||||
|
|
||||||
|
def validate_source_config(kind: str, config: dict[str, Any]) -> dict[str, Any]:
|
||||||
|
"""Validate a complete config and return it with profile defaults filled.
|
||||||
|
|
||||||
|
Unknown keys are rejected to make configuration additions explicit. Secret
|
||||||
|
masking is intentionally not interpreted here: callers must merge a PATCH
|
||||||
|
with its stored config first.
|
||||||
|
"""
|
||||||
|
profile = get_source_profile(kind)
|
||||||
|
if not isinstance(config, dict):
|
||||||
|
raise SourceProfileError("Source config must be an object.")
|
||||||
|
fields = {field.name: field for field in profile.fields}
|
||||||
|
unknown = set(config) - set(fields)
|
||||||
|
if unknown:
|
||||||
|
raise SourceProfileError(f"Unknown {kind} config field(s): {sorted(unknown)!r}")
|
||||||
|
|
||||||
|
validated: dict[str, Any] = {}
|
||||||
|
for field in profile.fields:
|
||||||
|
if field.name in config:
|
||||||
|
value = config[field.name]
|
||||||
|
elif field.required:
|
||||||
|
raise SourceProfileError(f"Missing required {kind} config field: {field.name!r}")
|
||||||
|
else:
|
||||||
|
value = field.default
|
||||||
|
_validate_field_value(kind, field, value)
|
||||||
|
validated[field.name] = value
|
||||||
|
return validated
|
||||||
|
|
||||||
|
|
||||||
|
def sanitize_source_config(kind: str, config: dict[str, Any]) -> dict[str, Any]:
|
||||||
|
"""Validate and return a response-safe config with secrets masked."""
|
||||||
|
profile = get_source_profile(kind)
|
||||||
|
sanitized = validate_source_config(kind, config)
|
||||||
|
for field in profile.fields:
|
||||||
|
if field.secret:
|
||||||
|
sanitized[field.name] = SECRET_MASK
|
||||||
|
return sanitized
|
||||||
|
|
||||||
|
|
||||||
|
def merge_source_config(kind: str, current: dict[str, Any], patch: dict[str, Any]) -> dict[str, Any]:
|
||||||
|
"""Merge a partial PATCH into stored config, retaining masked secrets.
|
||||||
|
|
||||||
|
An empty secret value is the public response mask and therefore means
|
||||||
|
"keep the old value". New sources use :func:`validate_source_config`
|
||||||
|
instead, so an explicitly empty secret can still be initially configured.
|
||||||
|
"""
|
||||||
|
profile = get_source_profile(kind)
|
||||||
|
current_validated = validate_source_config(kind, current)
|
||||||
|
if not isinstance(patch, dict):
|
||||||
|
raise SourceProfileError("Source config patch must be an object.")
|
||||||
|
fields = {field.name: field for field in profile.fields}
|
||||||
|
unknown = set(patch) - set(fields)
|
||||||
|
if unknown:
|
||||||
|
raise SourceProfileError(f"Unknown {kind} config field(s): {sorted(unknown)!r}")
|
||||||
|
|
||||||
|
merged = dict(current_validated)
|
||||||
|
for name, value in patch.items():
|
||||||
|
field = fields[name]
|
||||||
|
if field.secret and value == SECRET_MASK:
|
||||||
|
continue
|
||||||
|
merged[name] = value
|
||||||
|
return validate_source_config(kind, merged)
|
||||||
+248
-8
@@ -30,6 +30,7 @@ from __future__ import annotations
|
|||||||
import logging
|
import logging
|
||||||
import threading
|
import threading
|
||||||
from collections.abc import Callable
|
from collections.abc import Callable
|
||||||
|
from dataclasses import dataclass
|
||||||
from typing import TYPE_CHECKING
|
from typing import TYPE_CHECKING
|
||||||
|
|
||||||
import paho.mqtt.client as mqtt
|
import paho.mqtt.client as mqtt
|
||||||
@@ -47,14 +48,34 @@ MQTT_SETTINGS_KEYS = {
|
|||||||
"mqtt_username",
|
"mqtt_username",
|
||||||
"mqtt_password",
|
"mqtt_password",
|
||||||
"mqtt_tls_enabled",
|
"mqtt_tls_enabled",
|
||||||
|
"mqtt_client_id",
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class _SourceClientState:
|
||||||
|
"""One installed source generation and its in-flight callback count."""
|
||||||
|
|
||||||
|
client: mqtt.Client
|
||||||
|
generation: int
|
||||||
|
in_flight: int = 0
|
||||||
|
|
||||||
|
|
||||||
def _is_configured(settings: Settings) -> bool:
|
def _is_configured(settings: Settings) -> bool:
|
||||||
"""Return True if MQTT is enabled *and* the broker host is set."""
|
"""Return True if MQTT is enabled *and* the broker host is set."""
|
||||||
return bool(settings.mqtt_enabled and settings.mqtt_broker_host)
|
return bool(settings.mqtt_enabled and settings.mqtt_broker_host)
|
||||||
|
|
||||||
|
|
||||||
|
def mqtt_source_client_id(base_client_id: str, source_id: int) -> str:
|
||||||
|
"""Return a stable, deployment-scoped identity for one DSMR source."""
|
||||||
|
return f"{base_client_id}-dsmr-source-{source_id}"
|
||||||
|
|
||||||
|
|
||||||
|
def mqtt_test_client_id(base_client_id: str) -> str:
|
||||||
|
"""Return a transient test identity that cannot evict a long-lived client."""
|
||||||
|
return f"{base_client_id}-test"
|
||||||
|
|
||||||
|
|
||||||
class MqttManager:
|
class MqttManager:
|
||||||
"""Long-lived MQTT client wrapper.
|
"""Long-lived MQTT client wrapper.
|
||||||
|
|
||||||
@@ -72,11 +93,26 @@ class MqttManager:
|
|||||||
|
|
||||||
def __init__(self) -> None:
|
def __init__(self) -> None:
|
||||||
self._client: mqtt.Client | None = None
|
self._client: mqtt.Client | None = None
|
||||||
self._lock = threading.Lock()
|
self._lock = threading.RLock()
|
||||||
|
# Source replacement/removal is serialized independently from callback
|
||||||
|
# bookkeeping. In particular, paho's loop_stop() joins its network
|
||||||
|
# thread, whose callback completion also needs ``_lock``.
|
||||||
|
self._source_lifecycle_lock = threading.Lock()
|
||||||
|
self._source_idle = threading.Condition(self._lock)
|
||||||
self._connected = False
|
self._connected = False
|
||||||
# topic → handler registry; persists across reconnects so subscriptions
|
# topic → handler registry; persists across reconnects so subscriptions
|
||||||
# are automatically re-established when the client reconnects.
|
# are automatically re-established when the client reconnects.
|
||||||
self._subscriptions: dict[str, Callable[[bytes], None]] = {}
|
self._subscriptions: dict[str, Callable[[bytes], None]] = {}
|
||||||
|
# DSMR sources are independent connections: their credentials and TLS
|
||||||
|
# configuration belong to MeterSource.config, not app_config.
|
||||||
|
self._source_clients: dict[int, mqtt.Client] = {}
|
||||||
|
self._source_subscriptions: dict[int, dict[str, Callable[[bytes], None]]] = {}
|
||||||
|
self._source_connected: set[int] = set()
|
||||||
|
# Each replacement gets a distinct identity. A paho callback can run
|
||||||
|
# after its client was stopped, so source id alone is not sufficient.
|
||||||
|
self._source_generations: dict[int, int] = {}
|
||||||
|
self._source_states: dict[int, _SourceClientState] = {}
|
||||||
|
self._next_source_generation = 0
|
||||||
|
|
||||||
# ------------------------------------------------------------------
|
# ------------------------------------------------------------------
|
||||||
# Public properties
|
# Public properties
|
||||||
@@ -115,6 +151,11 @@ class MqttManager:
|
|||||||
"""
|
"""
|
||||||
with self._lock:
|
with self._lock:
|
||||||
self._stop_client()
|
self._stop_client()
|
||||||
|
with self._source_lifecycle_lock:
|
||||||
|
with self._lock:
|
||||||
|
source_ids = list(self._source_clients)
|
||||||
|
for source_id in source_ids:
|
||||||
|
self._stop_source_client(source_id)
|
||||||
|
|
||||||
def reconnect(self, settings: Settings) -> None:
|
def reconnect(self, settings: Settings) -> None:
|
||||||
"""Disconnect the current client (if any) and reconnect with *settings*.
|
"""Disconnect the current client (if any) and reconnect with *settings*.
|
||||||
@@ -133,23 +174,33 @@ class MqttManager:
|
|||||||
*,
|
*,
|
||||||
retain: bool = False,
|
retain: bool = False,
|
||||||
qos: int = 0,
|
qos: int = 0,
|
||||||
) -> None:
|
) -> bool:
|
||||||
"""Publish a message to *topic*.
|
"""Publish a message to *topic*.
|
||||||
|
|
||||||
If the client is not connected the call is silently skipped.
|
Return whether paho accepted the message for publication. If the
|
||||||
Errors are logged but do not raise.
|
client is not connected, or paho raises/rejects it, return ``False``;
|
||||||
|
errors still do not escape this best-effort boundary.
|
||||||
"""
|
"""
|
||||||
with self._lock:
|
with self._lock:
|
||||||
client = self._client
|
client = self._client
|
||||||
|
|
||||||
if client is None or not self._connected:
|
if client is None or not self._connected:
|
||||||
logger.debug("MQTT publish skipped — not connected (topic=%s).", topic)
|
logger.debug("MQTT publish skipped — not connected (topic=%s).", topic)
|
||||||
return
|
return False
|
||||||
|
|
||||||
try:
|
try:
|
||||||
client.publish(topic, payload=payload, qos=qos, retain=retain)
|
result = client.publish(topic, payload=payload, qos=qos, retain=retain)
|
||||||
|
# paho returns MQTTMessageInfo with an integer rc. Keep fakes and
|
||||||
|
# alternate clients compatible by treating an absent/non-int rc as
|
||||||
|
# accepted after the call itself succeeded.
|
||||||
|
rc = getattr(result, "rc", None)
|
||||||
|
if isinstance(rc, int) and rc != mqtt.MQTT_ERR_SUCCESS:
|
||||||
|
logger.warning("MQTT publish rejected (topic=%s, rc=%s).", topic, rc)
|
||||||
|
return False
|
||||||
|
return True
|
||||||
except Exception:
|
except Exception:
|
||||||
logger.exception("MQTT publish error (topic=%s).", topic)
|
logger.exception("MQTT publish error (topic=%s).", topic)
|
||||||
|
return False
|
||||||
|
|
||||||
def subscribe(self, topic: str, handler: Callable[[bytes], None]) -> None:
|
def subscribe(self, topic: str, handler: Callable[[bytes], None]) -> None:
|
||||||
"""Register *handler* to be called when a message arrives on *topic*.
|
"""Register *handler* to be called when a message arrives on *topic*.
|
||||||
@@ -196,15 +247,152 @@ class MqttManager:
|
|||||||
except Exception:
|
except Exception:
|
||||||
logger.exception("MQTT unsubscribe error (topic=%s).", topic)
|
logger.exception("MQTT unsubscribe error (topic=%s).", topic)
|
||||||
|
|
||||||
|
def replace_source(
|
||||||
|
self,
|
||||||
|
source_id: int,
|
||||||
|
*,
|
||||||
|
host: str,
|
||||||
|
port: int,
|
||||||
|
username: str,
|
||||||
|
password: str,
|
||||||
|
tls_enabled: bool,
|
||||||
|
subscriptions: dict[str, Callable[[bytes], None]],
|
||||||
|
base_client_id: str = "home-automation",
|
||||||
|
state_handler: Callable[[str], None] | None = None,
|
||||||
|
) -> bool:
|
||||||
|
"""Replace one source-owned client and its handlers.
|
||||||
|
|
||||||
|
This intentionally does not touch the legacy app-wide client or any
|
||||||
|
other source client. It is also safe for a source to be temporarily
|
||||||
|
unconfigured: handlers are retained in the source registry but no
|
||||||
|
connection is attempted until a host is supplied.
|
||||||
|
"""
|
||||||
|
with self._source_lifecycle_lock:
|
||||||
|
self._stop_source_client(source_id)
|
||||||
|
if not host:
|
||||||
|
self._report_source_state(state_handler, "error", source_id)
|
||||||
|
return False
|
||||||
|
self._next_source_generation += 1
|
||||||
|
generation = self._next_source_generation
|
||||||
|
captured_subscriptions = dict(subscriptions)
|
||||||
|
client = mqtt.Client(
|
||||||
|
callback_api_version=mqtt.CallbackAPIVersion.VERSION2,
|
||||||
|
client_id=mqtt_source_client_id(base_client_id, source_id),
|
||||||
|
)
|
||||||
|
|
||||||
|
def _on_connect(
|
||||||
|
connected_client: mqtt.Client,
|
||||||
|
_userdata: object,
|
||||||
|
_flags: mqtt.ConnectFlags,
|
||||||
|
reason_code: mqtt.ReasonCode,
|
||||||
|
_properties: mqtt.Properties | None,
|
||||||
|
) -> None:
|
||||||
|
with self._lock:
|
||||||
|
if not self._is_current_source_client(source_id, generation, connected_client):
|
||||||
|
return
|
||||||
|
if reason_code.is_failure:
|
||||||
|
self._source_connected.discard(source_id)
|
||||||
|
logger.warning("DSMR MQTT connection refused for source_id=%s", source_id)
|
||||||
|
state = "error"
|
||||||
|
else:
|
||||||
|
self._source_connected.add(source_id)
|
||||||
|
state = "online"
|
||||||
|
for topic in captured_subscriptions:
|
||||||
|
try:
|
||||||
|
connected_client.subscribe(topic)
|
||||||
|
except Exception:
|
||||||
|
logger.exception("DSMR MQTT re-subscribe failed for source_id=%s", source_id)
|
||||||
|
self._report_source_state(state_handler, state, source_id)
|
||||||
|
|
||||||
|
def _on_disconnect(
|
||||||
|
disconnected_client: mqtt.Client,
|
||||||
|
_userdata: object,
|
||||||
|
_flags: mqtt.DisconnectFlags,
|
||||||
|
_reason_code: mqtt.ReasonCode,
|
||||||
|
_properties: mqtt.Properties | None,
|
||||||
|
) -> None:
|
||||||
|
with self._lock:
|
||||||
|
if not self._is_current_source_client(source_id, generation, disconnected_client):
|
||||||
|
return
|
||||||
|
self._source_connected.discard(source_id)
|
||||||
|
self._report_source_state(state_handler, "error", source_id)
|
||||||
|
|
||||||
|
def _on_message(
|
||||||
|
message_client: mqtt.Client,
|
||||||
|
_userdata: object,
|
||||||
|
message: mqtt.MQTTMessage,
|
||||||
|
) -> None:
|
||||||
|
with self._lock:
|
||||||
|
if not self._is_current_source_client(source_id, generation, message_client):
|
||||||
|
return
|
||||||
|
handler = captured_subscriptions.get(message.topic)
|
||||||
|
state = self._source_states.get(source_id)
|
||||||
|
if handler is None or state is None:
|
||||||
|
return
|
||||||
|
# This permit covers the entire handler call. Teardown first
|
||||||
|
# invalidates the state and then waits for all permits, so an
|
||||||
|
# old callback cannot run after teardown returns.
|
||||||
|
state.in_flight += 1
|
||||||
|
try:
|
||||||
|
handler(message.payload)
|
||||||
|
except Exception:
|
||||||
|
logger.exception("DSMR source handler raised (source_id=%s)", source_id)
|
||||||
|
finally:
|
||||||
|
with self._lock:
|
||||||
|
state.in_flight -= 1
|
||||||
|
if state.in_flight == 0:
|
||||||
|
self._source_idle.notify_all()
|
||||||
|
|
||||||
|
client.on_connect = _on_connect
|
||||||
|
client.on_disconnect = _on_disconnect
|
||||||
|
client.on_message = _on_message
|
||||||
|
if tls_enabled:
|
||||||
|
try:
|
||||||
|
client.tls_set()
|
||||||
|
except Exception:
|
||||||
|
logger.exception("DSMR MQTT TLS setup failed for source_id=%s", source_id)
|
||||||
|
self._report_source_state(state_handler, "error", source_id)
|
||||||
|
return False
|
||||||
|
if username:
|
||||||
|
client.username_pw_set(username=username, password=password or None)
|
||||||
|
# Register ownership before network processing begins. A broker
|
||||||
|
# may deliver CONNACK synchronously from connect(), or on the loop
|
||||||
|
# thread before connect() returns.
|
||||||
|
with self._lock:
|
||||||
|
self._source_clients[source_id] = client
|
||||||
|
self._source_subscriptions[source_id] = captured_subscriptions
|
||||||
|
self._source_generations[source_id] = generation
|
||||||
|
self._source_states[source_id] = _SourceClientState(client, generation)
|
||||||
|
self._report_source_state(state_handler, "connecting", source_id)
|
||||||
|
client.loop_start()
|
||||||
|
try:
|
||||||
|
client.connect(host=host, port=port, keepalive=60)
|
||||||
|
except Exception:
|
||||||
|
logger.exception("DSMR MQTT connect failed (source_id=%s, host=%s)", source_id, host)
|
||||||
|
self._report_source_state(state_handler, "error", source_id)
|
||||||
|
self._stop_source_client(source_id)
|
||||||
|
return False
|
||||||
|
return True
|
||||||
|
|
||||||
|
def remove_source(self, source_id: int) -> None:
|
||||||
|
"""Drop one source client and its handlers, including queued callbacks."""
|
||||||
|
with self._source_lifecycle_lock:
|
||||||
|
self._stop_source_client(source_id)
|
||||||
|
|
||||||
|
def source_is_active(self, source_id: int) -> bool:
|
||||||
|
"""Whether a source-owned client is currently installed for callbacks."""
|
||||||
|
with self._lock:
|
||||||
|
return source_id in self._source_clients
|
||||||
|
|
||||||
# ------------------------------------------------------------------
|
# ------------------------------------------------------------------
|
||||||
# Internal helpers — must be called with self._lock held
|
# Internal helpers
|
||||||
# ------------------------------------------------------------------
|
# ------------------------------------------------------------------
|
||||||
|
|
||||||
def _start_client(self, settings: Settings) -> None:
|
def _start_client(self, settings: Settings) -> None:
|
||||||
"""Build a fresh paho Client, configure it, and call loop_start + connect."""
|
"""Build a fresh paho Client, configure it, and call loop_start + connect."""
|
||||||
client = mqtt.Client(
|
client = mqtt.Client(
|
||||||
callback_api_version=mqtt.CallbackAPIVersion.VERSION2,
|
callback_api_version=mqtt.CallbackAPIVersion.VERSION2,
|
||||||
client_id="home-automation",
|
client_id=settings.mqtt_client_id,
|
||||||
)
|
)
|
||||||
|
|
||||||
# Callbacks — VERSION2 on_connect signature:
|
# Callbacks — VERSION2 on_connect signature:
|
||||||
@@ -333,6 +521,58 @@ class MqttManager:
|
|||||||
logger.debug("MQTT loop_stop raised (ignoring).", exc_info=True)
|
logger.debug("MQTT loop_stop raised (ignoring).", exc_info=True)
|
||||||
logger.info("MQTT client stopped.")
|
logger.info("MQTT client stopped.")
|
||||||
|
|
||||||
|
def _stop_source_client(self, source_id: int) -> None:
|
||||||
|
"""Detach then stop a source client without blocking callback bookkeeping.
|
||||||
|
|
||||||
|
Callers hold ``_source_lifecycle_lock``. The first phase makes the
|
||||||
|
generation unreachable while holding ``_lock``. Paho operations and
|
||||||
|
the in-flight wait are deliberately outside that lock: loop_stop()
|
||||||
|
joins paho's network thread, and an active callback needs ``_lock`` to
|
||||||
|
release its permit in ``_on_message``'s finally block.
|
||||||
|
"""
|
||||||
|
with self._lock:
|
||||||
|
state = self._source_states.pop(source_id, None)
|
||||||
|
client = self._source_clients.pop(source_id, None)
|
||||||
|
self._source_subscriptions.pop(source_id, None)
|
||||||
|
self._source_connected.discard(source_id)
|
||||||
|
# Invalidate callbacks even when there was no successfully
|
||||||
|
# installed client (for example after a failed replacement).
|
||||||
|
self._source_generations.pop(source_id, None)
|
||||||
|
if client is not None:
|
||||||
|
try:
|
||||||
|
client.disconnect()
|
||||||
|
except Exception:
|
||||||
|
logger.debug("DSMR MQTT disconnect raised (source_id=%s)", source_id, exc_info=True)
|
||||||
|
try:
|
||||||
|
client.loop_stop()
|
||||||
|
except Exception:
|
||||||
|
logger.debug("DSMR MQTT loop_stop raised (source_id=%s)", source_id, exc_info=True)
|
||||||
|
if state is not None:
|
||||||
|
with self._lock:
|
||||||
|
while state.in_flight:
|
||||||
|
self._source_idle.wait()
|
||||||
|
|
||||||
|
def _is_current_source_client(
|
||||||
|
self, source_id: int, generation: int, client: mqtt.Client
|
||||||
|
) -> bool:
|
||||||
|
"""Check callback ownership while ``_lock`` is held."""
|
||||||
|
return (
|
||||||
|
self._source_generations.get(source_id) == generation
|
||||||
|
and self._source_clients.get(source_id) is client
|
||||||
|
)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _report_source_state(
|
||||||
|
state_handler: Callable[[str], None] | None, state: str, source_id: int
|
||||||
|
) -> None:
|
||||||
|
"""Invoke an optional health callback without exposing connection credentials."""
|
||||||
|
if state_handler is None:
|
||||||
|
return
|
||||||
|
try:
|
||||||
|
state_handler(state)
|
||||||
|
except Exception:
|
||||||
|
logger.exception("DSMR MQTT source state update failed for source_id=%s", source_id)
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# Module-level singleton — shared across lifespan and route handlers
|
# Module-level singleton — shared across lifespan and route handlers
|
||||||
|
|||||||
@@ -0,0 +1,192 @@
|
|||||||
|
"""Pure, privacy-preserving parser for DSMR and WarmteLink P1 telegrams."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass, field as dataclass_field
|
||||||
|
from decimal import Decimal, InvalidOperation
|
||||||
|
from enum import StrEnum
|
||||||
|
import hashlib
|
||||||
|
import re
|
||||||
|
|
||||||
|
|
||||||
|
_OBIS_LINE = re.compile(r"^(?P<code>\d+-\d+:\d+\.\d+\.\d+)(?P<values>(?:\([^)]*\))*)$")
|
||||||
|
_NUMBER_WITH_UNIT = re.compile(r"^(?P<number>[+-]?\d+(?:\.\d+)?)(?:\*(?P<unit>.+))?$")
|
||||||
|
_CHANNEL_OBIS = re.compile(r"^0-(?P<channel>[1-9]\d*):(24|96)\.")
|
||||||
|
_EQUIPMENT_ID_CODES = re.compile(r"^0-(?:0|[1-9]\d*):96\.1\.[01]$")
|
||||||
|
|
||||||
|
|
||||||
|
class IntegrityStatus(StrEnum):
|
||||||
|
"""Whether a frame has a verifiable standard DSMR checksum."""
|
||||||
|
|
||||||
|
VALID = "valid"
|
||||||
|
INVALID = "invalid"
|
||||||
|
UNVERIFIABLE = "unverifiable"
|
||||||
|
|
||||||
|
|
||||||
|
class P1ParseError(ValueError):
|
||||||
|
"""A parse error whose message never includes telegram contents."""
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class ObisField:
|
||||||
|
"""One sanitized OBIS line, including values not understood by the parser."""
|
||||||
|
|
||||||
|
code: str
|
||||||
|
raw_values: tuple[str, ...]
|
||||||
|
value: Decimal | None = None
|
||||||
|
unit: str | None = None
|
||||||
|
comparison_token: str | None = dataclass_field(default=None, repr=False)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class P1Channel:
|
||||||
|
"""Fields associated with one M-Bus channel, without its raw identifier."""
|
||||||
|
|
||||||
|
number: int
|
||||||
|
device_type: str | None
|
||||||
|
equipment_fingerprint: str | None = dataclass_field(repr=False)
|
||||||
|
readings: tuple[ObisField, ...]
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class P1Telegram:
|
||||||
|
"""A parsed telegram with only sanitized, persistence-safe data."""
|
||||||
|
|
||||||
|
frame_length: int
|
||||||
|
integrity: IntegrityStatus
|
||||||
|
integrity_reason: str
|
||||||
|
timestamp: str | None
|
||||||
|
equipment_fingerprint: str | None = dataclass_field(repr=False)
|
||||||
|
fields: tuple[ObisField, ...]
|
||||||
|
channels: tuple[P1Channel, ...]
|
||||||
|
|
||||||
|
|
||||||
|
class TelegramFramer:
|
||||||
|
"""Incrementally extract newline-terminated variable-length telegrams."""
|
||||||
|
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self._buffer = bytearray()
|
||||||
|
|
||||||
|
def feed(self, chunk: bytes) -> list[bytes]:
|
||||||
|
"""Append *chunk* and return every complete frame now available."""
|
||||||
|
|
||||||
|
self._buffer.extend(chunk)
|
||||||
|
frames: list[bytes] = []
|
||||||
|
while (bang := self._buffer.find(b"!")) >= 0:
|
||||||
|
newline = self._buffer.find(b"\n", bang)
|
||||||
|
if newline < 0:
|
||||||
|
break
|
||||||
|
standard_start = self._buffer.find(b"/")
|
||||||
|
start = standard_start if 0 <= standard_start < bang else 0
|
||||||
|
frames.append(bytes(self._buffer[start : newline + 1]))
|
||||||
|
del self._buffer[: newline + 1]
|
||||||
|
return frames
|
||||||
|
|
||||||
|
|
||||||
|
def dsmr_crc16(data: bytes) -> int:
|
||||||
|
"""Return the DSMR CRC-16 over *data* (normally from ``/`` through ``!``)."""
|
||||||
|
|
||||||
|
crc = 0
|
||||||
|
for byte in data:
|
||||||
|
crc ^= byte
|
||||||
|
for _ in range(8):
|
||||||
|
crc = (crc >> 1) ^ 0xA001 if crc & 1 else crc >> 1
|
||||||
|
return crc & 0xFFFF
|
||||||
|
|
||||||
|
|
||||||
|
def parse_telegram(frame: bytes) -> P1Telegram:
|
||||||
|
"""Parse one complete frame without retaining raw telegram bytes or IDs."""
|
||||||
|
|
||||||
|
bang = frame.find(b"!")
|
||||||
|
if bang < 0:
|
||||||
|
raise P1ParseError("telegram has no footer marker")
|
||||||
|
body = frame[:bang]
|
||||||
|
footer = frame[bang + 1 :].rstrip(b"\r\n")
|
||||||
|
integrity, reason = _integrity(frame, bang, footer)
|
||||||
|
fields, identifiers = _parse_obis_fields(body)
|
||||||
|
timestamp = _field_value(fields, "0-0:1.0.0")
|
||||||
|
return P1Telegram(
|
||||||
|
frame_length=len(frame),
|
||||||
|
integrity=integrity,
|
||||||
|
integrity_reason=reason,
|
||||||
|
timestamp=timestamp,
|
||||||
|
equipment_fingerprint=identifiers.get("0-0:96.1.1"),
|
||||||
|
fields=tuple(fields),
|
||||||
|
channels=_parse_channels(fields, identifiers),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _integrity(frame: bytes, bang: int, footer: bytes) -> tuple[IntegrityStatus, str]:
|
||||||
|
if not frame.startswith(b"/"):
|
||||||
|
return IntegrityStatus.UNVERIFIABLE, "missing standard DSMR '/' header"
|
||||||
|
if len(footer) != 4 or not all(chr(byte) in "0123456789abcdefABCDEF" for byte in footer):
|
||||||
|
return IntegrityStatus.UNVERIFIABLE, "footer is not a four-digit hexadecimal CRC"
|
||||||
|
expected = int(footer, 16)
|
||||||
|
actual = dsmr_crc16(frame[: bang + 1])
|
||||||
|
if actual == expected:
|
||||||
|
return IntegrityStatus.VALID, "CRC16 verified from '/' through '!'"
|
||||||
|
return IntegrityStatus.INVALID, f"CRC16 mismatch: expected {expected:04X}, calculated {actual:04X}"
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_obis_fields(body: bytes) -> tuple[list[ObisField], dict[str, str]]:
|
||||||
|
fields: list[ObisField] = []
|
||||||
|
identifiers: dict[str, str] = {}
|
||||||
|
for line in body.decode("ascii", errors="replace").splitlines()[1:]:
|
||||||
|
match = _OBIS_LINE.fullmatch(line)
|
||||||
|
if not match:
|
||||||
|
continue
|
||||||
|
code = match.group("code")
|
||||||
|
raw_values = tuple(re.findall(r"\(([^)]*)\)", match.group("values")))
|
||||||
|
if _EQUIPMENT_ID_CODES.fullmatch(code):
|
||||||
|
comparison_token = _fingerprint(raw_values[-1]) if raw_values else None
|
||||||
|
if comparison_token is not None:
|
||||||
|
identifiers[code] = comparison_token
|
||||||
|
fields.append(ObisField(code, ("<redacted>",), comparison_token=comparison_token))
|
||||||
|
continue
|
||||||
|
value, unit = _numeric_value(raw_values)
|
||||||
|
fields.append(ObisField(code, raw_values, value, unit))
|
||||||
|
return fields, identifiers
|
||||||
|
|
||||||
|
|
||||||
|
def _fingerprint(identifier: str) -> str:
|
||||||
|
"""Hash an identifier locally; callers never receive its original value."""
|
||||||
|
|
||||||
|
return hashlib.sha256(identifier.encode("ascii", errors="replace")).hexdigest()
|
||||||
|
|
||||||
|
|
||||||
|
def _numeric_value(raw_values: tuple[str, ...]) -> tuple[Decimal | None, str | None]:
|
||||||
|
if not raw_values:
|
||||||
|
return None, None
|
||||||
|
match = _NUMBER_WITH_UNIT.fullmatch(raw_values[-1])
|
||||||
|
if not match:
|
||||||
|
return None, None
|
||||||
|
try:
|
||||||
|
return Decimal(match.group("number")), match.group("unit")
|
||||||
|
except InvalidOperation:
|
||||||
|
return None, None
|
||||||
|
|
||||||
|
|
||||||
|
def _field_value(fields: list[ObisField], code: str) -> str | None:
|
||||||
|
field = next((item for item in fields if item.code == code), None)
|
||||||
|
return field.raw_values[-1] if field and field.raw_values else None
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_channels(fields: list[ObisField], identifiers: dict[str, str]) -> tuple[P1Channel, ...]:
|
||||||
|
by_channel: dict[int, list[ObisField]] = {}
|
||||||
|
for field in fields:
|
||||||
|
match = _CHANNEL_OBIS.match(field.code)
|
||||||
|
if match:
|
||||||
|
by_channel.setdefault(int(match.group("channel")), []).append(field)
|
||||||
|
return tuple(
|
||||||
|
P1Channel(
|
||||||
|
number=number,
|
||||||
|
device_type=_field_value(channel_fields, f"0-{number}:24.1.0"),
|
||||||
|
equipment_fingerprint=identifiers.get(f"0-{number}:96.1.0"),
|
||||||
|
readings=tuple(
|
||||||
|
field
|
||||||
|
for field in channel_fields
|
||||||
|
if field.code == f"0-{number}:24.2.1" and field.value is not None
|
||||||
|
),
|
||||||
|
)
|
||||||
|
for number, channel_fields in sorted(by_channel.items())
|
||||||
|
)
|
||||||
@@ -25,6 +25,7 @@ Design notes
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import logging
|
import logging
|
||||||
|
from decimal import Decimal, InvalidOperation
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any, Optional
|
from typing import Any, Optional
|
||||||
|
|
||||||
@@ -152,13 +153,86 @@ class TibberProfile(BaseModel):
|
|||||||
return self
|
return self
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# DistrictHeatingProfile — user-entered thermal contract structure
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
class DistrictHeatingFieldSpec(BaseModel):
|
||||||
|
"""A Decimal-safe thermal tariff field displayed to the user."""
|
||||||
|
|
||||||
|
model_config = {"extra": "forbid"}
|
||||||
|
|
||||||
|
unit: str
|
||||||
|
label: str
|
||||||
|
help: str
|
||||||
|
minimum: Decimal = Decimal("0")
|
||||||
|
default: Decimal | None = None
|
||||||
|
|
||||||
|
|
||||||
|
class DistrictHeatingVariableSpec(BaseModel):
|
||||||
|
model_config = {"extra": "forbid"}
|
||||||
|
|
||||||
|
heating: DistrictHeatingFieldSpec
|
||||||
|
hot_water_heating: DistrictHeatingFieldSpec
|
||||||
|
hot_water: DistrictHeatingFieldSpec
|
||||||
|
hot_water_tax: DistrictHeatingFieldSpec
|
||||||
|
|
||||||
|
|
||||||
|
class DistrictHeatingStandingSpec(BaseModel):
|
||||||
|
model_config = {"extra": "forbid"}
|
||||||
|
|
||||||
|
heating_network: DistrictHeatingFieldSpec
|
||||||
|
metering: DistrictHeatingFieldSpec
|
||||||
|
delivery_set: DistrictHeatingFieldSpec
|
||||||
|
hot_water_network: DistrictHeatingFieldSpec
|
||||||
|
other: DistrictHeatingFieldSpec
|
||||||
|
|
||||||
|
|
||||||
|
class DistrictHeatingProfile(BaseModel):
|
||||||
|
"""Complete structure description for a ``district_heating`` contract."""
|
||||||
|
|
||||||
|
model_config = {"extra": "forbid"}
|
||||||
|
|
||||||
|
kind: str
|
||||||
|
label: str
|
||||||
|
variable: DistrictHeatingVariableSpec
|
||||||
|
standing: DistrictHeatingStandingSpec
|
||||||
|
|
||||||
|
@model_validator(mode="after")
|
||||||
|
def _check_kind(self) -> "DistrictHeatingProfile":
|
||||||
|
if self.kind != "district_heating":
|
||||||
|
raise ValueError(
|
||||||
|
"DistrictHeatingProfile requires kind='district_heating', "
|
||||||
|
f"got {self.kind!r}"
|
||||||
|
)
|
||||||
|
units = {
|
||||||
|
"heating": "EUR/GJ",
|
||||||
|
"hot_water_heating": "EUR/m³",
|
||||||
|
"hot_water": "EUR/m³",
|
||||||
|
"hot_water_tax": "EUR/m³",
|
||||||
|
}
|
||||||
|
for key, unit in units.items():
|
||||||
|
field = getattr(self.variable, key)
|
||||||
|
if field.unit != unit or field.minimum != 0 or field.default is not None:
|
||||||
|
raise ValueError(f"district_heating.variable.{key} must be required {unit} with minimum 0")
|
||||||
|
for key in DistrictHeatingStandingSpec.model_fields:
|
||||||
|
field = getattr(self.standing, key)
|
||||||
|
if field.unit != "EUR/year" or field.minimum != 0 or field.default != 0:
|
||||||
|
raise ValueError(
|
||||||
|
f"district_heating.standing.{key} must be EUR/year with default and minimum 0"
|
||||||
|
)
|
||||||
|
return self
|
||||||
|
|
||||||
|
|
||||||
# A union type for type hints where either profile is acceptable.
|
# A union type for type hints where either profile is acceptable.
|
||||||
AnyProfile = ManualProfile | TibberProfile
|
AnyProfile = ManualProfile | TibberProfile | DistrictHeatingProfile
|
||||||
|
|
||||||
# Map kind → Pydantic model class used for validation.
|
# Map kind → Pydantic model class used for validation.
|
||||||
_PROFILE_MODELS: dict[str, type[ManualProfile] | type[TibberProfile]] = {
|
_PROFILE_MODELS: dict[str, type[ManualProfile] | type[TibberProfile] | type[DistrictHeatingProfile]] = {
|
||||||
"manual": ManualProfile,
|
"manual": ManualProfile,
|
||||||
"tibber": TibberProfile,
|
"tibber": TibberProfile,
|
||||||
|
"district_heating": DistrictHeatingProfile,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
@@ -204,6 +278,11 @@ def load_profile(kind: str) -> AnyProfile:
|
|||||||
f"Profile '{kind}': expected a YAML mapping, got {type(raw).__name__}"
|
f"Profile '{kind}': expected a YAML mapping, got {type(raw).__name__}"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
# YAML's implicit float conversion must never contaminate the thermal profile.
|
||||||
|
# Existing electricity profiles intentionally retain their established defaults.
|
||||||
|
if raw.get("kind", kind) == "district_heating":
|
||||||
|
_reject_yaml_floats(raw, path)
|
||||||
|
|
||||||
# Choose the right Pydantic model based on the ``kind`` field in the YAML.
|
# Choose the right Pydantic model based on the ``kind`` field in the YAML.
|
||||||
yaml_kind = raw.get("kind", kind)
|
yaml_kind = raw.get("kind", kind)
|
||||||
model_cls = _PROFILE_MODELS.get(yaml_kind)
|
model_cls = _PROFILE_MODELS.get(yaml_kind)
|
||||||
@@ -362,6 +441,82 @@ def _validate_tibber_values(values: dict[str, Any], profile: TibberProfile) -> d
|
|||||||
return filled
|
return filled
|
||||||
|
|
||||||
|
|
||||||
|
def _reject_yaml_floats(value: Any, path: Path) -> None:
|
||||||
|
"""Reject implicit YAML floats for district-heating profile metadata."""
|
||||||
|
if isinstance(value, float):
|
||||||
|
raise ProfileValidationError(
|
||||||
|
f"Profile '{path.stem}' must not contain YAML float values; use integer 0 or strings."
|
||||||
|
)
|
||||||
|
if isinstance(value, dict):
|
||||||
|
for child in value.values():
|
||||||
|
_reject_yaml_floats(child, path)
|
||||||
|
elif isinstance(value, list):
|
||||||
|
for child in value:
|
||||||
|
_reject_yaml_floats(child, path)
|
||||||
|
|
||||||
|
|
||||||
|
def _decimal_value(section: str, key: str, value: Any) -> str:
|
||||||
|
"""Validate and normalise one thermal amount without passing through float."""
|
||||||
|
if isinstance(value, bool) or isinstance(value, float) or not isinstance(value, (str, int, Decimal)):
|
||||||
|
raise ProfileValidationError(
|
||||||
|
f"Contract values field '{section}.{key}' must be a Decimal-compatible string or integer, "
|
||||||
|
f"got {type(value).__name__!r}"
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
amount = Decimal(str(value))
|
||||||
|
except (InvalidOperation, ValueError) as exc:
|
||||||
|
raise ProfileValidationError(
|
||||||
|
f"Contract values field '{section}.{key}' must be a Decimal-compatible value"
|
||||||
|
) from exc
|
||||||
|
if not amount.is_finite() or amount < 0:
|
||||||
|
raise ProfileValidationError(
|
||||||
|
f"Contract values field '{section}.{key}' must be a non-negative finite Decimal"
|
||||||
|
)
|
||||||
|
return format(amount, "f")
|
||||||
|
|
||||||
|
|
||||||
|
def _validate_district_heating_values(
|
||||||
|
values: dict[str, Any], profile: DistrictHeatingProfile
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
"""Validate thermal values and return a complete JSON-safe Decimal snapshot."""
|
||||||
|
if not isinstance(values, dict):
|
||||||
|
raise ProfileValidationError("District-heating contract values must be a mapping")
|
||||||
|
expected_sections = {"variable", "standing"}
|
||||||
|
unknown_sections = set(values) - expected_sections
|
||||||
|
if unknown_sections:
|
||||||
|
raise ProfileValidationError(
|
||||||
|
f"District-heating contract values contain unknown section(s): {sorted(unknown_sections)}"
|
||||||
|
)
|
||||||
|
|
||||||
|
def normalise_section(
|
||||||
|
section: str, specs: Any, *, defaults_allowed: bool
|
||||||
|
) -> dict[str, str]:
|
||||||
|
supplied = values.get(section, {})
|
||||||
|
if not isinstance(supplied, dict):
|
||||||
|
raise ProfileValidationError(f"Contract values section '{section}' must be a mapping")
|
||||||
|
expected = set(type(specs).model_fields)
|
||||||
|
unknown = set(supplied) - expected
|
||||||
|
if unknown:
|
||||||
|
raise ProfileValidationError(
|
||||||
|
f"Contract values section '{section}' contains unknown field(s): {sorted(unknown)}"
|
||||||
|
)
|
||||||
|
normalised: dict[str, str] = {}
|
||||||
|
for key in type(specs).model_fields:
|
||||||
|
if key not in supplied:
|
||||||
|
field = getattr(specs, key)
|
||||||
|
if not defaults_allowed or field.default is None:
|
||||||
|
raise ProfileValidationError(f"Contract values missing required field '{section}.{key}'")
|
||||||
|
normalised[key] = format(field.default, "f")
|
||||||
|
else:
|
||||||
|
normalised[key] = _decimal_value(section, key, supplied[key])
|
||||||
|
return normalised
|
||||||
|
|
||||||
|
return {
|
||||||
|
"variable": normalise_section("variable", profile.variable, defaults_allowed=False),
|
||||||
|
"standing": normalise_section("standing", profile.standing, defaults_allowed=True),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
def validate_values(kind: str, values: dict[str, Any]) -> dict[str, Any]:
|
def validate_values(kind: str, values: dict[str, Any]) -> dict[str, Any]:
|
||||||
"""Validate a contract-values dict against the named profile structure.
|
"""Validate a contract-values dict against the named profile structure.
|
||||||
|
|
||||||
@@ -395,5 +550,7 @@ def validate_values(kind: str, values: dict[str, Any]) -> dict[str, Any]:
|
|||||||
return _validate_manual_values(values, profile)
|
return _validate_manual_values(values, profile)
|
||||||
if isinstance(profile, TibberProfile):
|
if isinstance(profile, TibberProfile):
|
||||||
return _validate_tibber_values(values, profile)
|
return _validate_tibber_values(values, profile)
|
||||||
|
if isinstance(profile, DistrictHeatingProfile):
|
||||||
|
return _validate_district_heating_values(values, profile)
|
||||||
# Unreachable with current kinds, but guard for future extensions.
|
# Unreachable with current kinds, but guard for future extensions.
|
||||||
raise ProfileValidationError(f"No validator implemented for kind={kind!r}")
|
raise ProfileValidationError(f"No validator implemented for kind={kind!r}")
|
||||||
|
|||||||
@@ -0,0 +1,56 @@
|
|||||||
|
kind: district_heating
|
||||||
|
label: 区域供热
|
||||||
|
|
||||||
|
variable:
|
||||||
|
heating:
|
||||||
|
unit: EUR/GJ
|
||||||
|
label: 供暖热量
|
||||||
|
help: 按供暖用热量计收;请录入合同中的实际金额。
|
||||||
|
minimum: 0
|
||||||
|
hot_water_heating:
|
||||||
|
unit: EUR/m³
|
||||||
|
label: 热水加热
|
||||||
|
help: 按热水体积计收的加热部分;请录入合同中的实际金额。
|
||||||
|
minimum: 0
|
||||||
|
hot_water:
|
||||||
|
unit: EUR/m³
|
||||||
|
label: 热水用量
|
||||||
|
help: 按热水体积计收的用量部分;请录入合同中的实际金额。
|
||||||
|
minimum: 0
|
||||||
|
hot_water_tax:
|
||||||
|
unit: EUR/m³
|
||||||
|
label: 热水税费
|
||||||
|
help: 按热水体积计收的税费部分;请录入合同中的实际金额。
|
||||||
|
minimum: 0
|
||||||
|
|
||||||
|
standing:
|
||||||
|
heating_network:
|
||||||
|
unit: EUR/year
|
||||||
|
label: 供暖网络费
|
||||||
|
help: 年度固定费用;默认零,按合同实际金额录入。
|
||||||
|
minimum: 0
|
||||||
|
default: 0
|
||||||
|
metering:
|
||||||
|
unit: EUR/year
|
||||||
|
label: 计量费
|
||||||
|
help: 年度固定费用;默认零,按合同实际金额录入。
|
||||||
|
minimum: 0
|
||||||
|
default: 0
|
||||||
|
delivery_set:
|
||||||
|
unit: EUR/year
|
||||||
|
label: 交付装置费
|
||||||
|
help: 年度固定费用;默认零,按合同实际金额录入。
|
||||||
|
minimum: 0
|
||||||
|
default: 0
|
||||||
|
hot_water_network:
|
||||||
|
unit: EUR/year
|
||||||
|
label: 热水网络费
|
||||||
|
help: 年度固定费用;默认零,按合同实际金额录入。
|
||||||
|
minimum: 0
|
||||||
|
default: 0
|
||||||
|
other:
|
||||||
|
unit: EUR/year
|
||||||
|
label: 其他固定费
|
||||||
|
help: 年度固定费用;默认零,按合同实际金额录入。
|
||||||
|
minimum: 0
|
||||||
|
default: 0
|
||||||
+64
-28
@@ -1,7 +1,9 @@
|
|||||||
import logging
|
import logging
|
||||||
import os
|
import os
|
||||||
from contextlib import asynccontextmanager
|
from contextlib import asynccontextmanager
|
||||||
|
from datetime import UTC, datetime
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
from typing import Callable
|
||||||
|
|
||||||
from fastapi import FastAPI, HTTPException, Request
|
from fastapi import FastAPI, HTTPException, Request
|
||||||
from fastapi.responses import FileResponse
|
from fastapi.responses import FileResponse
|
||||||
@@ -16,8 +18,10 @@ from app.api.routes.api.config import router as api_config_router
|
|||||||
from app.api.routes.api.data import router as api_data_router
|
from app.api.routes.api.data import router as api_data_router
|
||||||
from app.api.routes.api.energy import router as api_energy_router
|
from app.api.routes.api.energy import router as api_energy_router
|
||||||
from app.api.routes.api.energy_contracts import router as api_energy_contracts_router
|
from app.api.routes.api.energy_contracts import router as api_energy_contracts_router
|
||||||
|
from app.api.routes.api.meter_costs import router as api_meter_costs_router
|
||||||
from app.api.routes.api.expose import router as api_expose_router
|
from app.api.routes.api.expose import router as api_expose_router
|
||||||
from app.api.routes.api.meters import router as api_meters_router
|
from app.api.routes.api.meters import router as api_meters_router
|
||||||
|
from app.api.routes.api.meter_sources import router as api_meter_sources_router
|
||||||
from app.api.routes.api.modbus import router as api_modbus_router
|
from app.api.routes.api.modbus import router as api_modbus_router
|
||||||
from app.api.routes.api.session import router as api_session_router
|
from app.api.routes.api.session import router as api_session_router
|
||||||
from app.api.routes import status
|
from app.api.routes import status
|
||||||
@@ -34,9 +38,11 @@ from app.services.config_page import build_runtime_settings, seed_missing_config
|
|||||||
from app.services.dsmr_ingest import apply_dsmr_subscription
|
from app.services.dsmr_ingest import apply_dsmr_subscription
|
||||||
from app.services.public_ip import check_public_ipv4_and_notify
|
from app.services.public_ip import check_public_ipv4_and_notify
|
||||||
from app.services.modbus_poll import poll_all_enabled_devices, BASE_POLL_TICK_SECONDS
|
from app.services.modbus_poll import poll_all_enabled_devices, BASE_POLL_TICK_SECONDS
|
||||||
from app.services.ha_discovery import publish_discovery, publish_states
|
from app.services.ha_discovery import initialize_legacy_thermal_cleanup, publish_discovery, publish_states
|
||||||
from app.services.tibber_prices import refresh_prices
|
from app.services.tibber_prices import run_tibber_refresh_best_effort
|
||||||
from app.services.energy_cost import compute_closed_periods
|
from app.services.energy_cost import compute_closed_periods
|
||||||
|
from app.services.meter_cost import compute_closed_periods as compute_closed_meter_cost_periods
|
||||||
|
from app.services.warmtelink_worker import warmtelink_worker_manager
|
||||||
from app.services.timezone import local_tz
|
from app.services.timezone import local_tz
|
||||||
from scripts.app_db_adopt import AppDatabaseAdoptionError, validate_app_runtime_db
|
from scripts.app_db_adopt import AppDatabaseAdoptionError, validate_app_runtime_db
|
||||||
|
|
||||||
@@ -86,15 +92,7 @@ def _run_scheduled_tibber_refresh() -> None:
|
|||||||
so that a single failed fetch does not crash the scheduler or affect the
|
so that a single failed fetch does not crash the scheduler or affect the
|
||||||
other background jobs.
|
other background jobs.
|
||||||
"""
|
"""
|
||||||
session_local = get_session_local()
|
run_tibber_refresh_best_effort()
|
||||||
session: Session = session_local()
|
|
||||||
try:
|
|
||||||
runtime_settings = build_runtime_settings(session, get_settings())
|
|
||||||
refresh_prices(session, runtime_settings)
|
|
||||||
except Exception:
|
|
||||||
logger.exception("_run_scheduled_tibber_refresh: unexpected error")
|
|
||||||
finally:
|
|
||||||
session.close()
|
|
||||||
|
|
||||||
|
|
||||||
def _run_scheduled_energy_cost() -> None:
|
def _run_scheduled_energy_cost() -> None:
|
||||||
@@ -110,22 +108,44 @@ def _run_scheduled_energy_cost() -> None:
|
|||||||
does not crash the scheduler or affect the other background jobs.
|
does not crash the scheduler or affect the other background jobs.
|
||||||
"""
|
"""
|
||||||
session_local = get_session_local()
|
session_local = get_session_local()
|
||||||
session: Session = session_local()
|
|
||||||
|
def run_scope(label: str, operation: Callable[[Session], None]) -> None:
|
||||||
|
"""Run one best-effort scope in an isolated transaction/session."""
|
||||||
|
session: Session | None = None
|
||||||
try:
|
try:
|
||||||
compute_closed_periods(session)
|
session = session_local()
|
||||||
# After billing periods are computed, push fresh energy-cost state values
|
operation(session)
|
||||||
# to MQTT/HA. publish_states is internally guarded by _should_publish
|
except Exception:
|
||||||
# (MQTT disabled / not connected → no-op), so this never raises due to
|
logger.exception("_run_scheduled_energy_cost: %s failed", label)
|
||||||
# unconfigured MQTT and does not block the billing job.
|
if session is not None:
|
||||||
try:
|
try:
|
||||||
from app.services.ha_discovery import publish_states
|
session.rollback()
|
||||||
publish_states(session)
|
|
||||||
except Exception:
|
except Exception:
|
||||||
logger.exception("_run_scheduled_energy_cost: publish_states failed (non-fatal)")
|
# A failed cleanup must not replace the operation/factory
|
||||||
except Exception:
|
# error or prevent the following independent scope.
|
||||||
logger.exception("_run_scheduled_energy_cost: unexpected error")
|
logger.exception("_run_scheduled_energy_cost: %s rollback failed", label)
|
||||||
finally:
|
finally:
|
||||||
|
if session is not None:
|
||||||
|
try:
|
||||||
session.close()
|
session.close()
|
||||||
|
except Exception:
|
||||||
|
# Sessions are intentionally isolated; close failures are
|
||||||
|
# diagnostic only and must remain best-effort too.
|
||||||
|
logger.exception("_run_scheduled_energy_cost: %s close failed", label)
|
||||||
|
|
||||||
|
# Electricity, thermal and HA publishing must not share failed transaction
|
||||||
|
# state or accidentally commit each other's partially-flushed changes.
|
||||||
|
run_scope("electricity computation", compute_closed_periods)
|
||||||
|
run_scope("thermal computation", compute_closed_meter_cost_periods)
|
||||||
|
|
||||||
|
def publish(session: Session) -> None:
|
||||||
|
# publish_states is internally guarded by _should_publish (MQTT
|
||||||
|
# disabled / disconnected -> no-op), but gets a clean Session anyway.
|
||||||
|
from app.services.ha_discovery import publish_states
|
||||||
|
|
||||||
|
publish_states(session)
|
||||||
|
|
||||||
|
run_scope("publish_states (non-fatal)", publish)
|
||||||
|
|
||||||
|
|
||||||
def _run_scheduled_ha_state_publish() -> None:
|
def _run_scheduled_ha_state_publish() -> None:
|
||||||
@@ -183,6 +203,7 @@ def ensure_auth_db_ready() -> None:
|
|||||||
initialize_auth_schema(session, get_settings())
|
initialize_auth_schema(session, get_settings())
|
||||||
seed_missing_config_from_bootstrap(session, get_settings())
|
seed_missing_config_from_bootstrap(session, get_settings())
|
||||||
sync_app_hostname_from_bootstrap(session, get_settings())
|
sync_app_hostname_from_bootstrap(session, get_settings())
|
||||||
|
initialize_legacy_thermal_cleanup(session)
|
||||||
except AppDatabaseAdoptionError as exc:
|
except AppDatabaseAdoptionError as exc:
|
||||||
raise RuntimeError(str(exc)) from exc
|
raise RuntimeError(str(exc)) from exc
|
||||||
except AuthBootstrapError as exc:
|
except AuthBootstrapError as exc:
|
||||||
@@ -237,6 +258,10 @@ async def lifespan(_: FastAPI):
|
|||||||
replace_existing=True,
|
replace_existing=True,
|
||||||
max_instances=1,
|
max_instances=1,
|
||||||
coalesce=True,
|
coalesce=True,
|
||||||
|
# APScheduler otherwise waits one full interval before its first run.
|
||||||
|
# This preserves the hourly cadence while requesting a non-blocking
|
||||||
|
# startup fetch as soon as the scheduler starts.
|
||||||
|
next_run_time=datetime.now(UTC),
|
||||||
)
|
)
|
||||||
# Energy cost billing: compute uncalculated closed 15-minute periods every minute.
|
# Energy cost billing: compute uncalculated closed 15-minute periods every minute.
|
||||||
# The job is a no-op when no active contract or DSMR data is present, so it is
|
# The job is a no-op when no active contract or DSMR data is present, so it is
|
||||||
@@ -272,18 +297,27 @@ async def lifespan(_: FastAPI):
|
|||||||
_startup_runtime_settings = build_runtime_settings(_startup_session, get_settings())
|
_startup_runtime_settings = build_runtime_settings(_startup_session, get_settings())
|
||||||
finally:
|
finally:
|
||||||
_startup_session.close()
|
_startup_session.close()
|
||||||
|
serial_started = False
|
||||||
|
try:
|
||||||
mqtt_manager.connect(_startup_runtime_settings)
|
mqtt_manager.connect(_startup_runtime_settings)
|
||||||
|
|
||||||
# DSMR ingest: subscribe to the configured MQTT topic when enabled. The same
|
# DSMR sources carry their own runtime configuration and are reconciled
|
||||||
# applier is called from PUT /api/config, so toggling DSMR via the UI takes
|
# after the MQTT manager is connected.
|
||||||
# effect without an app restart.
|
|
||||||
apply_dsmr_subscription(_startup_runtime_settings)
|
apply_dsmr_subscription(_startup_runtime_settings)
|
||||||
|
# Mark it before reconcile: a partial reconcile can already own a fd or
|
||||||
|
# a non-daemon thread and must receive the same orderly shutdown.
|
||||||
|
serial_started = True
|
||||||
|
warmtelink_worker_manager.start()
|
||||||
|
|
||||||
yield
|
yield
|
||||||
|
finally:
|
||||||
# MQTT: clean shutdown before the process exits.
|
# Serial descriptors/workers must be handled first on every exit path.
|
||||||
|
if serial_started:
|
||||||
|
try:
|
||||||
|
warmtelink_worker_manager.shutdown()
|
||||||
|
except Exception:
|
||||||
|
logger.exception("WarmteLink shutdown failed")
|
||||||
mqtt_manager.disconnect()
|
mqtt_manager.disconnect()
|
||||||
|
|
||||||
scheduler.shutdown(wait=False)
|
scheduler.shutdown(wait=False)
|
||||||
|
|
||||||
|
|
||||||
@@ -308,7 +342,9 @@ def create_app() -> FastAPI:
|
|||||||
app.include_router(api_data_router)
|
app.include_router(api_data_router)
|
||||||
app.include_router(api_energy_router)
|
app.include_router(api_energy_router)
|
||||||
app.include_router(api_energy_contracts_router)
|
app.include_router(api_energy_contracts_router)
|
||||||
|
app.include_router(api_meter_costs_router)
|
||||||
app.include_router(api_meters_router)
|
app.include_router(api_meters_router)
|
||||||
|
app.include_router(api_meter_sources_router)
|
||||||
app.include_router(api_expose_router)
|
app.include_router(api_expose_router)
|
||||||
app.include_router(api_modbus_router)
|
app.include_router(api_modbus_router)
|
||||||
app.include_router(api_session_router)
|
app.include_router(api_session_router)
|
||||||
|
|||||||
@@ -5,12 +5,16 @@ from app.models.config import AppConfigEntry
|
|||||||
from app.models.location import Location
|
from app.models.location import Location
|
||||||
from app.models.poo import PooRecord
|
from app.models.poo import PooRecord
|
||||||
from app.models.public_ip import PublicIPHistory, PublicIPState
|
from app.models.public_ip import PublicIPHistory, PublicIPState
|
||||||
|
from app.models.meter_source import MeterSource, MeterSourceBinding, MeterSourceChannel
|
||||||
|
|
||||||
__all__ = [
|
__all__ = [
|
||||||
"AppConfigEntry",
|
"AppConfigEntry",
|
||||||
"AuthSession",
|
"AuthSession",
|
||||||
"AuthUser",
|
"AuthUser",
|
||||||
"Location",
|
"Location",
|
||||||
|
"MeterSource",
|
||||||
|
"MeterSourceBinding",
|
||||||
|
"MeterSourceChannel",
|
||||||
"PooRecord",
|
"PooRecord",
|
||||||
"PublicIPHistory",
|
"PublicIPHistory",
|
||||||
"PublicIPState",
|
"PublicIPState",
|
||||||
|
|||||||
+264
-24
@@ -12,19 +12,134 @@ Six tables:
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import uuid as _uuid
|
import uuid as _uuid
|
||||||
from datetime import datetime
|
from datetime import datetime, timezone
|
||||||
|
from decimal import Decimal
|
||||||
from sqlalchemy import Boolean, DateTime, Float, ForeignKey, Integer, String
|
from typing import Any
|
||||||
from sqlalchemy.orm import Mapped, mapped_column, relationship
|
from sqlalchemy import (
|
||||||
from sqlalchemy.types import JSON
|
Boolean,
|
||||||
|
CheckConstraint,
|
||||||
|
DateTime,
|
||||||
|
Float,
|
||||||
|
ForeignKey,
|
||||||
|
Index,
|
||||||
|
Integer,
|
||||||
|
Numeric,
|
||||||
|
String,
|
||||||
|
UniqueConstraint,
|
||||||
|
event,
|
||||||
|
text,
|
||||||
|
)
|
||||||
|
from sqlalchemy.orm import Mapped, mapped_column, relationship, synonym, validates
|
||||||
|
from sqlalchemy.types import JSON, TypeDecorator
|
||||||
|
|
||||||
from app.db import Base
|
from app.db import Base
|
||||||
|
from app.models.meter_source import MeterSourceBinding
|
||||||
|
|
||||||
|
|
||||||
def _uuid4_str() -> str:
|
def _uuid4_str() -> str:
|
||||||
return str(_uuid.uuid4())
|
return str(_uuid.uuid4())
|
||||||
|
|
||||||
|
|
||||||
|
def _decimal_json(value: Any) -> Any:
|
||||||
|
"""Make auditable JSON portable without admitting binary numeric values."""
|
||||||
|
if isinstance(value, Decimal):
|
||||||
|
return format(value, "f")
|
||||||
|
if isinstance(value, float) or (isinstance(value, int) and not isinstance(value, bool)):
|
||||||
|
raise ValueError("JSON amounts and quantities must be decimal strings, not numeric JSON values")
|
||||||
|
if isinstance(value, dict):
|
||||||
|
return {key: _decimal_json(child) for key, child in value.items()}
|
||||||
|
if isinstance(value, list):
|
||||||
|
return [_decimal_json(child) for child in value]
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def _validate_fixed_decimal(value: Decimal, precision: int, scale: int, field: str) -> Decimal:
|
||||||
|
if not isinstance(value, Decimal):
|
||||||
|
raise ValueError(f"{field} must be a Decimal, not a binary float or other numeric type")
|
||||||
|
if not value.is_finite():
|
||||||
|
raise ValueError(f"{field} must be finite")
|
||||||
|
if -value.as_tuple().exponent > scale:
|
||||||
|
raise ValueError(f"{field} exceeds scale {scale}")
|
||||||
|
integer_digits = 0 if value.is_zero() else max(value.copy_abs().adjusted() + 1, 0)
|
||||||
|
if integer_digits > precision - scale:
|
||||||
|
raise ValueError(f"{field} exceeds precision {precision},{scale}")
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
class ExactDecimal(TypeDecorator[Decimal]):
|
||||||
|
"""Fixed-point Decimal which uses SQLite TEXT, never a binary float."""
|
||||||
|
|
||||||
|
impl = Numeric
|
||||||
|
cache_ok = True
|
||||||
|
|
||||||
|
def __init__(self, precision: int, scale: int) -> None:
|
||||||
|
self.precision = precision
|
||||||
|
self.scale = scale
|
||||||
|
super().__init__(precision=precision, scale=scale)
|
||||||
|
|
||||||
|
def load_dialect_impl(self, dialect):
|
||||||
|
if dialect.name == "sqlite":
|
||||||
|
return dialect.type_descriptor(String(self.precision + 2))
|
||||||
|
return dialect.type_descriptor(Numeric(self.precision, self.scale, asdecimal=True))
|
||||||
|
|
||||||
|
def process_bind_param(self, value: Decimal | None, dialect) -> Decimal | str | None:
|
||||||
|
if value is None:
|
||||||
|
return None
|
||||||
|
value = _validate_fixed_decimal(value, self.precision, self.scale, "decimal value")
|
||||||
|
if dialect.name == "sqlite":
|
||||||
|
return format(value, f".{self.scale}f")
|
||||||
|
return value
|
||||||
|
|
||||||
|
def process_result_value(self, value: Decimal | str | None, _dialect) -> Decimal | None:
|
||||||
|
return None if value is None else Decimal(value)
|
||||||
|
|
||||||
|
|
||||||
|
class DecimalJSON(TypeDecorator[dict]):
|
||||||
|
"""JSON which serializes Decimal values as strings on every write path."""
|
||||||
|
|
||||||
|
impl = JSON
|
||||||
|
cache_ok = True
|
||||||
|
|
||||||
|
def process_bind_param(self, value: Any, _dialect) -> Any:
|
||||||
|
return None if value is None else _decimal_json(value)
|
||||||
|
|
||||||
|
|
||||||
|
class UTCDateTime(TypeDecorator[datetime]):
|
||||||
|
"""UTC timestamps that preserve instant identity on SQLite and other dialects."""
|
||||||
|
|
||||||
|
impl = DateTime(timezone=True)
|
||||||
|
cache_ok = True
|
||||||
|
|
||||||
|
def __init__(self, field: str) -> None:
|
||||||
|
self.field = field
|
||||||
|
super().__init__()
|
||||||
|
|
||||||
|
def process_bind_param(self, value: datetime | None, _dialect) -> datetime | None:
|
||||||
|
return None if value is None else _normalise_utc_period(value, self.field)
|
||||||
|
|
||||||
|
def process_result_value(self, value: datetime | None, _dialect) -> datetime | None:
|
||||||
|
if value is None:
|
||||||
|
return None
|
||||||
|
if value.tzinfo is None or value.utcoffset() is None:
|
||||||
|
return value.replace(tzinfo=timezone.utc)
|
||||||
|
return value.astimezone(timezone.utc)
|
||||||
|
|
||||||
|
|
||||||
|
def _require_aware_period(start: datetime, end: datetime) -> None:
|
||||||
|
if start.tzinfo is None or start.utcoffset() is None:
|
||||||
|
raise ValueError("period_start must be timezone-aware")
|
||||||
|
if end.tzinfo is None or end.utcoffset() is None:
|
||||||
|
raise ValueError("period_end must be timezone-aware")
|
||||||
|
if end <= start:
|
||||||
|
raise ValueError("period_end must be after period_start")
|
||||||
|
|
||||||
|
|
||||||
|
def _normalise_utc_period(value: datetime, field: str) -> datetime:
|
||||||
|
if value.tzinfo is None or value.utcoffset() is None:
|
||||||
|
raise ValueError(f"{field} must be timezone-aware")
|
||||||
|
return value.astimezone(timezone.utc)
|
||||||
|
|
||||||
|
|
||||||
class Meter(Base):
|
class Meter(Base):
|
||||||
"""One physical electricity meter's installation epoch.
|
"""One physical electricity meter's installation epoch.
|
||||||
|
|
||||||
@@ -53,9 +168,7 @@ class Meter(Base):
|
|||||||
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
||||||
|
|
||||||
# Stable internal identity — used as HA Discovery unique_id anchor.
|
# Stable internal identity — used as HA Discovery unique_id anchor.
|
||||||
uuid: Mapped[str] = mapped_column(
|
uuid: Mapped[str] = mapped_column(String(36), unique=True, nullable=False, default=_uuid4_str)
|
||||||
String(36), unique=True, nullable=False, default=_uuid4_str
|
|
||||||
)
|
|
||||||
|
|
||||||
# Human-readable label for this physical meter (e.g. address, serial, tariff zone).
|
# Human-readable label for this physical meter (e.g. address, serial, tariff zone).
|
||||||
label: Mapped[str] = mapped_column(String(255), nullable=False)
|
label: Mapped[str] = mapped_column(String(255), nullable=False)
|
||||||
@@ -85,6 +198,10 @@ class Meter(Base):
|
|||||||
back_populates="meter", cascade="save-update, merge"
|
back_populates="meter", cascade="save-update, merge"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
source_bindings: Mapped[list["MeterSourceBinding"]] = relationship(
|
||||||
|
back_populates="meter", cascade="save-update, merge"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
class DsmrReading(Base):
|
class DsmrReading(Base):
|
||||||
"""One down-sampled DSMR telegram stored as a full JSON blob.
|
"""One down-sampled DSMR telegram stored as a full JSON blob.
|
||||||
@@ -93,8 +210,9 @@ class DsmrReading(Base):
|
|||||||
(that field overflows and must be manually reset to zero — a known DSMR
|
(that field overflows and must be manually reset to zero — a known DSMR
|
||||||
quirk — so relying on it for uniqueness risks silently dropping new data).
|
quirk — so relying on it for uniqueness risks silently dropping new data).
|
||||||
The table's own autoincrement ``id`` PK is the stable internal identity, and
|
The table's own autoincrement ``id`` PK is the stable internal identity, and
|
||||||
``recorded_at`` (the telegram timestamp) is the UNIQUE de-duplication key: a
|
``(meter_source_id, recorded_at)`` is the UNIQUE de-duplication key: each
|
||||||
single P1 meter emits exactly one telegram per timestamp.
|
configured P1 source emits at most one telegram per timestamp, while
|
||||||
|
different sources may legitimately emit at the same instant.
|
||||||
|
|
||||||
``recorded_at`` is a real column (not inside the payload) so time-range
|
``recorded_at`` is a real column (not inside the payload) so time-range
|
||||||
queries are efficient. The entire telegram frame is stored verbatim in
|
queries are efficient. The entire telegram frame is stored verbatim in
|
||||||
@@ -106,29 +224,54 @@ class DsmrReading(Base):
|
|||||||
|
|
||||||
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
||||||
|
|
||||||
# UTC timestamp of the sample — real column, UNIQUE (telegram-id-independent
|
# UTC timestamp of the sample. Idempotency is per configured source, so
|
||||||
# idempotency key). The unique index also serves time-range queries.
|
# distinct P1 sources may legitimately emit at the same instant.
|
||||||
recorded_at: Mapped[datetime] = mapped_column(
|
recorded_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||||
DateTime(timezone=True), nullable=False, unique=True
|
|
||||||
)
|
|
||||||
|
|
||||||
# Telegram's own id (DSMR Reader assigns it). Stored only as a reference /
|
# Telegram's own id (DSMR Reader assigns it). Stored only as a reference /
|
||||||
# debugging aid — NOT used for uniqueness or idempotency (it overflows and
|
# debugging aid — NOT used for uniqueness or idempotency (it overflows and
|
||||||
# gets reset to zero). Nullable because some DSMR sources may not emit one.
|
# gets reset to zero). Nullable because some DSMR sources may not emit one.
|
||||||
source_id: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
telegram_id: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
||||||
|
|
||||||
|
# Compatibility for the pre-M8 ingest implementation. This is an ORM
|
||||||
|
# alias only; the physical database column is ``telegram_id``.
|
||||||
|
source_id = synonym("telegram_id")
|
||||||
|
|
||||||
|
# The configured source is the durable identity of the cumulative reading
|
||||||
|
# stream. It is non-null after the revision-16 historical adoption.
|
||||||
|
meter_source_id: Mapped[int] = mapped_column(
|
||||||
|
ForeignKey("meter_source.id", ondelete="RESTRICT"), nullable=False, index=True
|
||||||
|
)
|
||||||
|
|
||||||
# Full telegram frame as a JSON object; values are typically JSON strings
|
# Full telegram frame as a JSON object; values are typically JSON strings
|
||||||
# (e.g. "20915.154") — callers must cast to Decimal before arithmetic.
|
# (e.g. "20915.154") — callers must cast to Decimal before arithmetic.
|
||||||
payload: Mapped[dict] = mapped_column(JSON, nullable=False)
|
payload: Mapped[dict] = mapped_column(JSON, nullable=False)
|
||||||
|
|
||||||
|
__table_args__ = (
|
||||||
|
UniqueConstraint(
|
||||||
|
"meter_source_id", "recorded_at", name="uq_dsmr_reading_source_recorded_at"
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@event.listens_for(DsmrReading, "before_insert")
|
||||||
|
def _supply_legacy_dsmr_source(_mapper, connection, target: DsmrReading) -> None:
|
||||||
|
"""Keep the pre-T04 single-source writer working during the schema handoff."""
|
||||||
|
if target.meter_source_id is None:
|
||||||
|
target.meter_source_id = connection.execute(
|
||||||
|
text("SELECT id FROM meter_source WHERE kind = 'dsmr_mqtt' ORDER BY id LIMIT 1")
|
||||||
|
).scalar_one()
|
||||||
|
|
||||||
|
|
||||||
class EnergyContract(Base):
|
class EnergyContract(Base):
|
||||||
"""Contract head: a named energy contract with a chosen pricing strategy.
|
"""Contract head: a named energy contract with a chosen pricing strategy.
|
||||||
|
|
||||||
``kind`` determines which price strategy is used (``manual`` for fixed
|
``kind`` determines which price strategy is used (``manual`` for fixed
|
||||||
dual-tariff rates entered by the user, ``tibber`` for dynamic API prices).
|
dual-tariff rates entered by the user, ``tibber`` for dynamic API prices).
|
||||||
Only one contract may be ``active`` at a time; the service layer enforces
|
A contract belongs to an energy ``scope`` (currently electricity; thermal
|
||||||
mutual exclusion. Specific pricing values live in ``EnergyContractVersion``
|
profiles are reserved for the next milestone). Only one contract may be
|
||||||
|
``active`` per scope; the service layer enforces mutual exclusion. Specific
|
||||||
|
pricing values live in ``EnergyContractVersion``
|
||||||
so that price changes can be tracked without modifying historical records.
|
so that price changes can be tracked without modifying historical records.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
@@ -144,6 +287,12 @@ class EnergyContract(Base):
|
|||||||
# migration simple and the strategy registry extensible.
|
# migration simple and the strategy registry extensible.
|
||||||
kind: Mapped[str] = mapped_column(String(32), nullable=False)
|
kind: Mapped[str] = mapped_column(String(32), nullable=False)
|
||||||
|
|
||||||
|
# Billing domain. The service registry derives this from ``kind`` so API
|
||||||
|
# callers cannot move a pricing strategy into an incompatible domain.
|
||||||
|
scope: Mapped[str] = mapped_column(
|
||||||
|
String(32), nullable=False, default="electricity", index=True
|
||||||
|
)
|
||||||
|
|
||||||
# Whether this is the currently active contract (at most one should be True).
|
# Whether this is the currently active contract (at most one should be True).
|
||||||
active: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
|
active: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
|
||||||
|
|
||||||
@@ -179,15 +328,11 @@ class EnergyContractVersion(Base):
|
|||||||
)
|
)
|
||||||
|
|
||||||
# Start of this version's validity window (inclusive, UTC).
|
# Start of this version's validity window (inclusive, UTC).
|
||||||
effective_from: Mapped[datetime] = mapped_column(
|
effective_from: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||||
DateTime(timezone=True), nullable=False
|
|
||||||
)
|
|
||||||
|
|
||||||
# End of this version's validity window (exclusive, UTC). NULL means open-ended
|
# End of this version's validity window (exclusive, UTC). NULL means open-ended
|
||||||
# (i.e. this is the most recent / current version).
|
# (i.e. this is the most recent / current version).
|
||||||
effective_to: Mapped[datetime | None] = mapped_column(
|
effective_to: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
|
||||||
DateTime(timezone=True), nullable=True
|
|
||||||
)
|
|
||||||
|
|
||||||
# Pricing values as a JSON object conforming to the profile structure for
|
# Pricing values as a JSON object conforming to the profile structure for
|
||||||
# ``contract.kind`` (validated by the application layer against the YAML profile).
|
# ``contract.kind`` (validated by the application layer against the YAML profile).
|
||||||
@@ -292,6 +437,12 @@ class EnergyCostPeriod(Base):
|
|||||||
ForeignKey("meter.id", ondelete="RESTRICT"), nullable=True
|
ForeignKey("meter.id", ondelete="RESTRICT"), nullable=True
|
||||||
)
|
)
|
||||||
|
|
||||||
|
# Nullable for historical and degraded rows. Every new normal period
|
||||||
|
# points at the one binding that supplied both cumulative endpoints.
|
||||||
|
source_binding_id: Mapped[int | None] = mapped_column(
|
||||||
|
ForeignKey("meter_source_binding.id", ondelete="RESTRICT"), nullable=True
|
||||||
|
)
|
||||||
|
|
||||||
# True when the period was computed with incomplete data (missing readings or
|
# True when the period was computed with incomplete data (missing readings or
|
||||||
# missing price); serves as a flag for later recomputation.
|
# missing price); serves as a flag for later recomputation.
|
||||||
degraded: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
|
degraded: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
|
||||||
@@ -307,6 +458,95 @@ class EnergyCostPeriod(Base):
|
|||||||
# Relationship back to the meter epoch.
|
# Relationship back to the meter epoch.
|
||||||
meter: Mapped["Meter | None"] = relationship(back_populates="cost_periods")
|
meter: Mapped["Meter | None"] = relationship(back_populates="cost_periods")
|
||||||
|
|
||||||
|
source_binding: Mapped["MeterSourceBinding | None"] = relationship(
|
||||||
|
back_populates="cost_periods"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class MeterCostPeriod(Base):
|
||||||
|
"""Auditable commodity-scoped ledger row for one half-open metering period.
|
||||||
|
|
||||||
|
A degraded row intentionally permits missing audit links; services must
|
||||||
|
still require them before writing a normal row.
|
||||||
|
"""
|
||||||
|
|
||||||
|
__tablename__ = "meter_cost_period"
|
||||||
|
|
||||||
|
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
||||||
|
commodity: Mapped[str] = mapped_column(String(32), nullable=False)
|
||||||
|
period_start: Mapped[datetime] = mapped_column(UTCDateTime("period_start"), nullable=False)
|
||||||
|
period_end: Mapped[datetime] = mapped_column(UTCDateTime("period_end"), nullable=False)
|
||||||
|
meter_id: Mapped[int | None] = mapped_column(
|
||||||
|
ForeignKey("meter.id", ondelete="RESTRICT"), nullable=True
|
||||||
|
)
|
||||||
|
source_binding_id: Mapped[int | None] = mapped_column(
|
||||||
|
ForeignKey("meter_source_binding.id", ondelete="RESTRICT"), nullable=True
|
||||||
|
)
|
||||||
|
contract_version_id: Mapped[int | None] = mapped_column(
|
||||||
|
ForeignKey("energy_contract_version.id", ondelete="RESTRICT"), nullable=True
|
||||||
|
)
|
||||||
|
|
||||||
|
# SQLite reliably round-trips at most fifteen significant decimal digits.
|
||||||
|
# WarmteLink itself reports 0.001 units, so nine cost fractional digits
|
||||||
|
# retain a six-place tariff times that source precision without float loss.
|
||||||
|
quantity: Mapped[Decimal] = mapped_column(ExactDecimal(15, 6), nullable=False)
|
||||||
|
cost: Mapped[Decimal] = mapped_column(ExactDecimal(15, 9), nullable=False)
|
||||||
|
currency: Mapped[str] = mapped_column(String(8), nullable=False)
|
||||||
|
cost_breakdown: Mapped[dict] = mapped_column(DecimalJSON(), nullable=False, default=dict)
|
||||||
|
pricing_snapshot: Mapped[dict] = mapped_column(DecimalJSON(), nullable=False, default=dict)
|
||||||
|
quality: Mapped[str] = mapped_column(String(32), nullable=False, default="valid")
|
||||||
|
degraded: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
|
||||||
|
degraded_reason: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
||||||
|
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||||
|
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||||
|
|
||||||
|
meter: Mapped["Meter | None"] = relationship()
|
||||||
|
source_binding: Mapped["MeterSourceBinding | None"] = relationship()
|
||||||
|
contract_version: Mapped["EnergyContractVersion | None"] = relationship()
|
||||||
|
|
||||||
|
__table_args__ = (
|
||||||
|
CheckConstraint(
|
||||||
|
"degraded OR (meter_id IS NOT NULL AND source_binding_id IS NOT NULL "
|
||||||
|
"AND contract_version_id IS NOT NULL)",
|
||||||
|
name="ck_meter_cost_period_normal_audit_links",
|
||||||
|
),
|
||||||
|
CheckConstraint("period_end > period_start", name="ck_meter_cost_period_positive_interval"),
|
||||||
|
CheckConstraint(
|
||||||
|
"NOT degraded OR (degraded_reason IS NOT NULL AND length(trim(degraded_reason)) > 0)",
|
||||||
|
name="ck_meter_cost_period_degraded_reason",
|
||||||
|
),
|
||||||
|
UniqueConstraint("commodity", "period_start", name="uq_meter_cost_period_commodity_start"),
|
||||||
|
Index("ix_meter_cost_period_commodity_start", "commodity", "period_start"),
|
||||||
|
Index("ix_meter_cost_period_source_binding_id", "source_binding_id"),
|
||||||
|
)
|
||||||
|
|
||||||
|
@validates("cost_breakdown", "pricing_snapshot")
|
||||||
|
def _validate_decimal_json(self, _key: str, value: dict) -> dict:
|
||||||
|
return _decimal_json(value)
|
||||||
|
|
||||||
|
@validates("quantity", "cost")
|
||||||
|
def _validate_fixed_decimal(self, key: str, value: Decimal) -> Decimal:
|
||||||
|
precision, scale = (15, 6) if key == "quantity" else (15, 9)
|
||||||
|
return _validate_fixed_decimal(value, precision, scale, key)
|
||||||
|
|
||||||
|
@validates("period_start", "period_end")
|
||||||
|
def _normalise_period(self, key: str, value: datetime) -> datetime:
|
||||||
|
return _normalise_utc_period(value, key)
|
||||||
|
|
||||||
|
|
||||||
|
@event.listens_for(MeterCostPeriod, "before_insert")
|
||||||
|
@event.listens_for(MeterCostPeriod, "before_update")
|
||||||
|
def _validate_meter_cost_period(_mapper, _connection, target: MeterCostPeriod) -> None:
|
||||||
|
_require_aware_period(target.period_start, target.period_end)
|
||||||
|
if not target.degraded and (
|
||||||
|
target.meter_id is None
|
||||||
|
or target.source_binding_id is None
|
||||||
|
or target.contract_version_id is None
|
||||||
|
):
|
||||||
|
raise ValueError("normal meter cost periods require meter, binding, and contract version")
|
||||||
|
if target.degraded and not target.degraded_reason:
|
||||||
|
raise ValueError("degraded meter cost periods require a degraded_reason")
|
||||||
|
|
||||||
|
|
||||||
# Index on recorded_at for efficient time-range queries on DSMR readings.
|
# Index on recorded_at for efficient time-range queries on DSMR readings.
|
||||||
# (The ORM-level index=True on recorded_at already creates ix_dsmr_reading_recorded_at;
|
# (The ORM-level index=True on recorded_at already creates ix_dsmr_reading_recorded_at;
|
||||||
|
|||||||
@@ -0,0 +1,165 @@
|
|||||||
|
"""Protocol-agnostic source, channel, and meter-binding identity models."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import uuid as _uuid
|
||||||
|
from datetime import datetime
|
||||||
|
from decimal import Decimal
|
||||||
|
from typing import TYPE_CHECKING
|
||||||
|
|
||||||
|
from sqlalchemy import (
|
||||||
|
Boolean,
|
||||||
|
CheckConstraint,
|
||||||
|
DateTime,
|
||||||
|
ForeignKey,
|
||||||
|
Index,
|
||||||
|
Integer,
|
||||||
|
Numeric,
|
||||||
|
String,
|
||||||
|
UniqueConstraint,
|
||||||
|
)
|
||||||
|
from sqlalchemy.orm import Mapped, mapped_column, relationship
|
||||||
|
from sqlalchemy.types import JSON
|
||||||
|
|
||||||
|
from app.db import Base
|
||||||
|
|
||||||
|
if TYPE_CHECKING:
|
||||||
|
from app.models.energy import EnergyCostPeriod, Meter
|
||||||
|
|
||||||
|
|
||||||
|
def _uuid4_str() -> str:
|
||||||
|
return str(_uuid.uuid4())
|
||||||
|
|
||||||
|
|
||||||
|
def half_open_intervals_overlap(
|
||||||
|
started_at: datetime,
|
||||||
|
ended_at: datetime | None,
|
||||||
|
other_started_at: datetime,
|
||||||
|
other_ended_at: datetime | None,
|
||||||
|
) -> bool:
|
||||||
|
"""Return whether two ``[started_at, ended_at)`` intervals overlap.
|
||||||
|
|
||||||
|
``None`` denotes an open-ended interval. Equal boundaries do not overlap,
|
||||||
|
which lets a source binding hand off at one exact timestamp.
|
||||||
|
"""
|
||||||
|
return (other_ended_at is None or started_at < other_ended_at) and (
|
||||||
|
ended_at is None or other_started_at < ended_at
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class MeterSource(Base):
|
||||||
|
"""A configured protocol connection that discovers one or more channels."""
|
||||||
|
|
||||||
|
__tablename__ = "meter_source"
|
||||||
|
|
||||||
|
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
||||||
|
uuid: Mapped[str] = mapped_column(String(36), unique=True, nullable=False, default=_uuid4_str)
|
||||||
|
name: Mapped[str] = mapped_column(String(255), nullable=False)
|
||||||
|
kind: Mapped[str] = mapped_column(String(64), nullable=False)
|
||||||
|
enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True)
|
||||||
|
config: Mapped[dict] = mapped_column(JSON, nullable=False, default=dict)
|
||||||
|
status: Mapped[str] = mapped_column(String(32), nullable=False, default="unknown")
|
||||||
|
last_seen_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
|
||||||
|
last_error: Mapped[str | None] = mapped_column(String(1024), nullable=True)
|
||||||
|
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||||
|
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||||
|
|
||||||
|
channels: Mapped[list["MeterSourceChannel"]] = relationship(
|
||||||
|
back_populates="source", cascade="save-update, merge"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class MeterSourceChannel(Base):
|
||||||
|
"""A stable cumulative measurement identity discovered from a source."""
|
||||||
|
|
||||||
|
__tablename__ = "meter_source_channel"
|
||||||
|
|
||||||
|
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
||||||
|
uuid: Mapped[str] = mapped_column(String(36), unique=True, nullable=False, default=_uuid4_str)
|
||||||
|
source_id: Mapped[int] = mapped_column(
|
||||||
|
ForeignKey("meter_source.id", ondelete="RESTRICT"), nullable=False, index=True
|
||||||
|
)
|
||||||
|
channel_key: Mapped[str] = mapped_column(String(128), nullable=False)
|
||||||
|
label: Mapped[str] = mapped_column(String(255), nullable=False)
|
||||||
|
suggested_commodity: Mapped[str | None] = mapped_column(String(32), nullable=True)
|
||||||
|
unit: Mapped[str] = mapped_column(String(32), nullable=False)
|
||||||
|
device_type: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||||
|
fingerprint: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||||
|
latest_value: Mapped[float | None] = mapped_column(Numeric(20, 6), nullable=True)
|
||||||
|
latest_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
|
||||||
|
latest_quality: Mapped[str | None] = mapped_column(String(32), nullable=True)
|
||||||
|
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||||
|
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||||
|
|
||||||
|
source: Mapped["MeterSource"] = relationship(back_populates="channels")
|
||||||
|
bindings: Mapped[list["MeterSourceBinding"]] = relationship(
|
||||||
|
back_populates="channel", cascade="save-update, merge"
|
||||||
|
)
|
||||||
|
warmtelink_readings: Mapped[list["WarmteLinkReading"]] = relationship(
|
||||||
|
back_populates="channel", cascade="save-update, merge"
|
||||||
|
)
|
||||||
|
|
||||||
|
__table_args__ = (
|
||||||
|
UniqueConstraint("source_id", "channel_key", name="uq_meter_source_channel_source_key"),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class WarmteLinkReading(Base):
|
||||||
|
"""One accepted scalar cumulative reading from a WarmteLink channel."""
|
||||||
|
|
||||||
|
__tablename__ = "warmtelink_reading"
|
||||||
|
|
||||||
|
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
||||||
|
channel_id: Mapped[int] = mapped_column(
|
||||||
|
ForeignKey("meter_source_channel.id", ondelete="RESTRICT"), nullable=False
|
||||||
|
)
|
||||||
|
# These timestamps retain their UTC-aware application semantics. SQLite
|
||||||
|
# stores them without an offset, so callers must always supply aware UTC.
|
||||||
|
recorded_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||||
|
received_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||||
|
# SQLite's ORM Numeric path can exactly round-trip this 12-integer-digit
|
||||||
|
# range at scale 3. That is ample for a long-lived cumulative meter while
|
||||||
|
# retaining the protocol's 0.001 resolution without float conversion.
|
||||||
|
value: Mapped[Decimal] = mapped_column(Numeric(15, 3), nullable=False)
|
||||||
|
unit: Mapped[str] = mapped_column(String(32), nullable=False)
|
||||||
|
quality: Mapped[str] = mapped_column(String(32), nullable=False)
|
||||||
|
equipment_fingerprint: Mapped[str] = mapped_column(String(64), nullable=False)
|
||||||
|
|
||||||
|
channel: Mapped["MeterSourceChannel"] = relationship(back_populates="warmtelink_readings")
|
||||||
|
|
||||||
|
__table_args__ = (
|
||||||
|
CheckConstraint(
|
||||||
|
"quality IN ('valid', 'invalid', 'unverifiable')",
|
||||||
|
name="ck_warmtelink_reading_quality",
|
||||||
|
),
|
||||||
|
UniqueConstraint("channel_id", "recorded_at", name="uq_warmtelink_reading_channel_recorded_at"),
|
||||||
|
Index("ix_warmtelink_reading_recorded_at", "recorded_at"),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class MeterSourceBinding(Base):
|
||||||
|
"""Connect one source channel to one physical meter for a half-open window."""
|
||||||
|
|
||||||
|
__tablename__ = "meter_source_binding"
|
||||||
|
|
||||||
|
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
||||||
|
uuid: Mapped[str] = mapped_column(String(36), unique=True, nullable=False, default=_uuid4_str)
|
||||||
|
meter_id: Mapped[int] = mapped_column(
|
||||||
|
ForeignKey("meter.id", ondelete="RESTRICT"), nullable=False, index=True
|
||||||
|
)
|
||||||
|
channel_id: Mapped[int] = mapped_column(
|
||||||
|
ForeignKey("meter_source_channel.id", ondelete="RESTRICT"), nullable=False, index=True
|
||||||
|
)
|
||||||
|
started_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||||
|
ended_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
|
||||||
|
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||||
|
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||||
|
|
||||||
|
meter: Mapped["Meter"] = relationship(back_populates="source_bindings")
|
||||||
|
channel: Mapped["MeterSourceChannel"] = relationship(back_populates="bindings")
|
||||||
|
cost_periods: Mapped[list["EnergyCostPeriod"]] = relationship(
|
||||||
|
back_populates="source_binding", cascade="save-update, merge", passive_deletes="all"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
Index("ix_meter_source_kind_enabled", MeterSource.kind, MeterSource.enabled)
|
||||||
+44
-7
@@ -76,6 +76,24 @@ class PricesResponse(BaseModel):
|
|||||||
"Null for tibber contracts and when no active contract exists."
|
"Null for tibber contracts and when no active contract exists."
|
||||||
),
|
),
|
||||||
)
|
)
|
||||||
|
contract_version_id: int | None = Field(
|
||||||
|
default=None,
|
||||||
|
description="Thermal active contract version identifier; omitted for electricity.",
|
||||||
|
)
|
||||||
|
effective_from: datetime | None = Field(
|
||||||
|
default=None,
|
||||||
|
description="Thermal contract version start; omitted for electricity.",
|
||||||
|
)
|
||||||
|
effective_to: datetime | None = Field(
|
||||||
|
default=None,
|
||||||
|
description="Thermal contract version end; omitted for electricity.",
|
||||||
|
)
|
||||||
|
values: dict[str, dict[str, str]] | None = Field(
|
||||||
|
default=None,
|
||||||
|
description="Thermal normalized Decimal-string contract values; omitted for electricity.",
|
||||||
|
)
|
||||||
|
|
||||||
|
model_config = {"ser_json_exclude_none": True}
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
@@ -95,13 +113,18 @@ class CostPeriodSchema(BaseModel):
|
|||||||
export_revenue: float = Field(description="Revenue from electricity fed to grid (EUR).")
|
export_revenue: float = Field(description="Revenue from electricity fed to grid (EUR).")
|
||||||
net_cost: float = Field(description="import_cost − export_revenue (EUR).")
|
net_cost: float = Field(description="import_cost − export_revenue (EUR).")
|
||||||
currency: str = Field(description="ISO 4217 currency code.")
|
currency: str = Field(description="ISO 4217 currency code.")
|
||||||
degraded: bool = Field(
|
degraded: bool = Field(description="True when the period was computed with incomplete data.")
|
||||||
description="True when the period was computed with incomplete data."
|
|
||||||
)
|
|
||||||
contract_version_id: int | None = Field(
|
contract_version_id: int | None = Field(
|
||||||
default=None,
|
default=None,
|
||||||
description="FK to the contract version used for this billing period (null when degraded).",
|
description="FK to the contract version used for this billing period (null when degraded).",
|
||||||
)
|
)
|
||||||
|
source_binding_id: int | None = Field(
|
||||||
|
default=None,
|
||||||
|
description=(
|
||||||
|
"FK to the source binding that supplied both cumulative endpoints "
|
||||||
|
"(null for legacy or degraded periods)."
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
model_config = {"from_attributes": True}
|
model_config = {"from_attributes": True}
|
||||||
|
|
||||||
@@ -121,15 +144,29 @@ class CostsResponse(BaseModel):
|
|||||||
class SummaryResponse(BaseModel):
|
class SummaryResponse(BaseModel):
|
||||||
"""Response for GET /api/energy/costs/summary.
|
"""Response for GET /api/energy/costs/summary.
|
||||||
|
|
||||||
All monetary values are in ``currency``.
|
Monetary values are in ``currency``; the ``*_kwh`` fields are energy totals
|
||||||
|
in kWh. ``metered_import``/``metered_export`` are **money**, not energy —
|
||||||
|
only the ``_kwh``-suffixed fields carry kWh.
|
||||||
|
|
||||||
``total_payable = metered_net + fixed_costs − credits``
|
``total_payable = metered_net + fixed_costs − credits``
|
||||||
"""
|
"""
|
||||||
|
|
||||||
currency: str
|
currency: str
|
||||||
metered_import: float = Field(description="Σ import_cost for non-degraded periods.")
|
metered_import: float = Field(
|
||||||
metered_export: float = Field(description="Σ export_revenue for non-degraded periods.")
|
description="Σ import_cost for non-degraded periods (money, in `currency`)."
|
||||||
metered_net: float = Field(description="Σ net_cost for non-degraded periods.")
|
)
|
||||||
|
metered_export: float = Field(
|
||||||
|
description="Σ export_revenue for non-degraded periods (money, in `currency`)."
|
||||||
|
)
|
||||||
|
metered_net: float = Field(
|
||||||
|
description="Σ net_cost for non-degraded periods (money, in `currency`)."
|
||||||
|
)
|
||||||
|
metered_import_kwh: float = Field(
|
||||||
|
description="Σ (d1_kwh + d2_kwh) for non-degraded periods (energy imported, kWh)."
|
||||||
|
)
|
||||||
|
metered_export_kwh: float = Field(
|
||||||
|
description="Σ (r1_kwh + r2_kwh) for non-degraded periods (energy exported, kWh)."
|
||||||
|
)
|
||||||
fixed_costs: float = Field(
|
fixed_costs: float = Field(
|
||||||
description="Standing charges (network_fee + management_fee) apportioned over the interval."
|
description="Standing charges (network_fee + management_fee) apportioned over the interval."
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -51,6 +51,7 @@ class ContractResponse(BaseModel):
|
|||||||
id: int
|
id: int
|
||||||
name: str
|
name: str
|
||||||
kind: str
|
kind: str
|
||||||
|
scope: str
|
||||||
active: bool
|
active: bool
|
||||||
currency: str
|
currency: str
|
||||||
created_at: datetime
|
created_at: datetime
|
||||||
@@ -69,6 +70,7 @@ class ContractDetailResponse(BaseModel):
|
|||||||
id: int
|
id: int
|
||||||
name: str
|
name: str
|
||||||
kind: str
|
kind: str
|
||||||
|
scope: str
|
||||||
active: bool
|
active: bool
|
||||||
currency: str
|
currency: str
|
||||||
created_at: datetime
|
created_at: datetime
|
||||||
@@ -101,6 +103,7 @@ class ContractCreate(BaseModel):
|
|||||||
|
|
||||||
name: str = Field(..., min_length=1, max_length=255)
|
name: str = Field(..., min_length=1, max_length=255)
|
||||||
kind: str = Field(..., min_length=1, max_length=32)
|
kind: str = Field(..., min_length=1, max_length=32)
|
||||||
|
scope: str | None = Field(default=None, min_length=1, max_length=32)
|
||||||
currency: str = Field(default="EUR", min_length=1, max_length=8)
|
currency: str = Field(default="EUR", min_length=1, max_length=8)
|
||||||
values: dict[str, Any]
|
values: dict[str, Any]
|
||||||
effective_from: datetime | None = Field(
|
effective_from: datetime | None = Field(
|
||||||
|
|||||||
@@ -49,10 +49,21 @@ class MeterResponse(BaseModel):
|
|||||||
reason: str
|
reason: str
|
||||||
note: str | None
|
note: str | None
|
||||||
created_at: datetime
|
created_at: datetime
|
||||||
|
bindings: list["MeterBindingSummary"] = Field(default_factory=list)
|
||||||
|
|
||||||
model_config = {"from_attributes": True}
|
model_config = {"from_attributes": True}
|
||||||
|
|
||||||
|
|
||||||
|
class MeterBindingSummary(BaseModel):
|
||||||
|
"""Stable, non-sensitive binding identity embedded in meter responses."""
|
||||||
|
|
||||||
|
uuid: str
|
||||||
|
source_channel_uuid: str
|
||||||
|
source_uuid: str
|
||||||
|
started_at: datetime
|
||||||
|
ended_at: datetime | None
|
||||||
|
|
||||||
|
|
||||||
class MeterListResponse(BaseModel):
|
class MeterListResponse(BaseModel):
|
||||||
"""Response schema for GET /api/energy/meters.
|
"""Response schema for GET /api/energy/meters.
|
||||||
|
|
||||||
@@ -107,6 +118,12 @@ class MeterDeclareRequest(BaseModel):
|
|||||||
max_length=32,
|
max_length=32,
|
||||||
description="Energy commodity this meter measures. Defaults to 'electricity'.",
|
description="Energy commodity this meter measures. Defaults to 'electricity'.",
|
||||||
)
|
)
|
||||||
|
source_channel_uuid: str | None = Field(
|
||||||
|
default=None,
|
||||||
|
min_length=1,
|
||||||
|
max_length=36,
|
||||||
|
description="Optional compatible source channel to bind atomically to this meter.",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
class MeterPatchRequest(BaseModel):
|
class MeterPatchRequest(BaseModel):
|
||||||
@@ -129,3 +146,9 @@ class MeterPatchRequest(BaseModel):
|
|||||||
"Triggers billing recompute over the affected window."
|
"Triggers billing recompute over the affected window."
|
||||||
),
|
),
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class MeterCloseRequest(BaseModel):
|
||||||
|
"""Close the active meter epoch at an exclusive end boundary."""
|
||||||
|
|
||||||
|
ended_at: datetime
|
||||||
|
|||||||
@@ -0,0 +1,62 @@
|
|||||||
|
"""Schemas for the commodity-scoped thermal cost ledger."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import datetime
|
||||||
|
from typing import Literal
|
||||||
|
|
||||||
|
from pydantic import BaseModel, Field
|
||||||
|
|
||||||
|
|
||||||
|
class MeterCostPeriodSchema(BaseModel):
|
||||||
|
"""One auditable thermal ledger row; all Decimal values are JSON strings."""
|
||||||
|
|
||||||
|
commodity: Literal["heating", "hot_water"]
|
||||||
|
period_start: datetime
|
||||||
|
period_end: datetime
|
||||||
|
meter_id: int | None
|
||||||
|
source_binding_id: int | None
|
||||||
|
contract_version_id: int | None
|
||||||
|
quantity: str
|
||||||
|
cost: str
|
||||||
|
currency: str
|
||||||
|
cost_breakdown: dict[str, str]
|
||||||
|
pricing_snapshot: dict[str, dict[str, str]]
|
||||||
|
quality: str
|
||||||
|
degraded: bool
|
||||||
|
degraded_reason: str | None
|
||||||
|
|
||||||
|
|
||||||
|
class MeterCostsResponse(BaseModel):
|
||||||
|
items: list[MeterCostPeriodSchema]
|
||||||
|
total: int = Field(description="Total matching rows before pagination.")
|
||||||
|
|
||||||
|
|
||||||
|
class ThermalFixedBreakdown(BaseModel):
|
||||||
|
"""D11 annual-standing charges accrued per settled local day, as Decimal strings."""
|
||||||
|
|
||||||
|
heating_network: str
|
||||||
|
metering: str
|
||||||
|
delivery_set: str
|
||||||
|
hot_water_network: str
|
||||||
|
other: str
|
||||||
|
|
||||||
|
|
||||||
|
class ThermalCostSummaryResponse(BaseModel):
|
||||||
|
currency: str
|
||||||
|
heating: str
|
||||||
|
hot_water_heating: str
|
||||||
|
hot_water: str
|
||||||
|
hot_water_tax: str
|
||||||
|
variable_subtotal: str
|
||||||
|
fixed_breakdown: ThermalFixedBreakdown
|
||||||
|
fixed_subtotal: str
|
||||||
|
all_in: str
|
||||||
|
period_count: int
|
||||||
|
degraded_count: int
|
||||||
|
|
||||||
|
|
||||||
|
class MeterCostRecomputeResponse(BaseModel):
|
||||||
|
processed: int
|
||||||
|
normal: int
|
||||||
|
degraded: int
|
||||||
@@ -0,0 +1,162 @@
|
|||||||
|
"""Public schemas for protocol-agnostic meter sources and bindings."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import datetime
|
||||||
|
from decimal import Decimal
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from pydantic import BaseModel, Field
|
||||||
|
|
||||||
|
|
||||||
|
class SourceConfigFieldResponse(BaseModel):
|
||||||
|
name: str
|
||||||
|
value_type: str
|
||||||
|
default: Any = None
|
||||||
|
required: bool
|
||||||
|
secret: bool
|
||||||
|
|
||||||
|
|
||||||
|
class SourceProfileResponse(BaseModel):
|
||||||
|
kind: str
|
||||||
|
fields: list[SourceConfigFieldResponse]
|
||||||
|
defaults: dict[str, Any]
|
||||||
|
capabilities: list[str]
|
||||||
|
allowed_units: list[str]
|
||||||
|
|
||||||
|
|
||||||
|
class SourceProfilesResponse(BaseModel):
|
||||||
|
items: list[SourceProfileResponse]
|
||||||
|
|
||||||
|
|
||||||
|
class MeterSourceCreate(BaseModel):
|
||||||
|
name: str = Field(..., min_length=1, max_length=255)
|
||||||
|
kind: str = Field(..., min_length=1, max_length=64)
|
||||||
|
config: dict[str, Any] = Field(default_factory=dict)
|
||||||
|
enabled: bool = True
|
||||||
|
|
||||||
|
|
||||||
|
class MeterSourcePatch(BaseModel):
|
||||||
|
name: str | None = Field(default=None, min_length=1, max_length=255)
|
||||||
|
config: dict[str, Any] | None = None
|
||||||
|
enabled: bool | None = None
|
||||||
|
|
||||||
|
|
||||||
|
class MeterSourceResponse(BaseModel):
|
||||||
|
uuid: str
|
||||||
|
name: str
|
||||||
|
kind: str
|
||||||
|
enabled: bool
|
||||||
|
config: dict[str, Any]
|
||||||
|
status: str
|
||||||
|
last_seen_at: datetime | None
|
||||||
|
last_error: str | None
|
||||||
|
created_at: datetime
|
||||||
|
updated_at: datetime
|
||||||
|
|
||||||
|
|
||||||
|
class MeterSourceListResponse(BaseModel):
|
||||||
|
items: list[MeterSourceResponse]
|
||||||
|
total: int
|
||||||
|
|
||||||
|
|
||||||
|
class DiscoverResponse(BaseModel):
|
||||||
|
requested: bool
|
||||||
|
supported: bool
|
||||||
|
status: str
|
||||||
|
request_id: int | None = None
|
||||||
|
detail: str | None = None
|
||||||
|
channels: list["DiscoverChannelResponse"] = Field(default_factory=list)
|
||||||
|
|
||||||
|
|
||||||
|
class DiscoverChannelResponse(BaseModel):
|
||||||
|
uuid: str
|
||||||
|
label: str
|
||||||
|
unit: str
|
||||||
|
latest_value: Decimal | None
|
||||||
|
latest_at: datetime | None
|
||||||
|
latest_quality: str | None
|
||||||
|
|
||||||
|
|
||||||
|
class ChannelBindingSummaryResponse(BaseModel):
|
||||||
|
count: int
|
||||||
|
meter_ids: list[int]
|
||||||
|
|
||||||
|
|
||||||
|
class CommodityResponse(BaseModel):
|
||||||
|
key: str
|
||||||
|
unit: str
|
||||||
|
capabilities: list[str]
|
||||||
|
|
||||||
|
|
||||||
|
class CommoditiesResponse(BaseModel):
|
||||||
|
items: list[CommodityResponse]
|
||||||
|
|
||||||
|
|
||||||
|
class MeterSourceChannelResponse(BaseModel):
|
||||||
|
uuid: str
|
||||||
|
label: str
|
||||||
|
suggested_commodity: str | None
|
||||||
|
unit: str
|
||||||
|
device_type: str | None
|
||||||
|
latest_value: Decimal | None
|
||||||
|
latest_at: datetime | None
|
||||||
|
latest_quality: str | None
|
||||||
|
binding_count: int
|
||||||
|
bound_meter_ids: list[int]
|
||||||
|
binding_summary: ChannelBindingSummaryResponse
|
||||||
|
|
||||||
|
|
||||||
|
class MeterSourceChannelListResponse(BaseModel):
|
||||||
|
items: list[MeterSourceChannelResponse]
|
||||||
|
total: int
|
||||||
|
source_status: str
|
||||||
|
|
||||||
|
|
||||||
|
class ChannelReadingResponse(BaseModel):
|
||||||
|
recorded_at: datetime
|
||||||
|
value: Decimal | None = None
|
||||||
|
quality: str | None = None
|
||||||
|
|
||||||
|
|
||||||
|
class ChannelReadingsResponse(BaseModel):
|
||||||
|
items: list[ChannelReadingResponse]
|
||||||
|
total: int
|
||||||
|
|
||||||
|
|
||||||
|
class BindingCreate(BaseModel):
|
||||||
|
source_channel_uuid: str = Field(..., min_length=1, max_length=36)
|
||||||
|
started_at: datetime
|
||||||
|
ended_at: datetime | None = None
|
||||||
|
|
||||||
|
|
||||||
|
class BindingPatch(BaseModel):
|
||||||
|
started_at: datetime | None = None
|
||||||
|
ended_at: datetime | None = None
|
||||||
|
|
||||||
|
|
||||||
|
class BindingTransferRequest(BaseModel):
|
||||||
|
from_binding_uuid: str = Field(..., min_length=1, max_length=36)
|
||||||
|
to_source_channel_uuid: str = Field(..., min_length=1, max_length=36)
|
||||||
|
effective_at: datetime
|
||||||
|
|
||||||
|
|
||||||
|
class BindingResponse(BaseModel):
|
||||||
|
uuid: str
|
||||||
|
meter_id: int
|
||||||
|
source_channel_uuid: str
|
||||||
|
source_uuid: str
|
||||||
|
started_at: datetime
|
||||||
|
ended_at: datetime | None
|
||||||
|
created_at: datetime
|
||||||
|
updated_at: datetime
|
||||||
|
|
||||||
|
|
||||||
|
class BindingListResponse(BaseModel):
|
||||||
|
items: list[BindingResponse]
|
||||||
|
total: int
|
||||||
|
|
||||||
|
|
||||||
|
class BindingTransferResponse(BaseModel):
|
||||||
|
closed_binding: BindingResponse
|
||||||
|
created_binding: BindingResponse
|
||||||
+13
-29
@@ -109,6 +109,7 @@ CONFIG_FIELDS: tuple[ConfigField, ...] = (
|
|||||||
ConfigField("MQTT", "MQTT_USERNAME", "mqtt_username", "MQTT Username"),
|
ConfigField("MQTT", "MQTT_USERNAME", "mqtt_username", "MQTT Username"),
|
||||||
ConfigField("MQTT", "MQTT_PASSWORD", "mqtt_password", "MQTT Password", secret=True),
|
ConfigField("MQTT", "MQTT_PASSWORD", "mqtt_password", "MQTT Password", secret=True),
|
||||||
ConfigField("MQTT", "MQTT_TLS_ENABLED", "mqtt_tls_enabled", "MQTT TLS Enabled", input_type="checkbox"),
|
ConfigField("MQTT", "MQTT_TLS_ENABLED", "mqtt_tls_enabled", "MQTT TLS Enabled", input_type="checkbox"),
|
||||||
|
ConfigField("MQTT", "MQTT_CLIENT_ID", "mqtt_client_id", "MQTT Client ID"),
|
||||||
ConfigField(
|
ConfigField(
|
||||||
"Home Assistant Discovery",
|
"Home Assistant Discovery",
|
||||||
"HA_DISCOVERY_ENABLED",
|
"HA_DISCOVERY_ENABLED",
|
||||||
@@ -129,27 +130,6 @@ CONFIG_FIELDS: tuple[ConfigField, ...] = (
|
|||||||
"HA State Topic Prefix",
|
"HA State Topic Prefix",
|
||||||
),
|
),
|
||||||
ConfigField("Modbus", "MODBUS_POLLING_ENABLED", "modbus_polling_enabled", "Modbus Polling Enabled", input_type="checkbox"),
|
ConfigField("Modbus", "MODBUS_POLLING_ENABLED", "modbus_polling_enabled", "Modbus Polling Enabled", input_type="checkbox"),
|
||||||
ConfigField(
|
|
||||||
"DSMR",
|
|
||||||
"DSMR_INGEST_ENABLED",
|
|
||||||
"dsmr_ingest_enabled",
|
|
||||||
"DSMR Ingest Enabled",
|
|
||||||
input_type="checkbox",
|
|
||||||
),
|
|
||||||
ConfigField("DSMR", "DSMR_MQTT_TOPIC", "dsmr_mqtt_topic", "DSMR MQTT Topic"),
|
|
||||||
ConfigField(
|
|
||||||
"DSMR",
|
|
||||||
"DSMR_SAMPLE_INTERVAL_S",
|
|
||||||
"dsmr_sample_interval_s",
|
|
||||||
"DSMR Sample Interval (s)",
|
|
||||||
input_type="number",
|
|
||||||
),
|
|
||||||
ConfigField(
|
|
||||||
"DSMR",
|
|
||||||
"DSMR_TARIFF_TOPIC",
|
|
||||||
"dsmr_tariff_topic",
|
|
||||||
"DSMR Tariff Topic",
|
|
||||||
),
|
|
||||||
ConfigField(
|
ConfigField(
|
||||||
"Tibber",
|
"Tibber",
|
||||||
"TIBBER_API_TOKEN",
|
"TIBBER_API_TOKEN",
|
||||||
@@ -243,7 +223,12 @@ def save_config_updates(session: Session, form_data: dict[str, str], bootstrap_s
|
|||||||
else:
|
else:
|
||||||
merged_values[field.env_name] = submitted_value
|
merged_values[field.env_name] = submitted_value
|
||||||
|
|
||||||
_validate_config_values(merged_values, bootstrap_settings)
|
validated_settings = _validate_config_values(merged_values, bootstrap_settings)
|
||||||
|
# Persist the canonical client identity as well as using it at runtime. A
|
||||||
|
# whitespace-padded value must not survive in app_config and unexpectedly
|
||||||
|
# reappear in another consumer of the stored settings.
|
||||||
|
if "MQTT_CLIENT_ID" in merged_values:
|
||||||
|
merged_values["MQTT_CLIENT_ID"] = validated_settings.mqtt_client_id
|
||||||
_persist_config_values(session, merged_values)
|
_persist_config_values(session, merged_values)
|
||||||
get_settings.cache_clear()
|
get_settings.cache_clear()
|
||||||
reset_db_caches()
|
reset_db_caches()
|
||||||
@@ -258,7 +243,9 @@ def save_config_value(
|
|||||||
) -> None:
|
) -> None:
|
||||||
current_values = _read_config_values(session)
|
current_values = _read_config_values(session)
|
||||||
current_values[env_name] = value
|
current_values[env_name] = value
|
||||||
_validate_config_values(current_values, bootstrap_settings)
|
validated_settings = _validate_config_values(current_values, bootstrap_settings)
|
||||||
|
if env_name == "MQTT_CLIENT_ID":
|
||||||
|
current_values[env_name] = validated_settings.mqtt_client_id
|
||||||
_persist_config_values(session, current_values)
|
_persist_config_values(session, current_values)
|
||||||
get_settings.cache_clear()
|
get_settings.cache_clear()
|
||||||
reset_db_caches()
|
reset_db_caches()
|
||||||
@@ -277,14 +264,14 @@ def _read_config_values(session: Session) -> dict[str, str]:
|
|||||||
return {row.key: row.value for row in rows}
|
return {row.key: row.value for row in rows}
|
||||||
|
|
||||||
|
|
||||||
def _validate_config_values(config_values: dict[str, str], bootstrap_settings: Settings) -> None:
|
def _validate_config_values(config_values: dict[str, str], bootstrap_settings: Settings) -> Settings:
|
||||||
payload = _settings_payload(bootstrap_settings)
|
payload = _settings_payload(bootstrap_settings)
|
||||||
for field in CONFIG_FIELDS:
|
for field in CONFIG_FIELDS:
|
||||||
if field.env_name in config_values:
|
if field.env_name in config_values:
|
||||||
payload[field.setting_attr] = config_values[field.env_name]
|
payload[field.setting_attr] = config_values[field.env_name]
|
||||||
|
|
||||||
try:
|
try:
|
||||||
Settings(_env_file=None, **payload)
|
return Settings(_env_file=None, **payload)
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
raise ConfigSaveError("invalid config submission") from exc
|
raise ConfigSaveError("invalid config submission") from exc
|
||||||
|
|
||||||
@@ -355,13 +342,10 @@ def _settings_payload(settings: Settings) -> dict[str, Any]:
|
|||||||
"mqtt_username": settings.mqtt_username,
|
"mqtt_username": settings.mqtt_username,
|
||||||
"mqtt_password": settings.mqtt_password,
|
"mqtt_password": settings.mqtt_password,
|
||||||
"mqtt_tls_enabled": settings.mqtt_tls_enabled,
|
"mqtt_tls_enabled": settings.mqtt_tls_enabled,
|
||||||
|
"mqtt_client_id": settings.mqtt_client_id,
|
||||||
"ha_discovery_enabled": settings.ha_discovery_enabled,
|
"ha_discovery_enabled": settings.ha_discovery_enabled,
|
||||||
"ha_discovery_prefix": settings.ha_discovery_prefix,
|
"ha_discovery_prefix": settings.ha_discovery_prefix,
|
||||||
"ha_state_topic_prefix": settings.ha_state_topic_prefix,
|
"ha_state_topic_prefix": settings.ha_state_topic_prefix,
|
||||||
"dsmr_ingest_enabled": settings.dsmr_ingest_enabled,
|
|
||||||
"dsmr_mqtt_topic": settings.dsmr_mqtt_topic,
|
|
||||||
"dsmr_sample_interval_s": settings.dsmr_sample_interval_s,
|
|
||||||
"dsmr_tariff_topic": settings.dsmr_tariff_topic,
|
|
||||||
"tibber_api_token": settings.tibber_api_token,
|
"tibber_api_token": settings.tibber_api_token,
|
||||||
"tibber_home_id": settings.tibber_home_id,
|
"tibber_home_id": settings.tibber_home_id,
|
||||||
}
|
}
|
||||||
|
|||||||
+63
-18
@@ -11,8 +11,8 @@ Design decisions
|
|||||||
setting its ``effective_to`` to the new version's ``effective_from``; raises
|
setting its ``effective_to`` to the new version's ``effective_from``; raises
|
||||||
``ContractVersionError`` if the new date is strictly earlier than the previous
|
``ContractVersionError`` if the new date is strictly earlier than the previous
|
||||||
version's ``effective_from``.
|
version's ``effective_from``.
|
||||||
- ``activate_contract``: mutual-exclusion; sets all other contracts' ``active``
|
- ``activate_contract``: scope-local mutual exclusion; sets other contracts in
|
||||||
to False, then sets the given contract's ``active`` to True.
|
the target scope inactive, then sets the given contract active.
|
||||||
- ``active_contract_version_at``: returns the single version of the currently
|
- ``active_contract_version_at``: returns the single version of the currently
|
||||||
active contract that covers *ts* (``effective_from ≤ ts < effective_to``,
|
active contract that covers *ts* (``effective_from ≤ ts < effective_to``,
|
||||||
or open-ended when ``effective_to`` is None).
|
or open-ended when ``effective_to`` is None).
|
||||||
@@ -32,7 +32,7 @@ import logging
|
|||||||
from datetime import UTC, datetime
|
from datetime import UTC, datetime
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
from sqlalchemy import select
|
from sqlalchemy import select, update
|
||||||
from sqlalchemy.orm import Session
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
from app.integrations.pricing.profiles import validate_values
|
from app.integrations.pricing.profiles import validate_values
|
||||||
@@ -41,6 +41,32 @@ from app.models.energy import EnergyContract, EnergyContractVersion
|
|||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
# This is deliberately separate from the pricing-profile loader. T12 needs to
|
||||||
|
# reserve the thermal domain before T13 supplies its actual profile.
|
||||||
|
CONTRACT_KIND_SCOPES: dict[str, str] = {
|
||||||
|
"manual": "electricity",
|
||||||
|
"tibber": "electricity",
|
||||||
|
"district_heating": "thermal",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
class ContractScopeError(ValueError):
|
||||||
|
"""Raised when a contract kind is unknown or its supplied scope disagrees."""
|
||||||
|
|
||||||
|
|
||||||
|
def contract_scope_for_kind(kind: str, requested_scope: str | None = None) -> str:
|
||||||
|
"""Return the registry-owned scope for *kind*, rejecting client mismatches."""
|
||||||
|
try:
|
||||||
|
scope = CONTRACT_KIND_SCOPES[kind]
|
||||||
|
except KeyError as exc:
|
||||||
|
raise ContractScopeError(f"Unknown energy contract kind: {kind!r}") from exc
|
||||||
|
if requested_scope is not None and requested_scope != scope:
|
||||||
|
raise ContractScopeError(
|
||||||
|
f"Contract kind {kind!r} belongs to scope {scope!r}, not {requested_scope!r}."
|
||||||
|
)
|
||||||
|
return scope
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# Internal helpers
|
# Internal helpers
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
@@ -86,10 +112,14 @@ def get_contract_or_none(session: Session, contract_id: int) -> EnergyContract |
|
|||||||
).scalar_one_or_none()
|
).scalar_one_or_none()
|
||||||
|
|
||||||
|
|
||||||
def list_contracts(session: Session) -> list[EnergyContract]:
|
def list_contracts(session: Session, *, scope: str = "electricity") -> list[EnergyContract]:
|
||||||
"""Return all contracts ordered by id (ascending)."""
|
"""Return contracts in one scope, ordered by id (ascending)."""
|
||||||
return list(
|
return list(
|
||||||
session.execute(select(EnergyContract).order_by(EnergyContract.id)).scalars().all()
|
session.execute(
|
||||||
|
select(EnergyContract)
|
||||||
|
.where(EnergyContract.scope == scope)
|
||||||
|
.order_by(EnergyContract.id)
|
||||||
|
).scalars().all()
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
@@ -117,6 +147,7 @@ def create_contract(
|
|||||||
name: str,
|
name: str,
|
||||||
kind: str,
|
kind: str,
|
||||||
currency: str = "EUR",
|
currency: str = "EUR",
|
||||||
|
scope: str | None = None,
|
||||||
values: dict[str, Any],
|
values: dict[str, Any],
|
||||||
effective_from: datetime,
|
effective_from: datetime,
|
||||||
) -> EnergyContract:
|
) -> EnergyContract:
|
||||||
@@ -129,11 +160,14 @@ def create_contract(
|
|||||||
name:
|
name:
|
||||||
Human-readable label for the contract.
|
Human-readable label for the contract.
|
||||||
kind:
|
kind:
|
||||||
Pricing strategy identifier (``"manual"`` or ``"tibber"``).
|
Pricing strategy identifier (``"manual"``, ``"tibber"``, or
|
||||||
|
``"district_heating"``).
|
||||||
currency:
|
currency:
|
||||||
ISO 4217 currency code (default ``"EUR"``).
|
ISO 4217 currency code (default ``"EUR"``).
|
||||||
values:
|
values:
|
||||||
Pricing values dict conforming to the named profile's structure.
|
Pricing values dict conforming to the named profile's structure. The
|
||||||
|
district-heating profile normalises its Decimal-safe values to strings
|
||||||
|
before the JSON snapshot is stored.
|
||||||
Validated via ``validate_values(kind, values)`` before any writes.
|
Validated via ``validate_values(kind, values)`` before any writes.
|
||||||
effective_from:
|
effective_from:
|
||||||
UTC datetime at which the first pricing version takes effect.
|
UTC datetime at which the first pricing version takes effect.
|
||||||
@@ -150,6 +184,7 @@ def create_contract(
|
|||||||
ProfileValidationError
|
ProfileValidationError
|
||||||
If *values* does not conform to the profile structure.
|
If *values* does not conform to the profile structure.
|
||||||
"""
|
"""
|
||||||
|
resolved_scope = contract_scope_for_kind(kind, scope)
|
||||||
# Validate (and fill defaults) before any DB write.
|
# Validate (and fill defaults) before any DB write.
|
||||||
filled_values = validate_values(kind, values)
|
filled_values = validate_values(kind, values)
|
||||||
|
|
||||||
@@ -157,6 +192,7 @@ def create_contract(
|
|||||||
contract = EnergyContract(
|
contract = EnergyContract(
|
||||||
name=name,
|
name=name,
|
||||||
kind=kind,
|
kind=kind,
|
||||||
|
scope=resolved_scope,
|
||||||
currency=currency,
|
currency=currency,
|
||||||
active=False, # New contracts are inactive; caller must explicitly activate.
|
active=False, # New contracts are inactive; caller must explicitly activate.
|
||||||
created_at=now,
|
created_at=now,
|
||||||
@@ -262,15 +298,18 @@ def add_version(
|
|||||||
def activate_contract(session: Session, contract: EnergyContract) -> None:
|
def activate_contract(session: Session, contract: EnergyContract) -> None:
|
||||||
"""Activate a contract with mutual exclusion.
|
"""Activate a contract with mutual exclusion.
|
||||||
|
|
||||||
Sets every other contract's ``active`` flag to False, then sets the given
|
Sets every other contract in the same scope inactive, then sets the given
|
||||||
contract's ``active`` to True. This guarantees at most one active contract
|
contract active. This guarantees at most one active contract per scope.
|
||||||
at any time.
|
|
||||||
|
|
||||||
Caller must commit after this returns.
|
Caller must commit after this returns.
|
||||||
"""
|
"""
|
||||||
# Deactivate all contracts (including the target; we re-activate below).
|
# This bulk update is a single write statement inside the caller's
|
||||||
for other in session.execute(select(EnergyContract)).scalars().all():
|
# transaction. SQLite serializes writers, and another scope is never touched.
|
||||||
other.active = False
|
session.execute(
|
||||||
|
update(EnergyContract)
|
||||||
|
.where(EnergyContract.scope == contract.scope, EnergyContract.id != contract.id)
|
||||||
|
.values(active=False)
|
||||||
|
)
|
||||||
contract.active = True
|
contract.active = True
|
||||||
contract.updated_at = datetime.now(UTC)
|
contract.updated_at = datetime.now(UTC)
|
||||||
logger.info("Activated contract %r (id=%d)", contract.name, contract.id)
|
logger.info("Activated contract %r (id=%d)", contract.name, contract.id)
|
||||||
@@ -286,7 +325,9 @@ def deactivate_contract(session: Session, contract: EnergyContract) -> None:
|
|||||||
logger.info("Deactivated contract %r (id=%d)", contract.name, contract.id)
|
logger.info("Deactivated contract %r (id=%d)", contract.name, contract.id)
|
||||||
|
|
||||||
|
|
||||||
def active_contract_versions(session: Session) -> list[EnergyContractVersion]:
|
def active_contract_versions(
|
||||||
|
session: Session, *, scope: str = "electricity"
|
||||||
|
) -> list[EnergyContractVersion]:
|
||||||
"""Return all versions of the currently active contract, ordered by effective_from ascending.
|
"""Return all versions of the currently active contract, ordered by effective_from ascending.
|
||||||
|
|
||||||
Returns an empty list when there is no active contract. The list spans the
|
Returns an empty list when there is no active contract. The list spans the
|
||||||
@@ -295,7 +336,9 @@ def active_contract_versions(session: Session) -> list[EnergyContractVersion]:
|
|||||||
cost / credit accumulation (Principle C).
|
cost / credit accumulation (Principle C).
|
||||||
"""
|
"""
|
||||||
active = session.execute(
|
active = session.execute(
|
||||||
select(EnergyContract).where(EnergyContract.active.is_(True)).limit(1)
|
select(EnergyContract)
|
||||||
|
.where(EnergyContract.active.is_(True), EnergyContract.scope == scope)
|
||||||
|
.limit(1)
|
||||||
).scalar_one_or_none()
|
).scalar_one_or_none()
|
||||||
|
|
||||||
if active is None:
|
if active is None:
|
||||||
@@ -313,7 +356,7 @@ def active_contract_versions(session: Session) -> list[EnergyContractVersion]:
|
|||||||
|
|
||||||
|
|
||||||
def active_contract_version_at(
|
def active_contract_version_at(
|
||||||
session: Session, ts: datetime
|
session: Session, ts: datetime, *, scope: str = "electricity"
|
||||||
) -> EnergyContractVersion | None:
|
) -> EnergyContractVersion | None:
|
||||||
"""Return the active contract's version that covers *ts*.
|
"""Return the active contract's version that covers *ts*.
|
||||||
|
|
||||||
@@ -336,7 +379,9 @@ def active_contract_version_at(
|
|||||||
EnergyContractVersion | None
|
EnergyContractVersion | None
|
||||||
"""
|
"""
|
||||||
active = session.execute(
|
active = session.execute(
|
||||||
select(EnergyContract).where(EnergyContract.active.is_(True)).limit(1)
|
select(EnergyContract)
|
||||||
|
.where(EnergyContract.active.is_(True), EnergyContract.scope == scope)
|
||||||
|
.limit(1)
|
||||||
).scalar_one_or_none()
|
).scalar_one_or_none()
|
||||||
|
|
||||||
if active is None:
|
if active is None:
|
||||||
|
|||||||
+397
-263
@@ -1,46 +1,21 @@
|
|||||||
"""DSMR telegram ingest service.
|
"""DSMR MQTT ingest, keyed by durable meter-source identity."""
|
||||||
|
|
||||||
Subscribes to the DSMR Reader MQTT topic (``dsmr/json``) and persists
|
|
||||||
down-sampled DSMR telegram frames to the ``dsmr_reading`` table.
|
|
||||||
|
|
||||||
Design decisions
|
|
||||||
----------------
|
|
||||||
- **Whole-frame storage**: the entire parsed telegram dict is stored as a JSON
|
|
||||||
blob in ``DsmrReading.payload``; no field allow-list is applied. This lets
|
|
||||||
future commodities (gas, heating, three-phase) be accommodated without a
|
|
||||||
table-schema change.
|
|
||||||
- **10-second down-sampling** (configurable via ``dsmr_sample_interval_s``):
|
|
||||||
only telegrams whose ``timestamp`` second falls on an exact multiple of the
|
|
||||||
interval are persisted. This reduces write volume from ~60 rows/min to ~6
|
|
||||||
rows/min while guaranteeing that every 15-minute boundary (second=00) is
|
|
||||||
captured.
|
|
||||||
- **Idempotency**: the telegram's own ``id`` field is stored as ``source_id``
|
|
||||||
with a UNIQUE constraint. A second delivery of the same telegram (e.g. after
|
|
||||||
a broker reconnect) is silently skipped.
|
|
||||||
- **Network-thread safety**: ``handle_message`` is called from paho's background
|
|
||||||
loop thread. It opens and closes its own short-lived SQLAlchemy session and
|
|
||||||
swallows all exceptions so that a buggy payload or transient DB error never
|
|
||||||
crashes the paho loop or drops the MQTT connection.
|
|
||||||
- **Numeric values kept as strings**: the DSMR Reader emits all numeric readings
|
|
||||||
as JSON strings (e.g. ``"20915.154"``). They are stored verbatim; conversion
|
|
||||||
to ``Decimal`` is deferred to the billing engine (T07) where precision matters.
|
|
||||||
- **Null phases**: some telegrams omit certain phase readings (``null`` in JSON);
|
|
||||||
these are stored as-is without special handling.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import json
|
import json
|
||||||
import logging
|
import logging
|
||||||
import threading
|
import threading
|
||||||
|
from dataclasses import dataclass
|
||||||
from datetime import datetime, timezone
|
from datetime import datetime, timezone
|
||||||
from typing import TYPE_CHECKING
|
from typing import TYPE_CHECKING
|
||||||
|
|
||||||
import sqlalchemy.exc
|
import sqlalchemy.exc
|
||||||
from sqlalchemy import select
|
from sqlalchemy import select
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
from app.db import get_session_local
|
from app.db import get_session_local
|
||||||
from app.models.energy import DsmrReading
|
from app.models.energy import DsmrReading, Meter
|
||||||
|
from app.models.meter_source import MeterSource, MeterSourceBinding, MeterSourceChannel
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
if TYPE_CHECKING:
|
||||||
from app.config import Settings
|
from app.config import Settings
|
||||||
@@ -48,250 +23,409 @@ if TYPE_CHECKING:
|
|||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
# Tracks the DSMR topic currently subscribed via the MQTT manager, so a config
|
@dataclass(frozen=True, slots=True)
|
||||||
# change can unsubscribe the old topic before subscribing the new one.
|
class DsmrSourceSnapshot:
|
||||||
_current_dsmr_topic: str | None = None
|
"""The only DSMR runtime configuration a network handler may use."""
|
||||||
|
|
||||||
# Tracks the DSMR tariff topic currently subscribed via the MQTT manager.
|
source_id: int
|
||||||
_current_tariff_topic: str | None = None
|
topic: str
|
||||||
|
tariff_topic: str
|
||||||
|
sample_interval_s: int
|
||||||
|
broker_host: str = ""
|
||||||
|
broker_port: int = 1883
|
||||||
|
username: str = ""
|
||||||
|
password: str = ""
|
||||||
|
tls_enabled: bool = False
|
||||||
|
|
||||||
# Current electricity tariff: 1 = dal/off-peak, 2 = normal/peak.
|
|
||||||
# Written by the paho network thread, read by the publish job — guarded by a lock.
|
_subscriptions: dict[int, DsmrSourceSnapshot] = {}
|
||||||
_current_tariff: int | None = None
|
_subscription_client_ids: dict[int, str] = {}
|
||||||
|
_subscription_lock = threading.RLock()
|
||||||
|
# A configuration value is not an ownership identity: disable and re-enable
|
||||||
|
# can produce an equal snapshot. Each installed handler therefore captures a
|
||||||
|
# fresh token and verifies object identity before it can write or update tariff.
|
||||||
|
_subscription_tokens: dict[int, object] = {}
|
||||||
|
_reconcile_lock = threading.RLock()
|
||||||
|
_tariffs: dict[int, int] = {}
|
||||||
_tariff_lock = threading.Lock()
|
_tariff_lock = threading.Lock()
|
||||||
|
# Kept only for legacy direct test callers of set_current_tariff(value). Runtime
|
||||||
|
# handlers never write this value; production callers resolve a binding first.
|
||||||
|
_current_tariff: int | None = None
|
||||||
|
_UNSET = object()
|
||||||
|
|
||||||
|
|
||||||
def get_current_tariff() -> int | None:
|
def get_current_tariff(meter_source_id: int | None = None) -> int | None:
|
||||||
"""Return the most recently received electricity tariff (1 or 2), or None."""
|
"""Return a source tariff, or resolve the active electricity binding.
|
||||||
with _tariff_lock:
|
|
||||||
return _current_tariff
|
|
||||||
|
|
||||||
|
The no-argument form is retained for the pre-M8 expose integration. It
|
||||||
def set_current_tariff(value: int | None) -> None:
|
opens a short session to select the current electricity binding, so a
|
||||||
"""Set the current electricity tariff (1 or 2), or clear it with None."""
|
source's MQTT callback can never make another source's tariff current.
|
||||||
with _tariff_lock:
|
``_current_tariff`` is solely a test-era fallback when no binding database
|
||||||
global _current_tariff
|
is available; runtime MQTT handlers do not update it.
|
||||||
_current_tariff = value
|
|
||||||
|
|
||||||
|
|
||||||
def handle_tariff_message(payload_bytes: bytes) -> None:
|
|
||||||
"""Parse one DSMR tariff MQTT payload and update the in-memory tariff state.
|
|
||||||
|
|
||||||
Called from the paho network thread; *must* swallow all exceptions so that
|
|
||||||
a bad payload never crashes the loop or drops the broker connection.
|
|
||||||
|
|
||||||
Accepts payload as bytes or str (paho can deliver either). Ignores
|
|
||||||
whitespace. Only accepts integer values 1 or 2; anything else is discarded
|
|
||||||
and the previous known tariff is preserved.
|
|
||||||
|
|
||||||
Parameters
|
|
||||||
----------
|
|
||||||
payload_bytes:
|
|
||||||
Raw bytes from the MQTT message (may also be a str in some paho versions).
|
|
||||||
"""
|
"""
|
||||||
try:
|
if meter_source_id is None:
|
||||||
# Decode bytes → str if needed; strip surrounding whitespace.
|
|
||||||
if isinstance(payload_bytes, (bytes, bytearray)):
|
|
||||||
raw = payload_bytes.decode("utf-8", errors="replace").strip()
|
|
||||||
else:
|
|
||||||
raw = str(payload_bytes).strip()
|
|
||||||
|
|
||||||
value = int(raw)
|
|
||||||
if value not in (1, 2):
|
|
||||||
logger.debug(
|
|
||||||
"dsmr_ingest.handle_tariff_message: unexpected tariff value %d (expected 1 or 2, ignored).",
|
|
||||||
value,
|
|
||||||
)
|
|
||||||
return
|
|
||||||
|
|
||||||
set_current_tariff(value)
|
|
||||||
logger.debug("dsmr_ingest.handle_tariff_message: tariff updated to %d.", value)
|
|
||||||
|
|
||||||
except Exception:
|
|
||||||
# Malformed payload (e.g. non-numeric); swallow silently to protect network thread.
|
|
||||||
logger.debug(
|
|
||||||
"dsmr_ingest.handle_tariff_message: could not parse payload %r (ignored).",
|
|
||||||
payload_bytes,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def apply_dsmr_subscription(settings: "Settings") -> None:
|
|
||||||
"""(Re)apply the DSMR MQTT subscriptions to match *settings* — restart-free.
|
|
||||||
|
|
||||||
Call this at startup and after every config save. It makes the live MQTT
|
|
||||||
subscriptions reflect the current ``dsmr_ingest_enabled`` / ``dsmr_mqtt_topic``
|
|
||||||
/ ``dsmr_sample_interval_s`` / ``dsmr_tariff_topic`` settings without an app
|
|
||||||
restart:
|
|
||||||
|
|
||||||
- **Disabled** → unsubscribe any active DSMR and tariff subscriptions.
|
|
||||||
- **Enabled** → (re)subscribe to ``dsmr_mqtt_topic`` with a handler bound to
|
|
||||||
a *fresh* settings snapshot, so a changed sample interval also takes effect.
|
|
||||||
Also subscribe to ``dsmr_tariff_topic`` when non-empty.
|
|
||||||
- **Topic changed** → unsubscribe the old topic before subscribing the new one.
|
|
||||||
|
|
||||||
Idempotent and safe to call when MQTT is not connected (the subscription is
|
|
||||||
queued in the manager and established on the next connect).
|
|
||||||
"""
|
|
||||||
# Imported here (not at module top) to avoid a circular import at app start.
|
|
||||||
from app.integrations.mqtt import mqtt_manager
|
|
||||||
|
|
||||||
global _current_dsmr_topic, _current_tariff_topic
|
|
||||||
|
|
||||||
if not settings.dsmr_ingest_enabled:
|
|
||||||
if _current_dsmr_topic is not None:
|
|
||||||
mqtt_manager.unsubscribe(_current_dsmr_topic)
|
|
||||||
logger.info("DSMR ingest disabled — unsubscribed from topic=%s.", _current_dsmr_topic)
|
|
||||||
_current_dsmr_topic = None
|
|
||||||
if _current_tariff_topic is not None:
|
|
||||||
mqtt_manager.unsubscribe(_current_tariff_topic)
|
|
||||||
logger.info(
|
|
||||||
"DSMR ingest disabled — unsubscribed from tariff topic=%s.",
|
|
||||||
_current_tariff_topic,
|
|
||||||
)
|
|
||||||
_current_tariff_topic = None
|
|
||||||
return
|
|
||||||
|
|
||||||
# --- Main DSMR telegram topic ---
|
|
||||||
topic = settings.dsmr_mqtt_topic
|
|
||||||
if _current_dsmr_topic is not None and _current_dsmr_topic != topic:
|
|
||||||
mqtt_manager.unsubscribe(_current_dsmr_topic)
|
|
||||||
|
|
||||||
# Re-subscribe (overwrites any existing handler for this topic) with a fresh
|
|
||||||
# settings snapshot so dsmr_sample_interval_s changes take effect too.
|
|
||||||
snapshot = settings
|
|
||||||
mqtt_manager.subscribe(topic, lambda payload: handle_message(payload, snapshot))
|
|
||||||
_current_dsmr_topic = topic
|
|
||||||
logger.info("DSMR ingest enabled — subscribed to topic=%s.", topic)
|
|
||||||
|
|
||||||
# --- DSMR tariff topic (dual-tariff slot indicator) ---
|
|
||||||
tariff_topic = settings.dsmr_tariff_topic if settings.dsmr_tariff_topic else ""
|
|
||||||
if tariff_topic:
|
|
||||||
if _current_tariff_topic is not None and _current_tariff_topic != tariff_topic:
|
|
||||||
mqtt_manager.unsubscribe(_current_tariff_topic)
|
|
||||||
mqtt_manager.subscribe(tariff_topic, lambda payload: handle_tariff_message(payload))
|
|
||||||
_current_tariff_topic = tariff_topic
|
|
||||||
logger.info("DSMR tariff topic — subscribed to topic=%s.", tariff_topic)
|
|
||||||
else:
|
|
||||||
# tariff_topic is empty → unsubscribe any existing tariff subscription.
|
|
||||||
if _current_tariff_topic is not None:
|
|
||||||
mqtt_manager.unsubscribe(_current_tariff_topic)
|
|
||||||
logger.info(
|
|
||||||
"DSMR tariff topic cleared — unsubscribed from topic=%s.",
|
|
||||||
_current_tariff_topic,
|
|
||||||
)
|
|
||||||
_current_tariff_topic = None
|
|
||||||
|
|
||||||
|
|
||||||
def handle_message(payload_bytes: bytes, settings: "Settings") -> None:
|
|
||||||
"""Parse one DSMR MQTT payload and persist it if it passes the sample filter.
|
|
||||||
|
|
||||||
Called from the paho network thread; *must* swallow all exceptions so that
|
|
||||||
a bad payload or transient error does not crash the loop or drop the broker
|
|
||||||
connection.
|
|
||||||
|
|
||||||
Parameters
|
|
||||||
----------
|
|
||||||
payload_bytes:
|
|
||||||
Raw bytes from the MQTT message.
|
|
||||||
settings:
|
|
||||||
Runtime settings snapshot (captured at subscription time). Used for
|
|
||||||
``dsmr_sample_interval_s``.
|
|
||||||
"""
|
|
||||||
try:
|
|
||||||
_handle_message_inner(payload_bytes, settings)
|
|
||||||
except Exception:
|
|
||||||
logger.exception("dsmr_ingest.handle_message: unexpected error (swallowed).")
|
|
||||||
|
|
||||||
|
|
||||||
def _handle_message_inner(payload_bytes: bytes, settings: "Settings") -> None:
|
|
||||||
"""Inner implementation — may raise; caller wraps in try/except."""
|
|
||||||
# --- 1. Parse JSON ---
|
|
||||||
try:
|
|
||||||
data: dict = json.loads(payload_bytes)
|
|
||||||
except (json.JSONDecodeError, ValueError):
|
|
||||||
logger.debug("dsmr_ingest: invalid JSON payload (skipped).")
|
|
||||||
return
|
|
||||||
|
|
||||||
if not isinstance(data, dict):
|
|
||||||
logger.debug("dsmr_ingest: payload is not a JSON object (skipped).")
|
|
||||||
return
|
|
||||||
|
|
||||||
# --- 2. Parse timestamp ---
|
|
||||||
raw_ts = data.get("timestamp")
|
|
||||||
if raw_ts is None:
|
|
||||||
logger.debug("dsmr_ingest: missing 'timestamp' field (skipped).")
|
|
||||||
return
|
|
||||||
|
|
||||||
try:
|
|
||||||
# Python 3.11+ accepts the trailing 'Z' directly; for 3.10 compat we
|
|
||||||
# replace 'Z' with '+00:00' before parsing.
|
|
||||||
ts_str = raw_ts if not isinstance(raw_ts, str) else raw_ts.replace("Z", "+00:00")
|
|
||||||
ts_utc: datetime = datetime.fromisoformat(ts_str)
|
|
||||||
# Ensure it is timezone-aware UTC.
|
|
||||||
if ts_utc.tzinfo is None:
|
|
||||||
ts_utc = ts_utc.replace(tzinfo=timezone.utc)
|
|
||||||
except (ValueError, TypeError, AttributeError):
|
|
||||||
logger.debug(
|
|
||||||
"dsmr_ingest: cannot parse 'timestamp' value %r (skipped).", raw_ts
|
|
||||||
)
|
|
||||||
return
|
|
||||||
|
|
||||||
# --- 3. Down-sample: only persist if second falls on interval boundary ---
|
|
||||||
interval = settings.dsmr_sample_interval_s
|
|
||||||
if interval > 0 and (ts_utc.second % interval) != 0:
|
|
||||||
# This telegram is between sample points; discard silently.
|
|
||||||
return
|
|
||||||
|
|
||||||
# --- 4. Extract source_id (telegram's own id) — stored only as a reference,
|
|
||||||
# NOT used for uniqueness/idempotency (it overflows and gets reset). ---
|
|
||||||
source_id: int | None = data.get("id")
|
|
||||||
if source_id is not None and not isinstance(source_id, int):
|
|
||||||
# Unexpected type — treat as missing rather than raising.
|
|
||||||
logger.debug(
|
|
||||||
"dsmr_ingest: 'id' field has unexpected type %s (ignoring).",
|
|
||||||
type(source_id).__name__,
|
|
||||||
)
|
|
||||||
source_id = None
|
|
||||||
|
|
||||||
# --- 5. Persist to database ---
|
|
||||||
# Idempotency is keyed on recorded_at (the telegram timestamp), which is
|
|
||||||
# telegram-id-independent: a single P1 meter emits one telegram per second,
|
|
||||||
# and down-sampling keeps at most one per interval-aligned second. The
|
|
||||||
# UNIQUE(recorded_at) constraint is the backstop for the IntegrityError race.
|
|
||||||
session_local = get_session_local()
|
session_local = get_session_local()
|
||||||
session = session_local()
|
session = session_local()
|
||||||
try:
|
try:
|
||||||
existing = session.scalar(
|
source_id = _current_electricity_source_id(session, datetime.now(timezone.utc))
|
||||||
select(DsmrReading).where(DsmrReading.recorded_at == ts_utc)
|
if source_id is not None:
|
||||||
)
|
return get_current_tariff(source_id)
|
||||||
if existing is not None:
|
|
||||||
logger.debug(
|
|
||||||
"dsmr_ingest: recorded_at=%s already in DB, skipping.",
|
|
||||||
ts_utc.isoformat(),
|
|
||||||
)
|
|
||||||
return
|
|
||||||
|
|
||||||
reading = DsmrReading(
|
|
||||||
recorded_at=ts_utc,
|
|
||||||
source_id=source_id,
|
|
||||||
payload=data, # full frame, verbatim
|
|
||||||
)
|
|
||||||
session.add(reading)
|
|
||||||
session.commit()
|
|
||||||
logger.debug(
|
|
||||||
"dsmr_ingest: persisted reading recorded_at=%s source_id=%s.",
|
|
||||||
ts_utc.isoformat(),
|
|
||||||
source_id,
|
|
||||||
)
|
|
||||||
except sqlalchemy.exc.IntegrityError:
|
|
||||||
# Race / duplicate: another insert beat us to the same recorded_at.
|
|
||||||
session.rollback()
|
|
||||||
logger.debug(
|
|
||||||
"dsmr_ingest: IntegrityError for recorded_at=%s (duplicate, skipped).",
|
|
||||||
ts_utc.isoformat(),
|
|
||||||
)
|
|
||||||
except Exception:
|
except Exception:
|
||||||
|
logger.debug("DSMR legacy tariff lookup could not resolve a binding", exc_info=True)
|
||||||
|
finally:
|
||||||
|
session.close()
|
||||||
|
return _current_tariff
|
||||||
|
with _tariff_lock:
|
||||||
|
return _tariffs.get(meter_source_id)
|
||||||
|
|
||||||
|
|
||||||
|
def set_current_tariff(meter_source_id: int, value: int | None | object = _UNSET) -> None:
|
||||||
|
"""Set or clear an individual source's tariff state."""
|
||||||
|
global _current_tariff
|
||||||
|
if value is _UNSET:
|
||||||
|
# Compatibility with older direct callers. Do not route runtime source
|
||||||
|
# updates through this global fallback.
|
||||||
|
_current_tariff = meter_source_id if meter_source_id in (1, 2) else None
|
||||||
|
return
|
||||||
|
with _tariff_lock:
|
||||||
|
if value is None:
|
||||||
|
_tariffs.pop(meter_source_id, None)
|
||||||
|
else:
|
||||||
|
_tariffs[meter_source_id] = value
|
||||||
|
|
||||||
|
|
||||||
|
def _current_electricity_source_id(session: Session, at: datetime) -> int | None:
|
||||||
|
"""Return the DSMR source bound to electricity at ``at``, if any."""
|
||||||
|
return session.scalar(
|
||||||
|
select(MeterSource.id)
|
||||||
|
.join(MeterSourceChannel, MeterSourceChannel.source_id == MeterSource.id)
|
||||||
|
.join(MeterSourceBinding, MeterSourceBinding.channel_id == MeterSourceChannel.id)
|
||||||
|
.join(Meter, Meter.id == MeterSourceBinding.meter_id)
|
||||||
|
.where(
|
||||||
|
Meter.commodity == "electricity",
|
||||||
|
MeterSourceBinding.started_at <= at,
|
||||||
|
(MeterSourceBinding.ended_at.is_(None)) | (MeterSourceBinding.ended_at > at),
|
||||||
|
)
|
||||||
|
.order_by(MeterSourceBinding.started_at.desc())
|
||||||
|
.limit(1)
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def get_current_electricity_tariff(session: Session, at: datetime | None = None) -> int | None:
|
||||||
|
"""Resolve tariff through the current electricity binding, never globally."""
|
||||||
|
source_id = _current_electricity_source_id(session, at or datetime.now(timezone.utc))
|
||||||
|
if source_id is None:
|
||||||
|
return None
|
||||||
|
return get_current_tariff(source_id)
|
||||||
|
|
||||||
|
|
||||||
|
def handle_tariff_message(payload_bytes: bytes, meter_source_id: int) -> None:
|
||||||
|
"""Parse one source's tariff payload without raising in paho's thread."""
|
||||||
|
try:
|
||||||
|
raw = (
|
||||||
|
payload_bytes.decode("utf-8", errors="replace").strip()
|
||||||
|
if isinstance(payload_bytes, (bytes, bytearray))
|
||||||
|
else str(payload_bytes).strip()
|
||||||
|
)
|
||||||
|
value = int(raw)
|
||||||
|
if value in (1, 2):
|
||||||
|
set_current_tariff(meter_source_id, value)
|
||||||
|
except Exception:
|
||||||
|
logger.debug("DSMR tariff payload ignored for source_id=%s", meter_source_id)
|
||||||
|
|
||||||
|
|
||||||
|
def _snapshot(source: MeterSource) -> DsmrSourceSnapshot:
|
||||||
|
config = source.config
|
||||||
|
return DsmrSourceSnapshot(
|
||||||
|
source_id=source.id,
|
||||||
|
topic=str(config.get("topic", "dsmr/json")),
|
||||||
|
tariff_topic=str(config.get("tariff_topic", "")),
|
||||||
|
sample_interval_s=int(config.get("sample_interval_s", 10)),
|
||||||
|
broker_host=str(config.get("broker_host", "")),
|
||||||
|
broker_port=int(config.get("broker_port", 1883)),
|
||||||
|
username=str(config.get("username", "")),
|
||||||
|
password=str(config.get("password", "")),
|
||||||
|
tls_enabled=bool(config.get("tls_enabled", False)),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _enabled_snapshots() -> list[DsmrSourceSnapshot]:
|
||||||
|
session_local = get_session_local()
|
||||||
|
session = session_local()
|
||||||
|
try:
|
||||||
|
sources = session.scalars(
|
||||||
|
select(MeterSource).where(MeterSource.kind == "dsmr_mqtt", MeterSource.enabled.is_(True))
|
||||||
|
).all()
|
||||||
|
return [_snapshot(source) for source in sources]
|
||||||
|
finally:
|
||||||
|
session.close()
|
||||||
|
|
||||||
|
|
||||||
|
def apply_dsmr_subscription(settings: "Settings | None" = None) -> None:
|
||||||
|
"""Reconcile enabled DSMR source subscriptions from the database.
|
||||||
|
|
||||||
|
``settings`` provides the DB-merged app-wide MQTT identity. Individual
|
||||||
|
DSMR broker settings continue to come solely from MeterSource records.
|
||||||
|
"""
|
||||||
|
from app.integrations.mqtt import mqtt_manager
|
||||||
|
from app.config import get_settings
|
||||||
|
|
||||||
|
base_client_id = (settings or get_settings()).mqtt_client_id
|
||||||
|
|
||||||
|
try:
|
||||||
|
desired = {snapshot.source_id: snapshot for snapshot in _enabled_snapshots()}
|
||||||
|
except Exception:
|
||||||
|
logger.exception("DSMR subscription reconcile failed while reading sources")
|
||||||
|
return
|
||||||
|
|
||||||
|
# Do not retain _subscription_lock while stopping MQTT clients: a callback
|
||||||
|
# may currently hold it through its complete dispatch, and MqttManager
|
||||||
|
# waits for that callback before teardown returns.
|
||||||
|
with _reconcile_lock:
|
||||||
|
# Every source owns a distinct MQTT client, so equal topics from
|
||||||
|
# different sources/brokers are dispatchable. A telegram and tariff
|
||||||
|
# topic on the *same* client would overwrite one handler, however.
|
||||||
|
rejected_source_ids: set[int] = set()
|
||||||
|
for source_id, snapshot in list(desired.items()):
|
||||||
|
if snapshot.tariff_topic and snapshot.topic == snapshot.tariff_topic:
|
||||||
|
logger.error("DSMR source_id=%s rejected: telegram/tariff topic collision", source_id)
|
||||||
|
rejected_source_ids.add(source_id)
|
||||||
|
desired.pop(source_id)
|
||||||
|
|
||||||
|
with _subscription_lock:
|
||||||
|
stale_source_ids = [
|
||||||
|
source_id
|
||||||
|
for source_id, current in _subscriptions.items()
|
||||||
|
if desired.get(source_id) != current
|
||||||
|
or _subscription_client_ids.get(source_id) != base_client_id
|
||||||
|
]
|
||||||
|
for source_id in stale_source_ids:
|
||||||
|
_subscriptions.pop(source_id, None)
|
||||||
|
_subscription_client_ids.pop(source_id, None)
|
||||||
|
_subscription_tokens.pop(source_id, None)
|
||||||
|
set_current_tariff(source_id, None)
|
||||||
|
|
||||||
|
# A disabled source has no installed MQTT owner. Persist that fact
|
||||||
|
# after invalidating its callback token, so a retained callback cannot
|
||||||
|
# revive an earlier online state while teardown is in progress.
|
||||||
|
for source_id in stale_source_ids:
|
||||||
|
_mark_disabled_source_inactive(source_id)
|
||||||
|
|
||||||
|
# A topic collision is a configuration error for an enabled source,
|
||||||
|
# not a disabled-state transition. It must therefore replace any
|
||||||
|
# earlier online state even when this process started without a
|
||||||
|
# matching runtime subscription to tear down.
|
||||||
|
for source_id in rejected_source_ids:
|
||||||
|
_mark_rejected_source_error(source_id)
|
||||||
|
|
||||||
|
for source_id in stale_source_ids:
|
||||||
|
mqtt_manager.remove_source(source_id)
|
||||||
|
|
||||||
|
for source_id, snapshot in desired.items():
|
||||||
|
with _subscription_lock:
|
||||||
|
current = _subscriptions.get(source_id)
|
||||||
|
current_client_id = _subscription_client_ids.get(source_id)
|
||||||
|
if (
|
||||||
|
current == snapshot
|
||||||
|
and current_client_id == base_client_id
|
||||||
|
and mqtt_manager.source_is_active(source_id)
|
||||||
|
):
|
||||||
|
continue
|
||||||
|
if current is not None:
|
||||||
|
# The client went inactive outside reconcile. Invalidate its
|
||||||
|
# old token before rebuilding the same snapshot.
|
||||||
|
with _subscription_lock:
|
||||||
|
if _subscriptions.get(source_id) == current:
|
||||||
|
_subscriptions.pop(source_id, None)
|
||||||
|
_subscription_client_ids.pop(source_id, None)
|
||||||
|
_subscription_tokens.pop(source_id, None)
|
||||||
|
mqtt_manager.remove_source(source_id)
|
||||||
|
token = object()
|
||||||
|
handlers = {
|
||||||
|
snapshot.topic: lambda payload, captured=snapshot, captured_token=token: (
|
||||||
|
handle_captured_message(payload, captured, captured_token)
|
||||||
|
)
|
||||||
|
}
|
||||||
|
if snapshot.tariff_topic:
|
||||||
|
handlers[snapshot.tariff_topic] = (
|
||||||
|
lambda payload, captured=snapshot, captured_token=token: (
|
||||||
|
handle_captured_tariff_message(payload, captured, captured_token)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
with _subscription_lock:
|
||||||
|
_subscriptions[source_id] = snapshot
|
||||||
|
_subscription_client_ids[source_id] = base_client_id
|
||||||
|
_subscription_tokens[source_id] = token
|
||||||
|
applied = mqtt_manager.replace_source(
|
||||||
|
source_id,
|
||||||
|
host=snapshot.broker_host,
|
||||||
|
port=snapshot.broker_port,
|
||||||
|
username=snapshot.username,
|
||||||
|
password=snapshot.password,
|
||||||
|
tls_enabled=snapshot.tls_enabled,
|
||||||
|
subscriptions=handlers,
|
||||||
|
base_client_id=base_client_id,
|
||||||
|
state_handler=lambda state, captured=snapshot, captured_token=token: (
|
||||||
|
handle_captured_source_state(captured, captured_token, state)
|
||||||
|
),
|
||||||
|
)
|
||||||
|
if not applied:
|
||||||
|
with _subscription_lock:
|
||||||
|
if _subscription_tokens.get(source_id) is token:
|
||||||
|
_subscriptions.pop(source_id, None)
|
||||||
|
_subscription_client_ids.pop(source_id, None)
|
||||||
|
_subscription_tokens.pop(source_id, None)
|
||||||
|
|
||||||
|
|
||||||
|
def handle_message(payload_bytes: bytes, snapshot: DsmrSourceSnapshot) -> None:
|
||||||
|
"""Persist one down-sampled frame under its captured source identity."""
|
||||||
|
try:
|
||||||
|
_handle_message_inner(payload_bytes, snapshot)
|
||||||
|
except Exception:
|
||||||
|
logger.exception("DSMR ingest handler failed for source_id=%s (swallowed)", snapshot.source_id)
|
||||||
|
|
||||||
|
|
||||||
|
def handle_captured_message(
|
||||||
|
payload_bytes: bytes, snapshot: DsmrSourceSnapshot, token: object | None = None
|
||||||
|
) -> None:
|
||||||
|
"""Run a broker callback only while its exact source generation is active."""
|
||||||
|
with _subscription_lock:
|
||||||
|
if token is not None:
|
||||||
|
if _subscription_tokens.get(snapshot.source_id) is not token:
|
||||||
|
return
|
||||||
|
elif _subscriptions.get(snapshot.source_id) != snapshot:
|
||||||
|
return
|
||||||
|
handle_message(payload_bytes, snapshot)
|
||||||
|
|
||||||
|
|
||||||
|
def handle_captured_tariff_message(
|
||||||
|
payload_bytes: bytes, snapshot: DsmrSourceSnapshot, token: object | None = None
|
||||||
|
) -> None:
|
||||||
|
"""Ignore tariff callbacks retained from a removed/replaced source."""
|
||||||
|
with _subscription_lock:
|
||||||
|
if token is not None:
|
||||||
|
if _subscription_tokens.get(snapshot.source_id) is not token:
|
||||||
|
return
|
||||||
|
elif _subscriptions.get(snapshot.source_id) != snapshot:
|
||||||
|
return
|
||||||
|
handle_tariff_message(payload_bytes, snapshot.source_id)
|
||||||
|
|
||||||
|
|
||||||
|
def handle_captured_source_state(snapshot: DsmrSourceSnapshot, token: object, state: str) -> None:
|
||||||
|
"""Persist one active generation's connection health in a short DB session."""
|
||||||
|
with _subscription_lock:
|
||||||
|
if _subscription_tokens.get(snapshot.source_id) is not token:
|
||||||
|
return
|
||||||
|
session_local = get_session_local()
|
||||||
|
session = session_local()
|
||||||
|
try:
|
||||||
|
source = session.get(MeterSource, snapshot.source_id)
|
||||||
|
if source is None or not source.enabled or source.kind != "dsmr_mqtt":
|
||||||
|
return
|
||||||
|
source.status = state
|
||||||
|
source.last_error = "MQTT connection failed." if state == "error" else None
|
||||||
|
source.updated_at = datetime.now(timezone.utc)
|
||||||
|
session.commit()
|
||||||
|
except Exception:
|
||||||
session.rollback()
|
session.rollback()
|
||||||
logger.exception("dsmr_ingest: DB error (swallowed).")
|
logger.exception("DSMR source health update failed for source_id=%s", snapshot.source_id)
|
||||||
|
finally:
|
||||||
|
session.close()
|
||||||
|
|
||||||
|
|
||||||
|
def _mark_disabled_source_inactive(source_id: int) -> None:
|
||||||
|
"""Clear an obsolete online health state for a disabled DSMR source.
|
||||||
|
|
||||||
|
The caller has already invalidated the source's generation token. This
|
||||||
|
helper deliberately opens its own short session so reconcile never shares
|
||||||
|
a callback-thread transaction. Deleted sources simply have no row left
|
||||||
|
to update.
|
||||||
|
"""
|
||||||
|
session_local = get_session_local()
|
||||||
|
session = session_local()
|
||||||
|
try:
|
||||||
|
source = session.get(MeterSource, source_id)
|
||||||
|
if source is None or source.enabled or source.kind != "dsmr_mqtt":
|
||||||
|
return
|
||||||
|
source.status = "unknown"
|
||||||
|
source.last_error = None
|
||||||
|
source.updated_at = datetime.now(timezone.utc)
|
||||||
|
session.commit()
|
||||||
|
except Exception:
|
||||||
|
session.rollback()
|
||||||
|
logger.exception("DSMR disabled source health update failed for source_id=%s", source_id)
|
||||||
|
finally:
|
||||||
|
session.close()
|
||||||
|
|
||||||
|
|
||||||
|
def _mark_rejected_source_error(source_id: int) -> None:
|
||||||
|
"""Persist a non-sensitive error for an enabled source rejected by reconcile."""
|
||||||
|
session_local = get_session_local()
|
||||||
|
session = session_local()
|
||||||
|
try:
|
||||||
|
source = session.get(MeterSource, source_id)
|
||||||
|
if source is None or not source.enabled or source.kind != "dsmr_mqtt":
|
||||||
|
return
|
||||||
|
source.status = "error"
|
||||||
|
source.last_error = "DSMR source configuration invalid."
|
||||||
|
source.updated_at = datetime.now(timezone.utc)
|
||||||
|
session.commit()
|
||||||
|
except Exception:
|
||||||
|
session.rollback()
|
||||||
|
logger.exception("DSMR rejected source health update failed for source_id=%s", source_id)
|
||||||
|
finally:
|
||||||
|
session.close()
|
||||||
|
|
||||||
|
|
||||||
|
def _handle_message_inner(payload_bytes: bytes, snapshot: DsmrSourceSnapshot) -> None:
|
||||||
|
try:
|
||||||
|
data = json.loads(payload_bytes)
|
||||||
|
except (json.JSONDecodeError, ValueError):
|
||||||
|
return
|
||||||
|
if not isinstance(data, dict):
|
||||||
|
return
|
||||||
|
try:
|
||||||
|
raw_ts = data["timestamp"]
|
||||||
|
ts_utc = datetime.fromisoformat(raw_ts.replace("Z", "+00:00"))
|
||||||
|
if ts_utc.tzinfo is None:
|
||||||
|
ts_utc = ts_utc.replace(tzinfo=timezone.utc)
|
||||||
|
except (KeyError, ValueError, TypeError, AttributeError):
|
||||||
|
return
|
||||||
|
if snapshot.sample_interval_s > 0 and ts_utc.second % snapshot.sample_interval_s:
|
||||||
|
return
|
||||||
|
telegram_id = data.get("id")
|
||||||
|
if telegram_id is not None and not isinstance(telegram_id, int):
|
||||||
|
telegram_id = None
|
||||||
|
|
||||||
|
session_local = get_session_local()
|
||||||
|
session = session_local()
|
||||||
|
try:
|
||||||
|
exists = session.scalar(
|
||||||
|
select(DsmrReading.id).where(
|
||||||
|
DsmrReading.meter_source_id == snapshot.source_id,
|
||||||
|
DsmrReading.recorded_at == ts_utc,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
if exists is None:
|
||||||
|
session.add(
|
||||||
|
DsmrReading(
|
||||||
|
meter_source_id=snapshot.source_id,
|
||||||
|
recorded_at=ts_utc,
|
||||||
|
telegram_id=telegram_id,
|
||||||
|
payload=data,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
source = session.get(MeterSource, snapshot.source_id)
|
||||||
|
if source is not None and source.enabled and source.kind == "dsmr_mqtt":
|
||||||
|
source.status = "online"
|
||||||
|
source.last_seen_at = datetime.now(timezone.utc)
|
||||||
|
source.last_error = None
|
||||||
|
source.updated_at = datetime.now(timezone.utc)
|
||||||
|
session.commit()
|
||||||
|
except sqlalchemy.exc.IntegrityError:
|
||||||
|
session.rollback()
|
||||||
|
except Exception:
|
||||||
|
session.rollback()
|
||||||
|
logger.exception("DSMR database write failed for source_id=%s", snapshot.source_id)
|
||||||
finally:
|
finally:
|
||||||
session.close()
|
session.close()
|
||||||
|
|||||||
+124
-26
@@ -18,6 +18,12 @@ M6 design document, extended in M7-T03 to be meter-aware:
|
|||||||
÷ 30 per day) and subtracts the energy-tax credit (heffingskorting,
|
÷ 30 per day) and subtracts the energy-tax credit (heffingskorting,
|
||||||
apportioned at EUR/year ÷ 365 per day).
|
apportioned at EUR/year ÷ 365 per day).
|
||||||
|
|
||||||
|
The summary reports **both** money and energy: ``metered_import`` /
|
||||||
|
``metered_export`` are monetary totals (Σ import_cost / Σ export_revenue),
|
||||||
|
while ``metered_import_kwh`` / ``metered_export_kwh`` are the corresponding
|
||||||
|
metered energy totals in kWh. The ``_kwh`` suffix is the only thing that
|
||||||
|
distinguishes them — always check it before labelling a value in a UI.
|
||||||
|
|
||||||
Design notes
|
Design notes
|
||||||
------------
|
------------
|
||||||
- **Decimal arithmetic throughout**: all monetary computations use
|
- **Decimal arithmetic throughout**: all monetary computations use
|
||||||
@@ -31,7 +37,7 @@ Design notes
|
|||||||
- **Register keys**: DSMR payload uses JSON strings like ``"20915.154"``
|
- **Register keys**: DSMR payload uses JSON strings like ``"20915.154"``
|
||||||
for cumulative kWh registers. ``register_at`` converts them to Decimal.
|
for cumulative kWh registers. ``register_at`` converts them to Decimal.
|
||||||
- **Degraded vs skip semantics**:
|
- **Degraded vs skip semantics**:
|
||||||
- *No meter coverage* (``meter_at`` returns None for t0): write a
|
- *No unique meter coverage* (no sole electricity meter at t0): write a
|
||||||
``degraded=True`` row with ``meter_id=None``.
|
``degraded=True`` row with ``meter_id=None``.
|
||||||
- *Cross-meter boundary* (m0.id != m1.id for t0/t1): write a ``degraded=True``
|
- *Cross-meter boundary* (m0.id != m1.id for t0/t1): write a ``degraded=True``
|
||||||
row with ``meter_id=m0.id``; losing this one period at the swap boundary is
|
row with ``meter_id=m0.id``; losing this one period at the swap boundary is
|
||||||
@@ -61,8 +67,8 @@ Meter-aware compute_period ordering rationale (M7-T03)
|
|||||||
The order of checks inside ``compute_period`` is:
|
The order of checks inside ``compute_period`` is:
|
||||||
|
|
||||||
1. **Immutability guard** (existing non-degraded row, overwrite=False) → return False.
|
1. **Immutability guard** (existing non-degraded row, overwrite=False) → return False.
|
||||||
2. **Meter determination** (m0 = meter_at(t0), m1 = meter_at(t1)):
|
2. **Meter determination** (m0/m1 each resolve to one electricity Meter):
|
||||||
- No meter (m0 is None) → write degraded, meter_id=None.
|
- No unique meter (m0 is None) → write degraded, meter_id=None.
|
||||||
- Cross-meter boundary (m0.id != m1.id) → write degraded, meter_id=m0.id.
|
- Cross-meter boundary (m0.id != m1.id) → write degraded, meter_id=m0.id.
|
||||||
3. **Active contract version check** → skip (no write) if absent.
|
3. **Active contract version check** → skip (no write) if absent.
|
||||||
4. **Boundary register readings** within m0's window → write degraded if missing.
|
4. **Boundary register readings** within m0's window → write degraded if missing.
|
||||||
@@ -92,8 +98,8 @@ from app.integrations.pricing.strategies import (
|
|||||||
get_strategy,
|
get_strategy,
|
||||||
)
|
)
|
||||||
from app.models.energy import DsmrReading, EnergyCostPeriod, Meter
|
from app.models.energy import DsmrReading, EnergyCostPeriod, Meter
|
||||||
|
from app.models.meter_source import MeterSource, MeterSourceBinding, MeterSourceChannel
|
||||||
from app.services.contracts import active_contract_version_at, active_contract_versions
|
from app.services.contracts import active_contract_version_at, active_contract_versions
|
||||||
from app.services.meters import meter_at
|
|
||||||
from app.services.timezone import local_date, local_now
|
from app.services.timezone import local_date, local_now
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
@@ -168,6 +174,23 @@ def _existing_period(session: Session, t0: datetime) -> EnergyCostPeriod | None:
|
|||||||
).scalar_one_or_none()
|
).scalar_one_or_none()
|
||||||
|
|
||||||
|
|
||||||
|
def _unique_electricity_meter_at(session: Session, boundary: datetime) -> Meter | None:
|
||||||
|
"""Return the sole electricity meter covering *boundary*, if one exists.
|
||||||
|
|
||||||
|
Billing must treat overlapping meter epochs as a structural ambiguity rather
|
||||||
|
than relying on ``meter_at``'s newest-started tie breaker. A cumulative
|
||||||
|
delta is safe only when exactly one electricity meter covers each endpoint.
|
||||||
|
"""
|
||||||
|
candidates = session.execute(
|
||||||
|
select(Meter).where(
|
||||||
|
Meter.commodity == "electricity",
|
||||||
|
Meter.started_at <= boundary,
|
||||||
|
(Meter.ended_at.is_(None)) | (Meter.ended_at > boundary),
|
||||||
|
)
|
||||||
|
).scalars().all()
|
||||||
|
return candidates[0] if len(candidates) == 1 else None
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# register_at — boundary reading lookup (meter-aware)
|
# register_at — boundary reading lookup (meter-aware)
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
@@ -177,6 +200,8 @@ def register_at(
|
|||||||
session: Session,
|
session: Session,
|
||||||
boundary: datetime,
|
boundary: datetime,
|
||||||
meter: Meter,
|
meter: Meter,
|
||||||
|
*,
|
||||||
|
meter_source_id: int | None = None,
|
||||||
) -> dict[str, Decimal] | None:
|
) -> dict[str, Decimal] | None:
|
||||||
"""Return the four cumulative kWh register values at *boundary*, within *meter*'s window.
|
"""Return the four cumulative kWh register values at *boundary*, within *meter*'s window.
|
||||||
|
|
||||||
@@ -234,6 +259,8 @@ def register_at(
|
|||||||
# Apply the upper bound only when the meter is closed (ended_at is not None).
|
# Apply the upper bound only when the meter is closed (ended_at is not None).
|
||||||
if meter_upper is not None:
|
if meter_upper is not None:
|
||||||
stmt = stmt.where(DsmrReading.recorded_at < meter_upper)
|
stmt = stmt.where(DsmrReading.recorded_at < meter_upper)
|
||||||
|
if meter_source_id is not None:
|
||||||
|
stmt = stmt.where(DsmrReading.meter_source_id == meter_source_id)
|
||||||
|
|
||||||
row: DsmrReading | None = session.execute(stmt).scalar_one_or_none()
|
row: DsmrReading | None = session.execute(stmt).scalar_one_or_none()
|
||||||
|
|
||||||
@@ -267,6 +294,33 @@ def register_at(
|
|||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _binding_at(
|
||||||
|
session: Session, boundary: datetime, meter: Meter
|
||||||
|
) -> tuple[MeterSourceBinding, int] | None:
|
||||||
|
"""Resolve the sole DSMR binding for *meter* at one period boundary.
|
||||||
|
|
||||||
|
Costing must not infer a cumulative domain from whichever reading happens
|
||||||
|
to be latest. A binding anchors both the physical meter epoch and its
|
||||||
|
source stream. Any missing or overlapping binding is therefore
|
||||||
|
deliberately unresolvable.
|
||||||
|
"""
|
||||||
|
candidates = session.execute(
|
||||||
|
select(MeterSourceBinding, MeterSourceChannel.source_id)
|
||||||
|
.join(MeterSourceChannel, MeterSourceChannel.id == MeterSourceBinding.channel_id)
|
||||||
|
.join(MeterSource, MeterSource.id == MeterSourceChannel.source_id)
|
||||||
|
.where(
|
||||||
|
MeterSourceBinding.meter_id == meter.id,
|
||||||
|
MeterSourceBinding.started_at <= boundary,
|
||||||
|
(MeterSourceBinding.ended_at.is_(None)) | (MeterSourceBinding.ended_at > boundary),
|
||||||
|
MeterSource.kind == "dsmr_mqtt",
|
||||||
|
)
|
||||||
|
).all()
|
||||||
|
if len(candidates) != 1:
|
||||||
|
return None
|
||||||
|
binding, source_id = candidates[0]
|
||||||
|
return binding, source_id
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# compute_period — single 15-minute period
|
# compute_period — single 15-minute period
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
@@ -296,9 +350,9 @@ def compute_period(session: Session, t0: datetime, *, overwrite: bool = False) -
|
|||||||
Side-effects
|
Side-effects
|
||||||
------------
|
------------
|
||||||
- Inserts or updates an ``EnergyCostPeriod`` row keyed on ``period_start=t0``.
|
- Inserts or updates an ``EnergyCostPeriod`` row keyed on ``period_start=t0``.
|
||||||
- If no meter covers t0 (``meter_at`` returns None for t0): inserts/updates
|
- If no unique meter covers t0: inserts/updates a degraded row with
|
||||||
a degraded row with ``meter_id=None``.
|
``meter_id=None``.
|
||||||
- If the period spans a meter boundary (``meter_at(t0).id != meter_at(t1).id``):
|
- If the period spans a meter boundary (m0.id != m1.id):
|
||||||
inserts/updates a degraded row with ``meter_id=m0.id`` (D5 decision).
|
inserts/updates a degraded row with ``meter_id=m0.id`` (D5 decision).
|
||||||
- If readings are missing at either boundary within the meter window:
|
- If readings are missing at either boundary within the meter window:
|
||||||
inserts/updates a degraded row with ``meter_id=m0.id``.
|
inserts/updates a degraded row with ``meter_id=m0.id``.
|
||||||
@@ -326,7 +380,7 @@ def compute_period(session: Session, t0: datetime, *, overwrite: bool = False) -
|
|||||||
# is corrected and a recompute_range is triggered.
|
# is corrected and a recompute_range is triggered.
|
||||||
#
|
#
|
||||||
# Ordering rationale:
|
# Ordering rationale:
|
||||||
# 1. No meter (m0 is None) → degraded(meter_id=None): no epoch for t0.
|
# 1. No unique meter (m0 is None) → degraded(meter_id=None): no unambiguous epoch for t0.
|
||||||
# 2. Cross-meter boundary (m0.id != m1.id) → degraded(meter_id=m0.id): D5.
|
# 2. Cross-meter boundary (m0.id != m1.id) → degraded(meter_id=m0.id): D5.
|
||||||
# 3. (Single meter, proceed) → contract check → readings → delta guard → price.
|
# 3. (Single meter, proceed) → contract check → readings → delta guard → price.
|
||||||
#
|
#
|
||||||
@@ -335,13 +389,13 @@ def compute_period(session: Session, t0: datetime, *, overwrite: bool = False) -
|
|||||||
# first, a missing-contract skip would silently discard the cross-table
|
# first, a missing-contract skip would silently discard the cross-table
|
||||||
# evidence; once a contract is added and recompute runs, the engine would
|
# evidence; once a contract is added and recompute runs, the engine would
|
||||||
# incorrectly use cross-table reads.
|
# incorrectly use cross-table reads.
|
||||||
m0 = meter_at(session, t0)
|
m0 = _unique_electricity_meter_at(session, t0)
|
||||||
m1 = meter_at(session, t1)
|
m1 = _unique_electricity_meter_at(session, t1)
|
||||||
|
|
||||||
if m0 is None:
|
if m0 is None:
|
||||||
# No meter epoch covers t0 — degraded with no meter attribution.
|
# No unambiguous meter epoch covers t0 — degraded with no attribution.
|
||||||
logger.debug(
|
logger.debug(
|
||||||
"compute_period(%s): no active meter at t0 — writing degraded (meter_id=None).",
|
"compute_period(%s): no unique active meter at t0 — writing degraded (meter_id=None).",
|
||||||
t0.isoformat(),
|
t0.isoformat(),
|
||||||
)
|
)
|
||||||
_upsert_degraded(session, t0, now, existing, meter_id=None)
|
_upsert_degraded(session, t0, now, existing, meter_id=None)
|
||||||
@@ -360,6 +414,16 @@ def compute_period(session: Session, t0: datetime, *, overwrite: bool = False) -
|
|||||||
_upsert_degraded(session, t0, now, existing, meter_id=m0.id)
|
_upsert_degraded(session, t0, now, existing, meter_id=m0.id)
|
||||||
return True
|
return True
|
||||||
|
|
||||||
|
# Both endpoints must resolve to the same binding and source before a
|
||||||
|
# cumulative subtraction is permitted. This is checked before contract
|
||||||
|
# lookup so structural inconsistencies remain visible as degraded rows.
|
||||||
|
bound0 = _binding_at(session, t0, m0)
|
||||||
|
bound1 = _binding_at(session, t1, m1)
|
||||||
|
if bound0 is None or bound1 is None or bound0[0].id != bound1[0].id or bound0[1] != bound1[1]:
|
||||||
|
_upsert_degraded(session, t0, now, existing, meter_id=m0.id)
|
||||||
|
return True
|
||||||
|
binding, meter_source_id = bound0
|
||||||
|
|
||||||
# --- Active contract version at t0 ---
|
# --- Active contract version at t0 ---
|
||||||
# If there is no active contract covering t0, skip the period entirely.
|
# If there is no active contract covering t0, skip the period entirely.
|
||||||
# We do not write a degraded row — there is no meaningful state to recover
|
# We do not write a degraded row — there is no meaningful state to recover
|
||||||
@@ -372,8 +436,8 @@ def compute_period(session: Session, t0: datetime, *, overwrite: bool = False) -
|
|||||||
return False
|
return False
|
||||||
|
|
||||||
# --- Boundary readings within m0's meter window ---
|
# --- Boundary readings within m0's meter window ---
|
||||||
start_regs = register_at(session, t0, m0)
|
start_regs = register_at(session, t0, m0, meter_source_id=meter_source_id)
|
||||||
end_regs = register_at(session, t1, m0)
|
end_regs = register_at(session, t1, m0, meter_source_id=meter_source_id)
|
||||||
|
|
||||||
if start_regs is None or end_regs is None:
|
if start_regs is None or end_regs is None:
|
||||||
# Missing readings within the meter window → degraded with m0 attribution.
|
# Missing readings within the meter window → degraded with m0 attribution.
|
||||||
@@ -415,9 +479,7 @@ def compute_period(session: Session, t0: datetime, *, overwrite: bool = False) -
|
|||||||
except TibberPriceNotFoundError:
|
except TibberPriceNotFoundError:
|
||||||
# Missing Tibber price → skip the period; it will be retried once the
|
# Missing Tibber price → skip the period; it will be retried once the
|
||||||
# price arrives (e.g. after the next Tibber refresh job runs).
|
# price arrives (e.g. after the next Tibber refresh job runs).
|
||||||
logger.debug(
|
logger.debug("compute_period(%s): no Tibber price found — skipping.", t0.isoformat())
|
||||||
"compute_period(%s): no Tibber price found — skipping.", t0.isoformat()
|
|
||||||
)
|
|
||||||
return False
|
return False
|
||||||
|
|
||||||
# --- Upsert the billing record ---
|
# --- Upsert the billing record ---
|
||||||
@@ -439,6 +501,7 @@ def compute_period(session: Session, t0: datetime, *, overwrite: bool = False) -
|
|||||||
existing.pricing = pricing
|
existing.pricing = pricing
|
||||||
existing.contract_version_id = version.id
|
existing.contract_version_id = version.id
|
||||||
existing.meter_id = m0.id
|
existing.meter_id = m0.id
|
||||||
|
existing.source_binding_id = binding.id
|
||||||
existing.degraded = False
|
existing.degraded = False
|
||||||
existing.computed_at = now
|
existing.computed_at = now
|
||||||
else:
|
else:
|
||||||
@@ -455,6 +518,7 @@ def compute_period(session: Session, t0: datetime, *, overwrite: bool = False) -
|
|||||||
pricing=pricing,
|
pricing=pricing,
|
||||||
contract_version_id=version.id,
|
contract_version_id=version.id,
|
||||||
meter_id=m0.id,
|
meter_id=m0.id,
|
||||||
|
source_binding_id=binding.id,
|
||||||
degraded=False,
|
degraded=False,
|
||||||
computed_at=now,
|
computed_at=now,
|
||||||
)
|
)
|
||||||
@@ -514,6 +578,7 @@ def _upsert_degraded(
|
|||||||
existing.pricing = {}
|
existing.pricing = {}
|
||||||
existing.contract_version_id = None
|
existing.contract_version_id = None
|
||||||
existing.meter_id = meter_id
|
existing.meter_id = meter_id
|
||||||
|
existing.source_binding_id = None
|
||||||
existing.degraded = True
|
existing.degraded = True
|
||||||
existing.computed_at = now
|
existing.computed_at = now
|
||||||
else:
|
else:
|
||||||
@@ -530,6 +595,7 @@ def _upsert_degraded(
|
|||||||
pricing={},
|
pricing={},
|
||||||
contract_version_id=None,
|
contract_version_id=None,
|
||||||
meter_id=meter_id,
|
meter_id=meter_id,
|
||||||
|
source_binding_id=None,
|
||||||
degraded=True,
|
degraded=True,
|
||||||
computed_at=now,
|
computed_at=now,
|
||||||
)
|
)
|
||||||
@@ -599,7 +665,9 @@ def compute_closed_periods(session: Session) -> int:
|
|||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
def recompute_range(session: Session, start: datetime, end: datetime) -> int:
|
def recompute_range(
|
||||||
|
session: Session, start: datetime, end: datetime, *, commit: bool = True, strict: bool = False
|
||||||
|
) -> int:
|
||||||
"""Recompute (overwrite) all 15-minute periods in ``[start, end)``.
|
"""Recompute (overwrite) all 15-minute periods in ``[start, end)``.
|
||||||
|
|
||||||
This is the *explicit opt-in* path for recovering from:
|
This is the *explicit opt-in* path for recovering from:
|
||||||
@@ -620,8 +688,16 @@ def recompute_range(session: Session, start: datetime, end: datetime) -> int:
|
|||||||
Parameters
|
Parameters
|
||||||
----------
|
----------
|
||||||
session:
|
session:
|
||||||
Active SQLAlchemy session. The function commits after all periods
|
Active SQLAlchemy session.
|
||||||
have been processed.
|
commit:
|
||||||
|
When true (the default), commit after all periods have been processed.
|
||||||
|
Callers composing this recompute with other writes may pass false and
|
||||||
|
own the surrounding transaction themselves.
|
||||||
|
strict:
|
||||||
|
When true, propagate a failed period computation to the caller. This
|
||||||
|
is for lifecycle transactions which must roll back their meter/binding
|
||||||
|
mutation together with the cost recompute. The default remains
|
||||||
|
best-effort for existing background and standalone callers.
|
||||||
start:
|
start:
|
||||||
Inclusive start datetime (floored to the nearest quarter-hour internally).
|
Inclusive start datetime (floored to the nearest quarter-hour internally).
|
||||||
end:
|
end:
|
||||||
@@ -652,12 +728,15 @@ def recompute_range(session: Session, start: datetime, end: datetime) -> int:
|
|||||||
if did_write:
|
if did_write:
|
||||||
written += 1
|
written += 1
|
||||||
except Exception:
|
except Exception:
|
||||||
|
if strict:
|
||||||
|
raise
|
||||||
logger.exception(
|
logger.exception(
|
||||||
"recompute_range: unexpected error for t0=%s — continuing.",
|
"recompute_range: unexpected error for t0=%s — continuing.",
|
||||||
t0.isoformat(),
|
t0.isoformat(),
|
||||||
)
|
)
|
||||||
t0 += timedelta(minutes=_PERIOD_MINUTES)
|
t0 += timedelta(minutes=_PERIOD_MINUTES)
|
||||||
|
|
||||||
|
if commit:
|
||||||
session.commit()
|
session.commit()
|
||||||
logger.info(
|
logger.info(
|
||||||
"recompute_range(%s, %s): wrote %d period(s).",
|
"recompute_range(%s, %s): wrote %d period(s).",
|
||||||
@@ -732,9 +811,11 @@ def summarize(session: Session, start: datetime, end: datetime) -> dict[str, Any
|
|||||||
dict with keys:
|
dict with keys:
|
||||||
|
|
||||||
currency str ISO 4217 currency (from contract, or "EUR" fallback)
|
currency str ISO 4217 currency (from contract, or "EUR" fallback)
|
||||||
metered_import float Σ import_cost from non-degraded periods
|
metered_import float Σ import_cost from non-degraded periods (money)
|
||||||
metered_export float Σ export_revenue from non-degraded periods
|
metered_export float Σ export_revenue from non-degraded periods (money)
|
||||||
metered_net float Σ net_cost from non-degraded periods
|
metered_net float Σ net_cost from non-degraded periods (money)
|
||||||
|
metered_import_kwh float Σ (d1_kwh + d2_kwh) from non-degraded periods (energy)
|
||||||
|
metered_export_kwh float Σ (r1_kwh + r2_kwh) from non-degraded periods (energy)
|
||||||
fixed_costs float standing charges for elapsed whole local days
|
fixed_costs float standing charges for elapsed whole local days
|
||||||
credits float energy-tax credit for elapsed whole local days
|
credits float energy-tax credit for elapsed whole local days
|
||||||
total_payable float metered_net + fixed_costs − credits
|
total_payable float metered_net + fixed_costs − credits
|
||||||
@@ -748,12 +829,16 @@ def summarize(session: Session, start: datetime, end: datetime) -> dict[str, Any
|
|||||||
end_utc = _as_utc(end)
|
end_utc = _as_utc(end)
|
||||||
|
|
||||||
# --- Fetch all EnergyCostPeriod rows in [start, end) ---
|
# --- Fetch all EnergyCostPeriod rows in [start, end) ---
|
||||||
rows = session.execute(
|
rows = (
|
||||||
|
session.execute(
|
||||||
select(EnergyCostPeriod).where(
|
select(EnergyCostPeriod).where(
|
||||||
EnergyCostPeriod.period_start >= start_utc,
|
EnergyCostPeriod.period_start >= start_utc,
|
||||||
EnergyCostPeriod.period_start < end_utc,
|
EnergyCostPeriod.period_start < end_utc,
|
||||||
)
|
)
|
||||||
).scalars().all()
|
)
|
||||||
|
.scalars()
|
||||||
|
.all()
|
||||||
|
)
|
||||||
|
|
||||||
good_rows = [r for r in rows if not r.degraded]
|
good_rows = [r for r in rows if not r.degraded]
|
||||||
degraded_rows = [r for r in rows if r.degraded]
|
degraded_rows = [r for r in rows if r.degraded]
|
||||||
@@ -763,6 +848,15 @@ def summarize(session: Session, start: datetime, end: datetime) -> dict[str, Any
|
|||||||
sum_export = sum((_to_decimal(r.export_revenue) for r in good_rows), Decimal("0"))
|
sum_export = sum((_to_decimal(r.export_revenue) for r in good_rows), Decimal("0"))
|
||||||
sum_net = sum((_to_decimal(r.net_cost) for r in good_rows), Decimal("0"))
|
sum_net = sum((_to_decimal(r.net_cost) for r in good_rows), Decimal("0"))
|
||||||
|
|
||||||
|
# Σ metered energy (kWh), summed across both tariff registers. Reuses the
|
||||||
|
# already-fetched ``good_rows`` so no extra query is issued.
|
||||||
|
sum_import_kwh = sum(
|
||||||
|
(_to_decimal(r.d1_kwh) + _to_decimal(r.d2_kwh) for r in good_rows), Decimal("0")
|
||||||
|
)
|
||||||
|
sum_export_kwh = sum(
|
||||||
|
(_to_decimal(r.r1_kwh) + _to_decimal(r.r2_kwh) for r in good_rows), Decimal("0")
|
||||||
|
)
|
||||||
|
|
||||||
# --- Interval length in days (window, not elapsed — kept for API compat) ---
|
# --- Interval length in days (window, not elapsed — kept for API compat) ---
|
||||||
total_seconds = (end_utc - start_utc).total_seconds()
|
total_seconds = (end_utc - start_utc).total_seconds()
|
||||||
days = _to_decimal(str(total_seconds)) / _to_decimal("86400")
|
days = _to_decimal(str(total_seconds)) / _to_decimal("86400")
|
||||||
@@ -846,7 +940,9 @@ def summarize(session: Session, start: datetime, end: datetime) -> dict[str, Any
|
|||||||
version_segments: list[tuple[_date, _date | None, dict]] = []
|
version_segments: list[tuple[_date, _date | None, dict]] = []
|
||||||
for v in versions:
|
for v in versions:
|
||||||
v_start_local = local_date(_as_utc(v.effective_from))
|
v_start_local = local_date(_as_utc(v.effective_from))
|
||||||
v_end_local = local_date(_as_utc(v.effective_to)) if v.effective_to is not None else None
|
v_end_local = (
|
||||||
|
local_date(_as_utc(v.effective_to)) if v.effective_to is not None else None
|
||||||
|
)
|
||||||
version_segments.append((v_start_local, v_end_local, v.values or {}))
|
version_segments.append((v_start_local, v_end_local, v.values or {}))
|
||||||
|
|
||||||
for v_start, v_end_excl, v_values in version_segments:
|
for v_start, v_end_excl, v_values in version_segments:
|
||||||
@@ -885,6 +981,8 @@ def summarize(session: Session, start: datetime, end: datetime) -> dict[str, Any
|
|||||||
"metered_import": float(sum_import),
|
"metered_import": float(sum_import),
|
||||||
"metered_export": float(sum_export),
|
"metered_export": float(sum_export),
|
||||||
"metered_net": float(sum_net),
|
"metered_net": float(sum_net),
|
||||||
|
"metered_import_kwh": float(sum_import_kwh),
|
||||||
|
"metered_export_kwh": float(sum_export_kwh),
|
||||||
"fixed_costs": float(fixed_dec),
|
"fixed_costs": float(fixed_dec),
|
||||||
"credits": float(credits_dec),
|
"credits": float(credits_dec),
|
||||||
"total_payable": float(total_payable),
|
"total_payable": float(total_payable),
|
||||||
|
|||||||
+499
-16
@@ -34,16 +34,30 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import json
|
import json
|
||||||
import logging
|
import logging
|
||||||
|
import re
|
||||||
|
from datetime import UTC, datetime
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
from sqlalchemy.orm import Session
|
from sqlalchemy.orm import Session
|
||||||
|
from sqlalchemy import inspect
|
||||||
|
|
||||||
from app.integrations.expose import ExposableEntity, build_catalog
|
from app.integrations.expose import ExposableEntity, build_catalog
|
||||||
|
from app.models.config import AppConfigEntry
|
||||||
from app.integrations.mqtt import mqtt_manager
|
from app.integrations.mqtt import mqtt_manager
|
||||||
from app.services.config_page import build_runtime_settings
|
from app.services.config_page import build_runtime_settings
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
_HA_SEGMENT_RE = re.compile(r"[^A-Za-z0-9_-]+")
|
||||||
|
# These durable, versioned application metadata entries prevent the minute
|
||||||
|
# publisher from repeatedly emitting already accepted known-bad v1.6.1 thermal
|
||||||
|
# topics. A failed publish is not acknowledged and remains retryable across
|
||||||
|
# restarts. They are intentionally application metadata, rather than broker
|
||||||
|
# state. An empty retained publish to an illegal v1.6.1 topic cannot be used
|
||||||
|
# as an acknowledgement: HA rejects that topic before it processes its payload.
|
||||||
|
_LEGACY_THERMAL_CLEANUP_KEY = "HA_DISCOVERY_LEGACY_THERMAL_CLEANUP_V1"
|
||||||
|
_REGISTRY_REPAIR_KEY = "HA_DISCOVERY_REGISTRY_REPAIR_V2"
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# Internal helpers
|
# Internal helpers
|
||||||
@@ -59,14 +73,25 @@ def _should_publish(settings: Any) -> bool:
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
def _node_id(device_uuid: str) -> str:
|
def _safe_segment(value: str) -> str:
|
||||||
"""Stable MQTT node_id derived from device uuid (replace hyphens for safety)."""
|
"""Return a deterministic HA discovery topic segment.
|
||||||
return device_uuid.replace("-", "_")
|
|
||||||
|
Home Assistant accepts only ``[A-Za-z0-9_-]+`` for node/object ids.
|
||||||
|
Replacing each run of other characters keeps ordinary UUID/key output
|
||||||
|
readable while ensuring composite identities cannot produce invalid topics.
|
||||||
|
"""
|
||||||
|
result = _HA_SEGMENT_RE.sub("_", value).strip("_")
|
||||||
|
return result or "home_automation"
|
||||||
|
|
||||||
|
|
||||||
|
def _node_id(identity: str) -> str:
|
||||||
|
"""Stable, strictly valid MQTT node_id from an internal identity."""
|
||||||
|
return _safe_segment(identity.replace("-", "_"))
|
||||||
|
|
||||||
|
|
||||||
def _object_id(entity: ExposableEntity) -> str:
|
def _object_id(entity: ExposableEntity) -> str:
|
||||||
"""Stable MQTT object_id derived from entity key (dots + hyphens → underscores)."""
|
"""Stable, strictly valid MQTT object_id derived from the entity key."""
|
||||||
return entity.key.replace(".", "_").replace("-", "_")
|
return _safe_segment(entity.key.replace("-", "_"))
|
||||||
|
|
||||||
|
|
||||||
def _discovery_topic(entity: ExposableEntity, prefix: str) -> str:
|
def _discovery_topic(entity: ExposableEntity, prefix: str) -> str:
|
||||||
@@ -74,7 +99,7 @@ def _discovery_topic(entity: ExposableEntity, prefix: str) -> str:
|
|||||||
|
|
||||||
Format: ``<prefix>/<component>/<node_id>/<object_id>/config``
|
Format: ``<prefix>/<component>/<node_id>/<object_id>/config``
|
||||||
"""
|
"""
|
||||||
node = _node_id(entity.device.identifiers[1]) # device uuid
|
node = _node_id(entity.device.internal_identity)
|
||||||
obj = _object_id(entity)
|
obj = _object_id(entity)
|
||||||
return f"{prefix}/{entity.component}/{node}/{obj}/config"
|
return f"{prefix}/{entity.component}/{node}/{obj}/config"
|
||||||
|
|
||||||
@@ -84,7 +109,7 @@ def _state_topic(entity: ExposableEntity, prefix: str) -> str:
|
|||||||
|
|
||||||
Format: ``<prefix>/<component>/<node_id>/<object_id>/state``
|
Format: ``<prefix>/<component>/<node_id>/<object_id>/state``
|
||||||
"""
|
"""
|
||||||
node = _node_id(entity.device.identifiers[1])
|
node = _node_id(entity.device.internal_identity)
|
||||||
obj = _object_id(entity)
|
obj = _object_id(entity)
|
||||||
return f"{prefix}/{entity.component}/{node}/{obj}/state"
|
return f"{prefix}/{entity.component}/{node}/{obj}/state"
|
||||||
|
|
||||||
@@ -98,13 +123,139 @@ def _availability_topic(device_uuid: str, prefix: str) -> str:
|
|||||||
return f"{prefix}/modbus/{node}/availability"
|
return f"{prefix}/modbus/{node}/availability"
|
||||||
|
|
||||||
|
|
||||||
|
def _availability_id(entity: ExposableEntity) -> str:
|
||||||
|
"""Return the identity which owns this entity's liveness topic.
|
||||||
|
|
||||||
|
M8 meters deliberately retain their own UUID as HA node/unique identity,
|
||||||
|
while their availability is supplied by a MeterSource UUID.
|
||||||
|
"""
|
||||||
|
return entity.device.availability_id or entity.device.internal_identity
|
||||||
|
|
||||||
|
|
||||||
def _unique_id(entity: ExposableEntity) -> str:
|
def _unique_id(entity: ExposableEntity) -> str:
|
||||||
"""Stable unique_id — device uuid + metric key (never from mutable fields)."""
|
"""Stable unique_id — device uuid + metric key (never from mutable fields)."""
|
||||||
device_uuid = entity.device.identifiers[1]
|
device_uuid = entity.device.internal_identity
|
||||||
# entity.key is already "modbus.<uuid>.<metric_key>" — use it as the seed
|
# entity.key is already "modbus.<uuid>.<metric_key>" — use it as the seed
|
||||||
return f"{device_uuid}_{entity.key.replace('.', '_')}"
|
return f"{device_uuid}_{entity.key.replace('.', '_')}"
|
||||||
|
|
||||||
|
|
||||||
|
def _migration_state(session: Session, key: str, default: str) -> str:
|
||||||
|
"""Read a versioned discovery-repair acknowledgement from ``app_config``."""
|
||||||
|
entry = session.query(AppConfigEntry).filter(AppConfigEntry.key == key).one_or_none()
|
||||||
|
return entry.value if entry is not None else default
|
||||||
|
|
||||||
|
|
||||||
|
def _has_repair_store(session: Session) -> bool:
|
||||||
|
"""Whether this session uses the app schema (not a lightweight unit DB)."""
|
||||||
|
return inspect(session.get_bind()).has_table("app_config")
|
||||||
|
|
||||||
|
|
||||||
|
def _set_migration_state(session: Session, key: str, value: str) -> None:
|
||||||
|
"""Durably acknowledge a completed discovery-repair step.
|
||||||
|
|
||||||
|
The state is operational metadata only; it never alters meters, readings,
|
||||||
|
contracts, costs, or expose toggles.
|
||||||
|
"""
|
||||||
|
entry = session.query(AppConfigEntry).filter(AppConfigEntry.key == key).one_or_none()
|
||||||
|
if entry is None:
|
||||||
|
session.add(AppConfigEntry(key=key, value=value, updated_at=datetime.now(UTC)))
|
||||||
|
else:
|
||||||
|
entry.value = value
|
||||||
|
entry.updated_at = datetime.now(UTC)
|
||||||
|
session.commit()
|
||||||
|
|
||||||
|
|
||||||
|
def _migration_json(session: Session, key: str) -> dict[str, Any]:
|
||||||
|
"""Read a versioned JSON migration ledger, treating corrupt values as pending."""
|
||||||
|
raw = _migration_state(session, key, "{}")
|
||||||
|
try:
|
||||||
|
value = json.loads(raw)
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
logger.warning("Ignoring malformed HA discovery migration ledger %s", key)
|
||||||
|
return {}
|
||||||
|
return value if isinstance(value, dict) else {}
|
||||||
|
|
||||||
|
|
||||||
|
def _set_migration_json(session: Session, key: str, value: dict[str, Any]) -> None:
|
||||||
|
_set_migration_state(session, key, json.dumps(value, sort_keys=True))
|
||||||
|
|
||||||
|
|
||||||
|
def _frozen_legacy_inventory(ledger: dict[str, Any]) -> list[str] | None:
|
||||||
|
"""Return a structurally valid immutable cleanup inventory, if present.
|
||||||
|
|
||||||
|
The ledger predates a schema migration, so it must tolerate both an absent
|
||||||
|
entry and old ``topics``-only values. An inventory becomes immutable only
|
||||||
|
once it is a list of non-empty topic strings; anything else is compatibility
|
||||||
|
data to be frozen during startup, not state a publisher may reinterpret.
|
||||||
|
"""
|
||||||
|
value = ledger.get("inventory")
|
||||||
|
if not isinstance(value, list) or not all(isinstance(topic, str) and topic for topic in value):
|
||||||
|
return None
|
||||||
|
return list(dict.fromkeys(value))
|
||||||
|
|
||||||
|
|
||||||
|
def _publish_ok(topic: str, payload: str | bytes, *, retain: bool = True) -> bool:
|
||||||
|
"""Publish best-effort while treating only an explicit ``False`` as failure.
|
||||||
|
|
||||||
|
Production ``MqttManager.publish`` returns ``bool``. Accepting ``None``
|
||||||
|
keeps existing third-party/mock publishers best-effort compatible.
|
||||||
|
"""
|
||||||
|
return mqtt_manager.publish(topic, payload, retain=retain) is not False
|
||||||
|
|
||||||
|
|
||||||
|
def initialize_legacy_thermal_cleanup(session: Session) -> None:
|
||||||
|
"""Create the legacy-cleanup ledger before expose toggles can be edited.
|
||||||
|
|
||||||
|
A newly installed database has no v1.6.1 retained thermal topics. Marking
|
||||||
|
that case complete during startup prevents a later first-time toggle from
|
||||||
|
manufacturing an old, illegal topic. Conversely, an upgraded database
|
||||||
|
freezes its exact enabled legacy topics before the UI can change a toggle.
|
||||||
|
"""
|
||||||
|
if not _has_repair_store(session):
|
||||||
|
return
|
||||||
|
|
||||||
|
ledger = _migration_json(session, _LEGACY_THERMAL_CLEANUP_KEY)
|
||||||
|
if ledger.get("complete") is True:
|
||||||
|
return
|
||||||
|
|
||||||
|
inventory = _frozen_legacy_inventory(ledger)
|
||||||
|
if inventory is not None:
|
||||||
|
# A previously frozen non-empty inventory must never be expanded or
|
||||||
|
# recomputed from mutable toggle/prefix state. A zero-item inventory
|
||||||
|
# is just the fresh-install terminal state written by an older build.
|
||||||
|
if not inventory:
|
||||||
|
_set_migration_json(
|
||||||
|
session,
|
||||||
|
_LEGACY_THERMAL_CLEANUP_KEY,
|
||||||
|
{"complete": True, "inventory": [], "topics": []},
|
||||||
|
)
|
||||||
|
return
|
||||||
|
|
||||||
|
# This deliberately propagates. Startup runs before the expose UI can
|
||||||
|
# write a toggle, so swallowing an enumeration or durable-write error would
|
||||||
|
# create a window in which the cleanup scope could be changed or lost.
|
||||||
|
legacy_entities = _legacy_thermal_entities(session)
|
||||||
|
from app.config import get_settings
|
||||||
|
|
||||||
|
settings = build_runtime_settings(session, get_settings())
|
||||||
|
inventory = list(dict.fromkeys(
|
||||||
|
_legacy_discovery_topic(entity, settings.ha_discovery_prefix)
|
||||||
|
for entity in legacy_entities
|
||||||
|
))
|
||||||
|
_set_migration_json(
|
||||||
|
session,
|
||||||
|
_LEGACY_THERMAL_CLEANUP_KEY,
|
||||||
|
{
|
||||||
|
"complete": not inventory,
|
||||||
|
"inventory": inventory,
|
||||||
|
"topics": sorted({
|
||||||
|
topic for topic in ledger.get("topics", [])
|
||||||
|
if isinstance(topic, str) and topic in inventory
|
||||||
|
}),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# Public: build discovery payload
|
# Public: build discovery payload
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
@@ -139,8 +290,7 @@ def build_discovery_payload(
|
|||||||
if state_prefix is None:
|
if state_prefix is None:
|
||||||
state_prefix = discovery_prefix
|
state_prefix = discovery_prefix
|
||||||
|
|
||||||
device_uuid = entity.device.identifiers[1]
|
avail_topic = _availability_topic(_availability_id(entity), state_prefix)
|
||||||
avail_topic = _availability_topic(device_uuid, state_prefix)
|
|
||||||
state_t = _state_topic(entity, state_prefix)
|
state_t = _state_topic(entity, state_prefix)
|
||||||
topic = _discovery_topic(entity, discovery_prefix)
|
topic = _discovery_topic(entity, discovery_prefix)
|
||||||
|
|
||||||
@@ -212,19 +362,90 @@ def publish_discovery(session: Session) -> None:
|
|||||||
logger.exception("publish_discovery: failed to build catalog; aborting")
|
logger.exception("publish_discovery: failed to build catalog; aborting")
|
||||||
return
|
return
|
||||||
|
|
||||||
|
# Clear the *illegal* v1.6.1 thermal keys once. Startup freezes the exact
|
||||||
|
# inventory while the legacy toggle state still describes v1.6.1. Thus a
|
||||||
|
# later UI toggle/prefix/meter change cannot erase or expand this repair.
|
||||||
|
# ``topics`` is only durable success progress; ``inventory`` is immutable.
|
||||||
|
# Startup freezes compatibility ledgers before the UI can mutate the
|
||||||
|
# toggle/prefix inputs. The publisher only consumes that frozen snapshot.
|
||||||
|
repaired_keys: set[str] = set()
|
||||||
|
if _has_repair_store(session):
|
||||||
|
ledger = _migration_json(session, _LEGACY_THERMAL_CLEANUP_KEY)
|
||||||
|
if not ledger.get("complete"):
|
||||||
|
inventory = _frozen_legacy_inventory(ledger)
|
||||||
|
if inventory is None:
|
||||||
|
logger.error(
|
||||||
|
"publish_discovery: legacy cleanup ledger was not frozen at startup; refusing cleanup"
|
||||||
|
)
|
||||||
|
|
||||||
|
if inventory is not None:
|
||||||
|
completed = {
|
||||||
|
topic for topic in ledger.get("topics", [])
|
||||||
|
if isinstance(topic, str) and topic in inventory
|
||||||
|
}
|
||||||
|
for topic in (topic for topic in inventory if topic not in completed):
|
||||||
|
try:
|
||||||
|
if _publish_ok(topic, b""):
|
||||||
|
completed.add(topic)
|
||||||
|
# Commit each accepted illegal-topic tombstone before
|
||||||
|
# attempting another one: a crash must not replay it.
|
||||||
|
_set_migration_json(
|
||||||
|
session,
|
||||||
|
_LEGACY_THERMAL_CLEANUP_KEY,
|
||||||
|
{
|
||||||
|
"complete": False,
|
||||||
|
"inventory": inventory,
|
||||||
|
"topics": sorted(completed),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
logger.warning("publish_discovery: broker rejected v1.6.1 cleanup for %s", topic)
|
||||||
|
except Exception:
|
||||||
|
logger.exception("publish_discovery: unable to clear v1.6.1 topic %s", topic)
|
||||||
|
if set(inventory).issubset(completed):
|
||||||
|
_set_migration_json(
|
||||||
|
session,
|
||||||
|
_LEGACY_THERMAL_CLEANUP_KEY,
|
||||||
|
{"complete": True, "inventory": inventory, "topics": sorted(completed)},
|
||||||
|
)
|
||||||
|
|
||||||
|
# Current-format stale topics are legal and deliberately remain retryable:
|
||||||
|
# they cover post-repair meter swaps and include all 14 thermal metrics.
|
||||||
|
try:
|
||||||
|
stale_entities = _stale_m8_entities(session)
|
||||||
|
except Exception:
|
||||||
|
logger.exception("publish_discovery: unable to enumerate stale M8 identities")
|
||||||
|
stale_entities = []
|
||||||
|
for old_entity in stale_entities:
|
||||||
|
try:
|
||||||
|
if not _publish_ok(_discovery_topic(old_entity, discovery_prefix), b""):
|
||||||
|
logger.warning("publish_discovery: broker rejected stale cleanup for %r", old_entity.key)
|
||||||
|
except Exception:
|
||||||
|
logger.exception("publish_discovery: unable to clear stale identity %r", old_entity.key)
|
||||||
|
|
||||||
|
# v1.6.1 used the same legal topics/unique_ids for Meter, Source, Modbus
|
||||||
|
# and electricity-cost entities, but attached their registry entries to
|
||||||
|
# wrongly merged HA devices. HA does not move those entries when only a
|
||||||
|
# discovery ``device`` block changes. One durable unload -> re-add cycle
|
||||||
|
# lets HA rebuild the existing unique_id entries against the new singleton
|
||||||
|
# identifiers, preserving entity keys, toggles and user customizations.
|
||||||
|
repaired_keys = _run_registry_repair(session, catalog, discovery_prefix, state_prefix, settings)
|
||||||
|
|
||||||
for entry in catalog:
|
for entry in catalog:
|
||||||
entity = entry.entity
|
entity = entry.entity
|
||||||
|
if entity.key in repaired_keys:
|
||||||
|
continue
|
||||||
try:
|
try:
|
||||||
topic, config = build_discovery_payload(entity, discovery_prefix, state_prefix)
|
topic, config = build_discovery_payload(entity, discovery_prefix, state_prefix)
|
||||||
if entry.enabled:
|
if entry.enabled:
|
||||||
payload = json.dumps(config)
|
payload = json.dumps(config)
|
||||||
mqtt_manager.publish(topic, payload, retain=True)
|
_publish_ok(topic, payload)
|
||||||
logger.debug(
|
logger.debug(
|
||||||
"publish_discovery: published config for %r → %s", entity.key, topic
|
"publish_discovery: published config for %r → %s", entity.key, topic
|
||||||
)
|
)
|
||||||
else:
|
else:
|
||||||
# Clear retained config for disabled entities.
|
# Clear retained config for disabled entities.
|
||||||
mqtt_manager.publish(topic, b"", retain=True)
|
_publish_ok(topic, b"")
|
||||||
logger.debug(
|
logger.debug(
|
||||||
"publish_discovery: cleared config for disabled entity %r → %s",
|
"publish_discovery: cleared config for disabled entity %r → %s",
|
||||||
entity.key,
|
entity.key,
|
||||||
@@ -236,6 +457,256 @@ def publish_discovery(session: Session) -> None:
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _is_registry_repair_entity(entity: ExposableEntity) -> bool:
|
||||||
|
"""Whether a legal v1.6.1 config needs an unload/re-add device split."""
|
||||||
|
return entity.key.startswith(("modbus.", "source.", "meter.", "energy.", "thermal_cost."))
|
||||||
|
|
||||||
|
|
||||||
|
def _ha_registry_bindings(settings: Any, unique_ids: set[str]) -> dict[str, set[str]] | None:
|
||||||
|
"""Read HA's registry, returning ``None`` when it is not observable."""
|
||||||
|
from app.integrations.homeassistant import (
|
||||||
|
HomeAssistantClient,
|
||||||
|
HomeAssistantConfigError,
|
||||||
|
HomeAssistantRequestError,
|
||||||
|
)
|
||||||
|
|
||||||
|
# Runtime settings from old callers/tests may not expose outbound HA
|
||||||
|
# fields. Treat absent/non-string credentials as intentionally optional.
|
||||||
|
if not isinstance(getattr(settings, "home_assistant_base_url", None), str) or not isinstance(
|
||||||
|
getattr(settings, "home_assistant_auth_token", None), str
|
||||||
|
):
|
||||||
|
return None
|
||||||
|
client = HomeAssistantClient(settings)
|
||||||
|
if not client.is_configured():
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
return client.discovery_registry_bindings(unique_ids)
|
||||||
|
except (HomeAssistantConfigError, HomeAssistantRequestError, TypeError, ValueError):
|
||||||
|
logger.warning("HA registry repair remains pending: HA registry is unavailable")
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _run_registry_repair(
|
||||||
|
session: Session,
|
||||||
|
catalog: list[Any],
|
||||||
|
discovery_prefix: str,
|
||||||
|
state_prefix: str,
|
||||||
|
settings: Any,
|
||||||
|
) -> set[str]:
|
||||||
|
"""Repair each v1.6.1 entity only after HA confirms every phase.
|
||||||
|
|
||||||
|
MQTT acknowledges broker receipt, not Home Assistant processing. The
|
||||||
|
ledger therefore holds each target at ``unloaded`` until HA's entity
|
||||||
|
registry no longer contains its stable unique_id, then at ``republished``
|
||||||
|
until HA reports the expected singleton device identifier. A partial
|
||||||
|
catalog simply adds targets on a later run; it cannot complete others.
|
||||||
|
When HA's registry is unavailable, ordinary discovery remains untouched.
|
||||||
|
"""
|
||||||
|
targets = [entry for entry in catalog if _is_registry_repair_entity(entry.entity)]
|
||||||
|
if not targets:
|
||||||
|
return set()
|
||||||
|
ledger = _migration_json(session, _REGISTRY_REPAIR_KEY)
|
||||||
|
pending_targets = [
|
||||||
|
entry for entry in targets if ledger.get(_unique_id(entry.entity), {}).get("phase") != "complete"
|
||||||
|
]
|
||||||
|
if not pending_targets:
|
||||||
|
return set()
|
||||||
|
unique_ids = {_unique_id(entry.entity) for entry in pending_targets}
|
||||||
|
bindings = _ha_registry_bindings(settings, unique_ids)
|
||||||
|
if bindings is None:
|
||||||
|
return set()
|
||||||
|
blocked: set[str] = set()
|
||||||
|
for entry in pending_targets:
|
||||||
|
entity = entry.entity
|
||||||
|
unique_id = _unique_id(entity)
|
||||||
|
expected = set(entity.device.identifiers)
|
||||||
|
phase = ledger.get(unique_id, {}).get("phase", "pending")
|
||||||
|
observed = bindings.get(unique_id)
|
||||||
|
if phase == "unloaded":
|
||||||
|
if observed is not None:
|
||||||
|
# HA was disconnected or otherwise missed the retained
|
||||||
|
# tombstone. Keep it retained until HA itself confirms delete.
|
||||||
|
try:
|
||||||
|
_publish_ok(_discovery_topic(entity, discovery_prefix), b"")
|
||||||
|
except Exception:
|
||||||
|
logger.exception("publish_discovery: unable to repeat registry unload for %r", entity.key)
|
||||||
|
blocked.add(entity.key)
|
||||||
|
continue
|
||||||
|
if not entry.enabled:
|
||||||
|
_set_registry_phase(session, ledger, unique_id, "complete", expected)
|
||||||
|
continue
|
||||||
|
_publish_registry_config(session, ledger, entry, discovery_prefix, state_prefix, expected, blocked)
|
||||||
|
elif phase == "republished":
|
||||||
|
if observed == expected:
|
||||||
|
_set_registry_phase(session, ledger, unique_id, "complete", expected)
|
||||||
|
elif observed is None and entry.enabled:
|
||||||
|
_publish_registry_config(session, ledger, entry, discovery_prefix, state_prefix, expected, blocked)
|
||||||
|
else:
|
||||||
|
# A stale/wrong device binding must pass through deletion again.
|
||||||
|
try:
|
||||||
|
if _publish_ok(_discovery_topic(entity, discovery_prefix), b""):
|
||||||
|
_set_registry_phase(session, ledger, unique_id, "unloaded", expected)
|
||||||
|
except Exception:
|
||||||
|
logger.exception("publish_discovery: unable to unload wrong registry entity %r", entity.key)
|
||||||
|
blocked.add(entity.key)
|
||||||
|
elif phase != "complete":
|
||||||
|
if observed == expected or (observed is None and not entry.enabled):
|
||||||
|
_set_registry_phase(session, ledger, unique_id, "complete", expected)
|
||||||
|
elif observed is None:
|
||||||
|
_publish_registry_config(session, ledger, entry, discovery_prefix, state_prefix, expected, blocked)
|
||||||
|
else:
|
||||||
|
try:
|
||||||
|
if _publish_ok(_discovery_topic(entity, discovery_prefix), b""):
|
||||||
|
_set_registry_phase(session, ledger, unique_id, "unloaded", expected)
|
||||||
|
blocked.add(entity.key)
|
||||||
|
else:
|
||||||
|
logger.warning("publish_discovery: broker rejected registry unload for %r", entity.key)
|
||||||
|
except Exception:
|
||||||
|
logger.exception("publish_discovery: unable to unload registry entity %r", entity.key)
|
||||||
|
return blocked
|
||||||
|
|
||||||
|
|
||||||
|
def _set_registry_phase(
|
||||||
|
session: Session, ledger: dict[str, Any], unique_id: str, phase: str, expected: set[str]
|
||||||
|
) -> None:
|
||||||
|
ledger[unique_id] = {"phase": phase, "identifier": sorted(expected)}
|
||||||
|
_set_migration_json(session, _REGISTRY_REPAIR_KEY, ledger)
|
||||||
|
|
||||||
|
|
||||||
|
def _publish_registry_config(
|
||||||
|
session: Session, ledger: dict[str, Any], entry: Any, discovery_prefix: str,
|
||||||
|
state_prefix: str, expected: set[str], blocked: set[str],
|
||||||
|
) -> None:
|
||||||
|
entity = entry.entity
|
||||||
|
topic, config = build_discovery_payload(entity, discovery_prefix, state_prefix)
|
||||||
|
try:
|
||||||
|
if _publish_ok(topic, json.dumps(config)):
|
||||||
|
_set_registry_phase(session, ledger, _unique_id(entity), "republished", expected)
|
||||||
|
else:
|
||||||
|
logger.warning("publish_discovery: broker rejected registry re-add for %r", entity.key)
|
||||||
|
except Exception:
|
||||||
|
logger.exception("publish_discovery: unable to re-add registry entity %r", entity.key)
|
||||||
|
blocked.add(entity.key)
|
||||||
|
|
||||||
|
|
||||||
|
def _legacy_discovery_topic(entity: ExposableEntity, prefix: str) -> str:
|
||||||
|
"""Return the exact v1.6.1 config topic for a known stale M8 entity.
|
||||||
|
|
||||||
|
This intentionally preserves the old hyphen-only conversion because the
|
||||||
|
point is to remove that exact retained broker key once, never to publish a
|
||||||
|
wildcard or manufacture a new invalid topic.
|
||||||
|
"""
|
||||||
|
node = entity.device.internal_identity.replace("-", "_")
|
||||||
|
obj = entity.key.replace(".", "_").replace("-", "_")
|
||||||
|
return f"{prefix}/{entity.component}/{node}/{obj}/config"
|
||||||
|
|
||||||
|
|
||||||
|
def _overlapping_thermal_pairs(session: Session) -> list[tuple[Any, Any]]:
|
||||||
|
"""Return only heating/water Meter epochs that could have coexisted."""
|
||||||
|
from app.models.energy import Meter
|
||||||
|
from sqlalchemy import select
|
||||||
|
|
||||||
|
meters = session.execute(select(Meter).where(
|
||||||
|
Meter.commodity.in_(("electricity", "heating", "hot_water"))
|
||||||
|
)).scalars().all()
|
||||||
|
heatings = [meter for meter in meters if meter.commodity == "heating"]
|
||||||
|
waters = [meter for meter in meters if meter.commodity == "hot_water"]
|
||||||
|
pairs: list[tuple[Any, Any]] = []
|
||||||
|
for heating in heatings:
|
||||||
|
for water in waters:
|
||||||
|
# A thermal identity can only have been published when both Meter
|
||||||
|
# epochs were current at the same instant. Do not form a Cartesian
|
||||||
|
# product of historical records: that would tombstone identities
|
||||||
|
# which have never existed in HA.
|
||||||
|
heating_start, water_start = heating.started_at, water.started_at
|
||||||
|
heating_end, water_end = heating.ended_at, water.ended_at
|
||||||
|
if (heating_end is not None and water_start >= heating_end) or (
|
||||||
|
water_end is not None and heating_start >= water_end
|
||||||
|
):
|
||||||
|
continue
|
||||||
|
pairs.append((heating, water))
|
||||||
|
return pairs
|
||||||
|
|
||||||
|
|
||||||
|
def _thermal_cleanup_entities(
|
||||||
|
pairs: list[tuple[Any, Any]], *, include_hot_water_total: bool
|
||||||
|
) -> list[ExposableEntity]:
|
||||||
|
"""Build exact synthetic config entries for known thermal identities."""
|
||||||
|
from app.integrations.expose import DeviceInfo
|
||||||
|
|
||||||
|
metrics = ["heating", "hot_water_heating", "water", "water_tax", "fixed", "all_in"]
|
||||||
|
if include_hot_water_total:
|
||||||
|
metrics.insert(2, "hot_water_total")
|
||||||
|
entities: list[ExposableEntity] = []
|
||||||
|
for heating, water in pairs:
|
||||||
|
identity = ".".join(sorted((heating.uuid, water.uuid)))
|
||||||
|
info = DeviceInfo(
|
||||||
|
identifiers=(f"home-automation:thermal-cost:{identity}",),
|
||||||
|
name="obsolete",
|
||||||
|
identity=identity,
|
||||||
|
)
|
||||||
|
for metric in metrics:
|
||||||
|
for suffix in ("total", "today"):
|
||||||
|
entities.append(ExposableEntity(
|
||||||
|
key=f"thermal_cost.{identity}.{metric}_{suffix}", component="sensor", device=info,
|
||||||
|
device_class=None, unit="", name="obsolete",
|
||||||
|
))
|
||||||
|
return entities
|
||||||
|
|
||||||
|
|
||||||
|
def _legacy_thermal_entities(session: Session) -> list[ExposableEntity]:
|
||||||
|
"""Return only v1.6.1 illegal topics that could retain a non-empty config.
|
||||||
|
|
||||||
|
v1.6.1 published an illegal thermal config only while its expose toggle was
|
||||||
|
enabled. A disabled toggle published its own tombstone, so querying the
|
||||||
|
durable toggle state prevents a fresh install from manufacturing warnings
|
||||||
|
for twelve never-used illegal topics.
|
||||||
|
"""
|
||||||
|
from app.models.expose import ExposedEntityToggle
|
||||||
|
|
||||||
|
entities = _thermal_cleanup_entities(_overlapping_thermal_pairs(session), include_hot_water_total=False)
|
||||||
|
keys = [entity.key for entity in entities]
|
||||||
|
enabled_keys = {
|
||||||
|
row.key for row in session.query(ExposedEntityToggle).filter(
|
||||||
|
ExposedEntityToggle.key.in_(keys), ExposedEntityToggle.enabled.is_(True)
|
||||||
|
)
|
||||||
|
}
|
||||||
|
return [entity for entity in entities if entity.key in enabled_keys]
|
||||||
|
|
||||||
|
|
||||||
|
def _stale_m8_entities(session: Session) -> list[ExposableEntity]:
|
||||||
|
"""Return *safe-format* configs that became stale after a later epoch swap.
|
||||||
|
|
||||||
|
Unlike the v1.6.1 cleanup, this intentionally excludes the active thermal
|
||||||
|
pair and includes the post-repair ``hot_water_total`` metric (14 configs
|
||||||
|
per stale pair). All generated topics use ``_discovery_topic``.
|
||||||
|
"""
|
||||||
|
from app.integrations.expose import DeviceInfo
|
||||||
|
from app.models.energy import Meter
|
||||||
|
from sqlalchemy import select
|
||||||
|
|
||||||
|
meters = session.execute(select(Meter).where(
|
||||||
|
Meter.commodity.in_(("electricity", "heating", "hot_water"))
|
||||||
|
)).scalars().all()
|
||||||
|
entities: list[ExposableEntity] = []
|
||||||
|
for meter in (meter for meter in meters if meter.ended_at is not None):
|
||||||
|
info = DeviceInfo(
|
||||||
|
identifiers=(f"home-automation:meter:{meter.uuid}",), name=meter.label, identity=meter.uuid
|
||||||
|
)
|
||||||
|
for suffix in ("total", "today"):
|
||||||
|
entities.append(ExposableEntity(
|
||||||
|
key=f"meter.{meter.uuid}.{suffix}", component="sensor", device=info,
|
||||||
|
device_class=None, unit="", name="obsolete",
|
||||||
|
))
|
||||||
|
stale_pairs = [
|
||||||
|
(heating, water)
|
||||||
|
for heating, water in _overlapping_thermal_pairs(session)
|
||||||
|
if heating.ended_at is not None or water.ended_at is not None
|
||||||
|
]
|
||||||
|
entities.extend(_thermal_cleanup_entities(stale_pairs, include_hot_water_total=True))
|
||||||
|
return entities
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# Public: publish states
|
# Public: publish states
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
@@ -288,7 +759,19 @@ def _publish_entity_state(
|
|||||||
Also publishes the availability topic for ``binary_sensor`` "online" entities.
|
Also publishes the availability topic for ``binary_sensor`` "online" entities.
|
||||||
"""
|
"""
|
||||||
state_t = _state_topic(entity, prefix)
|
state_t = _state_topic(entity, prefix)
|
||||||
device_uuid = entity.device.identifiers[1]
|
# Source-backed entities can have a different liveness identity from their
|
||||||
|
# HA device identity. Publish it before the state; a None value below is
|
||||||
|
# intentionally not converted to a synthetic zero.
|
||||||
|
if entity.device.provides_availability and entity.device.availability_getter is not None:
|
||||||
|
try:
|
||||||
|
available = bool(entity.device.availability_getter(session))
|
||||||
|
mqtt_manager.publish(
|
||||||
|
_availability_topic(_availability_id(entity), prefix),
|
||||||
|
"online" if available else "offline",
|
||||||
|
retain=False,
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
logger.exception("availability_getter raised for entity %r", entity.key)
|
||||||
|
|
||||||
if entity.component == "binary_sensor" and "online" in entity.key:
|
if entity.component == "binary_sensor" and "online" in entity.key:
|
||||||
# The online sensor represents device availability.
|
# The online sensor represents device availability.
|
||||||
@@ -303,7 +786,7 @@ def _publish_entity_state(
|
|||||||
# Default to offline when no reading is available.
|
# Default to offline when no reading is available.
|
||||||
online = (raw_value == "ON")
|
online = (raw_value == "ON")
|
||||||
avail_payload = "online" if online else "offline"
|
avail_payload = "online" if online else "offline"
|
||||||
avail_topic = _availability_topic(device_uuid, prefix)
|
avail_topic = _availability_topic(_availability_id(entity), prefix)
|
||||||
mqtt_manager.publish(avail_topic, avail_payload, retain=False)
|
mqtt_manager.publish(avail_topic, avail_payload, retain=False)
|
||||||
# The state of the binary_sensor itself
|
# The state of the binary_sensor itself
|
||||||
state_payload = "ON" if online else "OFF"
|
state_payload = "ON" if online else "OFF"
|
||||||
@@ -377,7 +860,7 @@ def publish_device_state(session: Session, device: Any) -> None:
|
|||||||
continue
|
continue
|
||||||
entity = entry.entity
|
entity = entry.entity
|
||||||
# Only process entities belonging to this device.
|
# Only process entities belonging to this device.
|
||||||
if entity.device.identifiers[1] != device_uuid:
|
if entity.device.internal_identity != device_uuid:
|
||||||
continue
|
continue
|
||||||
# Skip the online binary_sensor itself (availability handled above).
|
# Skip the online binary_sensor itself (availability handled above).
|
||||||
if entity.component == "binary_sensor" and "online" in entity.key:
|
if entity.component == "binary_sensor" and "online" in entity.key:
|
||||||
@@ -452,7 +935,7 @@ def clear_device_discovery(session: Session, device_uuid: str) -> None:
|
|||||||
for entry in catalog:
|
for entry in catalog:
|
||||||
entity = entry.entity
|
entity = entry.entity
|
||||||
# Only clear entities belonging to this device.
|
# Only clear entities belonging to this device.
|
||||||
if entity.device.identifiers[1] != device_uuid:
|
if entity.device.internal_identity != device_uuid:
|
||||||
continue
|
continue
|
||||||
try:
|
try:
|
||||||
topic, _config = build_discovery_payload(entity, discovery_prefix, state_prefix)
|
topic, _config = build_discovery_payload(entity, discovery_prefix, state_prefix)
|
||||||
|
|||||||
@@ -0,0 +1,298 @@
|
|||||||
|
"""Thermal (WarmteLink) 15-minute cost ledger.
|
||||||
|
|
||||||
|
This module deliberately does not share the electricity ledger: thermal has
|
||||||
|
two independently-bound cumulative domains and Decimal database columns.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
from datetime import UTC, date, datetime, time, timedelta
|
||||||
|
from decimal import Decimal
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from sqlalchemy import select
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from app.models.energy import Meter, MeterCostPeriod
|
||||||
|
from app.models.meter_source import MeterSource, MeterSourceBinding, MeterSourceChannel, WarmteLinkReading
|
||||||
|
from app.services.contracts import active_contract_version_at, active_contract_versions
|
||||||
|
from app.services.energy_cost import floor_to_quarter
|
||||||
|
from app.services import timezone as timezone_service
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
_PERIOD = timedelta(minutes=15)
|
||||||
|
_FRESHNESS = timedelta(seconds=120)
|
||||||
|
_LIMITS = {"heating": Decimal("0.1"), "hot_water": Decimal("1")}
|
||||||
|
_ACCEPTED_QUALITIES = {"valid", "unverifiable"}
|
||||||
|
_SETTLEMENT_TIME = time(1, 5)
|
||||||
|
|
||||||
|
|
||||||
|
def _utc(value: datetime) -> datetime:
|
||||||
|
return value.replace(tzinfo=UTC) if value.tzinfo is None else value.astimezone(UTC)
|
||||||
|
|
||||||
|
|
||||||
|
def _decimal(value: Any) -> Decimal:
|
||||||
|
return value if isinstance(value, Decimal) else Decimal(str(value))
|
||||||
|
|
||||||
|
|
||||||
|
def _existing(session: Session, commodity: str, start: datetime) -> MeterCostPeriod | None:
|
||||||
|
return session.execute(
|
||||||
|
select(MeterCostPeriod).where(
|
||||||
|
MeterCostPeriod.commodity == commodity, MeterCostPeriod.period_start == start
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
|
||||||
|
|
||||||
|
def _meter_at(session: Session, commodity: str, instant: datetime) -> Meter | None:
|
||||||
|
meters = session.execute(
|
||||||
|
select(Meter).where(
|
||||||
|
Meter.commodity == commodity,
|
||||||
|
Meter.started_at <= instant,
|
||||||
|
(Meter.ended_at.is_(None)) | (Meter.ended_at > instant),
|
||||||
|
)
|
||||||
|
).scalars().all()
|
||||||
|
return meters[0] if len(meters) == 1 else None
|
||||||
|
|
||||||
|
|
||||||
|
def _binding_at(
|
||||||
|
session: Session, meter: Meter, instant: datetime
|
||||||
|
) -> tuple[MeterSourceBinding, MeterSourceChannel] | None:
|
||||||
|
rows = session.execute(
|
||||||
|
select(MeterSourceBinding, MeterSourceChannel)
|
||||||
|
.join(MeterSourceChannel, MeterSourceChannel.id == MeterSourceBinding.channel_id)
|
||||||
|
.join(MeterSource, MeterSource.id == MeterSourceChannel.source_id)
|
||||||
|
.where(
|
||||||
|
MeterSourceBinding.meter_id == meter.id,
|
||||||
|
MeterSourceBinding.started_at <= instant,
|
||||||
|
(MeterSourceBinding.ended_at.is_(None)) | (MeterSourceBinding.ended_at > instant),
|
||||||
|
MeterSource.kind == "warmtelink_serial",
|
||||||
|
)
|
||||||
|
).all()
|
||||||
|
return rows[0] if len(rows) == 1 else None
|
||||||
|
|
||||||
|
|
||||||
|
def _reading_at(
|
||||||
|
session: Session,
|
||||||
|
channel: MeterSourceChannel,
|
||||||
|
meter: Meter,
|
||||||
|
binding: MeterSourceBinding,
|
||||||
|
target: datetime,
|
||||||
|
) -> WarmteLinkReading | None:
|
||||||
|
"""Choose the nearest accepted reading inside this cumulative domain.
|
||||||
|
|
||||||
|
Freshness alone is insufficient: a frame immediately before a meter or
|
||||||
|
source hand-off belongs to a different cumulative register and must never
|
||||||
|
be used as the other side of a delta.
|
||||||
|
"""
|
||||||
|
window_start, window_end = target - _FRESHNESS, target + _FRESHNESS
|
||||||
|
rows = session.execute(
|
||||||
|
select(WarmteLinkReading)
|
||||||
|
.where(
|
||||||
|
WarmteLinkReading.channel_id == channel.id,
|
||||||
|
WarmteLinkReading.recorded_at >= window_start,
|
||||||
|
WarmteLinkReading.recorded_at <= window_end,
|
||||||
|
)
|
||||||
|
).scalars().all()
|
||||||
|
domain_start = max(_utc(meter.started_at), _utc(binding.started_at))
|
||||||
|
domain_ends = (meter.ended_at, binding.ended_at)
|
||||||
|
domain_end = min((_utc(value) for value in domain_ends if value is not None), default=None)
|
||||||
|
accepted = [
|
||||||
|
row for row in rows
|
||||||
|
if row.quality in _ACCEPTED_QUALITIES
|
||||||
|
and _utc(row.recorded_at) >= domain_start
|
||||||
|
and (domain_end is None or _utc(row.recorded_at) < domain_end)
|
||||||
|
]
|
||||||
|
if not accepted:
|
||||||
|
return None
|
||||||
|
return min(
|
||||||
|
accepted,
|
||||||
|
key=lambda row: (abs((_utc(row.recorded_at) - target).total_seconds()), _utc(row.recorded_at)),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _degrade(
|
||||||
|
session: Session,
|
||||||
|
commodity: str,
|
||||||
|
start: datetime,
|
||||||
|
end: datetime,
|
||||||
|
existing: MeterCostPeriod | None,
|
||||||
|
reason: str,
|
||||||
|
meter_id: int | None = None,
|
||||||
|
binding_id: int | None = None,
|
||||||
|
) -> None:
|
||||||
|
now = datetime.now(UTC)
|
||||||
|
fields = dict(
|
||||||
|
period_end=end, meter_id=meter_id, source_binding_id=binding_id,
|
||||||
|
contract_version_id=None, quantity=Decimal("0"), cost=Decimal("0"), currency="EUR",
|
||||||
|
cost_breakdown={}, pricing_snapshot={}, quality="invalid", degraded=True,
|
||||||
|
degraded_reason=reason, updated_at=now,
|
||||||
|
)
|
||||||
|
if existing is None:
|
||||||
|
session.add(MeterCostPeriod(commodity=commodity, period_start=start, created_at=now, **fields))
|
||||||
|
else:
|
||||||
|
for key, value in fields.items():
|
||||||
|
setattr(existing, key, value)
|
||||||
|
|
||||||
|
|
||||||
|
def compute_period(
|
||||||
|
session: Session, commodity: str, period_start: datetime, *, overwrite: bool = False
|
||||||
|
) -> bool:
|
||||||
|
"""Compute one closed thermal period, recording every unsafe input as degraded."""
|
||||||
|
if commodity not in _LIMITS:
|
||||||
|
raise ValueError("commodity must be heating or hot_water")
|
||||||
|
start = floor_to_quarter(_utc(period_start))
|
||||||
|
end = start + _PERIOD
|
||||||
|
existing = _existing(session, commodity, start)
|
||||||
|
if existing is not None and not existing.degraded and not overwrite:
|
||||||
|
return False
|
||||||
|
|
||||||
|
meter0, meter1 = _meter_at(session, commodity, start), _meter_at(session, commodity, end)
|
||||||
|
if meter0 is None:
|
||||||
|
_degrade(session, commodity, start, end, existing, "missing_or_ambiguous_meter")
|
||||||
|
return True
|
||||||
|
if meter1 is None or meter1.id != meter0.id:
|
||||||
|
_degrade(session, commodity, start, end, existing, "cross_meter_epoch", meter0.id)
|
||||||
|
return True
|
||||||
|
bound0, bound1 = _binding_at(session, meter0, start), _binding_at(session, meter1, end)
|
||||||
|
if bound0 is None or bound1 is None:
|
||||||
|
_degrade(session, commodity, start, end, existing, "missing_or_ambiguous_binding", meter0.id)
|
||||||
|
return True
|
||||||
|
binding, channel = bound0
|
||||||
|
if bound1[0].id != binding.id or bound1[1].id != channel.id:
|
||||||
|
_degrade(session, commodity, start, end, existing, "cross_source_binding", meter0.id, binding.id)
|
||||||
|
return True
|
||||||
|
first = _reading_at(session, channel, meter0, binding, start)
|
||||||
|
last = _reading_at(session, channel, meter0, binding, end)
|
||||||
|
if first is None or last is None:
|
||||||
|
_degrade(session, commodity, start, end, existing, "missing_stale_or_invalid_reading", meter0.id, binding.id)
|
||||||
|
return True
|
||||||
|
delta = _decimal(last.value) - _decimal(first.value)
|
||||||
|
if delta < 0:
|
||||||
|
_degrade(session, commodity, start, end, existing, "negative_delta", meter0.id, binding.id)
|
||||||
|
return True
|
||||||
|
if delta > _LIMITS[commodity]:
|
||||||
|
_degrade(session, commodity, start, end, existing, "delta_limit_exceeded", meter0.id, binding.id)
|
||||||
|
return True
|
||||||
|
version = active_contract_version_at(session, start, scope="thermal")
|
||||||
|
if version is None:
|
||||||
|
_degrade(session, commodity, start, end, existing, "missing_contract", meter0.id, binding.id)
|
||||||
|
return True
|
||||||
|
|
||||||
|
values = {key: _decimal(value) for key, value in version.values["variable"].items()}
|
||||||
|
if commodity == "heating":
|
||||||
|
breakdown = {"heating": delta * values["heating"]}
|
||||||
|
else:
|
||||||
|
breakdown = {
|
||||||
|
key: delta * values[key]
|
||||||
|
for key in ("hot_water_heating", "hot_water", "hot_water_tax")
|
||||||
|
}
|
||||||
|
cost = sum(breakdown.values(), Decimal("0"))
|
||||||
|
now = datetime.now(UTC)
|
||||||
|
fields = dict(
|
||||||
|
period_end=end, meter_id=meter0.id, source_binding_id=binding.id,
|
||||||
|
contract_version_id=version.id, quantity=delta, cost=cost, currency=version.contract.currency,
|
||||||
|
cost_breakdown=breakdown, pricing_snapshot=dict(version.values),
|
||||||
|
quality="valid" if first.quality == last.quality == "valid" else "unverifiable",
|
||||||
|
degraded=False, degraded_reason=None, updated_at=now,
|
||||||
|
)
|
||||||
|
if existing is None:
|
||||||
|
session.add(MeterCostPeriod(commodity=commodity, period_start=start, created_at=now, **fields))
|
||||||
|
else:
|
||||||
|
for key, value in fields.items():
|
||||||
|
setattr(existing, key, value)
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
def compute_closed_periods(session: Session, *, now: datetime | None = None) -> int:
|
||||||
|
"""Retry incomplete thermal rows and fill recent closed periods without touching good rows."""
|
||||||
|
now = _utc(now or datetime.now(UTC))
|
||||||
|
first = floor_to_quarter(now - timedelta(days=7))
|
||||||
|
written = 0
|
||||||
|
cursor = first
|
||||||
|
while cursor + _PERIOD <= now:
|
||||||
|
for commodity in ("heating", "hot_water"):
|
||||||
|
if compute_period(session, commodity, cursor):
|
||||||
|
written += 1
|
||||||
|
cursor += _PERIOD
|
||||||
|
session.commit()
|
||||||
|
return written
|
||||||
|
|
||||||
|
|
||||||
|
def recompute_range(session: Session, start: datetime, end: datetime, *, commit: bool = True) -> int:
|
||||||
|
"""Recompute a thermal range.
|
||||||
|
|
||||||
|
The historical service entry point remains self-committing for the scheduler
|
||||||
|
and direct callers. HTTP callers pass ``commit=False`` so validation,
|
||||||
|
recomputation, response statistics, and the single commit share one
|
||||||
|
transaction owned by the route.
|
||||||
|
"""
|
||||||
|
cursor, end = floor_to_quarter(_utc(start)), _utc(end)
|
||||||
|
now, written = datetime.now(UTC), 0
|
||||||
|
while cursor < end:
|
||||||
|
if cursor + _PERIOD <= now:
|
||||||
|
for commodity in ("heating", "hot_water"):
|
||||||
|
if compute_period(session, commodity, cursor, overwrite=True):
|
||||||
|
written += 1
|
||||||
|
cursor += _PERIOD
|
||||||
|
if commit:
|
||||||
|
session.commit()
|
||||||
|
return written
|
||||||
|
|
||||||
|
|
||||||
|
def _settled_end_date(now: datetime) -> date:
|
||||||
|
local_now = timezone_service.to_local(now)
|
||||||
|
return local_now.date() if local_now.timetz().replace(tzinfo=None) >= _SETTLEMENT_TIME else local_now.date() - timedelta(days=1)
|
||||||
|
|
||||||
|
|
||||||
|
def summarize(session: Session, start: datetime, end: datetime, *, now: datetime | None = None) -> dict[str, Any]:
|
||||||
|
"""Return thermal variable/fixed totals; standing is charged once per contract/day."""
|
||||||
|
start, end = _utc(start), _utc(end)
|
||||||
|
rows = session.execute(select(MeterCostPeriod).where(
|
||||||
|
MeterCostPeriod.period_start >= start, MeterCostPeriod.period_start < end
|
||||||
|
)).scalars().all()
|
||||||
|
good = [row for row in rows if not row.degraded]
|
||||||
|
variable = sum((_decimal(row.cost) for row in good), Decimal("0"))
|
||||||
|
breakdown: dict[str, Decimal] = {key: Decimal("0") for key in (
|
||||||
|
"heating", "hot_water_heating", "hot_water", "hot_water_tax")}
|
||||||
|
for row in good:
|
||||||
|
for key, value in row.cost_breakdown.items():
|
||||||
|
breakdown[key] = breakdown.get(key, Decimal("0")) + _decimal(value)
|
||||||
|
|
||||||
|
# A summary is half-open. ``end`` at local midnight has no overlap with
|
||||||
|
# that next local date, and an empty/reversed range owns no standing day.
|
||||||
|
final_day = timezone_service.local_date(end - timedelta(microseconds=1))
|
||||||
|
final_day = min(final_day, _settled_end_date(now or datetime.now(UTC)))
|
||||||
|
day = timezone_service.local_date(start)
|
||||||
|
fixed_breakdown: dict[str, Decimal] = {key: Decimal("0") for key in (
|
||||||
|
"heating_network", "metering", "delivery_set", "hot_water_network", "other"
|
||||||
|
)}
|
||||||
|
versions = active_contract_versions(session, scope="thermal")
|
||||||
|
while start < end and day <= final_day:
|
||||||
|
day_start = datetime.combine(day, time.min, tzinfo=timezone_service.local_tz()).astimezone(UTC)
|
||||||
|
next_day_start = datetime.combine(
|
||||||
|
day + timedelta(days=1), time.min, tzinfo=timezone_service.local_tz()
|
||||||
|
).astimezone(UTC)
|
||||||
|
local_day_seconds = Decimal(str((next_day_start - day_start).total_seconds()))
|
||||||
|
# A rate revision part-way through a local date is attributable only
|
||||||
|
# to its effective interval. This preserves one contract-level daily
|
||||||
|
# charge while correctly handling first-version and intra-day changes.
|
||||||
|
for version in versions:
|
||||||
|
segment_start = max(day_start, _utc(version.effective_from))
|
||||||
|
version_end = _utc(version.effective_to) if version.effective_to is not None else next_day_start
|
||||||
|
segment_end = min(next_day_start, version_end)
|
||||||
|
if segment_start >= segment_end:
|
||||||
|
continue
|
||||||
|
values = version.values["standing"]
|
||||||
|
fraction = Decimal(str((segment_end - segment_start).total_seconds())) / local_day_seconds
|
||||||
|
for key in fixed_breakdown:
|
||||||
|
fixed_breakdown[key] += _decimal(values.get(key, "0")) / Decimal("365") * fraction
|
||||||
|
day += timedelta(days=1)
|
||||||
|
fixed = sum(fixed_breakdown.values(), Decimal("0"))
|
||||||
|
return {
|
||||||
|
"currency": good[0].currency if good else "EUR", "variable_cost": variable,
|
||||||
|
"fixed_cost": fixed, "fixed_breakdown": fixed_breakdown,
|
||||||
|
"total_cost": variable + fixed, "breakdown": breakdown,
|
||||||
|
"period_count": len(good), "degraded_count": len(rows) - len(good),
|
||||||
|
}
|
||||||
@@ -0,0 +1,529 @@
|
|||||||
|
"""Service layer for source/channel discovery and meter-source bindings.
|
||||||
|
|
||||||
|
All mutating functions receive a caller-owned :class:`~sqlalchemy.orm.Session`
|
||||||
|
and never commit. This lets HTTP handlers compose source and meter changes in
|
||||||
|
one transaction later without exposing any connection I/O here.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import UTC, datetime
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from sqlalchemy import or_, select
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from app.integrations.meter_sources import (
|
||||||
|
SourceProfileError,
|
||||||
|
get_source_profile,
|
||||||
|
merge_source_config,
|
||||||
|
validate_source_config,
|
||||||
|
)
|
||||||
|
from app.models.energy import Meter
|
||||||
|
from app.models.meter_source import (
|
||||||
|
MeterSource,
|
||||||
|
MeterSourceBinding,
|
||||||
|
MeterSourceChannel,
|
||||||
|
half_open_intervals_overlap,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class MeterSourceError(ValueError):
|
||||||
|
"""Base class for source-domain validation errors."""
|
||||||
|
|
||||||
|
|
||||||
|
class SourceNotFoundError(MeterSourceError):
|
||||||
|
"""Raised when the requested source does not exist."""
|
||||||
|
|
||||||
|
|
||||||
|
class ChannelNotFoundError(MeterSourceError):
|
||||||
|
"""Raised when the requested source channel does not exist."""
|
||||||
|
|
||||||
|
|
||||||
|
class MeterNotFoundError(MeterSourceError):
|
||||||
|
"""Raised when the requested meter does not exist."""
|
||||||
|
|
||||||
|
|
||||||
|
class BindingNotFoundError(MeterSourceError):
|
||||||
|
"""Raised when the requested binding does not exist."""
|
||||||
|
|
||||||
|
|
||||||
|
class BindingValidationError(MeterSourceError):
|
||||||
|
"""Raised for an incompatible unit, commodity, or invalid interval."""
|
||||||
|
|
||||||
|
|
||||||
|
class BindingOverlapError(BindingValidationError):
|
||||||
|
"""Raised when a meter or channel already has an overlapping binding."""
|
||||||
|
|
||||||
|
|
||||||
|
class SourceDeleteRestrictedError(MeterSourceError):
|
||||||
|
"""Raised when a source has retained channel, binding, or reading history."""
|
||||||
|
|
||||||
|
|
||||||
|
COMMODITY_UNITS = {"electricity": "kWh", "heating": "GJ", "hot_water": "m³"}
|
||||||
|
_UNSET = object()
|
||||||
|
|
||||||
|
|
||||||
|
def _utc_now() -> datetime:
|
||||||
|
return datetime.now(UTC)
|
||||||
|
|
||||||
|
|
||||||
|
def _as_utc(value: datetime) -> datetime:
|
||||||
|
return value.replace(tzinfo=UTC) if value.tzinfo is None else value.astimezone(UTC)
|
||||||
|
|
||||||
|
|
||||||
|
def get_source(session: Session, source_id: int) -> MeterSource:
|
||||||
|
source = session.get(MeterSource, source_id)
|
||||||
|
if source is None:
|
||||||
|
raise SourceNotFoundError(f"Meter source {source_id} was not found.")
|
||||||
|
return source
|
||||||
|
|
||||||
|
|
||||||
|
def list_sources(session: Session, *, kind: str | None = None) -> list[MeterSource]:
|
||||||
|
statement = select(MeterSource).order_by(MeterSource.id)
|
||||||
|
if kind is not None:
|
||||||
|
get_source_profile(kind)
|
||||||
|
statement = statement.where(MeterSource.kind == kind)
|
||||||
|
return list(session.execute(statement).scalars())
|
||||||
|
|
||||||
|
|
||||||
|
def create_source(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
name: str,
|
||||||
|
kind: str,
|
||||||
|
config: dict[str, Any],
|
||||||
|
enabled: bool = True,
|
||||||
|
) -> MeterSource:
|
||||||
|
"""Add a source after validating its complete kind-specific config."""
|
||||||
|
now = _utc_now()
|
||||||
|
source = MeterSource(
|
||||||
|
name=name,
|
||||||
|
kind=kind,
|
||||||
|
enabled=enabled,
|
||||||
|
config=validate_source_config(kind, config),
|
||||||
|
created_at=now,
|
||||||
|
updated_at=now,
|
||||||
|
)
|
||||||
|
session.add(source)
|
||||||
|
return source
|
||||||
|
|
||||||
|
|
||||||
|
def update_source(
|
||||||
|
session: Session,
|
||||||
|
source_id: int,
|
||||||
|
*,
|
||||||
|
name: str | None = None,
|
||||||
|
enabled: bool | None = None,
|
||||||
|
config_patch: dict[str, Any] | None = None,
|
||||||
|
) -> MeterSource:
|
||||||
|
"""Update source metadata and/or merge a partial source config without commit."""
|
||||||
|
source = get_source(session, source_id)
|
||||||
|
if name is not None:
|
||||||
|
source.name = name
|
||||||
|
if enabled is not None:
|
||||||
|
source.enabled = enabled
|
||||||
|
if config_patch is not None:
|
||||||
|
source.config = merge_source_config(source.kind, source.config, config_patch)
|
||||||
|
source.updated_at = _utc_now()
|
||||||
|
return source
|
||||||
|
|
||||||
|
|
||||||
|
def delete_source(session: Session, source_id: int) -> None:
|
||||||
|
"""Delete an entirely unused source; history is always retained instead."""
|
||||||
|
source = get_source(session, source_id)
|
||||||
|
has_channel = session.execute(
|
||||||
|
select(MeterSourceChannel.id).where(MeterSourceChannel.source_id == source.id).limit(1)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
has_binding = session.execute(
|
||||||
|
select(MeterSourceBinding.id)
|
||||||
|
.join(MeterSourceChannel)
|
||||||
|
.where(MeterSourceChannel.source_id == source.id)
|
||||||
|
.limit(1)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if has_channel is not None or has_binding is not None:
|
||||||
|
raise SourceDeleteRestrictedError(
|
||||||
|
f"Meter source {source_id} has dependent channels, bindings, or readings."
|
||||||
|
)
|
||||||
|
session.delete(source)
|
||||||
|
|
||||||
|
|
||||||
|
def get_channel(session: Session, channel_id: int) -> MeterSourceChannel:
|
||||||
|
channel = session.get(MeterSourceChannel, channel_id)
|
||||||
|
if channel is None:
|
||||||
|
raise ChannelNotFoundError(f"Meter source channel {channel_id} was not found.")
|
||||||
|
return channel
|
||||||
|
|
||||||
|
|
||||||
|
def upsert_discovered_channel(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
source_id: int,
|
||||||
|
channel_key: str,
|
||||||
|
label: str,
|
||||||
|
unit: str,
|
||||||
|
suggested_commodity: str | None = None,
|
||||||
|
device_type: str | None = None,
|
||||||
|
fingerprint: str | None = None,
|
||||||
|
latest_value: Any = None,
|
||||||
|
latest_at: datetime | None = None,
|
||||||
|
latest_quality: str | None = None,
|
||||||
|
) -> MeterSourceChannel:
|
||||||
|
"""Idempotently create or refresh a discovered channel's metadata.
|
||||||
|
|
||||||
|
``suggested_commodity`` remains metadata only; this function never creates
|
||||||
|
a meter or a binding.
|
||||||
|
"""
|
||||||
|
source = get_source(session, source_id)
|
||||||
|
if unit not in get_source_profile(source.kind).allowed_units:
|
||||||
|
raise SourceProfileError(f"Unit {unit!r} is not allowed for source kind {source.kind!r}.")
|
||||||
|
channel = session.execute(
|
||||||
|
select(MeterSourceChannel).where(
|
||||||
|
MeterSourceChannel.source_id == source.id,
|
||||||
|
MeterSourceChannel.channel_key == channel_key,
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
now = _utc_now()
|
||||||
|
if channel is None:
|
||||||
|
channel = MeterSourceChannel(
|
||||||
|
source_id=source.id,
|
||||||
|
channel_key=channel_key,
|
||||||
|
label=label,
|
||||||
|
unit=unit,
|
||||||
|
suggested_commodity=suggested_commodity,
|
||||||
|
device_type=device_type,
|
||||||
|
fingerprint=fingerprint,
|
||||||
|
latest_value=latest_value,
|
||||||
|
latest_at=latest_at,
|
||||||
|
latest_quality=latest_quality,
|
||||||
|
created_at=now,
|
||||||
|
updated_at=now,
|
||||||
|
)
|
||||||
|
session.add(channel)
|
||||||
|
return channel
|
||||||
|
|
||||||
|
if channel.unit != unit:
|
||||||
|
binding_id = session.execute(
|
||||||
|
select(MeterSourceBinding.id)
|
||||||
|
.where(MeterSourceBinding.channel_id == channel.id)
|
||||||
|
.limit(1)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if binding_id is not None:
|
||||||
|
raise BindingValidationError(
|
||||||
|
f"Cannot change unit of bound channel {channel.id} from {channel.unit!r} to {unit!r}."
|
||||||
|
)
|
||||||
|
|
||||||
|
channel.label = label
|
||||||
|
channel.unit = unit
|
||||||
|
channel.suggested_commodity = suggested_commodity
|
||||||
|
channel.device_type = device_type
|
||||||
|
channel.fingerprint = fingerprint
|
||||||
|
channel.latest_value = latest_value
|
||||||
|
channel.latest_at = latest_at
|
||||||
|
channel.latest_quality = latest_quality
|
||||||
|
channel.updated_at = now
|
||||||
|
return channel
|
||||||
|
|
||||||
|
|
||||||
|
def list_bindings(
|
||||||
|
session: Session, *, meter_id: int | None = None, channel_id: int | None = None
|
||||||
|
) -> list[MeterSourceBinding]:
|
||||||
|
statement = select(MeterSourceBinding).order_by(MeterSourceBinding.started_at, MeterSourceBinding.id)
|
||||||
|
if meter_id is not None:
|
||||||
|
statement = statement.where(MeterSourceBinding.meter_id == meter_id)
|
||||||
|
if channel_id is not None:
|
||||||
|
statement = statement.where(MeterSourceBinding.channel_id == channel_id)
|
||||||
|
return list(session.execute(statement).scalars())
|
||||||
|
|
||||||
|
|
||||||
|
def _get_meter(session: Session, meter_id: int) -> Meter:
|
||||||
|
meter = session.get(Meter, meter_id)
|
||||||
|
if meter is None:
|
||||||
|
raise MeterNotFoundError(f"Meter {meter_id} was not found.")
|
||||||
|
return meter
|
||||||
|
|
||||||
|
|
||||||
|
def _validate_binding(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
meter_id: int,
|
||||||
|
channel_id: int,
|
||||||
|
started_at: datetime,
|
||||||
|
ended_at: datetime | None,
|
||||||
|
excluding_ids: set[int] | None = None,
|
||||||
|
) -> None:
|
||||||
|
meter = _get_meter(session, meter_id)
|
||||||
|
channel = get_channel(session, channel_id)
|
||||||
|
expected_unit = COMMODITY_UNITS.get(meter.commodity)
|
||||||
|
if expected_unit is None:
|
||||||
|
raise BindingValidationError(f"Commodity {meter.commodity!r} cannot be bound to a source channel.")
|
||||||
|
if channel.unit != expected_unit:
|
||||||
|
raise BindingValidationError(
|
||||||
|
f"Meter commodity {meter.commodity!r} requires unit {expected_unit!r}, "
|
||||||
|
f"but channel has {channel.unit!r}."
|
||||||
|
)
|
||||||
|
if ended_at is not None and _as_utc(ended_at) <= _as_utc(started_at):
|
||||||
|
raise BindingValidationError("Binding ended_at must be strictly after started_at.")
|
||||||
|
if _as_utc(started_at) < _as_utc(meter.started_at):
|
||||||
|
raise BindingValidationError("Binding must not start before its meter epoch.")
|
||||||
|
if meter.ended_at is None:
|
||||||
|
if ended_at is not None:
|
||||||
|
# A historical binding on an active epoch is valid, but it must be
|
||||||
|
# wholly within that epoch (whose upper bound is open).
|
||||||
|
pass
|
||||||
|
else:
|
||||||
|
meter_end = _as_utc(meter.ended_at)
|
||||||
|
if ended_at is None or _as_utc(ended_at) > meter_end:
|
||||||
|
raise BindingValidationError("Closed meter bindings must end within the meter epoch.")
|
||||||
|
|
||||||
|
excluded = excluding_ids or set()
|
||||||
|
candidates = session.execute(
|
||||||
|
select(MeterSourceBinding).where(
|
||||||
|
or_(
|
||||||
|
MeterSourceBinding.meter_id == meter_id,
|
||||||
|
MeterSourceBinding.channel_id == channel_id,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalars()
|
||||||
|
for existing in candidates:
|
||||||
|
if existing.id in excluded:
|
||||||
|
continue
|
||||||
|
if half_open_intervals_overlap(
|
||||||
|
_as_utc(started_at),
|
||||||
|
_as_utc(ended_at) if ended_at is not None else None,
|
||||||
|
_as_utc(existing.started_at),
|
||||||
|
_as_utc(existing.ended_at) if existing.ended_at is not None else None,
|
||||||
|
):
|
||||||
|
side = "meter" if existing.meter_id == meter_id else "channel"
|
||||||
|
raise BindingOverlapError(f"Binding overlaps existing {side} binding {existing.id}.")
|
||||||
|
|
||||||
|
|
||||||
|
def create_binding(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
meter_id: int,
|
||||||
|
channel_id: int,
|
||||||
|
started_at: datetime,
|
||||||
|
ended_at: datetime | None = None,
|
||||||
|
) -> MeterSourceBinding:
|
||||||
|
"""Create a compatible non-overlapping half-open source binding."""
|
||||||
|
now = _utc_now()
|
||||||
|
if _as_utc(started_at) > now or (ended_at is not None and _as_utc(ended_at) > now):
|
||||||
|
raise BindingValidationError("Binding boundaries must not be in the future.")
|
||||||
|
_validate_binding(
|
||||||
|
session,
|
||||||
|
meter_id=meter_id,
|
||||||
|
channel_id=channel_id,
|
||||||
|
started_at=started_at,
|
||||||
|
ended_at=ended_at,
|
||||||
|
)
|
||||||
|
binding = MeterSourceBinding(
|
||||||
|
meter_id=meter_id,
|
||||||
|
channel_id=channel_id,
|
||||||
|
started_at=started_at,
|
||||||
|
ended_at=ended_at,
|
||||||
|
created_at=now,
|
||||||
|
updated_at=now,
|
||||||
|
)
|
||||||
|
session.add(binding)
|
||||||
|
return binding
|
||||||
|
|
||||||
|
|
||||||
|
def create_binding_for_meter_swap(
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
old_meter_id: int | None,
|
||||||
|
new_meter_id: int,
|
||||||
|
channel_id: int,
|
||||||
|
started_at: datetime,
|
||||||
|
) -> MeterSourceBinding:
|
||||||
|
"""Create a binding during a physical meter swap, handing off one channel if safe.
|
||||||
|
|
||||||
|
A channel is transferable only when exactly one of its bindings covered the
|
||||||
|
instant immediately before ``started_at`` and that binding belongs to the
|
||||||
|
meter which this declaration just closed. All other occupied or ambiguous
|
||||||
|
cases retain the normal fail-closed overlap behaviour.
|
||||||
|
|
||||||
|
This function deliberately does not commit. The caller must keep the meter
|
||||||
|
declaration, binding handoff, and any billing recompute in one transaction.
|
||||||
|
"""
|
||||||
|
new_meter = _get_meter(session, new_meter_id)
|
||||||
|
channel = get_channel(session, channel_id)
|
||||||
|
expected_unit = COMMODITY_UNITS.get(new_meter.commodity)
|
||||||
|
if expected_unit is None or channel.unit != expected_unit:
|
||||||
|
raise BindingValidationError(
|
||||||
|
f"Meter commodity {new_meter.commodity!r} requires unit {expected_unit!r}, "
|
||||||
|
f"but channel has {channel.unit!r}."
|
||||||
|
)
|
||||||
|
|
||||||
|
boundary = _as_utc(started_at)
|
||||||
|
if _as_utc(new_meter.started_at) != boundary:
|
||||||
|
raise BindingValidationError(
|
||||||
|
"Meter-swap binding must start at the new meter's started_at boundary."
|
||||||
|
)
|
||||||
|
covering_bindings = [
|
||||||
|
binding
|
||||||
|
for binding in session.execute(
|
||||||
|
select(MeterSourceBinding).where(MeterSourceBinding.channel_id == channel_id)
|
||||||
|
).scalars()
|
||||||
|
if _as_utc(binding.started_at) < boundary
|
||||||
|
and (binding.ended_at is None or _as_utc(binding.ended_at) >= boundary)
|
||||||
|
]
|
||||||
|
|
||||||
|
if not covering_bindings:
|
||||||
|
return create_binding(
|
||||||
|
session,
|
||||||
|
meter_id=new_meter_id,
|
||||||
|
channel_id=channel_id,
|
||||||
|
started_at=started_at,
|
||||||
|
)
|
||||||
|
|
||||||
|
if old_meter_id is None or len(covering_bindings) != 1:
|
||||||
|
raise BindingOverlapError("Channel is occupied or has an ambiguous binding at meter swap.")
|
||||||
|
|
||||||
|
old_meter = _get_meter(session, old_meter_id)
|
||||||
|
old_binding = covering_bindings[0]
|
||||||
|
if (
|
||||||
|
old_meter.commodity != new_meter.commodity
|
||||||
|
or old_meter.ended_at is None
|
||||||
|
or _as_utc(old_meter.ended_at) != boundary
|
||||||
|
or old_binding.meter_id != old_meter.id
|
||||||
|
):
|
||||||
|
raise BindingOverlapError("Channel is occupied by a binding that cannot be handed off.")
|
||||||
|
|
||||||
|
update_binding(session, old_binding.id, ended_at=started_at)
|
||||||
|
return create_binding(
|
||||||
|
session,
|
||||||
|
meter_id=new_meter_id,
|
||||||
|
channel_id=channel_id,
|
||||||
|
started_at=started_at,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def update_binding(
|
||||||
|
session: Session,
|
||||||
|
binding_id: int,
|
||||||
|
*,
|
||||||
|
meter_id: int | None = None,
|
||||||
|
channel_id: int | None = None,
|
||||||
|
started_at: datetime | None = None,
|
||||||
|
ended_at: datetime | None | object = _UNSET,
|
||||||
|
) -> MeterSourceBinding:
|
||||||
|
"""Correct a binding while preserving half-open timeline constraints."""
|
||||||
|
binding = session.get(MeterSourceBinding, binding_id)
|
||||||
|
if binding is None:
|
||||||
|
raise BindingNotFoundError(f"Meter source binding {binding_id} was not found.")
|
||||||
|
new_meter_id = binding.meter_id if meter_id is None else meter_id
|
||||||
|
new_channel_id = binding.channel_id if channel_id is None else channel_id
|
||||||
|
new_started_at = binding.started_at if started_at is None else started_at
|
||||||
|
new_ended_at = binding.ended_at if ended_at is _UNSET else ended_at
|
||||||
|
now = _utc_now()
|
||||||
|
if _as_utc(new_started_at) > now or (new_ended_at is not None and _as_utc(new_ended_at) > now):
|
||||||
|
raise BindingValidationError("Binding boundaries must not be in the future.")
|
||||||
|
_validate_binding(
|
||||||
|
session,
|
||||||
|
meter_id=new_meter_id,
|
||||||
|
channel_id=new_channel_id,
|
||||||
|
started_at=new_started_at,
|
||||||
|
ended_at=new_ended_at,
|
||||||
|
excluding_ids={binding.id},
|
||||||
|
)
|
||||||
|
binding.meter_id = new_meter_id
|
||||||
|
binding.channel_id = new_channel_id
|
||||||
|
binding.started_at = new_started_at
|
||||||
|
binding.ended_at = new_ended_at
|
||||||
|
binding.updated_at = _utc_now()
|
||||||
|
return binding
|
||||||
|
|
||||||
|
|
||||||
|
def close_binding(session: Session, binding_id: int, *, ended_at: datetime) -> MeterSourceBinding:
|
||||||
|
"""Close an existing binding at its exclusive end boundary."""
|
||||||
|
return update_binding(session, binding_id, ended_at=ended_at)
|
||||||
|
|
||||||
|
|
||||||
|
def close_open_bindings_for_meter(session: Session, meter_id: int, *, ended_at: datetime) -> list[MeterSourceBinding]:
|
||||||
|
"""Close every open binding on a meter at one shared boundary."""
|
||||||
|
bindings = list(session.execute(
|
||||||
|
select(MeterSourceBinding).where(
|
||||||
|
MeterSourceBinding.meter_id == meter_id, MeterSourceBinding.ended_at.is_(None)
|
||||||
|
)
|
||||||
|
).scalars())
|
||||||
|
for binding in bindings:
|
||||||
|
update_binding(session, binding.id, ended_at=ended_at)
|
||||||
|
return bindings
|
||||||
|
|
||||||
|
|
||||||
|
def transfer_binding(
|
||||||
|
session: Session, *, target_meter_id: int, from_binding_id: int, to_channel_id: int,
|
||||||
|
effective_at: datetime,
|
||||||
|
) -> tuple[MeterSourceBinding, MeterSourceBinding]:
|
||||||
|
"""Atomically close a binding and open its replacement on the target meter."""
|
||||||
|
source = session.get(MeterSourceBinding, from_binding_id)
|
||||||
|
if source is None:
|
||||||
|
raise BindingNotFoundError(f"Meter source binding {from_binding_id} was not found.")
|
||||||
|
target = _get_meter(session, target_meter_id)
|
||||||
|
old_meter = _get_meter(session, source.meter_id)
|
||||||
|
effective_at = _as_utc(effective_at)
|
||||||
|
now = _utc_now()
|
||||||
|
if effective_at > now:
|
||||||
|
raise BindingValidationError("Binding transfer effective_at must not be in the future.")
|
||||||
|
if old_meter.commodity != target.commodity:
|
||||||
|
raise BindingValidationError("Binding transfer meters must have the same commodity.")
|
||||||
|
if source.ended_at is not None:
|
||||||
|
raise BindingValidationError("Only an open binding can be transferred.")
|
||||||
|
if old_meter.id == target.id:
|
||||||
|
close_at = effective_at
|
||||||
|
else:
|
||||||
|
# Recovery is deliberately narrow: the source meter must be the one
|
||||||
|
# and only most-recent closed predecessor in this commodity's timeline.
|
||||||
|
# A manually closed meter may leave an intentional epoch gap before the
|
||||||
|
# target is declared, so adjacency is not required.
|
||||||
|
if old_meter.ended_at is None:
|
||||||
|
raise BindingValidationError("Source binding must belong to a closed predecessor meter.")
|
||||||
|
timeline = list(session.execute(
|
||||||
|
select(Meter).where(Meter.commodity == target.commodity)
|
||||||
|
).scalars())
|
||||||
|
predecessors = [
|
||||||
|
meter for meter in timeline
|
||||||
|
if meter.id != target.id
|
||||||
|
and meter.ended_at is not None
|
||||||
|
and _as_utc(meter.ended_at) <= _as_utc(target.started_at)
|
||||||
|
]
|
||||||
|
if not predecessors:
|
||||||
|
raise BindingValidationError("Source meter is not the unique immediately preceding meter.")
|
||||||
|
latest_end = max(_as_utc(meter.ended_at) for meter in predecessors)
|
||||||
|
latest = [meter for meter in predecessors if _as_utc(meter.ended_at) == latest_end]
|
||||||
|
if len(latest) != 1 or latest[0].id != old_meter.id:
|
||||||
|
raise BindingValidationError("Source meter is not the unique immediately preceding meter.")
|
||||||
|
# Reject any overlapping epoch around either endpoint. A separate
|
||||||
|
# meter inside the gap is already excluded by the predecessor check;
|
||||||
|
# one extending into either endpoint is an ambiguous timeline too.
|
||||||
|
for meter in timeline:
|
||||||
|
if meter.id in {old_meter.id, target.id}:
|
||||||
|
continue
|
||||||
|
meter_end = _as_utc(meter.ended_at) if meter.ended_at is not None else None
|
||||||
|
if (
|
||||||
|
half_open_intervals_overlap(
|
||||||
|
_as_utc(old_meter.started_at), _as_utc(old_meter.ended_at),
|
||||||
|
_as_utc(meter.started_at), meter_end,
|
||||||
|
)
|
||||||
|
or half_open_intervals_overlap(
|
||||||
|
_as_utc(target.started_at),
|
||||||
|
_as_utc(target.ended_at) if target.ended_at is not None else None,
|
||||||
|
_as_utc(meter.started_at), meter_end,
|
||||||
|
)
|
||||||
|
):
|
||||||
|
raise BindingValidationError("Source meter has an ambiguous commodity timeline.")
|
||||||
|
close_at = _as_utc(old_meter.ended_at)
|
||||||
|
if effective_at < _as_utc(target.started_at):
|
||||||
|
raise BindingValidationError("Transfer effective_at must be within the target meter epoch.")
|
||||||
|
if effective_at < _as_utc(source.started_at):
|
||||||
|
raise BindingValidationError("Transfer effective_at precedes the source binding.")
|
||||||
|
# Validate the target before mutating the old row, then close/create in one session.
|
||||||
|
_validate_binding(session, meter_id=target.id, channel_id=to_channel_id,
|
||||||
|
started_at=effective_at, ended_at=None,
|
||||||
|
excluding_ids={source.id})
|
||||||
|
update_binding(session, source.id, ended_at=close_at)
|
||||||
|
created = create_binding(session, meter_id=target.id, channel_id=to_channel_id,
|
||||||
|
started_at=effective_at)
|
||||||
|
return source, created
|
||||||
+76
-2
@@ -53,6 +53,7 @@ from sqlalchemy import select
|
|||||||
from sqlalchemy.orm import Session
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
from app.models.energy import Meter
|
from app.models.energy import Meter
|
||||||
|
from app.models.meter_source import MeterSourceBinding
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
@@ -104,6 +105,20 @@ class MeterIntervalError(MeterError):
|
|||||||
"""
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
def close_meter(session: Session, meter: Meter, *, ended_at: datetime) -> Meter:
|
||||||
|
"""Close an active meter at a valid, non-future exclusive boundary."""
|
||||||
|
boundary = _as_utc(ended_at)
|
||||||
|
if meter.ended_at is not None:
|
||||||
|
raise MeterIntervalError("Only an active meter can be closed.")
|
||||||
|
if boundary <= _as_utc(meter.started_at):
|
||||||
|
raise MeterIntervalError("Meter ended_at must be strictly after started_at.")
|
||||||
|
if boundary > datetime.now(UTC):
|
||||||
|
raise MeterIntervalError("Meter ended_at must not be in the future.")
|
||||||
|
_validate_bindings_fit_meter_end(session, meter, boundary)
|
||||||
|
meter.ended_at = boundary
|
||||||
|
return meter
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# Internal query helpers
|
# Internal query helpers
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
@@ -121,6 +136,23 @@ def _active_meter(session: Session, commodity: str) -> Optional[Meter]:
|
|||||||
).scalar_one_or_none()
|
).scalar_one_or_none()
|
||||||
|
|
||||||
|
|
||||||
|
def _validate_bindings_fit_meter_end(session: Session, meter: Meter, boundary: datetime) -> None:
|
||||||
|
"""Reject an epoch close that would put any retained binding out of bounds.
|
||||||
|
|
||||||
|
Closed binding history is immutable here. Open bindings may subsequently
|
||||||
|
be closed by the caller at the shared meter boundary, but only when that
|
||||||
|
produces a non-empty interval.
|
||||||
|
"""
|
||||||
|
for binding in session.execute(
|
||||||
|
select(MeterSourceBinding).where(MeterSourceBinding.meter_id == meter.id)
|
||||||
|
).scalars():
|
||||||
|
if binding.ended_at is None:
|
||||||
|
if _as_utc(binding.started_at) >= boundary:
|
||||||
|
raise MeterIntervalError("Open binding cannot be closed within the proposed meter epoch.")
|
||||||
|
elif _as_utc(binding.ended_at) > boundary:
|
||||||
|
raise MeterIntervalError("Closed binding extends beyond the proposed meter epoch.")
|
||||||
|
|
||||||
|
|
||||||
def _meter_before(session: Session, meter: Meter) -> Optional[Meter]:
|
def _meter_before(session: Session, meter: Meter) -> Optional[Meter]:
|
||||||
"""Return the meter whose ``ended_at`` equals *meter*'s ``started_at``.
|
"""Return the meter whose ``ended_at`` equals *meter*'s ``started_at``.
|
||||||
|
|
||||||
@@ -296,6 +328,8 @@ def declare_meter(
|
|||||||
If *started_at* is strictly earlier than the current active meter's
|
If *started_at* is strictly earlier than the current active meter's
|
||||||
``started_at`` (chronological backdate below the active epoch's start).
|
``started_at`` (chronological backdate below the active epoch's start).
|
||||||
"""
|
"""
|
||||||
|
if _as_utc(started_at) > datetime.now(UTC):
|
||||||
|
raise MeterIntervalError("Meter started_at must not be in the future.")
|
||||||
active = _active_meter(session, commodity)
|
active = _active_meter(session, commodity)
|
||||||
|
|
||||||
if active is not None:
|
if active is not None:
|
||||||
@@ -308,6 +342,9 @@ def declare_meter(
|
|||||||
"Declare a started_at on or after the active meter's start to avoid "
|
"Declare a started_at on or after the active meter's start to avoid "
|
||||||
"a chronologically inconsistent epoch ordering."
|
"a chronologically inconsistent epoch ordering."
|
||||||
)
|
)
|
||||||
|
# Validate before changing the epoch: retained closed binding history
|
||||||
|
# must never be silently truncated by a later declaration.
|
||||||
|
_validate_bindings_fit_meter_end(session, active, _as_utc(started_at))
|
||||||
# Close the current active meter at the swap point (contiguous handoff).
|
# Close the current active meter at the swap point (contiguous handoff).
|
||||||
active.ended_at = started_at
|
active.ended_at = started_at
|
||||||
logger.info(
|
logger.info(
|
||||||
@@ -400,6 +437,9 @@ def update_meter(
|
|||||||
invert).
|
invert).
|
||||||
b. It must be **strictly before** this meter's ``ended_at`` (if set),
|
b. It must be **strictly before** this meter's ``ended_at`` (if set),
|
||||||
so this meter's epoch remains non-empty.
|
so this meter's epoch remains non-empty.
|
||||||
|
c. Every binding on this meter and its affected predecessor must remain
|
||||||
|
wholly inside its proposed epoch. The service rejects the correction
|
||||||
|
rather than rewriting binding history.
|
||||||
|
|
||||||
Note: triggering a billing recompute (``recompute_range``) after a
|
Note: triggering a billing recompute (``recompute_range``) after a
|
||||||
retroactive ``started_at`` change is **out of scope** for this service
|
retroactive ``started_at`` change is **out of scope** for this service
|
||||||
@@ -429,6 +469,14 @@ def update_meter(
|
|||||||
If the new ``started_at`` would produce an invalid (empty or inverted)
|
If the new ``started_at`` would produce an invalid (empty or inverted)
|
||||||
epoch for this meter or the immediately preceding one.
|
epoch for this meter or the immediately preceding one.
|
||||||
"""
|
"""
|
||||||
|
# Validate the proposed epoch boundary before touching *any* mutable
|
||||||
|
# field. PATCH accepts label/note together with started_at, so doing this
|
||||||
|
# first keeps an invalid future timestamp from leaking a partial in-session
|
||||||
|
# update before the API's rollback boundary is reached.
|
||||||
|
proposed_started_at = _as_utc(started_at) if started_at is not None else None
|
||||||
|
if proposed_started_at is not None and proposed_started_at > datetime.now(UTC):
|
||||||
|
raise MeterIntervalError("Meter started_at must not be in the future.")
|
||||||
|
|
||||||
if label is not None:
|
if label is not None:
|
||||||
meter.label = label
|
meter.label = label
|
||||||
logger.info("Updated meter id=%d label=%r", meter.id, label)
|
logger.info("Updated meter id=%d label=%r", meter.id, label)
|
||||||
@@ -439,10 +487,11 @@ def update_meter(
|
|||||||
|
|
||||||
if started_at is not None:
|
if started_at is not None:
|
||||||
old_started_at = meter.started_at
|
old_started_at = meter.started_at
|
||||||
|
assert proposed_started_at is not None
|
||||||
|
|
||||||
# --- Validate upper bound: new started_at must be < this meter's ended_at (if set).
|
# --- Validate upper bound: new started_at must be < this meter's ended_at (if set).
|
||||||
if meter.ended_at is not None:
|
if meter.ended_at is not None:
|
||||||
if _as_utc(started_at) >= _as_utc(meter.ended_at):
|
if proposed_started_at >= _as_utc(meter.ended_at):
|
||||||
raise MeterIntervalError(
|
raise MeterIntervalError(
|
||||||
f"New started_at ({started_at.isoformat()}) must be strictly before "
|
f"New started_at ({started_at.isoformat()}) must be strictly before "
|
||||||
f"this meter's ended_at ({meter.ended_at.isoformat()}). "
|
f"this meter's ended_at ({meter.ended_at.isoformat()}). "
|
||||||
@@ -454,12 +503,37 @@ def update_meter(
|
|||||||
|
|
||||||
# --- Validate lower bound: new started_at must be strictly after prev's started_at.
|
# --- Validate lower bound: new started_at must be strictly after prev's started_at.
|
||||||
if prev is not None:
|
if prev is not None:
|
||||||
if _as_utc(started_at) <= _as_utc(prev.started_at):
|
if proposed_started_at <= _as_utc(prev.started_at):
|
||||||
raise MeterIntervalError(
|
raise MeterIntervalError(
|
||||||
f"New started_at ({started_at.isoformat()}) must be strictly after "
|
f"New started_at ({started_at.isoformat()}) must be strictly after "
|
||||||
f"the previous meter's started_at ({prev.started_at.isoformat()}). "
|
f"the previous meter's started_at ({prev.started_at.isoformat()}). "
|
||||||
"Moving the boundary that far back would collapse the previous meter's epoch."
|
"Moving the boundary that far back would collapse the previous meter's epoch."
|
||||||
)
|
)
|
||||||
|
# A boundary correction changes both adjacent meter epochs. Fail closed
|
||||||
|
# rather than silently rewriting binding history: every existing binding
|
||||||
|
# must still fit in its proposed epoch before either Meter is mutated.
|
||||||
|
affected_meters = [
|
||||||
|
(meter, proposed_started_at, _as_utc(meter.ended_at) if meter.ended_at is not None else None)
|
||||||
|
]
|
||||||
|
if prev is not None:
|
||||||
|
affected_meters.append((prev, _as_utc(prev.started_at), proposed_started_at))
|
||||||
|
for affected_meter, proposed_start, proposed_end in affected_meters:
|
||||||
|
bindings = session.execute(
|
||||||
|
select(MeterSourceBinding).where(MeterSourceBinding.meter_id == affected_meter.id)
|
||||||
|
).scalars()
|
||||||
|
for binding in bindings:
|
||||||
|
if _as_utc(binding.started_at) < proposed_start:
|
||||||
|
raise MeterIntervalError(
|
||||||
|
f"Binding {binding.id} starts before meter {affected_meter.id}'s epoch."
|
||||||
|
)
|
||||||
|
if proposed_end is not None and (
|
||||||
|
binding.ended_at is None or _as_utc(binding.ended_at) > proposed_end
|
||||||
|
):
|
||||||
|
raise MeterIntervalError(
|
||||||
|
f"Binding {binding.id} would fall outside meter {affected_meter.id}'s epoch."
|
||||||
|
)
|
||||||
|
|
||||||
|
if prev is not None:
|
||||||
# Maintain continuity: update the previous meter's ended_at to match the new start.
|
# Maintain continuity: update the previous meter's ended_at to match the new start.
|
||||||
prev.ended_at = started_at
|
prev.ended_at = started_at
|
||||||
logger.info(
|
logger.info(
|
||||||
|
|||||||
@@ -30,6 +30,7 @@ Design decisions
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import logging
|
import logging
|
||||||
|
from threading import Lock, Thread
|
||||||
from datetime import UTC, datetime
|
from datetime import UTC, datetime
|
||||||
|
|
||||||
from sqlalchemy import select
|
from sqlalchemy import select
|
||||||
@@ -41,7 +42,10 @@ from app.models.energy import EnergyContract, TibberPrice
|
|||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
def _active_tibber_contract_exists(session: Session) -> bool:
|
_background_refresh_lock = Lock()
|
||||||
|
|
||||||
|
|
||||||
|
def active_tibber_contract_exists(session: Session) -> bool:
|
||||||
"""Return True if there is an active contract with kind='tibber'."""
|
"""Return True if there is an active contract with kind='tibber'."""
|
||||||
row = session.execute(
|
row = session.execute(
|
||||||
select(EnergyContract).where(
|
select(EnergyContract).where(
|
||||||
@@ -84,14 +88,17 @@ def refresh_prices(session: Session, settings: object) -> int:
|
|||||||
logger.debug("refresh_prices: tibber_api_token is empty — no-op")
|
logger.debug("refresh_prices: tibber_api_token is empty — no-op")
|
||||||
return 0
|
return 0
|
||||||
|
|
||||||
if not _active_tibber_contract_exists(session):
|
if not active_tibber_contract_exists(session):
|
||||||
logger.debug("refresh_prices: no active tibber contract — no-op")
|
logger.debug("refresh_prices: no active tibber contract — no-op")
|
||||||
return 0
|
return 0
|
||||||
|
|
||||||
home_id: str = getattr(settings, "tibber_home_id", "") or ""
|
home_id: str = getattr(settings, "tibber_home_id", "") or ""
|
||||||
home_id_or_none: str | None = home_id.strip() or None
|
home_id_or_none: str | None = home_id.strip() or None
|
||||||
|
|
||||||
logger.info("refresh_prices: fetching Tibber price range (home_id=%r)", home_id_or_none)
|
# Neither the API token nor the selected home identifier is safe to emit in
|
||||||
|
# diagnostics. The fetch client receives them, but logs only describe the
|
||||||
|
# operation itself.
|
||||||
|
logger.info("refresh_prices: fetching Tibber price range")
|
||||||
|
|
||||||
# May raise TibberError or TibberAuthError — let them propagate.
|
# May raise TibberError or TibberAuthError — let them propagate.
|
||||||
price_points = fetch_price_range(token, home_id_or_none)
|
price_points = fetch_price_range(token, home_id_or_none)
|
||||||
@@ -137,3 +144,59 @@ def refresh_prices(session: Session, settings: object) -> int:
|
|||||||
|
|
||||||
logger.info("refresh_prices: upserted %d price points", upserted)
|
logger.info("refresh_prices: upserted %d price points", upserted)
|
||||||
return upserted
|
return upserted
|
||||||
|
|
||||||
|
|
||||||
|
def run_tibber_refresh_best_effort() -> bool:
|
||||||
|
"""Run one refresh with an isolated session, skipping concurrent requests.
|
||||||
|
|
||||||
|
This is shared by the hourly scheduler and immediate post-commit triggers.
|
||||||
|
It intentionally catches all failures: refresh is advisory and must never
|
||||||
|
make app startup or a successfully committed configuration/contract update
|
||||||
|
appear to have failed. The boolean reports whether this invocation owned
|
||||||
|
the work; it is primarily useful for tests and diagnostics.
|
||||||
|
"""
|
||||||
|
if not _background_refresh_lock.acquire(blocking=False):
|
||||||
|
logger.debug("Tibber refresh already running; skipping duplicate request")
|
||||||
|
return False
|
||||||
|
|
||||||
|
session: Session | None = None
|
||||||
|
try:
|
||||||
|
# Local imports keep the pure refresh service free of app startup import
|
||||||
|
# cycles, while every background invocation gets a fresh DB session.
|
||||||
|
from app.config import get_settings
|
||||||
|
from app.db import get_session_local
|
||||||
|
from app.services.config_page import build_runtime_settings
|
||||||
|
|
||||||
|
session = get_session_local()()
|
||||||
|
refresh_prices(session, build_runtime_settings(session, get_settings()))
|
||||||
|
except Exception as exc:
|
||||||
|
# Exception text can contain remote request details. Keep diagnostics
|
||||||
|
# useful without allowing a token or home id to escape through logging.
|
||||||
|
logger.warning("Tibber price refresh failed (%s)", type(exc).__name__)
|
||||||
|
if session is not None:
|
||||||
|
try:
|
||||||
|
session.rollback()
|
||||||
|
except Exception:
|
||||||
|
logger.warning("Tibber price refresh rollback failed")
|
||||||
|
finally:
|
||||||
|
if session is not None:
|
||||||
|
try:
|
||||||
|
session.close()
|
||||||
|
except Exception:
|
||||||
|
logger.warning("Tibber price refresh session close failed")
|
||||||
|
_background_refresh_lock.release()
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
def trigger_tibber_refresh() -> None:
|
||||||
|
"""Request a non-blocking, best-effort Tibber refresh after a DB commit."""
|
||||||
|
try:
|
||||||
|
Thread(
|
||||||
|
target=run_tibber_refresh_best_effort,
|
||||||
|
name="tibber-price-refresh",
|
||||||
|
daemon=True,
|
||||||
|
).start()
|
||||||
|
except Exception as exc:
|
||||||
|
# Starting the optional worker must not turn an already committed API
|
||||||
|
# operation into a failure; avoid logging exception text for secrecy.
|
||||||
|
logger.warning("Unable to start Tibber price refresh (%s)", type(exc).__name__)
|
||||||
|
|||||||
@@ -19,8 +19,8 @@ Priority for resolving the local timezone
|
|||||||
-----------------------------------------
|
-----------------------------------------
|
||||||
1. ``TZ`` environment variable — ``ZoneInfo(os.environ["TZ"])``.
|
1. ``TZ`` environment variable — ``ZoneInfo(os.environ["TZ"])``.
|
||||||
Set ``TZ=Europe/Amsterdam`` in the deployment env for correct NL handling.
|
Set ``TZ=Europe/Amsterdam`` in the deployment env for correct NL handling.
|
||||||
2. System local timezone fallback: ``datetime.now().astimezone().tzinfo``.
|
2. The DST-aware ``Europe/Amsterdam`` business timezone. This keeps local-day
|
||||||
This matches the behaviour callers already relied on implicitly.
|
calculations deterministic when a deployment does not set ``TZ``.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
@@ -40,7 +40,7 @@ def local_tz() -> "tzinfo":
|
|||||||
Resolution order:
|
Resolution order:
|
||||||
1. ``TZ`` environment variable (``ZoneInfo(TZ)``). Set
|
1. ``TZ`` environment variable (``ZoneInfo(TZ)``). Set
|
||||||
``TZ=Europe/Amsterdam`` in production for correct NL/DST handling.
|
``TZ=Europe/Amsterdam`` in production for correct NL/DST handling.
|
||||||
2. System local timezone via ``datetime.now().astimezone().tzinfo``.
|
2. The DST-aware ``Europe/Amsterdam`` business timezone.
|
||||||
|
|
||||||
**Monkeypatch this function in tests** to get deterministic timezone
|
**Monkeypatch this function in tests** to get deterministic timezone
|
||||||
behaviour regardless of CI host configuration::
|
behaviour regardless of CI host configuration::
|
||||||
@@ -51,9 +51,7 @@ def local_tz() -> "tzinfo":
|
|||||||
tz_env = os.environ.get("TZ", "").strip()
|
tz_env = os.environ.get("TZ", "").strip()
|
||||||
if tz_env:
|
if tz_env:
|
||||||
return ZoneInfo(tz_env)
|
return ZoneInfo(tz_env)
|
||||||
# System fallback — identical to the .astimezone() pattern already used
|
return ZoneInfo("Europe/Amsterdam")
|
||||||
# in homeassistant_inbound.py and poo.py.
|
|
||||||
return datetime.now().astimezone().tzinfo # type: ignore[return-value]
|
|
||||||
|
|
||||||
|
|
||||||
def to_local(dt: datetime) -> datetime:
|
def to_local(dt: datetime) -> datetime:
|
||||||
|
|||||||
@@ -0,0 +1,401 @@
|
|||||||
|
"""Privacy-preserving WarmteLink frame admission and minute sampling.
|
||||||
|
|
||||||
|
The serial worker added later owns I/O. This module deliberately only accepts
|
||||||
|
already parsed :class:`P1Telegram` instances (or a parser callable at its
|
||||||
|
small convenience entry point), so rejected telegram bytes never enter the
|
||||||
|
database or an exception message.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from collections.abc import Callable
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from datetime import UTC, datetime, timedelta
|
||||||
|
from decimal import Decimal
|
||||||
|
import re
|
||||||
|
from zoneinfo import ZoneInfo
|
||||||
|
|
||||||
|
from sqlalchemy import select
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from app.db import get_session_local
|
||||||
|
from app.integrations.p1 import IntegrityStatus, P1Channel, P1Telegram, parse_telegram
|
||||||
|
from app.models.meter_source import MeterSource, WarmteLinkReading
|
||||||
|
from app.services.meter_sources import upsert_discovered_channel
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class _ChannelSample:
|
||||||
|
key: str
|
||||||
|
label: str
|
||||||
|
value: Decimal
|
||||||
|
unit: str
|
||||||
|
device_type: str | None
|
||||||
|
fingerprint: str | None
|
||||||
|
identity: tuple[object, ...]
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class _FrameSnapshot:
|
||||||
|
recorded_at: datetime
|
||||||
|
fingerprint: str | None
|
||||||
|
samples: tuple[_ChannelSample, ...]
|
||||||
|
|
||||||
|
|
||||||
|
class WarmteLinkIngestor:
|
||||||
|
"""Keep unverifiable candidates isolated by source for one worker lifetime."""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
clock: Callable[[], datetime] | None = None,
|
||||||
|
) -> None:
|
||||||
|
self._clock = clock or (lambda: datetime.now(UTC))
|
||||||
|
self._previous: dict[int, _FrameSnapshot] = {}
|
||||||
|
|
||||||
|
def ingest(
|
||||||
|
self,
|
||||||
|
session: Session,
|
||||||
|
*,
|
||||||
|
source_id: int,
|
||||||
|
telegram: P1Telegram,
|
||||||
|
received_at: datetime | None = None,
|
||||||
|
) -> bool:
|
||||||
|
"""Apply one parsed telegram in the caller's transaction.
|
||||||
|
|
||||||
|
Returns whether the frame was admitted. Callers that own a session
|
||||||
|
must commit on success; :func:`handle_frame` is the failure-contained
|
||||||
|
worker-facing entry point.
|
||||||
|
"""
|
||||||
|
source = session.get(MeterSource, source_id)
|
||||||
|
if source is None:
|
||||||
|
raise ValueError("WarmteLink source was not found")
|
||||||
|
if source.kind != "warmtelink_serial":
|
||||||
|
raise ValueError("Source is not a WarmteLink serial source")
|
||||||
|
|
||||||
|
received_at = _utc(self._clock()) if received_at is None else _utc(received_at)
|
||||||
|
try:
|
||||||
|
integrity = _integrity_status(telegram.integrity)
|
||||||
|
except Exception:
|
||||||
|
self._previous.pop(source_id, None)
|
||||||
|
self._diagnose(source, "WarmteLink frame could not be normalized", now=received_at)
|
||||||
|
return False
|
||||||
|
|
||||||
|
if integrity is IntegrityStatus.INVALID:
|
||||||
|
self._previous.pop(source_id, None)
|
||||||
|
self._diagnose(source, "WarmteLink frame checksum is invalid", now=received_at)
|
||||||
|
return False
|
||||||
|
|
||||||
|
try:
|
||||||
|
snapshot = _snapshot(telegram, received_at)
|
||||||
|
fingerprints = _final_fingerprints(snapshot)
|
||||||
|
except Exception:
|
||||||
|
# A parser DTO remains an untrusted boundary. Do not let an
|
||||||
|
# unnormalizable DTO bridge two otherwise matching candidates.
|
||||||
|
self._previous.pop(source_id, None)
|
||||||
|
self._diagnose(source, "WarmteLink frame could not be normalized", now=received_at)
|
||||||
|
return False
|
||||||
|
if fingerprints is None:
|
||||||
|
self._previous.pop(source_id, None)
|
||||||
|
self._diagnose(source, "WarmteLink frame fingerprint is invalid", now=received_at)
|
||||||
|
return False
|
||||||
|
if integrity is IntegrityStatus.UNVERIFIABLE:
|
||||||
|
previous = self._previous.get(source_id)
|
||||||
|
if previous is None:
|
||||||
|
self._previous[source_id] = snapshot
|
||||||
|
self._diagnose(
|
||||||
|
source, "Awaiting a second matching unverifiable WarmteLink frame", now=received_at
|
||||||
|
)
|
||||||
|
return False
|
||||||
|
reason = _continuity_problem(previous, snapshot)
|
||||||
|
if reason is not None:
|
||||||
|
self._previous[source_id] = snapshot
|
||||||
|
self._diagnose(source, reason, now=received_at)
|
||||||
|
return False
|
||||||
|
try:
|
||||||
|
self._admit(session, source, snapshot, integrity.value, fingerprints, received_at)
|
||||||
|
except Exception:
|
||||||
|
# ``ingest`` owns flushes and may be used directly by tests or
|
||||||
|
# future callers. A failed write must never become a speculative
|
||||||
|
# predecessor for this source.
|
||||||
|
self._previous.pop(source_id, None)
|
||||||
|
raise
|
||||||
|
if integrity is IntegrityStatus.UNVERIFIABLE:
|
||||||
|
# Advance the sliding predecessor only once every database write
|
||||||
|
# for this frame has succeeded. ``handle_frame`` also clears it
|
||||||
|
# if the caller's later commit fails.
|
||||||
|
self._previous[source_id] = snapshot
|
||||||
|
else:
|
||||||
|
# A verified frame has no need for a speculative predecessor.
|
||||||
|
self._previous.pop(source_id, None)
|
||||||
|
return True
|
||||||
|
|
||||||
|
def handle_frame(
|
||||||
|
self,
|
||||||
|
source_id: int,
|
||||||
|
frame: bytes,
|
||||||
|
*,
|
||||||
|
session_factory: Callable[[], Session] = get_session_local,
|
||||||
|
parser: Callable[[bytes], P1Telegram] = parse_telegram,
|
||||||
|
) -> bool:
|
||||||
|
"""Parse and persist one frame, containing both parse and DB failures.
|
||||||
|
|
||||||
|
A failed write is rolled back before a fresh transaction records only
|
||||||
|
a generic source error. Consequently no partial latest/history update
|
||||||
|
survives and the following frame may recover normally.
|
||||||
|
"""
|
||||||
|
received_at = _utc(self._clock())
|
||||||
|
try:
|
||||||
|
telegram = parser(frame)
|
||||||
|
except Exception:
|
||||||
|
self._previous.pop(source_id, None)
|
||||||
|
self._record_error(
|
||||||
|
session_factory, source_id, "WarmteLink frame could not be parsed", now=received_at
|
||||||
|
)
|
||||||
|
return False
|
||||||
|
try:
|
||||||
|
with session_factory() as session:
|
||||||
|
admitted = self.ingest(
|
||||||
|
session, source_id=source_id, telegram=telegram, received_at=received_at
|
||||||
|
)
|
||||||
|
session.commit()
|
||||||
|
return admitted
|
||||||
|
except Exception:
|
||||||
|
self._previous.pop(source_id, None)
|
||||||
|
self._record_error(session_factory, source_id, "WarmteLink ingest failed", now=received_at)
|
||||||
|
return False
|
||||||
|
|
||||||
|
def _admit(
|
||||||
|
self,
|
||||||
|
session: Session,
|
||||||
|
source: MeterSource,
|
||||||
|
snapshot: _FrameSnapshot,
|
||||||
|
quality: str,
|
||||||
|
fingerprints: tuple[str, ...],
|
||||||
|
received_at: datetime,
|
||||||
|
) -> None:
|
||||||
|
for sample, fingerprint in zip(snapshot.samples, fingerprints, strict=True):
|
||||||
|
channel = upsert_discovered_channel(
|
||||||
|
session,
|
||||||
|
source_id=source.id,
|
||||||
|
channel_key=sample.key,
|
||||||
|
label=sample.label,
|
||||||
|
unit=sample.unit,
|
||||||
|
suggested_commodity=_suggestion(sample.unit),
|
||||||
|
device_type=sample.device_type,
|
||||||
|
fingerprint=fingerprint,
|
||||||
|
latest_value=sample.value,
|
||||||
|
latest_at=snapshot.recorded_at,
|
||||||
|
latest_quality=quality,
|
||||||
|
)
|
||||||
|
session.flush()
|
||||||
|
bucket = snapshot.recorded_at.replace(second=0, microsecond=0)
|
||||||
|
exists = session.scalar(
|
||||||
|
select(WarmteLinkReading.id)
|
||||||
|
.where(
|
||||||
|
WarmteLinkReading.channel_id == channel.id,
|
||||||
|
WarmteLinkReading.recorded_at >= bucket,
|
||||||
|
WarmteLinkReading.recorded_at < bucket + timedelta(minutes=1),
|
||||||
|
)
|
||||||
|
.limit(1)
|
||||||
|
)
|
||||||
|
if exists is None:
|
||||||
|
session.add(
|
||||||
|
WarmteLinkReading(
|
||||||
|
channel_id=channel.id,
|
||||||
|
recorded_at=snapshot.recorded_at,
|
||||||
|
received_at=received_at,
|
||||||
|
value=sample.value,
|
||||||
|
unit=sample.unit,
|
||||||
|
quality=quality,
|
||||||
|
equipment_fingerprint=fingerprint,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
source.status = "online"
|
||||||
|
source.last_seen_at = received_at
|
||||||
|
source.last_error = None
|
||||||
|
source.updated_at = received_at
|
||||||
|
|
||||||
|
def _diagnose(self, source: MeterSource, reason: str, *, now: datetime | None = None) -> None:
|
||||||
|
now = _utc(self._clock()) if now is None else now
|
||||||
|
source.status = "error"
|
||||||
|
source.last_error = reason
|
||||||
|
source.updated_at = now
|
||||||
|
|
||||||
|
def _record_error(
|
||||||
|
self,
|
||||||
|
session_factory: Callable[[], Session],
|
||||||
|
source_id: int,
|
||||||
|
message: str,
|
||||||
|
*,
|
||||||
|
now: datetime,
|
||||||
|
) -> None:
|
||||||
|
try:
|
||||||
|
with session_factory() as session:
|
||||||
|
source = session.get(MeterSource, source_id)
|
||||||
|
if source is not None:
|
||||||
|
self._diagnose(source, message, now=now)
|
||||||
|
session.commit()
|
||||||
|
except Exception:
|
||||||
|
# Error reporting itself must not kill another source worker.
|
||||||
|
return
|
||||||
|
|
||||||
|
|
||||||
|
def _snapshot(telegram: P1Telegram, received_at: datetime) -> _FrameSnapshot:
|
||||||
|
recorded_at = _parse_timestamp(telegram.timestamp, received_at)
|
||||||
|
samples = tuple(_sample(channel) for channel in telegram.channels)
|
||||||
|
if not samples:
|
||||||
|
raise ValueError("WarmteLink telegram contains no cumulative channels")
|
||||||
|
return _FrameSnapshot(recorded_at, telegram.equipment_fingerprint, samples)
|
||||||
|
|
||||||
|
|
||||||
|
def _sample(channel: P1Channel) -> _ChannelSample:
|
||||||
|
profile = _canonical_channel_profile(channel.number)
|
||||||
|
if channel.device_type != profile.device_type:
|
||||||
|
raise ValueError("WarmteLink channel device type is not canonical")
|
||||||
|
if len(channel.readings) != 1:
|
||||||
|
raise ValueError("WarmteLink channel has no unambiguous cumulative reading")
|
||||||
|
reading = channel.readings[0]
|
||||||
|
if reading.code != profile.reading_code or reading.value is None or reading.unit != profile.raw_unit:
|
||||||
|
raise ValueError("WarmteLink cumulative reading is incomplete")
|
||||||
|
# Parser annotations are not a trust boundary: fake or future parser DTOs
|
||||||
|
# must not put a float (including NaN) or a non-finite Decimal into the
|
||||||
|
# per-source unverifiable candidate state. Do not coerce here: accepting
|
||||||
|
# another numeric type would make the persistence and continuity paths
|
||||||
|
# disagree about the cumulative-value contract.
|
||||||
|
if not isinstance(reading.value, Decimal) or not reading.value.is_finite():
|
||||||
|
raise ValueError("WarmteLink cumulative reading is not a finite Decimal")
|
||||||
|
fingerprint = channel.equipment_fingerprint
|
||||||
|
return _ChannelSample(
|
||||||
|
key=profile.key,
|
||||||
|
label=profile.label,
|
||||||
|
value=reading.value,
|
||||||
|
unit=profile.unit,
|
||||||
|
device_type=profile.device_type,
|
||||||
|
fingerprint=fingerprint,
|
||||||
|
identity=(channel.number, profile.device_type, fingerprint, profile.reading_code, profile.unit),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class _CanonicalChannelProfile:
|
||||||
|
key: str
|
||||||
|
label: str
|
||||||
|
device_type: str
|
||||||
|
reading_code: str
|
||||||
|
raw_unit: str
|
||||||
|
unit: str
|
||||||
|
|
||||||
|
|
||||||
|
_CANONICAL_CHANNELS = {
|
||||||
|
1: _CanonicalChannelProfile(
|
||||||
|
key="channel-1",
|
||||||
|
label="WarmteLink channel 1",
|
||||||
|
device_type="006",
|
||||||
|
reading_code="0-1:24.2.1",
|
||||||
|
raw_unit="m3",
|
||||||
|
unit="m³",
|
||||||
|
),
|
||||||
|
2: _CanonicalChannelProfile(
|
||||||
|
key="channel-2",
|
||||||
|
label="WarmteLink channel 2",
|
||||||
|
device_type="012",
|
||||||
|
reading_code="0-2:24.2.1",
|
||||||
|
raw_unit="GJ",
|
||||||
|
unit="GJ",
|
||||||
|
),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _canonical_channel_profile(number: object) -> _CanonicalChannelProfile:
|
||||||
|
# Do not format or coerce parser supplied channel numbers: that could turn
|
||||||
|
# an arbitrary object into an identity key before it is rejected.
|
||||||
|
if type(number) is not int:
|
||||||
|
raise ValueError("WarmteLink channel number is not canonical")
|
||||||
|
try:
|
||||||
|
return _CANONICAL_CHANNELS[number]
|
||||||
|
except KeyError as exc:
|
||||||
|
raise ValueError("WarmteLink channel is not supported by the profile") from exc
|
||||||
|
|
||||||
|
|
||||||
|
def _integrity_status(value: object) -> IntegrityStatus:
|
||||||
|
"""Require a real parser integrity enum, never a look-alike value object."""
|
||||||
|
if not isinstance(value, IntegrityStatus):
|
||||||
|
raise ValueError("WarmteLink integrity status is not canonical")
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def _continuity_problem(previous: _FrameSnapshot, current: _FrameSnapshot) -> str | None:
|
||||||
|
if current.recorded_at <= previous.recorded_at:
|
||||||
|
return "Unverifiable WarmteLink timestamp is not strictly increasing"
|
||||||
|
if current.recorded_at - previous.recorded_at != timedelta(seconds=10):
|
||||||
|
return "Unverifiable WarmteLink frame cadence is not 10 seconds"
|
||||||
|
if current.fingerprint != previous.fingerprint:
|
||||||
|
return "WarmteLink equipment metadata changed"
|
||||||
|
if tuple(sample.identity for sample in current.samples) != tuple(sample.identity for sample in previous.samples):
|
||||||
|
return "WarmteLink channel metadata changed or channel set changed"
|
||||||
|
old_values = {sample.key: sample.value for sample in previous.samples}
|
||||||
|
if any(sample.value < old_values[sample.key] for sample in current.samples):
|
||||||
|
return "WarmteLink cumulative value decreased"
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
_FINGERPRINT_PATTERN = re.compile(r"[0-9a-f]{64}")
|
||||||
|
|
||||||
|
|
||||||
|
def _final_fingerprints(snapshot: _FrameSnapshot) -> tuple[str, ...] | None:
|
||||||
|
"""Return only canonical SHA-256 hexdigests safe to persist.
|
||||||
|
|
||||||
|
A parser DTO is an untrusted boundary: even a field named ``fingerprint``
|
||||||
|
can contain a raw equipment identifier. Validate the complete DTO before
|
||||||
|
choosing persisted values: the top-level value and every channel value
|
||||||
|
must independently be canonical hashes. This deliberately does not use
|
||||||
|
a top-level fallback for an absent or malformed channel fingerprint.
|
||||||
|
"""
|
||||||
|
values = (snapshot.fingerprint, *(sample.fingerprint for sample in snapshot.samples))
|
||||||
|
if any(not _is_canonical_fingerprint(value) for value in values):
|
||||||
|
return None
|
||||||
|
return tuple(sample.fingerprint for sample in snapshot.samples if sample.fingerprint is not None)
|
||||||
|
|
||||||
|
|
||||||
|
def _is_canonical_fingerprint(value: str | None) -> bool:
|
||||||
|
return value is not None and _FINGERPRINT_PATTERN.fullmatch(value) is not None
|
||||||
|
|
||||||
|
|
||||||
|
_AMSTERDAM = ZoneInfo("Europe/Amsterdam")
|
||||||
|
_MAX_CLOCK_SKEW = timedelta(minutes=5)
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_timestamp(value: str | None, received_at: datetime) -> datetime:
|
||||||
|
if value is None or len(value) != 13 or value[-1] not in {"S", "W"} or not value[:-1].isdigit():
|
||||||
|
raise ValueError("WarmteLink timestamp is unavailable")
|
||||||
|
naive = datetime.strptime(value[:-1], "%y%m%d%H%M%S")
|
||||||
|
# The S/W marker is only advisory: deployed devices have emitted W while
|
||||||
|
# on CEST. Validate both folds by a UTC round trip, which rejects spring
|
||||||
|
# gaps and leaves one (ordinary) or two (fall-back) real instants.
|
||||||
|
candidates: list[datetime] = []
|
||||||
|
for fold in (0, 1):
|
||||||
|
candidate = naive.replace(tzinfo=_AMSTERDAM, fold=fold).astimezone(UTC)
|
||||||
|
local = candidate.astimezone(_AMSTERDAM)
|
||||||
|
if local.replace(tzinfo=None) == naive and local.fold == fold and candidate not in candidates:
|
||||||
|
candidates.append(candidate)
|
||||||
|
if not candidates:
|
||||||
|
raise ValueError("WarmteLink timestamp is unavailable")
|
||||||
|
received_at = _utc(received_at)
|
||||||
|
recorded_at = min(candidates, key=lambda candidate: abs(candidate - received_at))
|
||||||
|
if abs(recorded_at - received_at) > _MAX_CLOCK_SKEW:
|
||||||
|
raise ValueError("WarmteLink timestamp is unavailable")
|
||||||
|
return recorded_at
|
||||||
|
|
||||||
|
|
||||||
|
def _suggestion(unit: str) -> str | None:
|
||||||
|
return {"GJ": "heating", "m³": "hot_water"}.get(unit)
|
||||||
|
|
||||||
|
|
||||||
|
def _canonical_unit(unit: str) -> str:
|
||||||
|
"""Map the P1 spelling of cubic metres to the source-profile unit."""
|
||||||
|
return "m³" if unit == "m3" else unit
|
||||||
|
|
||||||
|
|
||||||
|
def _utc(value: datetime) -> datetime:
|
||||||
|
return value.replace(tzinfo=UTC) if value.tzinfo is None else value.astimezone(UTC)
|
||||||
@@ -0,0 +1,436 @@
|
|||||||
|
"""Read-only WarmteLink serial workers and their lifecycle manager.
|
||||||
|
|
||||||
|
The worker deliberately owns no long-lived SQLAlchemy session and never
|
||||||
|
retains telegram bytes after handing a complete frame to the ingestor.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from collections.abc import Callable
|
||||||
|
from contextlib import suppress
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from datetime import UTC, datetime, timedelta
|
||||||
|
import logging
|
||||||
|
import threading
|
||||||
|
from typing import Protocol
|
||||||
|
|
||||||
|
import serial
|
||||||
|
from sqlalchemy import select
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from app.db import get_session_local
|
||||||
|
from app.integrations.p1 import TelegramFramer
|
||||||
|
from app.models.meter_source import MeterSource
|
||||||
|
from app.services.warmtelink_ingest import WarmteLinkIngestor
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
_BACKOFF_SECONDS = (1, 2, 4, 8, 16, 32, 60)
|
||||||
|
_JOIN_TIMEOUT_SECONDS = 5
|
||||||
|
_DISCOVERY_LOCK_TIMEOUT_SECONDS = 0.05
|
||||||
|
_DISCOVERY_WAIT_SECONDS = 0.1
|
||||||
|
_DISCOVERY_TIMEOUT_SECONDS = 5
|
||||||
|
|
||||||
|
|
||||||
|
class ReadOnlySerial(Protocol):
|
||||||
|
def read(self, size: int = 1) -> bytes: ...
|
||||||
|
|
||||||
|
def close(self) -> None: ...
|
||||||
|
|
||||||
|
|
||||||
|
SerialFactory = Callable[[dict], ReadOnlySerial]
|
||||||
|
SessionFactory = Callable[[], Session]
|
||||||
|
|
||||||
|
|
||||||
|
def _default_session_factory() -> Session:
|
||||||
|
"""Resolve the cached sessionmaker at call time, then open one session."""
|
||||||
|
return get_session_local()()
|
||||||
|
|
||||||
|
|
||||||
|
class WorkerClock(Protocol):
|
||||||
|
"""Injectable interruptible clock, keeping retry tests deterministic."""
|
||||||
|
|
||||||
|
def wait(self, stop_event: threading.Event, seconds: float) -> bool: ...
|
||||||
|
|
||||||
|
|
||||||
|
class _EventClock:
|
||||||
|
def wait(self, stop_event: threading.Event, seconds: float) -> bool:
|
||||||
|
return stop_event.wait(seconds)
|
||||||
|
|
||||||
|
|
||||||
|
def open_warmtelink_serial(config: dict) -> ReadOnlySerial:
|
||||||
|
"""Open the fixed WarmteLink P1 profile; no write-capable API is exposed."""
|
||||||
|
return serial.Serial(
|
||||||
|
port=config["path"], baudrate=115200, bytesize=serial.SEVENBITS,
|
||||||
|
parity=serial.PARITY_NONE, stopbits=serial.STOPBITS_ONE, timeout=1,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class _WorkerConfig:
|
||||||
|
source_id: int
|
||||||
|
config: dict
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class DiscoveryRequest:
|
||||||
|
"""One source-scoped request, completed only by its serial owner."""
|
||||||
|
|
||||||
|
request_id: int
|
||||||
|
source_id: int
|
||||||
|
deadline: datetime
|
||||||
|
status: str = "pending"
|
||||||
|
detail: str | None = None
|
||||||
|
completed: threading.Event = field(default_factory=threading.Event)
|
||||||
|
|
||||||
|
|
||||||
|
class WarmteLinkWorker:
|
||||||
|
"""One interruptible, read-only serial loop for one meter source."""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self, source_id: int, config: dict, *, session_factory: SessionFactory = _default_session_factory,
|
||||||
|
serial_factory: SerialFactory = open_warmtelink_serial,
|
||||||
|
stop_event: threading.Event | None = None,
|
||||||
|
ingestor: WarmteLinkIngestor | None = None,
|
||||||
|
clock: WorkerClock | None = None,
|
||||||
|
) -> None:
|
||||||
|
self.source_id = source_id
|
||||||
|
self.config = dict(config)
|
||||||
|
self._session_factory = session_factory
|
||||||
|
self._serial_factory = serial_factory
|
||||||
|
self._stop_event = stop_event or threading.Event()
|
||||||
|
self._ingestor = ingestor or WarmteLinkIngestor()
|
||||||
|
self._clock = clock or _EventClock()
|
||||||
|
self._serial: ReadOnlySerial | None = None
|
||||||
|
self._serial_lock = threading.Lock()
|
||||||
|
self._discovery_lock = threading.Lock()
|
||||||
|
self._discoveries: list[DiscoveryRequest] = []
|
||||||
|
# Never inherit a daemon flag from a caller's background thread: a serial
|
||||||
|
# descriptor and its orderly shutdown must remain visible to the process.
|
||||||
|
self._thread = threading.Thread(
|
||||||
|
target=self._run, name=f"warmtelink-{source_id}", daemon=False
|
||||||
|
)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def thread(self) -> threading.Thread:
|
||||||
|
return self._thread
|
||||||
|
|
||||||
|
def start(self) -> None:
|
||||||
|
self._thread.start()
|
||||||
|
|
||||||
|
def stop(self) -> None:
|
||||||
|
self._stop_event.set()
|
||||||
|
self._close_serial()
|
||||||
|
|
||||||
|
def request_discovery(self, request: DiscoveryRequest) -> None:
|
||||||
|
"""Queue a read request; this worker remains the sole serial owner."""
|
||||||
|
with self._discovery_lock:
|
||||||
|
self._discoveries.append(request)
|
||||||
|
|
||||||
|
def _finish_discoveries(self, status: str, detail: str | None = None) -> None:
|
||||||
|
now = datetime.now(UTC)
|
||||||
|
with self._discovery_lock:
|
||||||
|
pending, self._discoveries = self._discoveries, []
|
||||||
|
for request in pending:
|
||||||
|
if request.completed.is_set():
|
||||||
|
continue
|
||||||
|
if request.deadline <= now and status == "completed":
|
||||||
|
request.status, request.detail = "error", "Discovery timed out."
|
||||||
|
else:
|
||||||
|
request.status, request.detail = status, detail
|
||||||
|
request.completed.set()
|
||||||
|
|
||||||
|
def _expire_discoveries(self) -> None:
|
||||||
|
now = datetime.now(UTC)
|
||||||
|
with self._discovery_lock:
|
||||||
|
expired = [request for request in self._discoveries if request.deadline <= now]
|
||||||
|
self._discoveries = [request for request in self._discoveries if request.deadline > now]
|
||||||
|
for request in expired:
|
||||||
|
if request.completed.is_set():
|
||||||
|
continue
|
||||||
|
request.status, request.detail = "error", "Discovery timed out."
|
||||||
|
request.completed.set()
|
||||||
|
|
||||||
|
def join(self, timeout: float = _JOIN_TIMEOUT_SECONDS) -> bool:
|
||||||
|
self._thread.join(timeout)
|
||||||
|
return not self._thread.is_alive()
|
||||||
|
|
||||||
|
def _close_serial(self) -> None:
|
||||||
|
with self._serial_lock:
|
||||||
|
device, self._serial = self._serial, None
|
||||||
|
if device is not None:
|
||||||
|
with suppress(Exception):
|
||||||
|
device.close()
|
||||||
|
|
||||||
|
def _record_error(self, message: str) -> None:
|
||||||
|
try:
|
||||||
|
with self._session_factory() as session:
|
||||||
|
source = session.get(MeterSource, self.source_id)
|
||||||
|
if source is not None:
|
||||||
|
source.status = "error"
|
||||||
|
source.last_error = message
|
||||||
|
source.updated_at = datetime.now(UTC)
|
||||||
|
session.commit()
|
||||||
|
except Exception:
|
||||||
|
# A source-status failure must not end another source's worker.
|
||||||
|
return
|
||||||
|
|
||||||
|
def _run(self) -> None:
|
||||||
|
backoff_index = 0
|
||||||
|
while not self._stop_event.is_set():
|
||||||
|
try:
|
||||||
|
device = self._serial_factory(self.config)
|
||||||
|
with self._serial_lock:
|
||||||
|
if self._stop_event.is_set():
|
||||||
|
with suppress(Exception):
|
||||||
|
device.close()
|
||||||
|
return
|
||||||
|
self._serial = device
|
||||||
|
# A disconnect makes any bytes buffered from the previous
|
||||||
|
# descriptor untrustworthy. In particular, never let a
|
||||||
|
# trailing partial telegram be completed by a newly opened
|
||||||
|
# device.
|
||||||
|
framer = TelegramFramer()
|
||||||
|
while not self._stop_event.is_set():
|
||||||
|
self._expire_discoveries()
|
||||||
|
chunk = device.read(1024)
|
||||||
|
if not chunk:
|
||||||
|
# ``timeout`` reads are normal, but still yield so a bad
|
||||||
|
# fake/device cannot turn an empty read into a busy spin.
|
||||||
|
self._clock.wait(self._stop_event, 0.05)
|
||||||
|
continue
|
||||||
|
frames = framer.feed(chunk)
|
||||||
|
for frame in frames:
|
||||||
|
if self._stop_event.is_set():
|
||||||
|
break
|
||||||
|
admitted = self._ingestor.handle_frame(
|
||||||
|
self.source_id, frame, session_factory=self._session_factory
|
||||||
|
)
|
||||||
|
if admitted:
|
||||||
|
self._finish_discoveries("completed")
|
||||||
|
# A complete frame proves transport recovery even if its
|
||||||
|
# contents are rejected by the privacy/admission layer.
|
||||||
|
backoff_index = 0
|
||||||
|
except Exception:
|
||||||
|
self._finish_discoveries("error", "WarmteLink discovery failed.")
|
||||||
|
self._record_error("WarmteLink serial connection failed")
|
||||||
|
delay = _BACKOFF_SECONDS[min(backoff_index, len(_BACKOFF_SECONDS) - 1)]
|
||||||
|
backoff_index += 1
|
||||||
|
self._clock.wait(self._stop_event, delay)
|
||||||
|
finally:
|
||||||
|
self._close_serial()
|
||||||
|
|
||||||
|
|
||||||
|
class WarmteLinkWorkerManager:
|
||||||
|
"""Reconcile enabled serial sources into exactly one worker each."""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self, *, session_factory: SessionFactory = _default_session_factory,
|
||||||
|
serial_factory: SerialFactory = open_warmtelink_serial,
|
||||||
|
worker_factory: Callable[..., WarmteLinkWorker] = WarmteLinkWorker,
|
||||||
|
) -> None:
|
||||||
|
self._session_factory = session_factory
|
||||||
|
self._serial_factory = serial_factory
|
||||||
|
self._worker_factory = worker_factory
|
||||||
|
self._workers: dict[int, tuple[_WorkerConfig, WarmteLinkWorker]] = {}
|
||||||
|
self._lock = threading.Lock()
|
||||||
|
self._reapers: set[int] = set()
|
||||||
|
self._next_discovery_id = 0
|
||||||
|
self._shutting_down = False
|
||||||
|
|
||||||
|
@property
|
||||||
|
def worker_count(self) -> int:
|
||||||
|
with self._lock:
|
||||||
|
return len(self._workers)
|
||||||
|
|
||||||
|
def reconcile(self) -> None:
|
||||||
|
# Reading desired state under the same lock which applies it prevents a
|
||||||
|
# delayed pre-commit snapshot from rolling a newer commit backwards.
|
||||||
|
with self._lock:
|
||||||
|
if self._shutting_down:
|
||||||
|
return
|
||||||
|
desired = self._read_desired()
|
||||||
|
self._reconcile_locked(desired)
|
||||||
|
|
||||||
|
def start(self) -> None:
|
||||||
|
"""Enable reconciliation for a newly entered application lifespan."""
|
||||||
|
with self._lock:
|
||||||
|
self._shutting_down = False
|
||||||
|
self.reconcile()
|
||||||
|
|
||||||
|
def request_discovery(self, source_id: int) -> DiscoveryRequest:
|
||||||
|
"""Ask the current source worker for one bounded read/discovery attempt.
|
||||||
|
|
||||||
|
This intentionally does not reconcile or open a descriptor. Lifecycle
|
||||||
|
convergence remains separate; a request can neither replace nor stop a
|
||||||
|
worker when an HTTP client times out or disconnects.
|
||||||
|
"""
|
||||||
|
now = datetime.now(UTC)
|
||||||
|
request = DiscoveryRequest(0, source_id, now)
|
||||||
|
if not self._lock.acquire(timeout=_DISCOVERY_LOCK_TIMEOUT_SECONDS):
|
||||||
|
request.status, request.detail = "error", "Discovery queue is busy."
|
||||||
|
request.completed.set()
|
||||||
|
return request
|
||||||
|
try:
|
||||||
|
self._next_discovery_id += 1
|
||||||
|
request.request_id = self._next_discovery_id
|
||||||
|
request.deadline = now + timedelta(seconds=_DISCOVERY_TIMEOUT_SECONDS)
|
||||||
|
if self._shutting_down:
|
||||||
|
request.status, request.detail = "error", "WarmteLink manager is stopped."
|
||||||
|
request.completed.set()
|
||||||
|
elif (entry := self._workers.get(source_id)) is None:
|
||||||
|
request.status, request.detail = "error", "WarmteLink worker is not running."
|
||||||
|
request.completed.set()
|
||||||
|
else:
|
||||||
|
entry[1].request_discovery(request)
|
||||||
|
timer = threading.Timer(_DISCOVERY_TIMEOUT_SECONDS, self._timeout_discovery, args=(request,))
|
||||||
|
timer.daemon = True
|
||||||
|
timer.start()
|
||||||
|
finally:
|
||||||
|
self._lock.release()
|
||||||
|
# A tiny bounded wait makes an immediately available frame observable,
|
||||||
|
# without turning an HTTP call into serial I/O or an unbounded wait.
|
||||||
|
request.completed.wait(_DISCOVERY_WAIT_SECONDS)
|
||||||
|
return request
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _timeout_discovery(request: DiscoveryRequest) -> None:
|
||||||
|
"""Resolve a stale HTTP request without touching its healthy worker."""
|
||||||
|
if not request.completed.is_set():
|
||||||
|
request.status, request.detail = "error", "Discovery timed out."
|
||||||
|
request.completed.set()
|
||||||
|
|
||||||
|
def _read_desired(self) -> dict[int, _WorkerConfig]:
|
||||||
|
with self._session_factory() as session:
|
||||||
|
return {
|
||||||
|
source.id: _WorkerConfig(source.id, dict(source.config))
|
||||||
|
for source in session.execute(
|
||||||
|
select(MeterSource).where(
|
||||||
|
MeterSource.kind == "warmtelink_serial", MeterSource.enabled.is_(True)
|
||||||
|
)
|
||||||
|
).scalars()
|
||||||
|
}
|
||||||
|
|
||||||
|
def _record_manager_error(self, source_id: int) -> None:
|
||||||
|
"""Best-effort, deliberately non-sensitive lifecycle failure status."""
|
||||||
|
try:
|
||||||
|
with self._session_factory() as session:
|
||||||
|
source = session.get(MeterSource, source_id)
|
||||||
|
if source is not None:
|
||||||
|
source.status = "error"
|
||||||
|
source.last_error = "WarmteLink worker failed"
|
||||||
|
source.updated_at = datetime.now(UTC)
|
||||||
|
session.commit()
|
||||||
|
except Exception:
|
||||||
|
return
|
||||||
|
|
||||||
|
def _reconcile_locked(self, desired: dict[int, _WorkerConfig]) -> None:
|
||||||
|
stale = [
|
||||||
|
source_id for source_id, (config, _) in self._workers.items()
|
||||||
|
if source_id not in desired or desired[source_id] != config
|
||||||
|
]
|
||||||
|
blocked: set[int] = set()
|
||||||
|
for source_id in stale:
|
||||||
|
_, worker = self._workers[source_id]
|
||||||
|
try:
|
||||||
|
worker.stop()
|
||||||
|
stopped = worker.join()
|
||||||
|
except Exception:
|
||||||
|
self._record_manager_error(source_id)
|
||||||
|
blocked.add(source_id)
|
||||||
|
continue
|
||||||
|
if stopped:
|
||||||
|
self._workers.pop(source_id, None)
|
||||||
|
else:
|
||||||
|
logger.error("WarmteLink worker did not stop for source %s", source_id)
|
||||||
|
blocked.add(source_id)
|
||||||
|
self._schedule_reaper_locked(source_id, worker)
|
||||||
|
for source_id, config in desired.items():
|
||||||
|
if source_id in self._workers or source_id in blocked:
|
||||||
|
continue
|
||||||
|
worker: WarmteLinkWorker | None = None
|
||||||
|
try:
|
||||||
|
worker = self._worker_factory(
|
||||||
|
source_id, config.config, session_factory=self._session_factory,
|
||||||
|
serial_factory=self._serial_factory,
|
||||||
|
)
|
||||||
|
self._workers[source_id] = (config, worker)
|
||||||
|
worker.start()
|
||||||
|
except Exception:
|
||||||
|
self._record_manager_error(source_id)
|
||||||
|
# A failed start normally has no thread. If an unusual worker
|
||||||
|
# did start before raising, keep it tracked until it is reaped.
|
||||||
|
if not self._worker_is_alive(worker):
|
||||||
|
self._workers.pop(source_id, None)
|
||||||
|
else:
|
||||||
|
self._schedule_reaper_locked(source_id, worker)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _worker_is_alive(worker: object | None) -> bool:
|
||||||
|
thread = getattr(worker, "thread", None)
|
||||||
|
return bool(thread is not None and thread.is_alive())
|
||||||
|
|
||||||
|
def _schedule_reaper_locked(self, source_id: int, worker: WarmteLinkWorker) -> None:
|
||||||
|
if source_id in self._reapers:
|
||||||
|
return
|
||||||
|
self._reapers.add(source_id)
|
||||||
|
threading.Thread(
|
||||||
|
target=self._reap_worker, args=(source_id, worker),
|
||||||
|
# This bookkeeping watcher must not turn a deliberately bounded
|
||||||
|
# application shutdown into an unbounded process wait. The actual
|
||||||
|
# serial worker itself is explicitly non-daemon.
|
||||||
|
name=f"warmtelink-reaper-{source_id}", daemon=True,
|
||||||
|
).start()
|
||||||
|
|
||||||
|
def _reap_worker(self, source_id: int, worker: WarmteLinkWorker) -> None:
|
||||||
|
"""Wait for one timed-out worker, then converge without another API call."""
|
||||||
|
try:
|
||||||
|
while True:
|
||||||
|
try:
|
||||||
|
if worker.join():
|
||||||
|
break
|
||||||
|
except Exception:
|
||||||
|
self._record_manager_error(source_id)
|
||||||
|
return
|
||||||
|
# A custom worker can report a bounded join timeout immediately;
|
||||||
|
# yield before asking again so its reaper cannot busy-spin.
|
||||||
|
threading.Event().wait(0.05)
|
||||||
|
with self._lock:
|
||||||
|
current = self._workers.get(source_id)
|
||||||
|
if current is not None and current[1] is worker:
|
||||||
|
self._workers.pop(source_id)
|
||||||
|
self._reapers.discard(source_id)
|
||||||
|
should_reconcile = not self._shutting_down
|
||||||
|
if should_reconcile:
|
||||||
|
self.reconcile()
|
||||||
|
finally:
|
||||||
|
with self._lock:
|
||||||
|
self._reapers.discard(source_id)
|
||||||
|
|
||||||
|
def shutdown(self) -> None:
|
||||||
|
with self._lock:
|
||||||
|
self._shutting_down = True
|
||||||
|
workers = list(self._workers.items())
|
||||||
|
for source_id, (_, worker) in workers:
|
||||||
|
try:
|
||||||
|
worker.stop()
|
||||||
|
except Exception:
|
||||||
|
self._record_manager_error(source_id)
|
||||||
|
for source_id, (_, worker) in workers:
|
||||||
|
try:
|
||||||
|
stopped = worker.join()
|
||||||
|
except Exception:
|
||||||
|
self._record_manager_error(source_id)
|
||||||
|
continue
|
||||||
|
if not stopped:
|
||||||
|
logger.error("WarmteLink worker did not stop during shutdown for source %s", source_id)
|
||||||
|
with self._lock:
|
||||||
|
self._schedule_reaper_locked(source_id, worker)
|
||||||
|
else:
|
||||||
|
with self._lock:
|
||||||
|
current = self._workers.get(source_id)
|
||||||
|
if current is not None and current[1] is worker:
|
||||||
|
self._workers.pop(source_id)
|
||||||
|
|
||||||
|
|
||||||
|
warmtelink_worker_manager = WarmteLinkWorkerManager()
|
||||||
@@ -88,6 +88,8 @@ pyproject-hooks==1.2.0
|
|||||||
# via
|
# via
|
||||||
# build
|
# build
|
||||||
# pip-tools
|
# pip-tools
|
||||||
|
pyserial==3.5
|
||||||
|
# via -r requirements.in
|
||||||
pytest==8.4.2
|
pytest==8.4.2
|
||||||
# via -r dev-requirements.in
|
# via -r dev-requirements.in
|
||||||
python-dotenv==1.2.2
|
python-dotenv==1.2.2
|
||||||
|
|||||||
@@ -26,3 +26,7 @@ services:
|
|||||||
- "127.0.0.1:8002:8000"
|
- "127.0.0.1:8002:8000"
|
||||||
environment:
|
environment:
|
||||||
APP_DATABASE_URL: "sqlite:////app/data/app.db"
|
APP_DATABASE_URL: "sqlite:////app/data/app.db"
|
||||||
|
devices: !override
|
||||||
|
- "${WARMTELINK_DEVICE_PATH:?Set a stable /dev/serial/by-id path}:/dev/warmtelink:rw"
|
||||||
|
group_add: !override
|
||||||
|
- "${WARMTELINK_SERIAL_GID:?Set the host serial device GID}"
|
||||||
|
|||||||
+8
-1
@@ -6,6 +6,8 @@ services:
|
|||||||
restart: "no"
|
restart: "no"
|
||||||
init: true
|
init: true
|
||||||
command: ["python", "-m", "scripts.run_migrations"]
|
command: ["python", "-m", "scripts.run_migrations"]
|
||||||
|
environment:
|
||||||
|
TZ: "${TZ:-Europe/Amsterdam}"
|
||||||
volumes:
|
volumes:
|
||||||
- ./data:/app/data
|
- ./data:/app/data
|
||||||
- ./.env:/app/.env:ro
|
- ./.env:/app/.env:ro
|
||||||
@@ -17,13 +19,18 @@ services:
|
|||||||
user: "1000:1000"
|
user: "1000:1000"
|
||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
init: true
|
init: true
|
||||||
|
environment:
|
||||||
|
TZ: "${TZ:-Europe/Amsterdam}"
|
||||||
depends_on:
|
depends_on:
|
||||||
migration:
|
migration:
|
||||||
condition: service_completed_successfully
|
condition: service_completed_successfully
|
||||||
ports:
|
ports:
|
||||||
- "127.0.0.1:8881:8000"
|
- "127.0.0.1:8881:8000"
|
||||||
|
devices:
|
||||||
|
- "${WARMTELINK_DEVICE_PATH:?Set a stable /dev/serial/by-id path}:/dev/warmtelink:rw"
|
||||||
|
group_add:
|
||||||
|
- "${WARMTELINK_SERIAL_GID:?Set the host serial device GID}"
|
||||||
volumes:
|
volumes:
|
||||||
- ./data:/app/data
|
- ./data:/app/data
|
||||||
- ./.env:/app/.env:ro
|
- ./.env:/app/.env:ro
|
||||||
- /etc/localtime:/etc/localtime:ro
|
- /etc/localtime:/etc/localtime:ro
|
||||||
|
|
||||||
|
|||||||
@@ -19,7 +19,7 @@
|
|||||||
|
|
||||||
- `main.py`
|
- `main.py`
|
||||||
- FastAPI app factory
|
- FastAPI app factory
|
||||||
- lifespan(APScheduler 启停、MQTT 客户端起停、连接后触发 HA Discovery 发布;M6 新增 `tibber-refresh` 抓价 job + `energy-cost` 1 分钟计费 tick job;M6 启用时注册 DSMR MQTT 订阅)
|
- lifespan(APScheduler 启停、MQTT 客户端起停、连接后触发 HA Discovery 发布;注册 DSMR MQTT source,并启动/关闭每个 enabled WarmteLink 业务只读 serial worker;`tibber-refresh`、electricity 与 thermal cost tick 均由 scheduler 驱动)
|
||||||
- 基础路由注册
|
- 基础路由注册
|
||||||
- `config.py`
|
- `config.py`
|
||||||
- 环境变量驱动的 settings(含 M5 新增的 MQTT/HA Discovery/Modbus 配置项;M6 新增 `dsmr_ingest_enabled`、`dsmr_mqtt_topic`、`dsmr_sample_interval_s`、`tibber_api_token`(secret)、`tibber_home_id`)
|
- 环境变量驱动的 settings(含 M5 新增的 MQTT/HA Discovery/Modbus 配置项;M6 新增 `dsmr_ingest_enabled`、`dsmr_mqtt_topic`、`dsmr_sample_interval_s`、`tibber_api_token`(secret)、`tibber_home_id`)
|
||||||
@@ -29,22 +29,22 @@
|
|||||||
- 通用依赖注入
|
- 通用依赖注入
|
||||||
- `api/`
|
- `api/`
|
||||||
- HTTP routes
|
- HTTP routes
|
||||||
- `api/routes/api/`:JSON API(`/api/*` 前缀),供 React SPA 调用:会话/鉴权、配置读写、数据查询、记录 CRUD、Modbus 设备 CRUD + readings + metrics + test(`/api/modbus/*`)、Expose 勾选 + 重发 discovery(`/api/expose`)、MQTT 测试连接(`/api/config/mqtt/test`)、M6 新增合同 CRUD + 版本(`/api/energy/contracts*`)、pricing profile 列表(`/api/energy/profiles`)、价格/费用/汇总/DSMR 最新/重算/Tibber 测试(`/api/energy/prices`、`/api/energy/costs`、`/api/energy/costs/summary`、`/api/energy/dsmr/latest`、`/api/energy/costs/recompute`、`/api/energy/tibber/test`)
|
- `api/routes/api/`:JSON API(`/api/*` 前缀),供 React SPA 调用:会话/鉴权、配置读写、记录 CRUD、Modbus、Expose 与 MQTT 测试;Energy 包含 source profile/source/channel/history/discover/binding/Meter API、scope-aware contracts/prices/costs,以及兼容的 DSMR latest API
|
||||||
- 裸 ingestion 端点:`GET /public-ip/check`、`POST /homeassistant/publish`、`POST /poo/record`、`GET /poo/latest`、TickTick OAuth 等
|
- 裸 ingestion 端点:`GET /public-ip/check`、`POST /homeassistant/publish`、`POST /poo/record`、`GET /poo/latest`、TickTick OAuth 等
|
||||||
- `models/`
|
- `models/`
|
||||||
- SQLAlchemy models
|
- SQLAlchemy models
|
||||||
- 所有模型(auth / config / public_ip / location / poo / modbus / expose / energy)共用同一个 `Base`,均落在单一 `app.db` 中
|
- 所有模型(auth / config / public_ip / location / poo / modbus / expose / energy / meter_source)共用同一个 `Base`,均落在单一 `app.db` 中
|
||||||
- M5 新增:`ModbusDevice`(设备部署层)、`ModbusReading`(通用遥测,JSON payload)、`ExposedEntityToggle`(HA 实体暴露开关)
|
- M5 新增:`ModbusDevice`(设备部署层)、`ModbusReading`(通用遥测,JSON payload)、`ExposedEntityToggle`(HA 实体暴露开关)
|
||||||
- M6 新增:`DsmrReading`(整帧 DSMR telegram,10s 降采样)、`EnergyContract`(合同头,含 active 标记)、`EnergyContractVersion`(版本/时段,values JSON,只增不改)、`TibberPrice`(15 分钟价缓存,不可变)、`EnergyCostPeriod`(每 15 分钟计量电费,快照价,不可变)
|
- Energy:`MeterSource` / `MeterSourceChannel` / `MeterSourceBinding` 将协议连接、稳定测量 channel 与 Meter epoch 分离;`DsmrReading`、`WarmteLinkReading` 分别保存 JSON 与 Decimal scalar 历史;electricity `EnergyCostPeriod` 绑定 source binding,thermal `MeterCostPeriod` 保存审计账本;合同按 electricity / thermal scope 共存
|
||||||
- `schemas/`
|
- `schemas/`
|
||||||
- Pydantic schemas(M5 新增 `modbus.py`、`expose.py`;M6 新增 `energy_contract.py`、`energy.py`)
|
- Pydantic schemas(包括 `modbus.py`、`expose.py`、`energy_contract.py`、`energy.py`、`meter_source.py`)
|
||||||
- `services/`
|
- `services/`
|
||||||
- 业务服务层
|
- 业务服务层
|
||||||
- 当前已迁入 config page 的 DB 持久化逻辑
|
- 当前已迁入 config page 的 DB 持久化逻辑
|
||||||
- 当前已迁入 public IPv4 检查、状态持久化与变化通知逻辑
|
- 当前已迁入 public IPv4 检查、状态持久化与变化通知逻辑
|
||||||
- 当前已迁入 SMTP 发信与测试发信逻辑
|
- 当前已迁入 SMTP 发信与测试发信逻辑
|
||||||
- M5 新增:`modbus_poll.py`(采集 service,逐设备 poll + 落库 + 推 MQTT state)、`ha_discovery.py`(构建 HA Discovery payload、发布 retained config、发布 state)
|
- M5 新增:`modbus_poll.py`(采集 service,逐设备 poll + 落库 + 推 MQTT state)、`ha_discovery.py`(构建 HA Discovery payload、发布 retained config、发布 state)
|
||||||
- M6 新增:`tibber_prices.py`(httpx GraphQL 抓 15 分钟价,upsert `tibber_price`,幂等;仅 active=tibber 且 token 存在时运行)、`dsmr_ingest.py`(MQTT handler,整帧 JSON blob + 10s 降采样落库,`source_id` 幂等)、`energy_cost.py`(计费引擎:每 15 分钟寄存器差 × strategy 出价 → `energy_cost_period` 不可变快照;汇总 Σnet + 固定费 − heffingskorting;重算显式 opt-in)
|
- Energy:`dsmr_ingest.py` 按 source 入库;`warmtelink_ingest.py` 接纳连续确认的业务只读 P1 scalar,`warmtelink_worker.py` 管理 interruptible serial reconnect(pyserial 的 POSIX `O_RDWR` 打开由非 root、非 privileged、无 `m` 的 Docker `rw` device rule 支持;worker 只 read/close,绝不 write);`energy_cost.py` 与 `meter_cost.py` 分别计算 electricity/thermal 账本,均拒绝跨 Meter/binding 相减
|
||||||
- `integrations/`
|
- `integrations/`
|
||||||
- 外部系统适配层
|
- 外部系统适配层
|
||||||
- Home Assistant outbound adapter(REST 通道,原有)
|
- Home Assistant outbound adapter(REST 通道,原有)
|
||||||
|
|||||||
@@ -9,6 +9,8 @@
|
|||||||
- [`m5-iot-energy.md`](./m5-iot-energy.md) — IoT 集成与能耗采集(Modbus/Energy + MQTT/HA Discovery + 前端侧边栏)
|
- [`m5-iot-energy.md`](./m5-iot-energy.md) — IoT 集成与能耗采集(Modbus/Energy + MQTT/HA Discovery + 前端侧边栏)
|
||||||
- [`m6-tibber-dynamic-energy.md`](./m6-tibber-dynamic-energy.md) — 通用电价层 + DSMR 实时电表接入 + 实时买卖电费计算 + HA Energy 反哺
|
- [`m6-tibber-dynamic-energy.md`](./m6-tibber-dynamic-energy.md) — 通用电价层 + DSMR 实时电表接入 + 实时买卖电费计算 + HA Energy 反哺
|
||||||
- [`m7-meter-epochs-archival.md`](./m7-meter-epochs-archival.md) — 电表生命周期 / 换表归档(Meter epochs)
|
- [`m7-meter-epochs-archival.md`](./m7-meter-epochs-archival.md) — 电表生命周期 / 换表归档(Meter epochs)
|
||||||
|
- [`pre-m8-warmtelink-p1-poc.md`](./pre-m8-warmtelink-p1-poc.md) — WarmteLink P1 真机概念验证(已完成;正式 CLI 长测与供暖变化均经物理表复核)
|
||||||
|
- [`m8-warmtelink-energy.md`](./m8-warmtelink-energy.md) — WarmteLink P1、多数据源 Meter 与热力计费(M8-T01~T20 自动化技术验收已完成;交付后用户人工 walkthrough 待验收)
|
||||||
|
|
||||||
本文件定义**所有任务共用的格式与协作规则**,各个里程碑文档不再重复这些约定。
|
本文件定义**所有任务共用的格式与协作规则**,各个里程碑文档不再重复这些约定。
|
||||||
|
|
||||||
|
|||||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,333 @@
|
|||||||
|
# Pre-M8 — WarmteLink P1 真机概念验证
|
||||||
|
|
||||||
|
> **状态:已完成。** 2026-08-22 已完成临时脚本真机 bring-up;随后仓库内正式 probe 以
|
||||||
|
> `115200 7N1` 连续运行 10 分钟,形成脱敏的可重复验收证据;最终人工走查开启供暖后又观察到
|
||||||
|
> 区域供暖累计量从 `0.017 GJ` 增至 `0.018 GJ`,并与物理表一致。Pre-M8 现已解除
|
||||||
|
> **M8 Planning** 的入口限制;M8 的架构和实现范围仍未锁定。
|
||||||
|
|
||||||
|
## 1. 目的
|
||||||
|
|
||||||
|
在进入 M8 正式设计和实现前,用新家的 Vattenfall WarmteLink 建立一条只读 P1 验证链,回答:
|
||||||
|
|
||||||
|
1. USB→P1 线、主机串口权限和 WarmteLink P1 端口能否稳定输出可解析数据。
|
||||||
|
2. 真机使用什么串口参数,telegram 的 framing、CRC、时间戳和 OBIS/M-Bus channel 有何特征。
|
||||||
|
3. 实际能够读取哪些累计量,以及单位、精度、更新频率和累计语义。
|
||||||
|
4. P1 值是否与热量表/生活热水表面板一致。
|
||||||
|
|
||||||
|
Pre-M8 是 M8 的证据门:它只交付真机事实和可重复 probe,不决定数据库结构、Meter 数据源
|
||||||
|
绑定、后台采集服务或前端布局。
|
||||||
|
|
||||||
|
## 2. 执行边界
|
||||||
|
|
||||||
|
本阶段只建立下面这条最短链路:
|
||||||
|
|
||||||
|
```text
|
||||||
|
WarmteLink P1 → USB serial → 原始 telegram → 完整性状态 → 字段解析 → 终端输出
|
||||||
|
```
|
||||||
|
|
||||||
|
明确不做:
|
||||||
|
|
||||||
|
- 不写入 `app.db`,不新增 Alembic migration。
|
||||||
|
- 不新增 FastAPI API、后台常驻 worker、配置页面或 Energy 前端。
|
||||||
|
- 不发布 MQTT / Home Assistant Discovery。
|
||||||
|
- 不修改现有 DSMR Reader MQTT、电费计算或 Meter 逻辑。
|
||||||
|
- 不在本阶段决定 WarmteLink 应落在哪个正式 Device/Source 模型中。
|
||||||
|
- 不写串口、不修改 FTDI EEPROM;probe 必须严格只读。
|
||||||
|
|
||||||
|
## 3. 2026-08-22 真机事实
|
||||||
|
|
||||||
|
### 3.1 USB、权限与线材
|
||||||
|
|
||||||
|
- 新 USB→P1 线是 FTDI FT232R,稳定路径形态为
|
||||||
|
`/dev/serial/by-id/usb-FTDI_FT232R_USB_UART_<redacted>-if00-port0`;正式配置不得依赖
|
||||||
|
`/dev/ttyUSB1`。
|
||||||
|
- 运行用户加入 `dialout` 后可以直接读取 tty,无需 root,也不应把容器作为硬件 bring-up
|
||||||
|
的中间层。
|
||||||
|
- 已用另一根已知正常的 DSMR 线在电表上读到连续、CRC 正确的 DSMR telegram,证明宿主机
|
||||||
|
串口读取方法本身可用。
|
||||||
|
- 曾临时清除新 FTDI 线 EEPROM 的 `INVERT_RXD` 做对照;输出发生变化但仍不可解析,随后已将
|
||||||
|
EEPROM 逐字节恢复为原厂镜像并回读校验。正式方案不得依赖 EEPROM 修改。
|
||||||
|
|
||||||
|
### 3.2 串口参数矩阵
|
||||||
|
|
||||||
|
真机不是按最初假设的 `115200 8N1` 得到可读正文;当前线材与 WarmteLink 的实测最佳组合为:
|
||||||
|
|
||||||
|
```text
|
||||||
|
115200 baud, 7 data bits, no parity, 1 stop bit(7N1)
|
||||||
|
```
|
||||||
|
|
||||||
|
| 参数 | 实测结果 |
|
||||||
|
| --- | --- |
|
||||||
|
| `115200 7N1` | 正文稳定可读,可枚举 9 个 OBIS 字段 |
|
||||||
|
| `115200 7N2` | 同样可读;没有理由增加停止位,正式默认仍用 `7N1` |
|
||||||
|
| `115200 8N1/8E1/8O1` | 乱码,无有效 OBIS/CRC |
|
||||||
|
| `115200 7E1/7O1` | 乱码,无有效 OBIS/CRC |
|
||||||
|
| `120000–3000000`,分别用 `8N1`、`7N1` | 无完整报文;`120000` 仅残留少量可辨识文本 |
|
||||||
|
| XON/XOFF 开/关 | 不改变 `7N1` 的字段解析结果 |
|
||||||
|
|
||||||
|
[DSMR 5.0.2 P1 Companion Standard](https://www.netbeheernederland.nl/sites/default/files/2024-02/dsmr_5.0.2_p1_companion_standard.pdf)
|
||||||
|
规定 `115200 8N1`。因此 `7N1` 是当前设备/线材组合的实测事实,不应被文档或代码包装成
|
||||||
|
标准 DSMR framing;CLI 必须允许显式覆盖数据位、校验位和停止位。
|
||||||
|
|
||||||
|
### 3.3 Framing 与 CRC 异常
|
||||||
|
|
||||||
|
- 临时 bring-up 的一次 8 帧盘点中每帧均为 256 字节;正式 10 分钟采样显示长度并不固定,
|
||||||
|
详见 §3.6。
|
||||||
|
- 正文结构稳定,版本、时间戳、两个 M-Bus channel、单位和值均可重复解析。
|
||||||
|
- 实测头部为 `)TU)2NWA-MYRSKY`,偶见 `)TU{2NWA-MYRSKY`;没有标准要求的 `/` 起始符。
|
||||||
|
- 帧尾存在 `!`,但其后的字符并非每帧都稳定为四位十六进制;即使恰好是四位,也无法从
|
||||||
|
缺失的 `/` 起点完成标准 DSMR CRC16 验证。
|
||||||
|
|
||||||
|
因此正式 probe 必须把完整性明确表示为 `valid`、`invalid` 或 `unverifiable`,保留原始字节并
|
||||||
|
输出原因。它可以在 `unverifiable` 状态下枚举字段用于 PoC,但绝不能把该帧报告为 CRC 已通过。
|
||||||
|
M8 若要持久化这些读数,必须在 Planning 中单独锁定异常帧的接纳、重复确认和告警策略。
|
||||||
|
|
||||||
|
### 3.4 实际字段清单
|
||||||
|
|
||||||
|
连续 8 帧只出现以下 9 个字段;除 capture timestamp 外,字段集合和值均稳定:
|
||||||
|
|
||||||
|
| OBIS | 实测结构/值 | 结论 |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `1-3:0.2.8` | `(50)` | DSMR P1 输出版本 5.0 |
|
||||||
|
| `0-0:1.0.0` | `(YYMMDDhhmmssX)` | telegram/capture timestamp,每 10 秒变化 |
|
||||||
|
| `0-0:96.1.1` | `<redacted>` | WarmteLink/gateway equipment identifier |
|
||||||
|
| `0-1:24.1.0` | `(006)` | channel 1,M-Bus device type `0x06`,生活热水 |
|
||||||
|
| `0-1:96.1.0` | `<redacted>` | channel 1 equipment identifier;样本制造商可解码为 `KAM` |
|
||||||
|
| `0-1:24.2.1` | `(<timestamp>)(5.900*m3)` | 生活热水累计体积,输出到 `0.001 m³` 小数位 |
|
||||||
|
| `0-2:24.1.0` | `(012)` | channel 2,M-Bus device type `0x0C`,热量表 |
|
||||||
|
| `0-2:96.1.0` | `<redacted>` | channel 2 equipment identifier;样本制造商可解码为 `KAM` |
|
||||||
|
| `0-2:24.2.1` | `(<timestamp>)(0.017*GJ)` | 区域供暖累计热量,输出到 `0.001 GJ` 小数位 |
|
||||||
|
|
||||||
|
人工面板在同一时间显示 `5.900 m³` 和 `0.017 GJ`,与 P1 值完全一致;用户已确认这两个累计量
|
||||||
|
足以作为后续 Home Assistant Energy 展示的数据基础。
|
||||||
|
|
||||||
|
没有发现瞬时流量、当前热功率、供水温度或回水温度字段。完整 OMS/M-Bus 模型允许这些可选
|
||||||
|
量,但 WarmteLink 当前 P1 telegram 没有导出它们;Pre-M8 和 M8 不得假设它们可用。参考
|
||||||
|
[OMS Specification Vol. 2 Annex A](https://oms-group.org/wp-content/uploads/2024/05/OMS-Spec_Vol2_AnnexA_F121.pdf)。
|
||||||
|
|
||||||
|
### 3.5 对 M8 已经成立的事实
|
||||||
|
|
||||||
|
- 一个 WarmteLink serial source 同时暴露两个独立累计 measurement channel。
|
||||||
|
- channel 1 是生活热水 `m³`;channel 2 是区域供暖 `GJ`,不能按论坛样例固定 channel。
|
||||||
|
- 两个累计量都与物理表一致,可进入 M8 的 Meter/source 映射讨论。
|
||||||
|
- 当前没有瞬时流量、温度或热功率;M8 只承诺累计量采集与展示。
|
||||||
|
- 串口 framing 和 CRC 异常尚未消失,必须作为正式 ingestion 的显式质量状态处理。
|
||||||
|
|
||||||
|
### 3.6 正式 CLI 10 分钟复验(脱敏)
|
||||||
|
|
||||||
|
2026-08-22,仓库内 `python -m scripts.p1_probe` 在真实
|
||||||
|
`/dev/serial/by-id/<redacted>` 上以 `115200 7N1` 运行 `--duration 600 --show-changes`,原始
|
||||||
|
bytes 仅写入 `/tmp`。CLI 正常以 exit code 0 结束;期间没有 I/O error、未处理异常或断连。
|
||||||
|
|
||||||
|
- 读取到 60 个完整 frame、15,362 raw bytes;最后 2 bytes 是下一帧的不完整残片,60 个完整帧
|
||||||
|
合计 15,360 bytes。
|
||||||
|
- 设备 timestamp 从 `18:16:10` 至 `18:26:00`,严格每 10 秒递进。CLI 处理的 59 个相邻间隔中,
|
||||||
|
53 个为 10.0 s、1 个为 9.0 s、3 个为 0.0 s、2 个为 20.0 s,均值 9.81 s;0/20 s 配对来自
|
||||||
|
serial chunk 的批量交付,不代表设备 cadence 改变。
|
||||||
|
- 完整 frame 长度分布为 237 bytes × 2、239 × 1、254 × 12、256 × 31、258 × 11、275 × 3,
|
||||||
|
平均 256 bytes、范围 237–275。不能再把临时样本的「固定 256 bytes」视为帧格式契约;变长与
|
||||||
|
非标准 footer/分块边界一致。
|
||||||
|
- 60/60 的完整性均为 `unverifiable`,因为 60/60 缺少标准 `/` 起始符。footer 长度为 4 × 41、
|
||||||
|
6 × 16、23 × 3;仅 16/60 的 footer 恰为四位十六进制,但仍不能在缺失 `/` 时完成标准 CRC16
|
||||||
|
验证。**没有任何 frame 被报告为 CRC valid。**
|
||||||
|
- 60/60 帧均解析到同一组 9 个 OBIS code:`1-3:0.2.8`、`0-0:1.0.0`、`0-0:96.1.1`、
|
||||||
|
`0-1:24.1.0`、`0-1:96.1.0`、`0-1:24.2.1`、`0-2:24.1.0`、`0-2:96.1.0`、`0-2:24.2.1`。
|
||||||
|
equipment identifier 已脱敏,不进入 Git。
|
||||||
|
- channel 1 的 device type 是 `006`,累计值稳定为 `5.900 m³`;channel 2 的 device type 是
|
||||||
|
`012`,累计值稳定为 `0.017 GJ`。同日人工表盘复核也显示 `5.900 m³` / `0.017 GJ`,正式 CLI
|
||||||
|
因而复现了该基线。
|
||||||
|
- 60 帧中未见瞬时流量、热功率、供水温度或回水温度字段。10 分钟内累计值未变化只能证明这段
|
||||||
|
时间的累计值稳定;它不能证明发生消费时的更新频率,也不能证明 reset 或 wrap 行为。
|
||||||
|
|
||||||
|
### 3.7 最终人工走查:供暖累计量变化
|
||||||
|
|
||||||
|
正式长测交付后,用户又在终端直接运行只读 probe 十几分钟并开启供暖。channel 2 的区域供暖
|
||||||
|
累计量在本次运行中从 `0.017 GJ` 增至 `0.018 GJ`,同一时刻物理热量表也显示 `0.018 GJ`;
|
||||||
|
因此可以确认当前 P1 输出会在实际供暖消费下更新累计量,且 `0.001 GJ` 的变化与物理表一致。
|
||||||
|
|
||||||
|
这次人工走查没有改变完整性结论:telegram 仍缺少标准 `/`,数值与物理表一致不能替代 CRC
|
||||||
|
验证。走查也没有覆盖 reset、wrap 或精确更新延迟;这些仍须由 M8 的接纳、重复确认和告警策略
|
||||||
|
处理,而不能从一次累计量递增外推。
|
||||||
|
|
||||||
|
## 4. 正式 probe 的预期操作方式
|
||||||
|
|
||||||
|
实现后,在 workspace virtual environment 中运行只读 probe:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
source .venv/bin/activate
|
||||||
|
python -m scripts.p1_probe \
|
||||||
|
--device /dev/serial/by-id/<usb-p1-device> \
|
||||||
|
--baudrate 115200 \
|
||||||
|
--bytesize 7 \
|
||||||
|
--parity N \
|
||||||
|
--stopbits 1 \
|
||||||
|
--duration 600 \
|
||||||
|
--show-changes \
|
||||||
|
--raw-output /tmp/warmtelink-p1-telegram.bin
|
||||||
|
```
|
||||||
|
|
||||||
|
参数名可在实现时小幅调整,但必须保留这些能力:
|
||||||
|
|
||||||
|
- 设备路径由用户显式传入,文档推荐 `/dev/serial/by-id/...`。
|
||||||
|
- 串口默认采用本机实测 `115200 7N1`,同时允许显式覆盖 framing。
|
||||||
|
- 连续读取多帧,输出 telegram cadence、帧长度和读取/重连错误。
|
||||||
|
- 同时显示完整性/CRC 状态、原始 OBIS 字段和解析后的 channel、设备类型、值与单位。
|
||||||
|
- parser 不依赖字段固定顺序,也不把 GJ/m³ 固定到 channel 1 或 2。
|
||||||
|
- `--show-changes` 可只显示发生变化的字段,原始捕获可写入用户指定的 `/tmp` 路径。
|
||||||
|
- 捕获文件默认按原始 bytes 保存;未经脱敏不得提交到 Git。
|
||||||
|
- permission denied 时给出 `dialout` 指引;不得建议以 root 常驻运行。
|
||||||
|
|
||||||
|
Home Assistant Community 的
|
||||||
|
[最小 WarmteLink Bash 读取方法](https://community.home-assistant.io/t/solved-dsmr-add-warmtelink-as-data-source/485255/2)
|
||||||
|
仍可作为快速可见性检查,但论坛样例的 header、channel 和 device type 与本机均不同,不能作为
|
||||||
|
parser 契约。是否引入 [`dsmr_parser`](https://github.com/ndokter/dsmr_parser) 也由 T01 的实测
|
||||||
|
兼容性决定;若它要求标准 `/...!CRC` framing,则应保留小型专用 parser,而不是绕过其校验。
|
||||||
|
|
||||||
|
## 5. 实现任务
|
||||||
|
|
||||||
|
### PRE-M8-T01 — 纯函数 telegram framing、CRC 与 OBIS parser
|
||||||
|
|
||||||
|
- **Status**: `done`
|
||||||
|
- **Depends**: `none`
|
||||||
|
- **Context**: 先把串口 I/O 与解析分开,用脱敏 fixture 固定标准 DSMR 帧和本机异常帧行为。
|
||||||
|
|
||||||
|
**Files**
|
||||||
|
|
||||||
|
- `create scripts/p1_probe.py`
|
||||||
|
- `create tests/fixtures/dsmr_p1_valid.txt`
|
||||||
|
- `create tests/fixtures/warmtelink_p1_7n1.txt`
|
||||||
|
- `create tests/test_p1_probe.py`
|
||||||
|
|
||||||
|
**Steps**
|
||||||
|
|
||||||
|
1. 实现不依赖串口的增量 framing、DSMR CRC16 和通用 OBIS 行解析函数。
|
||||||
|
2. 用数据结构表达原始 header/footer、完整性状态、timestamp、channel、device type、equipment id、
|
||||||
|
value 和 unit;数值使用 `Decimal`,不使用二进制浮点保存累计量。
|
||||||
|
3. 加入一份完全脱敏的本机结构 fixture,并另造一份 CRC 正确的标准 DSMR fixture。
|
||||||
|
4. 对字段重排、分块输入、缺失 `/`、非十六进制 footer、CRC mismatch、未知字段和两个 channel
|
||||||
|
写单元测试。
|
||||||
|
|
||||||
|
**Out of scope / 不要碰**
|
||||||
|
|
||||||
|
- 不打开真实 serial device,不增加依赖,不写数据库/API/MQTT。
|
||||||
|
- 不因本机正文可读而伪造 `/` header 或把 CRC 状态升级为 valid。
|
||||||
|
|
||||||
|
**Acceptance criteria**
|
||||||
|
|
||||||
|
- [x] 标准 fixture 的 frame boundary 与 CRC 可验证为 `valid`。
|
||||||
|
- [x] 脱敏本机 fixture 被标为 `unverifiable`,但能按字段而非位置解析 `m³` 和 `GJ` channel。
|
||||||
|
- [x] 任意 chunk boundary 和字段顺序不影响结果,未知字段原样保留。
|
||||||
|
- [x] 累计量以 `Decimal` + 原单位返回。
|
||||||
|
- [x] `pytest tests/test_p1_probe.py`、`pytest`、`ruff check .` 全绿。
|
||||||
|
|
||||||
|
**Reviewer checklist**
|
||||||
|
|
||||||
|
- CRC 覆盖范围必须严格从 `/` 到 `!`(包含二者),不得对缺失字节做猜测性修补。
|
||||||
|
- fixture 必须脱敏且保留足以复现 framing 异常的字节结构。
|
||||||
|
- parser 不得硬编码 channel 1=GJ 或 channel 2=m³。
|
||||||
|
|
||||||
|
### PRE-M8-T02 — 只读 serial probe CLI
|
||||||
|
|
||||||
|
- **Status**: `done`
|
||||||
|
- **Depends**: `PRE-M8-T01`
|
||||||
|
- **Context**: 在纯 parser 通过后增加最小 serial I/O,使真机验证可以从仓库稳定复现。
|
||||||
|
|
||||||
|
**Files**
|
||||||
|
|
||||||
|
- `modify requirements.in`
|
||||||
|
- `modify requirements.txt`
|
||||||
|
- `modify dev-requirements.txt`
|
||||||
|
- `modify scripts/p1_probe.py`
|
||||||
|
- `modify tests/test_p1_probe.py`
|
||||||
|
|
||||||
|
**Steps**
|
||||||
|
|
||||||
|
1. 增加受约束的 `pyserial` runtime 依赖并用仓库既有 pip-compile 流程同步生成 requirements。
|
||||||
|
2. 实现 `--device`、framing 参数、`--duration`、`--show-changes` 和 `--raw-output`。
|
||||||
|
3. 默认使用 `115200 7N1`;串口只读,禁止 write、EEPROM 或自动修改设备配置。
|
||||||
|
4. 输出每帧完整性状态、全部字段、值变化、cadence 和错误;SIGINT/超时后关闭串口并正常退出。
|
||||||
|
5. 用 fake serial/chunk stream 测试 CLI,不要求 CI 存在 `/dev/ttyUSB*`。
|
||||||
|
|
||||||
|
**Out of scope / 不要碰**
|
||||||
|
|
||||||
|
- 不做 daemon、自动重连 worker、Docker device mapping、数据库、API、MQTT 或 HA Discovery。
|
||||||
|
- 不内置本机 FTDI 序列号,不自动扫描或改写任意 USB 设备。
|
||||||
|
|
||||||
|
**Acceptance criteria**
|
||||||
|
|
||||||
|
- [x] CLI 可用 `/dev/serial/by-id/...` 读取,且所有 framing 参数都可显式覆盖。
|
||||||
|
- [x] 默认参数准确反映本机 `115200 7N1`,帮助文本说明它是实测值而非 DSMR 标准默认。
|
||||||
|
- [x] raw output 保留原始 bytes;终端清楚区分 `valid`、`invalid`、`unverifiable`。
|
||||||
|
- [x] permission/busy/disconnect 错误非零退出并给出可执行诊断,绝不建议常驻 root。
|
||||||
|
- [x] 依赖输入与两个生成 requirements 文件同步。
|
||||||
|
- [x] `pytest tests/test_p1_probe.py`、`pytest`、`ruff check .` 全绿。
|
||||||
|
|
||||||
|
**Reviewer checklist**
|
||||||
|
|
||||||
|
- 确认所有 serial write path 均不存在。
|
||||||
|
- 确认测试完全 mock 硬件、没有 CI timing flake,退出路径总会关闭文件描述符。
|
||||||
|
- 确认 requirements 是生成结果而非仅手改 lock file。
|
||||||
|
|
||||||
|
### PRE-M8-T03 — 正式真机验收与 M8 交接
|
||||||
|
|
||||||
|
- **Status**: `done`
|
||||||
|
- **Depends**: `PRE-M8-T02`
|
||||||
|
- **Context**: 用仓库内 probe 替代本轮临时脚本,形成可重复、脱敏且能支撑 M8 Planning 的证据。
|
||||||
|
|
||||||
|
**Files**
|
||||||
|
|
||||||
|
- `modify docs/design/pre-m8-warmtelink-p1-poc.md`
|
||||||
|
- `modify docs/design/m8-warmtelink-energy.md`
|
||||||
|
- `modify docs/design/README.md`
|
||||||
|
- `modify docs/roadmap.md`
|
||||||
|
|
||||||
|
**Steps**
|
||||||
|
|
||||||
|
1. 在真实 `/dev/serial/by-id/...` 上运行 probe 至少 10 分钟,并保留原始捕获在 `/tmp`。
|
||||||
|
2. 汇总帧数、cadence、长度、完整性状态、全部字段和读数变化;不得提交原始设备标识。
|
||||||
|
3. 再次与物理表对照 GJ 和 m³,并记录累计/重置行为中本次能证实和不能证实的部分。
|
||||||
|
4. 更新本节事实、通过条件和 M8 入口;只有证据齐全后才把 Pre-M8 标记为完成。
|
||||||
|
|
||||||
|
**Out of scope / 不要碰**
|
||||||
|
|
||||||
|
- 不为完成 checklist 而修补原始字节或放宽 CRC 结果。
|
||||||
|
- 不进入 M8 schema/API/worker/frontend 实现。
|
||||||
|
|
||||||
|
**Acceptance criteria**
|
||||||
|
|
||||||
|
- [x] 仓库内 probe 在真机连续运行至少 10 分钟,无未处理异常退出。
|
||||||
|
- [x] 脱敏摘要列出两个累计 channel、单位、精度、cadence 和完整性异常。
|
||||||
|
- [x] `0.017 GJ`、`5.900 m³` 的基线或运行时新值与物理表再次对照。
|
||||||
|
- [x] 明确没有从当前 P1 输出读取到瞬时流量、功率或温度。
|
||||||
|
- [x] Pre-M8 状态与 roadmap/M8 入口同步;代码闸门保持全绿。
|
||||||
|
|
||||||
|
**Reviewer checklist**
|
||||||
|
|
||||||
|
- 证据必须来自正式 CLI,不得只复述本轮临时脚本结果。
|
||||||
|
- 任何 equipment id、FTDI serial 和未脱敏 raw capture 都不得进入 Git。
|
||||||
|
- CRC/framing 风险必须原样交给 M8,不能用“数值看起来正确”替代完整性判断。
|
||||||
|
|
||||||
|
## 6. 当前通过条件
|
||||||
|
|
||||||
|
- [x] USB、tty 权限和稳定 `/dev/serial/by-id/...` 路径已验证。
|
||||||
|
- [x] 串口参数矩阵已完成,实测正文可读参数为 `115200 7N1`。
|
||||||
|
- [x] 已枚举全部实际 channel/OBIS 字段,并确认没有轮换出现的额外测量量。
|
||||||
|
- [x] GJ 和生活热水 m³ 均与物理表面板完全一致。
|
||||||
|
- [x] 已记录精度、约 10 秒 telegram cadence 和当前字段集合。
|
||||||
|
- [x] framing/CRC 失败已保留为显式异常,没有误报为校验通过。
|
||||||
|
- [x] 仓库内 parser、fixtures、probe CLI 和自动化测试完成。
|
||||||
|
- [x] 正式 probe 完成至少 10 分钟真机复验并产出脱敏摘要。
|
||||||
|
- [x] 最终人工走查在供暖开启后观察到 `0.017 → 0.018 GJ`,并再次与物理表核对一致。
|
||||||
|
|
||||||
|
Pre-M8 已完成并向 M8 解锁 Planning;它只交付下列真机事实,不锁定正式架构或实现任务。
|
||||||
|
|
||||||
|
## 7. 向 M8 的交付物
|
||||||
|
|
||||||
|
Pre-M8 完成后只向 M8 交付事实,不交付正式架构:
|
||||||
|
|
||||||
|
- 脱敏 telegram 结构、fixture 和全部字段清单。
|
||||||
|
- GJ 与 m³ 的实际 channel、device type、单位、精度和更新时间。
|
||||||
|
- 串口参数、稳定设备路径形态与 `dialout` 权限要求。
|
||||||
|
- parser 适配结论和 `unverifiable` framing/CRC 异常样本。
|
||||||
|
- “一个 serial source 包含两个独立累计计量 channel”的实测结论。
|
||||||
|
- 当前 P1 不提供瞬时流量、热功率或温度的明确边界。
|
||||||
@@ -49,3 +49,16 @@
|
|||||||
- 更复杂的 backoff 策略
|
- 更复杂的 backoff 策略
|
||||||
|
|
||||||
这一轮重点是先把 app -> Home Assistant 的出站契约和可复用结构迁进来。
|
这一轮重点是先把 app -> Home Assistant 的出站契约和可复用结构迁进来。
|
||||||
|
|
||||||
|
## Energy、Source 与热力实体
|
||||||
|
|
||||||
|
Expose 框架还可以把已勾选的 Energy 实体通过 MQTT Home Assistant Discovery 发布;开关位于应用 Config 页的 HA Expose 面板,默认均为关闭。M8 增加了 source online、按 Meter UUID 锚定的累计量/today,以及 heating、hot-water-heating、water、water-tax、fixed、all-in total/today 等 thermal 实体。
|
||||||
|
|
||||||
|
- source 和 Meter identity 不依赖可变 label;换表会产生新 Meter UUID identity。
|
||||||
|
- thermal 组合成本 identity 由当前 heating/hot_water Meter UUID 的有序组合锚定,任一换表都会产生新 identity,避免不同累计域拼接。`Thermal Hot Water Total/Today` 精确为 `hot_water_heating + hot_water`,不包含 `hot_water_tax`。
|
||||||
|
- 每个 HA device 只发布一个完整、`home-automation:` namespaced identifier;内部 topic/unique-id seed 与该 identifier 分离。所有 discovery node/object segment 都会转为 HA 允许的 `[A-Za-z0-9_-]+` 字符集。
|
||||||
|
- v1.6.1 的 dot-containing thermal retained topics 会在启动、UI 尚未能修改 toggle 前,按当时实际重叠的 Meter epoch、enabled toggle 和 runtime discovery prefix 冻结为精确清单;每个成功 topic 都会持久记账,失败/未尝试项才会重试。冻结清单完成后跨重启不再枚举或发布旧 topic;fresh install 的空清单也会立即完成。没有 wildcard,也不会清理任何应用数据。旧 device 合并修复则以已配置的 Home Assistant WebSocket entity/device registry 实际观察 unload、重建和正确 device identifier;HA 不可达时普通 discovery 继续发布,repair 保持 pending。
|
||||||
|
- availability、unit、device/state class 与 today reset 由 provider 声明;operator 应在 HA 中核对,而不应假设同名实体可跨换表连续。
|
||||||
|
- 关闭 toggle 后 retained discovery 会被清理;关闭暴露不删除 source、Meter、合同、读数或成本历史。
|
||||||
|
|
||||||
|
WarmteLink 的 P1 质量会原样保留为 `unverifiable`(若适用),不因发布到 HA 而提升为 `valid`。不要将 raw telegram、equipment id、串口路径、合同金额或 API secret 作为 HA entity/state/attribute 发布。
|
||||||
|
|||||||
+54
-1
@@ -104,9 +104,62 @@ MQTT 上报的累计成本实体(`import_cost_total` / `export_revenue_total`
|
|||||||
- 若是全新空库,初始表不创建(无历史数据)。
|
- 若是全新空库,初始表不创建(无历史数据)。
|
||||||
- 回填幂等:重复跑迁移不会创建多条初始表;回填后对账(非降级周期 `meter_id IS NULL` 数必须为 0)。
|
- 回填幂等:重复跑迁移不会创建多条初始表;回填后对账(非降级周期 `meter_id IS NULL` 数必须为 0)。
|
||||||
|
|
||||||
|
## M8:Source binding 与热力 Meter
|
||||||
|
|
||||||
|
M8 将“协议连接”和物理 Meter 分开:`MeterSource` 产生稳定 channel,`MeterSourceBinding` 在 `[started_at, ended_at)` 内把 channel 接到一个 Meter epoch。DSMR electricity 与 WarmteLink heating `GJ`、hot_water `m³` 都使用这条链。
|
||||||
|
|
||||||
|
- 正常成本周期的两端必须解析到**同一** Meter 和 binding;跨 epoch、跨 binding、无 binding、读数陈旧或质量不可接纳时一律 degraded,不跨累计域相减。
|
||||||
|
- source switch 是关闭/新建 binding,不创建假 Meter swap;实际换表才创建新的 Meter epoch。对于 thermal,heating 与 hot_water 独立换表,热力组合 HA identity 因任一 UUID 改变而更新。
|
||||||
|
- `meter_cost_period` 为 heating 与 hot_water 保存 15 分钟 Decimal quantity/cost、binding、合同版本、price snapshot 和 degraded reason。固定费是合同级日汇总,只计一次。
|
||||||
|
|
||||||
|
部署与回滚串口 source 参见 [`warmtelink-energy.md`](./warmtelink-energy.md);保留旧 source/binding/history 可使审计和重算可重复,不能通过删除历史来“修复”边界周期。
|
||||||
|
|
||||||
|
## 生命周期操作与 stranded binding 恢复
|
||||||
|
|
||||||
|
M8-R08/R09 为 Meter 与 binding 增加了显式的生命周期操作。所有时间都由前端按本地日期时间输入,再按既有
|
||||||
|
Principle-A 约定交给后端;未来时间会被拒绝。
|
||||||
|
|
||||||
|
- **Close Meter**:只能关闭 active Meter。该 Meter 与它的所有 open binding 在同一个 `ended_at`、同一
|
||||||
|
事务中关闭;关闭后该 commodity 没有 active Meter。
|
||||||
|
- **Unbind**:可对任意 open binding 执行,只写 binding 的 `ended_at`,绝不删除历史。已关闭 Meter 上残留的
|
||||||
|
open binding 也可在 UI 中解绑,默认关闭时间为该 Meter 的 `ended_at`。
|
||||||
|
- **Transfer**:以单个原子请求切换 source channel,不采用浏览器端“先关闭再新建”的两步操作。同一 Meter
|
||||||
|
的 source switch 在同一 `effective_at` 关闭旧 binding、开启新 binding。失败时 binding、Meter 与受影响的
|
||||||
|
成本重算全部回滚,不会显示或留下部分成功。
|
||||||
|
|
||||||
|
新建或更新 binding 必须完整落在所属 Meter epoch 内:
|
||||||
|
`meter.started_at <= binding.started_at < binding.ended_at <= meter.ended_at`(Meter 已关闭时);
|
||||||
|
open-ended binding 只允许属于 active Meter。Close、Unbind、Transfer 与声明 Meter 都从最早受影响边界重算到
|
||||||
|
当前时间;electricity 与 heating/hot-water 分别使用对应的成本引擎,重算失败时整笔生命周期变更回滚。数据库
|
||||||
|
提交成功后才会 best-effort 重新发布 HA discovery;发布失败不会伪装成持久化失败。
|
||||||
|
|
||||||
|
### 换表自动交接与人工恢复
|
||||||
|
|
||||||
|
声明 `reason=meter_swap` 的新 Meter 时,若未选择 channel 且旧 active Meter 恰好有一条唯一、单位兼容的
|
||||||
|
open binding,系统会在新 Meter 起点自动把该 channel 原子交接过去。存在多个候选或时间线歧义时,声明会
|
||||||
|
fail closed 并整体回滚。不是这种唯一自动交接的声明,也会在新边界关闭旧 Meter 的 open binding,避免产生新的
|
||||||
|
“closed Meter + open binding”。
|
||||||
|
|
||||||
|
旧版本或历史异常可能已经留下 stranded binding:前一块 closed Meter 仍有 open DSMR/WarmteLink binding,
|
||||||
|
而当前 active Meter 没有 binding。不要修改数据库、不要跑 migration,也不会在启动时自动修复。请在
|
||||||
|
**Energy → Meters** 使用该 stranded binding 的 **Recover binding** 操作:选择当前 active Meter 与兼容
|
||||||
|
channel,提交一次 Transfer。来源必须是同 commodity、唯一且紧邻的前一块 closed Meter;旧 binding 固定在旧
|
||||||
|
Meter 的 `ended_at` 结束,新 binding 从选择的 `effective_at` 开始。默认是新 Meter 的 `started_at`;选择更晚
|
||||||
|
时间是允许的,但 UI 会提示这段明确的 unbound gap。非前序 Meter、单位不匹配、channel 在无关区间被占用或歧义
|
||||||
|
都会被拒绝,不会误关历史。
|
||||||
|
|
||||||
|
人工 walkthrough 只能使用开发库里**已经存在且由用户报告的** stranded row。不得为了演示通过 SQL、API、
|
||||||
|
脚本、migration 或直接改数据库制造这种历史异常;隔离开发库中没有该 row 时,记录此项为 `N/A/blocked` 即可。
|
||||||
|
Close、Unbind、same-Meter Transfer、Recover binding 与 meter-swap handoff 都会改变 Meter 或 binding 状态,
|
||||||
|
因此应作为彼此独立的验收场景:每个场景使用各自满足前置条件的 Meter/binding,或在执行前恢复独立前置状态,
|
||||||
|
不能把它们串成会互相破坏前提的单一故事。
|
||||||
|
|
||||||
|
上述恢复不新增 Alembic migration、不执行启动修复,也不删除 Meter、binding、reading 或成本历史;它只把用户
|
||||||
|
确认的时间线修正为可审计的闭区间。
|
||||||
|
|
||||||
## 非目标(本里程碑不做)
|
## 非目标(本里程碑不做)
|
||||||
|
|
||||||
- Gas / 区域供暖的计费(`commodity != "electricity"` 的 strategy)。
|
- Gas 计费 strategy。
|
||||||
- "家庭(home)"分组实体;多合同时间线积分。
|
- "家庭(home)"分组实体;多合同时间线积分。
|
||||||
- 自动识别换表(DSMR 帧无电表序列号,无法自动识别,靠用户显式声明)。
|
- 自动识别换表(DSMR 帧无电表序列号,无法自动识别,靠用户显式声明)。
|
||||||
- `last_reset` 信号(消除 HA 长期统计 blip)。
|
- `last_reset` 信号(消除 HA 长期统计 blip)。
|
||||||
|
|||||||
+69
-1
@@ -2,7 +2,7 @@
|
|||||||
|
|
||||||
本文档记录 `home-automation` 在 `v1.0.3` 之后的下一阶段规划。这一阶段不是小修补,而是几次较大的结构性改动:单库化、前端重写、以及远期的移动端试水。
|
本文档记录 `home-automation` 在 `v1.0.3` 之后的下一阶段规划。这一阶段不是小修补,而是几次较大的结构性改动:单库化、前端重写、以及远期的移动端试水。
|
||||||
|
|
||||||
> 每个里程碑的**可执行原子任务**展开在 [`docs/design/`](./design/README.md):M1 [`m1-db-consolidation.md`](./design/m1-db-consolidation.md)、M2 [`m2-frontend-v2.md`](./design/m2-frontend-v2.md)、M3 [`m3-token-mobile.md`](./design/m3-token-mobile.md)、M4 [`m4-login-hardening.md`](./design/m4-login-hardening.md)、M5 [`m5-iot-energy.md`](./design/m5-iot-energy.md)、M6 [`m6-tibber-dynamic-energy.md`](./design/m6-tibber-dynamic-energy.md)、M7 [`m7-meter-epochs-archival.md`](./design/m7-meter-epochs-archival.md)。这些文档为 Orchestrator→Implementer→Reviewer 的多模型流水线设计。
|
> 每个里程碑的设计与**可执行原子任务**展开在 [`docs/design/`](./design/README.md):M1 [`m1-db-consolidation.md`](./design/m1-db-consolidation.md)、M2 [`m2-frontend-v2.md`](./design/m2-frontend-v2.md)、M3 [`m3-token-mobile.md`](./design/m3-token-mobile.md)、M4 [`m4-login-hardening.md`](./design/m4-login-hardening.md)、M5 [`m5-iot-energy.md`](./design/m5-iot-energy.md)、M6 [`m6-tibber-dynamic-energy.md`](./design/m6-tibber-dynamic-energy.md)、M7 [`m7-meter-epochs-archival.md`](./design/m7-meter-epochs-archival.md)、Pre-M8 [`pre-m8-warmtelink-p1-poc.md`](./design/pre-m8-warmtelink-p1-poc.md)、M8 [`m8-warmtelink-energy.md`](./design/m8-warmtelink-energy.md)。Pre-M8 与 M8-T01~T20 的自动化技术验收均已完成;M8 交付后用户人工 walkthrough 待验收。
|
||||||
|
|
||||||
## 当前基线(v1.0.3)
|
## 当前基线(v1.0.3)
|
||||||
|
|
||||||
@@ -40,6 +40,8 @@
|
|||||||
| **M5** ✅ | IoT / 能耗采集 | 通用 Modbus 采集(YAML profile + JSON readings)+ MQTT/HA Discovery + 前端侧边栏 + Energy 视图 |
|
| **M5** ✅ | IoT / 能耗采集 | 通用 Modbus 采集(YAML profile + JSON readings)+ MQTT/HA Discovery + 前端侧边栏 + Energy 视图 |
|
||||||
| **M6** ✅ | 通用电价层 + DSMR 接入 + 实时电费计算 | 通用电价层(manual/tibber profile + 合同版本)+ DSMR 实时电表接入 + 每 15min 寄存器差×价计量电费(不可变快照)+ 日/月/年汇总 + 反哺 HA Energy + 前端合同/价格/费用视图 |
|
| **M6** ✅ | 通用电价层 + DSMR 接入 + 实时电费计算 | 通用电价层(manual/tibber profile + 合同版本)+ DSMR 实时电表接入 + 每 15min 寄存器差×价计量电费(不可变快照)+ 日/月/年汇总 + 反哺 HA Energy + 前端合同/价格/费用视图 |
|
||||||
| **M7** ✅ | 电表生命周期 / 换表归档 | 引入 Meter epoch,计费永不跨表算 delta,跨表/无表/异常 delta 一律降级,累计按当前表归零,追溯换表可重算,Meter CRUD API + 前端管理 UI |
|
| **M7** ✅ | 电表生命周期 / 换表归档 | 引入 Meter epoch,计费永不跨表算 delta,跨表/无表/异常 delta 一律降级,累计按当前表归零,追溯换表可重算,Meter CRUD API + 前端管理 UI |
|
||||||
|
| **Pre-M8** ✅ | WarmteLink P1 真机概念验证 | 正式只读 CLI 长测通过;人工开启供暖后累计量 `0.017 → 0.018 GJ` 且与物理表一致,所有 frame 的 CRC 状态仍为 `unverifiable` |
|
||||||
|
| **M8** ✅ | WarmteLink P1、多数据源 Meter 与热力计费 | M8-T01~T20 自动化技术验收完成:Source/Channel/Binding、WarmteLink、DSMR 迁移、thermal 合同/成本、HA/UI、部署与文档闭环;真实 serial/HA walkthrough 由用户交付后验收 |
|
||||||
| **M3** | 开放与移动端(远期试水) | token 鉴权 + React Native 移动端 |
|
| **M3** | 开放与移动端(远期试水) | token 鉴权 + React Native 移动端 |
|
||||||
|
|
||||||
排序原则:**先清地基,再在干净结构上盖楼。** M2 的新 API 和 React 必须建立在合并后的单库之上;M4 是公网安全加固,在 M5 IoT 集成之前先堵住裸密码这个洞;M5 在安全基座就绪后再做 IoT 接入。
|
排序原则:**先清地基,再在干净结构上盖楼。** M2 的新 API 和 React 必须建立在合并后的单库之上;M4 是公网安全加固,在 M5 IoT 集成之前先堵住裸密码这个洞;M5 在安全基座就绪后再做 IoT 接入。
|
||||||
@@ -257,6 +259,72 @@ httpx / paho-mqtt / pyyaml / apscheduler 均为 M5 已有依赖,M6 复用,
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## Pre-M8 — WarmteLink P1 真机概念验证(✅ 已完成)
|
||||||
|
|
||||||
|
### 目标
|
||||||
|
|
||||||
|
2026-08-22 的正式仓库 CLI 以 `115200 7N1` 连续运行 10 分钟,正常退出且没有 I/O error、未处理
|
||||||
|
异常或断连。一个 WarmteLink source 在 60/60 完整 frame 中暴露 channel 1 的生活热水累计量
|
||||||
|
`5.900 m³` 和 channel 2 的区域供暖累计量 `0.017 GJ`;二者再次与同日物理表一致。没有流量、
|
||||||
|
热功率或温度字段。
|
||||||
|
|
||||||
|
所有完整 frame 都缺少标准 `/`,CRC 因而为 `unverifiable`;这项异常由正式 probe 原样报告,
|
||||||
|
没有伪装成校验通过。frame 平均 256 bytes、范围 237–275,设备 timestamp 严格每 10 秒递进。
|
||||||
|
10 分钟内累计值无变化只能证明稳定基线,不能证明消费时更新频率或 reset/wrap 行为。
|
||||||
|
|
||||||
|
最终人工走查又在终端连续运行 probe 十几分钟并开启供暖;区域供暖累计量从 `0.017 GJ` 增至
|
||||||
|
`0.018 GJ`,物理热量表同步显示 `0.018 GJ`。这确认了实际消费时的累计递增与 `0.001 GJ`
|
||||||
|
精度,但不改变 CRC `unverifiable` 结论,也不证明精确更新延迟或 reset/wrap 行为。
|
||||||
|
|
||||||
|
本阶段不落库、不接 API/前端/HA,也不决定正式 Device/Source/Meter 关系。它只向 M8 提供脱敏的真机事实,避免在未知 firmware/字段语义上提前设计。
|
||||||
|
|
||||||
|
> 验证计划与通过条件:[`docs/design/pre-m8-warmtelink-p1-poc.md`](./design/pre-m8-warmtelink-p1-poc.md)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## M8 — WarmteLink P1、多数据源 Meter 与热力计费(✅ 自动化技术验收已完成)
|
||||||
|
|
||||||
|
### 目标
|
||||||
|
|
||||||
|
以统一的 `MeterSource → MeterSourceChannel → MeterSourceBinding → Meter epoch` 链路承载现有
|
||||||
|
DSMR MQTT 与新的 WarmteLink serial source;从一个 WarmteLink source 只读采集已由真机确认的
|
||||||
|
区域供暖累计 `GJ` 和生活热水累计 `m³`,并完成历史、计费、API、前端、HA 与部署闭环。
|
||||||
|
|
||||||
|
### 已锁定范围
|
||||||
|
|
||||||
|
- Source、Modbus Device、Meter 分离;channel 先发现、再由用户确认绑定,source 切换与物理换表
|
||||||
|
使用两条独立半开时间线。
|
||||||
|
- DSMR JSON 和 WarmteLink Decimal scalar 分表存储,但都显式关联 source/binding;既有 DSMR
|
||||||
|
历史和电费安全回填,正常电费数字保持不变,跨 source/binding 周期降级。
|
||||||
|
- WarmteLink `unverifiable` frame 需要连续双帧确认后接纳,质量标签不漂白;约 10 秒更新 latest、
|
||||||
|
每分钟保存 history,不保存 raw telegram 或 raw equipment identifier。
|
||||||
|
- 串口 worker 使用 `115200 7N1`、只读、短 DB session、可热更新和 `1…60s` 退避重连;现有
|
||||||
|
production base 与 base+dev Compose 通过 `.env` 必填的 `WARMTELINK_DEVICE_PATH`(stable by-id)
|
||||||
|
和 `WARMTELINK_SERIAL_GID` 直配,将设备固定映射为 `/dev/warmtelink:rw`,保持非 root 最小权限;
|
||||||
|
同一物理串口只能有一个 owner。
|
||||||
|
- 合同增加 electricity/thermal scope;一份 district-heating 合同覆盖 heating GJ 与 hot-water m³,
|
||||||
|
生成按 commodity 的 15 分钟成本和 contract-level 每日固定费。仓库不硬编码真实 tariff。
|
||||||
|
- UI 分成 Sources / Modbus Devices / Meters,并让 Contracts / Prices / Costs 按 scope 切换;新增
|
||||||
|
source、meter 与 thermal cost HA entities,开关默认关闭,旧 DSMR latest API 保持兼容。
|
||||||
|
|
||||||
|
### 原子实施链
|
||||||
|
|
||||||
|
- **M8-T01~T06**:已完成统一 source/channel/binding schema、DSMR 历史/runtime/电费迁移和管理 API。
|
||||||
|
- **M8-T07~T11**:已完成共享 P1 parser、WarmteLink 标量存储、质量接纳、serial worker、发现与历史 API。
|
||||||
|
- **M8-T12~T16**:已完成合同 scope、district-heating profile、thermal cost 账本/引擎/API。
|
||||||
|
- **M8-T17~T19**:已完成 HA、Sources/Meters UI 与 scope-aware 计费 UI;**M8-T20** 已完成 compose、文档与隔离自动化技术收尾。
|
||||||
|
|
||||||
|
完成判据不仅是单元闸门全绿,还包括以 mock/fake、合成数据库和隔离 Docker 完成的历史迁移对账、
|
||||||
|
OpenAPI/codegen、全部前端闸门、真实 `docker build` 与非 root 串口部署技术验收;并须交付完整的
|
||||||
|
九步用户 walkthrough 和证据模板。真实 serial/HA 的观察由用户在交付后自行验收,不是 agent/Reviewer
|
||||||
|
技术 PASS、T20 状态更新或 M8 autosquash/收尾的前置条件,也不得写成已执行。任何任务都不得删除旧
|
||||||
|
数据库、历史读数、旧 config 行或 volume;push/tag 仍需用户单独授权。
|
||||||
|
|
||||||
|
> 完整架构、HTTP 契约、质量/计费规则、依赖图与 M8-T01~M8-T20 任务卡:
|
||||||
|
> [`docs/design/m8-warmtelink-energy.md`](./design/m8-warmtelink-energy.md)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## M3 — 开放与移动端(远期试水)
|
## M3 — 开放与移动端(远期试水)
|
||||||
|
|
||||||
### 目标
|
### 目标
|
||||||
|
|||||||
@@ -0,0 +1,110 @@
|
|||||||
|
# WarmteLink、数据源与热力计费运维手册
|
||||||
|
|
||||||
|
本手册说明如何在不改变既有 DSMR、Modbus 和 electricity 功能的前提下,部署只读的 WarmteLink P1 采集、绑定热量/生活热水 Meter、配置热力合同,并按需暴露给 Home Assistant。所有示例均使用占位符;不要把设备标识、GID、数据库路径、token 或真实合同金额提交到仓库。
|
||||||
|
|
||||||
|
## 安全边界与开始前备份
|
||||||
|
|
||||||
|
- 应用容器继续使用基础 `docker-compose.yml` 中的非 root `user: "1000:1000"`;基础文件和 dev 合并配置都不设置 `privileged` 或额外 capability。
|
||||||
|
- Docker device cgroup 必须以 `rw` 映射,容器内固定为 `/dev/warmtelink`:这是 pyserial 3.5 在 POSIX 上以 `O_RDWR` 打开串口所需的最小系统权限,并不表示业务可写。规则绝不包含 `m`,且不授予 root、`privileged` 或额外 capability。WarmteLink worker 仍只调用 serial `read` / `close`,绝不调用 `write` 或发送写命令。
|
||||||
|
- 不删除或覆盖 `app_config`、`app.db`、旧数据库、Docker volume 或既有 source。禁用/解绑/回滚配置不是删除历史的替代方式。
|
||||||
|
- 维护前停止写入窗口,使用宿主机的备份流程复制 `./data/app.db` 到受保护的备份位置;确认备份可用后才运行 migration。不要把生产库复制到开发机或用于测试。
|
||||||
|
|
||||||
|
## 识别稳定串口并配置 Compose
|
||||||
|
|
||||||
|
在宿主机(不是容器)找出稳定 symlink;不要使用会在重启后变化的 `/dev/ttyUSB*` 名称:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ls -l /dev/serial/by-id/
|
||||||
|
stable_path=/dev/serial/by-id/<stable-by-id-name>
|
||||||
|
stat -c '%g %n' "$stable_path"
|
||||||
|
```
|
||||||
|
|
||||||
|
记录输出的数字 GID,而不是猜测 `dialout` 的数值。确认 path 指向预期的字符设备后,在部署机本地 `.env`(不提交)设置:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
WARMTELINK_DEVICE_PATH=/dev/serial/by-id/<stable-by-id-name>
|
||||||
|
WARMTELINK_SERIAL_GID=<host-serial-gid>
|
||||||
|
```
|
||||||
|
|
||||||
|
production 使用基础 compose;local dev 使用 base 与 dev 合并文件。两种环境都会从本地 `.env` 读取这两个变量:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose -f docker-compose.yml up -d
|
||||||
|
docker compose -f docker-compose.yml -f docker-compose.dev.yml up -d --build
|
||||||
|
```
|
||||||
|
|
||||||
|
启动前可用相同文件组合运行 `docker compose ... config`。结果的 `app` 必须仍显示 `user: "1000:1000"`,并只出现从 stable by-id path 到 `/dev/warmtelink` 的 `rw` device mapping 与 `group_add` GID;migration 不得有 device 或 group。不得出现 `privileged`、root user 或 `m` device permission。`rw` 仅满足 pyserial 的 POSIX `O_RDWR` 打开,不改变 worker 的只读业务行为。不得把设备路径或 GID 写入仓库的 `.env.example` 或文档。
|
||||||
|
|
||||||
|
同一物理串口在任意时刻只能有一个 owner。运行 Pre-M8 `p1_probe.py` 前,必须先停止 app(包括 dev stack),并在 probe 结束后再启动 app;不要让 probe 与 worker 同时打开该串口。
|
||||||
|
|
||||||
|
## Migration 与 source 配置
|
||||||
|
|
||||||
|
先在维护窗口运行 migration;它只升级 schema,绝不删除历史表或配置:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose -f docker-compose.yml run --rm migration
|
||||||
|
```
|
||||||
|
|
||||||
|
登录 Energy 页面,在 **Sources** 创建 `warmtelink_serial` source:
|
||||||
|
|
||||||
|
| 字段 | 值 |
|
||||||
|
| --- | --- |
|
||||||
|
| Path | `/dev/warmtelink` |
|
||||||
|
| Baud rate | `115200` |
|
||||||
|
| Data bits / parity / stop bits | `7` / `N` / `1` |
|
||||||
|
| Enabled | 先关闭,保存并检查配置后再开启 |
|
||||||
|
|
||||||
|
这些参数是固定 P1 profile。不要录入 telegram 或 equipment id;应用不会把原始 frame、设备身份或 serial 设置以外的协议标识保存到 source 配置中。开启后选择 **Discover**。成功发现后应有两个只读 channel:heating(`GJ`)和 hot water(`m³`);受 P1 CRC 限制,正常可接纳的质量可能显示为 `unverifiable`,这不是被错误提升为 `valid`。
|
||||||
|
|
||||||
|
在 **Meters** 分别创建或选择 `heating` 与 `hot_water` Meter,并各自选择与单位相符的 channel 创建 binding。source switch 只关闭旧 binding、在相同 Meter 上创建新 binding,不是换表;真正的物理换表才使用 Meter swap。跨 binding 或 Meter 边界的 15 分钟成本周期会明确标为 degraded,不能相减伪造成本。
|
||||||
|
|
||||||
|
## 运行检查与排障
|
||||||
|
|
||||||
|
启用 source 后,latest 通常约每 10 秒更新,持久化 history 最多每分钟一条。检查 source 状态、channel latest/quality 和绑定时间线,而不是从日志中寻找原始 telegram。
|
||||||
|
|
||||||
|
| 现象 | 安全排查 |
|
||||||
|
| --- | --- |
|
||||||
|
| `offline` 或无法 open serial | 核对 stable symlink 是否仍存在、`stat` 的 GID 是否等于 `WARMTELINK_SERIAL_GID`,再用 `docker compose ... config` 检查 non-root `rw`(非 `r`、非 `m`)device rule;不要用 root/privileged 绕过权限。 |
|
||||||
|
| 无 channels / discover 超时 | 确认 source 启用、`/dev/warmtelink` path 和固定 `115200 7N1`;检查电缆供电后重试 Discover。 |
|
||||||
|
| 短暂拔线 | worker 标记 offline 并以退避重连;插回后应恢复。检查 history 的 `(channel, recorded_at)` 唯一性,不能手工补重复行。 |
|
||||||
|
| 成本 degraded | 查两端读数 freshness(120 秒)、quality、Meter epoch 与 binding;不要通过修改累计值清除 degraded。 |
|
||||||
|
|
||||||
|
若需要停采集,在 UI 禁用该 source,确认 worker 关闭串口后再维护电缆。删除有 channel、binding 或历史的 source 会被 API 拒绝;保留记录以保证审计和成本重算。
|
||||||
|
|
||||||
|
## 热力合同、成本与 Home Assistant
|
||||||
|
|
||||||
|
在 **Contracts** 选择 `Thermal` scope,创建 `district_heating` 合同及版本。费率由 operator 按合同人工录入,字段为 heating(EUR/GJ)、hot-water heating / water / tax(EUR/m³)与五个年固定费字段;仓库不含任何真实默认金额。thermal 和 electricity 各可有一个 active 合同,彼此不互斥。
|
||||||
|
|
||||||
|
成本页的 15 分钟 ledger 分开显示 heating 与 hot-water 三项 variable breakdown;fixed 费只在合同级 summary 按本地自然日计提一次,all-in = variable + fixed。用显式 recompute 来验证测试时间窗时,应手算并核对 Decimal 金额,保留原有 electricity 合同和数字不变。
|
||||||
|
|
||||||
|
在 Config 的 HA Expose 中只开启需要的 source、Meter 与 thermal entities。核对 unit、state class、availability、today reset 和换表后 identity;关闭 toggle 后应用会清理 retained discovery。不要把 source secret、设备 identity 或合同金额放进 HA entity 名称、日志或截图。
|
||||||
|
|
||||||
|
## 安全回滚
|
||||||
|
|
||||||
|
1. 在 UI 禁用 WarmteLink source,确认状态离线且 worker 已停止;保留 channels、bindings、history、contracts 与成本账本。
|
||||||
|
2. 停止 app 后,在 UI 保持 source 禁用;这不会删除 `./data`、数据库、配置或 volumes。需要恢复 WarmteLink 时,确认本地 `.env` 的 stable by-id/GID 后再重新启用 source。
|
||||||
|
3. 确认 DSMR、Modbus、电价、既有 electricity 成本和前端正常。
|
||||||
|
4. schema migration 不应以 production downgrade 回滚;只有经过验证的备份恢复流程才处理灾难恢复,且必须由 operator 在隔离维护窗口执行。
|
||||||
|
|
||||||
|
## 上线验收清单
|
||||||
|
|
||||||
|
- production 与 base+dev Compose 都从本地 `.env` 获取 serial path/GID;app/migration 均为非 root,只有 app 有 `/dev/warmtelink:rw` 和 serial GID,绝无 `m`、root 或 privileged。该 `rw` 仅为 pyserial 的 `O_RDWR` 打开,worker 业务仍只读,且没有真实设备/GID 被记录在仓库。
|
||||||
|
- 下列项目是交付后由用户在备份数据库、可回滚部署、真实 serial 设备和真实 HA 环境执行的人工验收;自动化技术验收不能替代这些观察,也不得把它们伪称为已执行。
|
||||||
|
|
||||||
|
### 用户人工验收记录(交付后填写)
|
||||||
|
|
||||||
|
本模板在交付后由用户填写;它不描述当前宿主环境,也不代表任何项目已通过。每项均填写日期、隔离部署
|
||||||
|
标识、§12 项号、预期观察、实际观察、脱敏日志/截图引用、回滚结果和结果状态。不得记录 stable device id、
|
||||||
|
GID、secret、数据库路径或合同金额;交付时所有项目默认均为“未执行”,不得填造真实环境结果。
|
||||||
|
|
||||||
|
| §12 项号 | 日期 | 隔离部署标识 | 预期观察 | 实际观察 | 脱敏日志/截图引用 | 回滚结果 | 结果状态 |
|
||||||
|
| --- | --- | --- | --- | --- | --- | --- | --- |
|
||||||
|
| 1. 默认 stack 回归 | 待填写 | 待填写 | DSMR、Modbus、电价/电费、前端均无回归 | 待填写 | 待填写 | 待填写 | 未执行 |
|
||||||
|
| 2. stable by-id 与权限 | 待填写 | 待填写 | 非 root、非 privileged、`rw` 无 `m`;pyserial 可打开,worker 无 `write` | 待填写 | 待填写 | 待填写 | 未执行 |
|
||||||
|
| 3. Discover 与隐私 | 待填写 | 待填写 | 两 channel/unit/quality;日志、DB、API、UI 均完成脱敏检查 | 待填写 | 待填写 | 待填写 | 未执行 |
|
||||||
|
| 4. history/拔插重连 | 待填写 | 待填写 | latest、分钟 history、拔插恢复且无重复记录 | 待填写 | 待填写 | 待填写 | 未执行 |
|
||||||
|
| 5. source switch / Meter swap | 待填写 | 待填写 | 两条时间线正确、边界 degraded、HA identity 按设计变化 | 待填写 | 待填写 | 待填写 | 未执行 |
|
||||||
|
| 6. thermal 合同/成本 | 待填写 | 待填写 | 脱敏测试费率手算一致;15 分钟与 01:05 fixed 正确 | 待填写 | 待填写 | 待填写 | 未执行 |
|
||||||
|
| 7. 双 active scope | 待填写 | 待填写 | electricity 数字对照一致;UI/成本/HA 不串 scope | 待填写 | 待填写 | 待填写 | 未执行 |
|
||||||
|
| 8. HA toggles | 待填写 | 待填写 | unit、state class、availability、today、identity、retained cleanup 正确 | 待填写 | 待填写 | 待填写 | 未执行 |
|
||||||
|
| 9. 重启与默认 compose 回滚 | 待填写 | 待填写 | 历史恢复;回默认 compose 后数据完整 | 待填写 | 待填写 | 待填写 | 未执行 |
|
||||||
Vendored
+1316
-9
File diff suppressed because it is too large
Load Diff
@@ -65,6 +65,11 @@ const PROFILES_RESPONSE = {
|
|||||||
heffingskorting: { unit: 'EUR/year' },
|
heffingskorting: { unit: 'EUR/year' },
|
||||||
},
|
},
|
||||||
},
|
},
|
||||||
|
{
|
||||||
|
kind: 'district_heating', label: 'District heating',
|
||||||
|
variable: { heating: { unit: 'EUR/GJ', default: 0 } },
|
||||||
|
standing: { delivery_set: { unit: 'EUR/year', default: 0 } },
|
||||||
|
},
|
||||||
],
|
],
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -79,6 +84,21 @@ const CREATED_CONTRACT = {
|
|||||||
versions: [],
|
versions: [],
|
||||||
}
|
}
|
||||||
|
|
||||||
|
const D11_PROFILE_RESPONSE = {
|
||||||
|
profiles: [{
|
||||||
|
kind: 'district_heating', label: 'District heating',
|
||||||
|
variable: {
|
||||||
|
heating: { unit: 'EUR/GJ' }, hot_water_heating: { unit: 'EUR/m³' },
|
||||||
|
hot_water: { unit: 'EUR/m³' }, hot_water_tax: { unit: 'EUR/m³' },
|
||||||
|
},
|
||||||
|
standing: {
|
||||||
|
heating_network: { unit: 'EUR/year' }, metering: { unit: 'EUR/year' },
|
||||||
|
delivery_set: { unit: 'EUR/year' }, hot_water_network: { unit: 'EUR/year' },
|
||||||
|
other: { unit: 'EUR/year' },
|
||||||
|
},
|
||||||
|
}],
|
||||||
|
}
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
// Import component
|
// Import component
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
@@ -174,6 +194,49 @@ describe('ContractForm', () => {
|
|||||||
}, { timeout: 3000 })
|
}, { timeout: 3000 })
|
||||||
})
|
})
|
||||||
|
|
||||||
|
it('limits a thermal create form to its compatible profile and posts its scope', async () => {
|
||||||
|
const user = userEvent.setup()
|
||||||
|
mockGet.mockResolvedValue({ data: PROFILES_RESPONSE })
|
||||||
|
mockPost.mockResolvedValue({ data: CREATED_CONTRACT })
|
||||||
|
renderWithProviders(<ContractForm scope="thermal" defaultKind="district_heating" onClose={vi.fn()} onSaved={vi.fn()} />)
|
||||||
|
await waitFor(() => expect(screen.getByTestId('contract-field-variable.heating')).toBeInTheDocument())
|
||||||
|
await user.type(screen.getByTestId('contract-name'), 'Heat')
|
||||||
|
await user.click(screen.getByTestId('contract-form-submit'))
|
||||||
|
await waitFor(() => expect(mockPost).toHaveBeenCalledWith('/api/energy/contracts', expect.objectContaining({ body: expect.objectContaining({ kind: 'district_heating', scope: 'thermal' }) })))
|
||||||
|
})
|
||||||
|
|
||||||
|
it('posts all nine D11 values as unrounded Decimal strings, including zero standing fees', async () => {
|
||||||
|
const user = userEvent.setup()
|
||||||
|
mockGet.mockResolvedValue({ data: D11_PROFILE_RESPONSE })
|
||||||
|
mockPost.mockResolvedValue({ data: CREATED_CONTRACT })
|
||||||
|
renderWithProviders(<ContractForm scope="thermal" defaultKind="district_heating" onClose={vi.fn()} onSaved={vi.fn()} />)
|
||||||
|
await waitFor(() => expect(screen.getByTestId('contract-field-variable.heating')).toBeInTheDocument())
|
||||||
|
await user.type(screen.getByTestId('contract-name'), 'Precise heat')
|
||||||
|
const values: Record<string, string> = {
|
||||||
|
'variable.heating': '20.123456789123456789', 'variable.hot_water_heating': '8.200000000000000001',
|
||||||
|
'variable.hot_water': '1.234567890123456789', 'variable.hot_water_tax': '0.456789012345678901',
|
||||||
|
'standing.heating_network': '0', 'standing.metering': '0', 'standing.delivery_set': '0',
|
||||||
|
'standing.hot_water_network': '0', 'standing.other': '0',
|
||||||
|
}
|
||||||
|
for (const [key, value] of Object.entries(values)) {
|
||||||
|
const input = screen.getByTestId(`contract-field-${key}`)
|
||||||
|
await user.clear(input)
|
||||||
|
await user.type(input, value)
|
||||||
|
}
|
||||||
|
await user.click(screen.getByTestId('contract-form-submit'))
|
||||||
|
await waitFor(() => expect(mockPost).toHaveBeenCalled())
|
||||||
|
const body = mockPost.mock.calls[0][1].body
|
||||||
|
expect(body).toMatchObject({ scope: 'thermal', values: {
|
||||||
|
variable: {
|
||||||
|
heating: values['variable.heating'], hot_water_heating: values['variable.hot_water_heating'],
|
||||||
|
hot_water: values['variable.hot_water'], hot_water_tax: values['variable.hot_water_tax'],
|
||||||
|
},
|
||||||
|
standing: {
|
||||||
|
heating_network: '0', metering: '0', delivery_set: '0', hot_water_network: '0', other: '0',
|
||||||
|
},
|
||||||
|
} })
|
||||||
|
})
|
||||||
|
|
||||||
it('calls POST /api/energy/contracts/{id}/versions in add-version mode', async () => {
|
it('calls POST /api/energy/contracts/{id}/versions in add-version mode', async () => {
|
||||||
const user = userEvent.setup()
|
const user = userEvent.setup()
|
||||||
|
|
||||||
|
|||||||
@@ -38,6 +38,8 @@ export interface ContractFormProps {
|
|||||||
contractId?: number
|
contractId?: number
|
||||||
/** Existing contract kind (for add-version mode or edit). */
|
/** Existing contract kind (for add-version mode or edit). */
|
||||||
defaultKind?: string
|
defaultKind?: string
|
||||||
|
/** The list/create scope currently selected by the parent. */
|
||||||
|
scope?: 'electricity' | 'thermal'
|
||||||
onClose: () => void
|
onClose: () => void
|
||||||
onSaved: () => void
|
onSaved: () => void
|
||||||
}
|
}
|
||||||
@@ -64,7 +66,7 @@ interface LeafField {
|
|||||||
/** Dot-separated path within the section, e.g. "buy.normal" */
|
/** Dot-separated path within the section, e.g. "buy.normal" */
|
||||||
fieldPath: string
|
fieldPath: string
|
||||||
unit: string
|
unit: string
|
||||||
defaultValue?: number
|
defaultValue?: number | string
|
||||||
}
|
}
|
||||||
|
|
||||||
function extractLeafFields(obj: Record<string, unknown>, prefix = ''): LeafField[] {
|
function extractLeafFields(obj: Record<string, unknown>, prefix = ''): LeafField[] {
|
||||||
@@ -77,7 +79,9 @@ function extractLeafFields(obj: Record<string, unknown>, prefix = ''): LeafField
|
|||||||
fields.push({
|
fields.push({
|
||||||
fieldPath: path,
|
fieldPath: path,
|
||||||
unit: val.unit,
|
unit: val.unit,
|
||||||
defaultValue: typeof val.default === 'number' ? val.default : undefined,
|
defaultValue: typeof val.default === 'number' || typeof val.default === 'string'
|
||||||
|
? val.default
|
||||||
|
: undefined,
|
||||||
})
|
})
|
||||||
} else {
|
} else {
|
||||||
fields.push(...extractLeafFields(val as Record<string, unknown>, path))
|
fields.push(...extractLeafFields(val as Record<string, unknown>, path))
|
||||||
@@ -93,6 +97,7 @@ function extractLeafFields(obj: Record<string, unknown>, prefix = ''): LeafField
|
|||||||
function buildNestedValues(
|
function buildNestedValues(
|
||||||
sectionFields: Record<string, LeafField[]>,
|
sectionFields: Record<string, LeafField[]>,
|
||||||
fieldValues: Record<string, number | string>,
|
fieldValues: Record<string, number | string>,
|
||||||
|
decimalStrings: boolean,
|
||||||
): Record<string, unknown> {
|
): Record<string, unknown> {
|
||||||
const result: Record<string, unknown> = {}
|
const result: Record<string, unknown> = {}
|
||||||
|
|
||||||
@@ -100,7 +105,6 @@ function buildNestedValues(
|
|||||||
const sectionObj: Record<string, unknown> = {}
|
const sectionObj: Record<string, unknown> = {}
|
||||||
for (const field of fields) {
|
for (const field of fields) {
|
||||||
const raw = fieldValues[`${section}.${field.fieldPath}`]
|
const raw = fieldValues[`${section}.${field.fieldPath}`]
|
||||||
const numVal = typeof raw === 'number' ? raw : parseFloat(String(raw))
|
|
||||||
// Set nested path
|
// Set nested path
|
||||||
const parts = field.fieldPath.split('.')
|
const parts = field.fieldPath.split('.')
|
||||||
let current = sectionObj
|
let current = sectionObj
|
||||||
@@ -108,7 +112,10 @@ function buildNestedValues(
|
|||||||
if (!(parts[i] in current)) current[parts[i]] = {}
|
if (!(parts[i] in current)) current[parts[i]] = {}
|
||||||
current = current[parts[i]] as Record<string, unknown>
|
current = current[parts[i]] as Record<string, unknown>
|
||||||
}
|
}
|
||||||
current[parts[parts.length - 1]] = isNaN(numVal) ? 0 : numVal
|
// Values are Decimal JSON strings. Do not round-trip user money through JS Number.
|
||||||
|
current[parts[parts.length - 1]] = decimalStrings
|
||||||
|
? (raw === undefined || raw === '' ? '0' : String(raw))
|
||||||
|
: (Number.isFinite(Number(raw)) ? Number(raw) : 0)
|
||||||
}
|
}
|
||||||
result[section] = sectionObj
|
result[section] = sectionObj
|
||||||
}
|
}
|
||||||
@@ -130,7 +137,7 @@ function formatLabel(path: string): string {
|
|||||||
// Component
|
// Component
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
export function ContractForm({ contractId, defaultKind, onClose, onSaved }: ContractFormProps) {
|
export function ContractForm({ contractId, defaultKind, scope, onClose, onSaved }: ContractFormProps) {
|
||||||
const isAddVersion = contractId != null
|
const isAddVersion = contractId != null
|
||||||
|
|
||||||
// Profiles query
|
// Profiles query
|
||||||
@@ -161,7 +168,9 @@ export function ContractForm({ contractId, defaultKind, onClose, onSaved }: Cont
|
|||||||
const effectiveKind = isAddVersion ? (defaultKind ?? null) : selectedKind
|
const effectiveKind = isAddVersion ? (defaultKind ?? null) : selectedKind
|
||||||
|
|
||||||
// Build profile options from API response
|
// Build profile options from API response
|
||||||
const profiles = profilesQuery.data?.profiles ?? []
|
const profiles = (profilesQuery.data?.profiles ?? []).filter((p: Record<string, unknown>) =>
|
||||||
|
scope === 'thermal' ? p.kind === 'district_heating' : p.kind !== 'district_heating',
|
||||||
|
)
|
||||||
const profileOptions = profiles.map((p: Record<string, unknown>) => ({
|
const profileOptions = profiles.map((p: Record<string, unknown>) => ({
|
||||||
value: p.kind as string,
|
value: p.kind as string,
|
||||||
label: (p.label as string | undefined) ?? (p.kind as string),
|
label: (p.label as string | undefined) ?? (p.kind as string),
|
||||||
@@ -219,8 +228,7 @@ export function ContractForm({ contractId, defaultKind, onClose, onSaved }: Cont
|
|||||||
cursor = (cursor as Record<string, unknown>)[part]
|
cursor = (cursor as Record<string, unknown>)[part]
|
||||||
}
|
}
|
||||||
if (cursor != null && (typeof cursor === 'number' || typeof cursor === 'string')) {
|
if (cursor != null && (typeof cursor === 'number' || typeof cursor === 'string')) {
|
||||||
const numVal = typeof cursor === 'number' ? cursor : parseFloat(String(cursor))
|
seeded[`${section}.${leaf.fieldPath}`] = effectiveKind === 'district_heating' ? String(cursor) : Number(cursor)
|
||||||
seeded[`${section}.${leaf.fieldPath}`] = isNaN(numVal) ? 0 : numVal
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -231,6 +239,7 @@ export function ContractForm({ contractId, defaultKind, onClose, onSaved }: Cont
|
|||||||
contractDetailQuery.isError,
|
contractDetailQuery.isError,
|
||||||
contractDetailQuery.data,
|
contractDetailQuery.data,
|
||||||
sectionFields,
|
sectionFields,
|
||||||
|
effectiveKind,
|
||||||
])
|
])
|
||||||
|
|
||||||
// The effective field values: user edits override prefill; prefill is the base.
|
// The effective field values: user edits override prefill; prefill is the base.
|
||||||
@@ -285,7 +294,7 @@ export function ContractForm({ contractId, defaultKind, onClose, onSaved }: Cont
|
|||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
const values = buildNestedValues(sectionFields, fieldValues)
|
const values = buildNestedValues(sectionFields, fieldValues, effectiveKind === 'district_heating')
|
||||||
|
|
||||||
try {
|
try {
|
||||||
// Convert local date string to a naive local-midnight datetime string (no Z).
|
// Convert local date string to a naive local-midnight datetime string (no Z).
|
||||||
@@ -307,6 +316,7 @@ export function ContractForm({ contractId, defaultKind, onClose, onSaved }: Cont
|
|||||||
const body = {
|
const body = {
|
||||||
name: name.trim(),
|
name: name.trim(),
|
||||||
kind: effectiveKind,
|
kind: effectiveKind,
|
||||||
|
...(scope ? { scope } : {}),
|
||||||
currency,
|
currency,
|
||||||
values,
|
values,
|
||||||
...(effectiveFromISO ? { effective_from: effectiveFromISO } : {}),
|
...(effectiveFromISO ? { effective_from: effectiveFromISO } : {}),
|
||||||
@@ -409,19 +419,21 @@ export function ContractForm({ contractId, defaultKind, onClose, onSaved }: Cont
|
|||||||
</Title>
|
</Title>
|
||||||
{fields.map((field) => {
|
{fields.map((field) => {
|
||||||
const key = `${section}.${field.fieldPath}`
|
const key = `${section}.${field.fieldPath}`
|
||||||
const raw = fieldValues[key]
|
return effectiveKind === 'district_heating' ? (
|
||||||
const numVal = typeof raw === 'number' ? raw : parseFloat(String(raw))
|
<TextInput
|
||||||
return (
|
|
||||||
<NumberInput
|
|
||||||
key={key}
|
key={key}
|
||||||
label={formatLabel(field.fieldPath)}
|
label={formatLabel(field.fieldPath)}
|
||||||
description={field.unit}
|
description={field.unit}
|
||||||
value={isNaN(numVal) ? 0 : numVal}
|
inputMode="decimal"
|
||||||
onChange={(val) => handleFieldChange(key, val)}
|
value={String(fieldValues[key] ?? '0')}
|
||||||
decimalScale={6}
|
onChange={(event) => handleFieldChange(key, event.currentTarget.value)}
|
||||||
step={0.001}
|
|
||||||
data-testid={`contract-field-${key}`}
|
data-testid={`contract-field-${key}`}
|
||||||
/>
|
/>
|
||||||
|
) : (
|
||||||
|
<NumberInput key={key} label={formatLabel(field.fieldPath)} description={field.unit}
|
||||||
|
value={typeof fieldValues[key] === 'number' ? fieldValues[key] : Number(fieldValues[key] ?? 0)}
|
||||||
|
onChange={(value) => handleFieldChange(key, value)} decimalScale={6} step={0.001}
|
||||||
|
data-testid={`contract-field-${key}`} />
|
||||||
)
|
)
|
||||||
})}
|
})}
|
||||||
</Stack>
|
</Stack>
|
||||||
|
|||||||
@@ -66,6 +66,11 @@ const INACTIVE_CONTRACT = {
|
|||||||
updated_at: '2026-06-02T00:00:00Z',
|
updated_at: '2026-06-02T00:00:00Z',
|
||||||
}
|
}
|
||||||
|
|
||||||
|
const ACTIVE_THERMAL_CONTRACT = {
|
||||||
|
id: 3, name: 'Active Heat Contract', kind: 'district_heating', active: true, currency: 'EUR',
|
||||||
|
created_at: '2026-06-03T00:00:00Z', updated_at: '2026-06-03T00:00:00Z',
|
||||||
|
}
|
||||||
|
|
||||||
const PROFILES_RESPONSE = {
|
const PROFILES_RESPONSE = {
|
||||||
profiles: [
|
profiles: [
|
||||||
{
|
{
|
||||||
@@ -202,4 +207,39 @@ describe('ContractManager', () => {
|
|||||||
expect(screen.getByTestId('contract-form-modal')).toBeInTheDocument()
|
expect(screen.getByTestId('contract-form-modal')).toBeInTheDocument()
|
||||||
})
|
})
|
||||||
})
|
})
|
||||||
|
|
||||||
|
it('keeps the selector available while thermal data loads and requests each scope separately', async () => {
|
||||||
|
const user = userEvent.setup()
|
||||||
|
mockGet.mockImplementation((path: string, options?: { params?: { query?: { scope?: string } } }) => {
|
||||||
|
if (path === '/api/energy/contracts' && options?.params?.query?.scope === 'thermal') return new Promise(() => {})
|
||||||
|
if (path === '/api/energy/contracts') return Promise.resolve({ data: { items: [ACTIVE_CONTRACT], total: 1 } })
|
||||||
|
return Promise.resolve({ data: PROFILES_RESPONSE })
|
||||||
|
})
|
||||||
|
renderWithProviders(<ContractManager />)
|
||||||
|
await waitFor(() => expect(screen.getByTestId('contracts-scope-selector')).toBeInTheDocument())
|
||||||
|
await user.click(screen.getByText('Thermal'))
|
||||||
|
expect(screen.getByTestId('contracts-loading')).toBeInTheDocument()
|
||||||
|
expect(screen.getByTestId('contracts-scope-selector')).toBeInTheDocument()
|
||||||
|
await user.click(screen.getByText('Electricity'))
|
||||||
|
await waitFor(() => expect(screen.getByTestId('contracts-table')).toBeInTheDocument())
|
||||||
|
expect(mockGet).toHaveBeenCalledWith('/api/energy/contracts', { params: { query: { scope: 'thermal' } } })
|
||||||
|
})
|
||||||
|
|
||||||
|
it('keeps simultaneous active electricity and thermal contracts isolated across repeated switches', async () => {
|
||||||
|
const user = userEvent.setup()
|
||||||
|
mockGet.mockImplementation((_path: string, options?: { params?: { query?: { scope?: string } } }) =>
|
||||||
|
Promise.resolve({ data: { items: options?.params?.query?.scope === 'thermal'
|
||||||
|
? [ACTIVE_THERMAL_CONTRACT] : [ACTIVE_CONTRACT], total: 1 } }),
|
||||||
|
)
|
||||||
|
renderWithProviders(<ContractManager />)
|
||||||
|
await waitFor(() => expect(screen.getByText('My Active Contract')).toBeInTheDocument())
|
||||||
|
for (let i = 0; i < 2; i += 1) {
|
||||||
|
await user.click(screen.getByText('Thermal'))
|
||||||
|
await waitFor(() => expect(screen.getByText('Active Heat Contract')).toBeInTheDocument())
|
||||||
|
expect(screen.queryByText('My Active Contract')).not.toBeInTheDocument()
|
||||||
|
await user.click(screen.getByText('Electricity'))
|
||||||
|
await waitFor(() => expect(screen.getByText('My Active Contract')).toBeInTheDocument())
|
||||||
|
expect(screen.queryByText('Active Heat Contract')).not.toBeInTheDocument()
|
||||||
|
}
|
||||||
|
})
|
||||||
})
|
})
|
||||||
|
|||||||
@@ -10,6 +10,7 @@
|
|||||||
*/
|
*/
|
||||||
|
|
||||||
import { useState } from 'react'
|
import { useState } from 'react'
|
||||||
|
import { useQuery } from '@tanstack/react-query'
|
||||||
import {
|
import {
|
||||||
Table,
|
Table,
|
||||||
Button,
|
Button,
|
||||||
@@ -24,9 +25,9 @@ import {
|
|||||||
Modal,
|
Modal,
|
||||||
Accordion,
|
Accordion,
|
||||||
Code,
|
Code,
|
||||||
|
SegmentedControl,
|
||||||
} from '@mantine/core'
|
} from '@mantine/core'
|
||||||
import {
|
import {
|
||||||
useContracts,
|
|
||||||
useUpdateContract,
|
useUpdateContract,
|
||||||
type ContractResponse,
|
type ContractResponse,
|
||||||
type ContractDetailResponse,
|
type ContractDetailResponse,
|
||||||
@@ -244,7 +245,14 @@ function ContractTable({
|
|||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
export function ContractManager() {
|
export function ContractManager() {
|
||||||
const contractsQuery = useContracts()
|
const [scope, setScope] = useState<'electricity' | 'thermal'>('electricity')
|
||||||
|
const contractsQuery = useQuery({
|
||||||
|
queryKey: ['energy-contracts', scope],
|
||||||
|
queryFn: async () => {
|
||||||
|
const res = await apiClient.GET('/api/energy/contracts', { params: { query: { scope } } })
|
||||||
|
return res.data
|
||||||
|
},
|
||||||
|
})
|
||||||
const updateMutation = useUpdateContract()
|
const updateMutation = useUpdateContract()
|
||||||
|
|
||||||
const [showCreateForm, setShowCreateForm] = useState(false)
|
const [showCreateForm, setShowCreateForm] = useState(false)
|
||||||
@@ -252,6 +260,13 @@ export function ContractManager() {
|
|||||||
const [historyContract, setHistoryContract] = useState<ContractResponse | null>(null)
|
const [historyContract, setHistoryContract] = useState<ContractResponse | null>(null)
|
||||||
const [activatingId, setActivatingId] = useState<number | null>(null)
|
const [activatingId, setActivatingId] = useState<number | null>(null)
|
||||||
|
|
||||||
|
function handleScopeChange(nextScope: 'electricity' | 'thermal') {
|
||||||
|
setScope(nextScope)
|
||||||
|
setShowCreateForm(false)
|
||||||
|
setAddVersionContract(null)
|
||||||
|
setHistoryContract(null)
|
||||||
|
}
|
||||||
|
|
||||||
async function handleActivate(id: number) {
|
async function handleActivate(id: number) {
|
||||||
setActivatingId(id)
|
setActivatingId(id)
|
||||||
try {
|
try {
|
||||||
@@ -265,44 +280,40 @@ export function ContractManager() {
|
|||||||
// Render states
|
// Render states
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
if (contractsQuery.isLoading) {
|
const contracts = contractsQuery.data?.items ?? []
|
||||||
return (
|
|
||||||
<Center py="xl" data-testid="contracts-loading">
|
|
||||||
<Loader />
|
|
||||||
</Center>
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
if (contractsQuery.isError || !contractsQuery.data) {
|
|
||||||
return (
|
|
||||||
<Alert color="red" data-testid="contracts-load-error">
|
|
||||||
Failed to load contracts. Please refresh.
|
|
||||||
</Alert>
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
const contracts = contractsQuery.data.items
|
|
||||||
|
|
||||||
return (
|
return (
|
||||||
<Stack gap="lg" data-testid="contract-manager">
|
<Stack gap="lg" data-testid="contract-manager">
|
||||||
<Group justify="space-between" align="center">
|
<Group justify="space-between" align="center">
|
||||||
|
<Group gap="sm">
|
||||||
<Text fw={500}>Energy Contracts</Text>
|
<Text fw={500}>Energy Contracts</Text>
|
||||||
|
<SegmentedControl
|
||||||
|
value={scope}
|
||||||
|
onChange={(value) => handleScopeChange(value as 'electricity' | 'thermal')}
|
||||||
|
data={[{ label: 'Electricity', value: 'electricity' }, { label: 'Thermal', value: 'thermal' }]}
|
||||||
|
data-testid="contracts-scope-selector"
|
||||||
|
/>
|
||||||
|
</Group>
|
||||||
<Button onClick={() => setShowCreateForm(true)} data-testid="contract-new-button">
|
<Button onClick={() => setShowCreateForm(true)} data-testid="contract-new-button">
|
||||||
New Contract
|
New Contract
|
||||||
</Button>
|
</Button>
|
||||||
</Group>
|
</Group>
|
||||||
|
|
||||||
<ContractTable
|
{contractsQuery.isLoading && <Center py="xl" data-testid="contracts-loading"><Loader /></Center>}
|
||||||
|
{contractsQuery.isError && <Alert color="red" data-testid="contracts-load-error">Failed to load contracts. Please refresh.</Alert>}
|
||||||
|
{!contractsQuery.isLoading && !contractsQuery.isError && <ContractTable
|
||||||
contracts={contracts}
|
contracts={contracts}
|
||||||
onActivate={handleActivate}
|
onActivate={handleActivate}
|
||||||
onAddVersion={(c) => setAddVersionContract(c)}
|
onAddVersion={(c) => setAddVersionContract(c)}
|
||||||
onViewHistory={(c) => setHistoryContract(c)}
|
onViewHistory={(c) => setHistoryContract(c)}
|
||||||
activatingId={activatingId}
|
activatingId={activatingId}
|
||||||
/>
|
/>}
|
||||||
|
|
||||||
{/* Create new contract */}
|
{/* Create new contract */}
|
||||||
{showCreateForm && (
|
{showCreateForm && (
|
||||||
<ContractForm
|
<ContractForm
|
||||||
|
defaultKind={scope === 'thermal' ? 'district_heating' : undefined}
|
||||||
|
scope={scope}
|
||||||
onClose={() => setShowCreateForm(false)}
|
onClose={() => setShowCreateForm(false)}
|
||||||
onSaved={() => setShowCreateForm(false)}
|
onSaved={() => setShowCreateForm(false)}
|
||||||
/>
|
/>
|
||||||
@@ -313,6 +324,7 @@ export function ContractManager() {
|
|||||||
<ContractForm
|
<ContractForm
|
||||||
contractId={addVersionContract.id}
|
contractId={addVersionContract.id}
|
||||||
defaultKind={addVersionContract.kind}
|
defaultKind={addVersionContract.kind}
|
||||||
|
scope={scope}
|
||||||
onClose={() => setAddVersionContract(null)}
|
onClose={() => setAddVersionContract(null)}
|
||||||
onSaved={() => setAddVersionContract(null)}
|
onSaved={() => setAddVersionContract(null)}
|
||||||
/>
|
/>
|
||||||
|
|||||||
@@ -61,14 +61,89 @@ const COST_PERIOD = {
|
|||||||
|
|
||||||
const SUMMARY = {
|
const SUMMARY = {
|
||||||
currency: 'EUR',
|
currency: 'EUR',
|
||||||
|
// Money totals and kWh totals are deliberately distinct so the assertions
|
||||||
|
// below prove the cards read the *_kwh fields, not the monetary ones.
|
||||||
metered_import: 10.5,
|
metered_import: 10.5,
|
||||||
metered_export: 2.3,
|
metered_export: 2.3,
|
||||||
metered_net: 8.2,
|
metered_net: 8.2,
|
||||||
|
metered_import_kwh: 33.3,
|
||||||
|
metered_export_kwh: 44.4,
|
||||||
fixed_costs: 5.0,
|
fixed_costs: 5.0,
|
||||||
credits: 50.0,
|
credits: 50.0,
|
||||||
total_payable: 12.5,
|
total_payable: 12.5,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
const THERMAL_SUMMARY = {
|
||||||
|
currency: 'EUR', heating: '1.10', hot_water_heating: '2.20', hot_water: '3.30',
|
||||||
|
hot_water_tax: '0.40', variable_subtotal: '7.00', fixed_subtotal: '0.50', all_in: '7.50',
|
||||||
|
period_count: 4, degraded_count: 2,
|
||||||
|
fixed_breakdown: { heating_network: '0.1', metering: '0.1', delivery_set: '0', hot_water_network: '0.1', other: '0.2' },
|
||||||
|
}
|
||||||
|
const ROUNDED_THERMAL_SUMMARY = {
|
||||||
|
...THERMAL_SUMMARY,
|
||||||
|
heating: '20.12344',
|
||||||
|
hot_water_heating: '8.20005',
|
||||||
|
hot_water: '1.99995',
|
||||||
|
hot_water_tax: '0.40000',
|
||||||
|
variable_subtotal: '7.0000',
|
||||||
|
fixed_subtotal: '0.00004',
|
||||||
|
all_in: '9.99995',
|
||||||
|
fixed_breakdown: {
|
||||||
|
heating_network: '100.000000000000000001',
|
||||||
|
metering: '0.0000',
|
||||||
|
delivery_set: '20.20000',
|
||||||
|
hot_water_network: '30.30004',
|
||||||
|
other: '40.40005',
|
||||||
|
},
|
||||||
|
}
|
||||||
|
const THERMAL_VALUES = {
|
||||||
|
variable: {
|
||||||
|
heating: '20.123456789123456789', hot_water_heating: '8.200000000000000001',
|
||||||
|
hot_water: '1.234567890123456789', hot_water_tax: '0.456789012345678901',
|
||||||
|
},
|
||||||
|
standing: {
|
||||||
|
heating_network: '100.000000000000000001', metering: '0', delivery_set: '20.2',
|
||||||
|
hot_water_network: '30.3', other: '40.4',
|
||||||
|
},
|
||||||
|
}
|
||||||
|
const THERMAL_ROW = {
|
||||||
|
commodity: 'heating', period_start: '2026-06-22T10:00:00Z', period_end: '2026-06-22T10:15:00Z',
|
||||||
|
meter_id: 1, source_binding_id: 2, contract_version_id: 99, quantity: '1.2', cost: '0.123456789', currency: 'EUR',
|
||||||
|
cost_breakdown: { heating: '0.123456789' }, pricing_snapshot: THERMAL_VALUES,
|
||||||
|
quality: 'unverifiable', degraded: false, degraded_reason: null,
|
||||||
|
}
|
||||||
|
const ROUNDED_THERMAL_ROW = {
|
||||||
|
...THERMAL_ROW,
|
||||||
|
quantity: '1.23456',
|
||||||
|
cost: '0.12344',
|
||||||
|
cost_breakdown: { heating: '0.12345', hot_water: '0.10000' },
|
||||||
|
}
|
||||||
|
const THERMAL_ROW_OTHER_VERSION = {
|
||||||
|
...THERMAL_ROW, commodity: 'hot_water', period_start: '2026-06-22T10:15:00Z', period_end: '2026-06-22T10:30:00Z',
|
||||||
|
contract_version_id: 100, quantity: '2.3', cost: '4.339506172839506170',
|
||||||
|
cost_breakdown: { hot_water_heating: '1.2', hot_water: '2.8', hot_water_tax: '0.339506172839506170' },
|
||||||
|
pricing_snapshot: { ...THERMAL_VALUES, variable: { ...THERMAL_VALUES.variable, hot_water: '1.234567890123456789' } },
|
||||||
|
}
|
||||||
|
const THERMAL_DEGRADED_MISSING_CONTRACT = {
|
||||||
|
commodity: 'heating', period_start: '2026-06-22T10:30:00Z', period_end: '2026-06-22T10:45:00Z',
|
||||||
|
meter_id: 1, source_binding_id: 2, contract_version_id: null, quantity: '0', cost: '0', currency: 'EUR',
|
||||||
|
cost_breakdown: {}, pricing_snapshot: {}, quality: 'invalid', degraded: true, degraded_reason: 'missing_contract',
|
||||||
|
}
|
||||||
|
const THERMAL_DEGRADED_CROSS_EPOCH = {
|
||||||
|
commodity: 'hot_water', period_start: '2026-06-22T10:45:00Z', period_end: '2026-06-22T11:00:00Z',
|
||||||
|
meter_id: 2, source_binding_id: null, contract_version_id: null, quantity: '0', cost: '0', currency: 'EUR',
|
||||||
|
cost_breakdown: {}, pricing_snapshot: {}, quality: 'invalid', degraded: true, degraded_reason: 'cross_meter_epoch',
|
||||||
|
}
|
||||||
|
const ACTIVE_HEATING_METER = { id: 10, commodity: 'heating', ended_at: null }
|
||||||
|
const ACTIVE_HOT_WATER_METER = { id: 11, commodity: 'hot_water', ended_at: null }
|
||||||
|
const ENDED_HEATING_METER = { id: 9, commodity: 'heating', ended_at: '2026-06-01T00:00:00Z' }
|
||||||
|
const CLOSED_THERMAL_RANGE = {
|
||||||
|
startDate: '2020-01-01',
|
||||||
|
endDate: '2020-01-02',
|
||||||
|
start: '2020-01-01T00:00:00.000Z',
|
||||||
|
end: '2020-01-02T00:00:00.000Z',
|
||||||
|
}
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
// Import component
|
// Import component
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
@@ -145,9 +220,14 @@ describe('CostView', () => {
|
|||||||
expect(screen.getByTestId('summary-import')).toBeInTheDocument()
|
expect(screen.getByTestId('summary-import')).toBeInTheDocument()
|
||||||
})
|
})
|
||||||
|
|
||||||
expect(screen.getByTestId('summary-import')).toHaveTextContent('10.500')
|
// Main figure is energy (kWh), taken from the *_kwh fields.
|
||||||
expect(screen.getByTestId('summary-export')).toHaveTextContent('2.300')
|
expect(screen.getByTestId('summary-import')).toHaveTextContent('33.300')
|
||||||
|
expect(screen.getByTestId('summary-export')).toHaveTextContent('44.400')
|
||||||
expect(screen.getByTestId('summary-total')).toHaveTextContent('12.50')
|
expect(screen.getByTestId('summary-total')).toHaveTextContent('12.50')
|
||||||
|
|
||||||
|
// Sub-line carries the monetary equivalent, so money is still visible.
|
||||||
|
expect(screen.getByTestId('summary-import-sub')).toHaveTextContent('10.50 EUR')
|
||||||
|
expect(screen.getByTestId('summary-export-sub')).toHaveTextContent('2.30 EUR')
|
||||||
})
|
})
|
||||||
|
|
||||||
it('shows recompute confirmation modal on button click', async () => {
|
it('shows recompute confirmation modal on button click', async () => {
|
||||||
@@ -176,6 +256,32 @@ describe('CostView', () => {
|
|||||||
})
|
})
|
||||||
})
|
})
|
||||||
|
|
||||||
|
it('uses the same bottom-aligned toolbar structure in both scopes', async () => {
|
||||||
|
const user = userEvent.setup()
|
||||||
|
mockGet.mockImplementation((path: string) => {
|
||||||
|
if (path === '/api/energy/costs/summary') return Promise.resolve({ data: SUMMARY })
|
||||||
|
if (path === '/api/energy/meter-costs/summary') return Promise.resolve({ data: THERMAL_SUMMARY })
|
||||||
|
return Promise.resolve({ data: { items: [], total: 0 } })
|
||||||
|
})
|
||||||
|
|
||||||
|
renderWithProviders(<CostView />)
|
||||||
|
|
||||||
|
const electricityToolbar = screen.getByTestId('cost-toolbar')
|
||||||
|
expect(electricityToolbar).toContainElement(screen.getByTestId('costs-scope-selector'))
|
||||||
|
expect(electricityToolbar).toContainElement(screen.getByTestId('cost-range-control'))
|
||||||
|
expect(electricityToolbar).toContainElement(screen.getByTestId('cost-recompute-button'))
|
||||||
|
expect(screen.getByTestId('cost-recompute-button')).toHaveStyle({ marginLeft: 'auto' })
|
||||||
|
|
||||||
|
await user.click(screen.getByTestId('costs-scope-selector'))
|
||||||
|
await user.click(screen.getByText('Thermal'))
|
||||||
|
await waitFor(() => expect(screen.getByTestId('thermal-cost-toolbar')).toBeInTheDocument())
|
||||||
|
const thermalToolbar = screen.getByTestId('thermal-cost-toolbar')
|
||||||
|
expect(thermalToolbar).toContainElement(screen.getByTestId('costs-scope-selector'))
|
||||||
|
expect(thermalToolbar).toContainElement(screen.getByTestId('thermal-cost-range-control'))
|
||||||
|
expect(thermalToolbar).toContainElement(screen.getByTestId('thermal-recompute-button'))
|
||||||
|
expect(screen.getByTestId('thermal-recompute-button')).toHaveStyle({ marginLeft: 'auto' })
|
||||||
|
})
|
||||||
|
|
||||||
it('calls recompute mutation when confirmed', async () => {
|
it('calls recompute mutation when confirmed', async () => {
|
||||||
const user = userEvent.setup()
|
const user = userEvent.setup()
|
||||||
|
|
||||||
@@ -211,4 +317,180 @@ describe('CostView', () => {
|
|||||||
)
|
)
|
||||||
})
|
})
|
||||||
})
|
})
|
||||||
|
|
||||||
|
it('audits thermal rows and recomputes only a closed UTC quarter through the typed client', async () => {
|
||||||
|
const user = userEvent.setup()
|
||||||
|
mockGet.mockImplementation((path: string) => {
|
||||||
|
if (path === '/api/energy/meter-costs') return Promise.resolve({ data: { items: [THERMAL_ROW, THERMAL_ROW_OTHER_VERSION, THERMAL_DEGRADED_MISSING_CONTRACT, THERMAL_DEGRADED_CROSS_EPOCH], total: 4 } })
|
||||||
|
if (path === '/api/energy/meter-costs/summary') return Promise.resolve({ data: THERMAL_SUMMARY })
|
||||||
|
if (path === '/api/energy/costs') return Promise.resolve({ data: { items: [], total: 0 } })
|
||||||
|
if (path === '/api/energy/costs/summary') return Promise.resolve({ data: SUMMARY })
|
||||||
|
return Promise.resolve({ data: null })
|
||||||
|
})
|
||||||
|
mockPost.mockResolvedValue({ data: { processed: 1, normal: 0, degraded: 1 } })
|
||||||
|
renderWithProviders(<CostView />)
|
||||||
|
await user.click(screen.getByTestId('costs-scope-selector'))
|
||||||
|
await user.click(screen.getByText('Thermal'))
|
||||||
|
await waitFor(() => expect(screen.getByTestId('thermal-cost-summary')).toBeInTheDocument())
|
||||||
|
expect(screen.getByTestId('thermal-period-count')).toHaveTextContent('4 periods; 2 degraded')
|
||||||
|
expect(screen.getByTestId('thermal-summary-degraded')).toHaveTextContent('Expand a row to see its recorded reason')
|
||||||
|
expect(screen.queryByTestId('thermal-degraded-0')).not.toBeInTheDocument()
|
||||||
|
expect(screen.queryByTestId('thermal-degraded-1')).not.toBeInTheDocument()
|
||||||
|
expect(screen.getByTestId('thermal-degraded-2')).toHaveTextContent('missing_contract')
|
||||||
|
expect(screen.getByTestId('thermal-degraded-3')).toHaveTextContent('cross_meter_epoch')
|
||||||
|
await user.click(screen.getByTestId('thermal-cost-expand-0'))
|
||||||
|
expect(screen.getByTestId('thermal-cost-audit-0')).toHaveTextContent('Contract version: 99')
|
||||||
|
expect(screen.getByTestId('thermal-cost-audit-0')).toHaveTextContent('20.123456789123456789')
|
||||||
|
expect(screen.getByTestId('thermal-cost-audit-0')).toHaveTextContent('heating_network')
|
||||||
|
await user.click(screen.getByTestId('thermal-cost-expand-1'))
|
||||||
|
expect(screen.getByTestId('thermal-cost-audit-1')).toHaveTextContent('Contract version: 100')
|
||||||
|
expect(screen.getByTestId('thermal-cost-audit-1')).toHaveTextContent('1.234567890123456789')
|
||||||
|
expect(screen.getByTestId('thermal-cost-audit-1')).toHaveTextContent('hot_water_tax')
|
||||||
|
await user.click(screen.getByTestId('thermal-cost-expand-2'))
|
||||||
|
expect(screen.getByTestId('thermal-cost-audit-2')).toHaveTextContent('Contract version: none')
|
||||||
|
expect(screen.getByTestId('thermal-cost-audit-2')).toHaveTextContent('Pricing snapshot: {}')
|
||||||
|
expect(screen.getByTestId('thermal-fixed-breakdown')).toHaveTextContent('summary only')
|
||||||
|
expect(screen.getAllByText(/Fixed subtotal|All-in total/)).toHaveLength(2)
|
||||||
|
expect(screen.getByTestId('thermal-costs-table')).not.toHaveTextContent('Fixed')
|
||||||
|
await user.click(screen.getByTestId('thermal-cost-range-control'))
|
||||||
|
await user.click(screen.getByText('Custom'))
|
||||||
|
expect(screen.getByTestId('thermal-recompute-button')).toBeDisabled()
|
||||||
|
await user.type(screen.getByTestId('thermal-cost-custom-start'), CLOSED_THERMAL_RANGE.startDate)
|
||||||
|
await user.type(screen.getByTestId('thermal-cost-custom-end'), CLOSED_THERMAL_RANGE.endDate)
|
||||||
|
await waitFor(() => expect(screen.getByTestId('thermal-recompute-button')).toBeEnabled())
|
||||||
|
await user.click(screen.getByTestId('thermal-recompute-button'))
|
||||||
|
expect(screen.getByTestId('thermal-recompute-confirm-modal')).toHaveTextContent('closed 15-minute')
|
||||||
|
await user.click(screen.getByTestId('thermal-recompute-confirm'))
|
||||||
|
await waitFor(() => expect(mockPost).toHaveBeenCalledWith(
|
||||||
|
'/api/energy/meter-costs/recompute',
|
||||||
|
{ params: { query: { scope: 'thermal', start: CLOSED_THERMAL_RANGE.start, end: CLOSED_THERMAL_RANGE.end } } },
|
||||||
|
))
|
||||||
|
expect(new Date(CLOSED_THERMAL_RANGE.end).getUTCMinutes() % 15).toBe(0)
|
||||||
|
})
|
||||||
|
|
||||||
|
it('keeps the thermal confirmation cancellable and surfaces a 422 recompute failure', async () => {
|
||||||
|
const user = userEvent.setup()
|
||||||
|
mockGet.mockImplementation((path: string) => {
|
||||||
|
if (path === '/api/energy/meter-costs') return Promise.resolve({ data: { items: [THERMAL_ROW], total: 1 } })
|
||||||
|
if (path === '/api/energy/meter-costs/summary') return Promise.resolve({ data: THERMAL_SUMMARY })
|
||||||
|
if (path === '/api/energy/costs/summary') return Promise.resolve({ data: SUMMARY })
|
||||||
|
return Promise.resolve({ data: { items: [], total: 0 } })
|
||||||
|
})
|
||||||
|
mockPost.mockRejectedValue(new Error('422'))
|
||||||
|
renderWithProviders(<CostView />)
|
||||||
|
await user.click(screen.getByTestId('costs-scope-selector')); await user.click(screen.getByText('Thermal'))
|
||||||
|
await waitFor(() => expect(screen.getByTestId('thermal-recompute-button')).toBeInTheDocument())
|
||||||
|
await user.click(screen.getByTestId('thermal-cost-range-control')); await user.click(screen.getByText('Custom'))
|
||||||
|
expect(screen.getByTestId('thermal-recompute-button')).toBeDisabled()
|
||||||
|
await user.type(screen.getByTestId('thermal-cost-custom-start'), CLOSED_THERMAL_RANGE.startDate)
|
||||||
|
await user.type(screen.getByTestId('thermal-cost-custom-end'), CLOSED_THERMAL_RANGE.endDate)
|
||||||
|
await waitFor(() => expect(screen.getByTestId('thermal-recompute-button')).toBeEnabled())
|
||||||
|
await user.click(screen.getByTestId('thermal-recompute-button')); await user.click(screen.getByTestId('thermal-recompute-cancel'))
|
||||||
|
expect(screen.queryByTestId('thermal-recompute-confirm-modal')).not.toBeInTheDocument()
|
||||||
|
await user.click(screen.getByTestId('thermal-recompute-button')); await user.click(screen.getByTestId('thermal-recompute-confirm'))
|
||||||
|
await waitFor(() => expect(mockPost).toHaveBeenCalledWith(
|
||||||
|
'/api/energy/meter-costs/recompute',
|
||||||
|
{ params: { query: { scope: 'thermal', start: CLOSED_THERMAL_RANGE.start, end: CLOSED_THERMAL_RANGE.end } } },
|
||||||
|
))
|
||||||
|
await waitFor(() => expect(screen.getByTestId('thermal-recompute-error')).toBeInTheDocument())
|
||||||
|
})
|
||||||
|
|
||||||
|
it.each([
|
||||||
|
['only heating', [ACTIVE_HEATING_METER], 'hot-water meter is not configured', 'Not configured'],
|
||||||
|
['only hot water', [ACTIVE_HOT_WATER_METER], 'heating meter is not configured', 'Not configured'],
|
||||||
|
['both current meters', [ACTIVE_HEATING_METER, ACTIVE_HOT_WATER_METER], null, '1.1'],
|
||||||
|
['a replaced heating meter plus its current epoch', [ENDED_HEATING_METER, ACTIVE_HEATING_METER, ACTIVE_HOT_WATER_METER], null, '1.1'],
|
||||||
|
])('uses active meter epochs for %s without treating zero amounts as missing', async (_name, meterItems, missingText, heatingValue) => {
|
||||||
|
mockGet.mockImplementation((path: string) => {
|
||||||
|
if (path === '/api/energy/meters') return Promise.resolve({ data: { items: meterItems, total: meterItems.length } })
|
||||||
|
if (path === '/api/energy/meter-costs') return Promise.resolve({ data: { items: [THERMAL_ROW], total: 1 } })
|
||||||
|
if (path === '/api/energy/meter-costs/summary') return Promise.resolve({ data: { ...THERMAL_SUMMARY, heating: heatingValue === 'Not configured' ? '0' : THERMAL_SUMMARY.heating } })
|
||||||
|
if (path === '/api/energy/costs/summary') return Promise.resolve({ data: SUMMARY })
|
||||||
|
return Promise.resolve({ data: { items: [], total: 0 } })
|
||||||
|
})
|
||||||
|
const user = userEvent.setup()
|
||||||
|
renderWithProviders(<CostView />)
|
||||||
|
await user.click(screen.getByTestId('costs-scope-selector'))
|
||||||
|
await user.click(screen.getByText('Thermal'))
|
||||||
|
await waitFor(() => expect(screen.getByTestId('thermal-cost-summary')).toBeInTheDocument())
|
||||||
|
if (missingText) {
|
||||||
|
expect(screen.getByTestId('thermal-missing-current-meter')).toHaveTextContent(missingText)
|
||||||
|
expect(screen.getByTestId('thermal-cost-summary')).toHaveTextContent('Not configured')
|
||||||
|
} else {
|
||||||
|
expect(screen.queryByTestId('thermal-missing-current-meter')).not.toBeInTheDocument()
|
||||||
|
expect(screen.getByTestId('thermal-cost-summary')).toHaveTextContent(heatingValue)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
it('rounds ordinary thermal Decimal strings without changing the audit snapshot', async () => {
|
||||||
|
const user = userEvent.setup()
|
||||||
|
mockGet.mockImplementation((path: string) => {
|
||||||
|
if (path === '/api/energy/meters') {
|
||||||
|
return Promise.resolve({ data: { items: [ACTIVE_HEATING_METER, ACTIVE_HOT_WATER_METER], total: 2 } })
|
||||||
|
}
|
||||||
|
if (path === '/api/energy/meter-costs') {
|
||||||
|
return Promise.resolve({ data: { items: [ROUNDED_THERMAL_ROW], total: 1 } })
|
||||||
|
}
|
||||||
|
if (path === '/api/energy/meter-costs/summary') {
|
||||||
|
return Promise.resolve({ data: ROUNDED_THERMAL_SUMMARY })
|
||||||
|
}
|
||||||
|
if (path === '/api/energy/costs/summary') return Promise.resolve({ data: SUMMARY })
|
||||||
|
return Promise.resolve({ data: { items: [], total: 0 } })
|
||||||
|
})
|
||||||
|
|
||||||
|
renderWithProviders(<CostView />)
|
||||||
|
await user.click(screen.getByTestId('costs-scope-selector'))
|
||||||
|
await user.click(screen.getByText('Thermal'))
|
||||||
|
await waitFor(() => expect(screen.getByTestId('thermal-cost-summary')).toBeInTheDocument())
|
||||||
|
|
||||||
|
expect(screen.getByTestId('thermal-cost-summary')).toHaveTextContent('Heating20.1234')
|
||||||
|
expect(screen.getByTestId('thermal-cost-summary')).toHaveTextContent('Hot-water heating8.2001')
|
||||||
|
expect(screen.getByTestId('thermal-cost-summary')).toHaveTextContent('Hot water2')
|
||||||
|
expect(screen.getByTestId('thermal-cost-summary')).toHaveTextContent('Hot-water tax0.4')
|
||||||
|
expect(screen.getByTestId('thermal-cost-summary')).toHaveTextContent('Variable subtotal7')
|
||||||
|
expect(screen.getByTestId('thermal-cost-summary')).toHaveTextContent('Fixed subtotal0')
|
||||||
|
expect(screen.getByTestId('thermal-cost-summary')).toHaveTextContent('All-in total10')
|
||||||
|
expect(screen.getByTestId('thermal-fixed-breakdown')).toHaveTextContent('heating_network 100')
|
||||||
|
expect(screen.getByTestId('thermal-fixed-breakdown')).toHaveTextContent('other 40.4001')
|
||||||
|
expect(screen.getByTestId('thermal-cost-row-0')).toHaveTextContent('1.2346')
|
||||||
|
expect(screen.getByTestId('thermal-cost-row-0')).toHaveTextContent('0.1234 EUR')
|
||||||
|
expect(screen.getByTestId('thermal-cost-row-0')).toHaveTextContent('heating 0.1235, hot_water 0.1')
|
||||||
|
|
||||||
|
await user.click(screen.getByTestId('thermal-cost-expand-0'))
|
||||||
|
expect(screen.getByTestId('thermal-cost-audit-0')).toHaveTextContent('20.123456789123456789')
|
||||||
|
})
|
||||||
|
|
||||||
|
it('paginates the complete thermal ledger and resets offset when its range or scope changes', async () => {
|
||||||
|
const user = userEvent.setup()
|
||||||
|
const lastRow = { ...THERMAL_ROW, period_start: '2026-06-22T12:00:00Z', quantity: '501' }
|
||||||
|
mockGet.mockImplementation((path: string, options?: { params?: { query?: { offset?: number } } }) => {
|
||||||
|
if (path === '/api/energy/meters') return Promise.resolve({ data: { items: [ACTIVE_HEATING_METER, ACTIVE_HOT_WATER_METER], total: 2 } })
|
||||||
|
if (path === '/api/energy/meter-costs') {
|
||||||
|
const offset = options?.params?.query?.offset ?? 0
|
||||||
|
return Promise.resolve({ data: offset === 0 ? { items: Array.from({ length: 500 }, () => THERMAL_ROW), total: 501 } : { items: [lastRow], total: 501 } })
|
||||||
|
}
|
||||||
|
if (path === '/api/energy/meter-costs/summary') return Promise.resolve({ data: THERMAL_SUMMARY })
|
||||||
|
if (path === '/api/energy/costs/summary') return Promise.resolve({ data: SUMMARY })
|
||||||
|
return Promise.resolve({ data: { items: [], total: 0 } })
|
||||||
|
})
|
||||||
|
renderWithProviders(<CostView />)
|
||||||
|
await user.click(screen.getByTestId('costs-scope-selector'))
|
||||||
|
await user.click(screen.getByText('Thermal'))
|
||||||
|
await waitFor(() => expect(screen.getByTestId('thermal-ledger-count')).toHaveTextContent('Showing 1-500 of 501'))
|
||||||
|
expect(screen.getByTestId('thermal-ledger-prev')).toBeDisabled()
|
||||||
|
expect(screen.getByTestId('thermal-ledger-next')).toBeEnabled()
|
||||||
|
await user.click(screen.getByTestId('thermal-ledger-next'))
|
||||||
|
await waitFor(() => expect(screen.getByTestId('thermal-ledger-count')).toHaveTextContent('Showing 501-501 of 501'))
|
||||||
|
expect(screen.getByTestId('thermal-ledger-prev')).toBeEnabled()
|
||||||
|
expect(screen.getByTestId('thermal-ledger-next')).toBeDisabled()
|
||||||
|
expect(mockGet).toHaveBeenCalledWith('/api/energy/meter-costs', expect.objectContaining({ params: { query: expect.objectContaining({ scope: 'thermal', offset: 500, limit: 500 }) } }))
|
||||||
|
await user.click(screen.getByTestId('thermal-cost-range-control'))
|
||||||
|
await user.click(screen.getByText('This month'))
|
||||||
|
await waitFor(() => expect(screen.getByTestId('thermal-ledger-count')).toHaveTextContent('Showing 1-500 of 501'))
|
||||||
|
await user.click(screen.getByTestId('costs-scope-selector'))
|
||||||
|
await user.click(screen.getByText('Electricity'))
|
||||||
|
await user.click(screen.getByTestId('costs-scope-selector'))
|
||||||
|
await user.click(screen.getByText('Thermal'))
|
||||||
|
await waitFor(() => expect(screen.getByTestId('thermal-ledger-count')).toHaveTextContent('Showing 1-500 of 501'))
|
||||||
|
})
|
||||||
})
|
})
|
||||||
|
|||||||
@@ -13,6 +13,7 @@
|
|||||||
*/
|
*/
|
||||||
|
|
||||||
import { useState } from 'react'
|
import { useState } from 'react'
|
||||||
|
import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query'
|
||||||
import {
|
import {
|
||||||
Stack,
|
Stack,
|
||||||
Text,
|
Text,
|
||||||
@@ -43,6 +44,7 @@ import {
|
|||||||
} from 'recharts'
|
} from 'recharts'
|
||||||
import { useEnergyCosts, useEnergyCostSummary, useRecomputeCosts } from './hooks'
|
import { useEnergyCosts, useEnergyCostSummary, useRecomputeCosts } from './hooks'
|
||||||
import { formatLocalTime } from '../utils/datetime'
|
import { formatLocalTime } from '../utils/datetime'
|
||||||
|
import apiClient from '../api/client'
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
// Cost limit — prevent accidental full-table pulls
|
// Cost limit — prevent accidental full-table pulls
|
||||||
@@ -77,10 +79,12 @@ function getThisMonthRange(): { start: string; end: string } {
|
|||||||
interface SummaryCardProps {
|
interface SummaryCardProps {
|
||||||
label: string
|
label: string
|
||||||
value: string
|
value: string
|
||||||
|
/** Optional secondary line, e.g. the monetary equivalent of an energy figure. */
|
||||||
|
sub?: string
|
||||||
testId?: string
|
testId?: string
|
||||||
}
|
}
|
||||||
|
|
||||||
function SummaryCard({ label, value, testId }: SummaryCardProps) {
|
function SummaryCard({ label, value, sub, testId }: SummaryCardProps) {
|
||||||
return (
|
return (
|
||||||
<Paper withBorder p="sm" data-testid={testId}>
|
<Paper withBorder p="sm" data-testid={testId}>
|
||||||
<Stack gap={4}>
|
<Stack gap={4}>
|
||||||
@@ -90,6 +94,11 @@ function SummaryCard({ label, value, testId }: SummaryCardProps) {
|
|||||||
<Text fw={600} size="lg">
|
<Text fw={600} size="lg">
|
||||||
{value}
|
{value}
|
||||||
</Text>
|
</Text>
|
||||||
|
{sub !== undefined && (
|
||||||
|
<Text size="xs" c="dimmed" data-testid={testId ? `${testId}-sub` : undefined}>
|
||||||
|
{sub}
|
||||||
|
</Text>
|
||||||
|
)}
|
||||||
</Stack>
|
</Stack>
|
||||||
</Paper>
|
</Paper>
|
||||||
)
|
)
|
||||||
@@ -101,7 +110,60 @@ function SummaryCard({ label, value, testId }: SummaryCardProps) {
|
|||||||
|
|
||||||
type RangePreset = 'today' | 'month' | 'custom'
|
type RangePreset = 'today' | 'month' | 'custom'
|
||||||
|
|
||||||
|
/** Format an API Decimal string without converting it through binary floating point. */
|
||||||
|
function formatDecimal(value: string): string {
|
||||||
|
const negative = value.startsWith('-')
|
||||||
|
const unsigned = negative ? value.slice(1) : value
|
||||||
|
const [rawWhole = '0', rawFraction = ''] = unsigned.split('.')
|
||||||
|
let whole = rawWhole.replace(/^0+(?=\d)/, '') || '0'
|
||||||
|
let fraction = rawFraction.slice(0, 4)
|
||||||
|
|
||||||
|
if (rawFraction.length > 4 && rawFraction[4] >= '5') {
|
||||||
|
const digits = '0123456789'
|
||||||
|
const fractionDigits = fraction.split('')
|
||||||
|
let carry = true
|
||||||
|
for (let index = fractionDigits.length - 1; index >= 0 && carry; index -= 1) {
|
||||||
|
const digit = fractionDigits[index]
|
||||||
|
if (digit === '9') {
|
||||||
|
fractionDigits[index] = '0'
|
||||||
|
} else {
|
||||||
|
fractionDigits[index] = digits[digits.indexOf(digit) + 1]
|
||||||
|
carry = false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
fraction = fractionDigits.join('')
|
||||||
|
|
||||||
|
if (carry) {
|
||||||
|
const wholeDigits = whole.split('')
|
||||||
|
for (let index = wholeDigits.length - 1; index >= 0 && carry; index -= 1) {
|
||||||
|
const digit = wholeDigits[index]
|
||||||
|
if (digit === '9') {
|
||||||
|
wholeDigits[index] = '0'
|
||||||
|
} else {
|
||||||
|
wholeDigits[index] = digits[digits.indexOf(digit) + 1]
|
||||||
|
carry = false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
whole = `${carry ? '1' : ''}${wholeDigits.join('')}`
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fraction = fraction.replace(/0+$/, '')
|
||||||
|
const formatted = fraction ? `${whole}.${fraction}` : whole
|
||||||
|
return negative && formatted !== '0' ? `-${formatted}` : formatted
|
||||||
|
}
|
||||||
|
|
||||||
export function CostView() {
|
export function CostView() {
|
||||||
|
const [scope, setScope] = useState<'electricity' | 'thermal'>('electricity')
|
||||||
|
if (scope === 'thermal') return <ThermalCostView onScopeChange={setScope} />
|
||||||
|
return <ElectricityCostView onScopeChange={setScope} />
|
||||||
|
}
|
||||||
|
|
||||||
|
function ScopeSelector({ scope, onScopeChange }: { scope: 'electricity' | 'thermal'; onScopeChange: (scope: 'electricity' | 'thermal') => void }) {
|
||||||
|
return <SegmentedControl value={scope} onChange={(value) => onScopeChange(value as 'electricity' | 'thermal')} data={[{ label: 'Electricity', value: 'electricity' }, { label: 'Thermal', value: 'thermal' }]} data-testid="costs-scope-selector" />
|
||||||
|
}
|
||||||
|
|
||||||
|
function ElectricityCostView({ onScopeChange }: { onScopeChange: (scope: 'electricity' | 'thermal') => void }) {
|
||||||
const [rangePreset, setRangePreset] = useState<RangePreset>('today')
|
const [rangePreset, setRangePreset] = useState<RangePreset>('today')
|
||||||
// Date strings in YYYY-MM-DD format for custom range
|
// Date strings in YYYY-MM-DD format for custom range
|
||||||
const [customStartStr, setCustomStartStr] = useState('')
|
const [customStartStr, setCustomStartStr] = useState('')
|
||||||
@@ -136,7 +198,8 @@ export function CostView() {
|
|||||||
return (
|
return (
|
||||||
<Stack gap="lg" data-testid="cost-view">
|
<Stack gap="lg" data-testid="cost-view">
|
||||||
{/* Date range selector */}
|
{/* Date range selector */}
|
||||||
<Group align="flex-start" gap="md" wrap="wrap">
|
<Group align="flex-end" gap="md" wrap="wrap" data-testid="cost-toolbar">
|
||||||
|
<ScopeSelector scope="electricity" onScopeChange={onScopeChange} />
|
||||||
<Stack gap="xs">
|
<Stack gap="xs">
|
||||||
<Text size="sm" fw={500}>
|
<Text size="sm" fw={500}>
|
||||||
Date range
|
Date range
|
||||||
@@ -172,19 +235,18 @@ export function CostView() {
|
|||||||
</Group>
|
</Group>
|
||||||
)}
|
)}
|
||||||
|
|
||||||
<Group gap="sm" style={{ marginLeft: 'auto' }} align="flex-end">
|
|
||||||
<Button
|
<Button
|
||||||
variant="outline"
|
variant="outline"
|
||||||
color="orange"
|
color="orange"
|
||||||
size="sm"
|
size="sm"
|
||||||
onClick={() => setShowRecomputeConfirm(true)}
|
onClick={() => setShowRecomputeConfirm(true)}
|
||||||
loading={recomputeMutation.isPending}
|
loading={recomputeMutation.isPending}
|
||||||
|
style={{ marginLeft: 'auto' }}
|
||||||
data-testid="cost-recompute-button"
|
data-testid="cost-recompute-button"
|
||||||
>
|
>
|
||||||
Recompute
|
Recompute
|
||||||
</Button>
|
</Button>
|
||||||
</Group>
|
</Group>
|
||||||
</Group>
|
|
||||||
|
|
||||||
{/* Summary cards */}
|
{/* Summary cards */}
|
||||||
{summaryQuery.isLoading && (
|
{summaryQuery.isLoading && (
|
||||||
@@ -207,12 +269,14 @@ export function CostView() {
|
|||||||
<SimpleGrid cols={{ base: 2, sm: 3 }} spacing="sm" data-testid="cost-summary">
|
<SimpleGrid cols={{ base: 2, sm: 3 }} spacing="sm" data-testid="cost-summary">
|
||||||
<SummaryCard
|
<SummaryCard
|
||||||
label="Import (kWh)"
|
label="Import (kWh)"
|
||||||
value={summaryQuery.data.metered_import.toFixed(3)}
|
value={summaryQuery.data.metered_import_kwh.toFixed(3)}
|
||||||
|
sub={`${summaryQuery.data.metered_import.toFixed(2)} ${currency}`}
|
||||||
testId="summary-import"
|
testId="summary-import"
|
||||||
/>
|
/>
|
||||||
<SummaryCard
|
<SummaryCard
|
||||||
label="Export (kWh)"
|
label="Export (kWh)"
|
||||||
value={summaryQuery.data.metered_export.toFixed(3)}
|
value={summaryQuery.data.metered_export_kwh.toFixed(3)}
|
||||||
|
sub={`${summaryQuery.data.metered_export.toFixed(2)} ${currency}`}
|
||||||
testId="summary-export"
|
testId="summary-export"
|
||||||
/>
|
/>
|
||||||
<SummaryCard
|
<SummaryCard
|
||||||
@@ -401,3 +465,76 @@ export function CostView() {
|
|||||||
</Stack>
|
</Stack>
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function ThermalCostView({ onScopeChange }: { onScopeChange: (scope: 'electricity' | 'thermal') => void }) {
|
||||||
|
const [rangePreset, setRangePreset] = useState<RangePreset>('today')
|
||||||
|
const [customStartStr, setCustomStartStr] = useState('')
|
||||||
|
const [customEndStr, setCustomEndStr] = useState('')
|
||||||
|
const [showConfirm, setShowConfirm] = useState(false)
|
||||||
|
const [recomputeError, setRecomputeError] = useState<string | null>(null)
|
||||||
|
const [recomputeSuccess, setRecomputeSuccess] = useState<string | null>(null)
|
||||||
|
const [expandedRows, setExpandedRows] = useState<Set<number>>(() => new Set())
|
||||||
|
const [ledgerOffset, setLedgerOffset] = useState(0)
|
||||||
|
const { start, end } = (() => {
|
||||||
|
if (rangePreset === 'today') return getTodayRange()
|
||||||
|
if (rangePreset === 'month') return getThisMonthRange()
|
||||||
|
return {
|
||||||
|
start: customStartStr ? new Date(customStartStr).toISOString() : undefined,
|
||||||
|
end: customEndStr ? new Date(customEndStr).toISOString() : undefined,
|
||||||
|
}
|
||||||
|
})()
|
||||||
|
// The server only accepts complete UTC quarters. Never send a future end.
|
||||||
|
const closedEnd = (() => {
|
||||||
|
const now = new Date()
|
||||||
|
now.setUTCMinutes(Math.floor(now.getUTCMinutes() / 15) * 15, 0, 0)
|
||||||
|
const selectedEnd = end ? new Date(end) : now
|
||||||
|
return new Date(Math.min(selectedEnd.getTime(), now.getTime())).toISOString()
|
||||||
|
})()
|
||||||
|
const recomputeStart = start
|
||||||
|
const recomputeAvailable = !!recomputeStart && new Date(recomputeStart) < new Date(closedEnd)
|
||||||
|
const qc = useQueryClient()
|
||||||
|
const resetLedgerPage = () => {
|
||||||
|
setLedgerOffset(0)
|
||||||
|
setExpandedRows(new Set())
|
||||||
|
}
|
||||||
|
const rows = useQuery({ queryKey: ['meter-costs', 'thermal', start, end, ledgerOffset], queryFn: async () => {
|
||||||
|
const result = await apiClient.GET('/api/energy/meter-costs', { params: { query: { scope: 'thermal', start, end, limit: COSTS_MAX_LIMIT, offset: ledgerOffset } } })
|
||||||
|
return result.data
|
||||||
|
} })
|
||||||
|
const meters = useQuery({ queryKey: ['energy-meters', 'thermal'], queryFn: async () => {
|
||||||
|
const result = await apiClient.GET('/api/energy/meters')
|
||||||
|
return result.data
|
||||||
|
} })
|
||||||
|
const summary = useQuery({ queryKey: ['meter-cost-summary', 'thermal', start, end], queryFn: async () => {
|
||||||
|
const result = await apiClient.GET('/api/energy/meter-costs/summary', { params: { query: { scope: 'thermal', start, end } } })
|
||||||
|
return result.data
|
||||||
|
} })
|
||||||
|
const recompute = useMutation({ mutationFn: () => apiClient.POST('/api/energy/meter-costs/recompute', { params: { query: { scope: 'thermal', start: recomputeStart!, end: closedEnd } } }), onSuccess: (result) => {
|
||||||
|
void qc.invalidateQueries({ queryKey: ['meter-costs', 'thermal'] }); void qc.invalidateQueries({ queryKey: ['meter-cost-summary', 'thermal'] })
|
||||||
|
setRecomputeSuccess(`Recomputed ${result.data?.processed ?? 0} closed periods.`)
|
||||||
|
} })
|
||||||
|
const currency = summary.data?.currency ?? rows.data?.items[0]?.currency ?? 'EUR'
|
||||||
|
const fixed = summary.data?.fixed_breakdown
|
||||||
|
const hasCurrentHeatingMeter = meters.data?.items.some((meter) => meter.commodity === 'heating' && meter.ended_at === null)
|
||||||
|
const hasCurrentHotWaterMeter = meters.data?.items.some((meter) => meter.commodity === 'hot_water' && meter.ended_at === null)
|
||||||
|
const missingCurrentMeters = [
|
||||||
|
...(hasCurrentHeatingMeter === false ? ['heating'] : []),
|
||||||
|
...(hasCurrentHotWaterMeter === false ? ['hot-water'] : []),
|
||||||
|
]
|
||||||
|
const totalRows = rows.data?.total ?? 0
|
||||||
|
const shownStart = totalRows === 0 ? 0 : ledgerOffset + 1
|
||||||
|
const shownEnd = Math.min(ledgerOffset + (rows.data?.items.length ?? 0), totalRows)
|
||||||
|
return <Stack gap="lg" data-testid="thermal-cost-view">
|
||||||
|
<Group align="flex-end" gap="md" wrap="wrap" data-testid="thermal-cost-toolbar"><ScopeSelector scope="thermal" onScopeChange={onScopeChange} /><Stack gap="xs"><Text size="sm" fw={500}>Date range</Text><SegmentedControl value={rangePreset} onChange={(value) => { resetLedgerPage(); setRangePreset(value as RangePreset) }} data={[{ label: 'Today', value: 'today' }, { label: 'This month', value: 'month' }, { label: 'Custom', value: 'custom' }]} data-testid="thermal-cost-range-control" /></Stack>{rangePreset === 'custom' && <Group gap="sm" align="flex-end"><TextInput label="From" type="date" value={customStartStr} onChange={(event) => { resetLedgerPage(); setCustomStartStr(event.currentTarget.value) }} data-testid="thermal-cost-custom-start" /><TextInput label="To" type="date" value={customEndStr} onChange={(event) => { resetLedgerPage(); setCustomEndStr(event.currentTarget.value) }} data-testid="thermal-cost-custom-end" /></Group>}<Button variant="outline" color="orange" onClick={() => { setRecomputeError(null); setRecomputeSuccess(null); setShowConfirm(true) }} disabled={!recomputeAvailable} style={{ marginLeft: 'auto' }} data-testid="thermal-recompute-button">Recompute</Button></Group>
|
||||||
|
{(rows.isLoading || summary.isLoading) && <Center><Loader size="sm" /></Center>}
|
||||||
|
{(rows.isError || summary.isError) && <Alert color="red">Failed to load thermal costs.</Alert>}
|
||||||
|
{recomputeError && <Alert color="red" data-testid="thermal-recompute-error">{recomputeError}</Alert>}
|
||||||
|
{recomputeSuccess && <Alert color="green" data-testid="thermal-recompute-success">{recomputeSuccess}</Alert>}
|
||||||
|
{summary.data && <Stack gap="xs" data-testid="thermal-cost-summary"><Title order={6}>{rangePreset === 'today' ? 'Today' : rangePreset === 'month' ? 'This month' : 'Custom range'} ({currency})</Title><Text size="sm" data-testid="thermal-cost-range">{start ?? 'Select a start date'} — {end ?? 'Select an end date'}</Text><SimpleGrid cols={{ base: 2, sm: 3 }}>
|
||||||
|
<SummaryCard label="Heating" value={hasCurrentHeatingMeter === false ? 'Not configured' : formatDecimal(summary.data.heating)} /><SummaryCard label="Hot-water heating" value={hasCurrentHotWaterMeter === false ? 'Not configured' : formatDecimal(summary.data.hot_water_heating)} /><SummaryCard label="Hot water" value={hasCurrentHotWaterMeter === false ? 'Not configured' : formatDecimal(summary.data.hot_water)} /><SummaryCard label="Hot-water tax" value={hasCurrentHotWaterMeter === false ? 'Not configured' : formatDecimal(summary.data.hot_water_tax)} /><SummaryCard label="Variable subtotal" value={formatDecimal(summary.data.variable_subtotal)} /><SummaryCard label="Fixed subtotal" value={formatDecimal(summary.data.fixed_subtotal)} /><SummaryCard label="All-in total" value={formatDecimal(summary.data.all_in)} />
|
||||||
|
</SimpleGrid>{missingCurrentMeters.length > 0 && <Alert color="yellow" data-testid="thermal-missing-current-meter">Current {missingCurrentMeters.join(' and ')} meter{missingCurrentMeters.length > 1 ? 's are' : ' is'} not configured. Historical ledger rows do not establish a current meter.</Alert>}<Text size="sm" data-testid="thermal-period-count">{summary.data.period_count} periods; {summary.data.degraded_count} degraded</Text>{summary.data.degraded_count > 0 && <Alert color="orange" data-testid="thermal-summary-degraded">Some totals include degraded periods. Expand a row to see its recorded reason.</Alert>}{fixed && <Text size="sm" data-testid="thermal-fixed-breakdown">Fixed once per settled local day (summary only): {Object.entries(fixed).map(([key, value]) => `${key} ${formatDecimal(value)}`).join(' · ')}</Text>}</Stack>}
|
||||||
|
{rows.data?.items.length === 0 && <Alert color="gray" data-testid="thermal-costs-empty">No thermal cost data for this range. Check that heating or hot-water meters are bound and have settled readings.</Alert>}
|
||||||
|
{rows.data && <Stack gap="xs"><Text size="sm" c="dimmed" data-testid="thermal-ledger-count">Showing {shownStart}-{shownEnd} of {totalRows}</Text>{rows.data.items.length > 0 && <ScrollArea><Table striped withTableBorder data-testid="thermal-costs-table"><Table.Thead><Table.Tr><Table.Th>Time</Table.Th><Table.Th>Commodity</Table.Th><Table.Th>Quantity</Table.Th><Table.Th>Cost</Table.Th><Table.Th>Breakdown</Table.Th><Table.Th>Status</Table.Th><Table.Th></Table.Th></Table.Tr></Table.Thead><Table.Tbody>{rows.data.items.flatMap((item, index) => [<Table.Tr key={`${item.commodity}-${item.period_start}`} data-testid={`thermal-cost-row-${index}`}><Table.Td>{formatLocalTime(item.period_start)}</Table.Td><Table.Td>{item.commodity}</Table.Td><Table.Td>{formatDecimal(item.quantity)}</Table.Td><Table.Td>{formatDecimal(item.cost)} {item.currency}</Table.Td><Table.Td>{Object.entries(item.cost_breakdown).map(([key, value]) => `${key} ${formatDecimal(value)}`).join(', ')}</Table.Td><Table.Td>{item.degraded ? <Badge color="orange" data-testid={`thermal-degraded-${index}`}>{item.degraded_reason ?? 'degraded'}</Badge> : 'normal'}</Table.Td><Table.Td><Button size="xs" variant="subtle" onClick={() => setExpandedRows((current) => { const next = new Set(current); if (next.has(index)) next.delete(index); else next.add(index); return next })} data-testid={`thermal-cost-expand-${index}`}>{expandedRows.has(index) ? 'Hide audit' : 'Audit'}</Button></Table.Td></Table.Tr>, ...(expandedRows.has(index) ? [<Table.Tr key={`audit-${item.commodity}-${item.period_start}`} data-testid={`thermal-cost-audit-${index}`}><Table.Td colSpan={7}><Text size="xs">Contract version: {item.contract_version_id ?? 'none'}</Text><Text size="xs">Pricing snapshot: {JSON.stringify(item.pricing_snapshot)}</Text></Table.Td></Table.Tr>] : [])])}</Table.Tbody></Table></ScrollArea>}<Group justify="flex-end"><Button size="xs" variant="default" disabled={ledgerOffset === 0} onClick={() => { setExpandedRows(new Set()); setLedgerOffset((current) => Math.max(0, current - COSTS_MAX_LIMIT)) }} data-testid="thermal-ledger-prev">Previous</Button><Button size="xs" variant="default" disabled={ledgerOffset + (rows.data.items.length ?? 0) >= totalRows} onClick={() => { setExpandedRows(new Set()); setLedgerOffset((current) => current + COSTS_MAX_LIMIT) }} data-testid="thermal-ledger-next">Next</Button></Group></Stack>}
|
||||||
|
{showConfirm && <Modal opened onClose={() => setShowConfirm(false)} title="Recompute thermal costs?" data-testid="thermal-recompute-confirm-modal"><Stack><Text>This explicitly overwrites closed 15-minute thermal ledger rows for {recomputeStart ?? 'the selected start'} — {closedEnd}. Continue?</Text>{!recomputeAvailable && <Alert color="yellow">Select a range containing at least one closed UTC quarter.</Alert>}<Group justify="flex-end"><Button variant="default" onClick={() => setShowConfirm(false)} data-testid="thermal-recompute-cancel">Cancel</Button><Button color="orange" loading={recompute.isPending} disabled={!recomputeAvailable} onClick={async () => { try { await recompute.mutateAsync(); setShowConfirm(false) } catch { setRecomputeError('Failed to recompute thermal costs. Please try again.'); setShowConfirm(false) } }} data-testid="thermal-recompute-confirm">Recompute</Button></Group></Stack></Modal>}
|
||||||
|
</Stack>
|
||||||
|
}
|
||||||
|
|||||||
@@ -50,6 +50,9 @@ describe('DsmrPanel', () => {
|
|||||||
renderWithProviders(<DsmrPanel />)
|
renderWithProviders(<DsmrPanel />)
|
||||||
await waitFor(() => expect(screen.getByTestId('dsmr-empty')).toBeInTheDocument())
|
await waitFor(() => expect(screen.getByTestId('dsmr-empty')).toBeInTheDocument())
|
||||||
expect(screen.queryByTestId('dsmr-table')).not.toBeInTheDocument()
|
expect(screen.queryByTestId('dsmr-table')).not.toBeInTheDocument()
|
||||||
|
expect(screen.getByText(/In this DSMR Source, enable or edit the broker, topic, and profile/)).toBeInTheDocument()
|
||||||
|
expect(screen.getByText(/confirm the publisher is sending/)).toBeInTheDocument()
|
||||||
|
expect(screen.queryByText(/Enable DSMR ingest.*Config/)).not.toBeInTheDocument()
|
||||||
})
|
})
|
||||||
|
|
||||||
it('renders the latest telegram as a key/value table; null shown as dash', async () => {
|
it('renders the latest telegram as a key/value table; null shown as dash', async () => {
|
||||||
@@ -79,4 +82,11 @@ describe('DsmrPanel', () => {
|
|||||||
renderWithProviders(<DsmrPanel />)
|
renderWithProviders(<DsmrPanel />)
|
||||||
await waitFor(() => expect(screen.getByTestId('dsmr-error')).toBeInTheDocument())
|
await waitFor(() => expect(screen.getByTestId('dsmr-error')).toBeInTheDocument())
|
||||||
})
|
})
|
||||||
|
|
||||||
|
it('keeps the compatibility endpoint available for DSMR source detail', async () => {
|
||||||
|
mockGet.mockResolvedValue({ data: { found: true, recorded_at: '2026-06-23T12:16:00Z', payload: { tariff: 'low' } } })
|
||||||
|
renderWithProviders(<DsmrPanel />)
|
||||||
|
await waitFor(() => expect(mockGet).toHaveBeenCalledWith('/api/energy/dsmr/latest'))
|
||||||
|
expect(await screen.findByText('Latest DSMR reading (compatibility view)')).toBeInTheDocument()
|
||||||
|
})
|
||||||
})
|
})
|
||||||
|
|||||||
@@ -47,7 +47,7 @@ export function DsmrPanel() {
|
|||||||
<Stack gap="md" data-testid="dsmr-panel">
|
<Stack gap="md" data-testid="dsmr-panel">
|
||||||
<Group justify="space-between" align="center">
|
<Group justify="space-between" align="center">
|
||||||
<div>
|
<div>
|
||||||
<Text fw={600}>Latest DSMR reading</Text>
|
<Text fw={600}>Latest DSMR reading (compatibility view)</Text>
|
||||||
<Text size="xs" c="dimmed">
|
<Text size="xs" c="dimmed">
|
||||||
The most recent parsed telegram persisted to <code>dsmr_reading</code>.
|
The most recent parsed telegram persisted to <code>dsmr_reading</code>.
|
||||||
</Text>
|
</Text>
|
||||||
@@ -100,9 +100,9 @@ function DsmrContent({ isLoading, isError, data }: DsmrContentProps) {
|
|||||||
if (!data.found || !data.payload) {
|
if (!data.found || !data.payload) {
|
||||||
return (
|
return (
|
||||||
<Alert color="gray" data-testid="dsmr-empty">
|
<Alert color="gray" data-testid="dsmr-empty">
|
||||||
No DSMR data yet. Enable <strong>DSMR ingest</strong> in Config, make sure MQTT
|
No DSMR data yet. In this DSMR Source, enable or edit the broker, topic, and profile
|
||||||
is connected, and confirm the DSMR Reader is publishing to the configured topic
|
configuration, then confirm the publisher is sending to the configured topic (default
|
||||||
(default <code>dsmr/json</code>). Rows are stored about once every 10 seconds.
|
<code>dsmr/json</code>). Rows are stored about once every 10 seconds.
|
||||||
</Alert>
|
</Alert>
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|||||||
File diff suppressed because it is too large
Load Diff
@@ -34,11 +34,17 @@ import {
|
|||||||
useMeters,
|
useMeters,
|
||||||
useDeclareMeter,
|
useDeclareMeter,
|
||||||
useUpdateMeter,
|
useUpdateMeter,
|
||||||
|
useSources,
|
||||||
|
useSourceChannels,
|
||||||
|
useCreateBinding,
|
||||||
|
useCloseBinding,
|
||||||
|
useCloseMeter,
|
||||||
|
useTransferBinding,
|
||||||
type MeterResponse,
|
type MeterResponse,
|
||||||
type MeterReason,
|
type MeterReason,
|
||||||
} from './hooks'
|
} from './hooks'
|
||||||
import { ApiError } from '../api/client'
|
import { ApiError } from '../api/client'
|
||||||
import { formatLocalDate, parseBackendTimestamp } from '../utils/datetime'
|
import { formatLocalDate, formatLocalDateTime, parseBackendTimestamp } from '../utils/datetime'
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
// Helpers
|
// Helpers
|
||||||
@@ -73,23 +79,202 @@ function toLocalDateInputString(d: Date): string {
|
|||||||
return `${y}-${m}-${day}`
|
return `${y}-${m}-${day}`
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function toLocalDateTimeInputString(d = new Date()): string {
|
||||||
|
const pad = (value: number) => String(value).padStart(2, '0')
|
||||||
|
return `${toLocalDateInputString(d)}T${pad(d.getHours())}:${pad(d.getMinutes())}`
|
||||||
|
}
|
||||||
|
|
||||||
|
function expectedUnit(commodity: string): string {
|
||||||
|
return ({ electricity: 'kWh', heating: 'GJ', hot_water: 'm³' } as Record<string, string>)[commodity] ?? ''
|
||||||
|
}
|
||||||
|
|
||||||
|
function apiErrorMessage(err: unknown, fallback: string): string {
|
||||||
|
if (err instanceof ApiError) {
|
||||||
|
const body = err.body
|
||||||
|
if (typeof body === 'string') return body
|
||||||
|
if (body && typeof body === 'object' && 'detail' in body) {
|
||||||
|
const detail = (body as { detail?: unknown }).detail
|
||||||
|
if (typeof detail === 'string') return detail
|
||||||
|
if (Array.isArray(detail)) {
|
||||||
|
const messages = detail.map((item) => item && typeof item === 'object' && typeof item.msg === 'string'
|
||||||
|
? item.msg : String(item)).filter(Boolean)
|
||||||
|
if (messages.length) return messages.join('; ')
|
||||||
|
}
|
||||||
|
if (detail != null) return typeof detail === 'object' ? JSON.stringify(detail) : String(detail)
|
||||||
|
}
|
||||||
|
return `${fallback} (error ${err.status}).`
|
||||||
|
}
|
||||||
|
return fallback
|
||||||
|
}
|
||||||
|
|
||||||
|
function isValidLocalDateTime(value: string): boolean {
|
||||||
|
return value.trim() !== '' && !Number.isNaN(new Date(value).getTime())
|
||||||
|
}
|
||||||
|
|
||||||
|
type Eligibility = { eligible: boolean; reason?: string }
|
||||||
|
|
||||||
|
function localInstant(value: string): number | null {
|
||||||
|
const instant = new Date(value).getTime()
|
||||||
|
return Number.isNaN(instant) ? null : instant
|
||||||
|
}
|
||||||
|
|
||||||
|
function intervalsOverlap(start: number, end: number | null, otherStart: number, otherEnd: number | null): boolean {
|
||||||
|
return (end === null || otherStart < end) && (otherEnd === null || start < otherEnd)
|
||||||
|
}
|
||||||
|
|
||||||
|
function channelIntervalEligibility(
|
||||||
|
meters: MeterResponse[], channelUuid: string, startedAt: string, excludeBindingUuids: string[] = [],
|
||||||
|
): Eligibility {
|
||||||
|
const start = localInstant(startedAt)
|
||||||
|
if (start === null) return { eligible: false, reason: 'choose a valid start time first' }
|
||||||
|
const conflicts = meters.flatMap((meter) => (meter.bindings ?? []).filter((binding) =>
|
||||||
|
binding.source_channel_uuid === channelUuid && !excludeBindingUuids.includes(binding.uuid) &&
|
||||||
|
intervalsOverlap(start, null, parseBackendTimestamp(binding.started_at).getTime(), binding.ended_at ? parseBackendTimestamp(binding.ended_at).getTime() : null),
|
||||||
|
))
|
||||||
|
if (!conflicts.length) return { eligible: true }
|
||||||
|
if (conflicts.length > 1) return { eligible: false, reason: 'ambiguous: channel has overlapping binding history; resolve it first' }
|
||||||
|
return conflicts[0].ended_at === null
|
||||||
|
? { eligible: false, reason: 'occupied by an open binding; close or transfer it first' }
|
||||||
|
: { eligible: false, reason: 'overlaps closed binding history; choose a time at or after it ends' }
|
||||||
|
}
|
||||||
|
|
||||||
|
function meterContainsInstant(meter: MeterResponse, instant: number): boolean {
|
||||||
|
const start = parseBackendTimestamp(meter.started_at).getTime()
|
||||||
|
const end = meter.ended_at ? parseBackendTimestamp(meter.ended_at).getTime() : null
|
||||||
|
return instant >= start && (end === null || instant < end)
|
||||||
|
}
|
||||||
|
|
||||||
|
function recoveryTargetFor(meter: MeterResponse, meters: MeterResponse[]): MeterResponse | null {
|
||||||
|
if (meter.ended_at === null) return null
|
||||||
|
const active = meters.filter((candidate) => candidate.commodity === meter.commodity && candidate.ended_at === null)
|
||||||
|
if (active.length !== 1) return null
|
||||||
|
const target = active[0]
|
||||||
|
const targetStart = parseBackendTimestamp(target.started_at).getTime()
|
||||||
|
const predecessors = meters.filter((candidate) => candidate.commodity === meter.commodity && candidate.ended_at !== null &&
|
||||||
|
parseBackendTimestamp(candidate.ended_at).getTime() <= targetStart)
|
||||||
|
const latestEnd = Math.max(...predecessors.map((candidate) => parseBackendTimestamp(candidate.ended_at!).getTime()))
|
||||||
|
const immediate = predecessors.filter((candidate) => parseBackendTimestamp(candidate.ended_at!).getTime() === latestEnd)
|
||||||
|
if (immediate.length !== 1 || immediate[0].id !== meter.id) return null
|
||||||
|
const sourceStart = parseBackendTimestamp(meter.started_at).getTime()
|
||||||
|
const sourceEnd = parseBackendTimestamp(meter.ended_at).getTime()
|
||||||
|
for (const candidate of meters) {
|
||||||
|
if (candidate.id === meter.id || candidate.id === target.id || candidate.commodity !== meter.commodity) continue
|
||||||
|
const candidateStart = parseBackendTimestamp(candidate.started_at).getTime()
|
||||||
|
const candidateEnd = candidate.ended_at ? parseBackendTimestamp(candidate.ended_at).getTime() : null
|
||||||
|
if (intervalsOverlap(sourceStart, sourceEnd, candidateStart, candidateEnd) ||
|
||||||
|
intervalsOverlap(targetStart, null, candidateStart, candidateEnd)) return null
|
||||||
|
}
|
||||||
|
return target
|
||||||
|
}
|
||||||
|
|
||||||
|
function BindingTimeline({ binding, sources }: {
|
||||||
|
binding: NonNullable<MeterResponse['bindings']>[number]
|
||||||
|
sources: ReturnType<typeof useSources>
|
||||||
|
}) {
|
||||||
|
const source = sources.data?.items.find((item) => item.uuid === binding.source_uuid)
|
||||||
|
const channels = useSourceChannels(source?.uuid ?? null)
|
||||||
|
const channel = channels.data?.items.find((item) => item.uuid === binding.source_channel_uuid)
|
||||||
|
let endpoint = 'Source details unavailable'
|
||||||
|
if (sources.isLoading) endpoint = 'Loading source details…'
|
||||||
|
else if (!sources.isError && source) endpoint = source.name
|
||||||
|
else if (!sources.isError) endpoint = 'Source unavailable'
|
||||||
|
|
||||||
|
let channelName = 'Channel details unavailable'
|
||||||
|
if (source && channels.isLoading) channelName = 'Loading channel details…'
|
||||||
|
else if (source && !channels.isError && channel) channelName = channel.label
|
||||||
|
else if (source && !channels.isError) channelName = 'Channel unavailable'
|
||||||
|
|
||||||
|
return (
|
||||||
|
<Stack gap={0} mb="xs" data-testid={`binding-timeline-${binding.uuid}`}>
|
||||||
|
<Text size="xs">{endpoint} → {channelName}</Text>
|
||||||
|
<Text size="xs" c="dimmed">
|
||||||
|
[{formatLocalDateTime(binding.started_at)}, {binding.ended_at ? formatLocalDateTime(binding.ended_at) : 'open-ended'})
|
||||||
|
{' '}({binding.ended_at ? 'closed' : 'active'})
|
||||||
|
</Text>
|
||||||
|
</Stack>
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
// Declare meter form (modal)
|
// Declare meter form (modal)
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
interface DeclareMeterFormProps {
|
interface DeclareMeterFormProps {
|
||||||
|
meters: MeterResponse[]
|
||||||
onClose: () => void
|
onClose: () => void
|
||||||
onSaved: () => void
|
onSaved: () => void
|
||||||
}
|
}
|
||||||
|
|
||||||
function DeclareMeterForm({ onClose, onSaved }: DeclareMeterFormProps) {
|
function DeclareMeterForm({ meters, onClose, onSaved }: DeclareMeterFormProps) {
|
||||||
const [label, setLabel] = useState('')
|
const [label, setLabel] = useState('')
|
||||||
const [dateStr, setDateStr] = useState('')
|
const [dateStr, setDateStr] = useState('')
|
||||||
const [reason, setReason] = useState<string | null>(null)
|
const [reason, setReason] = useState<string | null>(null)
|
||||||
const [note, setNote] = useState('')
|
const [note, setNote] = useState('')
|
||||||
const [error, setError] = useState<string | null>(null)
|
const [error, setError] = useState<string | null>(null)
|
||||||
|
const [commodity, setCommodity] = useState<string | null>('electricity')
|
||||||
|
const [sourceUuid, setSourceUuid] = useState<string | null>(null)
|
||||||
|
const [channelUuid, setChannelUuid] = useState<string | null>(null)
|
||||||
|
const sources = useSources()
|
||||||
|
const channels = useSourceChannels(sourceUuid)
|
||||||
|
|
||||||
const declareMutation = useDeclareMeter()
|
const declareMutation = useDeclareMeter()
|
||||||
|
const unit = expectedUnit(commodity ?? 'electricity')
|
||||||
|
const oldMeter = meters.find((meter) => meter.commodity === commodity && meter.ended_at === null)
|
||||||
|
const channelEligibility = (uuid: string, startedAt: string): Eligibility => {
|
||||||
|
const channel = channels.data?.items.find((item) => item.uuid === uuid)
|
||||||
|
if (!channel) return { eligible: false, reason: 'channel is unavailable; reload the source' }
|
||||||
|
if (channel.unit !== unit) return { eligible: false, reason: `unit mismatch: ${channel.unit}; this meter needs ${unit}` }
|
||||||
|
if (!startedAt) return { eligible: false, reason: 'choose a start date first' }
|
||||||
|
const start = localInstant(toLocalMidnightNaive(startedAt))
|
||||||
|
if (start === null) return { eligible: false, reason: 'choose a valid start date first' }
|
||||||
|
if (startedAt > toLocalDateInputString(new Date())) return { eligible: false, reason: 'future start dates cannot bind a channel' }
|
||||||
|
|
||||||
|
// Channel aggregates include closed binding history. For a meter swap, only
|
||||||
|
// currently open bindings determine whether this channel can be handed off.
|
||||||
|
const openBindings = meters.flatMap((meter) =>
|
||||||
|
(meter.bindings ?? [])
|
||||||
|
.filter((binding) => binding.source_channel_uuid === channel.uuid && binding.ended_at === null)
|
||||||
|
.map((binding) => ({ meter, binding })),
|
||||||
|
)
|
||||||
|
const oldBinding = openBindings[0]
|
||||||
|
const isSingleOldBinding = reason === 'meter_swap' && oldMeter !== undefined &&
|
||||||
|
openBindings.length === 1 && oldBinding !== undefined &&
|
||||||
|
oldBinding.meter.id === oldMeter.id && oldBinding.meter.commodity === commodity
|
||||||
|
// A handoff must leave a non-empty interval on the old Meter. The backend
|
||||||
|
// rejects equality too, so surface it here instead of offering a request
|
||||||
|
// that is guaranteed to fail.
|
||||||
|
if (isSingleOldBinding && start <= parseBackendTimestamp(oldBinding.binding.started_at).getTime()) {
|
||||||
|
return { eligible: false, reason: 'handoff boundary must be strictly after the current binding start' }
|
||||||
|
}
|
||||||
|
const canHandoff = isSingleOldBinding
|
||||||
|
const interval = channelIntervalEligibility(
|
||||||
|
meters, channel.uuid, toLocalMidnightNaive(startedAt), canHandoff ? [oldBinding.binding.uuid] : [],
|
||||||
|
)
|
||||||
|
if (!interval.eligible) return interval
|
||||||
|
if (openBindings.length && !canHandoff) {
|
||||||
|
return openBindings.length > 1
|
||||||
|
? { eligible: false, reason: 'ambiguous: multiple open bindings; resolve them first' }
|
||||||
|
: { eligible: false, reason: 'occupied by an open binding; choose a different channel or close/transfer it first' }
|
||||||
|
}
|
||||||
|
return { eligible: true }
|
||||||
|
}
|
||||||
|
const channelOptions = channels.data?.items.map((channel) => {
|
||||||
|
const result = channelEligibility(channel.uuid, dateStr)
|
||||||
|
const canHandoff = !isUnboundChannel(channel.uuid) && result.eligible
|
||||||
|
return {
|
||||||
|
value: channel.uuid,
|
||||||
|
label: `${channel.label} (${channel.unit})${canHandoff ? ' — hand off from current meter' : result.reason ? ` — ${result.reason}` : ''}`,
|
||||||
|
disabled: !result.eligible,
|
||||||
|
}
|
||||||
|
}) ?? []
|
||||||
|
const selectedChannelUuid = channelUuid && channelEligibility(channelUuid, dateStr).eligible
|
||||||
|
? channelUuid
|
||||||
|
: null
|
||||||
|
function isUnboundChannel(uuid: string): boolean {
|
||||||
|
return !meters.some((meter) => meter.bindings?.some(
|
||||||
|
(binding) => binding.source_channel_uuid === uuid && binding.ended_at === null,
|
||||||
|
))
|
||||||
|
}
|
||||||
|
|
||||||
async function handleSubmit(e: React.FormEvent) {
|
async function handleSubmit(e: React.FormEvent) {
|
||||||
e.preventDefault()
|
e.preventDefault()
|
||||||
@@ -107,6 +292,29 @@ function DeclareMeterForm({ onClose, onSaved }: DeclareMeterFormProps) {
|
|||||||
setError('Reason is required.')
|
setError('Reason is required.')
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
if (sourceUuid && !selectedChannelUuid) {
|
||||||
|
const unavailable = channelOptions.length === 1 && channelOptions[0].disabled
|
||||||
|
? channelEligibility(channelOptions[0].value, dateStr).reason
|
||||||
|
: undefined
|
||||||
|
setError(unavailable ? `${unavailable[0].toUpperCase()}${unavailable.slice(1)}` : 'Select an eligible source channel or clear the optional source.')
|
||||||
|
return
|
||||||
|
}
|
||||||
|
// Omitting a channel for meter_swap asks the backend to auto-handoff the
|
||||||
|
// sole open binding. Keep that implicit path subject to the same strict
|
||||||
|
// boundary rule as an explicitly selected channel.
|
||||||
|
const start = localInstant(toLocalMidnightNaive(dateStr))
|
||||||
|
const oldMeter = meters.find((meter) => meter.commodity === commodity && meter.ended_at === null)
|
||||||
|
// The implicit backend handoff only considers bindings on the current
|
||||||
|
// commodity's old meter. Bindings are unit-compatible with their meter
|
||||||
|
// by the binding contract, so an unrelated heating/hot-water binding must
|
||||||
|
// neither make this ambiguous nor bypass this strict boundary check.
|
||||||
|
const autoHandoffCandidates = (oldMeter?.bindings ?? []).filter((binding) => binding.ended_at === null)
|
||||||
|
const oldBinding = autoHandoffCandidates[0]
|
||||||
|
if (reason === 'meter_swap' && start !== null && oldMeter !== undefined && autoHandoffCandidates.length === 1 &&
|
||||||
|
oldBinding !== undefined && start <= parseBackendTimestamp(oldBinding.started_at).getTime()) {
|
||||||
|
setError('Handoff boundary must be strictly after the current binding start.')
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
try {
|
try {
|
||||||
await declareMutation.mutateAsync({
|
await declareMutation.mutateAsync({
|
||||||
@@ -114,18 +322,12 @@ function DeclareMeterForm({ onClose, onSaved }: DeclareMeterFormProps) {
|
|||||||
started_at: toLocalMidnightNaive(dateStr),
|
started_at: toLocalMidnightNaive(dateStr),
|
||||||
reason: reason as MeterReason,
|
reason: reason as MeterReason,
|
||||||
note: note.trim() || undefined,
|
note: note.trim() || undefined,
|
||||||
commodity: 'electricity',
|
commodity: commodity ?? 'electricity',
|
||||||
|
...(selectedChannelUuid ? { source_channel_uuid: selectedChannelUuid } : {}),
|
||||||
})
|
})
|
||||||
onSaved()
|
onSaved()
|
||||||
onClose()
|
onClose()
|
||||||
} catch (err) {
|
} catch (err) { setError(apiErrorMessage(err, 'Failed to declare meter. Please try again.')) }
|
||||||
if (err instanceof ApiError) {
|
|
||||||
const detail = (err.body as { detail?: string } | null)?.detail
|
|
||||||
setError(detail ?? `Error ${err.status}: failed to declare meter.`)
|
|
||||||
} else {
|
|
||||||
setError('Failed to declare meter. Please try again.')
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
return (
|
return (
|
||||||
@@ -153,7 +355,11 @@ function DeclareMeterForm({ onClose, onSaved }: DeclareMeterFormProps) {
|
|||||||
type="date"
|
type="date"
|
||||||
required
|
required
|
||||||
value={dateStr}
|
value={dateStr}
|
||||||
onChange={(e) => setDateStr(e.currentTarget.value)}
|
onChange={(e) => {
|
||||||
|
const nextDateStr = e.currentTarget.value
|
||||||
|
setDateStr(nextDateStr)
|
||||||
|
setChannelUuid((uuid) => uuid && !channelEligibility(uuid, nextDateStr).eligible ? null : uuid)
|
||||||
|
}}
|
||||||
data-testid="meter-started-at"
|
data-testid="meter-started-at"
|
||||||
/>
|
/>
|
||||||
|
|
||||||
@@ -162,10 +368,21 @@ function DeclareMeterForm({ onClose, onSaved }: DeclareMeterFormProps) {
|
|||||||
required
|
required
|
||||||
data={REASON_OPTIONS}
|
data={REASON_OPTIONS}
|
||||||
value={reason}
|
value={reason}
|
||||||
onChange={setReason}
|
onChange={(value) => { setReason(value); setChannelUuid(null) }}
|
||||||
data-testid="meter-reason"
|
data-testid="meter-reason"
|
||||||
/>
|
/>
|
||||||
|
|
||||||
|
<Select label="Commodity" value={commodity} onChange={(value) => { setCommodity(value); setChannelUuid(null) }} data={[
|
||||||
|
{ value: 'electricity', label: 'Electricity' },
|
||||||
|
{ value: 'heating', label: 'Heating' },
|
||||||
|
{ value: 'hot_water', label: 'Hot water' },
|
||||||
|
]} />
|
||||||
|
<Select label="Bind source (optional)" value={sourceUuid} onChange={(value) => { setSourceUuid(value); setChannelUuid(null) }} data={sources.data?.items.filter((source) => typeof source.uuid === 'string').map((source) => ({ value: source.uuid, label: source.name })) ?? []} />
|
||||||
|
{reason === 'meter_swap' && <Alert color="blue">If the previous meter has exactly one compatible open binding, declaring this meter automatically hands that channel over atomically. Ambiguous bindings remain unavailable.</Alert>}
|
||||||
|
{sources.isLoading && <Text size="sm">Loading sources…</Text>}{sources.isError && <Alert color="red">Could not load sources. Retry after the connection recovers.</Alert>}
|
||||||
|
{sourceUuid && channels.isLoading && <Text size="sm">Loading source channels…</Text>}{sourceUuid && channels.isError && <Alert color="red">Could not load source channels. Choose another source or retry.</Alert>}
|
||||||
|
{sourceUuid && <Select label="Compatible source channel (optional)" value={selectedChannelUuid} onChange={setChannelUuid} description="Disabled channels explain unit, current interval, ambiguity, or required time. Closed history remains reusable." data={channelOptions} />}
|
||||||
|
|
||||||
<Textarea
|
<Textarea
|
||||||
label="Note (optional)"
|
label="Note (optional)"
|
||||||
value={note}
|
value={note}
|
||||||
@@ -252,14 +469,7 @@ function EditMeterForm({ meter, onClose, onSaved }: EditMeterFormProps) {
|
|||||||
await updateMutation.mutateAsync({ id: meter.id, body: patchBody })
|
await updateMutation.mutateAsync({ id: meter.id, body: patchBody })
|
||||||
onSaved(startedAtChanged)
|
onSaved(startedAtChanged)
|
||||||
onClose()
|
onClose()
|
||||||
} catch (err) {
|
} catch (err) { setError(apiErrorMessage(err, 'Failed to update meter. Please try again.')) }
|
||||||
if (err instanceof ApiError) {
|
|
||||||
const detail = (err.body as { detail?: string } | null)?.detail
|
|
||||||
setError(detail ?? `Error ${err.status}: failed to update meter.`)
|
|
||||||
} else {
|
|
||||||
setError('Failed to update meter. Please try again.')
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
return (
|
return (
|
||||||
@@ -333,10 +543,12 @@ function EditMeterForm({ meter, onClose, onSaved }: EditMeterFormProps) {
|
|||||||
|
|
||||||
interface MeterTableProps {
|
interface MeterTableProps {
|
||||||
meters: MeterResponse[]
|
meters: MeterResponse[]
|
||||||
|
sources: ReturnType<typeof useSources>
|
||||||
onEdit: (meter: MeterResponse) => void
|
onEdit: (meter: MeterResponse) => void
|
||||||
|
onClose: (meter: MeterResponse) => void
|
||||||
}
|
}
|
||||||
|
|
||||||
function MeterTable({ meters, onEdit }: MeterTableProps) {
|
function MeterTable({ meters, sources, onEdit, onClose }: MeterTableProps) {
|
||||||
if (meters.length === 0) {
|
if (meters.length === 0) {
|
||||||
return (
|
return (
|
||||||
<Text c="dimmed" ta="center" size="sm" data-testid="meters-empty">
|
<Text c="dimmed" ta="center" size="sm" data-testid="meters-empty">
|
||||||
@@ -356,6 +568,7 @@ function MeterTable({ meters, onEdit }: MeterTableProps) {
|
|||||||
<Table.Th>To</Table.Th>
|
<Table.Th>To</Table.Th>
|
||||||
<Table.Th>Status</Table.Th>
|
<Table.Th>Status</Table.Th>
|
||||||
<Table.Th>Reason</Table.Th>
|
<Table.Th>Reason</Table.Th>
|
||||||
|
<Table.Th>Binding timeline</Table.Th>
|
||||||
<Table.Th style={{ textAlign: 'right' }}>Actions</Table.Th>
|
<Table.Th style={{ textAlign: 'right' }}>Actions</Table.Th>
|
||||||
</Table.Tr>
|
</Table.Tr>
|
||||||
</Table.Thead>
|
</Table.Thead>
|
||||||
@@ -399,16 +612,26 @@ function MeterTable({ meters, onEdit }: MeterTableProps) {
|
|||||||
{meter.reason}
|
{meter.reason}
|
||||||
</Badge>
|
</Badge>
|
||||||
</Table.Td>
|
</Table.Td>
|
||||||
|
<Table.Td>
|
||||||
|
{meter.bindings?.length ? meter.bindings.map((binding) => (
|
||||||
|
<BindingTimeline key={binding.uuid} binding={binding} sources={sources} />
|
||||||
|
)) : <Text size="xs" c="dimmed">Unbound</Text>}
|
||||||
|
</Table.Td>
|
||||||
<Table.Td>
|
<Table.Td>
|
||||||
<Group justify="flex-end" gap="xs">
|
<Group justify="flex-end" gap="xs">
|
||||||
<Button
|
<Button
|
||||||
size="xs"
|
size="xs"
|
||||||
variant="outline"
|
variant="light"
|
||||||
onClick={() => onEdit(meter)}
|
onClick={() => onEdit(meter)}
|
||||||
data-testid={`meter-edit-${meter.id}`}
|
data-testid={`meter-edit-${meter.id}`}
|
||||||
>
|
>
|
||||||
Edit
|
Edit
|
||||||
</Button>
|
</Button>
|
||||||
|
{meter.bindings?.filter((binding) => binding.ended_at === null).map((binding) => (
|
||||||
|
<BindingActions key={binding.uuid} meter={meter} binding={binding} meters={meters} />
|
||||||
|
))}
|
||||||
|
{isActive && !meter.bindings?.some((binding) => binding.ended_at === null) && <DirectBindButton meter={meter} meters={meters} />}
|
||||||
|
{isActive && <Button size="xs" color="red" variant="light" onClick={() => onClose(meter)}>Close meter</Button>}
|
||||||
</Group>
|
</Group>
|
||||||
</Table.Td>
|
</Table.Td>
|
||||||
</Table.Tr>
|
</Table.Tr>
|
||||||
@@ -420,15 +643,184 @@ function MeterTable({ meters, onEdit }: MeterTableProps) {
|
|||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function DirectBindButton({ meter, meters }: { meter: MeterResponse; meters: MeterResponse[] }) {
|
||||||
|
const [opened, setOpened] = useState(false)
|
||||||
|
return <>{<Button size="xs" variant="light" onClick={() => setOpened(true)}>Bind source</Button>}{opened && <DirectBindModal meter={meter} meters={meters} onClose={() => setOpened(false)} />}</>
|
||||||
|
}
|
||||||
|
|
||||||
|
function DirectBindModal({ meter, meters, onClose }: { meter: MeterResponse; meters: MeterResponse[]; onClose: () => void }) {
|
||||||
|
const [sourceUuid, setSourceUuid] = useState<string | null>(null)
|
||||||
|
const [channelUuid, setChannelUuid] = useState<string | null>(null)
|
||||||
|
const [startedAt, setStartedAt] = useState(() => toLocalDateTimeInputString())
|
||||||
|
const [error, setError] = useState<string | null>(null)
|
||||||
|
const sources = useSources(); const channels = useSourceChannels(sourceUuid); const create = useCreateBinding()
|
||||||
|
const eligibilityAt = (channel: { uuid: string; unit: string }, value: string): Eligibility => {
|
||||||
|
if (channel.unit !== expectedUnit(meter.commodity)) return { eligible: false, reason: `unit mismatch: ${channel.unit}; this meter needs ${expectedUnit(meter.commodity)}` }
|
||||||
|
const start = localInstant(value)
|
||||||
|
if (start === null) return { eligible: false, reason: 'choose a valid binding start time first' }
|
||||||
|
if (value > toLocalDateTimeInputString()) return { eligible: false, reason: 'future binding start times are not allowed' }
|
||||||
|
if (!meterContainsInstant(meter, start)) return { eligible: false, reason: 'binding start must be within this meter epoch' }
|
||||||
|
return channelIntervalEligibility(meters, channel.uuid, value)
|
||||||
|
}
|
||||||
|
const eligibility = (channel: { uuid: string; unit: string }) => eligibilityAt(channel, startedAt)
|
||||||
|
const options = channels.data?.items.map((channel) => {
|
||||||
|
const result = eligibility(channel)
|
||||||
|
return { value: channel.uuid, label: `${channel.label} (${channel.unit})${result.reason ? ` — ${result.reason}` : ''}`, disabled: !result.eligible }
|
||||||
|
}) ?? []
|
||||||
|
const selectedChannel = channels.data?.items.find((channel) => channel.uuid === channelUuid)
|
||||||
|
const selectedEligible = selectedChannel !== undefined && eligibility(selectedChannel).eligible
|
||||||
|
async function save() {
|
||||||
|
if (create.isPending) return
|
||||||
|
if (!channelUuid) return setError('Select a genuinely unbound, unit-compatible channel.')
|
||||||
|
if (!isValidLocalDateTime(startedAt)) return setError('Choose a valid binding start time.')
|
||||||
|
if (!selectedEligible) return setError('The selected channel is no longer eligible. Choose an available channel.')
|
||||||
|
setError(null)
|
||||||
|
try { await create.mutateAsync({ id: meter.id, body: { source_channel_uuid: channelUuid, started_at: startedAt } }); onClose() } catch (err) { setError(apiErrorMessage(err, 'Could not bind this source.')) }
|
||||||
|
}
|
||||||
|
return <Modal opened onClose={onClose} title="Bind source" data-testid={`direct-bind-modal-${meter.id}`}><form onSubmit={(event) => { event.preventDefault(); void save() }}><Stack>
|
||||||
|
<Alert color="blue">This active meter has no open binding. Choose a currently unoccupied compatible channel.</Alert>
|
||||||
|
{sources.isLoading && <Text size="sm">Loading sources…</Text>}{sources.isError && <Alert color="red">Could not load sources. Retry after the connection recovers.</Alert>}
|
||||||
|
<Select label="Source" value={sourceUuid} onChange={(value) => { setSourceUuid(value); setChannelUuid(null) }} data={sources.data?.items.map((source) => ({ value: source.uuid, label: source.name })) ?? []} />
|
||||||
|
{sourceUuid && channels.isLoading && <Text size="sm">Loading source channels…</Text>}{sourceUuid && channels.isError && <Alert color="red">Could not load source channels. Choose another source or retry.</Alert>}
|
||||||
|
<Select label="Source channel" value={channelUuid} onChange={setChannelUuid} description="Disabled channels explain the unit, interval, ambiguity, or time constraint." data={options} />
|
||||||
|
<TextInput label="Binding start time" type="datetime-local" value={startedAt} onChange={(event) => { const value = event.currentTarget.value; setStartedAt(value); setChannelUuid((uuid) => { if (!isValidLocalDateTime(value)) return uuid; const channel = channels.data?.items.find((item) => item.uuid === uuid); return channel && !eligibilityAt(channel, value).eligible ? null : uuid }) }} required />
|
||||||
|
{error && <Alert color="red">{error}</Alert>}
|
||||||
|
<Group justify="flex-end"><Button type="button" variant="default" onClick={onClose}>Cancel</Button><Button type="submit" loading={create.isPending} disabled={create.isPending || (!!channelUuid && !selectedEligible)}>Bind source</Button></Group>
|
||||||
|
</Stack></form></Modal>
|
||||||
|
}
|
||||||
|
|
||||||
|
function BindingActions({ meter, binding, meters }: { meter: MeterResponse; binding: NonNullable<MeterResponse['bindings']>[number]; meters: MeterResponse[] }) {
|
||||||
|
const [unbindOpened, setUnbindOpened] = useState(false)
|
||||||
|
const [transferOpened, setTransferOpened] = useState(false)
|
||||||
|
const recoveryTarget = recoveryTargetFor(meter, meters)
|
||||||
|
// Cross-Meter recovery closes at the old Meter boundary. An anomalous
|
||||||
|
// retained binding beginning at or after that boundary would create a
|
||||||
|
// zero-length/negative source interval that the server correctly rejects.
|
||||||
|
const recoverySourceIsClosable = meter.ended_at === null ||
|
||||||
|
parseBackendTimestamp(binding.started_at).getTime() < parseBackendTimestamp(meter.ended_at).getTime()
|
||||||
|
return <>
|
||||||
|
{meter.ended_at === null ? <Button size="xs" variant="light" onClick={() => setTransferOpened(true)}>Transfer source</Button> : recoveryTarget && recoverySourceIsClosable && <Button size="xs" variant="light" onClick={() => setTransferOpened(true)}>Recover binding</Button>}
|
||||||
|
<Button size="xs" variant="light" onClick={() => setUnbindOpened(true)}>Unbind</Button>
|
||||||
|
{meter.ended_at !== null && recoveryTarget && !recoverySourceIsClosable && <Text size="xs" c="red">Cannot recover: the source binding starts at or after this Meter ended.</Text>}
|
||||||
|
{unbindOpened && <UnbindModal meter={meter} binding={binding} onClose={() => setUnbindOpened(false)} />}
|
||||||
|
{transferOpened && <TransferModal target={recoveryTarget ?? meter} sourceBinding={binding} meters={meters} recovery={!!recoveryTarget} onClose={() => setTransferOpened(false)} />}
|
||||||
|
</>
|
||||||
|
}
|
||||||
|
|
||||||
|
function UnbindModal({ meter, binding, onClose }: { meter: MeterResponse; binding: NonNullable<MeterResponse['bindings']>[number]; onClose: () => void }) {
|
||||||
|
const [endedAt, setEndedAt] = useState(() => meter.ended_at ? toLocalDateTimeInputString(parseBackendTimestamp(meter.ended_at)) : toLocalDateTimeInputString())
|
||||||
|
const [error, setError] = useState<string | null>(null)
|
||||||
|
const close = useCloseBinding()
|
||||||
|
async function save() {
|
||||||
|
if (close.isPending) return
|
||||||
|
if (!isValidLocalDateTime(endedAt)) return setError('Choose a valid unbind time.')
|
||||||
|
const instant = localInstant(endedAt)
|
||||||
|
if (instant === null || instant <= parseBackendTimestamp(binding.started_at).getTime()) {
|
||||||
|
return setError('Unbind time must be strictly after the binding start.')
|
||||||
|
}
|
||||||
|
if (endedAt > toLocalDateTimeInputString()) return setError('A future unbind time is not allowed.')
|
||||||
|
if (meter.ended_at && instant > parseBackendTimestamp(meter.ended_at).getTime()) {
|
||||||
|
return setError('Unbind time must not be after the meter end.')
|
||||||
|
}
|
||||||
|
setError(null)
|
||||||
|
try { await close.mutateAsync({ uuid: binding.uuid, ended_at: endedAt }); onClose() } catch (err) { setError(apiErrorMessage(err, 'Could not unbind this source. History was not deleted.')) }
|
||||||
|
}
|
||||||
|
return <Modal opened onClose={onClose} title="Unbind source" data-testid={`unbind-modal-${binding.uuid}`}><form onSubmit={(event) => { event.preventDefault(); void save() }}><Stack>
|
||||||
|
<Alert color="blue">Unbinding closes this binding at the selected time. It never deletes binding history.</Alert>
|
||||||
|
<TextInput label="Unbind time" type="datetime-local" value={endedAt} onChange={(event) => setEndedAt(event.currentTarget.value)} required />
|
||||||
|
{error && <Alert color="red">{error}</Alert>}
|
||||||
|
<Group justify="flex-end"><Button type="button" variant="default" onClick={onClose}>Cancel</Button><Button type="submit" loading={close.isPending} disabled={close.isPending}>Unbind</Button></Group>
|
||||||
|
</Stack></form></Modal>
|
||||||
|
}
|
||||||
|
|
||||||
|
function TransferModal({ target, sourceBinding, meters, recovery, onClose }: { target: MeterResponse; sourceBinding: NonNullable<MeterResponse['bindings']>[number]; meters: MeterResponse[]; recovery: boolean; onClose: () => void }) {
|
||||||
|
const [sourceUuid, setSourceUuid] = useState<string | null>(null)
|
||||||
|
const [channelUuid, setChannelUuid] = useState<string | null>(null)
|
||||||
|
const [effectiveAt, setEffectiveAt] = useState(() => recovery ? toLocalDateTimeInputString(parseBackendTimestamp(target.started_at)) : toLocalDateTimeInputString())
|
||||||
|
const [error, setError] = useState<string | null>(null)
|
||||||
|
const sources = useSources(); const channels = useSourceChannels(sourceUuid)
|
||||||
|
const transfer = useTransferBinding()
|
||||||
|
const channelEligibilityAt = (channel: { uuid: string; unit: string }, value: string): Eligibility => {
|
||||||
|
if (channel.unit !== expectedUnit(target.commodity)) return { eligible: false, reason: `unit mismatch: ${channel.unit}; target needs ${expectedUnit(target.commodity)}` }
|
||||||
|
const start = localInstant(value)
|
||||||
|
if (start === null) return { eligible: false, reason: 'choose a valid effective time first' }
|
||||||
|
if (value > toLocalDateTimeInputString()) return { eligible: false, reason: 'future effective times are not allowed' }
|
||||||
|
if (!meterContainsInstant(target, start)) return { eligible: false, reason: 'effective time must be within the target meter epoch' }
|
||||||
|
// A same-meter transfer closes its source at `effective_at`. Equality
|
||||||
|
// would therefore create the forbidden zero-length [start, start)
|
||||||
|
// interval. Recovery closes at the old meter boundary instead, so it
|
||||||
|
// deliberately keeps the normal target-epoch rule and may be equal to
|
||||||
|
// the source binding's (much earlier) start.
|
||||||
|
if (!recovery && start <= parseBackendTimestamp(sourceBinding.started_at).getTime()) {
|
||||||
|
return { eligible: false, reason: 'same-meter transfer must be strictly after the source binding start' }
|
||||||
|
}
|
||||||
|
return channelIntervalEligibility(meters, channel.uuid, value, [sourceBinding.uuid])
|
||||||
|
}
|
||||||
|
const channelEligibility = (channel: { uuid: string; unit: string }) => channelEligibilityAt(channel, effectiveAt)
|
||||||
|
const options = channels.data?.items.map((channel) => {
|
||||||
|
const result = channelEligibility(channel)
|
||||||
|
return { value: channel.uuid, label: `${channel.label} (${channel.unit})${result.reason ? ` — ${result.reason}` : ''}`, disabled: !result.eligible }
|
||||||
|
}) ?? []
|
||||||
|
const selectedChannel = channels.data?.items.find((channel) => channel.uuid === channelUuid)
|
||||||
|
const selectedEligible = selectedChannel !== undefined && channelEligibility(selectedChannel).eligible
|
||||||
|
const isFuture = effectiveAt > toLocalDateTimeInputString()
|
||||||
|
async function save() {
|
||||||
|
if (transfer.isPending) return
|
||||||
|
if (!channelUuid) return setError('Select a unit-compatible source channel.')
|
||||||
|
if (!isValidLocalDateTime(effectiveAt) || isFuture) return setError('Choose a valid non-future effective time.')
|
||||||
|
if (!selectedEligible) return setError('The selected channel is no longer eligible. Choose an available channel.')
|
||||||
|
setError(null)
|
||||||
|
try { await transfer.mutateAsync({ id: target.id, body: { from_binding_uuid: sourceBinding.uuid, to_source_channel_uuid: channelUuid, effective_at: effectiveAt } }); onClose() } catch (err) { setError(apiErrorMessage(err, 'Transfer failed. No partial source switch was saved.')) }
|
||||||
|
}
|
||||||
|
return <Modal opened onClose={onClose} title={recovery ? 'Recover stranded binding' : 'Transfer source binding'} data-testid={`transfer-modal-${sourceBinding.uuid}`}><form onSubmit={(event) => { event.preventDefault(); void save() }}><Stack>
|
||||||
|
<Alert color="blue">This is one atomic Transfer request: either the old binding closes and the new one opens together, or neither change is saved.</Alert>
|
||||||
|
{recovery && <Alert color="yellow">This binding is stranded on a closed meter. Recovery defaults to the new meter start. A later time is allowed, but creates an unbound gap before it.</Alert>}
|
||||||
|
{sources.isLoading && <Text size="sm">Loading sources…</Text>}{sources.isError && <Alert color="red">Could not load sources. Retry after the connection recovers.</Alert>}
|
||||||
|
<Select label="Source" value={sourceUuid} onChange={(value) => { setSourceUuid(value); setChannelUuid(null) }} data={sources.data?.items.map((source) => ({ value: source.uuid, label: source.name })) ?? []} />
|
||||||
|
{sourceUuid && channels.isLoading && <Text size="sm">Loading source channels…</Text>}{sourceUuid && channels.isError && <Alert color="red">Could not load source channels. Choose another source or retry.</Alert>}
|
||||||
|
<Select label="Source channel" value={channelUuid} onChange={setChannelUuid} description="Disabled channels name the specific unit or unrelated-open-interval conflict. The server also rejects ambiguity atomically." data={options} />
|
||||||
|
<TextInput label="Effective time" type="datetime-local" value={effectiveAt} onChange={(event) => { const value = event.currentTarget.value; setEffectiveAt(value); setChannelUuid((uuid) => { if (!isValidLocalDateTime(value)) return uuid; const channel = channels.data?.items.find((item) => item.uuid === uuid); return channel && !channelEligibilityAt(channel, value).eligible ? null : uuid }) }} required data-testid="transfer-effective-at" />
|
||||||
|
{recovery && effectiveAt && effectiveAt > toLocalDateTimeInputString(parseBackendTimestamp(target.started_at)) && <Alert color="yellow">Warning: this later time leaves an unbound gap from the new meter start until this transfer.</Alert>}
|
||||||
|
{isFuture && <Alert color="red">A future effective time is not allowed.</Alert>}
|
||||||
|
{error && <Alert color="red">{error}</Alert>}
|
||||||
|
<Group justify="flex-end"><Button type="button" variant="default" onClick={onClose}>Cancel</Button><Button type="submit" loading={transfer.isPending} disabled={transfer.isPending || (!!channelUuid && !selectedEligible)}>Transfer binding</Button></Group>
|
||||||
|
</Stack></form></Modal>
|
||||||
|
}
|
||||||
|
|
||||||
|
function CloseMeterModal({ meter, onClose }: { meter: MeterResponse; onClose: () => void }) {
|
||||||
|
const [endedAt, setEndedAt] = useState(() => toLocalDateTimeInputString())
|
||||||
|
const [error, setError] = useState<string | null>(null)
|
||||||
|
const close = useCloseMeter()
|
||||||
|
async function save() {
|
||||||
|
if (close.isPending) return
|
||||||
|
if (!isValidLocalDateTime(endedAt)) return setError('Choose a valid close time.')
|
||||||
|
const instant = localInstant(endedAt)
|
||||||
|
if (instant === null || instant <= parseBackendTimestamp(meter.started_at).getTime()) {
|
||||||
|
return setError('Close time must be strictly after the meter start.')
|
||||||
|
}
|
||||||
|
if (endedAt > toLocalDateTimeInputString()) return setError('A future close time is not allowed.')
|
||||||
|
setError(null)
|
||||||
|
try { await close.mutateAsync({ id: meter.id, ended_at: endedAt }); onClose() } catch (err) { setError(apiErrorMessage(err, 'Could not close this meter.')) }
|
||||||
|
}
|
||||||
|
return <Modal opened onClose={onClose} title={`Close Meter — ${meter.label}`} data-testid={`close-meter-modal-${meter.id}`}><form onSubmit={(event) => { event.preventDefault(); void save() }}><Stack>
|
||||||
|
<Alert color="yellow">Closing leaves no active {meter.commodity} meter. Every open binding on this meter closes at the same boundary.</Alert>
|
||||||
|
<TextInput label="Close time" type="datetime-local" value={endedAt} onChange={(event) => setEndedAt(event.currentTarget.value)} required />
|
||||||
|
{error && <Alert color="red">{error}</Alert>}
|
||||||
|
<Group justify="flex-end"><Button type="button" variant="default" onClick={onClose}>Cancel</Button><Button type="submit" color="red" loading={close.isPending} disabled={close.isPending}>Close meter</Button></Group>
|
||||||
|
</Stack></form></Modal>
|
||||||
|
}
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
// MeterManager — top-level
|
// MeterManager — top-level
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
export function MeterManager() {
|
export function MeterManager() {
|
||||||
const metersQuery = useMeters()
|
const metersQuery = useMeters()
|
||||||
|
const sources = useSources()
|
||||||
|
|
||||||
const [showDeclareForm, setShowDeclareForm] = useState(false)
|
const [showDeclareForm, setShowDeclareForm] = useState(false)
|
||||||
const [editMeter, setEditMeter] = useState<MeterResponse | null>(null)
|
const [editMeter, setEditMeter] = useState<MeterResponse | null>(null)
|
||||||
|
const [closeMeter, setCloseMeter] = useState<MeterResponse | null>(null)
|
||||||
const [recomputeNotice, setRecomputeNotice] = useState(false)
|
const [recomputeNotice, setRecomputeNotice] = useState(false)
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
@@ -456,7 +848,7 @@ export function MeterManager() {
|
|||||||
return (
|
return (
|
||||||
<Stack gap="lg" data-testid="meter-manager">
|
<Stack gap="lg" data-testid="meter-manager">
|
||||||
<Group justify="space-between" align="center">
|
<Group justify="space-between" align="center">
|
||||||
<Text fw={500}>Electricity Meters</Text>
|
<Text fw={500}>Meters</Text>
|
||||||
<Button
|
<Button
|
||||||
onClick={() => setShowDeclareForm(true)}
|
onClick={() => setShowDeclareForm(true)}
|
||||||
data-testid="meter-declare-button"
|
data-testid="meter-declare-button"
|
||||||
@@ -477,11 +869,12 @@ export function MeterManager() {
|
|||||||
</Notification>
|
</Notification>
|
||||||
)}
|
)}
|
||||||
|
|
||||||
<MeterTable meters={meters} onEdit={(m) => setEditMeter(m)} />
|
<MeterTable meters={meters} sources={sources} onEdit={(m) => setEditMeter(m)} onClose={(m) => setCloseMeter(m)} />
|
||||||
|
|
||||||
{/* Declare new meter */}
|
{/* Declare new meter */}
|
||||||
{showDeclareForm && (
|
{showDeclareForm && (
|
||||||
<DeclareMeterForm
|
<DeclareMeterForm
|
||||||
|
meters={meters}
|
||||||
onClose={() => setShowDeclareForm(false)}
|
onClose={() => setShowDeclareForm(false)}
|
||||||
onSaved={() => setShowDeclareForm(false)}
|
onSaved={() => setShowDeclareForm(false)}
|
||||||
/>
|
/>
|
||||||
@@ -498,6 +891,8 @@ export function MeterManager() {
|
|||||||
}}
|
}}
|
||||||
/>
|
/>
|
||||||
)}
|
)}
|
||||||
|
|
||||||
|
{closeMeter && <CloseMeterModal meter={closeMeter} onClose={() => setCloseMeter(null)} />}
|
||||||
</Stack>
|
</Stack>
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,15 @@
|
|||||||
|
import { describe, expect, it, vi, beforeEach } from 'vitest'
|
||||||
|
import { screen, waitFor } from '@testing-library/react'
|
||||||
|
import userEvent from '@testing-library/user-event'
|
||||||
|
import { renderWithProviders } from '../test-utils'
|
||||||
|
import { SourceForm } from './SourceForm'
|
||||||
|
|
||||||
|
const mockGet = vi.fn(); const mockPatch = vi.fn()
|
||||||
|
vi.mock('../api/client', () => ({ default: { GET: (...a: unknown[]) => mockGet(...a), POST: vi.fn(), PATCH: (...a: unknown[]) => mockPatch(...a), DELETE: vi.fn() }, ApiError: class ApiError extends Error { constructor(public status: number, public body: unknown) { super(`API error ${status}`) } }, registerLoginRedirect: vi.fn() }))
|
||||||
|
const profile = { kind: 'dsmr_mqtt', fields: [{ name: 'tls_enabled', value_type: 'bool', default: false }, { name: 'port', value_type: 'int', default: 1883 }, { name: 'topic', value_type: 'string', default: 'telegram' }, { name: 'password', value_type: 'string', secret: true }] }
|
||||||
|
const source = { uuid: 'source-1', name: 'DSMR', kind: 'dsmr_mqtt', enabled: true, config: { tls_enabled: true, port: 8883, topic: 'old', password: '********' } }
|
||||||
|
describe('SourceForm typed PATCH and secrets', () => {
|
||||||
|
beforeEach(() => { vi.clearAllMocks(); mockGet.mockResolvedValue({ data: { items: [profile] } }) })
|
||||||
|
it('preserves native bool/number/string and omits untouched masked secret', async () => { const user = userEvent.setup(); mockPatch.mockResolvedValue({ data: source }); renderWithProviders(<SourceForm source={source as never} onClose={vi.fn()} />); await user.click(await screen.findByRole('button', { name: 'Save Source' })); await waitFor(() => expect(mockPatch).toHaveBeenCalled()); expect(mockPatch).toHaveBeenCalledWith('/api/energy/sources/{source_uuid}', { params: { path: { source_uuid: 'source-1' } }, body: { name: 'DSMR', enabled: true, config: { tls_enabled: true, port: 8883, topic: 'old' } } }) })
|
||||||
|
it('sends a replacement secret and shows 422 detail', async () => { const user = userEvent.setup(); mockPatch.mockRejectedValue(new (await import('../api/client')).ApiError(422, { detail: 'invalid broker' })); renderWithProviders(<SourceForm source={source as never} onClose={vi.fn()} />); await user.type(await screen.findByLabelText('password'), 'new-secret'); await user.click(screen.getByRole('button', { name: 'Save Source' })); await waitFor(() => expect(mockPatch).toHaveBeenCalled()); expect(mockPatch.mock.calls[0][1].body.config).toMatchObject({ password: 'new-secret', tls_enabled: true, port: 8883 }); expect(await screen.findByText('invalid broker')).toBeInTheDocument() })
|
||||||
|
})
|
||||||
@@ -0,0 +1,33 @@
|
|||||||
|
import { useState } from 'react'
|
||||||
|
import { Alert, Button, Checkbox, Group, Modal, Select, Stack, TextInput } from '@mantine/core'
|
||||||
|
import { ApiError } from '../api/client'
|
||||||
|
import { useCreateSource, useSourceProfiles, useUpdateSource, type MeterSourceResponse } from './hooks'
|
||||||
|
|
||||||
|
export function SourceForm({ source, onClose }: { source?: MeterSourceResponse; onClose: () => void }) {
|
||||||
|
const profiles = useSourceProfiles(); const create = useCreateSource(); const update = useUpdateSource()
|
||||||
|
const [name, setName] = useState(source?.name ?? ''); const [kind, setKind] = useState<string | null>(source?.kind ?? null)
|
||||||
|
const [enabled, setEnabled] = useState(source?.enabled ?? true); const [config, setConfig] = useState<Record<string, string | boolean | number>>({}); const [error, setError] = useState<string | null>(null)
|
||||||
|
const profile = profiles.data?.items.find((item) => item.kind === kind)
|
||||||
|
async function submit(e: React.FormEvent) { e.preventDefault(); setError(null); if (!name.trim() || !kind) return setError('Name and source type are required.')
|
||||||
|
const values: Record<string, unknown> = {}; profile?.fields.forEach((field) => {
|
||||||
|
const changed = Object.prototype.hasOwnProperty.call(config, field.name)
|
||||||
|
const raw = changed ? config[field.name] : (source?.config[field.name] ?? field.default ?? '')
|
||||||
|
// A masked secret is deliberately absent from edit PATCHes until the user
|
||||||
|
// explicitly enters a replacement; sending an empty/masked value is unsafe.
|
||||||
|
if (field.secret && source && (!changed || raw === '')) return
|
||||||
|
if (field.value_type === 'bool' || field.value_type === 'boolean') values[field.name] = typeof raw === 'boolean' ? raw : raw === 'true'
|
||||||
|
else if (field.value_type === 'int' || field.value_type === 'integer') values[field.name] = typeof raw === 'number' ? raw : Number(raw)
|
||||||
|
else values[field.name] = typeof raw === 'string' ? raw : String(raw)
|
||||||
|
})
|
||||||
|
try { if (source) await update.mutateAsync({ uuid: source.uuid, body: { name: name.trim(), enabled, config: values } }); else await create.mutateAsync({ name: name.trim(), kind, enabled, config: values }); onClose() } catch (err) { setError(err instanceof ApiError ? String((err.body as { detail?: string })?.detail ?? `Error ${err.status}`) : 'Could not save source.') }
|
||||||
|
}
|
||||||
|
return <Modal opened onClose={onClose} title={source ? 'Edit Source' : 'New Source'}><form onSubmit={submit}><Stack>
|
||||||
|
<TextInput label="Name" required value={name} onChange={(e) => setName(e.currentTarget.value)} />
|
||||||
|
{profiles.isLoading && <Alert color="blue">Loading source profiles…</Alert>}{profiles.isError && <Alert color="red">Failed to load source profiles.</Alert>}
|
||||||
|
<Select label="Source type" required data={profiles.data?.items.map((p) => ({ value: p.kind, label: p.kind })) ?? []} value={kind} onChange={setKind} disabled={!!source} />
|
||||||
|
{kind === 'warmtelink_serial' && <Alert color="blue">Serial sources use <code>/dev/serial/by-id/…</code>; 115200 7N1.</Alert>}
|
||||||
|
{profile?.fields.map((field) => field.value_type === 'bool' || field.value_type === 'boolean' ? <Checkbox key={field.name} label={field.name} checked={Boolean(config[field.name] ?? source?.config[field.name] ?? field.default ?? false)} onChange={(e) => setConfig({ ...config, [field.name]: e.currentTarget.checked })} /> : <TextInput key={field.name} label={field.name} required={field.required} type={field.secret ? 'password' : (field.value_type === 'int' || field.value_type === 'integer' ? 'number' : 'text')} placeholder={field.secret && source ? 'Stored secret unchanged when blank' : undefined} value={String(config[field.name] ?? (field.secret ? '' : source?.config[field.name] ?? field.default ?? ''))} onChange={(e) => setConfig({ ...config, [field.name]: e.currentTarget.value })} />)}
|
||||||
|
<Checkbox label="Enabled" checked={enabled} onChange={(e) => setEnabled(e.currentTarget.checked)} />
|
||||||
|
{error && <Alert color="red">{error}</Alert>}<Group justify="flex-end"><Button variant="default" onClick={onClose}>Cancel</Button><Button type="submit" loading={create.isPending || update.isPending}>Save Source</Button></Group>
|
||||||
|
</Stack></form></Modal>
|
||||||
|
}
|
||||||
@@ -0,0 +1,17 @@
|
|||||||
|
import { describe, expect, it, vi, beforeEach } from 'vitest'
|
||||||
|
import { screen, waitFor } from '@testing-library/react'
|
||||||
|
import userEvent from '@testing-library/user-event'
|
||||||
|
import { renderWithProviders } from '../test-utils'
|
||||||
|
import { SourceManager } from './SourceManager'
|
||||||
|
const mockGet = vi.fn(); const mockPost = vi.fn(); const mockDelete = vi.fn()
|
||||||
|
vi.mock('../api/client', () => ({ default: { GET: (...a: unknown[]) => mockGet(...a), POST: (...a: unknown[]) => mockPost(...a), PATCH: vi.fn(), DELETE: (...a: unknown[]) => mockDelete(...a) }, ApiError: class ApiError extends Error { constructor(public status: number, public body: unknown) { super(`API error ${status}`) } }, registerLoginRedirect: vi.fn() }))
|
||||||
|
const source = { uuid: 's1', name: 'WarmteLink', kind: 'warmtelink_serial', enabled: true, status: 'online', config: {} }
|
||||||
|
describe('SourceManager API states', () => {
|
||||||
|
beforeEach(() => vi.clearAllMocks())
|
||||||
|
it('renders source list, detail and a completed bounded discovery refresh', async () => { const user = userEvent.setup(); mockGet.mockImplementation((path: string) => Promise.resolve({ data: path === '/api/energy/sources' ? { items: [source] } : path.includes('channels') ? { items: [] } : source })); mockPost.mockResolvedValue({ data: { status: 'completed' } }); renderWithProviders(<SourceManager />); await user.click(await screen.findByText('WarmteLink')); expect(await screen.findByText(/No channels discovered yet/)).toBeInTheDocument(); await user.click(screen.getByRole('button', { name: /Refresh discovered channels/ })); expect(await screen.findByText(/Discovery completed/)).toBeInTheDocument() })
|
||||||
|
it('explains that an enabled serial source reconnects automatically', async () => { const user = userEvent.setup(); mockGet.mockImplementation((path: string) => Promise.resolve({ data: path === '/api/energy/sources' ? { items: [{ ...source, status: 'error', last_error: 'serial unavailable' }] } : path.includes('channels') ? { items: [] } : { ...source, status: 'error', last_error: 'serial unavailable' } })); renderWithProviders(<SourceManager />); await user.click(await screen.findByText('WarmteLink')); expect(await screen.findAllByText(/reconnecting automatically/i)).not.toHaveLength(0); expect(screen.getByRole('button', { name: /Refresh discovered channels/ })).toBeInTheDocument() })
|
||||||
|
it('renders source list empty and error states', async () => { mockGet.mockResolvedValueOnce({ data: { items: [] } }); const { unmount } = renderWithProviders(<SourceManager />); expect(await screen.findByText(/No sources configured/)).toBeInTheDocument(); unmount(); mockGet.mockRejectedValueOnce(new Error('offline')); renderWithProviders(<SourceManager />); expect(await screen.findByText(/Failed to load sources/)).toBeInTheDocument() })
|
||||||
|
it('contains the wide four-column source table in a scroll area', async () => { mockGet.mockResolvedValue({ data: { items: [source] } }); renderWithProviders(<SourceManager />); expect(await screen.findByTestId('sources-table-scrollarea')).toBeInTheDocument(); expect(screen.getByTestId('sources-table')).toHaveStyle({ minWidth: '640px' }) })
|
||||||
|
it('safely deletes an unreferenced source and clears its selection', async () => { const user = userEvent.setup(); let items = [source]; mockGet.mockImplementation((path: string) => Promise.resolve({ data: path === '/api/energy/sources' ? { items } : path.includes('channels') ? { items: [] } : source })); mockDelete.mockImplementation(async () => { items = []; return { data: undefined } }); renderWithProviders(<SourceManager />); await user.click(await screen.findByText('WarmteLink')); await user.click(screen.getByRole('button', { name: 'Delete source' })); await waitFor(() => expect(mockDelete).toHaveBeenCalledWith('/api/energy/sources/{source_uuid}', { params: { path: { source_uuid: 's1' } } })); expect(await screen.findByText(/No sources configured/)).toBeInTheDocument(); expect(screen.queryByRole('button', { name: 'Delete source' })).not.toBeInTheDocument() })
|
||||||
|
it('keeps source visible and explains dependencies when deletion returns 409', async () => { const user = userEvent.setup(); mockGet.mockImplementation((path: string) => Promise.resolve({ data: path === '/api/energy/sources' ? { items: [source] } : path.includes('channels') ? { items: [] } : source })); const { ApiError } = await import('../api/client'); mockDelete.mockRejectedValue(new ApiError(409, { detail: 'dependent readings' })); renderWithProviders(<SourceManager />); await user.click(await screen.findByText('WarmteLink')); await user.click(screen.getByRole('button', { name: 'Delete source' })); expect(await screen.findByText(/dependent channels, readings, or meter bindings/)).toBeInTheDocument(); expect(screen.getByRole('button', { name: 'WarmteLink' })).toBeInTheDocument() })
|
||||||
|
})
|
||||||
@@ -0,0 +1,34 @@
|
|||||||
|
import { useState } from 'react'
|
||||||
|
import { Alert, Badge, Button, Center, Group, Loader, Paper, ScrollArea, Stack, Table, Text } from '@mantine/core'
|
||||||
|
import { DsmrPanel } from './DsmrPanel'
|
||||||
|
import { SourceForm } from './SourceForm'
|
||||||
|
import { SourceReadings } from './SourceReadings'
|
||||||
|
import { ApiError } from '../api/client'
|
||||||
|
import { useDeleteSource, useDiscoverSource, useSource, useSourceChannels, useSources, type MeterSourceResponse } from './hooks'
|
||||||
|
import { formatLocalDateTime } from '../utils/datetime'
|
||||||
|
|
||||||
|
export function SourceManager() {
|
||||||
|
const sources = useSources(); const [selected, setSelected] = useState<string | null>(null); const [form, setForm] = useState<MeterSourceResponse | undefined | null>(null)
|
||||||
|
if (sources.isLoading) return <Center data-testid="sources-loading"><Loader /></Center>
|
||||||
|
if (sources.isError || !sources.data) return <Alert color="red">Failed to load sources.</Alert>
|
||||||
|
return <Stack data-testid="source-manager"><Group justify="space-between"><Text fw={600}>Sources</Text><Button onClick={() => setForm(undefined)}>New Source</Button></Group>
|
||||||
|
{sources.data.items.length === 0 ? <Text c="dimmed">No sources configured yet.</Text> : <ScrollArea data-testid="sources-table-scrollarea"><Table data-testid="sources-table" style={{ minWidth: 640 }}><Table.Thead><Table.Tr><Table.Th>Name</Table.Th><Table.Th>Type</Table.Th><Table.Th>Status</Table.Th><Table.Th>Last seen</Table.Th></Table.Tr></Table.Thead><Table.Tbody>{sources.data.items.map((source) => <Table.Tr key={source.uuid}><Table.Td><Button variant="subtle" onClick={() => setSelected(source.uuid)}>{source.name}</Button></Table.Td><Table.Td>{source.kind}</Table.Td><Table.Td><Badge color={source.enabled && source.status === 'online' ? 'green' : 'gray'}>{source.enabled ? source.status : 'disabled'}</Badge>{source.enabled && source.status === 'error' && <Text c="orange" size="xs">Reconnecting automatically.</Text>}{source.last_error && <Text c="red" size="xs">{source.last_error}</Text>}</Table.Td><Table.Td>{source.last_seen_at ? formatLocalDateTime(source.last_seen_at) : '—'}</Table.Td></Table.Tr>)}</Table.Tbody></Table></ScrollArea>}
|
||||||
|
{selected && <SourceDetail uuid={selected} onEdit={setForm} onDeleted={() => setSelected(null)} />}{form !== null && <SourceForm source={form} onClose={() => setForm(null)} />}
|
||||||
|
</Stack>
|
||||||
|
}
|
||||||
|
function SourceDetail({ uuid, onEdit, onDeleted }: { uuid: string; onEdit: (source: MeterSourceResponse) => void; onDeleted: () => void }) {
|
||||||
|
const source = useSource(uuid); const channels = useSourceChannels(uuid); const discover = useDiscoverSource(); const remove = useDeleteSource(); const [deleteError, setDeleteError] = useState<string | null>(null)
|
||||||
|
async function deleteSource() { setDeleteError(null); try { await remove.mutateAsync(uuid); onDeleted() } catch (err) { if (err instanceof ApiError && err.status === 409) setDeleteError('This source cannot be deleted because it still has dependent channels, readings, or meter bindings. Remove those dependencies first; no data was deleted.'); else setDeleteError('Could not delete source. No data was deleted.') } }
|
||||||
|
if (source.isLoading) return <Loader />; if (source.isError || !source.data) return <Alert color="red">Failed to load source.</Alert>
|
||||||
|
const detail = source.data!
|
||||||
|
const result = discover.data?.data
|
||||||
|
return <Paper withBorder p="md"><Stack><Group justify="space-between"><Text fw={600}>{detail.name}</Text><Group><Button variant="default" onClick={() => onEdit(detail)}>Edit</Button><Button loading={discover.isPending} onClick={() => discover.mutate(uuid)}>Refresh discovered channels</Button><Button color="red" variant="outline" loading={remove.isPending} onClick={deleteSource}>Delete source</Button></Group></Group>
|
||||||
|
{deleteError && <Alert color="red">{deleteError}</Alert>}
|
||||||
|
{detail.enabled && detail.status === 'error' && <Alert color="orange">The worker is reconnecting automatically. Refresh only requests a bounded status update; it does not start ingestion.</Alert>}
|
||||||
|
{discover.isPending && <Alert color="blue">Discovery pending…</Alert>}{discover.isError && <Alert color="red">Discovery request failed. Check the source and try again.</Alert>}
|
||||||
|
{result && <Alert color={result.status === 'error' || result.status === 'timeout' ? 'red' : 'blue'}>Discovery {result.status}: {result.detail ?? (result.status === 'completed' ? 'Channels refreshed.' : 'Waiting for discovery.')}</Alert>}
|
||||||
|
{detail.kind.includes('dsmr') && <DsmrPanel />}
|
||||||
|
{channels.isLoading && <Loader />}{channels.isError && <Alert color="red">Failed to load channels.</Alert>}{channels.data?.items.length === 0 && <Text c="dimmed">No channels discovered yet.</Text>}
|
||||||
|
{channels.data?.items.map((channel) => <Stack key={channel.uuid} gap="xs"><Text>{channel.label} ({channel.unit}) — suggestion: {channel.suggested_commodity ?? 'none'} (review before binding)</Text><Text size="sm">Latest: {channel.latest_value ?? '—'}; quality: {channel.latest_quality ?? 'unknown'}; bindings: {channel.binding_count}; meter IDs: {channel.bound_meter_ids.length ? channel.bound_meter_ids.join(', ') : 'none'}</Text><SourceReadings sourceUuid={uuid} channel={channel} /></Stack>)}
|
||||||
|
</Stack></Paper>
|
||||||
|
}
|
||||||
@@ -0,0 +1,13 @@
|
|||||||
|
import { describe, expect, it, vi, beforeEach } from 'vitest'
|
||||||
|
import { screen, waitFor } from '@testing-library/react'
|
||||||
|
import { renderWithProviders } from '../test-utils'
|
||||||
|
import { SourceReadings } from './SourceReadings'
|
||||||
|
const mockGet = vi.fn()
|
||||||
|
vi.mock('../api/client', () => ({ default: { GET: (...a: unknown[]) => mockGet(...a), POST: vi.fn(), PATCH: vi.fn(), DELETE: vi.fn() }, ApiError: class extends Error {}, registerLoginRedirect: vi.fn() }))
|
||||||
|
const channel = { uuid: 'channel-1', label: 'Heat', unit: 'GJ', latest_value: '1.2', latest_quality: 'unverifiable' }
|
||||||
|
describe('SourceReadings', () => {
|
||||||
|
beforeEach(() => vi.clearAllMocks())
|
||||||
|
it('renders quality and history from the mocked channel API', async () => { mockGet.mockResolvedValue({ data: { items: [{ recorded_at: '2026-08-01T10:00:00Z', value: '1.1', quality: 'unverifiable' }] } }); renderWithProviders(<SourceReadings sourceUuid="s1" channel={channel as never} />); expect(await screen.findByText(/shown for review, not marked verified/)).toBeInTheDocument(); await waitFor(() => expect(screen.getByText('1.1')).toBeInTheDocument()) })
|
||||||
|
it('renders an API error and empty history', async () => { mockGet.mockRejectedValueOnce(new Error('down')); const { unmount } = renderWithProviders(<SourceReadings sourceUuid="s1" channel={channel as never} />); expect(await screen.findByText('Failed to load channel history.')).toBeInTheDocument(); unmount(); mockGet.mockResolvedValue({ data: { items: [] } }); renderWithProviders(<SourceReadings sourceUuid="s1" channel={channel as never} />); await waitFor(() => expect(screen.getByText('No channel history yet.')).toBeInTheDocument()) })
|
||||||
|
it('keeps the latest quality explanation visible while history is loading or fails', async () => { mockGet.mockImplementationOnce(() => new Promise(() => {})); const { unmount } = renderWithProviders(<SourceReadings sourceUuid="s1" channel={channel as never} />); expect(screen.getByText('Quality: unverifiable')).toBeInTheDocument(); expect(screen.getByText(/shown for review, not marked verified/)).toBeInTheDocument(); unmount(); mockGet.mockRejectedValueOnce(new Error('down')); renderWithProviders(<SourceReadings sourceUuid="s1" channel={channel as never} />); expect(await screen.findByText('Failed to load channel history.')).toBeInTheDocument(); expect(screen.getByText('Quality: unverifiable')).toBeInTheDocument(); expect(screen.getByText(/shown for review, not marked verified/)).toBeInTheDocument() })
|
||||||
|
})
|
||||||
@@ -0,0 +1,12 @@
|
|||||||
|
import { Alert, Badge, Center, Loader, Stack, Table, Text } from '@mantine/core'
|
||||||
|
import { useChannelReadings, type MeterSourceChannelResponse } from './hooks'
|
||||||
|
import { formatLocalDateTime } from '../utils/datetime'
|
||||||
|
|
||||||
|
export function SourceReadings({ sourceUuid, channel }: { sourceUuid: string; channel: MeterSourceChannelResponse }) {
|
||||||
|
const query = useChannelReadings(sourceUuid, channel.uuid)
|
||||||
|
return <Stack gap="xs"><Text fw={500}>{channel.label} — latest {channel.latest_value ?? '—'} {channel.unit}</Text>
|
||||||
|
<Text size="sm">Quality: {channel.latest_quality ?? 'unknown'}</Text>
|
||||||
|
{channel.latest_quality === 'unverifiable' && <Alert color="yellow">This reading is unverifiable: it is shown for review, not marked verified.</Alert>}
|
||||||
|
{query.isLoading ? <Center><Loader /></Center> : query.isError || !query.data ? <Alert color="red">Failed to load channel history.</Alert> : query.data.items.length === 0 ? <Text c="dimmed">No channel history yet.</Text> : <Table><Table.Thead><Table.Tr><Table.Th>Recorded</Table.Th><Table.Th>Value</Table.Th><Table.Th>Quality</Table.Th></Table.Tr></Table.Thead><Table.Tbody>{query.data.items.map((row) => <Table.Tr key={row.recorded_at}><Table.Td>{formatLocalDateTime(row.recorded_at)}</Table.Td><Table.Td>{row.value ?? '—'}</Table.Td><Table.Td><Badge color={row.quality === 'unverifiable' ? 'yellow' : 'gray'}>{row.quality ?? 'unknown'}</Badge></Table.Td></Table.Tr>)}</Table.Tbody></Table>}
|
||||||
|
</Stack>
|
||||||
|
}
|
||||||
@@ -14,6 +14,7 @@
|
|||||||
|
|
||||||
import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest'
|
import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest'
|
||||||
import { screen, waitFor, fireEvent } from '@testing-library/react'
|
import { screen, waitFor, fireEvent } from '@testing-library/react'
|
||||||
|
import userEvent from '@testing-library/user-event'
|
||||||
import { renderWithProviders } from '../test-utils'
|
import { renderWithProviders } from '../test-utils'
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
@@ -244,6 +245,70 @@ describe('TibberPrices', () => {
|
|||||||
expect(screen.getByTestId('tariff-sell-dal')).toHaveTextContent('0.0900')
|
expect(screen.getByTestId('tariff-sell-dal')).toHaveTextContent('0.0900')
|
||||||
})
|
})
|
||||||
|
|
||||||
|
it('keeps thermal prices scoped and renders the complete D11 Decimal snapshot as a table', async () => {
|
||||||
|
const user = userEvent.setup()
|
||||||
|
const thermal = {
|
||||||
|
kind: 'district_heating', currency: 'EUR', points: [], tariff: null,
|
||||||
|
contract_version_id: 42, effective_from: '2026-01-01T00:00:00Z', effective_to: '2026-12-31T00:00:00Z',
|
||||||
|
values: {
|
||||||
|
variable: {
|
||||||
|
heating: '20.123456789123456789', hot_water_heating: '8.200000000000000001',
|
||||||
|
hot_water: '1.234567890123456789', hot_water_tax: '0.456789012345678901',
|
||||||
|
},
|
||||||
|
standing: {
|
||||||
|
heating_network: '100.000000000000000001', metering: '0', delivery_set: '20.2',
|
||||||
|
hot_water_network: '30.3', other: '40.4',
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
mockGet.mockImplementation((_path: string, options?: { params?: { query?: { scope?: string } } }) =>
|
||||||
|
Promise.resolve({ data: options?.params?.query?.scope === 'thermal'
|
||||||
|
? thermal
|
||||||
|
: { kind: 'manual', currency: 'EUR', points: [], tariff: { buy_dal: 0.1, buy_normal: 0.2, sell_dal: 0.03, sell_normal: 0.04 } } }),
|
||||||
|
)
|
||||||
|
|
||||||
|
renderWithProviders(<TibberPrices />)
|
||||||
|
await waitFor(() => expect(screen.getByTestId('manual-tariff-table')).toBeInTheDocument())
|
||||||
|
await user.click(screen.getByText('Thermal'))
|
||||||
|
await waitFor(() => expect(screen.getByTestId('thermal-price-snapshot')).toBeInTheDocument())
|
||||||
|
|
||||||
|
const snapshot = screen.getByTestId('thermal-price-snapshot')
|
||||||
|
const table = screen.getByTestId('thermal-price-table')
|
||||||
|
expect(snapshot).toHaveTextContent('Version 42')
|
||||||
|
expect(snapshot).toHaveTextContent('effective 2026-01-01T00:00:00Z to 2026-12-31T00:00:00Z')
|
||||||
|
expect(snapshot).toHaveTextContent('not a 15-minute market spot price')
|
||||||
|
expect(screen.getAllByRole('columnheader').map((header) => header.textContent)).toEqual([
|
||||||
|
'Category', 'Charge', 'Rate', 'Unit',
|
||||||
|
])
|
||||||
|
expect(table.querySelectorAll('tbody tr')).toHaveLength(9)
|
||||||
|
expect(screen.getByTestId('thermal-price-row-variable-heating')).toHaveTextContent('VariableHeating20.123456789123456789EUR/GJ')
|
||||||
|
expect(screen.getByTestId('thermal-price-row-variable-hot_water_heating')).toHaveTextContent('Hot Water Heating8.200000000000000001EUR/m³')
|
||||||
|
expect(screen.getByTestId('thermal-price-row-standing-heating_network')).toHaveTextContent('StandingHeating Network100.000000000000000001EUR/year')
|
||||||
|
for (const value of Object.values(thermal.values.standing)) {
|
||||||
|
expect(table).toHaveTextContent(value)
|
||||||
|
}
|
||||||
|
expect(mockGet).toHaveBeenCalledWith('/api/energy/prices', expect.objectContaining({
|
||||||
|
params: { query: expect.objectContaining({ scope: 'thermal' }) },
|
||||||
|
}))
|
||||||
|
|
||||||
|
await user.click(screen.getByText('Electricity'))
|
||||||
|
await waitFor(() => expect(screen.getByTestId('manual-tariff-table')).toBeInTheDocument())
|
||||||
|
expect(screen.queryByTestId('thermal-price-snapshot')).not.toBeInTheDocument()
|
||||||
|
})
|
||||||
|
|
||||||
|
it('keeps unknown thermal snapshot fields with safe human-readable labels', async () => {
|
||||||
|
const user = userEvent.setup()
|
||||||
|
mockGet.mockImplementation((_path: string, options?: { params?: { query?: { scope?: string } } }) => Promise.resolve({
|
||||||
|
data: options?.params?.query?.scope === 'thermal'
|
||||||
|
? { kind: 'district_heating', currency: 'EUR', values: { future_fee: { experimental_charge: '1.000000000000000001' } } }
|
||||||
|
: { kind: 'manual', currency: 'EUR', points: [], tariff: { buy_dal: 0.1, buy_normal: 0.2, sell_dal: 0.03, sell_normal: 0.04 } },
|
||||||
|
}))
|
||||||
|
renderWithProviders(<TibberPrices />)
|
||||||
|
await user.click(await screen.findByText('Thermal'))
|
||||||
|
const row = await screen.findByTestId('thermal-price-row-future_fee-experimental_charge')
|
||||||
|
expect(row).toHaveTextContent('Future FeeExperimental Charge1.000000000000000001EUR')
|
||||||
|
})
|
||||||
|
|
||||||
it('marks the currently active price slot with a dot and a caption', async () => {
|
it('marks the currently active price slot with a dot and a caption', async () => {
|
||||||
installChartSize()
|
installChartSize()
|
||||||
|
|
||||||
|
|||||||
@@ -22,6 +22,8 @@ import {
|
|||||||
Badge,
|
Badge,
|
||||||
Group,
|
Group,
|
||||||
Paper,
|
Paper,
|
||||||
|
SegmentedControl,
|
||||||
|
ScrollArea,
|
||||||
} from '@mantine/core'
|
} from '@mantine/core'
|
||||||
import {
|
import {
|
||||||
LineChart,
|
LineChart,
|
||||||
@@ -34,7 +36,8 @@ import {
|
|||||||
ReferenceDot,
|
ReferenceDot,
|
||||||
ResponsiveContainer,
|
ResponsiveContainer,
|
||||||
} from 'recharts'
|
} from 'recharts'
|
||||||
import { useEnergyPrices } from './hooks'
|
import { useQuery } from '@tanstack/react-query'
|
||||||
|
import apiClient from '../api/client'
|
||||||
import { formatLocalDate, formatLocalTime, parseBackendTimestamp } from '../utils/datetime'
|
import { formatLocalDate, formatLocalTime, parseBackendTimestamp } from '../utils/datetime'
|
||||||
|
|
||||||
const BUY_COLOR = '#2196f3'
|
const BUY_COLOR = '#2196f3'
|
||||||
@@ -46,6 +49,31 @@ const FALLBACK_SLOT_MS = 60 * 60 * 1000
|
|||||||
/** How often the "current price" marker re-evaluates which slot is active. */
|
/** How often the "current price" marker re-evaluates which slot is active. */
|
||||||
const NOW_TICK_MS = 30 * 1000
|
const NOW_TICK_MS = 30 * 1000
|
||||||
|
|
||||||
|
/** D11 thermal profile units. The API snapshot is Decimal strings, while the
|
||||||
|
* currency comes from its contract metadata. Keep this display-only: no rates
|
||||||
|
* are derived or prefilled in the browser. */
|
||||||
|
function thermalUnit(section: string, key: string, currency: string): string {
|
||||||
|
if (section === 'standing') return `${currency}/year`
|
||||||
|
if (key === 'heating') return `${currency}/GJ`
|
||||||
|
if (key === 'hot_water_heating' || key === 'hot_water' || key === 'hot_water_tax') {
|
||||||
|
return `${currency}/m³`
|
||||||
|
}
|
||||||
|
return currency
|
||||||
|
}
|
||||||
|
|
||||||
|
function humanizeThermalField(value: string): string {
|
||||||
|
return value
|
||||||
|
.replace(/_/g, ' ')
|
||||||
|
.replace(/\b\w/g, (letter) => letter.toUpperCase())
|
||||||
|
}
|
||||||
|
|
||||||
|
function thermalSections(values: Record<string, Record<string, string>>): string[] {
|
||||||
|
const preferred = ['variable', 'standing']
|
||||||
|
return [...preferred.filter((section) => section in values), ...Object.keys(values)
|
||||||
|
.filter((section) => !preferred.includes(section))
|
||||||
|
.sort()]
|
||||||
|
}
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
// Time range helpers
|
// Time range helpers
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
@@ -288,46 +316,59 @@ function ManualTariffTable({ tariff, currency }: ManualTariffTableProps) {
|
|||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
export function TibberPrices() {
|
export function TibberPrices() {
|
||||||
|
const [scope, setScope] = useState<'electricity' | 'thermal'>('electricity')
|
||||||
const start = getTodayStart()
|
const start = getTodayStart()
|
||||||
const end = getTomorrowEnd()
|
const end = getTomorrowEnd()
|
||||||
|
const { data, isLoading, isError } = useQuery({
|
||||||
|
queryKey: ['energy-prices', scope, start, end],
|
||||||
|
queryFn: async () => {
|
||||||
|
const res = await apiClient.GET('/api/energy/prices', { params: { query: { scope, start, end } } })
|
||||||
|
return res.data
|
||||||
|
},
|
||||||
|
})
|
||||||
|
|
||||||
const { data, isLoading, isError } = useEnergyPrices(start, end)
|
const selector = (
|
||||||
|
<SegmentedControl
|
||||||
|
value={scope}
|
||||||
|
onChange={(value) => setScope(value as 'electricity' | 'thermal')}
|
||||||
|
data={[{ label: 'Electricity', value: 'electricity' }, { label: 'Thermal', value: 'thermal' }]}
|
||||||
|
data-testid="prices-scope-selector"
|
||||||
|
/>
|
||||||
|
)
|
||||||
|
|
||||||
if (isLoading) {
|
if (isLoading) {
|
||||||
return (
|
return (
|
||||||
<Center py="xl" data-testid="prices-loading">
|
<Stack><Group><Text fw={500}>Energy Prices</Text>{selector}</Group><Center py="xl" data-testid="prices-loading"><Loader /></Center></Stack>
|
||||||
<Loader />
|
|
||||||
</Center>
|
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
if (isError) {
|
if (isError) {
|
||||||
return (
|
return (
|
||||||
<Alert color="red" data-testid="prices-error">
|
<Stack><Group><Text fw={500}>Energy Prices</Text>{selector}</Group><Alert color="red" data-testid="prices-error">
|
||||||
Failed to load energy prices. Please refresh.
|
Failed to load energy prices. Please refresh.
|
||||||
</Alert>
|
</Alert></Stack>
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
if (!data) {
|
if (!data) {
|
||||||
return (
|
return (
|
||||||
<Alert color="gray" data-testid="prices-no-data">
|
<Stack><Group><Text fw={500}>Energy Prices</Text>{selector}</Group><Alert color="gray" data-testid="prices-no-data">
|
||||||
No pricing data available.
|
No pricing data available.
|
||||||
</Alert>
|
</Alert></Stack>
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// No active contract
|
// No active contract
|
||||||
if (!data.kind) {
|
if (!data.kind) {
|
||||||
return (
|
return (
|
||||||
<Paper withBorder p="md" data-testid="prices-no-contract">
|
<Stack><Group><Text fw={500}>Energy Prices</Text>{selector}</Group><Paper withBorder p="md" data-testid="prices-no-contract">
|
||||||
<Stack gap="xs">
|
<Stack gap="xs">
|
||||||
<Text fw={500}>No active contract</Text>
|
<Text fw={500}>No active contract</Text>
|
||||||
<Text size="sm" c="dimmed">
|
<Text size="sm" c="dimmed">
|
||||||
Activate an energy contract on the Contracts tab to see pricing data.
|
Activate an energy contract on the Contracts tab to see pricing data.
|
||||||
</Text>
|
</Text>
|
||||||
</Stack>
|
</Stack>
|
||||||
</Paper>
|
</Paper></Stack>
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -337,6 +378,7 @@ export function TibberPrices() {
|
|||||||
<Stack gap="lg" data-testid="tibber-prices">
|
<Stack gap="lg" data-testid="tibber-prices">
|
||||||
<Group gap="sm" align="center">
|
<Group gap="sm" align="center">
|
||||||
<Text fw={500}>Energy Prices</Text>
|
<Text fw={500}>Energy Prices</Text>
|
||||||
|
{selector}
|
||||||
<Badge variant="outline" size="sm">
|
<Badge variant="outline" size="sm">
|
||||||
{data.kind}
|
{data.kind}
|
||||||
</Badge>
|
</Badge>
|
||||||
@@ -364,6 +406,38 @@ export function TibberPrices() {
|
|||||||
Manual tariff data not available.
|
Manual tariff data not available.
|
||||||
</Alert>
|
</Alert>
|
||||||
)}
|
)}
|
||||||
|
|
||||||
|
{scope === 'thermal' && data.values && (
|
||||||
|
<Paper withBorder p="md" data-testid="thermal-price-snapshot">
|
||||||
|
<Stack gap="xs">
|
||||||
|
<Text fw={500}>Thermal contract snapshot</Text>
|
||||||
|
<Text size="sm">Version {data.contract_version_id ?? '—'} · effective {data.effective_from ?? '—'} to {data.effective_to ?? 'open'}</Text>
|
||||||
|
<Text size="sm" c="dimmed">This is a contract snapshot, not a 15-minute market spot price.</Text>
|
||||||
|
<ScrollArea type="auto">
|
||||||
|
<Table withTableBorder withColumnBorders data-testid="thermal-price-table">
|
||||||
|
<Table.Thead>
|
||||||
|
<Table.Tr>
|
||||||
|
<Table.Th>Category</Table.Th>
|
||||||
|
<Table.Th>Charge</Table.Th>
|
||||||
|
<Table.Th>Rate</Table.Th>
|
||||||
|
<Table.Th>Unit</Table.Th>
|
||||||
|
</Table.Tr>
|
||||||
|
</Table.Thead>
|
||||||
|
<Table.Tbody>
|
||||||
|
{thermalSections(data.values!).flatMap((section) => Object.entries(data.values![section]).map(([key, rate]) => (
|
||||||
|
<Table.Tr key={`${section}-${key}`} data-testid={`thermal-price-row-${section}-${key}`}>
|
||||||
|
<Table.Td>{humanizeThermalField(section)}</Table.Td>
|
||||||
|
<Table.Td>{humanizeThermalField(key)}</Table.Td>
|
||||||
|
<Table.Td>{rate}</Table.Td>
|
||||||
|
<Table.Td>{thermalUnit(section, key, currency)}</Table.Td>
|
||||||
|
</Table.Tr>
|
||||||
|
)))}
|
||||||
|
</Table.Tbody>
|
||||||
|
</Table>
|
||||||
|
</ScrollArea>
|
||||||
|
</Stack>
|
||||||
|
</Paper>
|
||||||
|
)}
|
||||||
</Stack>
|
</Stack>
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -83,6 +83,8 @@ const SUMMARY = {
|
|||||||
metered_import: 10.5,
|
metered_import: 10.5,
|
||||||
metered_export: 2.3,
|
metered_export: 2.3,
|
||||||
metered_net: 8.2,
|
metered_net: 8.2,
|
||||||
|
metered_import_kwh: 33.3,
|
||||||
|
metered_export_kwh: 44.4,
|
||||||
fixed_costs: 5.0,
|
fixed_costs: 5.0,
|
||||||
credits: 50.0,
|
credits: 50.0,
|
||||||
total_payable: 12.5,
|
total_payable: 12.5,
|
||||||
@@ -148,6 +150,17 @@ describe('useCreateContract', () => {
|
|||||||
|
|
||||||
expect(mockPost).toHaveBeenCalledWith('/api/energy/contracts', { body })
|
expect(mockPost).toHaveBeenCalledWith('/api/energy/contracts', { body })
|
||||||
})
|
})
|
||||||
|
|
||||||
|
it('preserves the caller-selected thermal scope in the typed create payload', async () => {
|
||||||
|
mockPost.mockResolvedValue({ data: { id: 9 } })
|
||||||
|
const { Wrapper } = makeWrapper()
|
||||||
|
const { useCreateContract } = await import('./hooks')
|
||||||
|
const { result } = renderHook(() => useCreateContract(), { wrapper: Wrapper })
|
||||||
|
await act(async () => {
|
||||||
|
await result.current.mutateAsync({ name: 'Heat', kind: 'district_heating', scope: 'thermal', currency: 'EUR', values: {} })
|
||||||
|
})
|
||||||
|
expect(mockPost).toHaveBeenCalledWith('/api/energy/contracts', expect.objectContaining({ body: expect.objectContaining({ scope: 'thermal' }) }))
|
||||||
|
})
|
||||||
})
|
})
|
||||||
|
|
||||||
describe('useEnergyPrices', () => {
|
describe('useEnergyPrices', () => {
|
||||||
|
|||||||
@@ -81,6 +81,7 @@ function makeWrapper() {
|
|||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
describe('useDevices', () => {
|
describe('useDevices', () => {
|
||||||
|
// Source hooks use the same typed client and QueryClient invalidation boundary.
|
||||||
beforeEach(() => vi.clearAllMocks())
|
beforeEach(() => vi.clearAllMocks())
|
||||||
|
|
||||||
it('calls GET /api/modbus/devices and returns device list', async () => {
|
it('calls GET /api/modbus/devices and returns device list', async () => {
|
||||||
@@ -98,6 +99,57 @@ describe('useDevices', () => {
|
|||||||
})
|
})
|
||||||
})
|
})
|
||||||
|
|
||||||
|
describe('useDeclareMeter source binding invalidation', () => {
|
||||||
|
beforeEach(() => vi.clearAllMocks())
|
||||||
|
it('invalidates every cache affected by atomic meter and binding creation', async () => {
|
||||||
|
mockPost.mockResolvedValue({ data: { id: 9 } })
|
||||||
|
const { qc, Wrapper } = makeWrapper()
|
||||||
|
const affectedKeys = [
|
||||||
|
['energy-meters'], ['energy-source-channels'], ['energy-meter-bindings', 9],
|
||||||
|
['energy-sources'], ['energy-source', 'source-1'], ['energy-channel-readings', 'source-1', 'channel-1'],
|
||||||
|
['expose-catalog'], ['energy-costs', 'electricity'], ['energy-costs-summary', 'electricity'],
|
||||||
|
['meter-costs', 'thermal', 'month'], ['meter-cost-summary', 'thermal'],
|
||||||
|
]
|
||||||
|
for (const queryKey of affectedKeys) qc.setQueryData(queryKey, { cached: true })
|
||||||
|
const { useDeclareMeter } = await import('./hooks')
|
||||||
|
const { result } = renderHook(() => useDeclareMeter(), { wrapper: Wrapper })
|
||||||
|
await act(async () => { await result.current.mutateAsync({ label: 'Heat', commodity: 'heating', started_at: '2026-08-01T00:00:00Z', reason: 'initial', source_channel_uuid: 'channel-1' } as never) })
|
||||||
|
expect(mockPost).toHaveBeenCalledWith('/api/energy/meters', expect.objectContaining({ body: expect.objectContaining({ source_channel_uuid: 'channel-1' }) }))
|
||||||
|
for (const queryKey of affectedKeys) expect(qc.getQueryState(queryKey)?.isInvalidated).toBe(true)
|
||||||
|
})
|
||||||
|
})
|
||||||
|
|
||||||
|
describe('meter lifecycle mutations', () => {
|
||||||
|
beforeEach(() => vi.clearAllMocks())
|
||||||
|
|
||||||
|
it.each([
|
||||||
|
['Declare auto-handoff', 'post', async (hooks: typeof import('./hooks')) => hooks.useDeclareMeter, { label: 'swap', commodity: 'electricity', started_at: '2026-08-24T12:34', reason: 'meter_swap' }],
|
||||||
|
['Close meter', 'post', async (hooks: typeof import('./hooks')) => hooks.useCloseMeter, { id: 9, ended_at: '2026-08-24T12:34' }],
|
||||||
|
['Unbind', 'patch', async (hooks: typeof import('./hooks')) => hooks.useCloseBinding, { uuid: 'old', ended_at: '2026-08-24T12:34' }],
|
||||||
|
['Transfer', 'post', async (hooks: typeof import('./hooks')) => hooks.useTransferBinding, { id: 10, body: { from_binding_uuid: 'old', to_source_channel_uuid: 'new', effective_at: '2026-08-24T12:34' } }],
|
||||||
|
['Direct bind', 'post', async (hooks: typeof import('./hooks')) => hooks.useCreateBinding, { id: 10, body: { source_channel_uuid: 'new', started_at: '2026-08-24T12:34' } }],
|
||||||
|
['Update electricity start', 'patch', async (hooks: typeof import('./hooks')) => hooks.useUpdateMeter, { id: 10, body: { started_at: '2026-08-24T12:34' } }],
|
||||||
|
['Update heating label', 'patch', async (hooks: typeof import('./hooks')) => hooks.useUpdateMeter, { id: 11, body: { label: 'Heating meter' } }],
|
||||||
|
['Update hot water start', 'patch', async (hooks: typeof import('./hooks')) => hooks.useUpdateMeter, { id: 12, body: { started_at: '2026-08-24T12:34' } }],
|
||||||
|
])('%s invalidates every lifecycle view using a fresh QueryClient', async (_name, method, getHook, payload) => {
|
||||||
|
mockPost.mockResolvedValue({ data: {} })
|
||||||
|
mockPatch.mockResolvedValue({ data: {} })
|
||||||
|
const { qc, Wrapper } = makeWrapper()
|
||||||
|
const affectedKeys = [
|
||||||
|
['energy-meters'], ['energy-sources'], ['energy-source', 'source-1'], ['energy-source-channels'], ['energy-meter-bindings'],
|
||||||
|
['energy-channel-readings', 'source-1', 'channel-1'], ['energy-costs', 'electricity'], ['energy-costs-summary', 'electricity'],
|
||||||
|
['meter-costs', 'thermal', 'month'], ['meter-cost-summary', 'thermal'], ['expose-catalog'],
|
||||||
|
]
|
||||||
|
for (const key of affectedKeys) qc.setQueryData(key, { cached: true })
|
||||||
|
const hooks = await import('./hooks')
|
||||||
|
const useHook = await getHook(hooks)
|
||||||
|
const result = renderHook(() => useHook(), { wrapper: Wrapper })
|
||||||
|
await act(async () => { await result.result.current.mutateAsync(payload as never) })
|
||||||
|
expect(method === 'post' ? mockPost : mockPatch).toHaveBeenCalled()
|
||||||
|
for (const key of affectedKeys) expect(qc.getQueryState(key)?.isInvalidated).toBe(true)
|
||||||
|
})
|
||||||
|
})
|
||||||
|
|
||||||
describe('useProfiles', () => {
|
describe('useProfiles', () => {
|
||||||
beforeEach(() => vi.clearAllMocks())
|
beforeEach(() => vi.clearAllMocks())
|
||||||
|
|
||||||
|
|||||||
@@ -237,6 +237,15 @@ export type SummaryResponse = components['schemas']['SummaryResponse']
|
|||||||
export type DsmrLatestResponse = components['schemas']['DsmrLatestResponse']
|
export type DsmrLatestResponse = components['schemas']['DsmrLatestResponse']
|
||||||
export type TibberTestResponse = components['schemas']['TibberTestResponse']
|
export type TibberTestResponse = components['schemas']['TibberTestResponse']
|
||||||
export type TibberTestPriceSchema = components['schemas']['TibberTestPriceSchema']
|
export type TibberTestPriceSchema = components['schemas']['TibberTestPriceSchema']
|
||||||
|
export type SourceProfileResponse = components['schemas']['SourceProfileResponse']
|
||||||
|
export type MeterSourceResponse = components['schemas']['MeterSourceResponse']
|
||||||
|
export type MeterSourceCreate = components['schemas']['MeterSourceCreate']
|
||||||
|
export type MeterSourcePatch = components['schemas']['MeterSourcePatch']
|
||||||
|
export type MeterSourceChannelResponse = components['schemas']['MeterSourceChannelResponse']
|
||||||
|
export type BindingResponse = components['schemas']['BindingResponse']
|
||||||
|
export type BindingCreate = components['schemas']['BindingCreate']
|
||||||
|
export type BindingTransferRequest = components['schemas']['BindingTransferRequest']
|
||||||
|
export type MeterCloseRequest = components['schemas']['MeterCloseRequest']
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
// Query: list all energy contracts
|
// Query: list all energy contracts
|
||||||
@@ -470,12 +479,9 @@ export function useDeclareMeter() {
|
|||||||
return useMutation({
|
return useMutation({
|
||||||
mutationFn: (body: MeterDeclareRequest) =>
|
mutationFn: (body: MeterDeclareRequest) =>
|
||||||
apiClient.POST('/api/energy/meters', { body }),
|
apiClient.POST('/api/energy/meters', { body }),
|
||||||
onSuccess: () => {
|
// A meter_swap can hand a binding over even when the optional channel was
|
||||||
void qc.invalidateQueries({ queryKey: ['energy-meters'] })
|
// omitted from this request, so every lifecycle write shares this boundary.
|
||||||
// Invalidate cost-related queries: a new meter may trigger recompute server-side.
|
onSuccess: () => invalidateLifecycleQueries(qc),
|
||||||
void qc.invalidateQueries({ queryKey: ['energy-costs'] })
|
|
||||||
void qc.invalidateQueries({ queryKey: ['energy-costs-summary'] })
|
|
||||||
},
|
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -491,15 +497,48 @@ export function useUpdateMeter() {
|
|||||||
params: { path: { meter_id: id } },
|
params: { path: { meter_id: id } },
|
||||||
body,
|
body,
|
||||||
}),
|
}),
|
||||||
onSuccess: () => {
|
onSuccess: () => invalidateLifecycleQueries(qc),
|
||||||
void qc.invalidateQueries({ queryKey: ['energy-meters'] })
|
|
||||||
// Retroactive started_at correction triggers recompute server-side.
|
|
||||||
void qc.invalidateQueries({ queryKey: ['energy-costs'] })
|
|
||||||
void qc.invalidateQueries({ queryKey: ['energy-costs-summary'] })
|
|
||||||
},
|
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Source → channel → binding hooks. These deliberately use the generated
|
||||||
|
// OpenAPI types; UI suggestions remain just suggestions until a user binds one.
|
||||||
|
export function useSourceProfiles() {
|
||||||
|
return useQuery({ queryKey: ['energy-source-profiles'], queryFn: async () => {
|
||||||
|
const res = await apiClient.GET('/api/energy/source-profiles'); return res.data
|
||||||
|
}, staleTime: 5 * 60 * 1000 })
|
||||||
|
}
|
||||||
|
export function useSources() {
|
||||||
|
return useQuery({ queryKey: ['energy-sources'], queryFn: async () => {
|
||||||
|
const res = await apiClient.GET('/api/energy/sources'); return res.data
|
||||||
|
}, refetchInterval: 5_000 })
|
||||||
|
}
|
||||||
|
export function useSource(uuid: string | null) {
|
||||||
|
return useQuery({ queryKey: ['energy-source', uuid], enabled: !!uuid, queryFn: async () => {
|
||||||
|
const res = await apiClient.GET('/api/energy/sources/{source_uuid}', { params: { path: { source_uuid: uuid! } } }); return res.data
|
||||||
|
}, refetchInterval: 3_000 })
|
||||||
|
}
|
||||||
|
function invalidateLifecycleQueries(qc: ReturnType<typeof useQueryClient>) {
|
||||||
|
void qc.invalidateQueries({ queryKey: ['energy-sources'] }); void qc.invalidateQueries({ queryKey: ['energy-source'] });
|
||||||
|
void qc.invalidateQueries({ queryKey: ['energy-source-channels'] }); void qc.invalidateQueries({ queryKey: ['energy-meters'] });
|
||||||
|
void qc.invalidateQueries({ queryKey: ['energy-channel-readings'] }); void qc.invalidateQueries({ queryKey: ['energy-meter-bindings'] })
|
||||||
|
void qc.invalidateQueries({ queryKey: ['expose-catalog'] })
|
||||||
|
void qc.invalidateQueries({ queryKey: ['energy-costs'] }); void qc.invalidateQueries({ queryKey: ['energy-costs-summary'] })
|
||||||
|
void qc.invalidateQueries({ queryKey: ['meter-costs', 'thermal'] })
|
||||||
|
void qc.invalidateQueries({ queryKey: ['meter-cost-summary', 'thermal'] })
|
||||||
|
}
|
||||||
|
export function useCreateSource() { const qc = useQueryClient(); return useMutation({ mutationFn: (body: MeterSourceCreate) => apiClient.POST('/api/energy/sources', { body }), onSuccess: () => invalidateLifecycleQueries(qc) }) }
|
||||||
|
export function useUpdateSource() { const qc = useQueryClient(); return useMutation({ mutationFn: ({ uuid, body }: { uuid: string; body: MeterSourcePatch }) => apiClient.PATCH('/api/energy/sources/{source_uuid}', { params: { path: { source_uuid: uuid } }, body }), onSuccess: () => invalidateLifecycleQueries(qc) }) }
|
||||||
|
export function useDeleteSource() { const qc = useQueryClient(); return useMutation({ mutationFn: (uuid: string) => apiClient.DELETE('/api/energy/sources/{source_uuid}', { params: { path: { source_uuid: uuid } } }), onSuccess: () => invalidateLifecycleQueries(qc) }) }
|
||||||
|
export function useDiscoverSource() { const qc = useQueryClient(); return useMutation({ mutationFn: (uuid: string) => apiClient.POST('/api/energy/sources/{source_uuid}/discover', { params: { path: { source_uuid: uuid } } }), onSuccess: () => invalidateLifecycleQueries(qc) }) }
|
||||||
|
export function useSourceChannels(uuid: string | null) { return useQuery({ queryKey: ['energy-source-channels', uuid], enabled: !!uuid, queryFn: async () => { const res = await apiClient.GET('/api/energy/sources/{source_uuid}/channels', { params: { path: { source_uuid: uuid! } } }); return res.data }, refetchInterval: 3_000 }) }
|
||||||
|
export function useChannelReadings(sourceUuid: string | null, channelUuid: string | null) { return useQuery({ queryKey: ['energy-channel-readings', sourceUuid, channelUuid], enabled: !!sourceUuid && !!channelUuid, queryFn: async () => { const res = await apiClient.GET('/api/energy/sources/{source_uuid}/channels/{channel_uuid}/readings', { params: { path: { source_uuid: sourceUuid!, channel_uuid: channelUuid! }, query: { limit: 60 } } }); return res.data }, refetchInterval: 5_000 }) }
|
||||||
|
export function useMeterBindings(id: number | null) { return useQuery({ queryKey: ['energy-meter-bindings', id], enabled: id != null, queryFn: async () => { const res = await apiClient.GET('/api/energy/meters/{meter_id}/bindings', { params: { path: { meter_id: id! } } }); return res.data } }) }
|
||||||
|
export function useCreateBinding() { const qc = useQueryClient(); return useMutation({ mutationFn: ({ id, body }: { id: number; body: BindingCreate }) => apiClient.POST('/api/energy/meters/{meter_id}/bindings', { params: { path: { meter_id: id } }, body }), onSuccess: () => invalidateLifecycleQueries(qc) }) }
|
||||||
|
export function useCloseBinding() { const qc = useQueryClient(); return useMutation({ mutationFn: ({ uuid, ended_at }: { uuid: string; ended_at: string }) => apiClient.PATCH('/api/energy/bindings/{binding_uuid}', { params: { path: { binding_uuid: uuid } }, body: { ended_at } }), onSuccess: () => invalidateLifecycleQueries(qc) }) }
|
||||||
|
export function useCloseMeter() { const qc = useQueryClient(); return useMutation({ mutationFn: ({ id, ended_at }: { id: number; ended_at: string }) => apiClient.POST('/api/energy/meters/{meter_id}/close', { params: { path: { meter_id: id } }, body: { ended_at } satisfies MeterCloseRequest }), onSuccess: () => invalidateLifecycleQueries(qc) }) }
|
||||||
|
export function useTransferBinding() { const qc = useQueryClient(); return useMutation({ mutationFn: ({ id, body }: { id: number; body: BindingTransferRequest }) => apiClient.POST('/api/energy/meters/{meter_id}/bindings/transfer', { params: { path: { meter_id: id } }, body }), onSuccess: () => invalidateLifecycleQueries(qc) }) }
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
// Query: time-range readings for a device (window + limit — never full-table)
|
// Query: time-range readings for a device (window + limit — never full-table)
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
|
|||||||
@@ -15,6 +15,7 @@
|
|||||||
|
|
||||||
import { describe, it, expect, vi, beforeEach } from 'vitest'
|
import { describe, it, expect, vi, beforeEach } from 'vitest'
|
||||||
import { screen, waitFor, fireEvent } from '@testing-library/react'
|
import { screen, waitFor, fireEvent } from '@testing-library/react'
|
||||||
|
import userEvent from '@testing-library/user-event'
|
||||||
import { renderWithProviders } from '../test-utils'
|
import { renderWithProviders } from '../test-utils'
|
||||||
import { EnergyPage } from './EnergyPage'
|
import { EnergyPage } from './EnergyPage'
|
||||||
|
|
||||||
@@ -135,6 +136,19 @@ describe('EnergyPage — device list', () => {
|
|||||||
})
|
})
|
||||||
})
|
})
|
||||||
|
|
||||||
|
describe('EnergyPage — M8 navigation labels', () => {
|
||||||
|
beforeEach(() => { vi.clearAllMocks(); setupDefaultMocks() })
|
||||||
|
it('has keyboard-operable Sources, Modbus Devices and Meters tabs, without a DSMR tab', async () => {
|
||||||
|
const user = userEvent.setup(); renderEnergy()
|
||||||
|
const sources = screen.getByRole('tab', { name: 'Sources' })
|
||||||
|
expect(screen.getByRole('tab', { name: 'Modbus Devices' })).toBeInTheDocument()
|
||||||
|
expect(screen.getByRole('tab', { name: 'Meters' })).toBeInTheDocument()
|
||||||
|
expect(screen.queryByRole('tab', { name: /^DSMR$/ })).not.toBeInTheDocument()
|
||||||
|
sources.focus(); await user.keyboard('{Enter}')
|
||||||
|
expect(await screen.findByTestId('panel-sources')).toBeInTheDocument()
|
||||||
|
})
|
||||||
|
})
|
||||||
|
|
||||||
describe('EnergyPage — create device', () => {
|
describe('EnergyPage — create device', () => {
|
||||||
beforeEach(() => {
|
beforeEach(() => {
|
||||||
vi.clearAllMocks()
|
vi.clearAllMocks()
|
||||||
|
|||||||
@@ -45,7 +45,7 @@ import { ContractManager } from '../energy/ContractManager'
|
|||||||
import { MeterManager } from '../energy/MeterManager'
|
import { MeterManager } from '../energy/MeterManager'
|
||||||
import { TibberPrices } from '../energy/TibberPrices'
|
import { TibberPrices } from '../energy/TibberPrices'
|
||||||
import { CostView } from '../energy/CostView'
|
import { CostView } from '../energy/CostView'
|
||||||
import { DsmrPanel } from '../energy/DsmrPanel'
|
import { SourceManager } from '../energy/SourceManager'
|
||||||
import type { ModbusDevice, ModbusTestReadResponse, MetricInfo } from '../energy/hooks'
|
import type { ModbusDevice, ModbusTestReadResponse, MetricInfo } from '../energy/hooks'
|
||||||
import { ApiError } from '../api/client'
|
import { ApiError } from '../api/client'
|
||||||
import { formatMetricValue } from '../energy/format'
|
import { formatMetricValue } from '../energy/format'
|
||||||
@@ -603,7 +603,7 @@ function DevicesTab() {
|
|||||||
return (
|
return (
|
||||||
<Stack gap="lg">
|
<Stack gap="lg">
|
||||||
<Group justify="space-between" align="center">
|
<Group justify="space-between" align="center">
|
||||||
<Title order={2}>Energy — Devices</Title>
|
<Title order={2}>Energy — Modbus Devices</Title>
|
||||||
<Button onClick={openCreate} data-testid="device-new-button">
|
<Button onClick={openCreate} data-testid="device-new-button">
|
||||||
New Device
|
New Device
|
||||||
</Button>
|
</Button>
|
||||||
@@ -659,8 +659,11 @@ export function EnergyPage() {
|
|||||||
<Container size="xl" pt="xl" pb="xl" data-testid="energy-page">
|
<Container size="xl" pt="xl" pb="xl" data-testid="energy-page">
|
||||||
<Tabs defaultValue="devices">
|
<Tabs defaultValue="devices">
|
||||||
<Tabs.List mb="lg">
|
<Tabs.List mb="lg">
|
||||||
|
<Tabs.Tab value="sources" data-testid="tab-sources">
|
||||||
|
Sources
|
||||||
|
</Tabs.Tab>
|
||||||
<Tabs.Tab value="devices" data-testid="tab-devices">
|
<Tabs.Tab value="devices" data-testid="tab-devices">
|
||||||
Devices
|
Modbus Devices
|
||||||
</Tabs.Tab>
|
</Tabs.Tab>
|
||||||
<Tabs.Tab value="meters" data-testid="tab-meters">
|
<Tabs.Tab value="meters" data-testid="tab-meters">
|
||||||
Meters
|
Meters
|
||||||
@@ -674,11 +677,12 @@ export function EnergyPage() {
|
|||||||
<Tabs.Tab value="costs" data-testid="tab-costs">
|
<Tabs.Tab value="costs" data-testid="tab-costs">
|
||||||
Costs
|
Costs
|
||||||
</Tabs.Tab>
|
</Tabs.Tab>
|
||||||
<Tabs.Tab value="dsmr" data-testid="tab-dsmr">
|
|
||||||
DSMR
|
|
||||||
</Tabs.Tab>
|
|
||||||
</Tabs.List>
|
</Tabs.List>
|
||||||
|
|
||||||
|
<Tabs.Panel value="sources" data-testid="panel-sources">
|
||||||
|
<SourceManager />
|
||||||
|
</Tabs.Panel>
|
||||||
|
|
||||||
<Tabs.Panel value="devices" data-testid="panel-devices">
|
<Tabs.Panel value="devices" data-testid="panel-devices">
|
||||||
<DevicesTab />
|
<DevicesTab />
|
||||||
</Tabs.Panel>
|
</Tabs.Panel>
|
||||||
@@ -699,9 +703,6 @@ export function EnergyPage() {
|
|||||||
<CostView />
|
<CostView />
|
||||||
</Tabs.Panel>
|
</Tabs.Panel>
|
||||||
|
|
||||||
<Tabs.Panel value="dsmr" data-testid="panel-dsmr">
|
|
||||||
<DsmrPanel />
|
|
||||||
</Tabs.Panel>
|
|
||||||
</Tabs>
|
</Tabs>
|
||||||
</Container>
|
</Container>
|
||||||
)
|
)
|
||||||
|
|||||||
+2366
-6
File diff suppressed because it is too large
Load Diff
+1533
-8
File diff suppressed because it is too large
Load Diff
@@ -7,6 +7,7 @@ paho-mqtt>=2.0,<3.0
|
|||||||
pymodbus>=3.6,<4.0
|
pymodbus>=3.6,<4.0
|
||||||
pydantic-settings>=2.6,<3.0
|
pydantic-settings>=2.6,<3.0
|
||||||
pyotp>=2.9,<3.0
|
pyotp>=2.9,<3.0
|
||||||
|
pyserial>=3.5,<4.0
|
||||||
python-multipart>=0.0.12,<1.0
|
python-multipart>=0.0.12,<1.0
|
||||||
pyyaml>=6.0,<7.0
|
pyyaml>=6.0,<7.0
|
||||||
sqlalchemy>=2.0,<3.0
|
sqlalchemy>=2.0,<3.0
|
||||||
|
|||||||
@@ -65,6 +65,8 @@ pymodbus==3.13.1
|
|||||||
# via -r requirements.in
|
# via -r requirements.in
|
||||||
pyotp==2.10.0
|
pyotp==2.10.0
|
||||||
# via -r requirements.in
|
# via -r requirements.in
|
||||||
|
pyserial==3.5
|
||||||
|
# via -r requirements.in
|
||||||
python-dotenv==1.2.2
|
python-dotenv==1.2.2
|
||||||
# via
|
# via
|
||||||
# pydantic-settings
|
# pydantic-settings
|
||||||
|
|||||||
@@ -15,7 +15,7 @@ if str(PROJECT_ROOT) not in sys.path:
|
|||||||
|
|
||||||
from app.config import get_settings
|
from app.config import get_settings
|
||||||
|
|
||||||
APP_BASELINE_REVISION = "20260625_14_meter_uuid"
|
APP_BASELINE_REVISION = "20260822_19_meter_cost_periods"
|
||||||
|
|
||||||
|
|
||||||
class AppDatabaseAdoptionError(RuntimeError):
|
class AppDatabaseAdoptionError(RuntimeError):
|
||||||
|
|||||||
@@ -0,0 +1,214 @@
|
|||||||
|
"""Read-only parser and command-line probe for DSMR and WarmteLink P1 telegrams."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import errno
|
||||||
|
import sys
|
||||||
|
import time
|
||||||
|
from typing import BinaryIO, Callable, TextIO
|
||||||
|
|
||||||
|
import serial
|
||||||
|
from app.integrations.p1 import IntegrityStatus, ObisField, P1Telegram, TelegramFramer, parse_telegram
|
||||||
|
|
||||||
|
__all__ = ["IntegrityStatus", "TelegramFramer", "build_parser", "parse_telegram", "run_probe"]
|
||||||
|
|
||||||
|
|
||||||
|
def build_parser() -> argparse.ArgumentParser:
|
||||||
|
"""Build the CLI parser for an explicitly selected, read-only serial device."""
|
||||||
|
|
||||||
|
parser = argparse.ArgumentParser(
|
||||||
|
description="Read-only WarmteLink P1 serial probe; it never writes to the device.",
|
||||||
|
epilog="Defaults are the locally measured WarmteLink 115200 7N1 framing, not DSMR defaults.",
|
||||||
|
)
|
||||||
|
parser.add_argument("--device", required=True, help="Explicit serial path, preferably /dev/serial/by-id/..."
|
||||||
|
)
|
||||||
|
parser.add_argument(
|
||||||
|
"--baudrate",
|
||||||
|
type=int,
|
||||||
|
default=115200,
|
||||||
|
help="Baud rate (default: 115200, the locally measured WarmteLink value).",
|
||||||
|
)
|
||||||
|
parser.add_argument(
|
||||||
|
"--bytesize",
|
||||||
|
type=int,
|
||||||
|
choices=(5, 6, 7, 8),
|
||||||
|
default=7,
|
||||||
|
help="Data bits (default: 7, locally measured; not the DSMR standard default).",
|
||||||
|
)
|
||||||
|
parser.add_argument(
|
||||||
|
"--parity",
|
||||||
|
choices=("N", "E", "O", "M", "S"),
|
||||||
|
type=lambda value: value.upper(),
|
||||||
|
default="N",
|
||||||
|
help="Parity (default: N, locally measured; not the DSMR standard default).",
|
||||||
|
)
|
||||||
|
parser.add_argument(
|
||||||
|
"--stopbits",
|
||||||
|
type=float,
|
||||||
|
choices=(1, 1.5, 2),
|
||||||
|
default=1,
|
||||||
|
help="Stop bits (default: 1, locally measured; not the DSMR standard default).",
|
||||||
|
)
|
||||||
|
parser.add_argument(
|
||||||
|
"--duration",
|
||||||
|
type=_positive_duration,
|
||||||
|
default=600.0,
|
||||||
|
help="Maximum capture time in seconds (default: 600).",
|
||||||
|
)
|
||||||
|
parser.add_argument(
|
||||||
|
"--show-changes",
|
||||||
|
action="store_true",
|
||||||
|
help="After the first frame, print only OBIS fields whose values changed.",
|
||||||
|
)
|
||||||
|
parser.add_argument(
|
||||||
|
"--raw-output",
|
||||||
|
type=argparse.FileType("wb"),
|
||||||
|
help="Optional path for the exact raw bytes read from the serial device.",
|
||||||
|
)
|
||||||
|
return parser
|
||||||
|
|
||||||
|
|
||||||
|
def _positive_duration(value: str) -> float:
|
||||||
|
try:
|
||||||
|
duration = float(value)
|
||||||
|
except ValueError as exc:
|
||||||
|
raise argparse.ArgumentTypeError("duration must be a positive number") from exc
|
||||||
|
if duration <= 0:
|
||||||
|
raise argparse.ArgumentTypeError("duration must be greater than zero")
|
||||||
|
return duration
|
||||||
|
|
||||||
|
|
||||||
|
def _serial_error_message(exc: BaseException) -> str:
|
||||||
|
"""Return a practical, non-root diagnostic for a serial open/read failure."""
|
||||||
|
|
||||||
|
error_number = getattr(exc, "errno", None)
|
||||||
|
message = str(exc)
|
||||||
|
if error_number == errno.EACCES or "permission denied" in message.lower():
|
||||||
|
return f"serial permission denied: {message}. Add your user to the dialout group; do not run it as root."
|
||||||
|
if error_number == errno.EBUSY or "resource busy" in message.lower():
|
||||||
|
return f"serial device is busy: {message}. Close the program currently using this device and retry."
|
||||||
|
if error_number in {errno.ENODEV, errno.ENOENT, errno.EIO}:
|
||||||
|
return f"serial device disconnected or unavailable: {message}. Check the cable and --device path."
|
||||||
|
return f"serial I/O failed: {message}. Check the cable, device path, and serial framing settings."
|
||||||
|
|
||||||
|
|
||||||
|
def _format_field(field: ObisField) -> str:
|
||||||
|
raw_values = ", ".join(field.raw_values) or "<no values>"
|
||||||
|
value = f" parsed={field.value} {field.unit or ''}" if field.value is not None else ""
|
||||||
|
return f" {field.code}: {raw_values}{value}".rstrip()
|
||||||
|
|
||||||
|
|
||||||
|
def _comparison_values(field: ObisField) -> tuple[str, ...]:
|
||||||
|
"""Return a local change-detection token without exposing identifiers."""
|
||||||
|
|
||||||
|
return (field.comparison_token,) if field.comparison_token is not None else field.raw_values
|
||||||
|
|
||||||
|
|
||||||
|
def _print_telegram(
|
||||||
|
telegram: P1Telegram,
|
||||||
|
frame_number: int,
|
||||||
|
cadence: float | None,
|
||||||
|
previous_fields: dict[str, tuple[str, ...]],
|
||||||
|
show_changes: bool,
|
||||||
|
output: TextIO,
|
||||||
|
) -> dict[str, tuple[str, ...]]:
|
||||||
|
cadence_text = "first frame" if cadence is None else f"cadence={cadence:.1f}s"
|
||||||
|
print(
|
||||||
|
f"frame {frame_number}: {telegram.integrity.value}; bytes={telegram.frame_length}; {cadence_text}",
|
||||||
|
file=output,
|
||||||
|
)
|
||||||
|
print(f" integrity: {telegram.integrity_reason}", file=output)
|
||||||
|
current_fields = {field.code: _comparison_values(field) for field in telegram.fields}
|
||||||
|
fields = telegram.fields
|
||||||
|
if show_changes and previous_fields:
|
||||||
|
fields = tuple(field for field in fields if previous_fields.get(field.code) != _comparison_values(field))
|
||||||
|
print(f" changed fields: {len(fields)}", file=output)
|
||||||
|
for field in fields:
|
||||||
|
print(_format_field(field), file=output)
|
||||||
|
for channel in telegram.channels:
|
||||||
|
print(
|
||||||
|
f" channel {channel.number}: device_type={channel.device_type or '<unknown>'}; "
|
||||||
|
f"readings={len(channel.readings)}",
|
||||||
|
file=output,
|
||||||
|
)
|
||||||
|
return current_fields
|
||||||
|
|
||||||
|
|
||||||
|
SerialFactory = Callable[..., serial.Serial]
|
||||||
|
|
||||||
|
|
||||||
|
def run_probe(
|
||||||
|
args: argparse.Namespace,
|
||||||
|
*,
|
||||||
|
serial_factory: SerialFactory = serial.Serial,
|
||||||
|
clock: Callable[[], float] = time.monotonic,
|
||||||
|
output: TextIO = sys.stdout,
|
||||||
|
error_output: TextIO = sys.stderr,
|
||||||
|
) -> int:
|
||||||
|
"""Capture and report frames until duration elapses or the user interrupts.
|
||||||
|
|
||||||
|
The only operation on ``serial_port`` is ``read``. It is always closed,
|
||||||
|
including after an interrupt, timeout, or a read error.
|
||||||
|
"""
|
||||||
|
|
||||||
|
raw_output: BinaryIO | None = args.raw_output
|
||||||
|
serial_port: serial.Serial | None = None
|
||||||
|
try:
|
||||||
|
serial_port = serial_factory(
|
||||||
|
port=args.device,
|
||||||
|
baudrate=args.baudrate,
|
||||||
|
bytesize=args.bytesize,
|
||||||
|
parity=args.parity,
|
||||||
|
stopbits=args.stopbits,
|
||||||
|
timeout=1,
|
||||||
|
)
|
||||||
|
framer = TelegramFramer()
|
||||||
|
deadline = clock() + args.duration
|
||||||
|
frame_number = 0
|
||||||
|
last_frame_at: float | None = None
|
||||||
|
previous_fields: dict[str, tuple[str, ...]] = {}
|
||||||
|
while clock() < deadline:
|
||||||
|
chunk = serial_port.read(4096)
|
||||||
|
if not chunk:
|
||||||
|
continue
|
||||||
|
if raw_output is not None:
|
||||||
|
raw_output.write(chunk)
|
||||||
|
raw_output.flush()
|
||||||
|
for frame in framer.feed(chunk):
|
||||||
|
now = clock()
|
||||||
|
telegram = parse_telegram(frame)
|
||||||
|
frame_number += 1
|
||||||
|
cadence = None if last_frame_at is None else now - last_frame_at
|
||||||
|
previous_fields = _print_telegram(
|
||||||
|
telegram,
|
||||||
|
frame_number,
|
||||||
|
cadence,
|
||||||
|
previous_fields,
|
||||||
|
args.show_changes,
|
||||||
|
output,
|
||||||
|
)
|
||||||
|
last_frame_at = now
|
||||||
|
return 0
|
||||||
|
except KeyboardInterrupt:
|
||||||
|
print("capture interrupted; serial device closed", file=output)
|
||||||
|
return 0
|
||||||
|
except (serial.SerialException, OSError) as exc:
|
||||||
|
print(_serial_error_message(exc), file=error_output)
|
||||||
|
return 1
|
||||||
|
finally:
|
||||||
|
if serial_port is not None:
|
||||||
|
serial_port.close()
|
||||||
|
if raw_output is not None:
|
||||||
|
raw_output.close()
|
||||||
|
|
||||||
|
|
||||||
|
def main(argv: list[str] | None = None) -> int:
|
||||||
|
"""Run the command-line probe."""
|
||||||
|
|
||||||
|
args = build_parser().parse_args(argv)
|
||||||
|
return run_probe(args)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__": # pragma: no cover - exercised through main()
|
||||||
|
raise SystemExit(main())
|
||||||
Vendored
+12
@@ -0,0 +1,12 @@
|
|||||||
|
/ISk5\2MT382-1000
|
||||||
|
|
||||||
|
1-3:0.2.8(50)
|
||||||
|
0-0:1.0.0(240822120000S)
|
||||||
|
0-0:96.1.1(TEST-GATEWAY)
|
||||||
|
0-1:24.1.0(006)
|
||||||
|
0-1:96.1.0(TEST-DHW)
|
||||||
|
0-1:24.2.1(240822120000S)(5.900*m3)
|
||||||
|
0-2:24.1.0(012)
|
||||||
|
0-2:96.1.0(TEST-HEAT)
|
||||||
|
0-2:24.2.1(240822120000S)(0.017*GJ)
|
||||||
|
!D18A
|
||||||
Vendored
+11
@@ -0,0 +1,11 @@
|
|||||||
|
)TU)2NWA-MYRSKY
|
||||||
|
0-2:24.2.1(240822120000S)(0.017*GJ)
|
||||||
|
0-1:96.1.0(KAM-REDACTED)
|
||||||
|
0-0:1.0.0(240822120000S)
|
||||||
|
0-2:24.1.0(012)
|
||||||
|
0-1:24.2.1(240822120000S)(5.900*m3)
|
||||||
|
0-0:96.1.1(WARMTE-REDACTED)
|
||||||
|
0-1:24.1.0(006)
|
||||||
|
0-2:96.1.0(KAM-REDACTED)
|
||||||
|
1-3:0.2.8(50)
|
||||||
|
!x7z?
|
||||||
+125
-109
@@ -187,15 +187,13 @@ def test_put_config_with_csrf_header_updates_app_name(
|
|||||||
assert app_name_field["value"] == "Updated via API"
|
assert app_name_field["value"] == "Updated via API"
|
||||||
|
|
||||||
|
|
||||||
def test_put_config_reapplies_dsmr_subscription(
|
def test_put_config_reapplies_dsmr_source_subscription(
|
||||||
client: TestClient, test_database_urls
|
client: TestClient, test_database_urls
|
||||||
) -> None:
|
) -> None:
|
||||||
"""Saving config must re-apply the DSMR subscription so enabling DSMR ingest
|
"""A config save triggers source subscription reconciliation."""
|
||||||
takes effect without an app restart (the route calls apply_dsmr_subscription
|
|
||||||
with the refreshed settings)."""
|
|
||||||
_login(client)
|
_login(client)
|
||||||
|
|
||||||
payload = _full_config_payload({"DSMR_INGEST_ENABLED": "true"})
|
payload = _full_config_payload()
|
||||||
with patch("app.services.dsmr_ingest.apply_dsmr_subscription") as spy:
|
with patch("app.services.dsmr_ingest.apply_dsmr_subscription") as spy:
|
||||||
response = client.put(
|
response = client.put(
|
||||||
"/api/config",
|
"/api/config",
|
||||||
@@ -205,8 +203,7 @@ def test_put_config_reapplies_dsmr_subscription(
|
|||||||
|
|
||||||
assert response.status_code == 200
|
assert response.status_code == 200
|
||||||
spy.assert_called_once()
|
spy.assert_called_once()
|
||||||
applied_settings = spy.call_args.args[0]
|
assert spy.call_args.args
|
||||||
assert applied_settings.dsmr_ingest_enabled is True
|
|
||||||
|
|
||||||
|
|
||||||
def test_put_config_blank_secret_keeps_existing_value(
|
def test_put_config_blank_secret_keeps_existing_value(
|
||||||
@@ -457,6 +454,7 @@ def test_get_config_includes_mqtt_section(client: TestClient) -> None:
|
|||||||
assert "MQTT_USERNAME" in env_names
|
assert "MQTT_USERNAME" in env_names
|
||||||
assert "MQTT_PASSWORD" in env_names
|
assert "MQTT_PASSWORD" in env_names
|
||||||
assert "MQTT_TLS_ENABLED" in env_names
|
assert "MQTT_TLS_ENABLED" in env_names
|
||||||
|
assert "MQTT_CLIENT_ID" in env_names
|
||||||
|
|
||||||
|
|
||||||
def test_get_config_includes_ha_discovery_section(client: TestClient) -> None:
|
def test_get_config_includes_ha_discovery_section(client: TestClient) -> None:
|
||||||
@@ -581,6 +579,18 @@ def test_put_config_invalid_mqtt_port_returns_422_and_does_not_write(
|
|||||||
assert rows.get("MQTT_BROKER_PORT") != "not-a-number"
|
assert rows.get("MQTT_BROKER_PORT") != "not-a-number"
|
||||||
|
|
||||||
|
|
||||||
|
def test_put_config_invalid_mqtt_client_id_returns_422(client: TestClient) -> None:
|
||||||
|
_login(client)
|
||||||
|
|
||||||
|
response = client.put(
|
||||||
|
"/api/config",
|
||||||
|
json={"updates": _full_config_payload({"MQTT_CLIENT_ID": "invalid id"})},
|
||||||
|
headers={"X-CSRF-Token": "token"},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert response.status_code == 422
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# M5-polish2 Area B: bool fields have input_type="checkbox"
|
# M5-polish2 Area B: bool fields have input_type="checkbox"
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
@@ -594,7 +604,6 @@ EXPECTED_CHECKBOX_FIELDS = {
|
|||||||
"MQTT_TLS_ENABLED",
|
"MQTT_TLS_ENABLED",
|
||||||
"HA_DISCOVERY_ENABLED",
|
"HA_DISCOVERY_ENABLED",
|
||||||
"MODBUS_POLLING_ENABLED",
|
"MODBUS_POLLING_ENABLED",
|
||||||
"DSMR_INGEST_ENABLED",
|
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
@@ -726,29 +735,46 @@ def test_put_config_mqtt_reconnect_uses_db_merged_settings(
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_put_config_mqtt_client_id_reconnects_and_trims_value(
|
||||||
|
client: TestClient, test_database_urls
|
||||||
|
) -> None:
|
||||||
|
_login(client)
|
||||||
|
mock_mgr = MagicMock()
|
||||||
|
|
||||||
|
with patch("app.api.routes.api.config.mqtt_manager", mock_mgr):
|
||||||
|
response = client.put(
|
||||||
|
"/api/config",
|
||||||
|
json={"updates": _full_config_payload({"MQTT_CLIENT_ID": " home-automation-dev "})},
|
||||||
|
headers={"X-CSRF-Token": "token"},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert response.status_code == 200
|
||||||
|
reconnect_settings = mock_mgr.reconnect.call_args.args[0]
|
||||||
|
assert reconnect_settings.mqtt_client_id == "home-automation-dev"
|
||||||
|
conn = sqlite3.connect(test_database_urls["app_path"])
|
||||||
|
try:
|
||||||
|
stored_value = conn.execute(
|
||||||
|
"SELECT value FROM app_config WHERE key = 'MQTT_CLIENT_ID'"
|
||||||
|
).fetchone()
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
assert stored_value == ("home-automation-dev",)
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# M6-T02: DSMR + Tibber CONFIG_FIELDS
|
# M6-T02: DSMR + Tibber CONFIG_FIELDS
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
def test_get_config_includes_dsmr_section(client: TestClient) -> None:
|
def test_get_config_excludes_legacy_dsmr_section(client: TestClient) -> None:
|
||||||
"""GET /api/config must include a DSMR section with expected fields including DSMR_TARIFF_TOPIC."""
|
"""DSMR is configured through MeterSource, never the legacy config form."""
|
||||||
_login(client)
|
_login(client)
|
||||||
|
|
||||||
response = client.get("/api/config")
|
response = client.get("/api/config")
|
||||||
body = response.json()
|
body = response.json()
|
||||||
|
|
||||||
section_names = {s["name"] for s in body["sections"]}
|
section_names = {s["name"] for s in body["sections"]}
|
||||||
assert "DSMR" in section_names, f"DSMR section missing; got {section_names}"
|
assert "DSMR" not in section_names
|
||||||
|
|
||||||
dsmr_section = next(s for s in body["sections"] if s["name"] == "DSMR")
|
|
||||||
env_names = {f["env_name"] for f in dsmr_section["fields"]}
|
|
||||||
assert "DSMR_INGEST_ENABLED" in env_names
|
|
||||||
assert "DSMR_MQTT_TOPIC" in env_names
|
|
||||||
assert "DSMR_SAMPLE_INTERVAL_S" in env_names
|
|
||||||
assert "DSMR_TARIFF_TOPIC" in env_names, (
|
|
||||||
f"DSMR_TARIFF_TOPIC must be present in DSMR section; got {env_names}"
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def test_get_config_includes_tibber_section(client: TestClient) -> None:
|
def test_get_config_includes_tibber_section(client: TestClient) -> None:
|
||||||
@@ -783,36 +809,46 @@ def test_get_config_tibber_api_token_is_secret(client: TestClient) -> None:
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
def test_get_config_dsmr_ingest_enabled_is_checkbox(client: TestClient) -> None:
|
def test_get_config_excludes_all_legacy_dsmr_fields(client: TestClient) -> None:
|
||||||
"""DSMR_INGEST_ENABLED must have input_type='checkbox' for correct frontend rendering."""
|
"""Old DSMR KV values stay in DB but are not returned by the config API."""
|
||||||
_login(client)
|
_login(client)
|
||||||
|
|
||||||
response = client.get("/api/config")
|
response = client.get("/api/config")
|
||||||
body = response.json()
|
body = response.json()
|
||||||
|
|
||||||
dsmr_section = next(s for s in body["sections"] if s["name"] == "DSMR")
|
fields = {field["env_name"] for section in body["sections"] for field in section["fields"]}
|
||||||
enabled_field = next(f for f in dsmr_section["fields"] if f["env_name"] == "DSMR_INGEST_ENABLED")
|
assert not fields.intersection({"DSMR_INGEST_ENABLED", "DSMR_MQTT_TOPIC", "DSMR_SAMPLE_INTERVAL_S", "DSMR_TARIFF_TOPIC"})
|
||||||
|
|
||||||
assert enabled_field["input_type"] == "checkbox", (
|
|
||||||
f"DSMR_INGEST_ENABLED input_type should be 'checkbox', got {enabled_field['input_type']!r}"
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def test_get_config_dsmr_sample_interval_input_type_is_number(client: TestClient) -> None:
|
def test_config_save_preserves_legacy_dsmr_kv_rows(client: TestClient, test_database_urls) -> None:
|
||||||
"""DSMR_SAMPLE_INTERVAL_S must have input_type='number'."""
|
"""A config-only save neither reads nor deletes retired DSMR configuration."""
|
||||||
_login(client)
|
_login(client)
|
||||||
|
conn = sqlite3.connect(test_database_urls["app_path"])
|
||||||
response = client.get("/api/config")
|
try:
|
||||||
body = response.json()
|
conn.execute(
|
||||||
|
"INSERT INTO app_config (key, value, updated_at) VALUES (?, ?, CURRENT_TIMESTAMP)",
|
||||||
dsmr_section = next(s for s in body["sections"] if s["name"] == "DSMR")
|
("DSMR_MQTT_TOPIC", "legacy/topic"),
|
||||||
interval_field = next(
|
|
||||||
f for f in dsmr_section["fields"] if f["env_name"] == "DSMR_SAMPLE_INTERVAL_S"
|
|
||||||
)
|
)
|
||||||
|
conn.execute(
|
||||||
assert interval_field["input_type"] == "number", (
|
"INSERT INTO app_config (key, value, updated_at) VALUES (?, ?, CURRENT_TIMESTAMP)",
|
||||||
f"DSMR_SAMPLE_INTERVAL_S input_type should be 'number', got {interval_field['input_type']!r}"
|
("DSMR_SAMPLE_INTERVAL_S", "37"),
|
||||||
)
|
)
|
||||||
|
conn.commit()
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
response = client.put(
|
||||||
|
"/api/config",
|
||||||
|
json={"updates": _full_config_payload({"APP_NAME": "config-only save"})},
|
||||||
|
headers={"X-CSRF-Token": "any-non-empty-value"},
|
||||||
|
)
|
||||||
|
assert response.status_code == 200
|
||||||
|
conn = sqlite3.connect(test_database_urls["app_path"])
|
||||||
|
try:
|
||||||
|
rows = dict(conn.execute("SELECT key, value FROM app_config WHERE key LIKE 'DSMR_%'"))
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
assert rows == {"DSMR_MQTT_TOPIC": "legacy/topic", "DSMR_SAMPLE_INTERVAL_S": "37"}
|
||||||
|
|
||||||
|
|
||||||
def test_put_config_blank_tibber_api_token_keeps_existing(
|
def test_put_config_blank_tibber_api_token_keeps_existing(
|
||||||
@@ -882,75 +918,6 @@ def test_put_config_new_tibber_api_token_overwrites_existing(
|
|||||||
assert rows.get("TIBBER_API_TOKEN") == "new-tibber-token"
|
assert rows.get("TIBBER_API_TOKEN") == "new-tibber-token"
|
||||||
|
|
||||||
|
|
||||||
def test_put_config_invalid_dsmr_sample_interval_returns_422_and_does_not_write(
|
|
||||||
client: TestClient, test_database_urls
|
|
||||||
) -> None:
|
|
||||||
"""Non-integer DSMR_SAMPLE_INTERVAL_S must return 422 and not persist the bad value."""
|
|
||||||
_login(client)
|
|
||||||
|
|
||||||
payload = _full_config_payload({"DSMR_SAMPLE_INTERVAL_S": "not-a-number"})
|
|
||||||
response = client.put(
|
|
||||||
"/api/config",
|
|
||||||
json={"updates": payload},
|
|
||||||
headers={"X-CSRF-Token": "token"},
|
|
||||||
)
|
|
||||||
|
|
||||||
assert response.status_code == 422
|
|
||||||
|
|
||||||
conn = sqlite3.connect(test_database_urls["app_path"])
|
|
||||||
try:
|
|
||||||
rows = dict(conn.execute("SELECT key, value FROM app_config").fetchall())
|
|
||||||
finally:
|
|
||||||
conn.close()
|
|
||||||
|
|
||||||
assert rows.get("DSMR_SAMPLE_INTERVAL_S") != "not-a-number"
|
|
||||||
|
|
||||||
|
|
||||||
def test_put_config_dsmr_tariff_topic_persists_and_reflects_in_get(
|
|
||||||
client: TestClient, test_database_urls
|
|
||||||
) -> None:
|
|
||||||
"""DSMR_TARIFF_TOPIC must persist via PUT and be readable via GET /api/config."""
|
|
||||||
_login(client)
|
|
||||||
|
|
||||||
new_topic = "meter/tariff/slot"
|
|
||||||
payload = _full_config_payload({"DSMR_TARIFF_TOPIC": new_topic})
|
|
||||||
with patch("app.services.dsmr_ingest.apply_dsmr_subscription"):
|
|
||||||
response = client.put(
|
|
||||||
"/api/config",
|
|
||||||
json={"updates": payload},
|
|
||||||
headers={"X-CSRF-Token": "token"},
|
|
||||||
)
|
|
||||||
assert response.status_code == 200
|
|
||||||
|
|
||||||
# The updated value must appear in the GET response.
|
|
||||||
get_resp = client.get("/api/config")
|
|
||||||
body = get_resp.json()
|
|
||||||
dsmr_section = next(s for s in body["sections"] if s["name"] == "DSMR")
|
|
||||||
tariff_field = next(f for f in dsmr_section["fields"] if f["env_name"] == "DSMR_TARIFF_TOPIC")
|
|
||||||
assert tariff_field["value"] == new_topic, (
|
|
||||||
f"Expected DSMR_TARIFF_TOPIC to be {new_topic!r}, got {tariff_field['value']!r}"
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def test_put_config_dsmr_tariff_topic_in_settings_payload(client: TestClient) -> None:
|
|
||||||
"""DSMR_TARIFF_TOPIC must appear in _settings_payload (GET /api/config returns it)."""
|
|
||||||
_login(client)
|
|
||||||
|
|
||||||
# The default value from Settings must appear in the DSMR section.
|
|
||||||
response = client.get("/api/config")
|
|
||||||
body = response.json()
|
|
||||||
|
|
||||||
dsmr_section = next(s for s in body["sections"] if s["name"] == "DSMR")
|
|
||||||
tariff_field = next(
|
|
||||||
(f for f in dsmr_section["fields"] if f["env_name"] == "DSMR_TARIFF_TOPIC"), None
|
|
||||||
)
|
|
||||||
assert tariff_field is not None, "DSMR_TARIFF_TOPIC must appear in DSMR config section"
|
|
||||||
# Default value should be the DSMR reader meter-stats topic.
|
|
||||||
assert tariff_field["value"] == "dsmr/meter-stats/electricity_tariff", (
|
|
||||||
f"Unexpected default for DSMR_TARIFF_TOPIC: {tariff_field['value']!r}"
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def test_get_config_tibber_api_token_value_masked_after_save(
|
def test_get_config_tibber_api_token_value_masked_after_save(
|
||||||
client: TestClient, test_database_urls
|
client: TestClient, test_database_urls
|
||||||
) -> None:
|
) -> None:
|
||||||
@@ -979,6 +946,55 @@ def test_get_config_tibber_api_token_value_masked_after_save(
|
|||||||
assert "some-tibber-token" not in resp_after.text
|
assert "some-tibber-token" not in resp_after.text
|
||||||
|
|
||||||
|
|
||||||
|
def test_put_tibber_config_triggers_refresh_only_for_active_tibber_contract(
|
||||||
|
client: TestClient, test_database_urls
|
||||||
|
) -> None:
|
||||||
|
"""Saved Tibber credentials request a refresh only when their contract is active."""
|
||||||
|
_login(client)
|
||||||
|
conn = sqlite3.connect(test_database_urls["app_path"])
|
||||||
|
try:
|
||||||
|
conn.execute(
|
||||||
|
"INSERT INTO energy_contract (name, kind, scope, active, currency, created_at, updated_at) "
|
||||||
|
"VALUES (?, ?, ?, ?, ?, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP)",
|
||||||
|
("Tibber", "tibber", "electricity", True, "EUR"),
|
||||||
|
)
|
||||||
|
conn.commit()
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
with patch("app.api.routes.api.config.trigger_tibber_refresh") as trigger:
|
||||||
|
response = client.put(
|
||||||
|
"/api/config",
|
||||||
|
json={"updates": _full_config_payload({"TIBBER_API_TOKEN": "new-secret-token"})},
|
||||||
|
headers={"X-CSRF-Token": "token"},
|
||||||
|
)
|
||||||
|
assert response.status_code == 200
|
||||||
|
trigger.assert_called_once()
|
||||||
|
|
||||||
|
with patch("app.api.routes.api.config.trigger_tibber_refresh") as trigger:
|
||||||
|
response = client.put(
|
||||||
|
"/api/config",
|
||||||
|
json={"updates": _full_config_payload({"TIBBER_API_TOKEN": ""})},
|
||||||
|
headers={"X-CSRF-Token": "token"},
|
||||||
|
)
|
||||||
|
assert response.status_code == 200
|
||||||
|
trigger.assert_not_called()
|
||||||
|
|
||||||
|
|
||||||
|
def test_put_tibber_config_does_not_refresh_without_active_tibber_contract(client: TestClient) -> None:
|
||||||
|
_login(client)
|
||||||
|
|
||||||
|
with patch("app.api.routes.api.config.trigger_tibber_refresh") as trigger:
|
||||||
|
response = client.put(
|
||||||
|
"/api/config",
|
||||||
|
json={"updates": _full_config_payload({"TIBBER_HOME_ID": "new-home-id"})},
|
||||||
|
headers={"X-CSRF-Token": "token"},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert response.status_code == 200
|
||||||
|
trigger.assert_not_called()
|
||||||
|
|
||||||
|
|
||||||
def test_post_mqtt_test_uses_db_broker_host(
|
def test_post_mqtt_test_uses_db_broker_host(
|
||||||
client: TestClient, test_database_urls
|
client: TestClient, test_database_urls
|
||||||
) -> None:
|
) -> None:
|
||||||
|
|||||||
+43
-11
@@ -304,6 +304,35 @@ def test_prices_no_contract_returns_empty(energy_client):
|
|||||||
assert body["points"] == []
|
assert body["points"] == []
|
||||||
assert body["tariff"] is None
|
assert body["tariff"] is None
|
||||||
assert "currency" in body
|
assert "currency" in body
|
||||||
|
assert client.get("/api/energy/prices?scope=electricity").json() == body
|
||||||
|
|
||||||
|
|
||||||
|
def test_prices_thermal_returns_active_contract_snapshot(energy_client):
|
||||||
|
client, engine, _app = energy_client
|
||||||
|
now = datetime.now(UTC)
|
||||||
|
values = {
|
||||||
|
"variable": {"heating": "20.123456", "hot_water_heating": "4", "hot_water": "2", "hot_water_tax": "1"},
|
||||||
|
"standing": {"heating_network": "0", "metering": "0", "delivery_set": "0", "hot_water_network": "0", "other": "0"},
|
||||||
|
}
|
||||||
|
with Session(engine) as session:
|
||||||
|
contract = EnergyContract(name="thermal", kind="district_heating", scope="thermal", active=True,
|
||||||
|
currency="EUR", created_at=now, updated_at=now)
|
||||||
|
session.add(contract)
|
||||||
|
session.flush()
|
||||||
|
session.add(EnergyContractVersion(contract_id=contract.id, effective_from=now - timedelta(days=1),
|
||||||
|
values=values, created_at=now))
|
||||||
|
session.commit()
|
||||||
|
_login(client)
|
||||||
|
response = client.get("/api/energy/prices?scope=thermal")
|
||||||
|
assert response.status_code == 200
|
||||||
|
body = response.json()
|
||||||
|
assert body == {
|
||||||
|
"kind": "district_heating", "currency": "EUR", "points": [],
|
||||||
|
"contract_version_id": body["contract_version_id"],
|
||||||
|
"effective_from": body["effective_from"], "values": values,
|
||||||
|
}
|
||||||
|
assert client.get("/api/energy/prices").json()["points"] == []
|
||||||
|
assert client.get("/api/energy/prices?scope=invalid").status_code == 422
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
@@ -334,6 +363,7 @@ def test_prices_manual_contract_returns_tariff(energy_client):
|
|||||||
assert abs(tariff["sell_dal"] - 0.10) < 1e-6
|
assert abs(tariff["sell_dal"] - 0.10) < 1e-6
|
||||||
# sell_normal = 0.10
|
# sell_normal = 0.10
|
||||||
assert abs(tariff["sell_normal"] - 0.10) < 1e-6
|
assert abs(tariff["sell_normal"] - 0.10) < 1e-6
|
||||||
|
assert client.get("/api/energy/prices?scope=electricity").json() == body
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
@@ -369,6 +399,7 @@ def test_prices_tibber_contract_returns_points(energy_client):
|
|||||||
assert abs(p["buy"] - 0.245) < 1e-6
|
assert abs(p["buy"] - 0.245) < 1e-6
|
||||||
assert abs(p["sell"] - (0.245 - 0.1108)) < 1e-4
|
assert abs(p["sell"] - (0.245 - 0.1108)) < 1e-4
|
||||||
assert p["level"] == "NORMAL"
|
assert p["level"] == "NORMAL"
|
||||||
|
assert client.get("/api/energy/prices", params={"scope": "electricity", "start": start, "end": end}).json() == body
|
||||||
|
|
||||||
|
|
||||||
def test_prices_tibber_sell_reflects_sell_fee(energy_client):
|
def test_prices_tibber_sell_reflects_sell_fee(energy_client):
|
||||||
@@ -430,9 +461,7 @@ def test_prices_tibber_limit_caps_results(energy_client):
|
|||||||
|
|
||||||
start = (datetime.now(UTC) - timedelta(hours=3)).isoformat()
|
start = (datetime.now(UTC) - timedelta(hours=3)).isoformat()
|
||||||
end = (datetime.now(UTC) + timedelta(hours=2)).isoformat()
|
end = (datetime.now(UTC) + timedelta(hours=2)).isoformat()
|
||||||
resp = client.get(
|
resp = client.get("/api/energy/prices", params={"start": start, "end": end, "limit": 2})
|
||||||
"/api/energy/prices", params={"start": start, "end": end, "limit": 2}
|
|
||||||
)
|
|
||||||
assert resp.status_code == 200
|
assert resp.status_code == 200
|
||||||
body = resp.json()
|
body = resp.json()
|
||||||
assert len(body["points"]) <= 2
|
assert len(body["points"]) <= 2
|
||||||
@@ -508,9 +537,16 @@ def test_costs_schema_fields_present(energy_client):
|
|||||||
item = resp.json()["items"][0]
|
item = resp.json()["items"][0]
|
||||||
for field in (
|
for field in (
|
||||||
"period_start",
|
"period_start",
|
||||||
"d1_kwh", "d2_kwh", "r1_kwh", "r2_kwh",
|
"d1_kwh",
|
||||||
"import_cost", "export_revenue", "net_cost",
|
"d2_kwh",
|
||||||
"currency", "degraded",
|
"r1_kwh",
|
||||||
|
"r2_kwh",
|
||||||
|
"import_cost",
|
||||||
|
"export_revenue",
|
||||||
|
"net_cost",
|
||||||
|
"currency",
|
||||||
|
"degraded",
|
||||||
|
"source_binding_id",
|
||||||
):
|
):
|
||||||
assert field in item, f"Missing field: {field}"
|
assert field in item, f"Missing field: {field}"
|
||||||
|
|
||||||
@@ -556,9 +592,7 @@ def test_summary_returns_correct_structure(energy_client):
|
|||||||
|
|
||||||
start = (datetime.now(UTC) - timedelta(hours=3)).isoformat()
|
start = (datetime.now(UTC) - timedelta(hours=3)).isoformat()
|
||||||
end = datetime.now(UTC).isoformat()
|
end = datetime.now(UTC).isoformat()
|
||||||
resp = client.get(
|
resp = client.get("/api/energy/costs/summary", params={"start": start, "end": end})
|
||||||
"/api/energy/costs/summary", params={"start": start, "end": end}
|
|
||||||
)
|
|
||||||
assert resp.status_code == 200
|
assert resp.status_code == 200
|
||||||
body = resp.json()
|
body = resp.json()
|
||||||
for field in (
|
for field in (
|
||||||
@@ -903,5 +937,3 @@ def test_tibber_test_token_not_in_response(energy_client):
|
|||||||
|
|
||||||
assert resp.status_code == 200
|
assert resp.status_code == 200
|
||||||
assert secret_token not in resp.text
|
assert secret_token not in resp.text
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -48,6 +48,7 @@ from __future__ import annotations
|
|||||||
|
|
||||||
from datetime import UTC, datetime, timedelta
|
from datetime import UTC, datetime, timedelta
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
from unittest.mock import patch
|
||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
from fastapi.testclient import TestClient
|
from fastapi.testclient import TestClient
|
||||||
@@ -55,6 +56,7 @@ from sqlalchemy import create_engine, select
|
|||||||
from sqlalchemy.orm import Session
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
from app.models.energy import EnergyContract, EnergyContractVersion
|
from app.models.energy import EnergyContract, EnergyContractVersion
|
||||||
|
from app.services.contracts import activate_contract
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# Shared helpers
|
# Shared helpers
|
||||||
@@ -103,6 +105,15 @@ _TIBBER_VALUES: dict[str, Any] = {
|
|||||||
},
|
},
|
||||||
}
|
}
|
||||||
|
|
||||||
|
_DISTRICT_HEATING_VALUES: dict[str, Any] = {
|
||||||
|
"variable": {
|
||||||
|
"heating": "12.34",
|
||||||
|
"hot_water_heating": "1.20",
|
||||||
|
"hot_water": "2.30",
|
||||||
|
"hot_water_tax": "0.10",
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
def _manual_payload(**overrides) -> dict[str, Any]:
|
def _manual_payload(**overrides) -> dict[str, Any]:
|
||||||
base: dict[str, Any] = {
|
base: dict[str, Any] = {
|
||||||
@@ -126,6 +137,17 @@ def _tibber_payload(**overrides) -> dict[str, Any]:
|
|||||||
return base
|
return base
|
||||||
|
|
||||||
|
|
||||||
|
def _district_heating_payload(**overrides) -> dict[str, Any]:
|
||||||
|
base: dict[str, Any] = {
|
||||||
|
"name": "District Heating",
|
||||||
|
"kind": "district_heating",
|
||||||
|
"currency": "EUR",
|
||||||
|
"values": _DISTRICT_HEATING_VALUES,
|
||||||
|
}
|
||||||
|
base.update(overrides)
|
||||||
|
return base
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# Fixtures
|
# Fixtures
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
@@ -155,7 +177,7 @@ def test_profiles_unauthenticated_returns_401(contracts_client):
|
|||||||
assert resp.status_code == 401
|
assert resp.status_code == 401
|
||||||
|
|
||||||
|
|
||||||
def test_profiles_returns_both_kinds(contracts_client):
|
def test_profiles_returns_all_kinds(contracts_client):
|
||||||
client, _ = contracts_client
|
client, _ = contracts_client
|
||||||
_login(client)
|
_login(client)
|
||||||
resp = client.get("/api/energy/profiles")
|
resp = client.get("/api/energy/profiles")
|
||||||
@@ -165,6 +187,7 @@ def test_profiles_returns_both_kinds(contracts_client):
|
|||||||
kinds = {p["kind"] for p in body["profiles"]}
|
kinds = {p["kind"] for p in body["profiles"]}
|
||||||
assert "manual" in kinds
|
assert "manual" in kinds
|
||||||
assert "tibber" in kinds
|
assert "tibber" in kinds
|
||||||
|
assert "district_heating" in kinds
|
||||||
|
|
||||||
|
|
||||||
def test_profiles_contain_structure(contracts_client):
|
def test_profiles_contain_structure(contracts_client):
|
||||||
@@ -176,8 +199,11 @@ def test_profiles_contain_structure(contracts_client):
|
|||||||
for profile in body["profiles"]:
|
for profile in body["profiles"]:
|
||||||
assert "kind" in profile
|
assert "kind" in profile
|
||||||
assert "label" in profile
|
assert "label" in profile
|
||||||
assert "energy" in profile
|
|
||||||
assert "standing" in profile
|
assert "standing" in profile
|
||||||
|
if profile["kind"] == "district_heating":
|
||||||
|
assert "variable" in profile
|
||||||
|
else:
|
||||||
|
assert "energy" in profile
|
||||||
assert "credits" in profile
|
assert "credits" in profile
|
||||||
|
|
||||||
|
|
||||||
@@ -345,6 +371,57 @@ def test_create_tibber_contract_success(contracts_client):
|
|||||||
assert len(body["versions"]) == 1
|
assert len(body["versions"]) == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_create_district_heating_contract_normalises_snapshot_and_scope(contracts_client):
|
||||||
|
client, _ = contracts_client
|
||||||
|
_login(client)
|
||||||
|
response = client.post(
|
||||||
|
"/api/energy/contracts",
|
||||||
|
json=_district_heating_payload(),
|
||||||
|
headers={"X-CSRF-Token": _CSRF},
|
||||||
|
)
|
||||||
|
assert response.status_code == 201
|
||||||
|
body = response.json()
|
||||||
|
assert body["kind"] == "district_heating"
|
||||||
|
assert body["scope"] == "thermal"
|
||||||
|
assert body["versions"][0]["values"] == {
|
||||||
|
"variable": _DISTRICT_HEATING_VALUES["variable"],
|
||||||
|
"standing": {
|
||||||
|
"heating_network": "0", "metering": "0", "delivery_set": "0",
|
||||||
|
"hot_water_network": "0", "other": "0",
|
||||||
|
},
|
||||||
|
}
|
||||||
|
assert client.get("/api/energy/contracts").json()["items"] == []
|
||||||
|
assert client.get("/api/energy/contracts?scope=thermal").json()["total"] == 1
|
||||||
|
mismatch = client.post(
|
||||||
|
"/api/energy/contracts",
|
||||||
|
json=_district_heating_payload(scope="electricity"),
|
||||||
|
headers={"X-CSRF-Token": _CSRF},
|
||||||
|
)
|
||||||
|
assert mismatch.status_code == 422
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
"values",
|
||||||
|
[
|
||||||
|
{"variable": {"heating": "1"}},
|
||||||
|
{"variable": {**_DISTRICT_HEATING_VALUES["variable"], "heating": -1}},
|
||||||
|
{"variable": {**_DISTRICT_HEATING_VALUES["variable"], "heating": 1.5}},
|
||||||
|
{"variable": {**_DISTRICT_HEATING_VALUES["variable"], "extra": "1"}},
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_create_district_heating_rejects_invalid_decimal_values(contracts_client, values):
|
||||||
|
client, engine = contracts_client
|
||||||
|
_login(client)
|
||||||
|
response = client.post(
|
||||||
|
"/api/energy/contracts",
|
||||||
|
json=_district_heating_payload(values=values),
|
||||||
|
headers={"X-CSRF-Token": _CSRF},
|
||||||
|
)
|
||||||
|
assert response.status_code == 422
|
||||||
|
with Session(engine) as session:
|
||||||
|
assert session.execute(select(EnergyContract)).scalars().all() == []
|
||||||
|
|
||||||
|
|
||||||
def test_create_contract_defaults_effective_from(contracts_client):
|
def test_create_contract_defaults_effective_from(contracts_client):
|
||||||
"""When effective_from is omitted, the version is created with a recent timestamp.
|
"""When effective_from is omitted, the version is created with a recent timestamp.
|
||||||
|
|
||||||
@@ -538,6 +615,52 @@ def test_activate_contract_mutual_exclusion(contracts_client):
|
|||||||
assert active_contracts[0].id == id_b
|
assert active_contracts[0].id == id_b
|
||||||
|
|
||||||
|
|
||||||
|
def test_activating_tibber_contract_triggers_refresh_only_after_commit(contracts_client):
|
||||||
|
"""Inactive→active Tibber is the only contract transition that requests refresh."""
|
||||||
|
client, engine = contracts_client
|
||||||
|
_login(client)
|
||||||
|
created = client.post(
|
||||||
|
"/api/energy/contracts",
|
||||||
|
json=_tibber_payload(),
|
||||||
|
headers={"X-CSRF-Token": _CSRF},
|
||||||
|
)
|
||||||
|
contract_id = created.json()["id"]
|
||||||
|
|
||||||
|
with patch("app.api.routes.api.energy_contracts.trigger_tibber_refresh") as trigger:
|
||||||
|
response = client.patch(
|
||||||
|
f"/api/energy/contracts/{contract_id}",
|
||||||
|
json={"active": True},
|
||||||
|
headers={"X-CSRF-Token": _CSRF},
|
||||||
|
)
|
||||||
|
assert response.status_code == 200
|
||||||
|
assert trigger.call_count == 1
|
||||||
|
with Session(engine) as session:
|
||||||
|
assert session.get(EnergyContract, contract_id).active is True
|
||||||
|
|
||||||
|
with patch("app.api.routes.api.energy_contracts.trigger_tibber_refresh") as trigger:
|
||||||
|
response = client.patch(
|
||||||
|
f"/api/energy/contracts/{contract_id}",
|
||||||
|
json={"active": True},
|
||||||
|
headers={"X-CSRF-Token": _CSRF},
|
||||||
|
)
|
||||||
|
assert response.status_code == 200
|
||||||
|
trigger.assert_not_called()
|
||||||
|
|
||||||
|
manual = client.post(
|
||||||
|
"/api/energy/contracts",
|
||||||
|
json=_manual_payload(),
|
||||||
|
headers={"X-CSRF-Token": _CSRF},
|
||||||
|
).json()
|
||||||
|
with patch("app.api.routes.api.energy_contracts.trigger_tibber_refresh") as trigger:
|
||||||
|
response = client.patch(
|
||||||
|
f"/api/energy/contracts/{manual['id']}",
|
||||||
|
json={"active": True},
|
||||||
|
headers={"X-CSRF-Token": _CSRF},
|
||||||
|
)
|
||||||
|
assert response.status_code == 200
|
||||||
|
trigger.assert_not_called()
|
||||||
|
|
||||||
|
|
||||||
def test_deactivate_contract(contracts_client):
|
def test_deactivate_contract(contracts_client):
|
||||||
"""PATCH active=false deactivates the contract without touching others."""
|
"""PATCH active=false deactivates the contract without touching others."""
|
||||||
client, _ = contracts_client
|
client, _ = contracts_client
|
||||||
@@ -567,6 +690,72 @@ def test_deactivate_contract(contracts_client):
|
|||||||
assert resp.json()["active"] is False
|
assert resp.json()["active"] is False
|
||||||
|
|
||||||
|
|
||||||
|
def test_scope_defaults_filtering_and_kind_mismatch(contracts_client):
|
||||||
|
"""Old clients default to electricity; a supplied incompatible scope is rejected."""
|
||||||
|
client, engine = contracts_client
|
||||||
|
_login(client)
|
||||||
|
created = client.post(
|
||||||
|
"/api/energy/contracts",
|
||||||
|
json=_manual_payload(),
|
||||||
|
headers={"X-CSRF-Token": _CSRF},
|
||||||
|
)
|
||||||
|
assert created.status_code == 201
|
||||||
|
assert created.json()["scope"] == "electricity"
|
||||||
|
assert client.get("/api/energy/contracts").json()["total"] == 1
|
||||||
|
assert client.get("/api/energy/contracts?scope=thermal").json()["items"] == []
|
||||||
|
|
||||||
|
mismatch = client.post(
|
||||||
|
"/api/energy/contracts",
|
||||||
|
json=_manual_payload(scope="thermal"),
|
||||||
|
headers={"X-CSRF-Token": _CSRF},
|
||||||
|
)
|
||||||
|
assert mismatch.status_code == 422
|
||||||
|
with Session(engine) as session:
|
||||||
|
assert len(session.execute(select(EnergyContract)).scalars().all()) == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_activation_is_scope_local_and_transaction_rollback_is_safe(contracts_client):
|
||||||
|
"""A thermal activation neither deactivates electricity nor survives rollback."""
|
||||||
|
client, engine = contracts_client
|
||||||
|
_login(client)
|
||||||
|
electricity = client.post(
|
||||||
|
"/api/energy/contracts",
|
||||||
|
json=_manual_payload(name="Electricity"),
|
||||||
|
headers={"X-CSRF-Token": _CSRF},
|
||||||
|
).json()
|
||||||
|
client.patch(
|
||||||
|
f"/api/energy/contracts/{electricity['id']}",
|
||||||
|
json={"active": True},
|
||||||
|
headers={"X-CSRF-Token": _CSRF},
|
||||||
|
)
|
||||||
|
now = datetime.now(UTC)
|
||||||
|
with Session(engine) as session:
|
||||||
|
thermal = EnergyContract(
|
||||||
|
name="Future thermal", kind="district_heating", scope="thermal", active=False,
|
||||||
|
currency="EUR", created_at=now, updated_at=now,
|
||||||
|
)
|
||||||
|
session.add(thermal)
|
||||||
|
session.commit()
|
||||||
|
thermal_id = thermal.id
|
||||||
|
|
||||||
|
with Session(engine) as session:
|
||||||
|
thermal = session.get(EnergyContract, thermal_id)
|
||||||
|
assert thermal is not None
|
||||||
|
activate_contract(session, thermal)
|
||||||
|
session.rollback() # Simulate a later write failure in this transaction.
|
||||||
|
|
||||||
|
with Session(engine) as session:
|
||||||
|
rows = {row.scope: row for row in session.execute(select(EnergyContract)).scalars()}
|
||||||
|
assert rows["electricity"].active is True
|
||||||
|
assert rows["thermal"].active is False
|
||||||
|
activate_contract(session, rows["thermal"])
|
||||||
|
session.commit()
|
||||||
|
|
||||||
|
with Session(engine) as session:
|
||||||
|
active = session.execute(select(EnergyContract).where(EnergyContract.active.is_(True))).scalars().all()
|
||||||
|
assert {row.scope for row in active} == {"electricity", "thermal"}
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# POST /api/energy/contracts/{id}/versions
|
# POST /api/energy/contracts/{id}/versions
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
@@ -748,3 +937,36 @@ def test_add_version_closes_previous_and_appends(contracts_client):
|
|||||||
.all()
|
.all()
|
||||||
)
|
)
|
||||||
assert len(all_versions) == 2
|
assert len(all_versions) == 2
|
||||||
|
|
||||||
|
|
||||||
|
def test_district_heating_version_timeline_keeps_normalised_snapshots(contracts_client):
|
||||||
|
client, _ = contracts_client
|
||||||
|
_login(client)
|
||||||
|
t0 = datetime(2026, 1, 1, tzinfo=UTC)
|
||||||
|
created = client.post(
|
||||||
|
"/api/energy/contracts",
|
||||||
|
json=_district_heating_payload(effective_from=t0.isoformat()),
|
||||||
|
headers={"X-CSRF-Token": _CSRF},
|
||||||
|
)
|
||||||
|
assert created.status_code == 201
|
||||||
|
contract_id = created.json()["id"]
|
||||||
|
t1 = datetime(2026, 6, 1, tzinfo=UTC)
|
||||||
|
updated_values = {
|
||||||
|
"variable": {**_DISTRICT_HEATING_VALUES["variable"], "heating": "13.500"},
|
||||||
|
"standing": {"metering": "10.00"},
|
||||||
|
}
|
||||||
|
response = client.post(
|
||||||
|
f"/api/energy/contracts/{contract_id}/versions",
|
||||||
|
json={"effective_from": t1.isoformat(), "values": updated_values},
|
||||||
|
headers={"X-CSRF-Token": _CSRF},
|
||||||
|
)
|
||||||
|
assert response.status_code == 201
|
||||||
|
old, new = response.json()["versions"]
|
||||||
|
assert old["effective_to"] is not None
|
||||||
|
assert old["values"]["variable"]["heating"] == "12.34"
|
||||||
|
assert new["effective_to"] is None
|
||||||
|
assert new["values"]["variable"]["heating"] == "13.500"
|
||||||
|
assert new["values"]["standing"] == {
|
||||||
|
"heating_network": "0", "metering": "10.00", "delivery_set": "0",
|
||||||
|
"hot_water_network": "0", "other": "0",
|
||||||
|
}
|
||||||
|
|||||||
+673
-3
@@ -31,15 +31,17 @@ Retroactive recompute integration
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
from datetime import UTC, datetime
|
from datetime import UTC, datetime, timedelta, timezone
|
||||||
|
from decimal import Decimal
|
||||||
from unittest.mock import patch
|
from unittest.mock import patch
|
||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
from fastapi.testclient import TestClient
|
from fastapi.testclient import TestClient
|
||||||
from sqlalchemy import create_engine, select
|
from sqlalchemy import create_engine, event, select
|
||||||
from sqlalchemy.orm import Session
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
from app.models.energy import Meter
|
from app.models.energy import EnergyCostPeriod, Meter, MeterCostPeriod
|
||||||
|
from app.models.meter_source import MeterSource, MeterSourceBinding, MeterSourceChannel
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# Shared helpers
|
# Shared helpers
|
||||||
@@ -67,6 +69,51 @@ def _declare_payload(**overrides) -> dict:
|
|||||||
return base
|
return base
|
||||||
|
|
||||||
|
|
||||||
|
def _add_bound_channel(
|
||||||
|
engine,
|
||||||
|
*,
|
||||||
|
meter_id: int,
|
||||||
|
started_at: datetime,
|
||||||
|
ended_at: datetime | None = None,
|
||||||
|
unit: str = "kWh",
|
||||||
|
) -> str:
|
||||||
|
"""Persist one test-only channel binding and return its public UUID."""
|
||||||
|
with Session(engine) as session:
|
||||||
|
source = MeterSource(
|
||||||
|
name="Test DSMR",
|
||||||
|
kind="dsmr_mqtt",
|
||||||
|
enabled=True,
|
||||||
|
config={},
|
||||||
|
status="online",
|
||||||
|
created_at=started_at,
|
||||||
|
updated_at=started_at,
|
||||||
|
)
|
||||||
|
session.add(source)
|
||||||
|
session.flush()
|
||||||
|
channel = MeterSourceChannel(
|
||||||
|
source_id=source.id,
|
||||||
|
channel_key=f"test-total-{meter_id}",
|
||||||
|
label="Test total",
|
||||||
|
unit=unit,
|
||||||
|
created_at=started_at,
|
||||||
|
updated_at=started_at,
|
||||||
|
)
|
||||||
|
session.add(channel)
|
||||||
|
session.flush()
|
||||||
|
session.add(
|
||||||
|
MeterSourceBinding(
|
||||||
|
meter_id=meter_id,
|
||||||
|
channel_id=channel.id,
|
||||||
|
started_at=started_at,
|
||||||
|
ended_at=ended_at,
|
||||||
|
created_at=started_at,
|
||||||
|
updated_at=started_at,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
session.commit()
|
||||||
|
return channel.uuid
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# Fixtures
|
# Fixtures
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
@@ -252,6 +299,207 @@ def test_declare_meter_swap_closes_previous(meters_client):
|
|||||||
assert ended_naive == t1
|
assert ended_naive == t1
|
||||||
|
|
||||||
|
|
||||||
|
def test_declare_meter_swap_hands_off_previous_meter_channel_atomically(meters_client):
|
||||||
|
client, engine = meters_client
|
||||||
|
_login(client)
|
||||||
|
t0 = datetime(2024, 6, 1, tzinfo=UTC)
|
||||||
|
boundary = datetime(2025, 3, 15, 12, tzinfo=UTC)
|
||||||
|
|
||||||
|
with patch("app.api.routes.api.meters.recompute_range", return_value=0):
|
||||||
|
old_response = client.post(
|
||||||
|
"/api/energy/meters",
|
||||||
|
json=_declare_payload(label="Old meter", started_at=t0.isoformat()),
|
||||||
|
headers={"X-CSRF-Token": _CSRF},
|
||||||
|
)
|
||||||
|
assert old_response.status_code == 201
|
||||||
|
old_id = old_response.json()["id"]
|
||||||
|
channel_uuid = _add_bound_channel(engine, meter_id=old_id, started_at=t0)
|
||||||
|
response = client.post(
|
||||||
|
"/api/energy/meters",
|
||||||
|
json=_declare_payload(
|
||||||
|
label="New meter",
|
||||||
|
started_at=boundary.isoformat(),
|
||||||
|
reason="meter_swap",
|
||||||
|
source_channel_uuid=channel_uuid,
|
||||||
|
),
|
||||||
|
headers={"X-CSRF-Token": _CSRF},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert response.status_code == 201
|
||||||
|
new_id = response.json()["id"]
|
||||||
|
with Session(engine) as session:
|
||||||
|
bindings = session.execute(
|
||||||
|
select(MeterSourceBinding).order_by(MeterSourceBinding.id)
|
||||||
|
).scalars().all()
|
||||||
|
old_binding_ended_at = bindings[0].ended_at
|
||||||
|
if old_binding_ended_at is not None and old_binding_ended_at.tzinfo is None:
|
||||||
|
old_binding_ended_at = old_binding_ended_at.replace(tzinfo=UTC)
|
||||||
|
new_binding_started_at = bindings[1].started_at
|
||||||
|
if new_binding_started_at.tzinfo is None:
|
||||||
|
new_binding_started_at = new_binding_started_at.replace(tzinfo=UTC)
|
||||||
|
assert [(bindings[0].meter_id, old_binding_ended_at), (bindings[1].meter_id, bindings[1].ended_at)] == [
|
||||||
|
(old_id, boundary),
|
||||||
|
(new_id, None),
|
||||||
|
]
|
||||||
|
assert new_binding_started_at == boundary
|
||||||
|
|
||||||
|
|
||||||
|
def test_declare_meter_swap_rejects_other_meter_channel_and_rolls_back(meters_client):
|
||||||
|
client, engine = meters_client
|
||||||
|
_login(client)
|
||||||
|
t0 = datetime(2024, 6, 1, tzinfo=UTC)
|
||||||
|
boundary = datetime(2025, 3, 15, 12, tzinfo=UTC)
|
||||||
|
|
||||||
|
with patch("app.api.routes.api.meters.recompute_range", return_value=0):
|
||||||
|
old_response = client.post(
|
||||||
|
"/api/energy/meters",
|
||||||
|
json=_declare_payload(label="Old meter", started_at=t0.isoformat()),
|
||||||
|
headers={"X-CSRF-Token": _CSRF},
|
||||||
|
)
|
||||||
|
old_id = old_response.json()["id"]
|
||||||
|
other = Meter(
|
||||||
|
label="Other meter",
|
||||||
|
commodity="electricity",
|
||||||
|
started_at=t0,
|
||||||
|
ended_at=boundary + timedelta(days=1),
|
||||||
|
reason="initial",
|
||||||
|
created_at=t0,
|
||||||
|
)
|
||||||
|
with Session(engine) as session:
|
||||||
|
session.add(other)
|
||||||
|
session.commit()
|
||||||
|
other_id = other.id
|
||||||
|
channel_uuid = _add_bound_channel(engine, meter_id=other_id, started_at=t0)
|
||||||
|
response = client.post(
|
||||||
|
"/api/energy/meters",
|
||||||
|
json=_declare_payload(
|
||||||
|
label="Rejected meter",
|
||||||
|
started_at=boundary.isoformat(),
|
||||||
|
reason="meter_swap",
|
||||||
|
source_channel_uuid=channel_uuid,
|
||||||
|
),
|
||||||
|
headers={"X-CSRF-Token": _CSRF},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert response.status_code == 422
|
||||||
|
with Session(engine) as session:
|
||||||
|
assert session.execute(select(Meter).where(Meter.label == "Rejected meter")).scalar_one_or_none() is None
|
||||||
|
assert session.get(Meter, old_id).ended_at is None
|
||||||
|
binding = session.execute(select(MeterSourceBinding)).scalar_one()
|
||||||
|
assert binding.ended_at is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_declare_meter_non_swap_cannot_take_previous_meter_channel(meters_client):
|
||||||
|
client, engine = meters_client
|
||||||
|
_login(client)
|
||||||
|
t0 = datetime(2024, 6, 1, tzinfo=UTC)
|
||||||
|
boundary = datetime(2025, 3, 15, 12, tzinfo=UTC)
|
||||||
|
with patch("app.api.routes.api.meters.recompute_range", return_value=0):
|
||||||
|
old_response = client.post(
|
||||||
|
"/api/energy/meters",
|
||||||
|
json=_declare_payload(label="Old meter", started_at=t0.isoformat()),
|
||||||
|
headers={"X-CSRF-Token": _CSRF},
|
||||||
|
)
|
||||||
|
old_id = old_response.json()["id"]
|
||||||
|
channel_uuid = _add_bound_channel(engine, meter_id=old_id, started_at=t0)
|
||||||
|
response = client.post(
|
||||||
|
"/api/energy/meters",
|
||||||
|
json=_declare_payload(
|
||||||
|
label="Moved meter",
|
||||||
|
started_at=boundary.isoformat(),
|
||||||
|
reason="home_move",
|
||||||
|
source_channel_uuid=channel_uuid,
|
||||||
|
),
|
||||||
|
headers={"X-CSRF-Token": _CSRF},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert response.status_code == 422
|
||||||
|
with Session(engine) as session:
|
||||||
|
assert session.get(Meter, old_id).ended_at is None
|
||||||
|
assert session.execute(select(MeterSourceBinding)).scalar_one().ended_at is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_declare_meter_recompute_failure_rolls_back_handoff(meters_client):
|
||||||
|
client, engine = meters_client
|
||||||
|
_login(client)
|
||||||
|
t0 = datetime(2024, 6, 1, tzinfo=UTC)
|
||||||
|
boundary = datetime(2025, 3, 15, 12, tzinfo=UTC)
|
||||||
|
with patch("app.api.routes.api.meters.recompute_range", return_value=0):
|
||||||
|
old_response = client.post(
|
||||||
|
"/api/energy/meters",
|
||||||
|
json=_declare_payload(label="Old meter", started_at=t0.isoformat()),
|
||||||
|
headers={"X-CSRF-Token": _CSRF},
|
||||||
|
)
|
||||||
|
old_id = old_response.json()["id"]
|
||||||
|
channel_uuid = _add_bound_channel(engine, meter_id=old_id, started_at=t0)
|
||||||
|
|
||||||
|
with patch("app.api.routes.api.meters.recompute_range", side_effect=RuntimeError("recompute failed")):
|
||||||
|
with pytest.raises(RuntimeError, match="recompute failed"):
|
||||||
|
client.post(
|
||||||
|
"/api/energy/meters",
|
||||||
|
json=_declare_payload(
|
||||||
|
label="Failed meter",
|
||||||
|
started_at=boundary.isoformat(),
|
||||||
|
reason="meter_swap",
|
||||||
|
source_channel_uuid=channel_uuid,
|
||||||
|
),
|
||||||
|
headers={"X-CSRF-Token": _CSRF},
|
||||||
|
)
|
||||||
|
|
||||||
|
with Session(engine) as session:
|
||||||
|
assert session.execute(select(Meter).where(Meter.label == "Failed meter")).scalar_one_or_none() is None
|
||||||
|
assert session.get(Meter, old_id).ended_at is None
|
||||||
|
assert session.execute(select(MeterSourceBinding)).scalar_one().ended_at is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_declare_meter_final_commit_failure_rolls_back_handoff_and_recompute(meters_client):
|
||||||
|
"""A real recompute remains uncommitted until the route's final commit succeeds."""
|
||||||
|
client, engine = meters_client
|
||||||
|
_login(client)
|
||||||
|
boundary = datetime.now(UTC) - timedelta(minutes=45)
|
||||||
|
boundary = boundary.replace(minute=boundary.minute - boundary.minute % 15, second=0, microsecond=0)
|
||||||
|
old_start = boundary - timedelta(days=1)
|
||||||
|
|
||||||
|
with Session(engine) as session:
|
||||||
|
old_meter = Meter(
|
||||||
|
label="Old meter",
|
||||||
|
commodity="electricity",
|
||||||
|
started_at=old_start,
|
||||||
|
reason="initial",
|
||||||
|
created_at=old_start,
|
||||||
|
)
|
||||||
|
session.add(old_meter)
|
||||||
|
session.commit()
|
||||||
|
old_id = old_meter.id
|
||||||
|
channel_uuid = _add_bound_channel(engine, meter_id=old_id, started_at=old_start)
|
||||||
|
|
||||||
|
def fail_final_commit(_session: Session) -> None:
|
||||||
|
raise RuntimeError("final commit failed")
|
||||||
|
|
||||||
|
event.listen(Session, "before_commit", fail_final_commit)
|
||||||
|
try:
|
||||||
|
with pytest.raises(RuntimeError, match="final commit failed"):
|
||||||
|
client.post(
|
||||||
|
"/api/energy/meters",
|
||||||
|
json=_declare_payload(
|
||||||
|
label="New meter",
|
||||||
|
started_at=boundary.isoformat(),
|
||||||
|
reason="meter_swap",
|
||||||
|
source_channel_uuid=channel_uuid,
|
||||||
|
),
|
||||||
|
headers={"X-CSRF-Token": _CSRF},
|
||||||
|
)
|
||||||
|
finally:
|
||||||
|
event.remove(Session, "before_commit", fail_final_commit)
|
||||||
|
|
||||||
|
with Session(engine) as session:
|
||||||
|
assert session.execute(select(Meter).where(Meter.label == "New meter")).scalar_one_or_none() is None
|
||||||
|
assert session.get(Meter, old_id).ended_at is None
|
||||||
|
binding = session.execute(select(MeterSourceBinding)).scalar_one()
|
||||||
|
assert binding.ended_at is None
|
||||||
|
assert session.execute(select(EnergyCostPeriod)).scalars().all() == []
|
||||||
|
|
||||||
|
|
||||||
def test_declare_meter_overlap_returns_422(meters_client):
|
def test_declare_meter_overlap_returns_422(meters_client):
|
||||||
"""Declaring a meter with started_at before active meter's started_at → 422."""
|
"""Declaring a meter with started_at before active meter's started_at → 422."""
|
||||||
client, _ = meters_client
|
client, _ = meters_client
|
||||||
@@ -355,6 +603,8 @@ def test_declare_meter_retroactive_triggers_recompute(meters_client):
|
|||||||
assert resp.status_code == 201
|
assert resp.status_code == 201
|
||||||
# recompute_range should have been called with start == t_past
|
# recompute_range should have been called with start == t_past
|
||||||
assert mock_recompute.called
|
assert mock_recompute.called
|
||||||
|
assert mock_recompute.call_args.kwargs["commit"] is False
|
||||||
|
assert mock_recompute.call_args.kwargs["strict"] is True
|
||||||
call_args = mock_recompute.call_args
|
call_args = mock_recompute.call_args
|
||||||
recompute_start = call_args[0][1] # positional arg index 1 (session is 0)
|
recompute_start = call_args[0][1] # positional arg index 1 (session is 0)
|
||||||
# Normalise for comparison
|
# Normalise for comparison
|
||||||
@@ -494,6 +744,8 @@ def test_patch_meter_started_at_retroactive_triggers_recompute(meters_client):
|
|||||||
assert resp.status_code == 200
|
assert resp.status_code == 200
|
||||||
# recompute should be triggered
|
# recompute should be triggered
|
||||||
assert mock_recompute.called
|
assert mock_recompute.called
|
||||||
|
assert mock_recompute.call_args.kwargs["commit"] is False
|
||||||
|
assert mock_recompute.call_args.kwargs["strict"] is True
|
||||||
call_args = mock_recompute.call_args
|
call_args = mock_recompute.call_args
|
||||||
recompute_start = call_args[0][1]
|
recompute_start = call_args[0][1]
|
||||||
if recompute_start.tzinfo is None:
|
if recompute_start.tzinfo is None:
|
||||||
@@ -535,6 +787,59 @@ def test_patch_meter_started_at_interval_violation_returns_422(meters_client):
|
|||||||
assert resp.status_code == 422
|
assert resp.status_code == 422
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("shift", ["later", "earlier"])
|
||||||
|
def test_patch_meter_started_at_rejects_boundary_shift_that_strands_binding(
|
||||||
|
meters_client, mock_publish_discovery, shift
|
||||||
|
):
|
||||||
|
"""Rejected boundary shifts leave adjacent meters/bindings untouched and emit no side effects."""
|
||||||
|
client, engine = meters_client
|
||||||
|
_login(client)
|
||||||
|
t0 = datetime(2024, 1, 1, tzinfo=UTC)
|
||||||
|
boundary = datetime(2025, 1, 1, tzinfo=UTC)
|
||||||
|
with patch("app.api.routes.api.meters.recompute_range", return_value=0):
|
||||||
|
meter_a = client.post(
|
||||||
|
"/api/energy/meters", json=_declare_payload(label="A", started_at=t0.isoformat()),
|
||||||
|
headers={"X-CSRF-Token": _CSRF},
|
||||||
|
).json()
|
||||||
|
meter_b = client.post(
|
||||||
|
"/api/energy/meters", json=_declare_payload(
|
||||||
|
label="B", started_at=boundary.isoformat(), reason="meter_swap"
|
||||||
|
), headers={"X-CSRF-Token": _CSRF},
|
||||||
|
).json()
|
||||||
|
if shift == "later":
|
||||||
|
_add_bound_channel(engine, meter_id=meter_b["id"], started_at=boundary)
|
||||||
|
proposed = boundary + timedelta(days=1)
|
||||||
|
else:
|
||||||
|
_add_bound_channel(engine, meter_id=meter_a["id"], started_at=t0)
|
||||||
|
with Session(engine) as session:
|
||||||
|
binding = session.scalar(select(MeterSourceBinding))
|
||||||
|
assert binding is not None
|
||||||
|
binding.ended_at = boundary
|
||||||
|
session.commit()
|
||||||
|
proposed = boundary - timedelta(days=1)
|
||||||
|
|
||||||
|
mock_publish_discovery.reset_mock()
|
||||||
|
with patch("app.api.routes.api.meters.recompute_range", return_value=0) as recompute:
|
||||||
|
response = client.patch(
|
||||||
|
f"/api/energy/meters/{meter_b['id']}", json={"started_at": proposed.isoformat()},
|
||||||
|
headers={"X-CSRF-Token": _CSRF},
|
||||||
|
)
|
||||||
|
assert response.status_code == 422
|
||||||
|
recompute.assert_not_called()
|
||||||
|
mock_publish_discovery.assert_not_called()
|
||||||
|
|
||||||
|
with Session(engine) as observer:
|
||||||
|
assert observer.get(Meter, meter_a["id"]).ended_at.replace(tzinfo=UTC) == boundary
|
||||||
|
assert observer.get(Meter, meter_b["id"]).started_at.replace(tzinfo=UTC) == boundary
|
||||||
|
binding = observer.scalar(select(MeterSourceBinding))
|
||||||
|
assert binding is not None
|
||||||
|
if shift == "later":
|
||||||
|
assert binding.meter_id == meter_b["id"] and binding.ended_at is None
|
||||||
|
else:
|
||||||
|
assert binding.meter_id == meter_a["id"]
|
||||||
|
assert binding.ended_at.replace(tzinfo=UTC) == boundary
|
||||||
|
|
||||||
|
|
||||||
def test_patch_meter_no_recompute_when_started_at_not_changed(meters_client):
|
def test_patch_meter_no_recompute_when_started_at_not_changed(meters_client):
|
||||||
"""PATCH that only changes label does NOT trigger recompute."""
|
"""PATCH that only changes label does NOT trigger recompute."""
|
||||||
client, _ = meters_client
|
client, _ = meters_client
|
||||||
@@ -560,6 +865,201 @@ def test_patch_meter_no_recompute_when_started_at_not_changed(meters_client):
|
|||||||
assert not mock_recompute.called
|
assert not mock_recompute.called
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("representation", ["aware_utc", "naive_local"])
|
||||||
|
def test_patch_meter_rejects_future_started_at_before_any_side_effect(
|
||||||
|
meters_client, mock_publish_discovery, monkeypatch, representation
|
||||||
|
):
|
||||||
|
"""Future aware and local-naive starts leave all persisted state untouched."""
|
||||||
|
client, engine = meters_client
|
||||||
|
_login(client)
|
||||||
|
started = datetime.now(UTC) - timedelta(hours=2)
|
||||||
|
with patch("app.api.routes.api.meters.recompute_range", return_value=0):
|
||||||
|
declared = client.post(
|
||||||
|
"/api/energy/meters",
|
||||||
|
json=_declare_payload(
|
||||||
|
label="Original", note="Original note", started_at=started.isoformat()
|
||||||
|
),
|
||||||
|
headers={"X-CSRF-Token": _CSRF},
|
||||||
|
)
|
||||||
|
meter_id = declared.json()["id"]
|
||||||
|
mock_publish_discovery.reset_mock()
|
||||||
|
future = datetime.now(UTC) + timedelta(hours=2)
|
||||||
|
if representation == "aware_utc":
|
||||||
|
proposed = future.isoformat()
|
||||||
|
else:
|
||||||
|
monkeypatch.setattr(
|
||||||
|
"app.services.timezone.local_tz", lambda: timezone(timedelta(hours=2))
|
||||||
|
)
|
||||||
|
proposed = (future + timedelta(hours=2)).replace(tzinfo=None).isoformat()
|
||||||
|
|
||||||
|
with patch("app.api.routes.api.meters.recompute_range", return_value=0) as recompute:
|
||||||
|
response = client.patch(
|
||||||
|
f"/api/energy/meters/{meter_id}",
|
||||||
|
json={"label": "Changed", "note": "Changed note", "started_at": proposed},
|
||||||
|
headers={"X-CSRF-Token": _CSRF},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert response.status_code == 422
|
||||||
|
recompute.assert_not_called()
|
||||||
|
mock_publish_discovery.assert_not_called()
|
||||||
|
with Session(engine) as observer:
|
||||||
|
meter = observer.get(Meter, meter_id)
|
||||||
|
assert meter is not None
|
||||||
|
assert meter.label == "Original"
|
||||||
|
assert meter.note == "Original note"
|
||||||
|
assert meter.started_at.replace(tzinfo=UTC) == started
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("commodity", ["heating", "hot_water"])
|
||||||
|
def test_patch_thermal_started_at_uses_thermal_recompute(meters_client, monkeypatch, commodity):
|
||||||
|
"""A successful thermal correction never routes through electricity recompute."""
|
||||||
|
from app.services import meter_cost
|
||||||
|
|
||||||
|
client, _ = meters_client
|
||||||
|
_login(client)
|
||||||
|
started = datetime.now(UTC) - timedelta(hours=3)
|
||||||
|
calls = []
|
||||||
|
monkeypatch.setattr(
|
||||||
|
meter_cost,
|
||||||
|
"recompute_range",
|
||||||
|
lambda db, start, end, *, commit: calls.append((start, end, commit)) or 0,
|
||||||
|
)
|
||||||
|
with patch("app.api.routes.api.meters.recompute_range", return_value=0) as electricity:
|
||||||
|
declared = client.post(
|
||||||
|
"/api/energy/meters",
|
||||||
|
json=_declare_payload(
|
||||||
|
commodity=commodity,
|
||||||
|
label=f"{commodity} meter",
|
||||||
|
started_at=started.isoformat(),
|
||||||
|
),
|
||||||
|
headers={"X-CSRF-Token": _CSRF},
|
||||||
|
)
|
||||||
|
assert declared.status_code == 201
|
||||||
|
calls.clear()
|
||||||
|
electricity.reset_mock()
|
||||||
|
response = client.patch(
|
||||||
|
f"/api/energy/meters/{declared.json()['id']}",
|
||||||
|
json={"started_at": (started + timedelta(minutes=30)).isoformat()},
|
||||||
|
headers={"X-CSRF-Token": _CSRF},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert response.status_code == 200
|
||||||
|
assert len(calls) == 1
|
||||||
|
assert calls[0][2] is False
|
||||||
|
electricity.assert_not_called()
|
||||||
|
|
||||||
|
|
||||||
|
def _create_adjacent_thermal_patch_state(engine, commodity: str):
|
||||||
|
"""Create an editable thermal boundary with two untouched bindings."""
|
||||||
|
old_start = datetime.now(UTC) - timedelta(hours=5)
|
||||||
|
boundary = old_start + timedelta(hours=1)
|
||||||
|
shifted = boundary + timedelta(minutes=30)
|
||||||
|
current_binding_start = shifted + timedelta(minutes=30)
|
||||||
|
with Session(engine) as session:
|
||||||
|
previous = Meter(
|
||||||
|
label="Previous thermal meter",
|
||||||
|
commodity=commodity,
|
||||||
|
started_at=old_start,
|
||||||
|
ended_at=boundary,
|
||||||
|
reason="initial",
|
||||||
|
created_at=old_start,
|
||||||
|
)
|
||||||
|
current = Meter(
|
||||||
|
label="Current thermal meter",
|
||||||
|
commodity=commodity,
|
||||||
|
started_at=boundary,
|
||||||
|
reason="meter_swap",
|
||||||
|
created_at=boundary,
|
||||||
|
)
|
||||||
|
session.add_all([previous, current])
|
||||||
|
session.commit()
|
||||||
|
previous_id, current_id = previous.id, current.id
|
||||||
|
unit = {"heating": "GJ", "hot_water": "m³"}[commodity]
|
||||||
|
_add_bound_channel(
|
||||||
|
engine, meter_id=previous_id, started_at=old_start, ended_at=boundary, unit=unit
|
||||||
|
)
|
||||||
|
_add_bound_channel(engine, meter_id=current_id, started_at=current_binding_start, unit=unit)
|
||||||
|
return previous_id, current_id, boundary, shifted, current_binding_start
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("commodity", ["heating", "hot_water"])
|
||||||
|
@pytest.mark.parametrize("failure", ["recompute", "flush", "commit"])
|
||||||
|
def test_patch_thermal_failure_rolls_back_lifecycle_and_cost_state(
|
||||||
|
meters_client, mock_publish_discovery, monkeypatch, commodity, failure
|
||||||
|
):
|
||||||
|
"""Thermal PATCH failures roll back meters, bindings, cost writes, and HA."""
|
||||||
|
from app.services import meter_cost
|
||||||
|
|
||||||
|
client, engine = meters_client
|
||||||
|
_login(client)
|
||||||
|
previous_id, current_id, boundary, shifted, binding_start = _create_adjacent_thermal_patch_state(
|
||||||
|
engine, commodity
|
||||||
|
)
|
||||||
|
|
||||||
|
def recompute_with_uncommitted_cost(db, start, end, *, commit):
|
||||||
|
assert commit is False
|
||||||
|
db.add(
|
||||||
|
MeterCostPeriod(
|
||||||
|
commodity=commodity,
|
||||||
|
period_start=shifted,
|
||||||
|
period_end=shifted + timedelta(minutes=15),
|
||||||
|
quantity=Decimal("0"),
|
||||||
|
cost=Decimal("0"),
|
||||||
|
currency="EUR",
|
||||||
|
cost_breakdown={},
|
||||||
|
pricing_snapshot={},
|
||||||
|
quality="invalid",
|
||||||
|
degraded=True,
|
||||||
|
degraded_reason="test rollback",
|
||||||
|
created_at=shifted,
|
||||||
|
updated_at=shifted,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
if failure == "recompute":
|
||||||
|
raise RuntimeError("thermal recompute failed")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
monkeypatch.setattr(meter_cost, "recompute_range", recompute_with_uncommitted_cost)
|
||||||
|
if failure == "flush":
|
||||||
|
monkeypatch.setattr(
|
||||||
|
Session,
|
||||||
|
"flush",
|
||||||
|
lambda *_args, **_kwargs: (_ for _ in ()).throw(RuntimeError("thermal flush failed")),
|
||||||
|
)
|
||||||
|
if failure == "commit":
|
||||||
|
def fail_commit(_session: Session) -> None:
|
||||||
|
raise RuntimeError("thermal commit failed")
|
||||||
|
|
||||||
|
event.listen(Session, "before_commit", fail_commit)
|
||||||
|
|
||||||
|
try:
|
||||||
|
expected = f"thermal {failure} failed"
|
||||||
|
with pytest.raises(RuntimeError, match=expected):
|
||||||
|
client.patch(
|
||||||
|
f"/api/energy/meters/{current_id}",
|
||||||
|
json={"started_at": shifted.isoformat()},
|
||||||
|
headers={"X-CSRF-Token": _CSRF},
|
||||||
|
)
|
||||||
|
finally:
|
||||||
|
if failure == "commit":
|
||||||
|
event.remove(Session, "before_commit", fail_commit)
|
||||||
|
if failure == "flush":
|
||||||
|
monkeypatch.undo()
|
||||||
|
|
||||||
|
mock_publish_discovery.assert_not_called()
|
||||||
|
with Session(engine) as observer:
|
||||||
|
previous = observer.get(Meter, previous_id)
|
||||||
|
current = observer.get(Meter, current_id)
|
||||||
|
assert previous is not None and previous.ended_at.replace(tzinfo=UTC) == boundary
|
||||||
|
assert current is not None and current.started_at.replace(tzinfo=UTC) == boundary
|
||||||
|
bindings = observer.execute(
|
||||||
|
select(MeterSourceBinding).order_by(MeterSourceBinding.meter_id)
|
||||||
|
).scalars().all()
|
||||||
|
assert bindings[0].ended_at.replace(tzinfo=UTC) == boundary
|
||||||
|
assert bindings[1].started_at.replace(tzinfo=UTC) == binding_start
|
||||||
|
assert observer.execute(select(MeterCostPeriod)).scalars().all() == []
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# Timeline continuity (recompute mocked to avoid slow computation over empty quarters)
|
# Timeline continuity (recompute mocked to avoid slow computation over empty quarters)
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
@@ -753,3 +1253,173 @@ def test_declare_meter_succeeds_when_publish_discovery_raises(meters_client):
|
|||||||
# The meter must be created successfully despite the discovery failure.
|
# The meter must be created successfully despite the discovery failure.
|
||||||
assert resp.status_code == 201
|
assert resp.status_code == 201
|
||||||
assert resp.json()["label"] == "Best Effort Meter"
|
assert resp.json()["label"] == "Best Effort Meter"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("commodity", ["heating", "hot_water"])
|
||||||
|
def test_thermal_declare_and_close_use_meter_cost_recompute(meters_client, monkeypatch, commodity):
|
||||||
|
"""Thermal lifecycle routes use the meter-cost helper's actual signature."""
|
||||||
|
from app.services import meter_cost
|
||||||
|
|
||||||
|
client, _ = meters_client
|
||||||
|
_login(client)
|
||||||
|
calls = []
|
||||||
|
monkeypatch.setattr(meter_cost, "recompute_range", lambda db, start, end, *, commit: calls.append((start, end, commit)) or 0)
|
||||||
|
started = datetime.now(UTC) - timedelta(hours=2)
|
||||||
|
declared = client.post("/api/energy/meters", headers={"X-CSRF-Token": _CSRF}, json={
|
||||||
|
"label": f"{commodity} meter", "started_at": started.isoformat(), "reason": "initial", "commodity": commodity,
|
||||||
|
})
|
||||||
|
assert declared.status_code == 201
|
||||||
|
closed = client.post(f"/api/energy/meters/{declared.json()['id']}/close", headers={"X-CSRF-Token": _CSRF}, json={
|
||||||
|
"ended_at": (started + timedelta(hours=1)).isoformat(),
|
||||||
|
})
|
||||||
|
assert closed.status_code == 200
|
||||||
|
assert len(calls) == 2 and all(call[2] is False for call in calls)
|
||||||
|
|
||||||
|
|
||||||
|
def test_close_meter_closes_open_bindings_and_enforces_auth_csrf(meters_client):
|
||||||
|
client, engine = meters_client
|
||||||
|
started = datetime.now(UTC) - timedelta(hours=2)
|
||||||
|
_login(client)
|
||||||
|
with patch("app.api.routes.api.meters.recompute_range", return_value=0):
|
||||||
|
declared = client.post("/api/energy/meters", headers={"X-CSRF-Token": _CSRF}, json=_declare_payload(
|
||||||
|
label="closable", started_at=started.isoformat(), reason="initial",
|
||||||
|
))
|
||||||
|
assert declared.status_code == 201
|
||||||
|
_add_bound_channel(engine, meter_id=declared.json()["id"], started_at=started)
|
||||||
|
boundary = started + timedelta(hours=1)
|
||||||
|
assert client.post(f"/api/energy/meters/{declared.json()['id']}/close", json={"ended_at": boundary.isoformat()}).status_code == 403
|
||||||
|
with patch("app.api.routes.api.meters.recompute_range", return_value=0):
|
||||||
|
closed = client.post(f"/api/energy/meters/{declared.json()['id']}/close", headers={"X-CSRF-Token": _CSRF}, json={
|
||||||
|
"ended_at": boundary.isoformat(),
|
||||||
|
})
|
||||||
|
assert closed.status_code == 200
|
||||||
|
with Session(engine) as session:
|
||||||
|
binding = session.scalar(select(MeterSourceBinding))
|
||||||
|
assert binding is not None and binding.ended_at.replace(tzinfo=UTC) == boundary
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("commodity,unit", [("electricity", "kWh"), ("heating", "GJ"), ("hot_water", "m³")])
|
||||||
|
@pytest.mark.parametrize("operation", ["close", "declare"])
|
||||||
|
def test_lifecycle_rejects_retained_closed_binding_beyond_proposed_end(
|
||||||
|
meters_client, mock_publish_discovery, commodity, unit, operation,
|
||||||
|
):
|
||||||
|
"""Close and declare fail closed before recompute or HA for every commodity."""
|
||||||
|
client, engine = meters_client
|
||||||
|
_login(client)
|
||||||
|
start = datetime.now(UTC) - timedelta(hours=3)
|
||||||
|
boundary = start + timedelta(hours=1)
|
||||||
|
retained_end = start + timedelta(hours=2)
|
||||||
|
declared = client.post(
|
||||||
|
"/api/energy/meters", headers={"X-CSRF-Token": _CSRF},
|
||||||
|
json=_declare_payload(label="retained-history", started_at=start.isoformat(), reason="initial", commodity=commodity),
|
||||||
|
)
|
||||||
|
assert declared.status_code == 201
|
||||||
|
meter_id = declared.json()["id"]
|
||||||
|
_add_bound_channel(engine, meter_id=meter_id, started_at=start, ended_at=retained_end, unit=unit)
|
||||||
|
mock_publish_discovery.reset_mock()
|
||||||
|
|
||||||
|
with patch("app.api.routes.api.meters._recompute_commodity", side_effect=AssertionError("must not recompute")):
|
||||||
|
if operation == "close":
|
||||||
|
response = client.post(
|
||||||
|
f"/api/energy/meters/{meter_id}/close", headers={"X-CSRF-Token": _CSRF},
|
||||||
|
json={"ended_at": boundary.isoformat()},
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
response = client.post(
|
||||||
|
"/api/energy/meters", headers={"X-CSRF-Token": _CSRF},
|
||||||
|
json=_declare_payload(
|
||||||
|
label="replacement", started_at=boundary.isoformat(), reason="meter_swap", commodity=commodity,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
assert response.status_code == 422
|
||||||
|
assert mock_publish_discovery.call_count == 0
|
||||||
|
with Session(engine) as observer:
|
||||||
|
meter = observer.get(Meter, meter_id)
|
||||||
|
binding = observer.scalar(select(MeterSourceBinding).where(MeterSourceBinding.meter_id == meter_id))
|
||||||
|
assert meter is not None and meter.ended_at is None
|
||||||
|
assert binding is not None and binding.ended_at.replace(tzinfo=UTC) == retained_end
|
||||||
|
assert observer.scalars(select(Meter).where(Meter.commodity == commodity)).all() == [meter]
|
||||||
|
|
||||||
|
|
||||||
|
def test_close_flushes_lifecycle_boundary_before_strict_recompute(meters_client):
|
||||||
|
"""The strict recompute query observes the just-closed meter and binding."""
|
||||||
|
client, engine = meters_client
|
||||||
|
_login(client)
|
||||||
|
started = datetime.now(UTC) - timedelta(hours=2)
|
||||||
|
boundary = started + timedelta(hours=1)
|
||||||
|
with patch("app.api.routes.api.meters.recompute_range", return_value=0):
|
||||||
|
declared = client.post(
|
||||||
|
"/api/energy/meters", headers={"X-CSRF-Token": _CSRF},
|
||||||
|
json=_declare_payload(label="flush-visible", started_at=started.isoformat(), reason="initial"),
|
||||||
|
)
|
||||||
|
meter_id = declared.json()["id"]
|
||||||
|
_add_bound_channel(engine, meter_id=meter_id, started_at=started)
|
||||||
|
|
||||||
|
def observe(session, *_args, **kwargs):
|
||||||
|
assert kwargs == {"commit": False, "strict": True}
|
||||||
|
observed_meter = session.get(Meter, meter_id)
|
||||||
|
observed_binding = session.scalar(select(MeterSourceBinding))
|
||||||
|
assert observed_meter is not None and observed_meter.ended_at is not None
|
||||||
|
assert observed_binding is not None and observed_binding.ended_at is not None
|
||||||
|
return 0
|
||||||
|
|
||||||
|
with patch("app.api.routes.api.meters.recompute_range", side_effect=observe):
|
||||||
|
response = client.post(
|
||||||
|
f"/api/energy/meters/{meter_id}/close", headers={"X-CSRF-Token": _CSRF},
|
||||||
|
json={"ended_at": boundary.isoformat()},
|
||||||
|
)
|
||||||
|
assert response.status_code == 200
|
||||||
|
|
||||||
|
|
||||||
|
def test_close_strict_compute_failure_rolls_back_persisted_lifecycle_state(meters_client, monkeypatch):
|
||||||
|
"""A real per-period strict failure rolls back the close in a fresh Session."""
|
||||||
|
from app.services import energy_cost
|
||||||
|
|
||||||
|
client, engine = meters_client
|
||||||
|
_login(client)
|
||||||
|
started = datetime.now(UTC) - timedelta(hours=2)
|
||||||
|
boundary = started + timedelta(hours=1)
|
||||||
|
with patch("app.api.routes.api.meters.recompute_range", return_value=0):
|
||||||
|
declared = client.post(
|
||||||
|
"/api/energy/meters", headers={"X-CSRF-Token": _CSRF},
|
||||||
|
json=_declare_payload(label="strict-rollback", started_at=started.isoformat(), reason="initial"),
|
||||||
|
)
|
||||||
|
meter_id = declared.json()["id"]
|
||||||
|
_add_bound_channel(engine, meter_id=meter_id, started_at=started)
|
||||||
|
monkeypatch.setattr(
|
||||||
|
energy_cost, "compute_period", lambda *_args, **_kwargs: (_ for _ in ()).throw(RuntimeError("strict boom"))
|
||||||
|
)
|
||||||
|
|
||||||
|
with pytest.raises(RuntimeError, match="strict boom"):
|
||||||
|
client.post(
|
||||||
|
f"/api/energy/meters/{meter_id}/close", headers={"X-CSRF-Token": _CSRF},
|
||||||
|
json={"ended_at": boundary.isoformat()},
|
||||||
|
)
|
||||||
|
with Session(engine) as observer:
|
||||||
|
assert observer.get(Meter, meter_id).ended_at is None
|
||||||
|
assert observer.scalar(select(MeterSourceBinding)).ended_at is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_close_flush_failure_rolls_back_persisted_lifecycle_state(meters_client, monkeypatch):
|
||||||
|
"""The mandatory pre-recompute flush shares the route rollback boundary."""
|
||||||
|
client, engine = meters_client
|
||||||
|
_login(client)
|
||||||
|
started = datetime.now(UTC) - timedelta(hours=2)
|
||||||
|
with patch("app.api.routes.api.meters.recompute_range", return_value=0):
|
||||||
|
declared = client.post(
|
||||||
|
"/api/energy/meters", headers={"X-CSRF-Token": _CSRF},
|
||||||
|
json=_declare_payload(label="flush-rollback", started_at=started.isoformat(), reason="initial"),
|
||||||
|
)
|
||||||
|
meter_id = declared.json()["id"]
|
||||||
|
_add_bound_channel(engine, meter_id=meter_id, started_at=started)
|
||||||
|
|
||||||
|
monkeypatch.setattr(Session, "flush", lambda *_args, **_kwargs: (_ for _ in ()).throw(RuntimeError("flush boom")))
|
||||||
|
with pytest.raises(RuntimeError, match="flush boom"):
|
||||||
|
client.post(
|
||||||
|
f"/api/energy/meters/{meter_id}/close", headers={"X-CSRF-Token": _CSRF},
|
||||||
|
json={"ended_at": (started + timedelta(hours=1)).isoformat()},
|
||||||
|
)
|
||||||
|
monkeypatch.undo()
|
||||||
|
with Session(engine) as observer:
|
||||||
|
assert observer.get(Meter, meter_id).ended_at is None
|
||||||
|
assert observer.scalar(select(MeterSourceBinding)).ended_at is None
|
||||||
|
|||||||
@@ -112,6 +112,36 @@ def test_app_start_seeds_missing_config_from_env_without_overwriting_existing_va
|
|||||||
assert rows["APP_NAME"] == "Database Owned Name"
|
assert rows["APP_NAME"] == "Database Owned Name"
|
||||||
assert rows["HOME_ASSISTANT_BASE_URL"] == "http://bootstrap-ha.local:8123"
|
assert rows["HOME_ASSISTANT_BASE_URL"] == "http://bootstrap-ha.local:8123"
|
||||||
assert rows["AUTH_SESSION_COOKIE_NAME"] == "home_automation_session"
|
assert rows["AUTH_SESSION_COOKIE_NAME"] == "home_automation_session"
|
||||||
|
assert rows["MQTT_CLIENT_ID"] == "home-automation"
|
||||||
|
|
||||||
|
get_settings.cache_clear()
|
||||||
|
reset_db_caches()
|
||||||
|
|
||||||
|
|
||||||
|
def test_startup_initializes_fresh_legacy_discovery_cleanup_ledger(
|
||||||
|
tmp_path, monkeypatch: pytest.MonkeyPatch
|
||||||
|
) -> None:
|
||||||
|
"""The cleanup ledger exists before the expose UI can write its first toggle."""
|
||||||
|
import app.main as main
|
||||||
|
|
||||||
|
app_database_url = _prepare_app_db(tmp_path)
|
||||||
|
monkeypatch.setenv("APP_DATABASE_URL", app_database_url)
|
||||||
|
monkeypatch.setenv("AUTH_BOOTSTRAP_USERNAME", "admin")
|
||||||
|
monkeypatch.setenv("AUTH_BOOTSTRAP_PASSWORD", "test-password")
|
||||||
|
get_settings.cache_clear()
|
||||||
|
reset_db_caches()
|
||||||
|
|
||||||
|
main.ensure_auth_db_ready()
|
||||||
|
|
||||||
|
conn = sqlite3.connect(tmp_path / "app_ready.db")
|
||||||
|
try:
|
||||||
|
value = conn.execute(
|
||||||
|
"SELECT value FROM app_config WHERE key = ?",
|
||||||
|
("HA_DISCOVERY_LEGACY_THERMAL_CLEANUP_V1",),
|
||||||
|
).fetchone()[0]
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
assert value == '{"complete": true, "inventory": [], "topics": []}'
|
||||||
|
|
||||||
get_settings.cache_clear()
|
get_settings.cache_clear()
|
||||||
reset_db_caches()
|
reset_db_caches()
|
||||||
@@ -151,3 +181,40 @@ def test_app_start_syncs_app_hostname_from_env_even_when_db_has_old_value(
|
|||||||
|
|
||||||
get_settings.cache_clear()
|
get_settings.cache_clear()
|
||||||
reset_db_caches()
|
reset_db_caches()
|
||||||
|
|
||||||
|
|
||||||
|
def test_lifespan_schedules_immediate_and_hourly_tibber_refresh(tmp_path, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||||
|
"""The single Tibber interval job starts immediately while retaining its hourly trigger."""
|
||||||
|
import app.main as main
|
||||||
|
|
||||||
|
app_database_url = _prepare_app_db(tmp_path)
|
||||||
|
added_jobs = []
|
||||||
|
|
||||||
|
class _Scheduler:
|
||||||
|
def __init__(self, **_kwargs): pass
|
||||||
|
def add_job(self, func, **kwargs): added_jobs.append((func, kwargs))
|
||||||
|
def start(self): pass
|
||||||
|
def shutdown(self, **_kwargs): pass
|
||||||
|
|
||||||
|
monkeypatch.setenv("APP_DATABASE_URL", app_database_url)
|
||||||
|
monkeypatch.setenv("AUTH_BOOTSTRAP_USERNAME", "admin")
|
||||||
|
monkeypatch.setenv("AUTH_BOOTSTRAP_PASSWORD", "test-password")
|
||||||
|
monkeypatch.setattr(main, "BackgroundScheduler", _Scheduler)
|
||||||
|
monkeypatch.setattr(main.mqtt_manager, "connect", lambda _settings: None)
|
||||||
|
monkeypatch.setattr(main.mqtt_manager, "disconnect", lambda: None)
|
||||||
|
monkeypatch.setattr(main, "apply_dsmr_subscription", lambda _settings: None)
|
||||||
|
monkeypatch.setattr(main.warmtelink_worker_manager, "start", lambda: None)
|
||||||
|
monkeypatch.setattr(main.warmtelink_worker_manager, "shutdown", lambda: None)
|
||||||
|
get_settings.cache_clear()
|
||||||
|
reset_db_caches()
|
||||||
|
|
||||||
|
anyio.run(_run_lifespan, create_app())
|
||||||
|
|
||||||
|
tibber_jobs = [kwargs for func, kwargs in added_jobs if func is main._run_scheduled_tibber_refresh]
|
||||||
|
assert len(tibber_jobs) == 1
|
||||||
|
assert tibber_jobs[0]["id"] == "tibber-refresh"
|
||||||
|
assert tibber_jobs[0]["max_instances"] == 1
|
||||||
|
assert tibber_jobs[0]["next_run_time"] is not None
|
||||||
|
|
||||||
|
get_settings.cache_clear()
|
||||||
|
reset_db_caches()
|
||||||
|
|||||||
@@ -72,6 +72,54 @@ def test_compose_uses_migration_job_before_app() -> None:
|
|||||||
assert dev["services"]["app"]["build"] == "."
|
assert dev["services"]["app"]["build"] == "."
|
||||||
|
|
||||||
|
|
||||||
|
def test_compose_defaults_business_timezone_and_dev_inherits_it() -> None:
|
||||||
|
"""Both production services receive an overridable Amsterdam timezone.
|
||||||
|
|
||||||
|
Compose merges service ``environment`` mappings, and the dev override does
|
||||||
|
not replace either mapping, so the base default applies to base+dev too.
|
||||||
|
"""
|
||||||
|
base = _read_yaml("docker-compose.yml")
|
||||||
|
dev = _read_yaml("docker-compose.dev.yml")
|
||||||
|
default_tz = "${TZ:-Europe/Amsterdam}"
|
||||||
|
|
||||||
|
for service_name in ("migration", "app"):
|
||||||
|
assert base["services"][service_name]["environment"]["TZ"] == default_tz
|
||||||
|
assert "TZ" not in dev["services"].get(service_name, {}).get("environment", {})
|
||||||
|
|
||||||
|
|
||||||
|
def test_compose_keeps_app_non_root_and_maps_minimal_warmtelink_serial_access() -> None:
|
||||||
|
"""Base Compose maps the configured device with only pyserial's required access.
|
||||||
|
|
||||||
|
Environment interpolation is deliberately left unresolved: operators supply
|
||||||
|
a host-specific stable by-id path and its numeric serial GID in local .env.
|
||||||
|
The dev compose file inherits this app configuration unchanged.
|
||||||
|
"""
|
||||||
|
base = _read_yaml("docker-compose.yml")
|
||||||
|
dev = _read_yaml("docker-compose.dev.yml")
|
||||||
|
|
||||||
|
base_app = base["services"]["app"]
|
||||||
|
base_migration = base["services"]["migration"]
|
||||||
|
assert base_app["user"] == "1000:1000"
|
||||||
|
assert base_migration["user"] == "1000:1000"
|
||||||
|
assert "privileged" not in base_app
|
||||||
|
assert "privileged" not in base_migration
|
||||||
|
assert "cap_add" not in base_app
|
||||||
|
assert "cap_add" not in base_migration
|
||||||
|
assert base_app["devices"] == [
|
||||||
|
"${WARMTELINK_DEVICE_PATH:?Set a stable /dev/serial/by-id path}:/dev/warmtelink:rw"
|
||||||
|
]
|
||||||
|
device_rule = base_app["devices"][0]
|
||||||
|
permissions = device_rule.rsplit(":", maxsplit=1)[1]
|
||||||
|
assert permissions == "rw"
|
||||||
|
assert "m" not in permissions
|
||||||
|
assert base_app["group_add"] == ["${WARMTELINK_SERIAL_GID:?Set the host serial device GID}"]
|
||||||
|
assert dev["services"]["app"]["devices"] == base_app["devices"]
|
||||||
|
assert dev["services"]["app"]["group_add"] == base_app["group_add"]
|
||||||
|
assert "devices" not in base_migration
|
||||||
|
assert "group_add" not in base_migration
|
||||||
|
assert "privileged" not in base_migration
|
||||||
|
|
||||||
|
|
||||||
def test_image_defaults_to_uvicorn_only() -> None:
|
def test_image_defaults_to_uvicorn_only() -> None:
|
||||||
dockerfile = (PROJECT_ROOT / "Dockerfile").read_text()
|
dockerfile = (PROJECT_ROOT / "Dockerfile").read_text()
|
||||||
entrypoint = (PROJECT_ROOT / "docker/entrypoint.sh").read_text()
|
entrypoint = (PROJECT_ROOT / "docker/entrypoint.sh").read_text()
|
||||||
|
|||||||
+427
-37
@@ -20,8 +20,9 @@ Covers:
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import json
|
import json
|
||||||
|
from datetime import datetime, timezone
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from unittest.mock import MagicMock, patch
|
from unittest.mock import patch
|
||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
from alembic import command
|
from alembic import command
|
||||||
@@ -30,6 +31,7 @@ from sqlalchemy import create_engine, select
|
|||||||
from sqlalchemy.orm import Session
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
from app.models.energy import DsmrReading
|
from app.models.energy import DsmrReading
|
||||||
|
from app.models.meter_source import MeterSource
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
@@ -63,11 +65,9 @@ def _make_settings(
|
|||||||
dsmr_ingest_enabled: bool = True,
|
dsmr_ingest_enabled: bool = True,
|
||||||
dsmr_mqtt_topic: str = "dsmr/json",
|
dsmr_mqtt_topic: str = "dsmr/json",
|
||||||
):
|
):
|
||||||
s = MagicMock()
|
del dsmr_ingest_enabled
|
||||||
s.dsmr_sample_interval_s = dsmr_sample_interval_s
|
from app.services.dsmr_ingest import DsmrSourceSnapshot
|
||||||
s.dsmr_ingest_enabled = dsmr_ingest_enabled
|
return DsmrSourceSnapshot(1, dsmr_mqtt_topic, "", dsmr_sample_interval_s)
|
||||||
s.dsmr_mqtt_topic = dsmr_mqtt_topic
|
|
||||||
return s
|
|
||||||
|
|
||||||
|
|
||||||
# The reference telegram sample from §6.3 of the design doc.
|
# The reference telegram sample from §6.3 of the design doc.
|
||||||
@@ -175,6 +175,264 @@ def test_second_00_persists_full_frame(dsmr_db):
|
|||||||
assert payload["phase_voltage_l2"] is None
|
assert payload["phase_voltage_l2"] is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_accepted_message_marks_source_online_and_updates_last_seen(dsmr_db):
|
||||||
|
"""A valid, accepted telegram clears stale diagnostics without changing its payload."""
|
||||||
|
engine, SessionLocal = dsmr_db
|
||||||
|
settings = _make_settings(dsmr_sample_interval_s=10)
|
||||||
|
with Session(engine) as session:
|
||||||
|
source = session.get(MeterSource, 1)
|
||||||
|
assert source is not None
|
||||||
|
source.enabled = True
|
||||||
|
source.status = "error"
|
||||||
|
source.last_error = "old error"
|
||||||
|
session.commit()
|
||||||
|
|
||||||
|
_call_handle_message(_SAMPLE_TELEGRAM, settings, SessionLocal)
|
||||||
|
|
||||||
|
with Session(engine) as session:
|
||||||
|
source = session.get(MeterSource, 1)
|
||||||
|
assert source is not None
|
||||||
|
assert source.status == "online"
|
||||||
|
assert source.last_error is None
|
||||||
|
assert source.last_seen_at is not None
|
||||||
|
|
||||||
|
|
||||||
|
def test_source_state_callback_persists_only_current_generation(dsmr_db, monkeypatch):
|
||||||
|
"""Connection callbacks use a short session and stale generations leave health untouched."""
|
||||||
|
engine, SessionLocal = dsmr_db
|
||||||
|
from app.services import dsmr_ingest
|
||||||
|
|
||||||
|
snapshot = _make_settings()
|
||||||
|
current = object()
|
||||||
|
monkeypatch.setattr(dsmr_ingest, "_subscription_tokens", {snapshot.source_id: current})
|
||||||
|
with Session(engine) as session:
|
||||||
|
source = session.get(MeterSource, snapshot.source_id)
|
||||||
|
assert source is not None
|
||||||
|
source.enabled = True
|
||||||
|
session.commit()
|
||||||
|
with patch.object(dsmr_ingest, "get_session_local", return_value=SessionLocal):
|
||||||
|
dsmr_ingest.handle_captured_source_state(snapshot, current, "connecting")
|
||||||
|
dsmr_ingest.handle_captured_source_state(snapshot, current, "error")
|
||||||
|
dsmr_ingest.handle_captured_source_state(snapshot, object(), "online")
|
||||||
|
|
||||||
|
with Session(engine) as session:
|
||||||
|
source = session.get(MeterSource, snapshot.source_id)
|
||||||
|
assert source is not None
|
||||||
|
assert source.status == "error"
|
||||||
|
assert source.last_error == "MQTT connection failed."
|
||||||
|
|
||||||
|
|
||||||
|
def test_source_health_is_isolated_between_sources(dsmr_db, monkeypatch):
|
||||||
|
"""Connection and message health changes only touch their own source row."""
|
||||||
|
engine, SessionLocal = dsmr_db
|
||||||
|
from app.services import dsmr_ingest
|
||||||
|
from app.services.dsmr_ingest import DsmrSourceSnapshot
|
||||||
|
|
||||||
|
first = _make_settings()
|
||||||
|
second = DsmrSourceSnapshot(2, "second/topic", "", 10)
|
||||||
|
first_token = object()
|
||||||
|
second_token = object()
|
||||||
|
monkeypatch.setattr(
|
||||||
|
dsmr_ingest, "_subscription_tokens", {first.source_id: first_token, second.source_id: second_token}
|
||||||
|
)
|
||||||
|
with Session(engine) as session:
|
||||||
|
first_source = session.get(MeterSource, first.source_id)
|
||||||
|
assert first_source is not None
|
||||||
|
first_source.enabled = True
|
||||||
|
session.add(MeterSource(
|
||||||
|
id=2, name="Second DSMR", kind="dsmr_mqtt", enabled=True, config={},
|
||||||
|
status="online", created_at=datetime.now(timezone.utc), updated_at=datetime.now(timezone.utc),
|
||||||
|
))
|
||||||
|
session.commit()
|
||||||
|
|
||||||
|
with patch.object(dsmr_ingest, "get_session_local", return_value=SessionLocal):
|
||||||
|
dsmr_ingest.handle_captured_source_state(first, first_token, "connecting")
|
||||||
|
dsmr_ingest.handle_captured_source_state(second, second_token, "error")
|
||||||
|
dsmr_ingest.handle_message(_payload(_SAMPLE_TELEGRAM), first)
|
||||||
|
|
||||||
|
with Session(engine) as session:
|
||||||
|
first_source = session.get(MeterSource, first.source_id)
|
||||||
|
second_source = session.get(MeterSource, second.source_id)
|
||||||
|
assert first_source is not None and second_source is not None
|
||||||
|
assert first_source.status == "online"
|
||||||
|
assert first_source.last_error is None
|
||||||
|
assert first_source.last_seen_at is not None
|
||||||
|
assert second_source.status == "error"
|
||||||
|
assert second_source.last_error == "MQTT connection failed."
|
||||||
|
assert second_source.last_seen_at is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_disable_reconcile_persists_unknown_and_rejects_retained_state_callback(dsmr_db, monkeypatch):
|
||||||
|
"""Disable invalidates the generation before clearing a stale online health state."""
|
||||||
|
engine, SessionLocal = dsmr_db
|
||||||
|
from app.services import dsmr_ingest
|
||||||
|
|
||||||
|
class FakeMqtt:
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self.removed: list[int] = []
|
||||||
|
self.state_handler = None
|
||||||
|
self.active: set[int] = set()
|
||||||
|
|
||||||
|
def replace_source(self, source_id: int, **kwargs: object) -> bool:
|
||||||
|
self.active.add(source_id)
|
||||||
|
self.state_handler = kwargs["state_handler"]
|
||||||
|
return True
|
||||||
|
|
||||||
|
def remove_source(self, source_id: int) -> None:
|
||||||
|
self.removed.append(source_id)
|
||||||
|
self.active.discard(source_id)
|
||||||
|
|
||||||
|
def source_is_active(self, source_id: int) -> bool:
|
||||||
|
return source_id in self.active
|
||||||
|
|
||||||
|
fake = FakeMqtt()
|
||||||
|
snapshot = _make_settings()
|
||||||
|
monkeypatch.setattr(dsmr_ingest, "_subscriptions", {})
|
||||||
|
monkeypatch.setattr(dsmr_ingest, "_subscription_client_ids", {})
|
||||||
|
monkeypatch.setattr(dsmr_ingest, "_subscription_tokens", {})
|
||||||
|
monkeypatch.setattr(dsmr_ingest, "_tariffs", {})
|
||||||
|
monkeypatch.setattr(dsmr_ingest, "get_session_local", lambda: SessionLocal)
|
||||||
|
monkeypatch.setattr("app.integrations.mqtt.mqtt_manager", fake)
|
||||||
|
monkeypatch.setattr(dsmr_ingest, "_enabled_snapshots", lambda: [snapshot])
|
||||||
|
with Session(engine) as session:
|
||||||
|
source = session.get(MeterSource, snapshot.source_id)
|
||||||
|
assert source is not None
|
||||||
|
source.enabled = True
|
||||||
|
source.status = "online"
|
||||||
|
session.commit()
|
||||||
|
dsmr_ingest.apply_dsmr_subscription()
|
||||||
|
assert fake.state_handler is not None
|
||||||
|
|
||||||
|
with Session(engine) as session:
|
||||||
|
source = session.get(MeterSource, snapshot.source_id)
|
||||||
|
assert source is not None
|
||||||
|
source.enabled = False
|
||||||
|
session.commit()
|
||||||
|
monkeypatch.setattr(dsmr_ingest, "_enabled_snapshots", lambda: [])
|
||||||
|
dsmr_ingest.apply_dsmr_subscription()
|
||||||
|
fake.state_handler("online")
|
||||||
|
|
||||||
|
with Session(engine) as session:
|
||||||
|
source = session.get(MeterSource, snapshot.source_id)
|
||||||
|
assert source is not None
|
||||||
|
assert source.status == "unknown"
|
||||||
|
assert source.last_error is None
|
||||||
|
assert fake.removed == [snapshot.source_id]
|
||||||
|
|
||||||
|
|
||||||
|
def test_topic_collision_persists_error_when_reconcile_removes_active_client(dsmr_db, monkeypatch):
|
||||||
|
"""A rejected enabled source cannot retain health from its removed client."""
|
||||||
|
engine, SessionLocal = dsmr_db
|
||||||
|
from app.services import dsmr_ingest
|
||||||
|
|
||||||
|
class FakeMqtt:
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self.removed: list[int] = []
|
||||||
|
self.active: set[int] = set()
|
||||||
|
|
||||||
|
def replace_source(self, source_id: int, **kwargs: object) -> bool:
|
||||||
|
self.active.add(source_id)
|
||||||
|
return True
|
||||||
|
|
||||||
|
def remove_source(self, source_id: int) -> None:
|
||||||
|
self.removed.append(source_id)
|
||||||
|
self.active.discard(source_id)
|
||||||
|
|
||||||
|
def source_is_active(self, source_id: int) -> bool:
|
||||||
|
return source_id in self.active
|
||||||
|
|
||||||
|
fake = FakeMqtt()
|
||||||
|
valid = _make_settings(dsmr_mqtt_topic="dsmr/telegram")
|
||||||
|
collision = _make_settings(dsmr_mqtt_topic="dsmr/telegram")
|
||||||
|
collision = dsmr_ingest.DsmrSourceSnapshot(
|
||||||
|
collision.source_id,
|
||||||
|
collision.topic,
|
||||||
|
collision.topic,
|
||||||
|
collision.sample_interval_s,
|
||||||
|
)
|
||||||
|
monkeypatch.setattr(dsmr_ingest, "_subscriptions", {})
|
||||||
|
monkeypatch.setattr(dsmr_ingest, "_subscription_client_ids", {})
|
||||||
|
monkeypatch.setattr(dsmr_ingest, "_subscription_tokens", {})
|
||||||
|
monkeypatch.setattr(dsmr_ingest, "_tariffs", {})
|
||||||
|
monkeypatch.setattr(dsmr_ingest, "get_session_local", lambda: SessionLocal)
|
||||||
|
monkeypatch.setattr("app.integrations.mqtt.mqtt_manager", fake)
|
||||||
|
with Session(engine) as session:
|
||||||
|
source = session.get(MeterSource, valid.source_id)
|
||||||
|
assert source is not None
|
||||||
|
source.enabled = True
|
||||||
|
source.status = "online"
|
||||||
|
session.commit()
|
||||||
|
|
||||||
|
monkeypatch.setattr(dsmr_ingest, "_enabled_snapshots", lambda: [valid])
|
||||||
|
dsmr_ingest.apply_dsmr_subscription()
|
||||||
|
monkeypatch.setattr(dsmr_ingest, "_enabled_snapshots", lambda: [collision])
|
||||||
|
dsmr_ingest.apply_dsmr_subscription()
|
||||||
|
|
||||||
|
with Session(engine) as session:
|
||||||
|
source = session.get(MeterSource, valid.source_id)
|
||||||
|
assert source is not None
|
||||||
|
assert source.enabled is True
|
||||||
|
assert source.status == "error"
|
||||||
|
assert source.last_error == "DSMR source configuration invalid."
|
||||||
|
assert fake.removed == [valid.source_id]
|
||||||
|
assert fake.active == set()
|
||||||
|
|
||||||
|
|
||||||
|
def test_topic_collision_persists_error_on_startup_without_runtime_client(dsmr_db, monkeypatch):
|
||||||
|
"""A collision corrects stale persisted online health without an installed client."""
|
||||||
|
engine, SessionLocal = dsmr_db
|
||||||
|
from app.services import dsmr_ingest
|
||||||
|
|
||||||
|
collision = dsmr_ingest.DsmrSourceSnapshot(1, "dsmr/telegram", "dsmr/telegram", 10)
|
||||||
|
monkeypatch.setattr(dsmr_ingest, "_subscriptions", {})
|
||||||
|
monkeypatch.setattr(dsmr_ingest, "_subscription_client_ids", {})
|
||||||
|
monkeypatch.setattr(dsmr_ingest, "_subscription_tokens", {})
|
||||||
|
monkeypatch.setattr(dsmr_ingest, "get_session_local", lambda: SessionLocal)
|
||||||
|
monkeypatch.setattr(dsmr_ingest, "_enabled_snapshots", lambda: [collision])
|
||||||
|
with Session(engine) as session:
|
||||||
|
source = session.get(MeterSource, collision.source_id)
|
||||||
|
assert source is not None
|
||||||
|
source.enabled = True
|
||||||
|
source.status = "online"
|
||||||
|
session.commit()
|
||||||
|
|
||||||
|
dsmr_ingest.apply_dsmr_subscription()
|
||||||
|
|
||||||
|
with Session(engine) as session:
|
||||||
|
source = session.get(MeterSource, collision.source_id)
|
||||||
|
assert source is not None
|
||||||
|
assert source.status == "error"
|
||||||
|
assert source.last_error == "DSMR source configuration invalid."
|
||||||
|
|
||||||
|
|
||||||
|
def test_topic_collision_does_not_change_a_disabled_source(dsmr_db, monkeypatch):
|
||||||
|
"""A concurrent disable is not overwritten by collision error handling."""
|
||||||
|
engine, SessionLocal = dsmr_db
|
||||||
|
from app.services import dsmr_ingest
|
||||||
|
|
||||||
|
collision = dsmr_ingest.DsmrSourceSnapshot(1, "dsmr/telegram", "dsmr/telegram", 10)
|
||||||
|
monkeypatch.setattr(dsmr_ingest, "_subscriptions", {})
|
||||||
|
monkeypatch.setattr(dsmr_ingest, "_subscription_client_ids", {})
|
||||||
|
monkeypatch.setattr(dsmr_ingest, "_subscription_tokens", {})
|
||||||
|
monkeypatch.setattr(dsmr_ingest, "get_session_local", lambda: SessionLocal)
|
||||||
|
monkeypatch.setattr(dsmr_ingest, "_enabled_snapshots", lambda: [collision])
|
||||||
|
with Session(engine) as session:
|
||||||
|
source = session.get(MeterSource, collision.source_id)
|
||||||
|
assert source is not None
|
||||||
|
source.enabled = False
|
||||||
|
source.status = "unknown"
|
||||||
|
source.last_error = None
|
||||||
|
session.commit()
|
||||||
|
|
||||||
|
dsmr_ingest.apply_dsmr_subscription()
|
||||||
|
|
||||||
|
with Session(engine) as session:
|
||||||
|
source = session.get(MeterSource, collision.source_id)
|
||||||
|
assert source is not None
|
||||||
|
assert source.status == "unknown"
|
||||||
|
assert source.last_error is None
|
||||||
|
|
||||||
|
|
||||||
def test_second_10_persists(dsmr_db):
|
def test_second_10_persists(dsmr_db):
|
||||||
"""A telegram with second=10 (another 10s boundary) must also be persisted."""
|
"""A telegram with second=10 (another 10s boundary) must also be persisted."""
|
||||||
_, SessionLocal = dsmr_db
|
_, SessionLocal = dsmr_db
|
||||||
@@ -240,6 +498,20 @@ def test_different_timestamps_are_independent(dsmr_db):
|
|||||||
assert _count_readings(SessionLocal) == 2
|
assert _count_readings(SessionLocal) == 2
|
||||||
|
|
||||||
|
|
||||||
|
def test_two_sources_can_store_the_same_timestamp_independently(dsmr_db):
|
||||||
|
"""The source identity, not timestamp alone, defines DSMR idempotency."""
|
||||||
|
_, SessionLocal = dsmr_db
|
||||||
|
first = _make_settings(dsmr_sample_interval_s=10)
|
||||||
|
from app.services.dsmr_ingest import DsmrSourceSnapshot
|
||||||
|
|
||||||
|
second = DsmrSourceSnapshot(2, "second/topic", "", 10)
|
||||||
|
_call_handle_message(_SAMPLE_TELEGRAM, first, SessionLocal)
|
||||||
|
_call_handle_message(_SAMPLE_TELEGRAM, second, SessionLocal)
|
||||||
|
|
||||||
|
rows = _get_readings(SessionLocal)
|
||||||
|
assert {row.meter_source_id for row in rows} == {1, 2}
|
||||||
|
|
||||||
|
|
||||||
def test_telegram_id_collision_does_not_drop_new_data(dsmr_db):
|
def test_telegram_id_collision_does_not_drop_new_data(dsmr_db):
|
||||||
"""Regression: the telegram id overflows / gets reset to zero in DSMR firmware.
|
"""Regression: the telegram id overflows / gets reset to zero in DSMR firmware.
|
||||||
Two DISTINCT telegrams (different timestamps) that happen to share the SAME
|
Two DISTINCT telegrams (different timestamps) that happen to share the SAME
|
||||||
@@ -259,12 +531,12 @@ def test_telegram_id_collision_does_not_drop_new_data(dsmr_db):
|
|||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# 4. Missing source_id — still persisted with source_id=None
|
# 4. Missing telegram id — still persisted with telegram_id=None
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
def test_missing_id_persisted_with_source_id_none(dsmr_db):
|
def test_missing_id_persisted_with_telegram_id_none(dsmr_db):
|
||||||
"""Telegram without an 'id' field must be stored with source_id=None."""
|
"""Telegram without an 'id' field must be stored with telegram_id=None."""
|
||||||
engine, SessionLocal = dsmr_db
|
engine, SessionLocal = dsmr_db
|
||||||
settings = _make_settings(dsmr_sample_interval_s=10)
|
settings = _make_settings(dsmr_sample_interval_s=10)
|
||||||
|
|
||||||
@@ -276,7 +548,7 @@ def test_missing_id_persisted_with_source_id_none(dsmr_db):
|
|||||||
readings = session.scalars(select(DsmrReading)).all()
|
readings = session.scalars(select(DsmrReading)).all()
|
||||||
|
|
||||||
assert len(readings) == 1
|
assert len(readings) == 1
|
||||||
assert readings[0].source_id is None
|
assert readings[0].telegram_id is None
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
@@ -419,80 +691,87 @@ def test_timestamp_with_utc_offset_suffix(dsmr_db):
|
|||||||
|
|
||||||
@pytest.fixture()
|
@pytest.fixture()
|
||||||
def reset_tariff(monkeypatch):
|
def reset_tariff(monkeypatch):
|
||||||
"""Reset _current_tariff to None before and after each tariff test."""
|
|
||||||
from app.services import dsmr_ingest as _di
|
from app.services import dsmr_ingest as _di
|
||||||
|
|
||||||
|
monkeypatch.setattr(_di, "_tariffs", {})
|
||||||
monkeypatch.setattr(_di, "_current_tariff", None)
|
monkeypatch.setattr(_di, "_current_tariff", None)
|
||||||
yield
|
|
||||||
# monkeypatch auto-restores on teardown
|
|
||||||
|
|
||||||
|
|
||||||
def test_tariff_message_value_2_sets_tariff(reset_tariff):
|
def test_tariff_message_value_2_sets_tariff(reset_tariff):
|
||||||
"""Payload b'2' must set the current tariff to 2 (normal/peak)."""
|
"""Payload b'2' must set the current tariff to 2 (normal/peak)."""
|
||||||
from app.services.dsmr_ingest import handle_tariff_message, get_current_tariff
|
from app.services.dsmr_ingest import handle_tariff_message, get_current_tariff
|
||||||
|
|
||||||
handle_tariff_message(b"2")
|
handle_tariff_message(b"2", 1)
|
||||||
assert get_current_tariff() == 2
|
assert get_current_tariff(1) == 2
|
||||||
|
|
||||||
|
|
||||||
def test_tariff_message_value_1_sets_tariff(reset_tariff):
|
def test_tariff_message_value_1_sets_tariff(reset_tariff):
|
||||||
"""Payload b'1' must set the current tariff to 1 (dal/off-peak)."""
|
"""Payload b'1' must set the current tariff to 1 (dal/off-peak)."""
|
||||||
from app.services.dsmr_ingest import handle_tariff_message, get_current_tariff
|
from app.services.dsmr_ingest import handle_tariff_message, get_current_tariff
|
||||||
|
|
||||||
handle_tariff_message(b"1")
|
handle_tariff_message(b"1", 1)
|
||||||
assert get_current_tariff() == 1
|
assert get_current_tariff(1) == 1
|
||||||
|
|
||||||
|
|
||||||
def test_tariff_message_updates_from_2_to_1(reset_tariff):
|
def test_tariff_message_updates_from_2_to_1(reset_tariff):
|
||||||
"""Subsequent payloads must overwrite the previous tariff value."""
|
"""Subsequent payloads must overwrite the previous tariff value."""
|
||||||
from app.services.dsmr_ingest import handle_tariff_message, get_current_tariff
|
from app.services.dsmr_ingest import handle_tariff_message, get_current_tariff
|
||||||
|
|
||||||
handle_tariff_message(b"2")
|
handle_tariff_message(b"2", 1)
|
||||||
assert get_current_tariff() == 2
|
assert get_current_tariff(1) == 2
|
||||||
handle_tariff_message(b"1")
|
handle_tariff_message(b"1", 1)
|
||||||
assert get_current_tariff() == 1
|
assert get_current_tariff(1) == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_tariffs_are_isolated_by_source(reset_tariff):
|
||||||
|
from app.services.dsmr_ingest import handle_tariff_message, get_current_tariff
|
||||||
|
|
||||||
|
handle_tariff_message(b"1", 1)
|
||||||
|
handle_tariff_message(b"2", 2)
|
||||||
|
assert get_current_tariff(1) == 1
|
||||||
|
assert get_current_tariff(2) == 2
|
||||||
|
|
||||||
|
|
||||||
def test_tariff_message_strips_whitespace(reset_tariff):
|
def test_tariff_message_strips_whitespace(reset_tariff):
|
||||||
"""Payloads with surrounding whitespace (e.g. b'2\\n') must be accepted."""
|
"""Payloads with surrounding whitespace (e.g. b'2\\n') must be accepted."""
|
||||||
from app.services.dsmr_ingest import handle_tariff_message, get_current_tariff
|
from app.services.dsmr_ingest import handle_tariff_message, get_current_tariff
|
||||||
|
|
||||||
handle_tariff_message(b"2\n")
|
handle_tariff_message(b"2\n", 1)
|
||||||
assert get_current_tariff() == 2
|
assert get_current_tariff(1) == 2
|
||||||
|
|
||||||
handle_tariff_message(b" 1 ")
|
handle_tariff_message(b" 1 ", 1)
|
||||||
assert get_current_tariff() == 1
|
assert get_current_tariff(1) == 1
|
||||||
|
|
||||||
|
|
||||||
def test_tariff_message_invalid_non_numeric_does_not_update(reset_tariff):
|
def test_tariff_message_invalid_non_numeric_does_not_update(reset_tariff):
|
||||||
"""Non-numeric payload must not update the tariff; previous value is preserved."""
|
"""Non-numeric payload must not update the tariff; previous value is preserved."""
|
||||||
from app.services.dsmr_ingest import handle_tariff_message, get_current_tariff, set_current_tariff
|
from app.services.dsmr_ingest import handle_tariff_message, get_current_tariff, set_current_tariff
|
||||||
|
|
||||||
set_current_tariff(2)
|
set_current_tariff(1, 2)
|
||||||
handle_tariff_message(b"x")
|
handle_tariff_message(b"x", 1)
|
||||||
# Must NOT raise and must NOT change the tariff.
|
# Must NOT raise and must NOT change the tariff.
|
||||||
assert get_current_tariff() == 2
|
assert get_current_tariff(1) == 2
|
||||||
|
|
||||||
|
|
||||||
def test_tariff_message_invalid_empty_does_not_update(reset_tariff):
|
def test_tariff_message_invalid_empty_does_not_update(reset_tariff):
|
||||||
"""Empty payload must not update the tariff; previous value is preserved."""
|
"""Empty payload must not update the tariff; previous value is preserved."""
|
||||||
from app.services.dsmr_ingest import handle_tariff_message, get_current_tariff, set_current_tariff
|
from app.services.dsmr_ingest import handle_tariff_message, get_current_tariff, set_current_tariff
|
||||||
|
|
||||||
set_current_tariff(1)
|
set_current_tariff(1, 1)
|
||||||
handle_tariff_message(b"")
|
handle_tariff_message(b"", 1)
|
||||||
assert get_current_tariff() == 1
|
assert get_current_tariff(1) == 1
|
||||||
|
|
||||||
|
|
||||||
def test_tariff_message_out_of_range_value_does_not_update(reset_tariff):
|
def test_tariff_message_out_of_range_value_does_not_update(reset_tariff):
|
||||||
"""Payload with out-of-range integer (not 1 or 2) must not update the tariff."""
|
"""Payload with out-of-range integer (not 1 or 2) must not update the tariff."""
|
||||||
from app.services.dsmr_ingest import handle_tariff_message, get_current_tariff, set_current_tariff
|
from app.services.dsmr_ingest import handle_tariff_message, get_current_tariff, set_current_tariff
|
||||||
|
|
||||||
set_current_tariff(2)
|
set_current_tariff(1, 2)
|
||||||
handle_tariff_message(b"3") # 3 is not a valid tariff
|
handle_tariff_message(b"3", 1) # 3 is not a valid tariff
|
||||||
assert get_current_tariff() == 2
|
assert get_current_tariff(1) == 2
|
||||||
|
|
||||||
handle_tariff_message(b"0") # 0 is not a valid tariff
|
handle_tariff_message(b"0", 1) # 0 is not a valid tariff
|
||||||
assert get_current_tariff() == 2
|
assert get_current_tariff(1) == 2
|
||||||
|
|
||||||
|
|
||||||
def test_tariff_message_does_not_raise_on_any_input(reset_tariff):
|
def test_tariff_message_does_not_raise_on_any_input(reset_tariff):
|
||||||
@@ -501,4 +780,115 @@ def test_tariff_message_does_not_raise_on_any_input(reset_tariff):
|
|||||||
|
|
||||||
# All of these must complete without raising.
|
# All of these must complete without raising.
|
||||||
for payload in (b"", b"x", b"99", b"\xff\xfe", b"None", b"2.0"):
|
for payload in (b"", b"x", b"99", b"\xff\xfe", b"None", b"2.0"):
|
||||||
handle_tariff_message(payload) # must not raise
|
handle_tariff_message(payload, 1) # must not raise
|
||||||
|
|
||||||
|
|
||||||
|
def test_electricity_tariff_resolves_current_binding_and_handoff(dsmr_db, reset_tariff):
|
||||||
|
"""Runtime tariffs remain isolated and are selected through the active binding."""
|
||||||
|
from datetime import timedelta
|
||||||
|
|
||||||
|
from app.models.energy import Meter
|
||||||
|
from app.models.meter_source import MeterSource, MeterSourceBinding, MeterSourceChannel
|
||||||
|
from app.services.dsmr_ingest import get_current_electricity_tariff, handle_tariff_message
|
||||||
|
|
||||||
|
_, SessionLocal = dsmr_db
|
||||||
|
now = datetime.now(timezone.utc).replace(microsecond=0)
|
||||||
|
with SessionLocal() as session:
|
||||||
|
meter = Meter(
|
||||||
|
label="electricity",
|
||||||
|
commodity="electricity",
|
||||||
|
started_at=now - timedelta(days=2),
|
||||||
|
ended_at=None,
|
||||||
|
reason="initial",
|
||||||
|
note=None,
|
||||||
|
created_at=now,
|
||||||
|
)
|
||||||
|
first = MeterSource(
|
||||||
|
name="first", kind="dsmr_mqtt", enabled=True, config={}, created_at=now, updated_at=now
|
||||||
|
)
|
||||||
|
second = MeterSource(
|
||||||
|
name="second", kind="dsmr_mqtt", enabled=True, config={}, created_at=now, updated_at=now
|
||||||
|
)
|
||||||
|
session.add_all([meter, first, second])
|
||||||
|
session.flush()
|
||||||
|
first_channel = MeterSourceChannel(
|
||||||
|
source_id=first.id, channel_key="electricity", label="first", unit="kWh",
|
||||||
|
created_at=now, updated_at=now
|
||||||
|
)
|
||||||
|
second_channel = MeterSourceChannel(
|
||||||
|
source_id=second.id, channel_key="electricity", label="second", unit="kWh",
|
||||||
|
created_at=now, updated_at=now
|
||||||
|
)
|
||||||
|
session.add_all([first_channel, second_channel])
|
||||||
|
session.flush()
|
||||||
|
handoff = now - timedelta(hours=1)
|
||||||
|
session.add_all([
|
||||||
|
MeterSourceBinding(meter_id=meter.id, channel_id=first_channel.id, started_at=now - timedelta(days=2), ended_at=handoff, created_at=now, updated_at=now),
|
||||||
|
MeterSourceBinding(meter_id=meter.id, channel_id=second_channel.id, started_at=handoff, ended_at=None, created_at=now, updated_at=now),
|
||||||
|
])
|
||||||
|
session.commit()
|
||||||
|
handle_tariff_message(b"1", first.id)
|
||||||
|
handle_tariff_message(b"2", second.id)
|
||||||
|
assert get_current_electricity_tariff(session, now - timedelta(days=3)) is None
|
||||||
|
assert get_current_electricity_tariff(session, handoff - timedelta(seconds=1)) == 1
|
||||||
|
assert get_current_electricity_tariff(session, handoff) == 2
|
||||||
|
assert get_current_electricity_tariff(session, now + timedelta(days=3)) == 2
|
||||||
|
|
||||||
|
|
||||||
|
def test_legacy_getter_resolves_runtime_tariff_through_current_binding(dsmr_db, reset_tariff):
|
||||||
|
"""The unchanged no-argument caller selects the bound source, not a global tariff."""
|
||||||
|
from datetime import timedelta
|
||||||
|
|
||||||
|
from app.models.energy import Meter
|
||||||
|
from app.models.meter_source import MeterSource, MeterSourceBinding, MeterSourceChannel
|
||||||
|
from app.services import dsmr_ingest
|
||||||
|
|
||||||
|
_, SessionLocal = dsmr_db
|
||||||
|
now = datetime.now(timezone.utc).replace(microsecond=0)
|
||||||
|
with SessionLocal() as session:
|
||||||
|
meter = Meter(
|
||||||
|
label="electricity", commodity="electricity", started_at=now - timedelta(days=1),
|
||||||
|
ended_at=None, reason="initial", note=None, created_at=now,
|
||||||
|
)
|
||||||
|
bound = MeterSource(
|
||||||
|
name="bound", kind="dsmr_mqtt", enabled=True, config={}, created_at=now, updated_at=now
|
||||||
|
)
|
||||||
|
other = MeterSource(
|
||||||
|
name="other", kind="dsmr_mqtt", enabled=True, config={}, created_at=now, updated_at=now
|
||||||
|
)
|
||||||
|
session.add_all([meter, bound, other])
|
||||||
|
session.flush()
|
||||||
|
channel = MeterSourceChannel(
|
||||||
|
source_id=bound.id, channel_key="electricity", label="bound", unit="kWh",
|
||||||
|
created_at=now, updated_at=now,
|
||||||
|
)
|
||||||
|
session.add(channel)
|
||||||
|
session.flush()
|
||||||
|
session.add(MeterSourceBinding(
|
||||||
|
meter_id=meter.id, channel_id=channel.id, started_at=now - timedelta(days=1), ended_at=None,
|
||||||
|
created_at=now, updated_at=now,
|
||||||
|
))
|
||||||
|
session.commit()
|
||||||
|
dsmr_ingest.handle_tariff_message(b"1", bound.id)
|
||||||
|
dsmr_ingest.handle_tariff_message(b"2", other.id)
|
||||||
|
|
||||||
|
with patch.object(dsmr_ingest, "get_session_local", return_value=SessionLocal):
|
||||||
|
assert dsmr_ingest.get_current_tariff() == 1
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("replacement", ["disable", "delete", "config-change"])
|
||||||
|
def test_retained_source_handler_cannot_write_after_reconcile(dsmr_db, monkeypatch, replacement):
|
||||||
|
"""A callback fetched before disable/delete/reconfigure is rejected before DB access."""
|
||||||
|
from app.services import dsmr_ingest
|
||||||
|
from app.services.dsmr_ingest import DsmrSourceSnapshot
|
||||||
|
|
||||||
|
_, SessionLocal = dsmr_db
|
||||||
|
old = DsmrSourceSnapshot(1, "old", "", 10, broker_host="one.test")
|
||||||
|
if replacement in {"disable", "delete"}:
|
||||||
|
active = {}
|
||||||
|
else:
|
||||||
|
active = {1: DsmrSourceSnapshot(1, "new", "", 10, broker_host="changed.test")}
|
||||||
|
monkeypatch.setattr(dsmr_ingest, "_subscriptions", active)
|
||||||
|
with patch.object(dsmr_ingest, "get_session_local", return_value=SessionLocal):
|
||||||
|
dsmr_ingest.handle_captured_message(_payload(_SAMPLE_TELEGRAM), old)
|
||||||
|
assert _count_readings(SessionLocal) == 0
|
||||||
|
|||||||
@@ -0,0 +1,290 @@
|
|||||||
|
"""Isolated revision-14/15 fixtures for the DSMR source-adoption migration."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
from datetime import datetime, timedelta, timezone
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
import sqlalchemy.exc
|
||||||
|
from alembic import command
|
||||||
|
from alembic.config import Config
|
||||||
|
from sqlalchemy import create_engine, event, inspect, text
|
||||||
|
|
||||||
|
from app.integrations.meter_sources import sanitize_source_config, validate_source_config
|
||||||
|
|
||||||
|
|
||||||
|
def _config(database_url: str) -> Config:
|
||||||
|
config = Config("alembic_app.ini")
|
||||||
|
config.set_main_option("sqlalchemy.url", database_url)
|
||||||
|
return config
|
||||||
|
|
||||||
|
|
||||||
|
def _engine(database_url: str):
|
||||||
|
engine = create_engine(database_url, connect_args={"check_same_thread": False})
|
||||||
|
|
||||||
|
@event.listens_for(engine, "connect")
|
||||||
|
def _foreign_keys(connection, _record) -> None:
|
||||||
|
connection.execute("PRAGMA foreign_keys=ON")
|
||||||
|
|
||||||
|
return engine
|
||||||
|
|
||||||
|
|
||||||
|
def _insert_meter(connection, label: str, started: datetime, ended: datetime | None) -> int:
|
||||||
|
connection.execute(
|
||||||
|
text(
|
||||||
|
"INSERT INTO meter (uuid, label, commodity, started_at, ended_at, reason, note, created_at) "
|
||||||
|
"VALUES (:uuid, :label, 'electricity', :started, :ended, 'initial', NULL, :started)"
|
||||||
|
),
|
||||||
|
{"uuid": f"{label:0<8}-0000-4000-8000-000000000000", "label": label,
|
||||||
|
"started": started, "ended": ended},
|
||||||
|
)
|
||||||
|
return int(connection.execute(text("SELECT last_insert_rowid()")).scalar_one())
|
||||||
|
|
||||||
|
|
||||||
|
def _insert_contract_version(connection, start: datetime) -> int:
|
||||||
|
connection.execute(
|
||||||
|
text(
|
||||||
|
"INSERT INTO energy_contract (name, kind, active, currency, created_at, updated_at) "
|
||||||
|
"VALUES ('Historic contract', 'manual', 1, 'EUR', :at, :at)"
|
||||||
|
),
|
||||||
|
{"at": start},
|
||||||
|
)
|
||||||
|
contract_id = int(connection.execute(text("SELECT last_insert_rowid()")).scalar_one())
|
||||||
|
connection.execute(
|
||||||
|
text(
|
||||||
|
"INSERT INTO energy_contract_version "
|
||||||
|
"(contract_id, effective_from, effective_to, \"values\", created_at) "
|
||||||
|
"VALUES (:contract_id, :at, NULL, :values, :at)"
|
||||||
|
),
|
||||||
|
{"contract_id": contract_id, "at": start, "values": json.dumps({"historic": True})},
|
||||||
|
)
|
||||||
|
return int(connection.execute(text("SELECT last_insert_rowid()")).scalar_one())
|
||||||
|
|
||||||
|
|
||||||
|
def _insert_cost(
|
||||||
|
connection,
|
||||||
|
period_start: datetime,
|
||||||
|
meter_id: int,
|
||||||
|
contract_version_id: int,
|
||||||
|
sequence: int,
|
||||||
|
) -> None:
|
||||||
|
amount = 2.5 + sequence
|
||||||
|
connection.execute(
|
||||||
|
text(
|
||||||
|
"INSERT INTO energy_cost_period "
|
||||||
|
"(period_start, d1_kwh, d2_kwh, r1_kwh, r2_kwh, import_cost, export_revenue, net_cost, "
|
||||||
|
"currency, pricing, contract_version_id, meter_id, degraded, computed_at) "
|
||||||
|
"VALUES (:start, :d1, :d2, :r1, :r2, :import_cost, :export_revenue, :net_cost, "
|
||||||
|
"'EUR', :pricing, :contract_version_id, :meter, 0, :computed_at)"
|
||||||
|
),
|
||||||
|
{
|
||||||
|
"start": period_start,
|
||||||
|
"d1": 1.0 + sequence,
|
||||||
|
"d2": 2.0 + sequence,
|
||||||
|
"r1": 3.0 + sequence,
|
||||||
|
"r2": 4.0 + sequence,
|
||||||
|
"import_cost": amount,
|
||||||
|
"export_revenue": 0.25 + sequence,
|
||||||
|
"net_cost": amount - (0.25 + sequence),
|
||||||
|
"pricing": json.dumps({"historic": True, "sequence": sequence}),
|
||||||
|
"contract_version_id": contract_version_id,
|
||||||
|
"meter": meter_id,
|
||||||
|
"computed_at": period_start + timedelta(seconds=sequence),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_populated_revision_14_adopts_dsmr_history_at_revision_16(tmp_path: Path):
|
||||||
|
database_url = f"sqlite:///{tmp_path / 'revision_14_history.db'}"
|
||||||
|
config = _config(database_url)
|
||||||
|
command.upgrade(config, "20260625_14_meter_uuid")
|
||||||
|
start = datetime(2026, 8, 1, tzinfo=timezone.utc)
|
||||||
|
engine = _engine(database_url)
|
||||||
|
try:
|
||||||
|
with engine.begin() as connection:
|
||||||
|
connection.execute(
|
||||||
|
text("INSERT INTO app_config (key, value, updated_at) VALUES (:key, :value, :at)"),
|
||||||
|
[
|
||||||
|
{"key": "DSMR_INGEST_ENABLED", "value": "true", "at": start},
|
||||||
|
{"key": "DSMR_MQTT_TOPIC", "value": "historic/dsmr", "at": start},
|
||||||
|
{"key": "DSMR_TARIFF_TOPIC", "value": "historic/tariff", "at": start},
|
||||||
|
{"key": "DSMR_SAMPLE_INTERVAL_S", "value": "0", "at": start},
|
||||||
|
{"key": "MQTT_BROKER_HOST", "value": "mqtt.example.invalid", "at": start},
|
||||||
|
{"key": "MQTT_BROKER_PORT", "value": "1884", "at": start},
|
||||||
|
{"key": "MQTT_USERNAME", "value": "historic-user", "at": start},
|
||||||
|
{"key": "MQTT_PASSWORD", "value": "historic-password", "at": start},
|
||||||
|
{"key": "MQTT_TLS_ENABLED", "value": "true", "at": start},
|
||||||
|
{"key": "UNRELATED_CONFIG", "value": "untouched", "at": start},
|
||||||
|
],
|
||||||
|
)
|
||||||
|
config_before = dict(connection.execute(text("SELECT key, value FROM app_config")).all())
|
||||||
|
for offset, telegram_id in ((0, 77), (20, 78), (40, 77)):
|
||||||
|
connection.execute(
|
||||||
|
text("INSERT INTO dsmr_reading (recorded_at, source_id, payload) VALUES (:at, :id, :payload)"),
|
||||||
|
{"at": start + timedelta(minutes=offset), "id": telegram_id,
|
||||||
|
"payload": json.dumps({"id": telegram_id, "keep": f"payload-{offset}"})},
|
||||||
|
)
|
||||||
|
first = _insert_meter(connection, "meterone", start - timedelta(hours=1), start + timedelta(minutes=20))
|
||||||
|
second = _insert_meter(connection, "metertwo", start + timedelta(minutes=20), start + timedelta(minutes=40))
|
||||||
|
third = _insert_meter(connection, "meterthree", start + timedelta(minutes=40), None)
|
||||||
|
_insert_meter(connection, "nodata", start + timedelta(days=1), None)
|
||||||
|
contract_version = _insert_contract_version(connection, start - timedelta(days=1))
|
||||||
|
# One normal period per epoch plus a period ending exactly at each
|
||||||
|
# replacement boundary. Meter/binding intervals are half-open, so
|
||||||
|
# the latter must remain unbound/degraded.
|
||||||
|
_insert_cost(connection, start, first, contract_version, 0)
|
||||||
|
_insert_cost(connection, start + timedelta(minutes=5), first, contract_version, 1)
|
||||||
|
_insert_cost(connection, start + timedelta(minutes=21), second, contract_version, 2)
|
||||||
|
_insert_cost(connection, start + timedelta(minutes=25), second, contract_version, 3)
|
||||||
|
_insert_cost(connection, start + timedelta(minutes=41), third, contract_version, 4)
|
||||||
|
cost_before = connection.execute(
|
||||||
|
text(
|
||||||
|
"SELECT id, period_start, d1_kwh, d2_kwh, r1_kwh, r2_kwh, import_cost, "
|
||||||
|
"export_revenue, net_cost, currency, pricing, contract_version_id, meter_id, "
|
||||||
|
"degraded, computed_at FROM energy_cost_period ORDER BY period_start"
|
||||||
|
)
|
||||||
|
).mappings().all()
|
||||||
|
finally:
|
||||||
|
engine.dispose()
|
||||||
|
|
||||||
|
command.upgrade(config, "20260822_16_dsmr_source_adoption")
|
||||||
|
engine = _engine(database_url)
|
||||||
|
try:
|
||||||
|
with engine.connect() as connection:
|
||||||
|
assert connection.execute(text("SELECT version_num FROM alembic_version")).scalar_one() == (
|
||||||
|
"20260822_16_dsmr_source_adoption"
|
||||||
|
)
|
||||||
|
assert connection.execute(text("SELECT COUNT(*) FROM dsmr_reading")).scalar_one() == 3
|
||||||
|
source = connection.execute(
|
||||||
|
text("SELECT id, enabled, config FROM meter_source WHERE kind = 'dsmr_mqtt'")
|
||||||
|
).one()
|
||||||
|
assert source.enabled == 1
|
||||||
|
source_config = json.loads(source.config)
|
||||||
|
assert source_config == {
|
||||||
|
"broker_host": "mqtt.example.invalid", "broker_port": 1884,
|
||||||
|
"username": "historic-user", "password": "historic-password", "tls_enabled": True,
|
||||||
|
"topic": "historic/dsmr", "tariff_topic": "historic/tariff", "sample_interval_s": 0,
|
||||||
|
}
|
||||||
|
assert validate_source_config("dsmr_mqtt", source_config) == source_config
|
||||||
|
assert sanitize_source_config("dsmr_mqtt", source_config)["sample_interval_s"] == 0
|
||||||
|
assert connection.execute(text("SELECT value FROM app_config WHERE key = 'DSMR_MQTT_TOPIC'")).scalar_one() == "historic/dsmr"
|
||||||
|
assert dict(connection.execute(text("SELECT key, value FROM app_config")).all()) == config_before
|
||||||
|
assert connection.execute(text("SELECT group_concat(telegram_id) FROM dsmr_reading")).scalar_one() == "77,78,77"
|
||||||
|
assert connection.execute(text("SELECT payload FROM dsmr_reading ORDER BY recorded_at")).scalars().all() == [
|
||||||
|
json.dumps({"id": 77, "keep": "payload-0"}),
|
||||||
|
json.dumps({"id": 78, "keep": "payload-20"}),
|
||||||
|
json.dumps({"id": 77, "keep": "payload-40"}),
|
||||||
|
]
|
||||||
|
assert connection.execute(text("SELECT COUNT(*) FROM meter_source_binding")).scalar_one() == 3
|
||||||
|
assert connection.execute(
|
||||||
|
text("SELECT COUNT(*) FROM meter_source_binding WHERE meter_id = (SELECT id FROM meter WHERE label = 'nodata')")
|
||||||
|
).scalar_one() == 0
|
||||||
|
periods = connection.execute(
|
||||||
|
text(
|
||||||
|
"SELECT id, period_start, d1_kwh, d2_kwh, r1_kwh, r2_kwh, import_cost, "
|
||||||
|
"export_revenue, net_cost, currency, pricing, contract_version_id, meter_id, "
|
||||||
|
"degraded, computed_at, source_binding_id FROM energy_cost_period ORDER BY period_start"
|
||||||
|
)
|
||||||
|
).mappings().all()
|
||||||
|
assert [
|
||||||
|
{key: value for key, value in period.items() if key not in {"degraded", "source_binding_id"}}
|
||||||
|
for period in periods
|
||||||
|
] == [
|
||||||
|
{key: value for key, value in period.items() if key != "degraded"}
|
||||||
|
for period in cost_before
|
||||||
|
]
|
||||||
|
assert [(period["degraded"], period["source_binding_id"] is not None) for period in periods] == [
|
||||||
|
(0, True), (1, False), (0, True), (1, False), (0, True)
|
||||||
|
]
|
||||||
|
assert [period["period_start"] for period in periods if period["source_binding_id"] is None] == [
|
||||||
|
(start + timedelta(minutes=5)).isoformat(sep=" "),
|
||||||
|
(start + timedelta(minutes=25)).isoformat(sep=" "),
|
||||||
|
]
|
||||||
|
bound_meter_ids = connection.execute(
|
||||||
|
text(
|
||||||
|
"SELECT binding.meter_id FROM energy_cost_period AS period "
|
||||||
|
"LEFT JOIN meter_source_binding AS binding "
|
||||||
|
"ON binding.id = period.source_binding_id ORDER BY period.period_start"
|
||||||
|
)
|
||||||
|
).scalars().all()
|
||||||
|
assert bound_meter_ids == [first, None, second, None, third]
|
||||||
|
assert connection.execute(text("PRAGMA foreign_key_check")).all() == []
|
||||||
|
finally:
|
||||||
|
engine.dispose()
|
||||||
|
|
||||||
|
|
||||||
|
def test_dsmr_source_timestamp_uniqueness_allows_two_sources(tmp_path: Path):
|
||||||
|
database_url = f"sqlite:///{tmp_path / 'two_sources.db'}"
|
||||||
|
config = _config(database_url)
|
||||||
|
command.upgrade(config, "20260822_16_dsmr_source_adoption")
|
||||||
|
engine = _engine(database_url)
|
||||||
|
timestamp = datetime(2026, 8, 1, tzinfo=timezone.utc)
|
||||||
|
try:
|
||||||
|
with engine.begin() as connection:
|
||||||
|
first_source = connection.execute(text("SELECT id FROM meter_source WHERE kind = 'dsmr_mqtt'")).scalar_one()
|
||||||
|
assert connection.execute(
|
||||||
|
text("SELECT enabled FROM meter_source WHERE id = :id"), {"id": first_source}
|
||||||
|
).scalar_one() == 0
|
||||||
|
connection.execute(
|
||||||
|
text(
|
||||||
|
"INSERT INTO meter_source (uuid, name, kind, enabled, config, status, created_at, updated_at) "
|
||||||
|
"VALUES ('22222222-2222-4222-8222-222222222222', 'Second DSMR', 'dsmr_mqtt', 0, '{}', "
|
||||||
|
"'unknown', :at, :at)"
|
||||||
|
), {"at": timestamp},
|
||||||
|
)
|
||||||
|
second_source = connection.execute(text("SELECT last_insert_rowid()")).scalar_one()
|
||||||
|
for source_id in (first_source, second_source):
|
||||||
|
connection.execute(
|
||||||
|
text(
|
||||||
|
"INSERT INTO dsmr_reading (recorded_at, telegram_id, meter_source_id, payload) "
|
||||||
|
"VALUES (:at, 9, :source, '{}')"
|
||||||
|
), {"at": timestamp, "source": source_id},
|
||||||
|
)
|
||||||
|
with pytest.raises(sqlalchemy.exc.IntegrityError):
|
||||||
|
connection.execute(
|
||||||
|
text(
|
||||||
|
"INSERT INTO dsmr_reading (recorded_at, telegram_id, meter_source_id, payload) "
|
||||||
|
"VALUES (:at, 10, :source, '{}')"
|
||||||
|
), {"at": timestamp, "source": first_source},
|
||||||
|
)
|
||||||
|
finally:
|
||||||
|
engine.dispose()
|
||||||
|
inspector = inspect(create_engine(database_url))
|
||||||
|
assert ("meter_source_id", "recorded_at") in {
|
||||||
|
tuple(item["column_names"]) for item in inspector.get_unique_constraints("dsmr_reading")
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def test_revision_15_without_legacy_dsmr_config_creates_unconfigured_source(tmp_path: Path):
|
||||||
|
database_url = f"sqlite:///{tmp_path / 'revision_15_no_dsmr_config.db'}"
|
||||||
|
config = _config(database_url)
|
||||||
|
command.upgrade(config, "20260822_15_meter_sources")
|
||||||
|
engine = _engine(database_url)
|
||||||
|
try:
|
||||||
|
with engine.begin() as connection:
|
||||||
|
connection.execute(
|
||||||
|
text(
|
||||||
|
"INSERT INTO app_config (key, value, updated_at) "
|
||||||
|
"VALUES ('UNRELATED_CONFIG', 'untouched', :at)"
|
||||||
|
),
|
||||||
|
{"at": datetime(2026, 8, 1, tzinfo=timezone.utc)},
|
||||||
|
)
|
||||||
|
finally:
|
||||||
|
engine.dispose()
|
||||||
|
|
||||||
|
command.upgrade(config, "20260822_16_dsmr_source_adoption")
|
||||||
|
engine = _engine(database_url)
|
||||||
|
try:
|
||||||
|
with engine.connect() as connection:
|
||||||
|
source = connection.execute(
|
||||||
|
text("SELECT enabled, config FROM meter_source WHERE kind = 'dsmr_mqtt'")
|
||||||
|
).one()
|
||||||
|
assert source.enabled == 0
|
||||||
|
assert json.loads(source.config) == {}
|
||||||
|
assert dict(connection.execute(text("SELECT key, value FROM app_config")).all()) == {
|
||||||
|
"UNRELATED_CONFIG": "untouched"
|
||||||
|
}
|
||||||
|
finally:
|
||||||
|
engine.dispose()
|
||||||
@@ -1,173 +1,203 @@
|
|||||||
"""Tests for restart-free DSMR subscription management (apply_dsmr_subscription).
|
"""DSMR source-driven MQTT subscription reconciliation tests."""
|
||||||
|
|
||||||
These verify that toggling DSMR ingest / changing its topic / changing its sample
|
|
||||||
interval via the config UI is reflected in the live MQTT subscription without an
|
|
||||||
app restart. A fake MQTT manager is injected so no real broker is touched.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
from unittest.mock import MagicMock
|
from dataclasses import replace
|
||||||
|
from types import SimpleNamespace
|
||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
|
|
||||||
from app.services import dsmr_ingest
|
from app.services import dsmr_ingest
|
||||||
|
from app.services.dsmr_ingest import DsmrSourceSnapshot
|
||||||
|
|
||||||
|
|
||||||
class _FakeMqtt:
|
class _FakeMqtt:
|
||||||
def __init__(self) -> None:
|
def __init__(self) -> None:
|
||||||
self.subscribe_calls: list[tuple[str, object]] = []
|
self.replace_calls: list[tuple[int, dict[str, object]]] = []
|
||||||
self.unsubscribe_calls: list[str] = []
|
self.remove_calls: list[int] = []
|
||||||
|
self.handlers: dict[int, dict[str, object]] = {}
|
||||||
|
|
||||||
def subscribe(self, topic: str, handler) -> None:
|
def replace_source(self, source_id: int, **kwargs: object) -> bool:
|
||||||
self.subscribe_calls.append((topic, handler))
|
self.replace_calls.append((source_id, kwargs))
|
||||||
|
self.handlers[source_id] = kwargs["subscriptions"] # type: ignore[assignment]
|
||||||
|
return True
|
||||||
|
|
||||||
def unsubscribe(self, topic: str) -> None:
|
def remove_source(self, source_id: int) -> None:
|
||||||
self.unsubscribe_calls.append(topic)
|
self.remove_calls.append(source_id)
|
||||||
|
self.handlers.pop(source_id, None)
|
||||||
|
|
||||||
|
def source_is_active(self, source_id: int) -> bool:
|
||||||
|
return source_id in self.handlers
|
||||||
|
|
||||||
|
|
||||||
def _settings(
|
def _source(
|
||||||
*,
|
source_id: int,
|
||||||
enabled: bool = True,
|
topic: str,
|
||||||
topic: str = "dsmr/json",
|
|
||||||
interval: int = 10,
|
|
||||||
tariff_topic: str = "",
|
tariff_topic: str = "",
|
||||||
):
|
interval: int = 10,
|
||||||
s = MagicMock()
|
**connection: object,
|
||||||
s.dsmr_ingest_enabled = enabled
|
) -> DsmrSourceSnapshot:
|
||||||
s.dsmr_mqtt_topic = topic
|
return DsmrSourceSnapshot(source_id, topic, tariff_topic, interval, **connection)
|
||||||
s.dsmr_sample_interval_s = interval
|
|
||||||
s.dsmr_tariff_topic = tariff_topic
|
|
||||||
return s
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture()
|
@pytest.fixture()
|
||||||
def fake_mqtt(monkeypatch):
|
def fake_mqtt(monkeypatch):
|
||||||
fake = _FakeMqtt()
|
fake = _FakeMqtt()
|
||||||
# apply_dsmr_subscription does `from app.integrations.mqtt import mqtt_manager`
|
|
||||||
# at call time, so patching the module attribute is picked up.
|
|
||||||
monkeypatch.setattr("app.integrations.mqtt.mqtt_manager", fake)
|
monkeypatch.setattr("app.integrations.mqtt.mqtt_manager", fake)
|
||||||
# Reset (and auto-restore) the module-level "currently subscribed topics".
|
monkeypatch.setattr(dsmr_ingest, "_subscriptions", {})
|
||||||
monkeypatch.setattr(dsmr_ingest, "_current_dsmr_topic", None)
|
monkeypatch.setattr(dsmr_ingest, "_subscription_client_ids", {})
|
||||||
monkeypatch.setattr(dsmr_ingest, "_current_tariff_topic", None)
|
monkeypatch.setattr(dsmr_ingest, "_subscription_tokens", {})
|
||||||
|
monkeypatch.setattr(dsmr_ingest, "_tariffs", {})
|
||||||
return fake
|
return fake
|
||||||
|
|
||||||
|
|
||||||
def test_enabled_subscribes_to_topic(fake_mqtt):
|
def test_reconcile_subscribes_each_enabled_source(fake_mqtt, monkeypatch):
|
||||||
dsmr_ingest.apply_dsmr_subscription(_settings(enabled=True, topic="dsmr/json"))
|
monkeypatch.setattr(dsmr_ingest, "_enabled_snapshots", lambda: [_source(1, "a"), _source(2, "b")])
|
||||||
|
dsmr_ingest.apply_dsmr_subscription()
|
||||||
assert len(fake_mqtt.subscribe_calls) == 1
|
assert [source_id for source_id, _ in fake_mqtt.replace_calls] == [1, 2]
|
||||||
assert fake_mqtt.subscribe_calls[0][0] == "dsmr/json"
|
|
||||||
assert fake_mqtt.unsubscribe_calls == []
|
|
||||||
assert dsmr_ingest._current_dsmr_topic == "dsmr/json"
|
|
||||||
|
|
||||||
|
|
||||||
def test_disabled_after_enabled_unsubscribes(fake_mqtt):
|
def test_changed_source_replaces_only_its_subscription(fake_mqtt, monkeypatch):
|
||||||
dsmr_ingest.apply_dsmr_subscription(_settings(enabled=True, topic="dsmr/json"))
|
monkeypatch.setattr(dsmr_ingest, "_enabled_snapshots", lambda: [_source(1, "a"), _source(2, "b")])
|
||||||
dsmr_ingest.apply_dsmr_subscription(_settings(enabled=False))
|
dsmr_ingest.apply_dsmr_subscription()
|
||||||
|
monkeypatch.setattr(dsmr_ingest, "_enabled_snapshots", lambda: [_source(1, "changed"), _source(2, "b")])
|
||||||
assert fake_mqtt.unsubscribe_calls == ["dsmr/json"]
|
dsmr_ingest.apply_dsmr_subscription()
|
||||||
assert dsmr_ingest._current_dsmr_topic is None
|
assert fake_mqtt.remove_calls == [1]
|
||||||
|
assert [source_id for source_id, _ in fake_mqtt.replace_calls] == [1, 2, 1]
|
||||||
|
|
||||||
|
|
||||||
def test_topic_change_unsubscribes_old_subscribes_new(fake_mqtt):
|
def test_disable_unsubscribes_and_clears_source_tariff(fake_mqtt, monkeypatch):
|
||||||
dsmr_ingest.apply_dsmr_subscription(_settings(topic="dsmr/json"))
|
monkeypatch.setattr(dsmr_ingest, "_enabled_snapshots", lambda: [_source(1, "a", "tariff/a")])
|
||||||
dsmr_ingest.apply_dsmr_subscription(_settings(topic="meter/dsmr"))
|
dsmr_ingest.apply_dsmr_subscription()
|
||||||
|
dsmr_ingest.set_current_tariff(1, 2)
|
||||||
assert fake_mqtt.unsubscribe_calls == ["dsmr/json"]
|
monkeypatch.setattr(dsmr_ingest, "_enabled_snapshots", lambda: [])
|
||||||
assert [t for t, _ in fake_mqtt.subscribe_calls] == ["dsmr/json", "meter/dsmr"]
|
dsmr_ingest.apply_dsmr_subscription()
|
||||||
assert dsmr_ingest._current_dsmr_topic == "meter/dsmr"
|
assert fake_mqtt.remove_calls == [1]
|
||||||
|
assert dsmr_ingest.get_current_tariff(1) is None
|
||||||
|
|
||||||
|
|
||||||
def test_reapply_same_topic_resubscribes_fresh_handler(fake_mqtt):
|
def test_same_snapshot_has_no_subscription_churn(fake_mqtt, monkeypatch):
|
||||||
# A changed sample interval must take effect — the handler is re-bound to a
|
monkeypatch.setattr(dsmr_ingest, "_enabled_snapshots", lambda: [_source(1, "a")])
|
||||||
# fresh settings snapshot, so re-applying the same topic re-subscribes.
|
dsmr_ingest.apply_dsmr_subscription()
|
||||||
dsmr_ingest.apply_dsmr_subscription(_settings(topic="dsmr/json", interval=10))
|
dsmr_ingest.apply_dsmr_subscription()
|
||||||
dsmr_ingest.apply_dsmr_subscription(_settings(topic="dsmr/json", interval=20))
|
assert len(fake_mqtt.replace_calls) == 1
|
||||||
|
assert fake_mqtt.remove_calls == []
|
||||||
assert len(fake_mqtt.subscribe_calls) == 2
|
|
||||||
assert fake_mqtt.unsubscribe_calls == [] # same topic, no churn
|
|
||||||
handler1 = fake_mqtt.subscribe_calls[0][1]
|
|
||||||
handler2 = fake_mqtt.subscribe_calls[1][1]
|
|
||||||
assert handler1 is not handler2 # fresh closure carrying the new settings
|
|
||||||
|
|
||||||
|
|
||||||
def test_disabled_when_never_enabled_is_noop(fake_mqtt):
|
def test_changed_base_client_id_replaces_all_enabled_sources(fake_mqtt, monkeypatch):
|
||||||
dsmr_ingest.apply_dsmr_subscription(_settings(enabled=False))
|
snapshot = _source(1, "topic", broker_host="broker.test")
|
||||||
|
monkeypatch.setattr(dsmr_ingest, "_enabled_snapshots", lambda: [snapshot])
|
||||||
|
|
||||||
assert fake_mqtt.subscribe_calls == []
|
dsmr_ingest.apply_dsmr_subscription(SimpleNamespace(mqtt_client_id="home-automation"))
|
||||||
assert fake_mqtt.unsubscribe_calls == []
|
dsmr_ingest.apply_dsmr_subscription(SimpleNamespace(mqtt_client_id="home-automation-dev"))
|
||||||
assert dsmr_ingest._current_dsmr_topic is None
|
|
||||||
|
assert fake_mqtt.remove_calls == [1]
|
||||||
|
assert [source_id for source_id, _ in fake_mqtt.replace_calls] == [1, 1]
|
||||||
|
assert fake_mqtt.replace_calls[-1][1]["base_client_id"] == "home-automation-dev"
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
def test_same_topic_on_different_brokers_is_allowed(fake_mqtt, monkeypatch):
|
||||||
# Tariff topic subscription management
|
monkeypatch.setattr(
|
||||||
# ---------------------------------------------------------------------------
|
dsmr_ingest,
|
||||||
|
"_enabled_snapshots",
|
||||||
|
lambda: [_source(1, "same", broker_host="one.test"), _source(2, "same", broker_host="two.test")],
|
||||||
def test_enabled_with_tariff_topic_subscribes_both(fake_mqtt):
|
|
||||||
"""When enabled and tariff_topic is non-empty, both topics must be subscribed."""
|
|
||||||
dsmr_ingest.apply_dsmr_subscription(
|
|
||||||
_settings(enabled=True, topic="dsmr/json", tariff_topic="dsmr/meter-stats/electricity_tariff")
|
|
||||||
)
|
)
|
||||||
|
dsmr_ingest.apply_dsmr_subscription()
|
||||||
subscribed_topics = [t for t, _ in fake_mqtt.subscribe_calls]
|
assert [source_id for source_id, _ in fake_mqtt.replace_calls] == [1, 2]
|
||||||
assert "dsmr/json" in subscribed_topics
|
|
||||||
assert "dsmr/meter-stats/electricity_tariff" in subscribed_topics
|
|
||||||
assert len(fake_mqtt.subscribe_calls) == 2
|
|
||||||
assert dsmr_ingest._current_tariff_topic == "dsmr/meter-stats/electricity_tariff"
|
|
||||||
|
|
||||||
|
|
||||||
def test_enabled_with_empty_tariff_topic_subscribes_only_main(fake_mqtt):
|
@pytest.mark.parametrize(
|
||||||
"""When tariff_topic is empty, only the main DSMR topic is subscribed."""
|
"sources",
|
||||||
dsmr_ingest.apply_dsmr_subscription(
|
[
|
||||||
_settings(enabled=True, topic="dsmr/json", tariff_topic="")
|
[_source(1, "shared", "shared")],
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_duplicate_telegram_or_tariff_topic_is_rejected(fake_mqtt, monkeypatch, sources):
|
||||||
|
"""One MQTT topic cannot safely dispatch to more than one source handler."""
|
||||||
|
monkeypatch.setattr(dsmr_ingest, "_enabled_snapshots", lambda: sources)
|
||||||
|
dsmr_ingest.apply_dsmr_subscription()
|
||||||
|
assert fake_mqtt.replace_calls == []
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
"field,value",
|
||||||
|
[
|
||||||
|
("broker_host", "changed.test"),
|
||||||
|
("broker_port", 2883),
|
||||||
|
("username", "different-user"),
|
||||||
|
("password", "different-password"),
|
||||||
|
("tls_enabled", True),
|
||||||
|
("sample_interval_s", 30),
|
||||||
|
("tariff_topic", "tariff/changed"),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_each_source_config_change_replaces_only_that_source(fake_mqtt, monkeypatch, field, value):
|
||||||
|
first = _source(1, "a", "tariff/a", broker_host="one.test", username="one")
|
||||||
|
second = _source(2, "b", "tariff/b", broker_host="two.test", username="two")
|
||||||
|
monkeypatch.setattr(dsmr_ingest, "_enabled_snapshots", lambda: [first, second])
|
||||||
|
dsmr_ingest.apply_dsmr_subscription()
|
||||||
|
changed = _source(1, "a", "tariff/a", broker_host="one.test", username="one")
|
||||||
|
changed = replace(changed, **{field: value})
|
||||||
|
monkeypatch.setattr(dsmr_ingest, "_enabled_snapshots", lambda: [changed, second])
|
||||||
|
dsmr_ingest.apply_dsmr_subscription()
|
||||||
|
assert fake_mqtt.remove_calls == [1]
|
||||||
|
assert [source_id for source_id, _ in fake_mqtt.replace_calls] == [1, 2, 1]
|
||||||
|
|
||||||
|
|
||||||
|
def test_failed_replace_is_not_marked_applied_and_is_retried(fake_mqtt, monkeypatch):
|
||||||
|
snapshot = _source(1, "a", broker_host="one.test")
|
||||||
|
monkeypatch.setattr(dsmr_ingest, "_enabled_snapshots", lambda: [snapshot])
|
||||||
|
original_replace = fake_mqtt.replace_source
|
||||||
|
outcomes = iter([False, True])
|
||||||
|
|
||||||
|
def replace_once_fails(source_id: int, **kwargs: object) -> bool:
|
||||||
|
original_replace(source_id, **kwargs)
|
||||||
|
return next(outcomes)
|
||||||
|
|
||||||
|
fake_mqtt.replace_source = replace_once_fails # type: ignore[method-assign]
|
||||||
|
dsmr_ingest.apply_dsmr_subscription()
|
||||||
|
dsmr_ingest.apply_dsmr_subscription()
|
||||||
|
assert [source_id for source_id, _ in fake_mqtt.replace_calls] == [1, 1]
|
||||||
|
|
||||||
|
|
||||||
|
def test_inactive_manager_source_is_rebuilt_on_next_reconcile(fake_mqtt, monkeypatch):
|
||||||
|
snapshot = _source(1, "a", broker_host="one.test")
|
||||||
|
monkeypatch.setattr(dsmr_ingest, "_enabled_snapshots", lambda: [snapshot])
|
||||||
|
dsmr_ingest.apply_dsmr_subscription()
|
||||||
|
fake_mqtt.handlers.clear() # models MqttManager.disconnect() tearing down clients
|
||||||
|
dsmr_ingest.apply_dsmr_subscription()
|
||||||
|
assert [source_id for source_id, _ in fake_mqtt.replace_calls] == [1, 1]
|
||||||
|
|
||||||
|
|
||||||
|
def test_same_snapshot_aba_rejects_retained_handler(fake_mqtt, monkeypatch):
|
||||||
|
"""An equal re-enabled snapshot has a fresh callback identity token."""
|
||||||
|
snapshot = _source(1, "same", broker_host="one.test")
|
||||||
|
received: list[tuple[bytes, DsmrSourceSnapshot]] = []
|
||||||
|
monkeypatch.setattr(dsmr_ingest, "_enabled_snapshots", lambda: [snapshot])
|
||||||
|
monkeypatch.setattr(
|
||||||
|
dsmr_ingest, "handle_message", lambda payload, captured: received.append((payload, captured))
|
||||||
)
|
)
|
||||||
|
dsmr_ingest.apply_dsmr_subscription()
|
||||||
|
old_handler = fake_mqtt.handlers[1]["same"]
|
||||||
|
monkeypatch.setattr(dsmr_ingest, "_enabled_snapshots", lambda: [])
|
||||||
|
dsmr_ingest.apply_dsmr_subscription()
|
||||||
|
monkeypatch.setattr(dsmr_ingest, "_enabled_snapshots", lambda: [snapshot])
|
||||||
|
dsmr_ingest.apply_dsmr_subscription()
|
||||||
|
|
||||||
assert len(fake_mqtt.subscribe_calls) == 1
|
old_handler(b"stale") # type: ignore[operator]
|
||||||
assert fake_mqtt.subscribe_calls[0][0] == "dsmr/json"
|
fake_mqtt.handlers[1]["same"](b"fresh") # type: ignore[operator]
|
||||||
assert dsmr_ingest._current_tariff_topic is None
|
assert received == [(b"fresh", snapshot)]
|
||||||
|
|
||||||
|
|
||||||
def test_disabled_after_tariff_subscription_unsubscribes_both(fake_mqtt):
|
def test_source_state_handler_is_generation_scoped(fake_mqtt, monkeypatch):
|
||||||
"""Disabling ingest must also unsubscribe the tariff topic."""
|
snapshot = _source(1, "same", broker_host="one.test")
|
||||||
dsmr_ingest.apply_dsmr_subscription(
|
monkeypatch.setattr(dsmr_ingest, "_enabled_snapshots", lambda: [snapshot])
|
||||||
_settings(enabled=True, topic="dsmr/json", tariff_topic="dsmr/meter-stats/electricity_tariff")
|
dsmr_ingest.apply_dsmr_subscription()
|
||||||
)
|
stale = fake_mqtt.replace_calls[-1][1]["state_handler"]
|
||||||
dsmr_ingest.apply_dsmr_subscription(_settings(enabled=False))
|
monkeypatch.setattr(dsmr_ingest, "_enabled_snapshots", lambda: [])
|
||||||
|
dsmr_ingest.apply_dsmr_subscription()
|
||||||
|
monkeypatch.setattr(dsmr_ingest, "_enabled_snapshots", lambda: [snapshot])
|
||||||
|
dsmr_ingest.apply_dsmr_subscription()
|
||||||
|
|
||||||
assert "dsmr/json" in fake_mqtt.unsubscribe_calls
|
monkeypatch.setattr(dsmr_ingest, "get_session_local", lambda: (_ for _ in ()).throw(AssertionError()))
|
||||||
assert "dsmr/meter-stats/electricity_tariff" in fake_mqtt.unsubscribe_calls
|
stale("error") # type: ignore[operator]
|
||||||
assert dsmr_ingest._current_dsmr_topic is None
|
|
||||||
assert dsmr_ingest._current_tariff_topic is None
|
|
||||||
|
|
||||||
|
|
||||||
def test_tariff_topic_change_resubscribes(fake_mqtt):
|
|
||||||
"""Changing the tariff topic must unsubscribe the old one and subscribe the new one."""
|
|
||||||
old_tariff = "dsmr/meter-stats/electricity_tariff"
|
|
||||||
new_tariff = "meter/tariff"
|
|
||||||
|
|
||||||
dsmr_ingest.apply_dsmr_subscription(
|
|
||||||
_settings(enabled=True, topic="dsmr/json", tariff_topic=old_tariff)
|
|
||||||
)
|
|
||||||
dsmr_ingest.apply_dsmr_subscription(
|
|
||||||
_settings(enabled=True, topic="dsmr/json", tariff_topic=new_tariff)
|
|
||||||
)
|
|
||||||
|
|
||||||
assert old_tariff in fake_mqtt.unsubscribe_calls
|
|
||||||
subscribed_topics = [t for t, _ in fake_mqtt.subscribe_calls]
|
|
||||||
assert new_tariff in subscribed_topics
|
|
||||||
assert dsmr_ingest._current_tariff_topic == new_tariff
|
|
||||||
|
|
||||||
|
|
||||||
def test_tariff_topic_cleared_unsubscribes(fake_mqtt):
|
|
||||||
"""Setting tariff_topic to empty after it was subscribed must unsubscribe it."""
|
|
||||||
dsmr_ingest.apply_dsmr_subscription(
|
|
||||||
_settings(enabled=True, topic="dsmr/json", tariff_topic="dsmr/meter-stats/electricity_tariff")
|
|
||||||
)
|
|
||||||
dsmr_ingest.apply_dsmr_subscription(
|
|
||||||
_settings(enabled=True, topic="dsmr/json", tariff_topic="")
|
|
||||||
)
|
|
||||||
|
|
||||||
assert "dsmr/meter-stats/electricity_tariff" in fake_mqtt.unsubscribe_calls
|
|
||||||
assert dsmr_ingest._current_tariff_topic is None
|
|
||||||
|
|||||||
+1091
-216
File diff suppressed because it is too large
Load Diff
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user