Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
8180082f90 | ||
|
|
b472f91f19 | ||
|
|
c9ce05d95a | ||
|
|
33ca3da593 | ||
|
|
d5623b9fcb | ||
|
|
b1b6a309cb | ||
|
|
8dc3f71aaf | ||
|
|
2be4f78f8a | ||
|
|
1a958cb102 | ||
|
|
c851bad829 | ||
|
|
09abe05f66 | ||
|
|
0924e8df52 | ||
|
|
72ac7e7300 | ||
|
|
18083822ea | ||
|
|
3beeb5a461 | ||
|
|
631b14e2ec | ||
|
|
231c340ea6 | ||
|
|
c24b6684cc | ||
|
|
d9e82038dc | ||
|
|
e59c192097 | ||
|
|
2a47dab272 | ||
|
|
9db7f63274 | ||
|
|
ebf96de4f1 | ||
|
|
5b9d60e80a | ||
|
|
963e43e3e4 | ||
|
|
39c11ae606 | ||
|
|
3eec701448 | ||
|
|
489e5b596a | ||
|
|
567ddb9779 | ||
|
|
0fb51d338c | ||
|
|
b812d5ac46 | ||
|
|
a9458394f2 | ||
|
|
4884a19e3d | ||
|
|
afe653bafa | ||
|
|
25a08c47a4 | ||
|
|
ffc693e995 | ||
|
|
5855fff451 | ||
|
|
1ea2f659e0 | ||
|
|
2e125dbd53 | ||
|
|
28486a83c7 | ||
|
|
a78401c2ef | ||
|
|
009856a50d | ||
|
|
43c2ddce1a | ||
|
|
d7f04aee8c | ||
|
|
8dbb59a3b7 | ||
|
|
2992bbb0ef | ||
|
|
22faeb45bb | ||
|
|
c37dfacfc7 | ||
|
|
16b050d821 | ||
|
|
d3d914b117 | ||
|
|
fc4af857e3 | ||
|
|
0958d9a2e9 | ||
|
|
b405aea88b | ||
|
|
bfc7aa3031 | ||
|
|
2f63b9630c | ||
|
|
d07a083e03 | ||
|
|
b65f700d56 | ||
|
|
f4cea3874b | ||
|
|
134f0abb5f | ||
|
|
3e04b15656 | ||
|
|
f2e8f6a8e7 |
@@ -7,8 +7,14 @@ APP_DATABASE_URL=sqlite:////app/data/app.db
|
||||
AUTH_BOOTSTRAP_USERNAME=admin
|
||||
AUTH_BOOTSTRAP_PASSWORD=change-me
|
||||
|
||||
# Required by Docker Compose for the WarmteLink serial device. Set these only in
|
||||
# your local .env; use a stable /dev/serial/by-id path and its numeric host GID.
|
||||
# WARMTELINK_DEVICE_PATH=/dev/serial/by-id/<stable-by-id-name>
|
||||
# WARMTELINK_SERIAL_GID=<host-serial-gid>
|
||||
|
||||
# Optional: runtime overrides.
|
||||
# Leave these commented out to use the application's built-in defaults.
|
||||
# TZ=Europe/Amsterdam
|
||||
# APP_DEBUG=
|
||||
# AUTH_SESSION_COOKIE_NAME=
|
||||
# AUTH_SESSION_TTL_HOURS=
|
||||
@@ -40,6 +46,8 @@ MQTT_BROKER_PORT=1883
|
||||
MQTT_USERNAME=
|
||||
MQTT_PASSWORD=
|
||||
MQTT_TLS_ENABLED=false
|
||||
# MQTT_CLIENT_ID must be a non-empty ASCII slug; use a distinct value per deployment.
|
||||
MQTT_CLIENT_ID=home-automation
|
||||
|
||||
# Optional: Home Assistant MQTT Discovery.
|
||||
# Requires MQTT_ENABLED=true and a running MQTT broker.
|
||||
|
||||
@@ -0,0 +1,208 @@
|
||||
# AGENTS.md — Home Automation Backend
|
||||
|
||||
本文件是本仓库 coding agent 指引的 **single source of truth**;`CLAUDE.md` 通过符号链接指向本文件。它定义本项目的**工作流程、文档位置、commit 规范**。支持对应项目指引的 agent 在动手前应完整读取本文件。
|
||||
|
||||
## 项目速览
|
||||
|
||||
- 个人用 home-automation 应用:**FastAPI + React SPA + SQLite + SQLAlchemy + Alembic**,前后端同源托管。
|
||||
- 单 admin 鉴权(Argon2 + server-side session cookie),runtime config 落 `app_config` 表。
|
||||
- 模块:public IPv4 monitor、SMTP 通知、location / poo recorder、Home Assistant in/out、TickTick OAuth、Modbus / DSMR 能耗采集、MQTT / HA Discovery、动态电价与电费计算。
|
||||
- 已发布 `v1.5.1`。M1、M2、M4-M7 已完成;M3 token / 移动端仍为远期方向。
|
||||
- **当前现实**:已收敛为单一 `app.db`、一套 DeclarativeBase 和一条 Alembic 链;只有历史数据迁移 runbook 会读取旧 location / poo 数据库。
|
||||
- 明确不做:Notion 模块。
|
||||
|
||||
## 文档地图与「开工前必读」
|
||||
|
||||
文档都在 `docs/`:
|
||||
|
||||
| 路径 | 作用 |
|
||||
| --- | --- |
|
||||
| `docs/roadmap.md` | 全局规划与里程碑总览 |
|
||||
| `docs/design/README.md` | **协作契约**:任务卡格式、原子任务定义、校验闸门、数据安全红线 |
|
||||
| `docs/design/m1-db-consolidation.md` | M1 原子任务(含真实代码现状盘点 + 人工 runbook) |
|
||||
| `docs/design/m2-frontend-v2.md` | M2 原子任务 + API 契约 + 前端校验闸门 |
|
||||
| `docs/design/m3-token-mobile.md` | M3(远期,暂缓) |
|
||||
| `docs/design/m4-login-hardening.md` | M4 登录加固(已完成) |
|
||||
| `docs/design/m5-iot-energy.md` | M5 IoT / 能耗采集(已完成) |
|
||||
| `docs/design/m6-tibber-dynamic-energy.md` | M6 动态电价、DSMR 与电费计算(已完成) |
|
||||
| `docs/design/m7-meter-epochs-archival.md` | M7 电表生命周期 / 换表归档(已完成) |
|
||||
| `docs/*.md`(auth / public-ip-monitor / location-recorder …) | 各模块说明,按需读 |
|
||||
|
||||
**开工时读取顺序**:
|
||||
1. `docs/design/README.md`(每轮都读,它是流程与验收的共同契约)。
|
||||
2. 本轮对应的 milestone 文档(如 `docs/design/m1-db-consolidation.md`),定位要做的任务卡。
|
||||
3. 任务卡 `Files` 列出的源文件 + 该模块的 `docs/*.md`(按需)。
|
||||
4. `docs/roadmap.md` 仅在需要全局视角时读。
|
||||
|
||||
## 工作流程
|
||||
|
||||
### 实现模式(由用户的提示词决定)
|
||||
|
||||
- **默认逐步**:给一个 milestone 文档,按其中原子任务**一步一步**实现。
|
||||
- **(a) 只实现一步**:用户说"只实现一步 / 这一个任务"时,**只做那一个任务卡**,跑完校验闸门后停下,等用户确认,不要顺手往下做。
|
||||
- **(b) 完成整个 milestone**:仅当用户在提示词里**显式要求启用 sub-agent**时,才起 implementer / reviewer / fixer sub-agent(按下方**『默认能力档位』**选择模型,用户人工指定则覆盖),按任务依赖顺序跑完整条链。
|
||||
- **Sub-agent 纪律**:只在用户显式要求时才 spawn sub-agent;单步/小改动在主线内联完成。当前 harness 支持独立 sub-agent 时,使用其原生机制按下方**『默认能力档位』**派发;不支持时不得假装已创建 sub-agent,应明确说明限制,并仅在用户允许 fallback 时由主 agent 继续。
|
||||
|
||||
### 默认能力档位(实现模式 sub-agent;可被人工指定覆盖)
|
||||
|
||||
起 implementer / reviewer / fixer sub-agent 时,**默认**按下列能力档位选择当前 harness 支持的模型,无需用户每次人工指定:
|
||||
|
||||
| 角色 | 通用模型要求 | 推理档位 | Harness 示例(非强制) |
|
||||
| --- | --- | --- | --- |
|
||||
| **Implementer** | 平衡型代码实现模型 | `medium` 或等效档位 | Claude Code:Sonnet;Codex/OpenAI:GPT-5.6 Terra (`gpt-5.6-terra`) |
|
||||
| **Fixer**(返工) | 平衡型代码实现模型 | `medium` 或等效档位 | Claude Code:Sonnet;Codex/OpenAI:GPT-5.6 Terra (`gpt-5.6-terra`) |
|
||||
| **Reviewer** | 当前 harness 支持的最强通用推理 / 代码模型 | `extra-high` / `xhigh` 或等效档位 | Claude Code:Opus;Codex/OpenAI:GPT-5.6 Sol (`gpt-5.6-sol`) |
|
||||
|
||||
- **示例非强制**:示例模型只表示当前推荐映射,不构成跨 harness 的硬性模型 ID;当前 harness 不支持时,选择最符合「通用模型要求」的可用模型。
|
||||
- **选择优先级**:用户显式指定 > 当前 harness 的原生角色配置 > 上表的 harness 示例 > 按通用模型要求自动选择。
|
||||
- **推理档位说明**:若 harness 提供独立的 reasoning-effort 设置,按上表设置;若不提供,在 spawn prompt 中明确 implementer/fixer 按平衡深度思考,reviewer 按对抗性外部审计强度复核。
|
||||
|
||||
### 角色(Orchestrator → Implementer → Reviewer → Fixer)
|
||||
|
||||
- 我(主线)= **Orchestrator**:挑依赖已满足的下一个任务、派发、转述结果、维护任务 `Status`。
|
||||
- **Implementer**(平衡型代码实现模型,medium 或等效档位):一次一个任务,严格按任务卡,不扩范围。
|
||||
- **Reviewer**(最强通用推理 / 代码模型,extra-high / xhigh 或等效档位):实现完成后起 Reviewer sub-agent,按任务卡 `Acceptance criteria` + `Reviewer checklist` 复核、**独立重跑校验闸门**,驱动返工直到本轮 PASS。
|
||||
- **Fixer**(平衡型代码实现模型,medium 或等效档位):按 reviewer 的编号返工清单返工;**每轮返工起一个干净的 Fixer**(与首次实现的 Implementer 分开冷启动),先读对应 `review-notes/<task>-review-<n>.md` 再改。
|
||||
|
||||
#### Reviewer 盲审纪律(M1 教训)
|
||||
|
||||
M1 里 review **从未触发过一次 rework**,根因是 orchestrator 把自己的结论 / 辩护喂给了 reviewer,造成 context bleed、review 沦为橡皮图章。所以:
|
||||
|
||||
- reviewer 必须**使用全新、独立的 sub-agent / thread 冷启动,并最小化喂料**——spawn prompt 只给:① 任务卡(`Acceptance criteria` + `Reviewer checklist`)、② 对应的 `review-notes/<task>-impl|rework-<n>.md` 路径、③ 要审的 diff / commit 范围。
|
||||
- **不要**在 prompt 里塞 orchestrator 自己的判断、"我觉得没问题"、对实现选择的辩护,或上一轮 reviewer 的倾向性结论。让它**独立得出结论、独立重跑校验闸门**。
|
||||
- 事后另起的整库**独立盲审**(如对抗复审)同理:使用全新独立的 agent / thread、最小上下文,把它当"**外部审计**"而非"确认自己没错"。
|
||||
|
||||
### 校验闸门(每个任务结束都要全绿)
|
||||
|
||||
根目录、激活 `.venv` 后:
|
||||
```bash
|
||||
pytest # 权威闸门(CI 跑的就是它)
|
||||
ruff check . # line-length=100
|
||||
python scripts/export_openapi.py && git diff --exit-code openapi/ # 改了路由/schema 才需要,且产物须入库
|
||||
```
|
||||
前端任务(M2)在 `frontend/` 下另跑 `npm run lint && npm run typecheck && npm run test && npm run build`(详见 m2 文档 §8)。
|
||||
**不过闸门就不算完成**,不得跳过、不得留红给下一轮。
|
||||
|
||||
**Repo-meta 例外**:纯文档、agent 指引、符号链接等不影响可执行代码、构建与 API 契约的变更,可在用户明确同意时跳过代码闸门。仍须完成针对性校验(如链接目标、文件类型、diff 与 Git 状态),并在结果中明确记录未运行哪些闸门。
|
||||
|
||||
#### API 契约同步:`openapi/` 与 `schema.d.ts` 是**两步**(v1.4.0 后教训)
|
||||
|
||||
**只跑 `export_openapi.py` 不够。** 前端的 `frontend/src/api/schema.d.ts` 是由 `openapi/openapi.json` 二次生成的,CI(`.github/workflows/frontend.yml` 的 *Check codegen is in sync*)会重跑 codegen 并 `git diff --exit-code src/api/schema.d.ts`。漏了第二步 → 本地闸门全绿、远端 CI 红。真出过:`d07a083` 改了 `/api/energy/prices` 的 docstring 并同步了 `openapi.json`,但没重跑 codegen,只差一行注释就把 CI 挂了。
|
||||
|
||||
所以**只要动了路由 / Pydantic schema / 路由 docstring**(docstring 也会进 OpenAPI description!),两步都要跑,两个产物都要入库:
|
||||
|
||||
```bash
|
||||
# 1) 后端契约
|
||||
python scripts/export_openapi.py && git diff --exit-code openapi/
|
||||
# 2) 前端类型(在 frontend/ 下)
|
||||
npm run codegen && git diff --exit-code src/api/schema.d.ts
|
||||
```
|
||||
|
||||
- 判据:`git diff --exit-code openapi/` 有输出 → **必然**还要跑一次 `npm run codegen`。
|
||||
- 反过来也成立:`schema.d.ts` 不要手改,它是生成物。
|
||||
- Reviewer 审"动了路由 / schema / 路由 docstring"类任务时,把**这两个产物是否都已重新生成并入库**当作 acceptance 的一部分。
|
||||
|
||||
### 构建上下文完整性(M1 Dockerfile 教训)
|
||||
|
||||
`docker build` **不在 pytest/ruff 闸门里**——M1 删了 `alembic_location/poo` 后忘了同步 `Dockerfile` 的 `COPY`,单元闸门全绿却把坏掉的镜像构建一路漏到 release tag。所以:
|
||||
|
||||
- 任务**删除 / 移动 / 重命名文件或目录**时,必须 grep 构建清单是否还在引用它们:`Dockerfile`(尤其 `COPY` 源)、`docker/`、`*.ini`、CI workflow、`requirements*.txt` 等。
|
||||
- 已有回归测试 `tests/test_deployment.py::test_dockerfile_copy_sources_exist` 守"Dockerfile `COPY` 源必须存在于构建上下文";新增 / 改动 `COPY` 时确保它仍覆盖得到。
|
||||
- Reviewer 审"删 / 移文件"类任务时,**必须顺带核对构建清单引用**,把它当 acceptance 的一部分。
|
||||
|
||||
## 每轮简报(`review-notes/`)
|
||||
|
||||
由 milestone 任务卡驱动的每轮实现、返工或 review,都要在 `review-notes/` 下产出**中文简报**。该目录**已在 `.gitignore` 忽略**,纯本地、不入库——它是 agent 之间和与人之间的交接载体,不是仓库产物。纯讨论、只读分析与不进入正式任务链的 repo-meta 变更无需产出简报,除非用户明确要求。
|
||||
|
||||
- **实现 / 返工简报**:每轮实现完成后(无论首次实现还是返工),写一份。文件名建议 `<task-id>-impl-<n>.md` / `<task-id>-rework-<n>.md`(如 `M1-T03-impl-1.md`、`M1-T03-rework-1.md`)。至少包含:
|
||||
1. **本轮修改的具体内容**(改了哪些文件、做了什么、为什么)。
|
||||
2. **自动化测试结果**(`pytest` / `ruff` / 前端闸门的实际输出或结论,通过/失败逐项写清)。
|
||||
3. **若需人工 walkthrough**:写明具体步骤(怎么启动、点哪里、预期看到什么);若无需人工验证,明确写"无需人工 walkthrough"。
|
||||
- **review 简报**:每轮 review 后写一份,文件名建议 `<task-id>-review-<n>.md`(如 `M1-T03-review-1.md`)。至少包含:评审结论(`PASS` 或带编号的返工清单)、对照任务卡 `Acceptance criteria` + `Reviewer checklist` 的逐条核对、reviewer 独立重跑校验闸门的结果。
|
||||
|
||||
**用途**:① reviewer 审核时参考对应的实现简报;② implementer 返工时参考对应的 review 简报;③ 人类(用户)通读这些简报确认有无问题。简报之间用文件名里的 `<task-id>` 与轮次 `<n>` 对应起来。
|
||||
|
||||
### Orchestrator 派发契约(让简报真正被读到)
|
||||
|
||||
**关键**:sub-agent 冷启动、不继承主线上下文,**不会因为本文件提到简报就自动去读**对应文件。简报能流转,靠的是 orchestrator(主线)在**每次 spawn 时把路径显式写进 prompt**,而不是被动约定。所以派发时必须做到:
|
||||
|
||||
- **显式告诉它「先读哪个简报」**:
|
||||
- 派 implementer 做**首次实现** → 传任务卡位置(milestone 文档路径 + task id);无前置简报。
|
||||
- 派 implementer 做**返工** → 必须传对应的 `review-notes/<task>-review-<n>.md` 路径,并要求**先读它**再改。
|
||||
- 派 reviewer → 必须传对应的 `review-notes/<task>-impl|rework-<n>.md` 路径 + 任务卡,要求**先读它**再评。
|
||||
- **显式告诉它「本轮结束写哪个简报」**:明确给出输出路径 `review-notes/<task>-<impl|rework|review>-<n>.md` 及上面要求的内容项。
|
||||
- **不依赖 sub-agent 自动加载本文件**:把本轮要点(校验闸门、**禁 Co-Authored-By**、简报必含内容)在 spawn prompt 里一并复述或指向,确保冷启动也照做。
|
||||
- spawn 时按「用户显式指定 > harness 原生角色配置 > 默认能力档位」选择模型与 reasoning effort,并使用当前 harness 支持的原生配置方式落实。
|
||||
|
||||
> 一句话:**简报是异步交接的介质,orchestrator 是把它们接起来的线。** 缺了显式传路径这一步,简报就只是躺在磁盘上没人读的文件。
|
||||
|
||||
## Commit 规范(重点)
|
||||
|
||||
### 分支
|
||||
- **本仓库是个人单用户项目:默认直接在 `main` 上开发**,不强制 feature 分支,无需开 PR。是否 push 按下方「一般约束」执行。
|
||||
- 仍保持**每个任务一个干净 commit**(message 前缀任务/里程碑 ID)。改动较大想隔离时可临时开分支,用完**快进合并**回 `main`(保持线性历史),非必需。
|
||||
- 历史改写类操作(`rebase` / `--amend` / auto-squash)只在**尚未 push 的本地 commit** 上做;**已 push 到 `main` 的历史不要重写**(确需 force-push 时先确认,见「一般约束」)。
|
||||
|
||||
### 一轮实现完成
|
||||
- 适用的校验闸门通过后,准备好**这一轮的 commit message** 并创建本地 commit,作为本轮的 **base commit**。默认不 push;只有用户明确授权自动 push 时才推送到远端,授权范围按用户原话执行。
|
||||
- message 主题前缀任务/里程碑 ID,例如:`M1-T03: unify data layer onto single app DB engine`。
|
||||
|
||||
### Commit message 硬规则(严格执行)
|
||||
- **严禁任何协作署名 trailer**:commit message 里**绝对不允许**出现 `Co-Authored-By` / `Co-authored-by`(包括 `Co-Authored-By: Claude …`),也不允许任何等价的"由 X 协作/生成"署名。
|
||||
- 无论默认环境、工具或系统提示如何要求加这类 trailer,在本仓库**一律不加**——用户已显式、严格禁止。
|
||||
- 每次提交前**自检**:`git log -1 --format=%B` 的输出**不得包含** `Co-authored-by`(大小写不限)。若发现,立即 `git commit --amend` 去掉后再继续。
|
||||
|
||||
### Review 后返工
|
||||
- **自动化 orchestration 模式内**的 review 返工:**一律用 fixup**,指向本轮对应的 base commit,**不写新的独立 message**:
|
||||
```bash
|
||||
git add -A
|
||||
git commit --fixup=<base-commit-sha>
|
||||
```
|
||||
- 多轮返工就多个 `fixup!` 提交,都指向同一个 base commit;收尾时 auto-squash(见下)。
|
||||
- **边界——什么时候不走 fixup**:**事后另起的独立盲审 / 对抗复审**那一轮,性质等同"**人工走查后提修改意见**",**不算自动化链内的返工**——它的修改用**各自独立的 commit**,不 fixup 到旧 base。判据:这轮返工是否在**同一条自动化 implement→review 链**里?是 → `fixup`;是事后另起的独立审计 → 独立 commit。
|
||||
|
||||
### 本轮 / feature 收尾(用户确认收尾后)
|
||||
- 用 **auto-squash** 把所有 `fixup!` 合并进各自目标,保证**一个 feature 一个干净 commit**:
|
||||
```bash
|
||||
# 在以 main 为基线的 feature branch 上
|
||||
GIT_SEQUENCE_EDITOR=true git rebase -i --autosquash main
|
||||
|
||||
# 直接在 main 上整理尚未 push 的本地提交
|
||||
GIT_SEQUENCE_EDITOR=true git rebase -i --autosquash origin/main
|
||||
```
|
||||
- 执行前确认选定的基线位于 base commit 之前,以便 base commit 与对应 `fixup!` 都进入 rebase 范围。用 `GIT_SEQUENCE_EDITOR=true` 让它**非交互**执行(不弹编辑器,自动接受 autosquash 排好的 todo)。
|
||||
- autosquash **改写历史**:仅在 push / 开 PR **之前**做。若该分支已 push,需要 force-push——属对外操作,**先取得用户确认再做**。
|
||||
|
||||
### 一般约束
|
||||
- **个人单用户仓库:默认直接在 `main` 上开发并创建本地 commit**。默认不 push;只有用户明确授权自动 push 时才推送到远端。
|
||||
- 始终需要**单独、明确授权**的操作:**force-push / 改写已推送历史**,以及**打 tag**(会触发镜像 CI / 对外发布;且打 tag 前须按下方「发版前置走查」真跑一次 `docker build`)。
|
||||
|
||||
## 发版前置走查(打 tag 前必做)
|
||||
|
||||
单元闸门绿 ≠ 真的能跑、能构建、能用。M1 出过"绿了但 docker 构建坏了"的事故,所以**打版本 tag(触发镜像 CI)之前**,除了 `pytest` / `ruff` 全绿,还要:
|
||||
|
||||
- **真起 app**:迁移(`python -m scripts.run_migrations`)→ `uvicorn app.main:app ...`,确认能正常启动、关键路由不 500。
|
||||
- **真跑镜像构建**:本地 `docker build`(多阶段就跑完整条),确认构建通过、`COPY` 源都在。
|
||||
- **关键功能人工瞄一眼**:尤其前端 / 可视化类(M2 的热力图、首页地图)——自动闸门判断不了"渲染对不对、UX 顺不顺",这部分**靠看跑起来的 app,不靠读代码**。
|
||||
- 上述任一不过 → **不打 tag**。tag 一旦 push 会触发 docker 镜像 CI / 对外发布,属对外操作,**先确认**。
|
||||
|
||||
## 数据安全红线(不可违反)
|
||||
|
||||
- 任何脚本 / migration **都不得删除或覆盖用户数据文件**(旧 `.db`、备份、volume)。删除只能是人工、事后、保留归档的独立步骤(见 `docs/design/m1-db-consolidation.md` §6 runbook)。
|
||||
- 涉及历史数据的迁移**先在备份副本上演练**;迁移脚本必须幂等且搬完对账行数。
|
||||
- Review 时只要发现"删文件 / drop 有数据的表 / truncate"出现在自动化任务里,直接判返工。
|
||||
|
||||
## 常用命令
|
||||
|
||||
```bash
|
||||
# 环境
|
||||
python -m venv .venv && source .venv/bin/activate && pip install -r dev-requirements.txt
|
||||
# 迁移(初始化/适配 DB)
|
||||
python -m scripts.run_migrations
|
||||
# 起服务
|
||||
uvicorn app.main:app --reload --host 0.0.0.0 --port 8000
|
||||
# 测试 / lint / OpenAPI 导出
|
||||
pytest
|
||||
ruff check .
|
||||
python scripts/export_openapi.py
|
||||
```
|
||||
@@ -1,181 +0,0 @@
|
||||
# CLAUDE.md — Home Automation Backend
|
||||
|
||||
本文件每次会话自动加载。它定义本项目的**工作流程、文档位置、commit 规范**。请在动手前先读完。
|
||||
|
||||
## 项目速览
|
||||
|
||||
- 个人用 home-automation 后端:**FastAPI + SQLite + SQLAlchemy + Alembic**,服务端模板(Jinja,M2 将换成 React SPA)。
|
||||
- 单 admin 鉴权(Argon2 + server-side session cookie),runtime config 落 `app_config` 表。
|
||||
- 模块:public IPv4 monitor、SMTP 通知、location recorder、poo recorder、Home Assistant in/out、TickTick OAuth。
|
||||
- 已发布 `v1.0.3`。下一阶段方向:**M1 单库化 → M2 React 前端 → M3 token/移动端(远期,M2 后再说)**。
|
||||
- **当前现实**:在 M1 完成前仍是**三个独立 SQLite 库**(app / location / poo),三套 DeclarativeBase、三条 Alembic 链。不要假设已经单库——以代码现状为准。
|
||||
- 明确不做:Notion 模块。
|
||||
|
||||
## 文档地图与「开工前必读」
|
||||
|
||||
文档都在 `docs/`:
|
||||
|
||||
| 路径 | 作用 |
|
||||
| --- | --- |
|
||||
| `docs/roadmap.md` | 全局规划与里程碑总览 |
|
||||
| `docs/design/README.md` | **协作契约**:任务卡格式、原子任务定义、校验闸门、数据安全红线 |
|
||||
| `docs/design/m1-db-consolidation.md` | M1 原子任务(含真实代码现状盘点 + 人工 runbook) |
|
||||
| `docs/design/m2-frontend-v2.md` | M2 原子任务 + API 契约 + 前端校验闸门 |
|
||||
| `docs/design/m3-token-mobile.md` | M3(远期,暂缓) |
|
||||
| `docs/*.md`(auth / public-ip-monitor / location-recorder …) | 各模块说明,按需读 |
|
||||
|
||||
**开工时读取顺序**:
|
||||
1. `docs/design/README.md`(每轮都读,它是流程与验收的共同契约)。
|
||||
2. 本轮对应的 milestone 文档(如 `docs/design/m1-db-consolidation.md`),定位要做的任务卡。
|
||||
3. 任务卡 `Files` 列出的源文件 + 该模块的 `docs/*.md`(按需)。
|
||||
4. `docs/roadmap.md` 仅在需要全局视角时读。
|
||||
|
||||
## 工作流程
|
||||
|
||||
### 实现模式(由用户的提示词决定)
|
||||
|
||||
- **默认逐步**:给一个 milestone 文档,按其中原子任务**一步一步**实现。
|
||||
- **(a) 只实现一步**:用户说"只实现一步 / 这一个任务"时,**只做那一个任务卡**,跑完校验闸门后停下,等用户确认,不要顺手往下做。
|
||||
- **(b) 完成整个 milestone**:仅当用户在提示词里**显式要求启用 sub-agent**时,才起 implementer / reviewer / fixer sub-agent(模型用下方**『默认模型档位』**,用户人工指定则覆盖),按任务依赖顺序跑完整条链。
|
||||
- **Sub-agent 纪律**:只在用户显式要求时才 spawn sub-agent;单步/小改动在主线内联完成。起 sub-agent 时按下方**『默认模型档位』**选择模型(用户人工指定则以人工指定为准),用 Agent 工具的 `model` 字段落实。
|
||||
|
||||
### 默认模型档位(实现模式 sub-agent;可被人工指定覆盖)
|
||||
|
||||
起 implementer / reviewer / fixer sub-agent 时,**默认**用下列模型档位,无需用户每次人工指定:
|
||||
|
||||
| 角色 | 默认模型 | 推理档位 |
|
||||
| --- | --- | --- |
|
||||
| **Implementer** | **Sonnet** | high reasoning effort |
|
||||
| **Fixer**(返工) | **Sonnet** | high reasoning effort |
|
||||
| **Reviewer** | **Opus** | extra-high reasoning effort |
|
||||
|
||||
- **人工指定覆盖**:若用户在提示词里**显式指定了其他模型**(针对任一角色),则**以用户人工指定为准**,覆盖上述默认。
|
||||
- 用 Agent 工具的 `model` 字段落实模型选择;该字段当前仅支持 `sonnet` / `opus` / `haiku` / `fable`。
|
||||
- **推理档位说明**:Agent 工具未暴露独立的 reasoning-effort 旋钮,"high / extra-high reasoning" 通过 spawn prompt 里的显式指令传达(要求 implementer/fixer 动手前充分推理边界条件;要求 reviewer 以对抗性外部审计心态最高强度复核)。若所在 harness 提供真正的 effort 设置,则一并按此档位设置。
|
||||
|
||||
### 角色(Orchestrator → Implementer → Reviewer → Fixer)
|
||||
|
||||
- 我(主线)= **Orchestrator**:挑依赖已满足的下一个任务、派发、转述结果、维护任务 `Status`。
|
||||
- **Implementer**(默认 **Sonnet**,high reasoning;见上方『默认模型档位』):一次一个任务,严格按任务卡,不扩范围。
|
||||
- **Reviewer**(默认 **Opus**,extra-high reasoning):实现完成后起 Reviewer sub-agent,按任务卡 `Acceptance criteria` + `Reviewer checklist` 复核、**独立重跑校验闸门**,驱动返工直到本轮 PASS。
|
||||
- **Fixer**(默认 **Sonnet**,high reasoning):按 reviewer 的编号返工清单返工;**每轮返工起一个干净的 Fixer**(与首次实现的 Implementer 分开冷启动),先读对应 `review-notes/<task>-review-<n>.md` 再改。
|
||||
|
||||
#### Reviewer 盲审纪律(M1 教训)
|
||||
|
||||
M1 里 review **从未触发过一次 rework**,根因是 orchestrator 把自己的结论 / 辩护喂给了 reviewer,造成 context bleed、review 沦为橡皮图章。所以:
|
||||
|
||||
- reviewer 必须**冷启动(Clear-Agent)、最小化喂料**——spawn prompt 只给:① 任务卡(`Acceptance criteria` + `Reviewer checklist`)、② 对应的 `review-notes/<task>-impl|rework-<n>.md` 路径、③ 要审的 diff / commit 范围。
|
||||
- **不要**在 prompt 里塞 orchestrator 自己的判断、"我觉得没问题"、对实现选择的辩护,或上一轮 reviewer 的倾向性结论。让它**独立得出结论、独立重跑校验闸门**。
|
||||
- 事后另起的整库**独立盲审**(如对抗复审)同理:Clear-Agent、最小上下文,把它当"**外部审计**"而非"确认自己没错"。
|
||||
|
||||
### 校验闸门(每个任务结束都要全绿)
|
||||
|
||||
根目录、激活 `.venv` 后:
|
||||
```bash
|
||||
pytest # 权威闸门(CI 跑的就是它)
|
||||
ruff check . # line-length=100
|
||||
python scripts/export_openapi.py && git diff --exit-code openapi/ # 改了路由/schema 才需要,且产物须入库
|
||||
```
|
||||
前端任务(M2)在 `frontend/` 下另跑 `npm run lint && npm run typecheck && npm run test && npm run build`(详见 m2 文档 §8)。
|
||||
**不过闸门就不算完成**,不得跳过、不得留红给下一轮。
|
||||
|
||||
### 构建上下文完整性(M1 Dockerfile 教训)
|
||||
|
||||
`docker build` **不在 pytest/ruff 闸门里**——M1 删了 `alembic_location/poo` 后忘了同步 `Dockerfile` 的 `COPY`,单元闸门全绿却把坏掉的镜像构建一路漏到 release tag。所以:
|
||||
|
||||
- 任务**删除 / 移动 / 重命名文件或目录**时,必须 grep 构建清单是否还在引用它们:`Dockerfile`(尤其 `COPY` 源)、`docker/`、`*.ini`、CI workflow、`requirements*.txt` 等。
|
||||
- 已有回归测试 `tests/test_deployment.py::test_dockerfile_copy_sources_exist` 守"Dockerfile `COPY` 源必须存在于构建上下文";新增 / 改动 `COPY` 时确保它仍覆盖得到。
|
||||
- Reviewer 审"删 / 移文件"类任务时,**必须顺带核对构建清单引用**,把它当 acceptance 的一部分。
|
||||
|
||||
## 每轮简报(`review-notes/`)
|
||||
|
||||
每轮工作都要在 `review-notes/` 下产出**中文简报**。该目录**已在 `.gitignore` 忽略**,纯本地、不入库——它是 agent 之间和与人之间的交接载体,不是仓库产物。
|
||||
|
||||
- **实现 / 返工简报**:每轮实现完成后(无论首次实现还是返工),写一份。文件名建议 `<task-id>-impl-<n>.md` / `<task-id>-rework-<n>.md`(如 `M1-T03-impl-1.md`、`M1-T03-rework-1.md`)。至少包含:
|
||||
1. **本轮修改的具体内容**(改了哪些文件、做了什么、为什么)。
|
||||
2. **自动化测试结果**(`pytest` / `ruff` / 前端闸门的实际输出或结论,通过/失败逐项写清)。
|
||||
3. **若需人工 walkthrough**:写明具体步骤(怎么启动、点哪里、预期看到什么);若无需人工验证,明确写"无需人工 walkthrough"。
|
||||
- **review 简报**:每轮 review 后写一份,文件名建议 `<task-id>-review-<n>.md`(如 `M1-T03-review-1.md`)。至少包含:评审结论(`PASS` 或带编号的返工清单)、对照任务卡 `Acceptance criteria` + `Reviewer checklist` 的逐条核对、reviewer 独立重跑校验闸门的结果。
|
||||
|
||||
**用途**:① reviewer 审核时参考对应的实现简报;② implementer 返工时参考对应的 review 简报;③ 人类(用户)通读这些简报确认有无问题。简报之间用文件名里的 `<task-id>` 与轮次 `<n>` 对应起来。
|
||||
|
||||
### Orchestrator 派发契约(让简报真正被读到)
|
||||
|
||||
**关键**:sub-agent 冷启动、不继承主线上下文,**不会因为本文件提到简报就自动去读**对应文件。简报能流转,靠的是 orchestrator(主线)在**每次 spawn 时把路径显式写进 prompt**,而不是被动约定。所以派发时必须做到:
|
||||
|
||||
- **显式告诉它「先读哪个简报」**:
|
||||
- 派 implementer 做**首次实现** → 传任务卡位置(milestone 文档路径 + task id);无前置简报。
|
||||
- 派 implementer 做**返工** → 必须传对应的 `review-notes/<task>-review-<n>.md` 路径,并要求**先读它**再改。
|
||||
- 派 reviewer → 必须传对应的 `review-notes/<task>-impl|rework-<n>.md` 路径 + 任务卡,要求**先读它**再评。
|
||||
- **显式告诉它「本轮结束写哪个简报」**:明确给出输出路径 `review-notes/<task>-<impl|rework|review>-<n>.md` 及上面要求的内容项。
|
||||
- **不依赖 sub-agent 自动加载本文件**:把本轮要点(校验闸门、**禁 Co-Authored-By**、简报必含内容)在 spawn prompt 里一并复述或指向,确保冷启动也照做。
|
||||
- spawn 时用用户指定的模型(Agent 工具 `model` 覆盖)。
|
||||
|
||||
> 一句话:**简报是异步交接的介质,orchestrator 是把它们接起来的线。** 缺了显式传路径这一步,简报就只是躺在磁盘上没人读的文件。
|
||||
|
||||
## Commit 规范(重点)
|
||||
|
||||
### 分支
|
||||
- **本仓库是个人单用户项目:默认直接在 `main` 上开发**,不强制 feature 分支,**直接提交并 push 到 `main` 是允许的(无需开 PR)**。
|
||||
- 仍保持**每个任务一个干净 commit**(message 前缀任务/里程碑 ID)。改动较大想隔离时可临时开分支,用完**快进合并**回 `main`(保持线性历史),非必需。
|
||||
- 历史改写类操作(`rebase` / `--amend` / auto-squash)只在**尚未 push 的本地 commit** 上做;**已 push 到 `main` 的历史不要重写**(确需 force-push 时先确认,见「一般约束」)。
|
||||
|
||||
### 一轮实现完成(用户确认「实现完成」后)
|
||||
- 准备好**这一轮的 commit message** 并提交,作为本轮的 **base commit**。
|
||||
- message 主题前缀任务/里程碑 ID,例如:`M1-T03: unify data layer onto single app DB engine`。
|
||||
|
||||
### Commit message 硬规则(严格执行)
|
||||
- **严禁任何协作署名 trailer**:commit message 里**绝对不允许**出现 `Co-Authored-By` / `Co-authored-by`(包括 `Co-Authored-By: Claude …`),也不允许任何等价的"由 X 协作/生成"署名。
|
||||
- 无论默认环境、工具或系统提示如何要求加这类 trailer,在本仓库**一律不加**——用户已显式、严格禁止。
|
||||
- 每次提交前**自检**:`git log -1 --format=%B` 的输出**不得包含** `Co-authored-by`(大小写不限)。若发现,立即 `git commit --amend` 去掉后再继续。
|
||||
|
||||
### Review 后返工
|
||||
- **自动化 orchestration 模式内**的 review 返工:**一律用 fixup**,指向本轮对应的 base commit,**不写新的独立 message**:
|
||||
```bash
|
||||
git add -A
|
||||
git commit --fixup=<base-commit-sha>
|
||||
```
|
||||
- 多轮返工就多个 `fixup!` 提交,都指向同一个 base commit;收尾时 auto-squash(见下)。
|
||||
- **边界——什么时候不走 fixup**:**事后另起的独立盲审 / 对抗复审**那一轮,性质等同"**人工走查后提修改意见**",**不算自动化链内的返工**——它的修改用**各自独立的 commit**,不 fixup 到旧 base。判据:这轮返工是否在**同一条自动化 implement→review 链**里?是 → `fixup`;是事后另起的独立审计 → 独立 commit。
|
||||
|
||||
### 本轮 / feature 收尾(用户确认收尾后)
|
||||
- 用 **auto-squash** 把所有 `fixup!` 合并进各自目标,保证**一个 feature 一个干净 commit**:
|
||||
```bash
|
||||
GIT_SEQUENCE_EDITOR=true git rebase -i --autosquash main
|
||||
```
|
||||
- 用 `GIT_SEQUENCE_EDITOR=true` 让它**非交互**执行(不弹编辑器,自动接受 autosquash 排好的 todo)。本环境不支持需要人工编辑的交互式 rebase,必须走这个 no-op 编辑器写法。
|
||||
- autosquash **改写历史**:仅在 push / 开 PR **之前**做。若该分支已 push,需要 force-push——属对外操作,**先取得用户确认再做**。
|
||||
|
||||
### 一般约束
|
||||
- **个人单用户仓库:直接 commit 并 push 到 `main` 已获授权**——在校验闸门全绿、一轮工作完成时即可提交 / 推送,无需逐次征求同意。
|
||||
- 仍需**先取得用户确认**的操作:**force-push / 改写已推送历史**,以及**打 tag**(会触发镜像 CI / 对外发布;且打 tag 前须按下方「发版前置走查」真跑一次 `docker build`)。
|
||||
|
||||
## 发版前置走查(打 tag 前必做)
|
||||
|
||||
单元闸门绿 ≠ 真的能跑、能构建、能用。M1 出过"绿了但 docker 构建坏了"的事故,所以**打版本 tag(触发镜像 CI)之前**,除了 `pytest` / `ruff` 全绿,还要:
|
||||
|
||||
- **真起 app**:迁移(`python -m scripts.run_migrations`)→ `uvicorn app.main:app ...`,确认能正常启动、关键路由不 500。
|
||||
- **真跑镜像构建**:本地 `docker build`(多阶段就跑完整条),确认构建通过、`COPY` 源都在。
|
||||
- **关键功能人工瞄一眼**:尤其前端 / 可视化类(M2 的热力图、首页地图)——自动闸门判断不了"渲染对不对、UX 顺不顺",这部分**靠看跑起来的 app,不靠读代码**。
|
||||
- 上述任一不过 → **不打 tag**。tag 一旦 push 会触发 docker 镜像 CI / 对外发布,属对外操作,**先确认**。
|
||||
|
||||
## 数据安全红线(不可违反)
|
||||
|
||||
- 任何脚本 / migration **都不得删除或覆盖用户数据文件**(旧 `.db`、备份、volume)。删除只能是人工、事后、保留归档的独立步骤(见 `docs/design/m1-db-consolidation.md` §6 runbook)。
|
||||
- 涉及历史数据的迁移**先在备份副本上演练**;迁移脚本必须幂等且搬完对账行数。
|
||||
- Review 时只要发现"删文件 / drop 有数据的表 / truncate"出现在自动化任务里,直接判返工。
|
||||
|
||||
## 常用命令
|
||||
|
||||
```bash
|
||||
# 环境
|
||||
python -m venv .venv && source .venv/bin/activate && pip install -r dev-requirements.txt
|
||||
# 迁移(初始化/适配 DB)
|
||||
python -m scripts.run_migrations
|
||||
# 起服务
|
||||
uvicorn app.main:app --reload --host 0.0.0.0 --port 8000
|
||||
# 测试 / lint / OpenAPI 导出
|
||||
pytest
|
||||
ruff check .
|
||||
python scripts/export_openapi.py
|
||||
```
|
||||
@@ -21,6 +21,8 @@
|
||||
- **通用电价合同层**:YAML profile 定合同结构(manual 固定/双费率 / tibber 动态电价);`EnergyContract`+`EnergyContractVersion` 存 UI 可填的数值,改价加新版本旧版本保留;price strategy 按 kind 出价
|
||||
- **实时买卖电费计算**:每 15 分钟按寄存器差值(`_1`=dal/低、`_2`=normal/高)× 买/卖价算计量电费,快照不可变;日/月/年汇总加固定费减 heffingskorting
|
||||
- **反哺 Home Assistant Energy**:当前买/卖价 + 累计买电支出/卖电收入(`total_increasing`)发成 HA 实体,可直接挂 HA Energy 仪表盘
|
||||
- **多数据源 Meter 与 WarmteLink**:DSMR MQTT 与只读 WarmteLink P1 serial source 统一为 Source → Channel → Binding → Meter;WarmteLink 提供 heating `GJ` 与 hot-water `m³` 的 Decimal history、质量与重连
|
||||
- **热力合同与成本**:electricity / thermal scope 可各有一个 active 合同;热力按 15 分钟账本计算 variable、fixed 与 all-in 成本,并可按需暴露给 HA
|
||||
- pytest 测试与 OpenAPI 导出脚本
|
||||
- Docker / Compose 部署入口
|
||||
|
||||
@@ -44,6 +46,8 @@
|
||||
- 电价合同(`energy_contract` 表)与版本(`energy_contract_version` 表,values JSON)
|
||||
- Tibber 15 分钟电价缓存(`tibber_price` 表,不可变)
|
||||
- 每 15 分钟计量电费(`energy_cost_period` 表,快照价,不可变)
|
||||
- meter source、channel 与 binding(`meter_source`、`meter_source_channel`、`meter_source_binding`)
|
||||
- WarmteLink scalar 历史(`warmtelink_reading`)与热力 15 分钟成本账本(`meter_cost_period`)
|
||||
|
||||
配置层只保留一个数据库环境变量:
|
||||
|
||||
@@ -55,7 +59,7 @@
|
||||
python -m scripts.run_migrations
|
||||
```
|
||||
|
||||
该命令会通过 Alembic 将 `app.db` 初始化或升级到最新 head(含全部表,包括 M5 新增的 `modbus_device`、`modbus_reading`、`exposed_entity_toggle`,以及 M6 新增的 `dsmr_reading`、`energy_contract`、`energy_contract_version`、`tibber_price`、`energy_cost_period`)。
|
||||
该命令会通过 Alembic 将 `app.db` 初始化或升级到最新 head(包括 Modbus、DSMR、Source/Channel/Binding、WarmteLink、electricity/thermal 合同与成本账本)。
|
||||
|
||||
## 当前目录
|
||||
|
||||
@@ -63,7 +67,7 @@ python -m scripts.run_migrations
|
||||
|
||||
- `app/`: FastAPI 应用代码(包含 JSON API、业务服务、数据模型)
|
||||
- `frontend/`: React SPA 前端(Vite + React + TypeScript + Mantine)
|
||||
- `alembic_app/`: App DB 的 Alembic migration 环境(管理所有表,含 M5 新增的 `modbus_device`、`modbus_reading`、`exposed_entity_toggle`,以及 M6 新增的 `dsmr_reading`、`energy_contract`、`energy_contract_version`、`tibber_price`、`energy_cost_period`)
|
||||
- `alembic_app/`: App DB 的唯一 Alembic migration 环境(管理所有 app 表,包括 Modbus、DSMR、Meter source、WarmteLink、合同与成本账本)
|
||||
- `tests/`: pytest 测试
|
||||
- `docs/`: 当前系统说明文档
|
||||
- `scripts/`: 辅助脚本,例如 OpenAPI 导出
|
||||
@@ -551,6 +555,20 @@ python scripts/export_openapi.py
|
||||
- `docker-compose.dev.yml`:本地开发显式叠加层——追加 `build: .`、独立 project /
|
||||
容器名(`-dev` 后缀)、暴露 8001,并把 DB 指向挂载的 `./data` 副本,可与生产栈在同一台机器上并存
|
||||
|
||||
WarmteLink serial access is configured directly by both Compose combinations. Before starting either
|
||||
one, set these host-specific values in your uncommitted local `.env` (use a stable `/dev/serial/by-id/...`
|
||||
path, never a transient `/dev/ttyUSB*` name):
|
||||
|
||||
```dotenv
|
||||
WARMTELINK_DEVICE_PATH=/dev/serial/by-id/<stable-by-id-name>
|
||||
WARMTELINK_SERIAL_GID=<host-serial-gid>
|
||||
```
|
||||
|
||||
Only `app` receives the device as `/dev/warmtelink:rw` and the serial group; `migration` does not.
|
||||
The app remains non-root, non-privileged, and has no added capabilities. One physical serial port may
|
||||
have only one owner: stop the app before running the Pre-M8 P1 probe. Never remove `./data`, databases,
|
||||
or volumes while changing this configuration.
|
||||
|
||||
本地开发启动方式(显式叠加 dev 层):
|
||||
|
||||
```bash
|
||||
|
||||
@@ -20,6 +20,7 @@ from app.models.energy import ( # noqa: F401
|
||||
TibberPrice,
|
||||
EnergyCostPeriod,
|
||||
)
|
||||
from app.models.meter_source import MeterSource, MeterSourceBinding, MeterSourceChannel # noqa: F401
|
||||
|
||||
config = context.config
|
||||
|
||||
|
||||
@@ -0,0 +1,107 @@
|
||||
"""add protocol-agnostic meter source, channel, and binding tables
|
||||
|
||||
Revision ID: 20260822_15_meter_sources
|
||||
Revises: 20260625_14_meter_uuid
|
||||
Create Date: 2026-08-22 00:00:00.000000
|
||||
|
||||
This revision is additive on upgrade. It deliberately does not backfill
|
||||
existing DSMR data; that adoption is a later, separately audited migration.
|
||||
"""
|
||||
|
||||
from typing import Sequence, Union
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
|
||||
|
||||
revision: str = "20260822_15_meter_sources"
|
||||
down_revision: Union[str, None] = "20260625_14_meter_uuid"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.create_table(
|
||||
"meter_source",
|
||||
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
|
||||
sa.Column("uuid", sa.String(length=36), nullable=False),
|
||||
sa.Column("name", sa.String(length=255), nullable=False),
|
||||
sa.Column("kind", sa.String(length=64), nullable=False),
|
||||
sa.Column("enabled", sa.Boolean(), nullable=False),
|
||||
sa.Column("config", sa.JSON(), nullable=False),
|
||||
sa.Column("status", sa.String(length=32), nullable=False),
|
||||
sa.Column("last_seen_at", sa.DateTime(timezone=True), nullable=True),
|
||||
sa.Column("last_error", sa.String(length=1024), nullable=True),
|
||||
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.PrimaryKeyConstraint("id"),
|
||||
sa.UniqueConstraint("uuid", name="uq_meter_source_uuid"),
|
||||
)
|
||||
op.create_index("ix_meter_source_kind_enabled", "meter_source", ["kind", "enabled"])
|
||||
|
||||
op.create_table(
|
||||
"meter_source_channel",
|
||||
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
|
||||
sa.Column("uuid", sa.String(length=36), nullable=False),
|
||||
sa.Column("source_id", sa.Integer(), nullable=False),
|
||||
sa.Column("channel_key", sa.String(length=128), nullable=False),
|
||||
sa.Column("label", sa.String(length=255), nullable=False),
|
||||
sa.Column("suggested_commodity", sa.String(length=32), nullable=True),
|
||||
sa.Column("unit", sa.String(length=32), nullable=False),
|
||||
sa.Column("device_type", sa.String(length=64), nullable=True),
|
||||
sa.Column("fingerprint", sa.String(length=64), nullable=True),
|
||||
sa.Column("latest_value", sa.Numeric(precision=20, scale=6), nullable=True),
|
||||
sa.Column("latest_at", sa.DateTime(timezone=True), nullable=True),
|
||||
sa.Column("latest_quality", sa.String(length=32), nullable=True),
|
||||
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.ForeignKeyConstraint(["source_id"], ["meter_source.id"], ondelete="RESTRICT"),
|
||||
sa.PrimaryKeyConstraint("id"),
|
||||
sa.UniqueConstraint("uuid", name="uq_meter_source_channel_uuid"),
|
||||
sa.UniqueConstraint("source_id", "channel_key", name="uq_meter_source_channel_source_key"),
|
||||
)
|
||||
op.create_index("ix_meter_source_channel_source_id", "meter_source_channel", ["source_id"])
|
||||
|
||||
op.create_table(
|
||||
"meter_source_binding",
|
||||
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
|
||||
sa.Column("uuid", sa.String(length=36), nullable=False),
|
||||
sa.Column("meter_id", sa.Integer(), nullable=False),
|
||||
sa.Column("channel_id", sa.Integer(), nullable=False),
|
||||
sa.Column("started_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.Column("ended_at", sa.DateTime(timezone=True), nullable=True),
|
||||
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.ForeignKeyConstraint(["meter_id"], ["meter.id"], ondelete="RESTRICT"),
|
||||
sa.ForeignKeyConstraint(
|
||||
["channel_id"], ["meter_source_channel.id"], ondelete="RESTRICT"
|
||||
),
|
||||
sa.PrimaryKeyConstraint("id"),
|
||||
sa.UniqueConstraint("uuid", name="uq_meter_source_binding_uuid"),
|
||||
)
|
||||
op.create_index("ix_meter_source_binding_meter_id", "meter_source_binding", ["meter_id"])
|
||||
op.create_index("ix_meter_source_binding_channel_id", "meter_source_binding", ["channel_id"])
|
||||
|
||||
with op.batch_alter_table("energy_cost_period", schema=None) as batch_op:
|
||||
batch_op.add_column(sa.Column("source_binding_id", sa.Integer(), nullable=True))
|
||||
batch_op.create_foreign_key(
|
||||
"fk_energy_cost_period_source_binding_id",
|
||||
"meter_source_binding",
|
||||
["source_binding_id"],
|
||||
["id"],
|
||||
ondelete="RESTRICT",
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
with op.batch_alter_table("energy_cost_period", schema=None) as batch_op:
|
||||
batch_op.drop_constraint("fk_energy_cost_period_source_binding_id", type_="foreignkey")
|
||||
batch_op.drop_column("source_binding_id")
|
||||
|
||||
op.drop_index("ix_meter_source_binding_channel_id", table_name="meter_source_binding")
|
||||
op.drop_index("ix_meter_source_binding_meter_id", table_name="meter_source_binding")
|
||||
op.drop_table("meter_source_binding")
|
||||
op.drop_index("ix_meter_source_channel_source_id", table_name="meter_source_channel")
|
||||
op.drop_table("meter_source_channel")
|
||||
op.drop_index("ix_meter_source_kind_enabled", table_name="meter_source")
|
||||
op.drop_table("meter_source")
|
||||
@@ -0,0 +1,269 @@
|
||||
"""adopt historical DSMR rows into the source and binding model
|
||||
|
||||
Revision ID: 20260822_16_dsmr_source_adoption
|
||||
Revises: 20260822_15_meter_sources
|
||||
Create Date: 2026-08-22 00:00:00.000000
|
||||
|
||||
The upgrade is deliberately data-preserving: it creates one migration-owned
|
||||
DSMR source/channel, moves the telegram identifier to ``telegram_id``, and
|
||||
audits every reading and cost row before committing. Old ``app_config`` rows,
|
||||
payload JSON, and cost snapshots are never deleted or rewritten.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import uuid
|
||||
from datetime import datetime, timezone
|
||||
from typing import Sequence, Union
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
|
||||
|
||||
revision: str = "20260822_16_dsmr_source_adoption"
|
||||
down_revision: Union[str, None] = "20260822_15_meter_sources"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def _as_bool(value: str | None) -> bool:
|
||||
return value is not None and value.strip().lower() in {"1", "true", "yes", "on"}
|
||||
|
||||
|
||||
def _iso_now() -> str:
|
||||
return datetime.now(tz=timezone.utc).replace(tzinfo=None).isoformat(sep=" ")
|
||||
|
||||
|
||||
def _config(connection: sa.Connection) -> dict[str, object]:
|
||||
rows = connection.execute(sa.text("SELECT key, value FROM app_config")).all()
|
||||
values = {str(key): str(value) for key, value in rows}
|
||||
# An unconfigured historical DSMR installation needs a disabled identity,
|
||||
# not guessed connection details. Preserve every legacy value we model
|
||||
# when any legacy DSMR/MQTT configuration was explicitly present.
|
||||
legacy_keys = {
|
||||
"MQTT_BROKER_HOST", "MQTT_BROKER_PORT", "MQTT_USERNAME", "MQTT_PASSWORD",
|
||||
"MQTT_TLS_ENABLED", "DSMR_MQTT_TOPIC", "DSMR_TARIFF_TOPIC", "DSMR_SAMPLE_INTERVAL_S",
|
||||
}
|
||||
if not legacy_keys & values.keys():
|
||||
return {}
|
||||
return {
|
||||
"broker_host": values.get("MQTT_BROKER_HOST", ""),
|
||||
"broker_port": int(values.get("MQTT_BROKER_PORT", "1883")),
|
||||
"username": values.get("MQTT_USERNAME", ""),
|
||||
"password": values.get("MQTT_PASSWORD", ""),
|
||||
"tls_enabled": _as_bool(values.get("MQTT_TLS_ENABLED")),
|
||||
"topic": values.get("DSMR_MQTT_TOPIC", "dsmr/json"),
|
||||
"tariff_topic": values.get("DSMR_TARIFF_TOPIC", "dsmr/meter-stats/electricity_tariff"),
|
||||
"sample_interval_s": int(values.get("DSMR_SAMPLE_INTERVAL_S", "10")),
|
||||
}
|
||||
|
||||
|
||||
def _count(connection: sa.Connection, table: str) -> int:
|
||||
return int(connection.execute(sa.text(f"SELECT COUNT(*) FROM {table}")).scalar_one())
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
connection = op.get_bind()
|
||||
readings_before = _count(connection, "dsmr_reading")
|
||||
costs_before = _count(connection, "energy_cost_period")
|
||||
sources_before = _count(connection, "meter_source")
|
||||
channels_before = _count(connection, "meter_source_channel")
|
||||
bindings_before = _count(connection, "meter_source_binding")
|
||||
now = _iso_now()
|
||||
|
||||
# A source exists even without historical configuration/readings. It stays
|
||||
# disabled unless the old explicit DSMR switch was enabled, so no broker or
|
||||
# topic is guessed at runtime.
|
||||
config = _config(connection)
|
||||
source_result = connection.execute(
|
||||
sa.text(
|
||||
"INSERT INTO meter_source "
|
||||
"(uuid, name, kind, enabled, config, status, last_seen_at, last_error, created_at, updated_at) "
|
||||
"VALUES (:uuid, :name, 'dsmr_mqtt', :enabled, :config, 'unknown', NULL, NULL, :now, :now)"
|
||||
),
|
||||
{
|
||||
"uuid": str(uuid.uuid4()),
|
||||
"name": "Migrated DSMR source",
|
||||
"enabled": _as_bool(
|
||||
connection.execute(
|
||||
sa.text("SELECT value FROM app_config WHERE key = 'DSMR_INGEST_ENABLED'")
|
||||
).scalar_one_or_none()
|
||||
),
|
||||
"config": __import__("json").dumps(config),
|
||||
"now": now,
|
||||
},
|
||||
)
|
||||
source_id = source_result.lastrowid
|
||||
if source_id is None:
|
||||
raise RuntimeError("DSMR source adoption failed to create a source")
|
||||
channel_result = connection.execute(
|
||||
sa.text(
|
||||
"INSERT INTO meter_source_channel "
|
||||
"(uuid, source_id, channel_key, label, suggested_commodity, unit, device_type, fingerprint, "
|
||||
"latest_value, latest_at, latest_quality, created_at, updated_at) "
|
||||
"VALUES (:uuid, :source_id, 'electricity-total', 'DSMR electricity total', 'electricity', "
|
||||
"'kWh', NULL, NULL, NULL, NULL, NULL, :now, :now)"
|
||||
),
|
||||
{"uuid": str(uuid.uuid4()), "source_id": source_id, "now": now},
|
||||
)
|
||||
channel_id = channel_result.lastrowid
|
||||
if channel_id is None:
|
||||
raise RuntimeError("DSMR source adoption failed to create an electricity channel")
|
||||
if _count(connection, "meter_source") != sources_before + 1:
|
||||
raise RuntimeError("DSMR source adoption source row-count audit failed")
|
||||
if _count(connection, "meter_source_channel") != channels_before + 1:
|
||||
raise RuntimeError("DSMR source adoption channel row-count audit failed")
|
||||
|
||||
# Rename/add while nullable, back-fill all rows, then make the FK non-null
|
||||
# and replace the legacy timestamp-only uniqueness in a SQLite batch rebuild.
|
||||
with op.batch_alter_table("dsmr_reading", schema=None) as batch_op:
|
||||
batch_op.alter_column("source_id", new_column_name="telegram_id")
|
||||
batch_op.add_column(sa.Column("meter_source_id", sa.Integer(), nullable=True))
|
||||
connection.execute(
|
||||
sa.text("UPDATE dsmr_reading SET meter_source_id = :source_id WHERE meter_source_id IS NULL"),
|
||||
{"source_id": source_id},
|
||||
)
|
||||
with op.batch_alter_table("dsmr_reading", schema=None) as batch_op:
|
||||
batch_op.drop_constraint("uq_dsmr_reading_recorded_at", type_="unique")
|
||||
batch_op.alter_column("meter_source_id", existing_type=sa.Integer(), nullable=False)
|
||||
batch_op.create_foreign_key(
|
||||
"fk_dsmr_reading_meter_source_id", "meter_source", ["meter_source_id"], ["id"],
|
||||
ondelete="RESTRICT",
|
||||
)
|
||||
batch_op.create_unique_constraint(
|
||||
"uq_dsmr_reading_source_recorded_at", ["meter_source_id", "recorded_at"]
|
||||
)
|
||||
batch_op.create_index("ix_dsmr_reading_meter_source_id", ["meter_source_id"])
|
||||
adopted_readings = int(
|
||||
connection.execute(
|
||||
sa.text("SELECT COUNT(*) FROM dsmr_reading WHERE meter_source_id = :source_id"),
|
||||
{"source_id": source_id},
|
||||
).scalar_one()
|
||||
)
|
||||
if adopted_readings != readings_before:
|
||||
raise RuntimeError("DSMR source adoption reading source audit failed")
|
||||
|
||||
# Bind each electricity meter only where it overlaps the actual DSMR data.
|
||||
data_window = connection.execute(
|
||||
sa.text("SELECT MIN(recorded_at), MAX(recorded_at) FROM dsmr_reading")
|
||||
).one()
|
||||
expected_binding_count = 0
|
||||
if data_window[0] is not None:
|
||||
meters = connection.execute(
|
||||
sa.text(
|
||||
"SELECT id, started_at, ended_at FROM meter WHERE commodity = 'electricity' "
|
||||
"ORDER BY started_at, id"
|
||||
)
|
||||
).all()
|
||||
for meter_id, started_at, ended_at in meters:
|
||||
# Intersect [meter start, meter end) with the inclusive historical
|
||||
# samples. A closed boundary at the final sample remains valid for
|
||||
# the preceding interval; an empty intersection gets no fake binding.
|
||||
if started_at > data_window[1] or (ended_at is not None and ended_at <= data_window[0]):
|
||||
continue
|
||||
expected_binding_count += 1
|
||||
binding_start = max(started_at, data_window[0])
|
||||
binding_end = ended_at
|
||||
connection.execute(
|
||||
sa.text(
|
||||
"INSERT INTO meter_source_binding "
|
||||
"(uuid, meter_id, channel_id, started_at, ended_at, created_at, updated_at) "
|
||||
"VALUES (:uuid, :meter_id, :channel_id, :started_at, :ended_at, :now, :now)"
|
||||
),
|
||||
{
|
||||
"uuid": str(uuid.uuid4()), "meter_id": meter_id, "channel_id": channel_id,
|
||||
"started_at": binding_start, "ended_at": binding_end, "now": now,
|
||||
},
|
||||
)
|
||||
|
||||
# A cost period may be linked only if exactly one binding covers both its
|
||||
# start and end. Historical boundary/unknown rows remain auditable but are
|
||||
# explicitly degraded instead of being silently attributed to a current meter.
|
||||
periods = connection.execute(
|
||||
sa.text("SELECT id, meter_id, period_start, degraded FROM energy_cost_period")
|
||||
).all()
|
||||
resolvable_normal_periods: dict[int, int] = {}
|
||||
unresolved_period_ids: set[int] = set()
|
||||
for period_id, meter_id, period_start, degraded_before in periods:
|
||||
candidates = []
|
||||
if meter_id is not None:
|
||||
candidates = connection.execute(
|
||||
sa.text(
|
||||
"SELECT id FROM meter_source_binding "
|
||||
"WHERE meter_id = :meter_id AND started_at <= :start "
|
||||
"AND (ended_at IS NULL OR julianday(ended_at) > julianday(:start, '+15 minutes'))"
|
||||
),
|
||||
{"meter_id": meter_id, "start": period_start},
|
||||
).all()
|
||||
if len(candidates) == 1:
|
||||
if not degraded_before:
|
||||
resolvable_normal_periods[period_id] = candidates[0][0]
|
||||
connection.execute(
|
||||
sa.text("UPDATE energy_cost_period SET source_binding_id = :binding_id WHERE id = :id"),
|
||||
{"binding_id": candidates[0][0], "id": period_id},
|
||||
)
|
||||
else:
|
||||
unresolved_period_ids.add(period_id)
|
||||
connection.execute(
|
||||
sa.text(
|
||||
"UPDATE energy_cost_period SET degraded = 1, source_binding_id = NULL WHERE id = :id"
|
||||
),
|
||||
{"id": period_id},
|
||||
)
|
||||
|
||||
readings_after = _count(connection, "dsmr_reading")
|
||||
costs_after = _count(connection, "energy_cost_period")
|
||||
if readings_after != readings_before or costs_after != costs_before:
|
||||
raise RuntimeError("DSMR source adoption row-count audit failed")
|
||||
if _count(connection, "meter_source_binding") != bindings_before + expected_binding_count:
|
||||
raise RuntimeError("DSMR source adoption binding row-count audit failed")
|
||||
if int(
|
||||
connection.execute(
|
||||
sa.text("SELECT COUNT(*) FROM meter_source_binding WHERE channel_id = :channel_id"),
|
||||
{"channel_id": channel_id},
|
||||
).scalar_one()
|
||||
) != expected_binding_count:
|
||||
raise RuntimeError("DSMR source adoption binding channel audit failed")
|
||||
for period_id, binding_id in resolvable_normal_periods.items():
|
||||
bound, degraded = connection.execute(
|
||||
sa.text("SELECT source_binding_id, degraded FROM energy_cost_period WHERE id = :id"),
|
||||
{"id": period_id},
|
||||
).one()
|
||||
if bound != binding_id or degraded:
|
||||
raise RuntimeError("DSMR source adoption resolvable cost audit failed")
|
||||
if unresolved_period_ids:
|
||||
unresolved_count = int(
|
||||
connection.execute(
|
||||
sa.text(
|
||||
"SELECT COUNT(*) FROM energy_cost_period "
|
||||
"WHERE id IN :period_ids AND (degraded != 1 OR source_binding_id IS NOT NULL)"
|
||||
).bindparams(sa.bindparam("period_ids", expanding=True)),
|
||||
{"period_ids": list(unresolved_period_ids)},
|
||||
).scalar_one()
|
||||
)
|
||||
if unresolved_count:
|
||||
raise RuntimeError("DSMR source adoption unresolved cost audit failed")
|
||||
orphan_rows = connection.execute(sa.text("PRAGMA foreign_key_check")).all()
|
||||
if orphan_rows:
|
||||
raise RuntimeError("DSMR source adoption foreign-key audit failed")
|
||||
normal_unbound = int(
|
||||
connection.execute(
|
||||
sa.text("SELECT COUNT(*) FROM energy_cost_period WHERE degraded = 0 AND source_binding_id IS NULL")
|
||||
).scalar_one()
|
||||
)
|
||||
if normal_unbound:
|
||||
raise RuntimeError(f"DSMR source adoption left {normal_unbound} normal cost period(s) unbound")
|
||||
if _count(connection, "meter_source") < 1:
|
||||
raise RuntimeError("DSMR source adoption source audit failed")
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
# Schema-only downgrade for isolated test databases. It intentionally does
|
||||
# not delete migration-created source/channel/binding rows.
|
||||
with op.batch_alter_table("dsmr_reading", schema=None) as batch_op:
|
||||
batch_op.drop_index("ix_dsmr_reading_meter_source_id")
|
||||
batch_op.drop_constraint("uq_dsmr_reading_source_recorded_at", type_="unique")
|
||||
batch_op.drop_constraint("fk_dsmr_reading_meter_source_id", type_="foreignkey")
|
||||
batch_op.drop_column("meter_source_id")
|
||||
batch_op.alter_column("telegram_id", new_column_name="source_id")
|
||||
batch_op.create_unique_constraint("uq_dsmr_reading_recorded_at", ["recorded_at"])
|
||||
@@ -0,0 +1,52 @@
|
||||
"""add normalized WarmteLink scalar reading history
|
||||
|
||||
Revision ID: 20260822_17_warmtelink_readings
|
||||
Revises: 20260822_16_dsmr_source_adoption
|
||||
Create Date: 2026-08-22 00:00:00.000000
|
||||
|
||||
The upgrade is additive: existing business rows are neither changed nor
|
||||
removed. The downgrade is schema-only and is exercised only on isolated test
|
||||
databases.
|
||||
"""
|
||||
|
||||
from typing import Sequence, Union
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
|
||||
|
||||
revision: str = "20260822_17_warmtelink_readings"
|
||||
down_revision: Union[str, None] = "20260822_16_dsmr_source_adoption"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.create_table(
|
||||
"warmtelink_reading",
|
||||
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
|
||||
sa.Column("channel_id", sa.Integer(), nullable=False),
|
||||
sa.Column("recorded_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.Column("received_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.Column("value", sa.Numeric(precision=15, scale=3), nullable=False),
|
||||
sa.Column("unit", sa.String(length=32), nullable=False),
|
||||
sa.Column("quality", sa.String(length=32), nullable=False),
|
||||
sa.Column("equipment_fingerprint", sa.String(length=64), nullable=False),
|
||||
sa.CheckConstraint(
|
||||
"quality IN ('valid', 'invalid', 'unverifiable')",
|
||||
name="ck_warmtelink_reading_quality",
|
||||
),
|
||||
sa.ForeignKeyConstraint(
|
||||
["channel_id"], ["meter_source_channel.id"], ondelete="RESTRICT"
|
||||
),
|
||||
sa.PrimaryKeyConstraint("id"),
|
||||
sa.UniqueConstraint(
|
||||
"channel_id", "recorded_at", name="uq_warmtelink_reading_channel_recorded_at"
|
||||
),
|
||||
)
|
||||
op.create_index("ix_warmtelink_reading_recorded_at", "warmtelink_reading", ["recorded_at"])
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_index("ix_warmtelink_reading_recorded_at", table_name="warmtelink_reading")
|
||||
op.drop_table("warmtelink_reading")
|
||||
@@ -0,0 +1,107 @@
|
||||
"""add a billing scope to energy contracts
|
||||
|
||||
Revision ID: 20260822_18_contract_scopes
|
||||
Revises: 20260822_17_warmtelink_readings
|
||||
Create Date: 2026-08-22 00:00:00.000000
|
||||
|
||||
The upgrade preserves every existing contract, version and cost row. Existing
|
||||
contracts predate scopes and therefore deterministically belong to electricity.
|
||||
"""
|
||||
|
||||
from typing import Sequence, Union
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
|
||||
|
||||
revision: str = "20260822_18_contract_scopes"
|
||||
down_revision: Union[str, None] = "20260822_17_warmtelink_readings"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def _count(connection: sa.Connection, table: str) -> int:
|
||||
return int(connection.execute(sa.text(f"SELECT COUNT(*) FROM {table}")).scalar_one())
|
||||
|
||||
|
||||
def _orphan_count(connection: sa.Connection) -> int:
|
||||
version_orphans = connection.execute(
|
||||
sa.text(
|
||||
"SELECT COUNT(*) FROM energy_contract_version v "
|
||||
"LEFT JOIN energy_contract c ON c.id = v.contract_id WHERE c.id IS NULL"
|
||||
)
|
||||
).scalar_one()
|
||||
cost_orphans = connection.execute(
|
||||
sa.text(
|
||||
"SELECT COUNT(*) FROM energy_cost_period p "
|
||||
"LEFT JOIN energy_contract_version v ON v.id = p.contract_version_id "
|
||||
"WHERE p.contract_version_id IS NOT NULL AND v.id IS NULL"
|
||||
)
|
||||
).scalar_one()
|
||||
return int(version_orphans) + int(cost_orphans)
|
||||
|
||||
|
||||
def _audit_scope_upgrade(connection: sa.Connection, before: dict[str, int], orphan_before: int) -> None:
|
||||
after = {table: _count(connection, table) for table in before}
|
||||
if after != before:
|
||||
raise RuntimeError("contract scope migration row-count audit failed")
|
||||
if _orphan_count(connection) != orphan_before:
|
||||
raise RuntimeError("contract scope migration FK audit failed")
|
||||
invalid_scope_count = connection.execute(
|
||||
sa.text("SELECT COUNT(*) FROM energy_contract WHERE scope IS NULL OR scope != 'electricity'")
|
||||
).scalar_one()
|
||||
if invalid_scope_count:
|
||||
raise RuntimeError("contract scope migration backfill audit failed")
|
||||
|
||||
# Kept on Alembic's Config attributes rather than an environment switch so
|
||||
# isolated migration tests can deterministically exercise the rollback
|
||||
# boundary without changing production behavior.
|
||||
failure_injector = op.get_context().config.attributes.get("m8_t12_post_ddl_audit_failure")
|
||||
if callable(failure_injector):
|
||||
failure_injector()
|
||||
|
||||
|
||||
def _apply_scope_schema() -> None:
|
||||
# SQLite batch mode reconstructs the table. The server default gives every
|
||||
# historical row its deterministic value during reconstruction.
|
||||
with op.batch_alter_table("energy_contract", schema=None) as batch_op:
|
||||
batch_op.add_column(
|
||||
sa.Column("scope", sa.String(length=32), nullable=False, server_default="electricity")
|
||||
)
|
||||
batch_op.create_index("ix_energy_contract_scope", ["scope"])
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
connection = op.get_bind()
|
||||
before = {
|
||||
table: _count(connection, table)
|
||||
for table in ("energy_contract", "energy_contract_version", "energy_cost_period")
|
||||
}
|
||||
orphan_before = _orphan_count(connection)
|
||||
|
||||
if connection.dialect.name != "sqlite":
|
||||
_apply_scope_schema()
|
||||
_audit_scope_upgrade(connection, before, orphan_before)
|
||||
return
|
||||
|
||||
# Alembic marks SQLite batch DDL as non-transactional. SQLite itself can
|
||||
# nevertheless atomically roll back CREATE/COPY/DROP/RENAME when an
|
||||
# explicit transaction owns the complete batch operation. Keep the audit
|
||||
# inside that boundary so a failed audit cannot strand a revision-17 DB
|
||||
# with a revision-18 table shape.
|
||||
connection.exec_driver_sql("BEGIN IMMEDIATE")
|
||||
try:
|
||||
_apply_scope_schema()
|
||||
_audit_scope_upgrade(connection, before, orphan_before)
|
||||
except BaseException:
|
||||
connection.exec_driver_sql("ROLLBACK")
|
||||
raise
|
||||
else:
|
||||
connection.exec_driver_sql("COMMIT")
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
# Schema reversibility is only exercised against isolated temporary test DBs.
|
||||
with op.batch_alter_table("energy_contract", schema=None) as batch_op:
|
||||
batch_op.drop_index("ix_energy_contract_scope")
|
||||
batch_op.drop_column("scope")
|
||||
@@ -0,0 +1,95 @@
|
||||
"""add generic commodity-scoped meter cost periods
|
||||
|
||||
Revision ID: 20260822_19_meter_cost_periods
|
||||
Revises: 20260822_18_contract_scopes
|
||||
Create Date: 2026-08-22 00:00:00.000000
|
||||
|
||||
This additive migration creates a separate audit ledger for non-electricity
|
||||
meter costs. It deliberately does not alter, migrate, or delete rows from the
|
||||
existing electricity-only energy_cost_period table.
|
||||
"""
|
||||
|
||||
from typing import Sequence, Union
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
|
||||
|
||||
revision: str = "20260822_19_meter_cost_periods"
|
||||
down_revision: Union[str, None] = "20260822_18_contract_scopes"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
class ExactDecimal(sa.TypeDecorator):
|
||||
"""Use SQLite text storage while retaining Numeric semantics elsewhere."""
|
||||
|
||||
impl = sa.Numeric
|
||||
cache_ok = True
|
||||
|
||||
def __init__(self, precision: int, scale: int) -> None:
|
||||
self.precision = precision
|
||||
self.scale = scale
|
||||
super().__init__(precision=precision, scale=scale)
|
||||
|
||||
def load_dialect_impl(self, dialect):
|
||||
if dialect.name == "sqlite":
|
||||
return dialect.type_descriptor(sa.String(self.precision + 2))
|
||||
return dialect.type_descriptor(sa.Numeric(self.precision, self.scale, asdecimal=True))
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.create_table(
|
||||
"meter_cost_period",
|
||||
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
|
||||
sa.Column("commodity", sa.String(length=32), nullable=False),
|
||||
sa.Column("period_start", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.Column("period_end", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.Column("meter_id", sa.Integer(), nullable=True),
|
||||
sa.Column("source_binding_id", sa.Integer(), nullable=True),
|
||||
sa.Column("contract_version_id", sa.Integer(), nullable=True),
|
||||
# SQLite NUMERIC coercion binds Decimal values as binary floats. Store
|
||||
# fixed-width decimal text there, while retaining Numeric semantics on
|
||||
# other supported dialects.
|
||||
sa.Column("quantity", ExactDecimal(15, 6), nullable=False),
|
||||
sa.Column("cost", ExactDecimal(15, 9), nullable=False),
|
||||
sa.Column("currency", sa.String(length=8), nullable=False),
|
||||
sa.Column("cost_breakdown", sa.JSON(), nullable=False),
|
||||
sa.Column("pricing_snapshot", sa.JSON(), nullable=False),
|
||||
sa.Column("quality", sa.String(length=32), nullable=False),
|
||||
sa.Column("degraded", sa.Boolean(), nullable=False, server_default=sa.false()),
|
||||
sa.Column("degraded_reason", sa.String(length=255), nullable=True),
|
||||
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.ForeignKeyConstraint(["meter_id"], ["meter.id"], ondelete="RESTRICT"),
|
||||
sa.ForeignKeyConstraint(
|
||||
["source_binding_id"], ["meter_source_binding.id"], ondelete="RESTRICT"
|
||||
),
|
||||
sa.ForeignKeyConstraint(
|
||||
["contract_version_id"], ["energy_contract_version.id"], ondelete="RESTRICT"
|
||||
),
|
||||
sa.CheckConstraint(
|
||||
"degraded OR (meter_id IS NOT NULL AND source_binding_id IS NOT NULL "
|
||||
"AND contract_version_id IS NOT NULL)",
|
||||
name="ck_meter_cost_period_normal_audit_links",
|
||||
),
|
||||
sa.CheckConstraint("period_end > period_start", name="ck_meter_cost_period_positive_interval"),
|
||||
sa.CheckConstraint(
|
||||
"NOT degraded OR (degraded_reason IS NOT NULL AND length(trim(degraded_reason)) > 0)",
|
||||
name="ck_meter_cost_period_degraded_reason",
|
||||
),
|
||||
sa.PrimaryKeyConstraint("id"),
|
||||
sa.UniqueConstraint("commodity", "period_start", name="uq_meter_cost_period_commodity_start"),
|
||||
)
|
||||
op.create_index(
|
||||
"ix_meter_cost_period_commodity_start", "meter_cost_period", ["commodity", "period_start"]
|
||||
)
|
||||
op.create_index(
|
||||
"ix_meter_cost_period_source_binding_id", "meter_cost_period", ["source_binding_id"]
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_index("ix_meter_cost_period_source_binding_id", table_name="meter_cost_period")
|
||||
op.drop_index("ix_meter_cost_period_commodity_start", table_name="meter_cost_period")
|
||||
op.drop_table("meter_cost_period")
|
||||
@@ -9,7 +9,7 @@ from sqlalchemy.orm import Session
|
||||
from app.api.routes.api.deps import require_csrf, require_session
|
||||
from app.config import Settings, get_settings
|
||||
from app.dependencies import get_app_settings, get_db
|
||||
from app.integrations.mqtt import MQTT_SETTINGS_KEYS, mqtt_manager
|
||||
from app.integrations.mqtt import MQTT_SETTINGS_KEYS, mqtt_manager, mqtt_test_client_id
|
||||
from app.schemas.config import (
|
||||
ConfigField,
|
||||
ConfigResponse,
|
||||
@@ -22,6 +22,7 @@ from app.schemas.config import (
|
||||
from app.services.auth import AuthenticatedSession
|
||||
from app.services.config_page import ConfigSaveError, build_config_sections, save_config_updates
|
||||
from app.services.email import EmailConfigurationError, EmailDeliveryError, send_smtp_test_email
|
||||
from app.services.tibber_prices import active_tibber_contract_exists, trigger_tibber_refresh
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -65,6 +66,10 @@ def put_config(
|
||||
# Detect whether any MQTT-related key is being submitted (non-secret change
|
||||
# or non-blank secret change) so we know to reconnect after saving.
|
||||
mqtt_keys_submitted = any(k.lower() in MQTT_SETTINGS_KEYS for k in body.updates)
|
||||
tibber_values_before = (
|
||||
settings.tibber_api_token,
|
||||
settings.tibber_home_id,
|
||||
)
|
||||
|
||||
try:
|
||||
save_config_updates(db, body.updates, settings)
|
||||
@@ -92,6 +97,13 @@ def put_config(
|
||||
from app.services.dsmr_ingest import apply_dsmr_subscription
|
||||
apply_dsmr_subscription(refreshed_settings)
|
||||
|
||||
tibber_values_changed = tibber_values_before != (
|
||||
refreshed_settings.tibber_api_token,
|
||||
refreshed_settings.tibber_home_id,
|
||||
)
|
||||
if tibber_values_changed and active_tibber_contract_exists(db):
|
||||
trigger_tibber_refresh()
|
||||
|
||||
sections_raw = build_config_sections(db, refreshed_settings)
|
||||
return ConfigUpdateResponse(sections=_sections_from_raw(sections_raw))
|
||||
|
||||
@@ -233,7 +245,7 @@ def _run_mqtt_test(settings: Settings) -> None:
|
||||
|
||||
client = mqtt.Client(
|
||||
callback_api_version=mqtt.CallbackAPIVersion.VERSION2,
|
||||
client_id="home-automation-test",
|
||||
client_id=mqtt_test_client_id(settings.mqtt_client_id),
|
||||
)
|
||||
|
||||
def _on_connect(
|
||||
|
||||
@@ -44,7 +44,7 @@ from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from datetime import UTC, datetime, timedelta
|
||||
from typing import Any
|
||||
from typing import Any, Literal
|
||||
|
||||
from fastapi import APIRouter, Depends, Query, status
|
||||
from fastapi.responses import JSONResponse
|
||||
@@ -133,13 +133,23 @@ def _manual_tariff_from_values(values: dict[str, Any]) -> ManualTariffSchema:
|
||||
)
|
||||
|
||||
|
||||
def _electricity_prices_response(response: PricesResponse) -> JSONResponse:
|
||||
"""Preserve the exact pre-scope electricity response body."""
|
||||
return JSONResponse(
|
||||
content=response.model_dump(
|
||||
mode="json", include={"kind", "currency", "points", "tariff"}
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# GET /api/energy/prices
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@router.get("/prices", response_model=PricesResponse)
|
||||
@router.get("/prices", response_model=PricesResponse, response_model_exclude_none=True)
|
||||
def get_prices(
|
||||
scope: Literal["electricity", "thermal"] = Query("electricity"),
|
||||
start: datetime | None = Query(
|
||||
default=None,
|
||||
description="Inclusive start of the time window (ISO 8601). "
|
||||
@@ -169,7 +179,7 @@ def get_prices(
|
||||
|
||||
Response ``points`` carries per-slot:
|
||||
- ``buy = total`` (Tibber all-inclusive price)
|
||||
- ``sell = total − energy_tax − sell_adjust`` (from active version values)
|
||||
- ``sell = total − energy_tax − sell_fee − sell_adjust`` (from active version values)
|
||||
- ``level`` (Tibber price level, may be null)
|
||||
|
||||
``tariff`` is null.
|
||||
@@ -186,6 +196,17 @@ def get_prices(
|
||||
"""
|
||||
now = datetime.now(UTC)
|
||||
|
||||
if scope == "thermal":
|
||||
version = active_contract_version_at(db, now, scope="thermal")
|
||||
if version is None:
|
||||
return PricesResponse(kind=None, currency="EUR", points=[], tariff=None)
|
||||
return PricesResponse(
|
||||
kind="district_heating", currency=version.contract.currency,
|
||||
contract_version_id=version.id, effective_from=_as_utc(version.effective_from),
|
||||
effective_to=_as_utc(version.effective_to) if version.effective_to else None,
|
||||
values=version.values, points=[], tariff=None,
|
||||
)
|
||||
|
||||
# Default window: today + tomorrow.
|
||||
if start is None:
|
||||
start = now.replace(hour=0, minute=0, second=0, microsecond=0)
|
||||
@@ -199,12 +220,12 @@ def get_prices(
|
||||
version = active_contract_version_at(db, start_utc)
|
||||
|
||||
if version is None:
|
||||
return PricesResponse(
|
||||
return _electricity_prices_response(PricesResponse(
|
||||
kind=None,
|
||||
currency="EUR",
|
||||
points=[],
|
||||
tariff=None,
|
||||
)
|
||||
))
|
||||
|
||||
contract = version.contract
|
||||
currency = contract.currency
|
||||
@@ -222,7 +243,8 @@ def get_prices(
|
||||
)
|
||||
rows = list(reversed(db.execute(stmt).scalars().all()))
|
||||
|
||||
# Derive sell price per-point using version values (energy_tax + sell_adjust).
|
||||
# Derive sell price per-point using version values
|
||||
# (energy_tax + sell_fee + sell_adjust).
|
||||
from decimal import Decimal
|
||||
|
||||
def _d(v: Any) -> Decimal:
|
||||
@@ -230,12 +252,13 @@ def get_prices(
|
||||
|
||||
energy = version.values.get("energy", {}) if version.values else {}
|
||||
energy_tax = _d(energy.get("energy_tax", 0))
|
||||
sell_fee = _d(energy.get("sell_fee", 0))
|
||||
sell_adjust = _d(energy.get("sell_adjust", 0))
|
||||
|
||||
points = []
|
||||
for row in rows:
|
||||
total = _d(row.total)
|
||||
sell = float(total - energy_tax - sell_adjust)
|
||||
sell = float(total - energy_tax - sell_fee - sell_adjust)
|
||||
points.append(
|
||||
PricePointSchema(
|
||||
starts_at=_as_utc(row.starts_at),
|
||||
@@ -245,30 +268,30 @@ def get_prices(
|
||||
)
|
||||
)
|
||||
|
||||
return PricesResponse(
|
||||
return _electricity_prices_response(PricesResponse(
|
||||
kind="tibber",
|
||||
currency=currency,
|
||||
points=points,
|
||||
tariff=None,
|
||||
)
|
||||
))
|
||||
|
||||
elif contract.kind == "manual":
|
||||
tariff = _manual_tariff_from_values(version.values or {})
|
||||
return PricesResponse(
|
||||
return _electricity_prices_response(PricesResponse(
|
||||
kind="manual",
|
||||
currency=currency,
|
||||
points=[],
|
||||
tariff=tariff,
|
||||
)
|
||||
))
|
||||
|
||||
else:
|
||||
# Unknown kind — return empty response gracefully.
|
||||
return PricesResponse(
|
||||
return _electricity_prices_response(PricesResponse(
|
||||
kind=contract.kind,
|
||||
currency=currency,
|
||||
points=[],
|
||||
tariff=None,
|
||||
)
|
||||
))
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@@ -46,7 +46,9 @@ from app.schemas.energy_contract import (
|
||||
from app.services.auth import AuthenticatedSession
|
||||
from app.services import timezone as _tz_mod
|
||||
from app.services.contracts import (
|
||||
CONTRACT_KIND_SCOPES,
|
||||
ContractVersionError,
|
||||
ContractScopeError,
|
||||
activate_contract,
|
||||
add_version,
|
||||
create_contract,
|
||||
@@ -54,6 +56,7 @@ from app.services.contracts import (
|
||||
get_contract_or_none,
|
||||
list_contracts,
|
||||
)
|
||||
from app.services.tibber_prices import trigger_tibber_refresh
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -98,6 +101,7 @@ def _contract_detail(db: Session, contract) -> ContractDetailResponse:
|
||||
id=contract.id,
|
||||
name=contract.name,
|
||||
kind=contract.kind,
|
||||
scope=contract.scope,
|
||||
active=contract.active,
|
||||
currency=contract.currency,
|
||||
created_at=contract.created_at,
|
||||
@@ -163,16 +167,22 @@ def get_profiles(
|
||||
|
||||
@router.get("/contracts", response_model=ContractListResponse)
|
||||
def list_energy_contracts(
|
||||
scope: str = "electricity",
|
||||
db: Session = Depends(get_db),
|
||||
_auth: AuthenticatedSession = Depends(require_session),
|
||||
) -> ContractListResponse:
|
||||
"""List all energy contracts with their active status.
|
||||
|
||||
Returns a flat list (no embedded version history); use
|
||||
Scope defaults to ``electricity`` for old clients. Returns a flat list (no embedded version history); use
|
||||
GET /api/energy/contracts/{id} to fetch the full version history for a
|
||||
specific contract.
|
||||
"""
|
||||
contracts = list_contracts(db)
|
||||
if scope not in set(CONTRACT_KIND_SCOPES.values()):
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
|
||||
detail=f"Unknown energy contract scope: {scope!r}",
|
||||
)
|
||||
contracts = list_contracts(db, scope=scope)
|
||||
items = [ContractResponse.model_validate(c) for c in contracts]
|
||||
return ContractListResponse(items=items, total=len(items))
|
||||
|
||||
@@ -208,10 +218,11 @@ def create_energy_contract(
|
||||
name=body.name,
|
||||
kind=body.kind,
|
||||
currency=body.currency,
|
||||
scope=body.scope,
|
||||
values=body.values,
|
||||
effective_from=effective_from,
|
||||
)
|
||||
except (ProfileNotFoundError, ProfileValidationError) as exc:
|
||||
except (ProfileNotFoundError, ProfileValidationError, ContractScopeError) as exc:
|
||||
_raise_422_for_profile_error(exc)
|
||||
|
||||
db.commit()
|
||||
@@ -256,13 +267,14 @@ def patch_energy_contract(
|
||||
"""Partially update a contract: rename or change activation status.
|
||||
|
||||
- ``name``: updates the human-readable label.
|
||||
- ``active=true``: activates this contract (all others are deactivated).
|
||||
- ``active=true``: activates this contract (same-scope contracts are deactivated).
|
||||
- ``active=false``: deactivates this contract (no effect on others).
|
||||
|
||||
At most one contract may be active at any time; the service layer enforces
|
||||
mutual exclusion.
|
||||
At most one contract may be active per scope; the service layer enforces
|
||||
scope-local mutual exclusion.
|
||||
"""
|
||||
contract = _get_contract_or_404(db, contract_id)
|
||||
was_active = contract.active
|
||||
|
||||
if body.name is not None:
|
||||
contract.name = body.name
|
||||
@@ -275,6 +287,8 @@ def patch_energy_contract(
|
||||
|
||||
db.commit()
|
||||
db.refresh(contract)
|
||||
if body.active is True and not was_active and contract.kind == "tibber":
|
||||
trigger_tibber_refresh()
|
||||
return _contract_detail(db, contract)
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,150 @@
|
||||
"""Authenticated API for the thermal 15-minute cost ledger."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import UTC, datetime, timedelta
|
||||
from decimal import Decimal
|
||||
from typing import Literal
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Query, status
|
||||
from sqlalchemy import func, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.api.routes.api.deps import require_csrf, require_session
|
||||
from app.dependencies import get_db
|
||||
from app.models.energy import MeterCostPeriod
|
||||
from app.schemas.meter_cost import (
|
||||
MeterCostPeriodSchema,
|
||||
MeterCostRecomputeResponse,
|
||||
MeterCostsResponse,
|
||||
ThermalCostSummaryResponse,
|
||||
)
|
||||
from app.services.auth import AuthenticatedSession
|
||||
from app.services.meter_cost import recompute_range, summarize
|
||||
from app.services.timezone import local_midnight_utc, local_now
|
||||
|
||||
router = APIRouter(prefix="/api/energy/meter-costs", tags=["api-energy"])
|
||||
|
||||
_LIMIT_MAX = 5000
|
||||
_RECOMPUTE_MAX_DAYS = 31
|
||||
_QUARTER = timedelta(minutes=15)
|
||||
|
||||
|
||||
def _utc(value: datetime) -> datetime:
|
||||
return value.replace(tzinfo=UTC) if value.tzinfo is None else value.astimezone(UTC)
|
||||
|
||||
|
||||
def _decimal_strings(value: object) -> object:
|
||||
if isinstance(value, dict):
|
||||
return {str(key): _decimal_strings(item) for key, item in value.items()}
|
||||
if isinstance(value, Decimal):
|
||||
return format(value, "f")
|
||||
return str(value) if isinstance(value, (int, float)) else value
|
||||
|
||||
|
||||
def _row_schema(row: MeterCostPeriod) -> MeterCostPeriodSchema:
|
||||
return MeterCostPeriodSchema(
|
||||
commodity=row.commodity,
|
||||
period_start=_utc(row.period_start), period_end=_utc(row.period_end),
|
||||
meter_id=row.meter_id, source_binding_id=row.source_binding_id,
|
||||
contract_version_id=row.contract_version_id, quantity=format(row.quantity, "f"),
|
||||
cost=format(row.cost, "f"), currency=row.currency,
|
||||
cost_breakdown=_decimal_strings(row.cost_breakdown),
|
||||
pricing_snapshot=_decimal_strings(row.pricing_snapshot), quality=row.quality,
|
||||
degraded=row.degraded, degraded_reason=row.degraded_reason,
|
||||
)
|
||||
|
||||
|
||||
@router.get("", response_model=MeterCostsResponse)
|
||||
def get_meter_costs(
|
||||
scope: Literal["thermal"] = Query("thermal"),
|
||||
commodity: Literal["heating", "hot_water"] | None = None,
|
||||
start: datetime | None = None,
|
||||
end: datetime | None = None,
|
||||
limit: int = Query(500, ge=1, le=_LIMIT_MAX),
|
||||
offset: int = Query(0, ge=0),
|
||||
db: Session = Depends(get_db),
|
||||
_auth: AuthenticatedSession = Depends(require_session),
|
||||
) -> MeterCostsResponse:
|
||||
"""List thermal rows in a half-open time window with stable pagination."""
|
||||
del scope
|
||||
if start is not None and end is not None and _utc(end) <= _utc(start):
|
||||
raise HTTPException(status.HTTP_422_UNPROCESSABLE_ENTITY, "'end' must be after 'start'.")
|
||||
clauses = []
|
||||
if commodity is not None:
|
||||
clauses.append(MeterCostPeriod.commodity == commodity)
|
||||
if start is not None:
|
||||
clauses.append(MeterCostPeriod.period_start >= _utc(start))
|
||||
if end is not None:
|
||||
clauses.append(MeterCostPeriod.period_start < _utc(end))
|
||||
total = db.scalar(select(func.count()).select_from(MeterCostPeriod).where(*clauses)) or 0
|
||||
rows = db.execute(
|
||||
select(MeterCostPeriod).where(*clauses).order_by(MeterCostPeriod.period_start, MeterCostPeriod.id)
|
||||
.offset(offset).limit(limit)
|
||||
).scalars().all()
|
||||
return MeterCostsResponse(items=[_row_schema(row) for row in rows], total=total)
|
||||
|
||||
|
||||
@router.get("/summary", response_model=ThermalCostSummaryResponse)
|
||||
def get_meter_cost_summary(
|
||||
scope: Literal["thermal"] = Query("thermal"),
|
||||
start: datetime | None = None,
|
||||
end: datetime | None = None,
|
||||
db: Session = Depends(get_db),
|
||||
_auth: AuthenticatedSession = Depends(require_session),
|
||||
) -> ThermalCostSummaryResponse:
|
||||
"""Summarize thermal variable and once-per-contract daily fixed costs."""
|
||||
del scope
|
||||
if start is None or end is None:
|
||||
today = local_now().date()
|
||||
start = start or local_midnight_utc(today)
|
||||
end = end or local_midnight_utc(today + timedelta(days=1))
|
||||
start, end = _utc(start), _utc(end)
|
||||
if end <= start:
|
||||
raise HTTPException(status.HTTP_422_UNPROCESSABLE_ENTITY, "'end' must be after 'start'.")
|
||||
result = summarize(db, start, end)
|
||||
breakdown = result["breakdown"]
|
||||
fixed_breakdown = result["fixed_breakdown"]
|
||||
fixed = result["fixed_cost"]
|
||||
return ThermalCostSummaryResponse(
|
||||
currency=result["currency"], heating=format(breakdown["heating"], "f"),
|
||||
hot_water_heating=format(breakdown["hot_water_heating"], "f"),
|
||||
hot_water=format(breakdown["hot_water"], "f"), hot_water_tax=format(breakdown["hot_water_tax"], "f"),
|
||||
variable_subtotal=format(result["variable_cost"], "f"),
|
||||
fixed_breakdown={key: format(value, "f") for key, value in fixed_breakdown.items()},
|
||||
fixed_subtotal=format(fixed, "f"),
|
||||
all_in=format(result["total_cost"], "f"), period_count=result["period_count"],
|
||||
degraded_count=result["degraded_count"],
|
||||
)
|
||||
|
||||
|
||||
@router.post("/recompute", response_model=MeterCostRecomputeResponse)
|
||||
def post_meter_cost_recompute(
|
||||
scope: Literal["thermal"] = Query("thermal"),
|
||||
start: datetime = Query(...), end: datetime = Query(...),
|
||||
db: Session = Depends(get_db), _auth: AuthenticatedSession = Depends(require_session),
|
||||
_csrf: None = Depends(require_csrf),
|
||||
) -> MeterCostRecomputeResponse:
|
||||
"""Atomically overwrite closed, UTC-quarter thermal rows in a bounded window."""
|
||||
del scope
|
||||
start, end = _utc(start), _utc(end)
|
||||
if end <= start or end - start > timedelta(days=_RECOMPUTE_MAX_DAYS):
|
||||
raise HTTPException(status.HTTP_422_UNPROCESSABLE_ENTITY, "invalid or overlarge recompute window")
|
||||
if start.minute % 15 or start.second or start.microsecond or end.minute % 15 or end.second or end.microsecond:
|
||||
raise HTTPException(status.HTTP_422_UNPROCESSABLE_ENTITY, "start and end must align to UTC quarters")
|
||||
if end > datetime.now(UTC):
|
||||
raise HTTPException(status.HTTP_422_UNPROCESSABLE_ENTITY, "recompute window must be closed")
|
||||
try:
|
||||
processed = recompute_range(db, start, end, commit=False)
|
||||
# Ensure pending upserts participate in this transaction before the
|
||||
# counts are read; a flush/query failure must still roll everything back.
|
||||
db.flush()
|
||||
rows = db.execute(select(MeterCostPeriod.degraded).where(
|
||||
MeterCostPeriod.period_start >= start, MeterCostPeriod.period_start < end
|
||||
)).scalars().all()
|
||||
degraded = sum(bool(value) for value in rows)
|
||||
db.commit()
|
||||
except Exception:
|
||||
db.rollback()
|
||||
raise
|
||||
return MeterCostRecomputeResponse(processed=processed, normal=len(rows) - degraded, degraded=degraded)
|
||||
@@ -0,0 +1,414 @@
|
||||
"""Authenticated HTTP contract for meter sources, channels, and bindings."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import UTC, datetime
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Query, Response, status
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.api.routes.api.deps import require_csrf, require_session
|
||||
from app.config import get_settings
|
||||
from app.dependencies import get_db
|
||||
from app.integrations.meter_sources import SourceProfileError, list_source_profiles, sanitize_source_config
|
||||
from app.models.energy import DsmrReading, Meter
|
||||
from app.models.meter_source import MeterSource, MeterSourceBinding, MeterSourceChannel, WarmteLinkReading
|
||||
from app.schemas.meter_source import (
|
||||
BindingCreate, BindingListResponse, BindingPatch, BindingResponse, ChannelBindingSummaryResponse,
|
||||
BindingTransferRequest, BindingTransferResponse,
|
||||
ChannelReadingResponse,
|
||||
ChannelReadingsResponse, CommoditiesResponse, CommodityResponse, DiscoverResponse,
|
||||
DiscoverChannelResponse,
|
||||
MeterSourceChannelListResponse, MeterSourceChannelResponse, MeterSourceCreate,
|
||||
MeterSourceListResponse, MeterSourcePatch, MeterSourceResponse, SourceConfigFieldResponse,
|
||||
SourceProfileResponse, SourceProfilesResponse,
|
||||
)
|
||||
from app.services.auth import AuthenticatedSession
|
||||
from app.services.config_page import build_runtime_settings
|
||||
from app.services.dsmr_ingest import apply_dsmr_subscription
|
||||
from app.services.meter_sources import (
|
||||
BindingNotFoundError, ChannelNotFoundError, MeterNotFoundError,
|
||||
MeterSourceError, SourceDeleteRestrictedError, SourceNotFoundError, create_binding,
|
||||
create_source, delete_source, list_bindings, list_sources, transfer_binding, update_binding, update_source,
|
||||
)
|
||||
from app.services.energy_cost import recompute_range as electricity_recompute_range
|
||||
from app.services import timezone as _tz_mod
|
||||
from app.services.warmtelink_worker import warmtelink_worker_manager
|
||||
|
||||
router = APIRouter(prefix="/api/energy", tags=["api-energy-meter-sources"])
|
||||
|
||||
|
||||
def _reconcile_runtimes_after_commit(db: Session) -> None:
|
||||
"""Best-effort runtime convergence after a durable source CRUD commit."""
|
||||
try:
|
||||
warmtelink_worker_manager.reconcile()
|
||||
except Exception:
|
||||
# The manager records individual source failures itself. Do not turn a
|
||||
# successful durable create/update/delete into a misleading HTTP 500.
|
||||
pass
|
||||
try:
|
||||
apply_dsmr_subscription(build_runtime_settings(db, get_settings()))
|
||||
except Exception:
|
||||
# DSMR owns independent source clients. Its failure must neither undo
|
||||
# durable CRUD nor prevent the WarmteLink manager from converging.
|
||||
pass
|
||||
finally:
|
||||
# DSMR health callbacks use short independent sessions. Make a CRUD
|
||||
# response observe any durable status change they just committed.
|
||||
db.expire_all()
|
||||
|
||||
|
||||
def _as_utc(value: datetime) -> datetime:
|
||||
if value.tzinfo is None:
|
||||
return value.replace(tzinfo=_tz_mod.local_tz()).astimezone(UTC)
|
||||
return value.astimezone(UTC)
|
||||
|
||||
|
||||
def _source_or_404(db: Session, uuid: str) -> MeterSource:
|
||||
source = db.execute(select(MeterSource).where(MeterSource.uuid == uuid)).scalar_one_or_none()
|
||||
if source is None:
|
||||
raise HTTPException(status_code=404, detail="Meter source not found.")
|
||||
return source
|
||||
|
||||
|
||||
def _channel_or_404(db: Session, source: MeterSource, uuid: str) -> MeterSourceChannel:
|
||||
channel = db.execute(
|
||||
select(MeterSourceChannel).where(
|
||||
MeterSourceChannel.uuid == uuid, MeterSourceChannel.source_id == source.id
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
if channel is None:
|
||||
raise HTTPException(status_code=404, detail="Meter source channel not found.")
|
||||
return channel
|
||||
|
||||
|
||||
def _source_response(source: MeterSource) -> MeterSourceResponse:
|
||||
return MeterSourceResponse(
|
||||
uuid=source.uuid, name=source.name, kind=source.kind, enabled=source.enabled,
|
||||
config=sanitize_source_config(source.kind, source.config), status=source.status,
|
||||
last_seen_at=source.last_seen_at, last_error=source.last_error,
|
||||
created_at=source.created_at, updated_at=source.updated_at,
|
||||
)
|
||||
|
||||
|
||||
def binding_response(binding: MeterSourceBinding) -> BindingResponse:
|
||||
return BindingResponse(
|
||||
uuid=binding.uuid, meter_id=binding.meter_id, source_channel_uuid=binding.channel.uuid,
|
||||
source_uuid=binding.channel.source.uuid, started_at=binding.started_at, ended_at=binding.ended_at,
|
||||
created_at=binding.created_at, updated_at=binding.updated_at,
|
||||
)
|
||||
|
||||
|
||||
def _binding_error(exc: MeterSourceError) -> HTTPException:
|
||||
if isinstance(exc, (SourceNotFoundError, ChannelNotFoundError, MeterNotFoundError, BindingNotFoundError)):
|
||||
return HTTPException(status_code=404, detail=str(exc))
|
||||
return HTTPException(status_code=422, detail=str(exc))
|
||||
|
||||
|
||||
def _recompute_binding_commodity(db: Session, commodity: str, start: datetime) -> None:
|
||||
end = datetime.now(UTC)
|
||||
if start >= end:
|
||||
return
|
||||
if commodity == "electricity":
|
||||
electricity_recompute_range(db, start, end, commit=False, strict=True)
|
||||
else:
|
||||
from app.services.meter_cost import recompute_range
|
||||
recompute_range(db, start, end, commit=False)
|
||||
|
||||
|
||||
def _republish_after_commit(db: Session) -> None:
|
||||
try:
|
||||
from app.services.ha_discovery import publish_discovery
|
||||
publish_discovery(db)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
@router.get("/source-profiles", response_model=SourceProfilesResponse)
|
||||
def source_profiles(_auth: AuthenticatedSession = Depends(require_session)) -> SourceProfilesResponse:
|
||||
"""Return profile metadata; default secrets are never populated with stored values."""
|
||||
profiles = []
|
||||
for profile in list_source_profiles():
|
||||
fields = [
|
||||
SourceConfigFieldResponse(name=f.name, value_type=f.value_type.__name__, default=f.default,
|
||||
required=f.required, secret=f.secret)
|
||||
for f in profile.fields
|
||||
]
|
||||
profiles.append(SourceProfileResponse(
|
||||
kind=profile.kind, fields=fields,
|
||||
defaults={f.name: f.default for f in profile.fields if not f.required},
|
||||
capabilities=sorted(profile.capabilities), allowed_units=sorted(profile.allowed_units),
|
||||
))
|
||||
return SourceProfilesResponse(items=profiles)
|
||||
|
||||
|
||||
@router.get("/commodities", response_model=CommoditiesResponse)
|
||||
def commodities(_auth: AuthenticatedSession = Depends(require_session)) -> CommoditiesResponse:
|
||||
return CommoditiesResponse(items=[
|
||||
CommodityResponse(key="electricity", unit="kWh", capabilities=["meter", "binding", "cost"]),
|
||||
CommodityResponse(key="heating", unit="GJ", capabilities=["meter", "binding"]),
|
||||
CommodityResponse(key="hot_water", unit="m³", capabilities=["meter", "binding"]),
|
||||
])
|
||||
|
||||
|
||||
@router.get("/sources", response_model=MeterSourceListResponse)
|
||||
def get_sources(db: Session = Depends(get_db), _auth: AuthenticatedSession = Depends(require_session)) -> MeterSourceListResponse:
|
||||
items = [_source_response(source) for source in list_sources(db)]
|
||||
return MeterSourceListResponse(items=items, total=len(items))
|
||||
|
||||
|
||||
@router.post("/sources", response_model=MeterSourceResponse, status_code=status.HTTP_201_CREATED)
|
||||
def post_source(body: MeterSourceCreate, db: Session = Depends(get_db),
|
||||
_auth: AuthenticatedSession = Depends(require_session), _csrf: None = Depends(require_csrf)) -> MeterSourceResponse:
|
||||
try:
|
||||
source = create_source(db, name=body.name, kind=body.kind, config=body.config, enabled=body.enabled)
|
||||
db.commit()
|
||||
_reconcile_runtimes_after_commit(db)
|
||||
return _source_response(_source_or_404(db, source.uuid))
|
||||
except (SourceProfileError, MeterSourceError) as exc:
|
||||
db.rollback()
|
||||
raise HTTPException(status_code=422, detail=str(exc)) from exc
|
||||
|
||||
|
||||
@router.get("/sources/{source_uuid}", response_model=MeterSourceResponse)
|
||||
def get_source_detail(source_uuid: str, db: Session = Depends(get_db),
|
||||
_auth: AuthenticatedSession = Depends(require_session)) -> MeterSourceResponse:
|
||||
return _source_response(_source_or_404(db, source_uuid))
|
||||
|
||||
|
||||
@router.patch("/sources/{source_uuid}", response_model=MeterSourceResponse)
|
||||
def patch_source(source_uuid: str, body: MeterSourcePatch, db: Session = Depends(get_db),
|
||||
_auth: AuthenticatedSession = Depends(require_session), _csrf: None = Depends(require_csrf)) -> MeterSourceResponse:
|
||||
source = _source_or_404(db, source_uuid)
|
||||
try:
|
||||
updated = update_source(db, source.id, name=body.name, enabled=body.enabled, config_patch=body.config)
|
||||
db.commit()
|
||||
_reconcile_runtimes_after_commit(db)
|
||||
return _source_response(_source_or_404(db, updated.uuid))
|
||||
except (SourceProfileError, MeterSourceError) as exc:
|
||||
db.rollback()
|
||||
raise _binding_error(exc) from exc
|
||||
|
||||
|
||||
@router.delete(
|
||||
"/sources/{source_uuid}", status_code=status.HTTP_204_NO_CONTENT, response_model=None
|
||||
)
|
||||
def remove_source(source_uuid: str, db: Session = Depends(get_db),
|
||||
_auth: AuthenticatedSession = Depends(require_session), _csrf: None = Depends(require_csrf)) -> None:
|
||||
source = _source_or_404(db, source_uuid)
|
||||
# DSMR readings are not a relationship on MeterSource to avoid loading a large history.
|
||||
if db.execute(select(DsmrReading.id).where(DsmrReading.meter_source_id == source.id).limit(1)).scalar() is not None:
|
||||
raise HTTPException(status_code=409, detail="Meter source has dependent readings.")
|
||||
try:
|
||||
delete_source(db, source.id)
|
||||
db.commit()
|
||||
_reconcile_runtimes_after_commit(db)
|
||||
return Response(status_code=status.HTTP_204_NO_CONTENT)
|
||||
except SourceDeleteRestrictedError as exc:
|
||||
db.rollback()
|
||||
raise HTTPException(status_code=409, detail=str(exc)) from exc
|
||||
|
||||
|
||||
@router.post("/sources/{source_uuid}/discover", response_model=DiscoverResponse)
|
||||
def discover_source(source_uuid: str, db: Session = Depends(get_db),
|
||||
_auth: AuthenticatedSession = Depends(require_session), _csrf: None = Depends(require_csrf)) -> DiscoverResponse:
|
||||
source = _source_or_404(db, source_uuid)
|
||||
if source.kind == "warmtelink_serial":
|
||||
if not source.enabled:
|
||||
return DiscoverResponse(
|
||||
requested=False, supported=True, status="error",
|
||||
detail="The WarmteLink source is disabled.", channels=_discover_channels(db, source),
|
||||
)
|
||||
# This merely schedules lifecycle convergence. It never opens a serial
|
||||
# descriptor or waits for a frame in the request thread; the one managed
|
||||
# worker remains the sole owner of serial I/O and can keep reconnecting.
|
||||
request = warmtelink_worker_manager.request_discovery(source.id)
|
||||
if request.completed.is_set():
|
||||
# A worker may have accepted a frame during the bounded wait.
|
||||
# Refresh only durable accepted metadata, never candidates/raw data.
|
||||
db.expire_all()
|
||||
source = _source_or_404(db, source_uuid)
|
||||
return DiscoverResponse(
|
||||
requested=request.status != "error", supported=True, status=request.status,
|
||||
request_id=request.request_id or None, detail=request.detail,
|
||||
channels=_discover_channels(db, source),
|
||||
)
|
||||
return DiscoverResponse(requested=False, supported=True, status="managed_by_runtime",
|
||||
detail="This source is discovered by its runtime subscription; no connection was opened.")
|
||||
|
||||
|
||||
@router.get("/sources/{source_uuid}/channels", response_model=MeterSourceChannelListResponse)
|
||||
def source_channels(source_uuid: str, db: Session = Depends(get_db),
|
||||
_auth: AuthenticatedSession = Depends(require_session)) -> MeterSourceChannelListResponse:
|
||||
source = _source_or_404(db, source_uuid)
|
||||
channels = db.execute(select(MeterSourceChannel).where(MeterSourceChannel.source_id == source.id)).scalars().all()
|
||||
items = []
|
||||
for channel in channels:
|
||||
bindings = list_bindings(db, channel_id=channel.id)
|
||||
meter_ids = [binding.meter_id for binding in bindings]
|
||||
items.append(MeterSourceChannelResponse(
|
||||
uuid=channel.uuid, label=channel.label, suggested_commodity=channel.suggested_commodity,
|
||||
unit=channel.unit, device_type=channel.device_type, latest_value=channel.latest_value,
|
||||
latest_at=channel.latest_at, latest_quality=channel.latest_quality, binding_count=len(bindings),
|
||||
bound_meter_ids=meter_ids,
|
||||
binding_summary=ChannelBindingSummaryResponse(count=len(bindings), meter_ids=meter_ids),
|
||||
))
|
||||
return MeterSourceChannelListResponse(items=items, total=len(items), source_status=source.status)
|
||||
|
||||
|
||||
@router.get("/sources/{source_uuid}/channels/{channel_uuid}/readings", response_model=ChannelReadingsResponse)
|
||||
def channel_readings(source_uuid: str, channel_uuid: str, limit: int = Query(default=100, ge=1, le=1000),
|
||||
from_: datetime | None = Query(default=None, alias="from"),
|
||||
to: datetime | None = Query(default=None), db: Session = Depends(get_db),
|
||||
_auth: AuthenticatedSession = Depends(require_session)) -> ChannelReadingsResponse:
|
||||
source = _source_or_404(db, source_uuid)
|
||||
channel = _channel_or_404(db, source, channel_uuid)
|
||||
if from_ is not None and to is not None and _as_utc(from_) >= _as_utc(to):
|
||||
raise HTTPException(status_code=422, detail="'from' must be earlier than 'to'.")
|
||||
if source.kind == "warmtelink_serial":
|
||||
statement = select(WarmteLinkReading).where(WarmteLinkReading.channel_id == channel.id)
|
||||
model = WarmteLinkReading
|
||||
else:
|
||||
# DSMR remains a source-level protocol history. Its channel is the
|
||||
# public electricity identity, while payload/telegram diagnostics stay
|
||||
# private to ingestion and the legacy latest endpoint.
|
||||
statement = select(DsmrReading).where(DsmrReading.meter_source_id == source.id)
|
||||
model = DsmrReading
|
||||
if from_ is not None:
|
||||
statement = statement.where(model.recorded_at >= _as_utc(from_))
|
||||
if to is not None:
|
||||
statement = statement.where(model.recorded_at < _as_utc(to))
|
||||
rows = list(db.execute(statement.order_by(model.recorded_at.asc()).limit(limit)).scalars())
|
||||
return ChannelReadingsResponse(
|
||||
items=[ChannelReadingResponse(
|
||||
recorded_at=row.recorded_at,
|
||||
value=getattr(row, "value", None), quality=getattr(row, "quality", None),
|
||||
) for row in rows],
|
||||
total=len(rows),
|
||||
)
|
||||
|
||||
|
||||
def _discover_channels(db: Session, source: MeterSource) -> list[DiscoverChannelResponse]:
|
||||
"""Return only public, accepted channel metadata for discover responses."""
|
||||
return [
|
||||
DiscoverChannelResponse(
|
||||
uuid=channel.uuid, label=channel.label, unit=channel.unit,
|
||||
latest_value=channel.latest_value, latest_at=channel.latest_at,
|
||||
latest_quality=channel.latest_quality,
|
||||
)
|
||||
for channel in db.execute(
|
||||
select(MeterSourceChannel).where(MeterSourceChannel.source_id == source.id)
|
||||
).scalars()
|
||||
]
|
||||
|
||||
|
||||
@router.get("/meters/{meter_id}/bindings", response_model=BindingListResponse)
|
||||
def meter_bindings(meter_id: int, db: Session = Depends(get_db),
|
||||
_auth: AuthenticatedSession = Depends(require_session)) -> BindingListResponse:
|
||||
if db.get(Meter, meter_id) is None:
|
||||
raise HTTPException(status_code=404, detail="Meter not found.")
|
||||
items = [binding_response(binding) for binding in list_bindings(db, meter_id=meter_id)]
|
||||
return BindingListResponse(items=items, total=len(items))
|
||||
|
||||
|
||||
@router.post("/meters/{meter_id}/bindings", response_model=BindingResponse, status_code=status.HTTP_201_CREATED)
|
||||
def post_meter_binding(meter_id: int, body: BindingCreate, db: Session = Depends(get_db),
|
||||
_auth: AuthenticatedSession = Depends(require_session), _csrf: None = Depends(require_csrf)) -> BindingResponse:
|
||||
channel = db.execute(select(MeterSourceChannel).where(MeterSourceChannel.uuid == body.source_channel_uuid)).scalar_one_or_none()
|
||||
if channel is None:
|
||||
raise HTTPException(status_code=404, detail="Meter source channel not found.")
|
||||
try:
|
||||
binding = create_binding(db, meter_id=meter_id, channel_id=channel.id, started_at=_as_utc(body.started_at),
|
||||
ended_at=_as_utc(body.ended_at) if body.ended_at else None)
|
||||
meter = db.get(Meter, meter_id)
|
||||
db.flush()
|
||||
_recompute_binding_commodity(db, meter.commodity, _as_utc(body.started_at))
|
||||
db.commit()
|
||||
db.refresh(binding)
|
||||
_republish_after_commit(db)
|
||||
return binding_response(binding)
|
||||
except MeterSourceError as exc:
|
||||
db.rollback()
|
||||
raise _binding_error(exc) from exc
|
||||
except Exception:
|
||||
db.rollback()
|
||||
raise
|
||||
|
||||
|
||||
@router.patch("/bindings/{binding_uuid}", response_model=BindingResponse)
|
||||
def patch_binding(binding_uuid: str, body: BindingPatch, db: Session = Depends(get_db),
|
||||
_auth: AuthenticatedSession = Depends(require_session), _csrf: None = Depends(require_csrf)) -> BindingResponse:
|
||||
binding = db.execute(select(MeterSourceBinding).where(MeterSourceBinding.uuid == binding_uuid)).scalar_one_or_none()
|
||||
if binding is None:
|
||||
raise HTTPException(status_code=404, detail="Meter source binding not found.")
|
||||
try:
|
||||
# ``ended_at`` has three meaningful states in the service layer: omitted
|
||||
# keeps the existing boundary, null reopens the interval, and a datetime
|
||||
# changes the exclusive end. Do not collapse omitted into null here.
|
||||
changes: dict[str, datetime | None] = {}
|
||||
if "started_at" in body.model_fields_set:
|
||||
changes["started_at"] = _as_utc(body.started_at) if body.started_at is not None else None
|
||||
if "ended_at" in body.model_fields_set:
|
||||
changes["ended_at"] = _as_utc(body.ended_at) if body.ended_at is not None else None
|
||||
old_started_at = _as_utc(binding.started_at)
|
||||
old_ended_at = _as_utc(binding.ended_at) if binding.ended_at is not None else None
|
||||
updated = update_binding(db, binding.id, **changes)
|
||||
meter = db.get(Meter, updated.meter_id)
|
||||
if "started_at" in changes:
|
||||
earliest = min(old_started_at, _as_utc(updated.started_at))
|
||||
elif "ended_at" in changes:
|
||||
new_ended_at = _as_utc(updated.ended_at) if updated.ended_at is not None else None
|
||||
changed_ends = [value for value in (old_ended_at, new_ended_at) if value is not None]
|
||||
earliest = min(changed_ends) if changed_ends else old_started_at
|
||||
else:
|
||||
earliest = old_started_at
|
||||
db.flush()
|
||||
_recompute_binding_commodity(db, meter.commodity, earliest)
|
||||
db.commit()
|
||||
db.refresh(updated)
|
||||
_republish_after_commit(db)
|
||||
return binding_response(updated)
|
||||
except MeterSourceError as exc:
|
||||
db.rollback()
|
||||
raise _binding_error(exc) from exc
|
||||
except Exception:
|
||||
db.rollback()
|
||||
raise
|
||||
|
||||
|
||||
@router.post("/meters/{meter_id}/bindings/transfer", response_model=BindingTransferResponse)
|
||||
def post_binding_transfer(
|
||||
meter_id: int, body: BindingTransferRequest, db: Session = Depends(get_db),
|
||||
_auth: AuthenticatedSession = Depends(require_session), _csrf: None = Depends(require_csrf),
|
||||
) -> BindingTransferResponse:
|
||||
source = db.execute(select(MeterSourceBinding).where(
|
||||
MeterSourceBinding.uuid == body.from_binding_uuid
|
||||
)).scalar_one_or_none()
|
||||
channel = db.execute(select(MeterSourceChannel).where(
|
||||
MeterSourceChannel.uuid == body.to_source_channel_uuid
|
||||
)).scalar_one_or_none()
|
||||
if source is None or channel is None:
|
||||
raise HTTPException(status_code=404, detail="Meter source binding or channel not found.")
|
||||
effective_at = _as_utc(body.effective_at)
|
||||
try:
|
||||
closed, created = transfer_binding(db, target_meter_id=meter_id, from_binding_id=source.id,
|
||||
to_channel_id=channel.id, effective_at=effective_at)
|
||||
meter = db.get(Meter, meter_id)
|
||||
if closed.meter_id == meter.id:
|
||||
earliest = effective_at
|
||||
else:
|
||||
earliest = min(_as_utc(closed.ended_at), effective_at)
|
||||
db.flush()
|
||||
_recompute_binding_commodity(db, meter.commodity, earliest)
|
||||
db.commit()
|
||||
db.refresh(closed)
|
||||
db.refresh(created)
|
||||
except MeterSourceError as exc:
|
||||
db.rollback()
|
||||
raise _binding_error(exc) from exc
|
||||
except Exception:
|
||||
db.rollback()
|
||||
raise
|
||||
_republish_after_commit(db)
|
||||
return BindingTransferResponse(closed_binding=binding_response(closed), created_binding=binding_response(created))
|
||||
+144
-26
@@ -48,16 +48,27 @@ from typing import Optional
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, status
|
||||
from sqlalchemy.orm import Session
|
||||
from sqlalchemy import select
|
||||
|
||||
from app.api.routes.api.deps import require_csrf, require_session
|
||||
from app.dependencies import get_db
|
||||
from app.models.energy import Meter
|
||||
from app.models.meter_source import MeterSourceChannel
|
||||
from app.schemas.meter import (
|
||||
MeterCloseRequest,
|
||||
MeterDeclareRequest,
|
||||
MeterBindingSummary,
|
||||
MeterListResponse,
|
||||
MeterPatchRequest,
|
||||
MeterResponse,
|
||||
)
|
||||
from app.services.meter_sources import (
|
||||
ChannelNotFoundError,
|
||||
MeterSourceError,
|
||||
create_binding,
|
||||
create_binding_for_meter_swap,
|
||||
close_open_bindings_for_meter,
|
||||
)
|
||||
from app.services import timezone as _tz_mod
|
||||
from app.services.auth import AuthenticatedSession
|
||||
from app.services.energy_cost import recompute_range
|
||||
@@ -65,6 +76,7 @@ from app.services.meters import (
|
||||
MeterIntervalError,
|
||||
MeterOverlapError,
|
||||
declare_meter,
|
||||
close_meter,
|
||||
list_meters,
|
||||
update_meter,
|
||||
)
|
||||
@@ -142,7 +154,7 @@ def _trigger_recompute(db: Session, start: datetime, label: str) -> int:
|
||||
# started_at is in the future — nothing to recompute.
|
||||
logger.info("%s: started_at (%s) is in the future, skipping recompute.", label, start)
|
||||
return 0
|
||||
n = recompute_range(db, start, end)
|
||||
n = recompute_range(db, start, end, commit=False, strict=True)
|
||||
logger.info(
|
||||
"%s: recomputed %d period(s) in window [%s, %s).",
|
||||
label,
|
||||
@@ -153,6 +165,33 @@ def _trigger_recompute(db: Session, start: datetime, label: str) -> int:
|
||||
return n
|
||||
|
||||
|
||||
def _recompute_commodity(db: Session, commodity: str, start: datetime, label: str) -> int:
|
||||
if commodity == "electricity":
|
||||
return _trigger_recompute(db, start, label)
|
||||
from app.services.meter_cost import recompute_range as thermal_recompute_range
|
||||
|
||||
end = datetime.now(UTC)
|
||||
if start >= end:
|
||||
return 0
|
||||
return thermal_recompute_range(db, start, end, commit=False)
|
||||
|
||||
|
||||
def _meter_response(meter: Meter) -> MeterResponse:
|
||||
"""Serialize meter plus binding summaries without exposing source config."""
|
||||
response = MeterResponse.model_validate(meter)
|
||||
response.bindings = [
|
||||
MeterBindingSummary(
|
||||
uuid=binding.uuid,
|
||||
source_channel_uuid=binding.channel.uuid,
|
||||
source_uuid=binding.channel.source.uuid,
|
||||
started_at=binding.started_at,
|
||||
ended_at=binding.ended_at,
|
||||
)
|
||||
for binding in meter.source_bindings
|
||||
]
|
||||
return response
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# GET /api/energy/meters
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -170,7 +209,7 @@ def list_energy_meters(
|
||||
has the latest ``started_at``.
|
||||
"""
|
||||
meters = list_meters(db)
|
||||
items = [MeterResponse.model_validate(m) for m in meters]
|
||||
items = [_meter_response(m) for m in meters]
|
||||
return MeterListResponse(items=items, total=len(items))
|
||||
|
||||
|
||||
@@ -211,6 +250,9 @@ def declare_energy_meter(
|
||||
started_at_utc = _localize_started_at(body.started_at)
|
||||
|
||||
try:
|
||||
old_meter = db.execute(
|
||||
select(Meter).where(Meter.commodity == body.commodity, Meter.ended_at.is_(None))
|
||||
).scalar_one_or_none()
|
||||
new_meter = declare_meter(
|
||||
db,
|
||||
label=body.label,
|
||||
@@ -219,20 +261,62 @@ def declare_energy_meter(
|
||||
commodity=body.commodity,
|
||||
note=body.note,
|
||||
)
|
||||
except MeterOverlapError as exc:
|
||||
db.flush() # assign PK before an optional binding and recompute
|
||||
# A closed predecessor must never retain an open interval. For a
|
||||
# meter swap with no selected channel we can safely hand off exactly
|
||||
# one compatible open channel; ambiguity is fail-closed.
|
||||
auto_channel = None
|
||||
if body.source_channel_uuid is None and old_meter is not None and body.reason.value == "meter_swap":
|
||||
candidates = [b for b in old_meter.source_bindings if b.ended_at is None and b.channel.unit == {"electricity": "kWh", "heating": "GJ", "hot_water": "m³"}.get(body.commodity)]
|
||||
if len(candidates) > 1:
|
||||
raise MeterSourceError("Meter swap has ambiguous open bindings; select a channel explicitly.")
|
||||
if len(candidates) == 1:
|
||||
auto_channel = candidates[0].channel
|
||||
if body.source_channel_uuid is not None:
|
||||
channel = db.execute(
|
||||
select(MeterSourceChannel).where(MeterSourceChannel.uuid == body.source_channel_uuid)
|
||||
).scalar_one_or_none()
|
||||
if channel is None:
|
||||
raise ChannelNotFoundError("Meter source channel was not found.")
|
||||
if body.reason.value == "meter_swap":
|
||||
create_binding_for_meter_swap(
|
||||
db,
|
||||
old_meter_id=old_meter.id if old_meter is not None else None,
|
||||
new_meter_id=new_meter.id,
|
||||
channel_id=channel.id,
|
||||
started_at=started_at_utc,
|
||||
)
|
||||
else:
|
||||
create_binding(
|
||||
db,
|
||||
meter_id=new_meter.id,
|
||||
channel_id=channel.id,
|
||||
started_at=started_at_utc,
|
||||
)
|
||||
elif auto_channel is not None:
|
||||
create_binding_for_meter_swap(db, old_meter_id=old_meter.id, new_meter_id=new_meter.id,
|
||||
channel_id=auto_channel.id, started_at=started_at_utc)
|
||||
if old_meter is not None:
|
||||
close_open_bindings_for_meter(db, old_meter.id, ended_at=started_at_utc)
|
||||
|
||||
# Keep recompute in this transaction: a failure must not leave a new
|
||||
# meter, its predecessor, or either binding at a half-applied boundary.
|
||||
now = datetime.now(UTC)
|
||||
if started_at_utc < now:
|
||||
db.flush()
|
||||
_recompute_commodity(db, body.commodity, started_at_utc, "POST /api/energy/meters")
|
||||
db.commit()
|
||||
except (MeterIntervalError, MeterOverlapError, MeterSourceError) as exc:
|
||||
db.rollback()
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
|
||||
status_code=(status.HTTP_404_NOT_FOUND if isinstance(exc, ChannelNotFoundError)
|
||||
else status.HTTP_422_UNPROCESSABLE_ENTITY),
|
||||
detail=str(exc),
|
||||
)
|
||||
except Exception:
|
||||
db.rollback()
|
||||
raise
|
||||
|
||||
db.flush() # assign PK before recompute (recompute uses session, needs meter in DB)
|
||||
|
||||
# Retroactive recompute: re-judge attribution from the new boundary onward.
|
||||
now = datetime.now(UTC)
|
||||
if started_at_utc < now:
|
||||
_trigger_recompute(db, started_at_utc, "POST /api/energy/meters")
|
||||
|
||||
db.commit()
|
||||
db.refresh(new_meter)
|
||||
|
||||
# Trigger HA discovery re-publish so the new active meter's energy-cost
|
||||
@@ -247,7 +331,31 @@ def declare_energy_meter(
|
||||
new_meter.label,
|
||||
started_at_utc.isoformat(),
|
||||
)
|
||||
return MeterResponse.model_validate(new_meter)
|
||||
return _meter_response(new_meter)
|
||||
|
||||
|
||||
@router.post("/meters/{meter_id}/close", response_model=MeterResponse)
|
||||
def close_energy_meter(
|
||||
meter_id: int, body: MeterCloseRequest, db: Session = Depends(get_db),
|
||||
_auth: AuthenticatedSession = Depends(require_session), _csrf: None = Depends(require_csrf),
|
||||
) -> MeterResponse:
|
||||
meter = _get_meter_or_404(db, meter_id)
|
||||
boundary = _localize_started_at(body.ended_at)
|
||||
try:
|
||||
close_meter(db, meter, ended_at=boundary)
|
||||
close_open_bindings_for_meter(db, meter.id, ended_at=boundary)
|
||||
db.flush()
|
||||
_recompute_commodity(db, meter.commodity, boundary, f"POST /api/energy/meters/{meter_id}/close")
|
||||
db.commit()
|
||||
except (MeterIntervalError, MeterSourceError) as exc:
|
||||
db.rollback()
|
||||
raise HTTPException(status_code=422, detail=str(exc)) from exc
|
||||
except Exception:
|
||||
db.rollback()
|
||||
raise
|
||||
db.refresh(meter)
|
||||
_trigger_discovery_republish(db)
|
||||
return _meter_response(meter)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -298,23 +406,33 @@ def patch_energy_meter(
|
||||
note=body.note,
|
||||
started_at=new_started_at_utc,
|
||||
)
|
||||
|
||||
# Retroactive recompute if started_at was changed.
|
||||
if new_started_at_utc is not None and old_started_at is not None:
|
||||
# Normalise old_started_at to UTC-aware for comparison.
|
||||
if old_started_at.tzinfo is None:
|
||||
old_started_at = old_started_at.replace(tzinfo=UTC)
|
||||
# Window = [min(old, new), now) — covers all periods whose attribution
|
||||
# may have changed due to the boundary shift in either direction.
|
||||
window_start = min(old_started_at, new_started_at_utc)
|
||||
db.flush()
|
||||
_recompute_commodity(
|
||||
db,
|
||||
meter.commodity,
|
||||
window_start,
|
||||
f"PATCH /api/energy/meters/{meter_id}",
|
||||
)
|
||||
|
||||
db.commit()
|
||||
except MeterIntervalError as exc:
|
||||
db.rollback()
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
|
||||
detail=str(exc),
|
||||
)
|
||||
|
||||
# Retroactive recompute if started_at was changed.
|
||||
if new_started_at_utc is not None and old_started_at is not None:
|
||||
# Normalise old_started_at to UTC-aware for comparison.
|
||||
if old_started_at.tzinfo is None:
|
||||
old_started_at = old_started_at.replace(tzinfo=UTC)
|
||||
# Window = [min(old, new), now) — covers all periods whose attribution
|
||||
# may have changed due to the boundary shift in either direction.
|
||||
window_start = min(old_started_at, new_started_at_utc)
|
||||
_trigger_recompute(db, window_start, f"PATCH /api/energy/meters/{meter_id}")
|
||||
|
||||
db.commit()
|
||||
except Exception:
|
||||
db.rollback()
|
||||
raise
|
||||
db.refresh(meter)
|
||||
|
||||
# Trigger HA discovery re-publish so label renames on the active meter
|
||||
@@ -328,4 +446,4 @@ def patch_energy_meter(
|
||||
meter.label,
|
||||
meter.started_at,
|
||||
)
|
||||
return MeterResponse.model_validate(meter)
|
||||
return _meter_response(meter)
|
||||
|
||||
@@ -488,6 +488,7 @@ def test_read(
|
||||
device.port,
|
||||
device.unit_id,
|
||||
[{"start": b.start, "count": b.count} for b in profile.blocks],
|
||||
function_code=profile.function_code,
|
||||
)
|
||||
payload: dict[str, Any] = decode_profile(profile, registers)
|
||||
return ModbusTestReadResponse(ok=True, payload=payload)
|
||||
|
||||
+14
-1
@@ -1,7 +1,8 @@
|
||||
from functools import lru_cache
|
||||
from pathlib import Path
|
||||
import re
|
||||
|
||||
from pydantic import computed_field
|
||||
from pydantic import computed_field, field_validator
|
||||
from pydantic_settings import BaseSettings, SettingsConfigDict
|
||||
|
||||
|
||||
@@ -53,6 +54,7 @@ class Settings(BaseSettings):
|
||||
mqtt_username: str = ""
|
||||
mqtt_password: str = ""
|
||||
mqtt_tls_enabled: bool = False
|
||||
mqtt_client_id: str = "home-automation"
|
||||
|
||||
# Home Assistant MQTT Discovery (T08 wires into CONFIG_FIELDS/UI; T11 does publishing).
|
||||
ha_discovery_enabled: bool = False
|
||||
@@ -81,6 +83,17 @@ class Settings(BaseSettings):
|
||||
extra="ignore",
|
||||
)
|
||||
|
||||
@field_validator("mqtt_client_id", mode="before")
|
||||
@classmethod
|
||||
def validate_mqtt_client_id(cls, value: object) -> str:
|
||||
"""Normalize a broker-safe base client identity used by every MQTT client."""
|
||||
if not isinstance(value, str):
|
||||
raise ValueError("MQTT client ID must be a string")
|
||||
normalized = value.strip()
|
||||
if not re.fullmatch(r"[A-Za-z0-9][A-Za-z0-9_-]{0,63}", normalized):
|
||||
raise ValueError("MQTT client ID must be a non-empty ASCII slug")
|
||||
return normalized
|
||||
|
||||
@computed_field
|
||||
@property
|
||||
def is_development(self) -> bool:
|
||||
|
||||
+332
-1
@@ -27,7 +27,7 @@ from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from dataclasses import dataclass, field
|
||||
from datetime import timedelta
|
||||
from datetime import UTC, datetime, timedelta
|
||||
from typing import Any, Callable, Optional, Protocol
|
||||
|
||||
from sqlalchemy.orm import Session
|
||||
@@ -74,6 +74,17 @@ class DeviceInfo:
|
||||
their state is being published.
|
||||
"""
|
||||
|
||||
availability_id: Optional[str] = None
|
||||
"""Stable id used for the shared availability topic, when different from
|
||||
this HA device's identity. A Meter is identified by its own UUID, while
|
||||
its liveness comes from the source/channel feeding it.
|
||||
"""
|
||||
|
||||
availability_getter: Optional[Callable[["Session"], bool]] = field(
|
||||
default=None, repr=False
|
||||
)
|
||||
"""Return whether the source behind this device is currently usable."""
|
||||
|
||||
|
||||
@dataclass
|
||||
class ExposableEntity:
|
||||
@@ -877,3 +888,323 @@ def _energy_cost_provider(session: Session) -> list[ExposableEntity]:
|
||||
|
||||
# Register the energy cost provider at module load time.
|
||||
register_provider(_energy_cost_provider)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# M8 source / meter / thermal-cost provider
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_SOURCE_STALE_AFTER = timedelta(minutes=5)
|
||||
|
||||
|
||||
def _utc_now() -> datetime:
|
||||
"""Small clock seam for live-value bounds and deterministic tests."""
|
||||
return datetime.now(UTC)
|
||||
|
||||
|
||||
def _as_utc(value: datetime) -> datetime:
|
||||
return value.replace(tzinfo=UTC) if value.tzinfo is None else value.astimezone(UTC)
|
||||
|
||||
|
||||
def _source_is_online(source: Any, channel: Any | None = None) -> bool:
|
||||
"""Do not turn an old cumulative value into a plausible live HA state."""
|
||||
now = _utc_now()
|
||||
if not source.enabled or source.status != "online" or source.last_seen_at is None:
|
||||
return False
|
||||
source_age = now - _as_utc(source.last_seen_at)
|
||||
if not timedelta(0) <= source_age <= _SOURCE_STALE_AFTER:
|
||||
return False
|
||||
if channel is None:
|
||||
return True
|
||||
if channel.latest_at is None or channel.latest_quality not in {"valid", "unverifiable"}:
|
||||
return False
|
||||
channel_age = now - _as_utc(channel.latest_at)
|
||||
return timedelta(0) <= channel_age <= _SOURCE_STALE_AFTER
|
||||
|
||||
|
||||
def _dsmr_latest(session: Session, source_id: int, *, start: datetime | None = None,
|
||||
end: datetime | None = None, not_after: datetime | None = None) -> Any:
|
||||
"""Latest DSMR row in the source's (optionally bounded) cumulative domain."""
|
||||
from app.models.energy import DsmrReading
|
||||
from sqlalchemy import select
|
||||
|
||||
query = select(DsmrReading).where(DsmrReading.meter_source_id == source_id)
|
||||
if start is not None:
|
||||
query = query.where(DsmrReading.recorded_at >= start)
|
||||
if end is not None:
|
||||
query = query.where(DsmrReading.recorded_at < end)
|
||||
if not_after is not None:
|
||||
query = query.where(DsmrReading.recorded_at <= not_after)
|
||||
return session.execute(query.order_by(DsmrReading.recorded_at.desc()).limit(1)).scalar_one_or_none()
|
||||
|
||||
|
||||
def _dsmr_total(reading: Any) -> Any:
|
||||
"""Return imported electricity total from a real DSMR telegram, or None."""
|
||||
from decimal import Decimal, InvalidOperation
|
||||
|
||||
try:
|
||||
payload = reading.payload or {}
|
||||
return Decimal(str(payload["electricity_delivered_1"])) + Decimal(
|
||||
str(payload["electricity_delivered_2"])
|
||||
)
|
||||
except (InvalidOperation, KeyError, TypeError, ValueError):
|
||||
return None
|
||||
|
||||
|
||||
def _m8_energy_provider(session: Session) -> list[ExposableEntity]:
|
||||
"""Expose accepted source snapshots and current M8 meters.
|
||||
|
||||
The provider intentionally reads the thermal service's public ``summarize``
|
||||
result for money. Keeping formulas in ``meter_cost`` prevents HA from
|
||||
becoming a second, subtly different billing implementation.
|
||||
"""
|
||||
from app.models.energy import Meter
|
||||
from app.models.meter_source import MeterSource, MeterSourceBinding, MeterSourceChannel
|
||||
from sqlalchemy import select
|
||||
|
||||
sources = session.execute(select(MeterSource)).scalars().all()
|
||||
entities: list[ExposableEntity] = []
|
||||
|
||||
for source in sources:
|
||||
source_info = DeviceInfo(
|
||||
identifiers=("meter-source", source.uuid), name=source.name,
|
||||
availability_id=source.uuid,
|
||||
availability_getter=lambda sess, source_id=source.id: _source_online_by_id(sess, source_id),
|
||||
)
|
||||
entities.append(ExposableEntity(
|
||||
key=f"source.{source.uuid}.online", component="binary_sensor", device=source_info,
|
||||
device_class="connectivity", unit="", name=f"{source.name} Online",
|
||||
value_getter=lambda sess, source_id=source.id: "ON" if _source_online_by_id(sess, source_id) else "OFF",
|
||||
))
|
||||
|
||||
active_meters = session.execute(
|
||||
select(Meter).where(
|
||||
Meter.ended_at.is_(None), Meter.commodity.in_(("electricity", "heating", "hot_water"))
|
||||
)
|
||||
).scalars().all()
|
||||
active_by_commodity = {meter.commodity: meter for meter in active_meters}
|
||||
for meter in active_meters:
|
||||
bound = session.execute(
|
||||
select(MeterSourceBinding, MeterSourceChannel, MeterSource)
|
||||
.join(MeterSourceChannel, MeterSourceChannel.id == MeterSourceBinding.channel_id)
|
||||
.join(MeterSource, MeterSource.id == MeterSourceChannel.source_id)
|
||||
.where(MeterSourceBinding.meter_id == meter.id, MeterSourceBinding.ended_at.is_(None))
|
||||
).one_or_none()
|
||||
if bound is None:
|
||||
continue
|
||||
binding, channel, source = bound
|
||||
info = DeviceInfo(
|
||||
identifiers=("meter", meter.uuid), name=meter.label,
|
||||
# Keep this opaque and Meter-anchored. In particular, do not use a
|
||||
# source UUID here: two channels of one source can be independently
|
||||
# stale/invalid and must not overwrite each other's availability.
|
||||
availability_id=f"meter-availability-{meter.uuid}",
|
||||
availability_getter=lambda sess, source_id=source.id, channel_id=channel.id:
|
||||
_bound_channel_online(sess, source_id, channel_id),
|
||||
)
|
||||
if meter.commodity == "electricity":
|
||||
unit, device_class = "kWh", "energy"
|
||||
elif meter.commodity == "heating":
|
||||
unit, device_class = "GJ", "energy"
|
||||
else:
|
||||
unit, device_class = "m³", "volume"
|
||||
for suffix, getter in (
|
||||
("total", _meter_total_getter(binding.id, source.id, channel.id)),
|
||||
("today", _meter_today_getter(binding.id, source.id, channel.id)),
|
||||
):
|
||||
entities.append(ExposableEntity(
|
||||
key=f"meter.{meter.uuid}.{suffix}", component="sensor", device=info,
|
||||
device_class=device_class, unit=unit,
|
||||
name=f"{meter.label} {suffix.title()}", value_getter=getter,
|
||||
state_class="total_increasing",
|
||||
))
|
||||
|
||||
heating, hot_water = active_by_commodity.get("heating"), active_by_commodity.get("hot_water")
|
||||
if heating is not None and hot_water is not None:
|
||||
identity = ".".join(sorted((heating.uuid, hot_water.uuid)))
|
||||
currency = _thermal_currency(session)
|
||||
cost_info = DeviceInfo(
|
||||
identifiers=("thermal-cost", identity), name="Thermal Energy Cost",
|
||||
provides_availability=False,
|
||||
)
|
||||
labels = {
|
||||
"heating": "Heating", "hot_water_heating": "Hot Water Heating", "water": "Water",
|
||||
"water_tax": "Water Tax", "fixed": "Fixed", "all_in": "All-in",
|
||||
}
|
||||
for suffix, window in (("total", None), ("today", "today")):
|
||||
for metric, label in labels.items():
|
||||
entities.append(ExposableEntity(
|
||||
key=f"thermal_cost.{identity}.{metric}_{suffix}", component="sensor", device=cost_info,
|
||||
device_class="monetary", unit=currency, name=f"Thermal {label} {suffix.title()}",
|
||||
value_getter=_thermal_cost_getter(metric, window),
|
||||
state_class="total" if suffix == "total" else "total_increasing",
|
||||
))
|
||||
return entities
|
||||
|
||||
|
||||
def _source_online_by_id(session: Session, source_id: int) -> bool:
|
||||
from app.models.meter_source import MeterSource
|
||||
source = session.get(MeterSource, source_id)
|
||||
if source is not None and source.kind == "dsmr_mqtt":
|
||||
now = _utc_now()
|
||||
# Inspect the actual latest telegram before calculating freshness: a
|
||||
# clock-skewed future telegram must not make an older one look live.
|
||||
latest = _dsmr_latest(session, source_id)
|
||||
if not source.enabled or latest is None:
|
||||
return False
|
||||
age = now - _as_utc(latest.recorded_at)
|
||||
return timedelta(0) <= age <= _SOURCE_STALE_AFTER
|
||||
return source is not None and _source_is_online(source)
|
||||
|
||||
|
||||
def _bound_channel_online(session: Session, source_id: int, channel_id: int) -> bool:
|
||||
from app.models.meter_source import MeterSource, MeterSourceChannel
|
||||
source, channel = session.get(MeterSource, source_id), session.get(MeterSourceChannel, channel_id)
|
||||
if source is not None and source.kind == "dsmr_mqtt":
|
||||
return _source_online_by_id(session, source_id)
|
||||
return source is not None and channel is not None and _source_is_online(source, channel)
|
||||
|
||||
|
||||
def _meter_total_getter(binding_id: int, source_id: int, channel_id: int) -> Callable[[Session], Any]:
|
||||
def _getter(session: Session) -> Any:
|
||||
from app.models.energy import Meter
|
||||
from app.models.meter_source import MeterSource, MeterSourceBinding, MeterSourceChannel
|
||||
if not _bound_channel_online(session, source_id, channel_id):
|
||||
return None
|
||||
binding = session.get(MeterSourceBinding, binding_id)
|
||||
if (
|
||||
binding is None or binding.ended_at is not None or binding.channel_id != channel_id
|
||||
or (meter := session.get(Meter, binding.meter_id)) is None or meter.ended_at is not None
|
||||
):
|
||||
return None
|
||||
source = session.get(MeterSource, source_id)
|
||||
channel = session.get(MeterSourceChannel, channel_id)
|
||||
if source is None or channel is None:
|
||||
return None
|
||||
start = max(_as_utc(meter.started_at), _as_utc(binding.started_at))
|
||||
# Both windows are half-open. The current records have no end, but
|
||||
# retaining this form makes a future close fail safely.
|
||||
end = min(
|
||||
(value for value in (_as_utc(meter.ended_at) if meter.ended_at else None,
|
||||
_as_utc(binding.ended_at) if binding.ended_at else None) if value is not None),
|
||||
default=None,
|
||||
)
|
||||
now = _utc_now()
|
||||
if source.kind == "dsmr_mqtt":
|
||||
latest = _dsmr_latest(session, source_id, start=start, end=end, not_after=now)
|
||||
if latest is None:
|
||||
return None
|
||||
return _dsmr_total(latest)
|
||||
if channel.latest_at is None or channel.latest_quality not in {"valid", "unverifiable"}:
|
||||
return None
|
||||
latest_at = _as_utc(channel.latest_at)
|
||||
if latest_at < start or latest_at > now or (end is not None and latest_at >= end):
|
||||
return None
|
||||
return channel.latest_value
|
||||
return _getter
|
||||
|
||||
|
||||
def _meter_today_getter(binding_id: int, source_id: int, channel_id: int) -> Callable[[Session], Any]:
|
||||
def _getter(session: Session) -> Any:
|
||||
from app.models.energy import Meter
|
||||
from app.models.meter_source import MeterSource, MeterSourceBinding, WarmteLinkReading
|
||||
from app.services import timezone as tz
|
||||
from sqlalchemy import select
|
||||
binding = session.get(MeterSourceBinding, binding_id)
|
||||
if binding is None or binding.ended_at is not None or binding.channel_id != channel_id:
|
||||
return None
|
||||
meter = session.get(Meter, binding.meter_id)
|
||||
if meter is None or meter.ended_at is not None:
|
||||
return None
|
||||
now = _utc_now()
|
||||
day = (tz.local_now() - _TODAY_RESET_GRACE).date()
|
||||
start = max(tz.local_midnight_utc(day), _as_utc(meter.started_at), _as_utc(binding.started_at))
|
||||
bounds = [tz.local_midnight_utc(day + timedelta(days=1))]
|
||||
bounds.extend(value for value in (
|
||||
_as_utc(meter.ended_at) if meter.ended_at else None,
|
||||
_as_utc(binding.ended_at) if binding.ended_at else None,
|
||||
) if value is not None)
|
||||
end = min(bounds)
|
||||
source = session.get(MeterSource, source_id)
|
||||
if source is None or not source.enabled:
|
||||
return None
|
||||
if source.kind != "dsmr_mqtt" and source.status != "online":
|
||||
return None
|
||||
if source.kind == "dsmr_mqtt":
|
||||
first = _dsmr_latest(session, source_id, start=start, end=end, not_after=now)
|
||||
if first is None:
|
||||
return None
|
||||
from app.models.energy import DsmrReading
|
||||
from sqlalchemy import select as dsmr_select
|
||||
rows = session.execute(dsmr_select(DsmrReading).where(
|
||||
DsmrReading.meter_source_id == source_id, DsmrReading.recorded_at >= start,
|
||||
DsmrReading.recorded_at < end,
|
||||
DsmrReading.recorded_at <= now,
|
||||
).order_by(DsmrReading.recorded_at)).scalars().all()
|
||||
if len(rows) < 2:
|
||||
return None
|
||||
first_total, last_total = _dsmr_total(rows[0]), _dsmr_total(rows[-1])
|
||||
if first_total is None or last_total is None:
|
||||
return None
|
||||
value = last_total - first_total
|
||||
return value if value >= 0 else None
|
||||
query = select(WarmteLinkReading).where(
|
||||
WarmteLinkReading.channel_id == channel_id,
|
||||
WarmteLinkReading.recorded_at >= start,
|
||||
WarmteLinkReading.quality.in_(("valid", "unverifiable")),
|
||||
)
|
||||
if end is not None:
|
||||
query = query.where(WarmteLinkReading.recorded_at < end)
|
||||
query = query.where(WarmteLinkReading.recorded_at <= now)
|
||||
readings = session.execute(query.order_by(WarmteLinkReading.recorded_at)).scalars().all()
|
||||
if len(readings) < 2:
|
||||
return None
|
||||
value = readings[-1].value - readings[0].value
|
||||
return value if value >= 0 else None
|
||||
return _getter
|
||||
|
||||
|
||||
def _thermal_currency(session: Session) -> str:
|
||||
from app.services.contracts import active_contract_versions
|
||||
versions = active_contract_versions(session, scope="thermal")
|
||||
return versions[-1].contract.currency if versions else "EUR"
|
||||
|
||||
|
||||
def _thermal_cost_getter(metric: str, window: str | None) -> Callable[[Session], Any]:
|
||||
def _getter(session: Session) -> Any:
|
||||
from app.services import timezone as tz
|
||||
from app.services.meter_cost import summarize
|
||||
now = _utc_now()
|
||||
from app.models.energy import Meter
|
||||
meters = session.query(Meter).filter(
|
||||
Meter.commodity.in_(("heating", "hot_water")), Meter.ended_at.is_(None)
|
||||
).all()
|
||||
if len(meters) != 2:
|
||||
return None
|
||||
epoch_start = max(_as_utc(m.started_at) for m in meters)
|
||||
if window == "today":
|
||||
day = (tz.local_now() - _TODAY_RESET_GRACE).date()
|
||||
start = max(tz.local_midnight_utc(day), epoch_start)
|
||||
# During the reset grace ``day`` is yesterday, whose local midnight
|
||||
# remains the cap; otherwise do not summarize readings from later today.
|
||||
end = min(tz.local_midnight_utc(day + timedelta(days=1)), now)
|
||||
else:
|
||||
# A combined thermal identity begins when its newest constituent
|
||||
# meter epoch begins; including pre-swap rows would mix identities.
|
||||
start, end = epoch_start, now
|
||||
result = summarize(session, start, end, now=now)
|
||||
if result["period_count"] == 0 and result["fixed_cost"] == 0:
|
||||
return None
|
||||
if metric == "fixed":
|
||||
return result["fixed_cost"]
|
||||
if metric == "all_in":
|
||||
return result["total_cost"]
|
||||
if metric == "water":
|
||||
return result["breakdown"]["hot_water"]
|
||||
if metric == "water_tax":
|
||||
return result["breakdown"]["hot_water_tax"]
|
||||
return result["breakdown"][metric]
|
||||
return _getter
|
||||
|
||||
|
||||
register_provider(_m8_energy_provider)
|
||||
|
||||
@@ -0,0 +1,173 @@
|
||||
"""Registry and configuration helpers for meter-source integrations.
|
||||
|
||||
The registry is deliberately I/O-free. Workers and HTTP handlers use these
|
||||
helpers to share one config contract without opening a broker or serial port.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import Any
|
||||
|
||||
|
||||
SECRET_MASK = ""
|
||||
|
||||
|
||||
class SourceProfileError(ValueError):
|
||||
"""Raised when a source kind or its configuration is invalid."""
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class SourceConfigField:
|
||||
"""One source configuration field and its public metadata."""
|
||||
|
||||
name: str
|
||||
value_type: type
|
||||
default: Any = None
|
||||
required: bool = False
|
||||
secret: bool = False
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class MeterSourceProfile:
|
||||
"""A supported source kind's config, capabilities, and channel units."""
|
||||
|
||||
kind: str
|
||||
fields: tuple[SourceConfigField, ...]
|
||||
capabilities: frozenset[str]
|
||||
allowed_units: frozenset[str]
|
||||
|
||||
|
||||
DSMR_MQTT_PROFILE = MeterSourceProfile(
|
||||
kind="dsmr_mqtt",
|
||||
fields=(
|
||||
SourceConfigField("broker_host", str, default=""),
|
||||
SourceConfigField("broker_port", int, default=1883),
|
||||
SourceConfigField("username", str, default="", secret=True),
|
||||
SourceConfigField("password", str, default="", secret=True),
|
||||
SourceConfigField("tls_enabled", bool, default=False),
|
||||
SourceConfigField("topic", str, default="dsmr/json"),
|
||||
SourceConfigField("tariff_topic", str, default="dsmr/meter-stats/electricity_tariff"),
|
||||
SourceConfigField("sample_interval_s", int, default=10),
|
||||
),
|
||||
capabilities=frozenset({"discover", "mqtt_subscribe", "tariff"}),
|
||||
allowed_units=frozenset({"kWh"}),
|
||||
)
|
||||
|
||||
WARMTELINK_SERIAL_PROFILE = MeterSourceProfile(
|
||||
kind="warmtelink_serial",
|
||||
fields=(
|
||||
SourceConfigField("path", str, required=True),
|
||||
SourceConfigField("baudrate", int, default=115200),
|
||||
SourceConfigField("data_bits", int, default=7),
|
||||
SourceConfigField("parity", str, default="N"),
|
||||
SourceConfigField("stop_bits", int, default=1),
|
||||
),
|
||||
capabilities=frozenset({"discover", "read_only_serial"}),
|
||||
allowed_units=frozenset({"GJ", "m³"}),
|
||||
)
|
||||
|
||||
SOURCE_PROFILES: dict[str, MeterSourceProfile] = {
|
||||
DSMR_MQTT_PROFILE.kind: DSMR_MQTT_PROFILE,
|
||||
WARMTELINK_SERIAL_PROFILE.kind: WARMTELINK_SERIAL_PROFILE,
|
||||
}
|
||||
|
||||
|
||||
def get_source_profile(kind: str) -> MeterSourceProfile:
|
||||
"""Return the profile for *kind*, or raise a stable validation error."""
|
||||
try:
|
||||
return SOURCE_PROFILES[kind]
|
||||
except KeyError as exc:
|
||||
raise SourceProfileError(f"Unsupported meter source kind: {kind!r}") from exc
|
||||
|
||||
|
||||
def list_source_profiles() -> list[MeterSourceProfile]:
|
||||
"""Return profiles in deterministic kind order for a future API/UI."""
|
||||
return [SOURCE_PROFILES[kind] for kind in sorted(SOURCE_PROFILES)]
|
||||
|
||||
|
||||
def _check_type(field: SourceConfigField, value: Any) -> None:
|
||||
# bool is a subclass of int; accept it only for explicitly boolean fields.
|
||||
if type(value) is not field.value_type:
|
||||
raise SourceProfileError(
|
||||
f"Config field {field.name!r} must be a {field.value_type.__name__}."
|
||||
)
|
||||
|
||||
|
||||
def _validate_field_value(kind: str, field: SourceConfigField, value: Any) -> None:
|
||||
_check_type(field, value)
|
||||
if field.name == "path" and not value.startswith("/dev/"):
|
||||
raise SourceProfileError("warmtelink_serial config path must start with '/dev/'.")
|
||||
if field.name in {"broker_port", "baudrate"} and value <= 0:
|
||||
raise SourceProfileError(f"Config field {field.name!r} must be greater than zero.")
|
||||
if field.name == "sample_interval_s" and value < 0:
|
||||
raise SourceProfileError("Config field 'sample_interval_s' must not be negative.")
|
||||
if field.name == "data_bits" and value != 7:
|
||||
raise SourceProfileError("warmtelink_serial data_bits must be 7.")
|
||||
if field.name == "parity" and value != "N":
|
||||
raise SourceProfileError("warmtelink_serial parity must be 'N'.")
|
||||
if field.name == "stop_bits" and value != 1:
|
||||
raise SourceProfileError("warmtelink_serial stop_bits must be 1.")
|
||||
|
||||
|
||||
def validate_source_config(kind: str, config: dict[str, Any]) -> dict[str, Any]:
|
||||
"""Validate a complete config and return it with profile defaults filled.
|
||||
|
||||
Unknown keys are rejected to make configuration additions explicit. Secret
|
||||
masking is intentionally not interpreted here: callers must merge a PATCH
|
||||
with its stored config first.
|
||||
"""
|
||||
profile = get_source_profile(kind)
|
||||
if not isinstance(config, dict):
|
||||
raise SourceProfileError("Source config must be an object.")
|
||||
fields = {field.name: field for field in profile.fields}
|
||||
unknown = set(config) - set(fields)
|
||||
if unknown:
|
||||
raise SourceProfileError(f"Unknown {kind} config field(s): {sorted(unknown)!r}")
|
||||
|
||||
validated: dict[str, Any] = {}
|
||||
for field in profile.fields:
|
||||
if field.name in config:
|
||||
value = config[field.name]
|
||||
elif field.required:
|
||||
raise SourceProfileError(f"Missing required {kind} config field: {field.name!r}")
|
||||
else:
|
||||
value = field.default
|
||||
_validate_field_value(kind, field, value)
|
||||
validated[field.name] = value
|
||||
return validated
|
||||
|
||||
|
||||
def sanitize_source_config(kind: str, config: dict[str, Any]) -> dict[str, Any]:
|
||||
"""Validate and return a response-safe config with secrets masked."""
|
||||
profile = get_source_profile(kind)
|
||||
sanitized = validate_source_config(kind, config)
|
||||
for field in profile.fields:
|
||||
if field.secret:
|
||||
sanitized[field.name] = SECRET_MASK
|
||||
return sanitized
|
||||
|
||||
|
||||
def merge_source_config(kind: str, current: dict[str, Any], patch: dict[str, Any]) -> dict[str, Any]:
|
||||
"""Merge a partial PATCH into stored config, retaining masked secrets.
|
||||
|
||||
An empty secret value is the public response mask and therefore means
|
||||
"keep the old value". New sources use :func:`validate_source_config`
|
||||
instead, so an explicitly empty secret can still be initially configured.
|
||||
"""
|
||||
profile = get_source_profile(kind)
|
||||
current_validated = validate_source_config(kind, current)
|
||||
if not isinstance(patch, dict):
|
||||
raise SourceProfileError("Source config patch must be an object.")
|
||||
fields = {field.name: field for field in profile.fields}
|
||||
unknown = set(patch) - set(fields)
|
||||
if unknown:
|
||||
raise SourceProfileError(f"Unknown {kind} config field(s): {sorted(unknown)!r}")
|
||||
|
||||
merged = dict(current_validated)
|
||||
for name, value in patch.items():
|
||||
field = fields[name]
|
||||
if field.secret and value == SECRET_MASK:
|
||||
continue
|
||||
merged[name] = value
|
||||
return validate_source_config(kind, merged)
|
||||
@@ -1,8 +1,11 @@
|
||||
"""Modbus TCP driver — thin wrapper around pymodbus.
|
||||
|
||||
This module provides a single public function ``read_blocks`` that performs
|
||||
one or more FC04 (Read Input Registers) block reads against a Modbus TCP
|
||||
gateway and returns a flat ``dict[register_address -> 16-bit_value]`` map.
|
||||
one or more block reads against a Modbus TCP gateway — using either FC04
|
||||
(Read Input Registers) or FC03 (Read Holding Registers), selected per call
|
||||
via the ``function_code`` argument — and returns a flat
|
||||
``dict[register_address -> 16-bit_value]`` map. The function code comes from
|
||||
the device profile (e.g. SDM120 uses FC04, DDSU666 uses FC03).
|
||||
|
||||
Design decisions
|
||||
----------------
|
||||
@@ -80,9 +83,10 @@ def read_blocks(
|
||||
unit_id: int,
|
||||
blocks: Sequence[Block],
|
||||
*,
|
||||
function_code: int = 4,
|
||||
timeout: float = 3.0,
|
||||
) -> dict[int, int]:
|
||||
"""Read one or more contiguous register blocks via FC04 (input registers).
|
||||
"""Read one or more contiguous register blocks via FC03 or FC04.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
@@ -96,6 +100,11 @@ def read_blocks(
|
||||
Sequence of ``{"start": int, "count": int}`` dicts describing the
|
||||
contiguous register ranges to read. ``count`` is the number of
|
||||
16-bit registers (not bytes).
|
||||
function_code:
|
||||
Modbus read function code: ``4`` for FC04 (Read Input Registers,
|
||||
default — SDM120) or ``3`` for FC03 (Read Holding Registers —
|
||||
DDSU666 and other devices that expose measurements as holding
|
||||
registers). Comes from the device profile's ``function_code`` field.
|
||||
timeout:
|
||||
TCP connect/read timeout in seconds (default 3 s).
|
||||
|
||||
@@ -107,12 +116,21 @@ def read_blocks(
|
||||
|
||||
Raises
|
||||
------
|
||||
ModbusDriverError
|
||||
If ``function_code`` is neither 3 nor 4 (validated before any
|
||||
connection is attempted).
|
||||
ModbusConnectionError
|
||||
If the TCP connection to the gateway fails.
|
||||
ModbusResponseError
|
||||
If the gateway returns a Modbus exception frame or an unexpected
|
||||
number of registers.
|
||||
"""
|
||||
if function_code not in (3, 4):
|
||||
raise ModbusDriverError(
|
||||
f"Unsupported read function code FC{function_code:02d} "
|
||||
f"(only FC03 holding-register and FC04 input-register reads are supported)"
|
||||
)
|
||||
|
||||
client = ModbusTcpClient(host, port=port, timeout=timeout)
|
||||
try:
|
||||
connected = client.connect()
|
||||
@@ -125,7 +143,7 @@ def read_blocks(
|
||||
for block in blocks:
|
||||
start: int = block["start"]
|
||||
count: int = block["count"]
|
||||
_read_block(client, unit_id, start, count, registers)
|
||||
_read_block(client, unit_id, start, count, registers, function_code=function_code)
|
||||
|
||||
return registers
|
||||
|
||||
@@ -145,10 +163,19 @@ def _read_block(
|
||||
start: int,
|
||||
count: int,
|
||||
result: dict[int, int],
|
||||
*,
|
||||
function_code: int,
|
||||
) -> None:
|
||||
"""Read one block and merge into *result*. Raises on any error."""
|
||||
"""Read one block and merge into *result*. Raises on any error.
|
||||
|
||||
``function_code`` is assumed already validated to be 3 or 4 by the caller
|
||||
(``read_blocks``); 3 dispatches FC03 (holding) and 4 dispatches FC04 (input).
|
||||
"""
|
||||
try:
|
||||
response = client.read_input_registers(start, count=count, device_id=unit_id)
|
||||
if function_code == 3:
|
||||
response = client.read_holding_registers(start, count=count, device_id=unit_id)
|
||||
else: # function_code == 4 (input registers)
|
||||
response = client.read_input_registers(start, count=count, device_id=unit_id)
|
||||
except ConnectionException as exc:
|
||||
raise ModbusConnectionError(
|
||||
f"Lost connection while reading registers 0x{start:04X}+{count}: {exc}"
|
||||
|
||||
@@ -0,0 +1,82 @@
|
||||
name: ddsu666
|
||||
description: CHINT DDSU666 single-phase smart meter
|
||||
function_code: 3 # holding registers (FC03) — DDSU666 has NO input registers (no FC04)
|
||||
word_order: big # high register first — ASSUMED; verify with a known voltage reading
|
||||
byte_order: big # high byte first within each register (confirmed by manual CRC example)
|
||||
|
||||
# NOTE: the manual gives no worked float-decode example, so word_order is a best-guess
|
||||
# (standard big-endian, high register first, matching sdm120). After wiring the meter,
|
||||
# read 0x2000 (voltage) — it should decode to ~230 V. If it decodes to garbage, the
|
||||
# device uses the opposite word order and this profile (and the decoder) need adjusting.
|
||||
# Byte order is confirmed big-endian from the manual (Appendix A, Table A.4: 0x1388 -> 13 88).
|
||||
|
||||
blocks:
|
||||
# Instantaneous quantities 0x2000–0x200F: voltage, current, P, Q, (rsv), PF, (rsv), Freq.
|
||||
# 16 contiguous registers — single bulk read. (DDSU666 manual Table 9.)
|
||||
- { start: 0x2000, count: 0x0010 }
|
||||
# Active energy — import (0x4000) and export (0x400A) read as two small blocks rather
|
||||
# than one span, to avoid touching the undocumented/reserved 0x4002–0x4009 gap.
|
||||
- { start: 0x4000, count: 0x0002 }
|
||||
- { start: 0x400A, count: 0x0002 }
|
||||
|
||||
metrics:
|
||||
# Addresses are the raw Modbus protocol addresses (hex) from DDSU666 manual Table 9,
|
||||
# read via FC03. Each float32 occupies two consecutive 16-bit registers.
|
||||
|
||||
- key: voltage
|
||||
address: 0x2000 # U — Voltage (V)
|
||||
type: float32
|
||||
unit: "V"
|
||||
device_class: voltage
|
||||
ha_component: sensor
|
||||
|
||||
- key: current
|
||||
address: 0x2002 # I — Current (A)
|
||||
type: float32
|
||||
unit: "A"
|
||||
device_class: current
|
||||
ha_component: sensor
|
||||
|
||||
- key: active_power
|
||||
address: 0x2004 # P — Active power. Manual unit is kW (NOT W like sdm120).
|
||||
type: float32
|
||||
unit: "kW"
|
||||
device_class: power
|
||||
ha_component: sensor
|
||||
|
||||
- key: reactive_power
|
||||
address: 0x2006 # Q — Reactive power (kvar)
|
||||
type: float32
|
||||
unit: "kvar"
|
||||
device_class: reactive_power
|
||||
ha_component: sensor
|
||||
|
||||
- key: power_factor
|
||||
address: 0x200A # PF — Power factor (dimensionless)
|
||||
type: float32
|
||||
unit: ""
|
||||
device_class: power_factor
|
||||
ha_component: sensor
|
||||
|
||||
- key: frequency
|
||||
address: 0x200E # Freq — Frequency (Hz)
|
||||
type: float32
|
||||
unit: "Hz"
|
||||
device_class: frequency
|
||||
ha_component: sensor
|
||||
|
||||
- key: import_energy
|
||||
address: 0x4000 # Ep — positive/forward active energy (kWh)
|
||||
type: float32
|
||||
unit: "kWh"
|
||||
device_class: energy
|
||||
state_class: total_increasing
|
||||
ha_component: sensor
|
||||
|
||||
- key: export_energy
|
||||
address: 0x400A # -Ep — reverse active energy (kWh)
|
||||
type: float32
|
||||
unit: "kWh"
|
||||
device_class: energy
|
||||
state_class: total_increasing
|
||||
ha_component: sensor
|
||||
+233
-3
@@ -30,6 +30,7 @@ from __future__ import annotations
|
||||
import logging
|
||||
import threading
|
||||
from collections.abc import Callable
|
||||
from dataclasses import dataclass
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
import paho.mqtt.client as mqtt
|
||||
@@ -47,14 +48,34 @@ MQTT_SETTINGS_KEYS = {
|
||||
"mqtt_username",
|
||||
"mqtt_password",
|
||||
"mqtt_tls_enabled",
|
||||
"mqtt_client_id",
|
||||
}
|
||||
|
||||
|
||||
@dataclass
|
||||
class _SourceClientState:
|
||||
"""One installed source generation and its in-flight callback count."""
|
||||
|
||||
client: mqtt.Client
|
||||
generation: int
|
||||
in_flight: int = 0
|
||||
|
||||
|
||||
def _is_configured(settings: Settings) -> bool:
|
||||
"""Return True if MQTT is enabled *and* the broker host is set."""
|
||||
return bool(settings.mqtt_enabled and settings.mqtt_broker_host)
|
||||
|
||||
|
||||
def mqtt_source_client_id(base_client_id: str, source_id: int) -> str:
|
||||
"""Return a stable, deployment-scoped identity for one DSMR source."""
|
||||
return f"{base_client_id}-dsmr-source-{source_id}"
|
||||
|
||||
|
||||
def mqtt_test_client_id(base_client_id: str) -> str:
|
||||
"""Return a transient test identity that cannot evict a long-lived client."""
|
||||
return f"{base_client_id}-test"
|
||||
|
||||
|
||||
class MqttManager:
|
||||
"""Long-lived MQTT client wrapper.
|
||||
|
||||
@@ -72,11 +93,26 @@ class MqttManager:
|
||||
|
||||
def __init__(self) -> None:
|
||||
self._client: mqtt.Client | None = None
|
||||
self._lock = threading.Lock()
|
||||
self._lock = threading.RLock()
|
||||
# Source replacement/removal is serialized independently from callback
|
||||
# bookkeeping. In particular, paho's loop_stop() joins its network
|
||||
# thread, whose callback completion also needs ``_lock``.
|
||||
self._source_lifecycle_lock = threading.Lock()
|
||||
self._source_idle = threading.Condition(self._lock)
|
||||
self._connected = False
|
||||
# topic → handler registry; persists across reconnects so subscriptions
|
||||
# are automatically re-established when the client reconnects.
|
||||
self._subscriptions: dict[str, Callable[[bytes], None]] = {}
|
||||
# DSMR sources are independent connections: their credentials and TLS
|
||||
# configuration belong to MeterSource.config, not app_config.
|
||||
self._source_clients: dict[int, mqtt.Client] = {}
|
||||
self._source_subscriptions: dict[int, dict[str, Callable[[bytes], None]]] = {}
|
||||
self._source_connected: set[int] = set()
|
||||
# Each replacement gets a distinct identity. A paho callback can run
|
||||
# after its client was stopped, so source id alone is not sufficient.
|
||||
self._source_generations: dict[int, int] = {}
|
||||
self._source_states: dict[int, _SourceClientState] = {}
|
||||
self._next_source_generation = 0
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Public properties
|
||||
@@ -115,6 +151,11 @@ class MqttManager:
|
||||
"""
|
||||
with self._lock:
|
||||
self._stop_client()
|
||||
with self._source_lifecycle_lock:
|
||||
with self._lock:
|
||||
source_ids = list(self._source_clients)
|
||||
for source_id in source_ids:
|
||||
self._stop_source_client(source_id)
|
||||
|
||||
def reconnect(self, settings: Settings) -> None:
|
||||
"""Disconnect the current client (if any) and reconnect with *settings*.
|
||||
@@ -196,15 +237,152 @@ class MqttManager:
|
||||
except Exception:
|
||||
logger.exception("MQTT unsubscribe error (topic=%s).", topic)
|
||||
|
||||
def replace_source(
|
||||
self,
|
||||
source_id: int,
|
||||
*,
|
||||
host: str,
|
||||
port: int,
|
||||
username: str,
|
||||
password: str,
|
||||
tls_enabled: bool,
|
||||
subscriptions: dict[str, Callable[[bytes], None]],
|
||||
base_client_id: str = "home-automation",
|
||||
state_handler: Callable[[str], None] | None = None,
|
||||
) -> bool:
|
||||
"""Replace one source-owned client and its handlers.
|
||||
|
||||
This intentionally does not touch the legacy app-wide client or any
|
||||
other source client. It is also safe for a source to be temporarily
|
||||
unconfigured: handlers are retained in the source registry but no
|
||||
connection is attempted until a host is supplied.
|
||||
"""
|
||||
with self._source_lifecycle_lock:
|
||||
self._stop_source_client(source_id)
|
||||
if not host:
|
||||
self._report_source_state(state_handler, "error", source_id)
|
||||
return False
|
||||
self._next_source_generation += 1
|
||||
generation = self._next_source_generation
|
||||
captured_subscriptions = dict(subscriptions)
|
||||
client = mqtt.Client(
|
||||
callback_api_version=mqtt.CallbackAPIVersion.VERSION2,
|
||||
client_id=mqtt_source_client_id(base_client_id, source_id),
|
||||
)
|
||||
|
||||
def _on_connect(
|
||||
connected_client: mqtt.Client,
|
||||
_userdata: object,
|
||||
_flags: mqtt.ConnectFlags,
|
||||
reason_code: mqtt.ReasonCode,
|
||||
_properties: mqtt.Properties | None,
|
||||
) -> None:
|
||||
with self._lock:
|
||||
if not self._is_current_source_client(source_id, generation, connected_client):
|
||||
return
|
||||
if reason_code.is_failure:
|
||||
self._source_connected.discard(source_id)
|
||||
logger.warning("DSMR MQTT connection refused for source_id=%s", source_id)
|
||||
state = "error"
|
||||
else:
|
||||
self._source_connected.add(source_id)
|
||||
state = "online"
|
||||
for topic in captured_subscriptions:
|
||||
try:
|
||||
connected_client.subscribe(topic)
|
||||
except Exception:
|
||||
logger.exception("DSMR MQTT re-subscribe failed for source_id=%s", source_id)
|
||||
self._report_source_state(state_handler, state, source_id)
|
||||
|
||||
def _on_disconnect(
|
||||
disconnected_client: mqtt.Client,
|
||||
_userdata: object,
|
||||
_flags: mqtt.DisconnectFlags,
|
||||
_reason_code: mqtt.ReasonCode,
|
||||
_properties: mqtt.Properties | None,
|
||||
) -> None:
|
||||
with self._lock:
|
||||
if not self._is_current_source_client(source_id, generation, disconnected_client):
|
||||
return
|
||||
self._source_connected.discard(source_id)
|
||||
self._report_source_state(state_handler, "error", source_id)
|
||||
|
||||
def _on_message(
|
||||
message_client: mqtt.Client,
|
||||
_userdata: object,
|
||||
message: mqtt.MQTTMessage,
|
||||
) -> None:
|
||||
with self._lock:
|
||||
if not self._is_current_source_client(source_id, generation, message_client):
|
||||
return
|
||||
handler = captured_subscriptions.get(message.topic)
|
||||
state = self._source_states.get(source_id)
|
||||
if handler is None or state is None:
|
||||
return
|
||||
# This permit covers the entire handler call. Teardown first
|
||||
# invalidates the state and then waits for all permits, so an
|
||||
# old callback cannot run after teardown returns.
|
||||
state.in_flight += 1
|
||||
try:
|
||||
handler(message.payload)
|
||||
except Exception:
|
||||
logger.exception("DSMR source handler raised (source_id=%s)", source_id)
|
||||
finally:
|
||||
with self._lock:
|
||||
state.in_flight -= 1
|
||||
if state.in_flight == 0:
|
||||
self._source_idle.notify_all()
|
||||
|
||||
client.on_connect = _on_connect
|
||||
client.on_disconnect = _on_disconnect
|
||||
client.on_message = _on_message
|
||||
if tls_enabled:
|
||||
try:
|
||||
client.tls_set()
|
||||
except Exception:
|
||||
logger.exception("DSMR MQTT TLS setup failed for source_id=%s", source_id)
|
||||
self._report_source_state(state_handler, "error", source_id)
|
||||
return False
|
||||
if username:
|
||||
client.username_pw_set(username=username, password=password or None)
|
||||
# Register ownership before network processing begins. A broker
|
||||
# may deliver CONNACK synchronously from connect(), or on the loop
|
||||
# thread before connect() returns.
|
||||
with self._lock:
|
||||
self._source_clients[source_id] = client
|
||||
self._source_subscriptions[source_id] = captured_subscriptions
|
||||
self._source_generations[source_id] = generation
|
||||
self._source_states[source_id] = _SourceClientState(client, generation)
|
||||
self._report_source_state(state_handler, "connecting", source_id)
|
||||
client.loop_start()
|
||||
try:
|
||||
client.connect(host=host, port=port, keepalive=60)
|
||||
except Exception:
|
||||
logger.exception("DSMR MQTT connect failed (source_id=%s, host=%s)", source_id, host)
|
||||
self._report_source_state(state_handler, "error", source_id)
|
||||
self._stop_source_client(source_id)
|
||||
return False
|
||||
return True
|
||||
|
||||
def remove_source(self, source_id: int) -> None:
|
||||
"""Drop one source client and its handlers, including queued callbacks."""
|
||||
with self._source_lifecycle_lock:
|
||||
self._stop_source_client(source_id)
|
||||
|
||||
def source_is_active(self, source_id: int) -> bool:
|
||||
"""Whether a source-owned client is currently installed for callbacks."""
|
||||
with self._lock:
|
||||
return source_id in self._source_clients
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Internal helpers — must be called with self._lock held
|
||||
# Internal helpers
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _start_client(self, settings: Settings) -> None:
|
||||
"""Build a fresh paho Client, configure it, and call loop_start + connect."""
|
||||
client = mqtt.Client(
|
||||
callback_api_version=mqtt.CallbackAPIVersion.VERSION2,
|
||||
client_id="home-automation",
|
||||
client_id=settings.mqtt_client_id,
|
||||
)
|
||||
|
||||
# Callbacks — VERSION2 on_connect signature:
|
||||
@@ -333,6 +511,58 @@ class MqttManager:
|
||||
logger.debug("MQTT loop_stop raised (ignoring).", exc_info=True)
|
||||
logger.info("MQTT client stopped.")
|
||||
|
||||
def _stop_source_client(self, source_id: int) -> None:
|
||||
"""Detach then stop a source client without blocking callback bookkeeping.
|
||||
|
||||
Callers hold ``_source_lifecycle_lock``. The first phase makes the
|
||||
generation unreachable while holding ``_lock``. Paho operations and
|
||||
the in-flight wait are deliberately outside that lock: loop_stop()
|
||||
joins paho's network thread, and an active callback needs ``_lock`` to
|
||||
release its permit in ``_on_message``'s finally block.
|
||||
"""
|
||||
with self._lock:
|
||||
state = self._source_states.pop(source_id, None)
|
||||
client = self._source_clients.pop(source_id, None)
|
||||
self._source_subscriptions.pop(source_id, None)
|
||||
self._source_connected.discard(source_id)
|
||||
# Invalidate callbacks even when there was no successfully
|
||||
# installed client (for example after a failed replacement).
|
||||
self._source_generations.pop(source_id, None)
|
||||
if client is not None:
|
||||
try:
|
||||
client.disconnect()
|
||||
except Exception:
|
||||
logger.debug("DSMR MQTT disconnect raised (source_id=%s)", source_id, exc_info=True)
|
||||
try:
|
||||
client.loop_stop()
|
||||
except Exception:
|
||||
logger.debug("DSMR MQTT loop_stop raised (source_id=%s)", source_id, exc_info=True)
|
||||
if state is not None:
|
||||
with self._lock:
|
||||
while state.in_flight:
|
||||
self._source_idle.wait()
|
||||
|
||||
def _is_current_source_client(
|
||||
self, source_id: int, generation: int, client: mqtt.Client
|
||||
) -> bool:
|
||||
"""Check callback ownership while ``_lock`` is held."""
|
||||
return (
|
||||
self._source_generations.get(source_id) == generation
|
||||
and self._source_clients.get(source_id) is client
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _report_source_state(
|
||||
state_handler: Callable[[str], None] | None, state: str, source_id: int
|
||||
) -> None:
|
||||
"""Invoke an optional health callback without exposing connection credentials."""
|
||||
if state_handler is None:
|
||||
return
|
||||
try:
|
||||
state_handler(state)
|
||||
except Exception:
|
||||
logger.exception("DSMR MQTT source state update failed for source_id=%s", source_id)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Module-level singleton — shared across lifespan and route handlers
|
||||
|
||||
@@ -0,0 +1,192 @@
|
||||
"""Pure, privacy-preserving parser for DSMR and WarmteLink P1 telegrams."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, field as dataclass_field
|
||||
from decimal import Decimal, InvalidOperation
|
||||
from enum import StrEnum
|
||||
import hashlib
|
||||
import re
|
||||
|
||||
|
||||
_OBIS_LINE = re.compile(r"^(?P<code>\d+-\d+:\d+\.\d+\.\d+)(?P<values>(?:\([^)]*\))*)$")
|
||||
_NUMBER_WITH_UNIT = re.compile(r"^(?P<number>[+-]?\d+(?:\.\d+)?)(?:\*(?P<unit>.+))?$")
|
||||
_CHANNEL_OBIS = re.compile(r"^0-(?P<channel>[1-9]\d*):(24|96)\.")
|
||||
_EQUIPMENT_ID_CODES = re.compile(r"^0-(?:0|[1-9]\d*):96\.1\.[01]$")
|
||||
|
||||
|
||||
class IntegrityStatus(StrEnum):
|
||||
"""Whether a frame has a verifiable standard DSMR checksum."""
|
||||
|
||||
VALID = "valid"
|
||||
INVALID = "invalid"
|
||||
UNVERIFIABLE = "unverifiable"
|
||||
|
||||
|
||||
class P1ParseError(ValueError):
|
||||
"""A parse error whose message never includes telegram contents."""
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ObisField:
|
||||
"""One sanitized OBIS line, including values not understood by the parser."""
|
||||
|
||||
code: str
|
||||
raw_values: tuple[str, ...]
|
||||
value: Decimal | None = None
|
||||
unit: str | None = None
|
||||
comparison_token: str | None = dataclass_field(default=None, repr=False)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class P1Channel:
|
||||
"""Fields associated with one M-Bus channel, without its raw identifier."""
|
||||
|
||||
number: int
|
||||
device_type: str | None
|
||||
equipment_fingerprint: str | None = dataclass_field(repr=False)
|
||||
readings: tuple[ObisField, ...]
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class P1Telegram:
|
||||
"""A parsed telegram with only sanitized, persistence-safe data."""
|
||||
|
||||
frame_length: int
|
||||
integrity: IntegrityStatus
|
||||
integrity_reason: str
|
||||
timestamp: str | None
|
||||
equipment_fingerprint: str | None = dataclass_field(repr=False)
|
||||
fields: tuple[ObisField, ...]
|
||||
channels: tuple[P1Channel, ...]
|
||||
|
||||
|
||||
class TelegramFramer:
|
||||
"""Incrementally extract newline-terminated variable-length telegrams."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self._buffer = bytearray()
|
||||
|
||||
def feed(self, chunk: bytes) -> list[bytes]:
|
||||
"""Append *chunk* and return every complete frame now available."""
|
||||
|
||||
self._buffer.extend(chunk)
|
||||
frames: list[bytes] = []
|
||||
while (bang := self._buffer.find(b"!")) >= 0:
|
||||
newline = self._buffer.find(b"\n", bang)
|
||||
if newline < 0:
|
||||
break
|
||||
standard_start = self._buffer.find(b"/")
|
||||
start = standard_start if 0 <= standard_start < bang else 0
|
||||
frames.append(bytes(self._buffer[start : newline + 1]))
|
||||
del self._buffer[: newline + 1]
|
||||
return frames
|
||||
|
||||
|
||||
def dsmr_crc16(data: bytes) -> int:
|
||||
"""Return the DSMR CRC-16 over *data* (normally from ``/`` through ``!``)."""
|
||||
|
||||
crc = 0
|
||||
for byte in data:
|
||||
crc ^= byte
|
||||
for _ in range(8):
|
||||
crc = (crc >> 1) ^ 0xA001 if crc & 1 else crc >> 1
|
||||
return crc & 0xFFFF
|
||||
|
||||
|
||||
def parse_telegram(frame: bytes) -> P1Telegram:
|
||||
"""Parse one complete frame without retaining raw telegram bytes or IDs."""
|
||||
|
||||
bang = frame.find(b"!")
|
||||
if bang < 0:
|
||||
raise P1ParseError("telegram has no footer marker")
|
||||
body = frame[:bang]
|
||||
footer = frame[bang + 1 :].rstrip(b"\r\n")
|
||||
integrity, reason = _integrity(frame, bang, footer)
|
||||
fields, identifiers = _parse_obis_fields(body)
|
||||
timestamp = _field_value(fields, "0-0:1.0.0")
|
||||
return P1Telegram(
|
||||
frame_length=len(frame),
|
||||
integrity=integrity,
|
||||
integrity_reason=reason,
|
||||
timestamp=timestamp,
|
||||
equipment_fingerprint=identifiers.get("0-0:96.1.1"),
|
||||
fields=tuple(fields),
|
||||
channels=_parse_channels(fields, identifiers),
|
||||
)
|
||||
|
||||
|
||||
def _integrity(frame: bytes, bang: int, footer: bytes) -> tuple[IntegrityStatus, str]:
|
||||
if not frame.startswith(b"/"):
|
||||
return IntegrityStatus.UNVERIFIABLE, "missing standard DSMR '/' header"
|
||||
if len(footer) != 4 or not all(chr(byte) in "0123456789abcdefABCDEF" for byte in footer):
|
||||
return IntegrityStatus.UNVERIFIABLE, "footer is not a four-digit hexadecimal CRC"
|
||||
expected = int(footer, 16)
|
||||
actual = dsmr_crc16(frame[: bang + 1])
|
||||
if actual == expected:
|
||||
return IntegrityStatus.VALID, "CRC16 verified from '/' through '!'"
|
||||
return IntegrityStatus.INVALID, f"CRC16 mismatch: expected {expected:04X}, calculated {actual:04X}"
|
||||
|
||||
|
||||
def _parse_obis_fields(body: bytes) -> tuple[list[ObisField], dict[str, str]]:
|
||||
fields: list[ObisField] = []
|
||||
identifiers: dict[str, str] = {}
|
||||
for line in body.decode("ascii", errors="replace").splitlines()[1:]:
|
||||
match = _OBIS_LINE.fullmatch(line)
|
||||
if not match:
|
||||
continue
|
||||
code = match.group("code")
|
||||
raw_values = tuple(re.findall(r"\(([^)]*)\)", match.group("values")))
|
||||
if _EQUIPMENT_ID_CODES.fullmatch(code):
|
||||
comparison_token = _fingerprint(raw_values[-1]) if raw_values else None
|
||||
if comparison_token is not None:
|
||||
identifiers[code] = comparison_token
|
||||
fields.append(ObisField(code, ("<redacted>",), comparison_token=comparison_token))
|
||||
continue
|
||||
value, unit = _numeric_value(raw_values)
|
||||
fields.append(ObisField(code, raw_values, value, unit))
|
||||
return fields, identifiers
|
||||
|
||||
|
||||
def _fingerprint(identifier: str) -> str:
|
||||
"""Hash an identifier locally; callers never receive its original value."""
|
||||
|
||||
return hashlib.sha256(identifier.encode("ascii", errors="replace")).hexdigest()
|
||||
|
||||
|
||||
def _numeric_value(raw_values: tuple[str, ...]) -> tuple[Decimal | None, str | None]:
|
||||
if not raw_values:
|
||||
return None, None
|
||||
match = _NUMBER_WITH_UNIT.fullmatch(raw_values[-1])
|
||||
if not match:
|
||||
return None, None
|
||||
try:
|
||||
return Decimal(match.group("number")), match.group("unit")
|
||||
except InvalidOperation:
|
||||
return None, None
|
||||
|
||||
|
||||
def _field_value(fields: list[ObisField], code: str) -> str | None:
|
||||
field = next((item for item in fields if item.code == code), None)
|
||||
return field.raw_values[-1] if field and field.raw_values else None
|
||||
|
||||
|
||||
def _parse_channels(fields: list[ObisField], identifiers: dict[str, str]) -> tuple[P1Channel, ...]:
|
||||
by_channel: dict[int, list[ObisField]] = {}
|
||||
for field in fields:
|
||||
match = _CHANNEL_OBIS.match(field.code)
|
||||
if match:
|
||||
by_channel.setdefault(int(match.group("channel")), []).append(field)
|
||||
return tuple(
|
||||
P1Channel(
|
||||
number=number,
|
||||
device_type=_field_value(channel_fields, f"0-{number}:24.1.0"),
|
||||
equipment_fingerprint=identifiers.get(f"0-{number}:96.1.0"),
|
||||
readings=tuple(
|
||||
field
|
||||
for field in channel_fields
|
||||
if field.code == f"0-{number}:24.2.1" and field.value is not None
|
||||
),
|
||||
)
|
||||
for number, channel_fields in sorted(by_channel.items())
|
||||
)
|
||||
@@ -25,6 +25,7 @@ Design notes
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from decimal import Decimal, InvalidOperation
|
||||
from pathlib import Path
|
||||
from typing import Any, Optional
|
||||
|
||||
@@ -123,6 +124,7 @@ class ManualProfile(BaseModel):
|
||||
class TibberEnergySpec(BaseModel):
|
||||
source: str # must be "tibber_api"
|
||||
energy_tax: FieldSpec # subtracted from total to derive sell price
|
||||
sell_fee: FieldSpec # verkoopvergoeding (feed-in fee); always subtracted from sell; default 0.0248
|
||||
sell_adjust: FieldSpec # additional sell-price adjustment; default 0
|
||||
|
||||
|
||||
@@ -151,13 +153,86 @@ class TibberProfile(BaseModel):
|
||||
return self
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# DistrictHeatingProfile — user-entered thermal contract structure
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class DistrictHeatingFieldSpec(BaseModel):
|
||||
"""A Decimal-safe thermal tariff field displayed to the user."""
|
||||
|
||||
model_config = {"extra": "forbid"}
|
||||
|
||||
unit: str
|
||||
label: str
|
||||
help: str
|
||||
minimum: Decimal = Decimal("0")
|
||||
default: Decimal | None = None
|
||||
|
||||
|
||||
class DistrictHeatingVariableSpec(BaseModel):
|
||||
model_config = {"extra": "forbid"}
|
||||
|
||||
heating: DistrictHeatingFieldSpec
|
||||
hot_water_heating: DistrictHeatingFieldSpec
|
||||
hot_water: DistrictHeatingFieldSpec
|
||||
hot_water_tax: DistrictHeatingFieldSpec
|
||||
|
||||
|
||||
class DistrictHeatingStandingSpec(BaseModel):
|
||||
model_config = {"extra": "forbid"}
|
||||
|
||||
heating_network: DistrictHeatingFieldSpec
|
||||
metering: DistrictHeatingFieldSpec
|
||||
delivery_set: DistrictHeatingFieldSpec
|
||||
hot_water_network: DistrictHeatingFieldSpec
|
||||
other: DistrictHeatingFieldSpec
|
||||
|
||||
|
||||
class DistrictHeatingProfile(BaseModel):
|
||||
"""Complete structure description for a ``district_heating`` contract."""
|
||||
|
||||
model_config = {"extra": "forbid"}
|
||||
|
||||
kind: str
|
||||
label: str
|
||||
variable: DistrictHeatingVariableSpec
|
||||
standing: DistrictHeatingStandingSpec
|
||||
|
||||
@model_validator(mode="after")
|
||||
def _check_kind(self) -> "DistrictHeatingProfile":
|
||||
if self.kind != "district_heating":
|
||||
raise ValueError(
|
||||
"DistrictHeatingProfile requires kind='district_heating', "
|
||||
f"got {self.kind!r}"
|
||||
)
|
||||
units = {
|
||||
"heating": "EUR/GJ",
|
||||
"hot_water_heating": "EUR/m³",
|
||||
"hot_water": "EUR/m³",
|
||||
"hot_water_tax": "EUR/m³",
|
||||
}
|
||||
for key, unit in units.items():
|
||||
field = getattr(self.variable, key)
|
||||
if field.unit != unit or field.minimum != 0 or field.default is not None:
|
||||
raise ValueError(f"district_heating.variable.{key} must be required {unit} with minimum 0")
|
||||
for key in DistrictHeatingStandingSpec.model_fields:
|
||||
field = getattr(self.standing, key)
|
||||
if field.unit != "EUR/year" or field.minimum != 0 or field.default != 0:
|
||||
raise ValueError(
|
||||
f"district_heating.standing.{key} must be EUR/year with default and minimum 0"
|
||||
)
|
||||
return self
|
||||
|
||||
|
||||
# A union type for type hints where either profile is acceptable.
|
||||
AnyProfile = ManualProfile | TibberProfile
|
||||
AnyProfile = ManualProfile | TibberProfile | DistrictHeatingProfile
|
||||
|
||||
# Map kind → Pydantic model class used for validation.
|
||||
_PROFILE_MODELS: dict[str, type[ManualProfile] | type[TibberProfile]] = {
|
||||
_PROFILE_MODELS: dict[str, type[ManualProfile] | type[TibberProfile] | type[DistrictHeatingProfile]] = {
|
||||
"manual": ManualProfile,
|
||||
"tibber": TibberProfile,
|
||||
"district_heating": DistrictHeatingProfile,
|
||||
}
|
||||
|
||||
|
||||
@@ -203,6 +278,11 @@ def load_profile(kind: str) -> AnyProfile:
|
||||
f"Profile '{kind}': expected a YAML mapping, got {type(raw).__name__}"
|
||||
)
|
||||
|
||||
# YAML's implicit float conversion must never contaminate the thermal profile.
|
||||
# Existing electricity profiles intentionally retain their established defaults.
|
||||
if raw.get("kind", kind) == "district_heating":
|
||||
_reject_yaml_floats(raw, path)
|
||||
|
||||
# Choose the right Pydantic model based on the ``kind`` field in the YAML.
|
||||
yaml_kind = raw.get("kind", kind)
|
||||
model_cls = _PROFILE_MODELS.get(yaml_kind)
|
||||
@@ -268,10 +348,13 @@ def _fill_defaults_manual(values: dict[str, Any], profile: ManualProfile) -> dic
|
||||
|
||||
|
||||
def _fill_defaults_tibber(values: dict[str, Any], profile: TibberProfile) -> dict[str, Any]:
|
||||
"""Return a copy of *values* with sell_adjust and management_fee defaults applied."""
|
||||
"""Return a copy of *values* with sell_fee, sell_adjust and management_fee defaults applied."""
|
||||
filled = dict(values)
|
||||
|
||||
energy = dict(filled.get("energy", {}))
|
||||
# Apply default for sell_fee (default=0.0248) if absent.
|
||||
if "sell_fee" not in energy and profile.energy.sell_fee.default is not None:
|
||||
energy["sell_fee"] = profile.energy.sell_fee.default
|
||||
# Apply default for sell_adjust (default=0) if absent.
|
||||
if "sell_adjust" not in energy and profile.energy.sell_adjust.default is not None:
|
||||
energy["sell_adjust"] = profile.energy.sell_adjust.default
|
||||
@@ -347,6 +430,7 @@ def _validate_tibber_values(values: dict[str, Any], profile: TibberProfile) -> d
|
||||
|
||||
# Required energy fields.
|
||||
_require_numeric("energy", "energy_tax", energy)
|
||||
_require_numeric("energy", "sell_fee", energy)
|
||||
_require_numeric("energy", "sell_adjust", energy)
|
||||
# Required standing fields.
|
||||
_require_numeric("standing", "management_fee", standing)
|
||||
@@ -357,13 +441,90 @@ def _validate_tibber_values(values: dict[str, Any], profile: TibberProfile) -> d
|
||||
return filled
|
||||
|
||||
|
||||
def _reject_yaml_floats(value: Any, path: Path) -> None:
|
||||
"""Reject implicit YAML floats for district-heating profile metadata."""
|
||||
if isinstance(value, float):
|
||||
raise ProfileValidationError(
|
||||
f"Profile '{path.stem}' must not contain YAML float values; use integer 0 or strings."
|
||||
)
|
||||
if isinstance(value, dict):
|
||||
for child in value.values():
|
||||
_reject_yaml_floats(child, path)
|
||||
elif isinstance(value, list):
|
||||
for child in value:
|
||||
_reject_yaml_floats(child, path)
|
||||
|
||||
|
||||
def _decimal_value(section: str, key: str, value: Any) -> str:
|
||||
"""Validate and normalise one thermal amount without passing through float."""
|
||||
if isinstance(value, bool) or isinstance(value, float) or not isinstance(value, (str, int, Decimal)):
|
||||
raise ProfileValidationError(
|
||||
f"Contract values field '{section}.{key}' must be a Decimal-compatible string or integer, "
|
||||
f"got {type(value).__name__!r}"
|
||||
)
|
||||
try:
|
||||
amount = Decimal(str(value))
|
||||
except (InvalidOperation, ValueError) as exc:
|
||||
raise ProfileValidationError(
|
||||
f"Contract values field '{section}.{key}' must be a Decimal-compatible value"
|
||||
) from exc
|
||||
if not amount.is_finite() or amount < 0:
|
||||
raise ProfileValidationError(
|
||||
f"Contract values field '{section}.{key}' must be a non-negative finite Decimal"
|
||||
)
|
||||
return format(amount, "f")
|
||||
|
||||
|
||||
def _validate_district_heating_values(
|
||||
values: dict[str, Any], profile: DistrictHeatingProfile
|
||||
) -> dict[str, Any]:
|
||||
"""Validate thermal values and return a complete JSON-safe Decimal snapshot."""
|
||||
if not isinstance(values, dict):
|
||||
raise ProfileValidationError("District-heating contract values must be a mapping")
|
||||
expected_sections = {"variable", "standing"}
|
||||
unknown_sections = set(values) - expected_sections
|
||||
if unknown_sections:
|
||||
raise ProfileValidationError(
|
||||
f"District-heating contract values contain unknown section(s): {sorted(unknown_sections)}"
|
||||
)
|
||||
|
||||
def normalise_section(
|
||||
section: str, specs: Any, *, defaults_allowed: bool
|
||||
) -> dict[str, str]:
|
||||
supplied = values.get(section, {})
|
||||
if not isinstance(supplied, dict):
|
||||
raise ProfileValidationError(f"Contract values section '{section}' must be a mapping")
|
||||
expected = set(type(specs).model_fields)
|
||||
unknown = set(supplied) - expected
|
||||
if unknown:
|
||||
raise ProfileValidationError(
|
||||
f"Contract values section '{section}' contains unknown field(s): {sorted(unknown)}"
|
||||
)
|
||||
normalised: dict[str, str] = {}
|
||||
for key in type(specs).model_fields:
|
||||
if key not in supplied:
|
||||
field = getattr(specs, key)
|
||||
if not defaults_allowed or field.default is None:
|
||||
raise ProfileValidationError(f"Contract values missing required field '{section}.{key}'")
|
||||
normalised[key] = format(field.default, "f")
|
||||
else:
|
||||
normalised[key] = _decimal_value(section, key, supplied[key])
|
||||
return normalised
|
||||
|
||||
return {
|
||||
"variable": normalise_section("variable", profile.variable, defaults_allowed=False),
|
||||
"standing": normalise_section("standing", profile.standing, defaults_allowed=True),
|
||||
}
|
||||
|
||||
|
||||
def validate_values(kind: str, values: dict[str, Any]) -> dict[str, Any]:
|
||||
"""Validate a contract-values dict against the named profile structure.
|
||||
|
||||
Fields that carry a ``default`` in the profile (e.g. ``ode``,
|
||||
``sell_adjust``, tibber ``management_fee``) are silently filled in when
|
||||
absent from *values*. Fields with no default that are absent, or fields
|
||||
whose value is not a number, cause a ``ProfileValidationError``.
|
||||
``sell_fee``, ``sell_adjust``, tibber ``management_fee``) are silently
|
||||
filled in when absent from *values*. Fields with no default that are
|
||||
absent, or fields whose value is not a number, cause a
|
||||
``ProfileValidationError``.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
@@ -389,5 +550,7 @@ def validate_values(kind: str, values: dict[str, Any]) -> dict[str, Any]:
|
||||
return _validate_manual_values(values, profile)
|
||||
if isinstance(profile, TibberProfile):
|
||||
return _validate_tibber_values(values, profile)
|
||||
if isinstance(profile, DistrictHeatingProfile):
|
||||
return _validate_district_heating_values(values, profile)
|
||||
# Unreachable with current kinds, but guard for future extensions.
|
||||
raise ProfileValidationError(f"No validator implemented for kind={kind!r}")
|
||||
|
||||
@@ -0,0 +1,56 @@
|
||||
kind: district_heating
|
||||
label: 区域供热
|
||||
|
||||
variable:
|
||||
heating:
|
||||
unit: EUR/GJ
|
||||
label: 供暖热量
|
||||
help: 按供暖用热量计收;请录入合同中的实际金额。
|
||||
minimum: 0
|
||||
hot_water_heating:
|
||||
unit: EUR/m³
|
||||
label: 热水加热
|
||||
help: 按热水体积计收的加热部分;请录入合同中的实际金额。
|
||||
minimum: 0
|
||||
hot_water:
|
||||
unit: EUR/m³
|
||||
label: 热水用量
|
||||
help: 按热水体积计收的用量部分;请录入合同中的实际金额。
|
||||
minimum: 0
|
||||
hot_water_tax:
|
||||
unit: EUR/m³
|
||||
label: 热水税费
|
||||
help: 按热水体积计收的税费部分;请录入合同中的实际金额。
|
||||
minimum: 0
|
||||
|
||||
standing:
|
||||
heating_network:
|
||||
unit: EUR/year
|
||||
label: 供暖网络费
|
||||
help: 年度固定费用;默认零,按合同实际金额录入。
|
||||
minimum: 0
|
||||
default: 0
|
||||
metering:
|
||||
unit: EUR/year
|
||||
label: 计量费
|
||||
help: 年度固定费用;默认零,按合同实际金额录入。
|
||||
minimum: 0
|
||||
default: 0
|
||||
delivery_set:
|
||||
unit: EUR/year
|
||||
label: 交付装置费
|
||||
help: 年度固定费用;默认零,按合同实际金额录入。
|
||||
minimum: 0
|
||||
default: 0
|
||||
hot_water_network:
|
||||
unit: EUR/year
|
||||
label: 热水网络费
|
||||
help: 年度固定费用;默认零,按合同实际金额录入。
|
||||
minimum: 0
|
||||
default: 0
|
||||
other:
|
||||
unit: EUR/year
|
||||
label: 其他固定费
|
||||
help: 年度固定费用;默认零,按合同实际金额录入。
|
||||
minimum: 0
|
||||
default: 0
|
||||
@@ -2,9 +2,10 @@ kind: tibber
|
||||
label: Tibber 动态电价(15 分钟)
|
||||
|
||||
energy:
|
||||
source: tibber_api # buy = total (from API); sell = total − energy_tax − sell_adjust
|
||||
source: tibber_api # buy = total (from API); sell = total − energy_tax − sell_fee − sell_adjust
|
||||
energy_tax: { unit: EUR/kWh } # subtracted from total to derive sell price (incl. VAT)
|
||||
sell_adjust: { unit: EUR/kWh, default: 0 } # additional sell-price adjustment (residual spread)
|
||||
sell_fee: { unit: EUR/kWh, default: 0.0248 } # verkoopvergoeding (feed-in fee, incl. VAT); always subtracted from sell
|
||||
sell_adjust: { unit: EUR/kWh, default: 0 } # manual sell-price adjustment; net-metering: set = −energy_tax to refund the tax
|
||||
|
||||
standing: # fixed charges; UI fills per month, engine prorates to days
|
||||
management_fee: { unit: EUR/month, default: 5.99 }
|
||||
|
||||
@@ -209,12 +209,23 @@ def _tibber_strategy(
|
||||
query on ``starts_at``.
|
||||
|
||||
Formula (§3.4):
|
||||
- ``buy = total`` (Tibber's all-inclusive price, already includes tax)
|
||||
- ``sell = total − energy_tax − sell_adjust``
|
||||
- ``buy = total`` (Tibber's all-inclusive price; already includes energy
|
||||
tax, VAT and the buy-side ``inkoopvergoeding``)
|
||||
- ``sell = total − energy_tax − sell_fee − sell_adjust``
|
||||
- ``import_cost = (Δd1 + Δd2) × buy``
|
||||
- ``export_revenue = (Δr1 + Δr2) × sell``
|
||||
- ``net_cost = import_cost − export_revenue``
|
||||
|
||||
``sell_fee`` models Tibber's per-kWh **verkoopvergoeding** (feed-in fee,
|
||||
€0.0248/kWh incl. VAT since 2026-01-01). It is always deducted from the
|
||||
feed-in payout: even under the net-metering (saldering) scheme, Tibber pays
|
||||
``total − verkoopvergoeding`` per returned kWh (Tibber NL: "€0,28 − €0,0248
|
||||
= €0,2552"). ``total`` already contains the equal buy-side
|
||||
``inkoopvergoeding``, so the two fees do **not** cancel — the feed-in price
|
||||
sits ``sell_fee`` below the buy price. ``sell_adjust`` is a separate manual
|
||||
correction: under net metering it carries back the refunded energy tax
|
||||
(``sell_adjust = −energy_tax``), leaving ``sell = total − sell_fee``.
|
||||
|
||||
Tibber does not differentiate tariff slots (dal vs normal) — the 15-minute
|
||||
API price applies to the full delivered/returned volume.
|
||||
|
||||
@@ -248,11 +259,12 @@ def _tibber_strategy(
|
||||
|
||||
energy = values.get("energy", {})
|
||||
energy_tax = _to_decimal(energy.get("energy_tax", 0))
|
||||
sell_fee = _to_decimal(energy.get("sell_fee", 0))
|
||||
sell_adjust = _to_decimal(energy.get("sell_adjust", 0))
|
||||
|
||||
total = _to_decimal(price_row.total)
|
||||
buy = total
|
||||
sell = total - energy_tax - sell_adjust
|
||||
sell = total - energy_tax - sell_fee - sell_adjust
|
||||
|
||||
total_delivered = deltas.d1 + deltas.d2
|
||||
total_returned = deltas.r1 + deltas.r2
|
||||
@@ -269,6 +281,7 @@ def _tibber_strategy(
|
||||
"buy": str(buy),
|
||||
"sell": str(sell),
|
||||
"energy_tax": str(energy_tax),
|
||||
"sell_fee": str(sell_fee),
|
||||
"sell_adjust": str(sell_adjust),
|
||||
}
|
||||
|
||||
|
||||
@@ -30,20 +30,39 @@ logger = logging.getLogger(__name__)
|
||||
_TIBBER_API_URL = "https://api.tibber.com/v1-beta/gql"
|
||||
_DEFAULT_TIMEOUT = 15.0
|
||||
|
||||
# GraphQL query to fetch a range of 15-minute price nodes.
|
||||
# ``priceInfoRange(resolution: QUARTER_HOURLY, first: 96)`` fetches up to
|
||||
# 96 quarter-hourly slots which covers today + tomorrow (2 × 24 × 4 = 192 max,
|
||||
# but the Tibber API typically starts from the current slot and returns at
|
||||
# most the remaining hours of today plus tomorrow, so 96 is a good cap for
|
||||
# "today + tomorrow").
|
||||
# GraphQL query to fetch the forward-looking today + tomorrow price curve at
|
||||
# 15-minute resolution.
|
||||
#
|
||||
# ``priceInfo(resolution: QUARTER_HOURLY)`` returns two node lists:
|
||||
# * ``today`` — always the full current local day (96 quarter-hourly slots,
|
||||
# 00:00 → 23:45 local), regardless of the current time.
|
||||
# * ``tomorrow`` — the full next local day (96 slots) once Tibber publishes the
|
||||
# day-ahead prices (around 13:00–15:00 local); empty before that.
|
||||
#
|
||||
# This is deliberately NOT ``priceInfoRange``: that field is a historical cursor
|
||||
# connection whose range ends at "now" (it never returns future slots), so it
|
||||
# cannot supply upcoming prices. ``priceInfo`` is forward-looking, so every
|
||||
# 15-minute slot's price is present in the DB *before* the slot closes — which is
|
||||
# what makes per-slot billing accurate (each period finds its own exact slot
|
||||
# instead of falling back to a stale earlier price) and keeps the live current-
|
||||
# price entity fresh. The hourly refresh job re-runs this query, so tomorrow's
|
||||
# prices are picked up within an hour of publication without a restart.
|
||||
_PRICE_RANGE_QUERY = """
|
||||
{
|
||||
viewer {
|
||||
homes {
|
||||
id
|
||||
currentSubscription {
|
||||
priceInfoRange(resolution: QUARTER_HOURLY, first: 96) {
|
||||
nodes {
|
||||
priceInfo(resolution: QUARTER_HOURLY) {
|
||||
today {
|
||||
startsAt
|
||||
total
|
||||
energy
|
||||
tax
|
||||
currency
|
||||
level
|
||||
}
|
||||
tomorrow {
|
||||
startsAt
|
||||
total
|
||||
energy
|
||||
@@ -230,11 +249,15 @@ def fetch_price_range(
|
||||
*,
|
||||
timeout: float = _DEFAULT_TIMEOUT,
|
||||
) -> list[PricePoint]:
|
||||
"""Fetch a range of 15-minute price nodes from the Tibber API.
|
||||
"""Fetch the forward-looking today + tomorrow 15-minute price curve from Tibber.
|
||||
|
||||
Sends the ``priceInfoRange(resolution: QUARTER_HOURLY, first: 96)`` query
|
||||
and parses every returned node into a ``PricePoint``. The number of nodes
|
||||
is not assumed — all returned nodes are parsed regardless of count.
|
||||
Sends the ``priceInfo(resolution: QUARTER_HOURLY) { today tomorrow }`` query
|
||||
and parses every node from both lists (today first, then tomorrow) into a
|
||||
``PricePoint``. ``priceInfo`` is forward-looking — ``today`` is always the
|
||||
full current local day and ``tomorrow`` is populated once Tibber publishes the
|
||||
day-ahead prices — so upcoming slots are returned, unlike ``priceInfoRange``
|
||||
which only reaches "now". ``tomorrow`` may be empty (before publication); the
|
||||
number of nodes is not assumed and all returned nodes are parsed.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
@@ -268,10 +291,15 @@ def fetch_price_range(
|
||||
home = _pick_home(homes, home_id)
|
||||
|
||||
try:
|
||||
nodes = home["currentSubscription"]["priceInfoRange"]["nodes"]
|
||||
price_info = home["currentSubscription"]["priceInfo"]
|
||||
today = price_info["today"]
|
||||
tomorrow = price_info["tomorrow"]
|
||||
except (KeyError, TypeError) as exc:
|
||||
raise TibberError("Tibber API response missing priceInfoRange nodes") from exc
|
||||
raise TibberError("Tibber API response missing priceInfo today/tomorrow") from exc
|
||||
|
||||
# tomorrow is null/empty until Tibber publishes the day-ahead prices; treat
|
||||
# a missing list as empty so we still return today's slots.
|
||||
nodes = list(today or []) + list(tomorrow or [])
|
||||
return [_parse_node(node, "QUARTER_HOURLY") for node in nodes]
|
||||
|
||||
|
||||
|
||||
+70
-35
@@ -1,7 +1,9 @@
|
||||
import logging
|
||||
import os
|
||||
from contextlib import asynccontextmanager
|
||||
from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
from typing import Callable
|
||||
|
||||
from fastapi import FastAPI, HTTPException, Request
|
||||
from fastapi.responses import FileResponse
|
||||
@@ -16,8 +18,10 @@ from app.api.routes.api.config import router as api_config_router
|
||||
from app.api.routes.api.data import router as api_data_router
|
||||
from app.api.routes.api.energy import router as api_energy_router
|
||||
from app.api.routes.api.energy_contracts import router as api_energy_contracts_router
|
||||
from app.api.routes.api.meter_costs import router as api_meter_costs_router
|
||||
from app.api.routes.api.expose import router as api_expose_router
|
||||
from app.api.routes.api.meters import router as api_meters_router
|
||||
from app.api.routes.api.meter_sources import router as api_meter_sources_router
|
||||
from app.api.routes.api.modbus import router as api_modbus_router
|
||||
from app.api.routes.api.session import router as api_session_router
|
||||
from app.api.routes import status
|
||||
@@ -35,8 +39,10 @@ from app.services.dsmr_ingest import apply_dsmr_subscription
|
||||
from app.services.public_ip import check_public_ipv4_and_notify
|
||||
from app.services.modbus_poll import poll_all_enabled_devices, BASE_POLL_TICK_SECONDS
|
||||
from app.services.ha_discovery import publish_discovery, publish_states
|
||||
from app.services.tibber_prices import refresh_prices
|
||||
from app.services.tibber_prices import run_tibber_refresh_best_effort
|
||||
from app.services.energy_cost import compute_closed_periods
|
||||
from app.services.meter_cost import compute_closed_periods as compute_closed_meter_cost_periods
|
||||
from app.services.warmtelink_worker import warmtelink_worker_manager
|
||||
from app.services.timezone import local_tz
|
||||
from scripts.app_db_adopt import AppDatabaseAdoptionError, validate_app_runtime_db
|
||||
|
||||
@@ -86,15 +92,7 @@ def _run_scheduled_tibber_refresh() -> None:
|
||||
so that a single failed fetch does not crash the scheduler or affect the
|
||||
other background jobs.
|
||||
"""
|
||||
session_local = get_session_local()
|
||||
session: Session = session_local()
|
||||
try:
|
||||
runtime_settings = build_runtime_settings(session, get_settings())
|
||||
refresh_prices(session, runtime_settings)
|
||||
except Exception:
|
||||
logger.exception("_run_scheduled_tibber_refresh: unexpected error")
|
||||
finally:
|
||||
session.close()
|
||||
run_tibber_refresh_best_effort()
|
||||
|
||||
|
||||
def _run_scheduled_energy_cost() -> None:
|
||||
@@ -110,22 +108,44 @@ def _run_scheduled_energy_cost() -> None:
|
||||
does not crash the scheduler or affect the other background jobs.
|
||||
"""
|
||||
session_local = get_session_local()
|
||||
session: Session = session_local()
|
||||
try:
|
||||
compute_closed_periods(session)
|
||||
# After billing periods are computed, push fresh energy-cost state values
|
||||
# to MQTT/HA. publish_states is internally guarded by _should_publish
|
||||
# (MQTT disabled / not connected → no-op), so this never raises due to
|
||||
# unconfigured MQTT and does not block the billing job.
|
||||
|
||||
def run_scope(label: str, operation: Callable[[Session], None]) -> None:
|
||||
"""Run one best-effort scope in an isolated transaction/session."""
|
||||
session: Session | None = None
|
||||
try:
|
||||
from app.services.ha_discovery import publish_states
|
||||
publish_states(session)
|
||||
session = session_local()
|
||||
operation(session)
|
||||
except Exception:
|
||||
logger.exception("_run_scheduled_energy_cost: publish_states failed (non-fatal)")
|
||||
except Exception:
|
||||
logger.exception("_run_scheduled_energy_cost: unexpected error")
|
||||
finally:
|
||||
session.close()
|
||||
logger.exception("_run_scheduled_energy_cost: %s failed", label)
|
||||
if session is not None:
|
||||
try:
|
||||
session.rollback()
|
||||
except Exception:
|
||||
# A failed cleanup must not replace the operation/factory
|
||||
# error or prevent the following independent scope.
|
||||
logger.exception("_run_scheduled_energy_cost: %s rollback failed", label)
|
||||
finally:
|
||||
if session is not None:
|
||||
try:
|
||||
session.close()
|
||||
except Exception:
|
||||
# Sessions are intentionally isolated; close failures are
|
||||
# diagnostic only and must remain best-effort too.
|
||||
logger.exception("_run_scheduled_energy_cost: %s close failed", label)
|
||||
|
||||
# Electricity, thermal and HA publishing must not share failed transaction
|
||||
# state or accidentally commit each other's partially-flushed changes.
|
||||
run_scope("electricity computation", compute_closed_periods)
|
||||
run_scope("thermal computation", compute_closed_meter_cost_periods)
|
||||
|
||||
def publish(session: Session) -> None:
|
||||
# publish_states is internally guarded by _should_publish (MQTT
|
||||
# disabled / disconnected -> no-op), but gets a clean Session anyway.
|
||||
from app.services.ha_discovery import publish_states
|
||||
|
||||
publish_states(session)
|
||||
|
||||
run_scope("publish_states (non-fatal)", publish)
|
||||
|
||||
|
||||
def _run_scheduled_ha_state_publish() -> None:
|
||||
@@ -237,6 +257,10 @@ async def lifespan(_: FastAPI):
|
||||
replace_existing=True,
|
||||
max_instances=1,
|
||||
coalesce=True,
|
||||
# APScheduler otherwise waits one full interval before its first run.
|
||||
# This preserves the hourly cadence while requesting a non-blocking
|
||||
# startup fetch as soon as the scheduler starts.
|
||||
next_run_time=datetime.now(UTC),
|
||||
)
|
||||
# Energy cost billing: compute uncalculated closed 15-minute periods every minute.
|
||||
# The job is a no-op when no active contract or DSMR data is present, so it is
|
||||
@@ -272,19 +296,28 @@ async def lifespan(_: FastAPI):
|
||||
_startup_runtime_settings = build_runtime_settings(_startup_session, get_settings())
|
||||
finally:
|
||||
_startup_session.close()
|
||||
mqtt_manager.connect(_startup_runtime_settings)
|
||||
serial_started = False
|
||||
try:
|
||||
mqtt_manager.connect(_startup_runtime_settings)
|
||||
|
||||
# DSMR ingest: subscribe to the configured MQTT topic when enabled. The same
|
||||
# applier is called from PUT /api/config, so toggling DSMR via the UI takes
|
||||
# effect without an app restart.
|
||||
apply_dsmr_subscription(_startup_runtime_settings)
|
||||
# DSMR sources carry their own runtime configuration and are reconciled
|
||||
# after the MQTT manager is connected.
|
||||
apply_dsmr_subscription(_startup_runtime_settings)
|
||||
# Mark it before reconcile: a partial reconcile can already own a fd or
|
||||
# a non-daemon thread and must receive the same orderly shutdown.
|
||||
serial_started = True
|
||||
warmtelink_worker_manager.start()
|
||||
|
||||
yield
|
||||
|
||||
# MQTT: clean shutdown before the process exits.
|
||||
mqtt_manager.disconnect()
|
||||
|
||||
scheduler.shutdown(wait=False)
|
||||
yield
|
||||
finally:
|
||||
# Serial descriptors/workers must be handled first on every exit path.
|
||||
if serial_started:
|
||||
try:
|
||||
warmtelink_worker_manager.shutdown()
|
||||
except Exception:
|
||||
logger.exception("WarmteLink shutdown failed")
|
||||
mqtt_manager.disconnect()
|
||||
scheduler.shutdown(wait=False)
|
||||
|
||||
|
||||
def create_app() -> FastAPI:
|
||||
@@ -308,7 +341,9 @@ def create_app() -> FastAPI:
|
||||
app.include_router(api_data_router)
|
||||
app.include_router(api_energy_router)
|
||||
app.include_router(api_energy_contracts_router)
|
||||
app.include_router(api_meter_costs_router)
|
||||
app.include_router(api_meters_router)
|
||||
app.include_router(api_meter_sources_router)
|
||||
app.include_router(api_expose_router)
|
||||
app.include_router(api_modbus_router)
|
||||
app.include_router(api_session_router)
|
||||
|
||||
@@ -5,12 +5,16 @@ from app.models.config import AppConfigEntry
|
||||
from app.models.location import Location
|
||||
from app.models.poo import PooRecord
|
||||
from app.models.public_ip import PublicIPHistory, PublicIPState
|
||||
from app.models.meter_source import MeterSource, MeterSourceBinding, MeterSourceChannel
|
||||
|
||||
__all__ = [
|
||||
"AppConfigEntry",
|
||||
"AuthSession",
|
||||
"AuthUser",
|
||||
"Location",
|
||||
"MeterSource",
|
||||
"MeterSourceBinding",
|
||||
"MeterSourceChannel",
|
||||
"PooRecord",
|
||||
"PublicIPHistory",
|
||||
"PublicIPState",
|
||||
|
||||
+264
-24
@@ -12,19 +12,134 @@ Six tables:
|
||||
from __future__ import annotations
|
||||
|
||||
import uuid as _uuid
|
||||
from datetime import datetime
|
||||
|
||||
from sqlalchemy import Boolean, DateTime, Float, ForeignKey, Integer, String
|
||||
from sqlalchemy.orm import Mapped, mapped_column, relationship
|
||||
from sqlalchemy.types import JSON
|
||||
from datetime import datetime, timezone
|
||||
from decimal import Decimal
|
||||
from typing import Any
|
||||
from sqlalchemy import (
|
||||
Boolean,
|
||||
CheckConstraint,
|
||||
DateTime,
|
||||
Float,
|
||||
ForeignKey,
|
||||
Index,
|
||||
Integer,
|
||||
Numeric,
|
||||
String,
|
||||
UniqueConstraint,
|
||||
event,
|
||||
text,
|
||||
)
|
||||
from sqlalchemy.orm import Mapped, mapped_column, relationship, synonym, validates
|
||||
from sqlalchemy.types import JSON, TypeDecorator
|
||||
|
||||
from app.db import Base
|
||||
from app.models.meter_source import MeterSourceBinding
|
||||
|
||||
|
||||
def _uuid4_str() -> str:
|
||||
return str(_uuid.uuid4())
|
||||
|
||||
|
||||
def _decimal_json(value: Any) -> Any:
|
||||
"""Make auditable JSON portable without admitting binary numeric values."""
|
||||
if isinstance(value, Decimal):
|
||||
return format(value, "f")
|
||||
if isinstance(value, float) or (isinstance(value, int) and not isinstance(value, bool)):
|
||||
raise ValueError("JSON amounts and quantities must be decimal strings, not numeric JSON values")
|
||||
if isinstance(value, dict):
|
||||
return {key: _decimal_json(child) for key, child in value.items()}
|
||||
if isinstance(value, list):
|
||||
return [_decimal_json(child) for child in value]
|
||||
return value
|
||||
|
||||
|
||||
def _validate_fixed_decimal(value: Decimal, precision: int, scale: int, field: str) -> Decimal:
|
||||
if not isinstance(value, Decimal):
|
||||
raise ValueError(f"{field} must be a Decimal, not a binary float or other numeric type")
|
||||
if not value.is_finite():
|
||||
raise ValueError(f"{field} must be finite")
|
||||
if -value.as_tuple().exponent > scale:
|
||||
raise ValueError(f"{field} exceeds scale {scale}")
|
||||
integer_digits = 0 if value.is_zero() else max(value.copy_abs().adjusted() + 1, 0)
|
||||
if integer_digits > precision - scale:
|
||||
raise ValueError(f"{field} exceeds precision {precision},{scale}")
|
||||
return value
|
||||
|
||||
|
||||
class ExactDecimal(TypeDecorator[Decimal]):
|
||||
"""Fixed-point Decimal which uses SQLite TEXT, never a binary float."""
|
||||
|
||||
impl = Numeric
|
||||
cache_ok = True
|
||||
|
||||
def __init__(self, precision: int, scale: int) -> None:
|
||||
self.precision = precision
|
||||
self.scale = scale
|
||||
super().__init__(precision=precision, scale=scale)
|
||||
|
||||
def load_dialect_impl(self, dialect):
|
||||
if dialect.name == "sqlite":
|
||||
return dialect.type_descriptor(String(self.precision + 2))
|
||||
return dialect.type_descriptor(Numeric(self.precision, self.scale, asdecimal=True))
|
||||
|
||||
def process_bind_param(self, value: Decimal | None, dialect) -> Decimal | str | None:
|
||||
if value is None:
|
||||
return None
|
||||
value = _validate_fixed_decimal(value, self.precision, self.scale, "decimal value")
|
||||
if dialect.name == "sqlite":
|
||||
return format(value, f".{self.scale}f")
|
||||
return value
|
||||
|
||||
def process_result_value(self, value: Decimal | str | None, _dialect) -> Decimal | None:
|
||||
return None if value is None else Decimal(value)
|
||||
|
||||
|
||||
class DecimalJSON(TypeDecorator[dict]):
|
||||
"""JSON which serializes Decimal values as strings on every write path."""
|
||||
|
||||
impl = JSON
|
||||
cache_ok = True
|
||||
|
||||
def process_bind_param(self, value: Any, _dialect) -> Any:
|
||||
return None if value is None else _decimal_json(value)
|
||||
|
||||
|
||||
class UTCDateTime(TypeDecorator[datetime]):
|
||||
"""UTC timestamps that preserve instant identity on SQLite and other dialects."""
|
||||
|
||||
impl = DateTime(timezone=True)
|
||||
cache_ok = True
|
||||
|
||||
def __init__(self, field: str) -> None:
|
||||
self.field = field
|
||||
super().__init__()
|
||||
|
||||
def process_bind_param(self, value: datetime | None, _dialect) -> datetime | None:
|
||||
return None if value is None else _normalise_utc_period(value, self.field)
|
||||
|
||||
def process_result_value(self, value: datetime | None, _dialect) -> datetime | None:
|
||||
if value is None:
|
||||
return None
|
||||
if value.tzinfo is None or value.utcoffset() is None:
|
||||
return value.replace(tzinfo=timezone.utc)
|
||||
return value.astimezone(timezone.utc)
|
||||
|
||||
|
||||
def _require_aware_period(start: datetime, end: datetime) -> None:
|
||||
if start.tzinfo is None or start.utcoffset() is None:
|
||||
raise ValueError("period_start must be timezone-aware")
|
||||
if end.tzinfo is None or end.utcoffset() is None:
|
||||
raise ValueError("period_end must be timezone-aware")
|
||||
if end <= start:
|
||||
raise ValueError("period_end must be after period_start")
|
||||
|
||||
|
||||
def _normalise_utc_period(value: datetime, field: str) -> datetime:
|
||||
if value.tzinfo is None or value.utcoffset() is None:
|
||||
raise ValueError(f"{field} must be timezone-aware")
|
||||
return value.astimezone(timezone.utc)
|
||||
|
||||
|
||||
class Meter(Base):
|
||||
"""One physical electricity meter's installation epoch.
|
||||
|
||||
@@ -53,9 +168,7 @@ class Meter(Base):
|
||||
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
||||
|
||||
# Stable internal identity — used as HA Discovery unique_id anchor.
|
||||
uuid: Mapped[str] = mapped_column(
|
||||
String(36), unique=True, nullable=False, default=_uuid4_str
|
||||
)
|
||||
uuid: Mapped[str] = mapped_column(String(36), unique=True, nullable=False, default=_uuid4_str)
|
||||
|
||||
# Human-readable label for this physical meter (e.g. address, serial, tariff zone).
|
||||
label: Mapped[str] = mapped_column(String(255), nullable=False)
|
||||
@@ -85,6 +198,10 @@ class Meter(Base):
|
||||
back_populates="meter", cascade="save-update, merge"
|
||||
)
|
||||
|
||||
source_bindings: Mapped[list["MeterSourceBinding"]] = relationship(
|
||||
back_populates="meter", cascade="save-update, merge"
|
||||
)
|
||||
|
||||
|
||||
class DsmrReading(Base):
|
||||
"""One down-sampled DSMR telegram stored as a full JSON blob.
|
||||
@@ -93,8 +210,9 @@ class DsmrReading(Base):
|
||||
(that field overflows and must be manually reset to zero — a known DSMR
|
||||
quirk — so relying on it for uniqueness risks silently dropping new data).
|
||||
The table's own autoincrement ``id`` PK is the stable internal identity, and
|
||||
``recorded_at`` (the telegram timestamp) is the UNIQUE de-duplication key: a
|
||||
single P1 meter emits exactly one telegram per timestamp.
|
||||
``(meter_source_id, recorded_at)`` is the UNIQUE de-duplication key: each
|
||||
configured P1 source emits at most one telegram per timestamp, while
|
||||
different sources may legitimately emit at the same instant.
|
||||
|
||||
``recorded_at`` is a real column (not inside the payload) so time-range
|
||||
queries are efficient. The entire telegram frame is stored verbatim in
|
||||
@@ -106,29 +224,54 @@ class DsmrReading(Base):
|
||||
|
||||
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
||||
|
||||
# UTC timestamp of the sample — real column, UNIQUE (telegram-id-independent
|
||||
# idempotency key). The unique index also serves time-range queries.
|
||||
recorded_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True), nullable=False, unique=True
|
||||
)
|
||||
# UTC timestamp of the sample. Idempotency is per configured source, so
|
||||
# distinct P1 sources may legitimately emit at the same instant.
|
||||
recorded_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||
|
||||
# Telegram's own id (DSMR Reader assigns it). Stored only as a reference /
|
||||
# debugging aid — NOT used for uniqueness or idempotency (it overflows and
|
||||
# gets reset to zero). Nullable because some DSMR sources may not emit one.
|
||||
source_id: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
||||
telegram_id: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
||||
|
||||
# Compatibility for the pre-M8 ingest implementation. This is an ORM
|
||||
# alias only; the physical database column is ``telegram_id``.
|
||||
source_id = synonym("telegram_id")
|
||||
|
||||
# The configured source is the durable identity of the cumulative reading
|
||||
# stream. It is non-null after the revision-16 historical adoption.
|
||||
meter_source_id: Mapped[int] = mapped_column(
|
||||
ForeignKey("meter_source.id", ondelete="RESTRICT"), nullable=False, index=True
|
||||
)
|
||||
|
||||
# Full telegram frame as a JSON object; values are typically JSON strings
|
||||
# (e.g. "20915.154") — callers must cast to Decimal before arithmetic.
|
||||
payload: Mapped[dict] = mapped_column(JSON, nullable=False)
|
||||
|
||||
__table_args__ = (
|
||||
UniqueConstraint(
|
||||
"meter_source_id", "recorded_at", name="uq_dsmr_reading_source_recorded_at"
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
@event.listens_for(DsmrReading, "before_insert")
|
||||
def _supply_legacy_dsmr_source(_mapper, connection, target: DsmrReading) -> None:
|
||||
"""Keep the pre-T04 single-source writer working during the schema handoff."""
|
||||
if target.meter_source_id is None:
|
||||
target.meter_source_id = connection.execute(
|
||||
text("SELECT id FROM meter_source WHERE kind = 'dsmr_mqtt' ORDER BY id LIMIT 1")
|
||||
).scalar_one()
|
||||
|
||||
|
||||
class EnergyContract(Base):
|
||||
"""Contract head: a named energy contract with a chosen pricing strategy.
|
||||
|
||||
``kind`` determines which price strategy is used (``manual`` for fixed
|
||||
dual-tariff rates entered by the user, ``tibber`` for dynamic API prices).
|
||||
Only one contract may be ``active`` at a time; the service layer enforces
|
||||
mutual exclusion. Specific pricing values live in ``EnergyContractVersion``
|
||||
A contract belongs to an energy ``scope`` (currently electricity; thermal
|
||||
profiles are reserved for the next milestone). Only one contract may be
|
||||
``active`` per scope; the service layer enforces mutual exclusion. Specific
|
||||
pricing values live in ``EnergyContractVersion``
|
||||
so that price changes can be tracked without modifying historical records.
|
||||
"""
|
||||
|
||||
@@ -144,6 +287,12 @@ class EnergyContract(Base):
|
||||
# migration simple and the strategy registry extensible.
|
||||
kind: Mapped[str] = mapped_column(String(32), nullable=False)
|
||||
|
||||
# Billing domain. The service registry derives this from ``kind`` so API
|
||||
# callers cannot move a pricing strategy into an incompatible domain.
|
||||
scope: Mapped[str] = mapped_column(
|
||||
String(32), nullable=False, default="electricity", index=True
|
||||
)
|
||||
|
||||
# Whether this is the currently active contract (at most one should be True).
|
||||
active: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
|
||||
|
||||
@@ -179,15 +328,11 @@ class EnergyContractVersion(Base):
|
||||
)
|
||||
|
||||
# Start of this version's validity window (inclusive, UTC).
|
||||
effective_from: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True), nullable=False
|
||||
)
|
||||
effective_from: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||
|
||||
# End of this version's validity window (exclusive, UTC). NULL means open-ended
|
||||
# (i.e. this is the most recent / current version).
|
||||
effective_to: Mapped[datetime | None] = mapped_column(
|
||||
DateTime(timezone=True), nullable=True
|
||||
)
|
||||
effective_to: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
|
||||
|
||||
# Pricing values as a JSON object conforming to the profile structure for
|
||||
# ``contract.kind`` (validated by the application layer against the YAML profile).
|
||||
@@ -292,6 +437,12 @@ class EnergyCostPeriod(Base):
|
||||
ForeignKey("meter.id", ondelete="RESTRICT"), nullable=True
|
||||
)
|
||||
|
||||
# Nullable for historical and degraded rows. Every new normal period
|
||||
# points at the one binding that supplied both cumulative endpoints.
|
||||
source_binding_id: Mapped[int | None] = mapped_column(
|
||||
ForeignKey("meter_source_binding.id", ondelete="RESTRICT"), nullable=True
|
||||
)
|
||||
|
||||
# True when the period was computed with incomplete data (missing readings or
|
||||
# missing price); serves as a flag for later recomputation.
|
||||
degraded: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
|
||||
@@ -307,6 +458,95 @@ class EnergyCostPeriod(Base):
|
||||
# Relationship back to the meter epoch.
|
||||
meter: Mapped["Meter | None"] = relationship(back_populates="cost_periods")
|
||||
|
||||
source_binding: Mapped["MeterSourceBinding | None"] = relationship(
|
||||
back_populates="cost_periods"
|
||||
)
|
||||
|
||||
|
||||
class MeterCostPeriod(Base):
|
||||
"""Auditable commodity-scoped ledger row for one half-open metering period.
|
||||
|
||||
A degraded row intentionally permits missing audit links; services must
|
||||
still require them before writing a normal row.
|
||||
"""
|
||||
|
||||
__tablename__ = "meter_cost_period"
|
||||
|
||||
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
||||
commodity: Mapped[str] = mapped_column(String(32), nullable=False)
|
||||
period_start: Mapped[datetime] = mapped_column(UTCDateTime("period_start"), nullable=False)
|
||||
period_end: Mapped[datetime] = mapped_column(UTCDateTime("period_end"), nullable=False)
|
||||
meter_id: Mapped[int | None] = mapped_column(
|
||||
ForeignKey("meter.id", ondelete="RESTRICT"), nullable=True
|
||||
)
|
||||
source_binding_id: Mapped[int | None] = mapped_column(
|
||||
ForeignKey("meter_source_binding.id", ondelete="RESTRICT"), nullable=True
|
||||
)
|
||||
contract_version_id: Mapped[int | None] = mapped_column(
|
||||
ForeignKey("energy_contract_version.id", ondelete="RESTRICT"), nullable=True
|
||||
)
|
||||
|
||||
# SQLite reliably round-trips at most fifteen significant decimal digits.
|
||||
# WarmteLink itself reports 0.001 units, so nine cost fractional digits
|
||||
# retain a six-place tariff times that source precision without float loss.
|
||||
quantity: Mapped[Decimal] = mapped_column(ExactDecimal(15, 6), nullable=False)
|
||||
cost: Mapped[Decimal] = mapped_column(ExactDecimal(15, 9), nullable=False)
|
||||
currency: Mapped[str] = mapped_column(String(8), nullable=False)
|
||||
cost_breakdown: Mapped[dict] = mapped_column(DecimalJSON(), nullable=False, default=dict)
|
||||
pricing_snapshot: Mapped[dict] = mapped_column(DecimalJSON(), nullable=False, default=dict)
|
||||
quality: Mapped[str] = mapped_column(String(32), nullable=False, default="valid")
|
||||
degraded: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
|
||||
degraded_reason: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
||||
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||
|
||||
meter: Mapped["Meter | None"] = relationship()
|
||||
source_binding: Mapped["MeterSourceBinding | None"] = relationship()
|
||||
contract_version: Mapped["EnergyContractVersion | None"] = relationship()
|
||||
|
||||
__table_args__ = (
|
||||
CheckConstraint(
|
||||
"degraded OR (meter_id IS NOT NULL AND source_binding_id IS NOT NULL "
|
||||
"AND contract_version_id IS NOT NULL)",
|
||||
name="ck_meter_cost_period_normal_audit_links",
|
||||
),
|
||||
CheckConstraint("period_end > period_start", name="ck_meter_cost_period_positive_interval"),
|
||||
CheckConstraint(
|
||||
"NOT degraded OR (degraded_reason IS NOT NULL AND length(trim(degraded_reason)) > 0)",
|
||||
name="ck_meter_cost_period_degraded_reason",
|
||||
),
|
||||
UniqueConstraint("commodity", "period_start", name="uq_meter_cost_period_commodity_start"),
|
||||
Index("ix_meter_cost_period_commodity_start", "commodity", "period_start"),
|
||||
Index("ix_meter_cost_period_source_binding_id", "source_binding_id"),
|
||||
)
|
||||
|
||||
@validates("cost_breakdown", "pricing_snapshot")
|
||||
def _validate_decimal_json(self, _key: str, value: dict) -> dict:
|
||||
return _decimal_json(value)
|
||||
|
||||
@validates("quantity", "cost")
|
||||
def _validate_fixed_decimal(self, key: str, value: Decimal) -> Decimal:
|
||||
precision, scale = (15, 6) if key == "quantity" else (15, 9)
|
||||
return _validate_fixed_decimal(value, precision, scale, key)
|
||||
|
||||
@validates("period_start", "period_end")
|
||||
def _normalise_period(self, key: str, value: datetime) -> datetime:
|
||||
return _normalise_utc_period(value, key)
|
||||
|
||||
|
||||
@event.listens_for(MeterCostPeriod, "before_insert")
|
||||
@event.listens_for(MeterCostPeriod, "before_update")
|
||||
def _validate_meter_cost_period(_mapper, _connection, target: MeterCostPeriod) -> None:
|
||||
_require_aware_period(target.period_start, target.period_end)
|
||||
if not target.degraded and (
|
||||
target.meter_id is None
|
||||
or target.source_binding_id is None
|
||||
or target.contract_version_id is None
|
||||
):
|
||||
raise ValueError("normal meter cost periods require meter, binding, and contract version")
|
||||
if target.degraded and not target.degraded_reason:
|
||||
raise ValueError("degraded meter cost periods require a degraded_reason")
|
||||
|
||||
|
||||
# Index on recorded_at for efficient time-range queries on DSMR readings.
|
||||
# (The ORM-level index=True on recorded_at already creates ix_dsmr_reading_recorded_at;
|
||||
|
||||
@@ -0,0 +1,165 @@
|
||||
"""Protocol-agnostic source, channel, and meter-binding identity models."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import uuid as _uuid
|
||||
from datetime import datetime
|
||||
from decimal import Decimal
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from sqlalchemy import (
|
||||
Boolean,
|
||||
CheckConstraint,
|
||||
DateTime,
|
||||
ForeignKey,
|
||||
Index,
|
||||
Integer,
|
||||
Numeric,
|
||||
String,
|
||||
UniqueConstraint,
|
||||
)
|
||||
from sqlalchemy.orm import Mapped, mapped_column, relationship
|
||||
from sqlalchemy.types import JSON
|
||||
|
||||
from app.db import Base
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from app.models.energy import EnergyCostPeriod, Meter
|
||||
|
||||
|
||||
def _uuid4_str() -> str:
|
||||
return str(_uuid.uuid4())
|
||||
|
||||
|
||||
def half_open_intervals_overlap(
|
||||
started_at: datetime,
|
||||
ended_at: datetime | None,
|
||||
other_started_at: datetime,
|
||||
other_ended_at: datetime | None,
|
||||
) -> bool:
|
||||
"""Return whether two ``[started_at, ended_at)`` intervals overlap.
|
||||
|
||||
``None`` denotes an open-ended interval. Equal boundaries do not overlap,
|
||||
which lets a source binding hand off at one exact timestamp.
|
||||
"""
|
||||
return (other_ended_at is None or started_at < other_ended_at) and (
|
||||
ended_at is None or other_started_at < ended_at
|
||||
)
|
||||
|
||||
|
||||
class MeterSource(Base):
|
||||
"""A configured protocol connection that discovers one or more channels."""
|
||||
|
||||
__tablename__ = "meter_source"
|
||||
|
||||
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
||||
uuid: Mapped[str] = mapped_column(String(36), unique=True, nullable=False, default=_uuid4_str)
|
||||
name: Mapped[str] = mapped_column(String(255), nullable=False)
|
||||
kind: Mapped[str] = mapped_column(String(64), nullable=False)
|
||||
enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True)
|
||||
config: Mapped[dict] = mapped_column(JSON, nullable=False, default=dict)
|
||||
status: Mapped[str] = mapped_column(String(32), nullable=False, default="unknown")
|
||||
last_seen_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
|
||||
last_error: Mapped[str | None] = mapped_column(String(1024), nullable=True)
|
||||
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||
|
||||
channels: Mapped[list["MeterSourceChannel"]] = relationship(
|
||||
back_populates="source", cascade="save-update, merge"
|
||||
)
|
||||
|
||||
|
||||
class MeterSourceChannel(Base):
|
||||
"""A stable cumulative measurement identity discovered from a source."""
|
||||
|
||||
__tablename__ = "meter_source_channel"
|
||||
|
||||
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
||||
uuid: Mapped[str] = mapped_column(String(36), unique=True, nullable=False, default=_uuid4_str)
|
||||
source_id: Mapped[int] = mapped_column(
|
||||
ForeignKey("meter_source.id", ondelete="RESTRICT"), nullable=False, index=True
|
||||
)
|
||||
channel_key: Mapped[str] = mapped_column(String(128), nullable=False)
|
||||
label: Mapped[str] = mapped_column(String(255), nullable=False)
|
||||
suggested_commodity: Mapped[str | None] = mapped_column(String(32), nullable=True)
|
||||
unit: Mapped[str] = mapped_column(String(32), nullable=False)
|
||||
device_type: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||
fingerprint: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||
latest_value: Mapped[float | None] = mapped_column(Numeric(20, 6), nullable=True)
|
||||
latest_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
|
||||
latest_quality: Mapped[str | None] = mapped_column(String(32), nullable=True)
|
||||
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||
|
||||
source: Mapped["MeterSource"] = relationship(back_populates="channels")
|
||||
bindings: Mapped[list["MeterSourceBinding"]] = relationship(
|
||||
back_populates="channel", cascade="save-update, merge"
|
||||
)
|
||||
warmtelink_readings: Mapped[list["WarmteLinkReading"]] = relationship(
|
||||
back_populates="channel", cascade="save-update, merge"
|
||||
)
|
||||
|
||||
__table_args__ = (
|
||||
UniqueConstraint("source_id", "channel_key", name="uq_meter_source_channel_source_key"),
|
||||
)
|
||||
|
||||
|
||||
class WarmteLinkReading(Base):
|
||||
"""One accepted scalar cumulative reading from a WarmteLink channel."""
|
||||
|
||||
__tablename__ = "warmtelink_reading"
|
||||
|
||||
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
||||
channel_id: Mapped[int] = mapped_column(
|
||||
ForeignKey("meter_source_channel.id", ondelete="RESTRICT"), nullable=False
|
||||
)
|
||||
# These timestamps retain their UTC-aware application semantics. SQLite
|
||||
# stores them without an offset, so callers must always supply aware UTC.
|
||||
recorded_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||
received_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||
# SQLite's ORM Numeric path can exactly round-trip this 12-integer-digit
|
||||
# range at scale 3. That is ample for a long-lived cumulative meter while
|
||||
# retaining the protocol's 0.001 resolution without float conversion.
|
||||
value: Mapped[Decimal] = mapped_column(Numeric(15, 3), nullable=False)
|
||||
unit: Mapped[str] = mapped_column(String(32), nullable=False)
|
||||
quality: Mapped[str] = mapped_column(String(32), nullable=False)
|
||||
equipment_fingerprint: Mapped[str] = mapped_column(String(64), nullable=False)
|
||||
|
||||
channel: Mapped["MeterSourceChannel"] = relationship(back_populates="warmtelink_readings")
|
||||
|
||||
__table_args__ = (
|
||||
CheckConstraint(
|
||||
"quality IN ('valid', 'invalid', 'unverifiable')",
|
||||
name="ck_warmtelink_reading_quality",
|
||||
),
|
||||
UniqueConstraint("channel_id", "recorded_at", name="uq_warmtelink_reading_channel_recorded_at"),
|
||||
Index("ix_warmtelink_reading_recorded_at", "recorded_at"),
|
||||
)
|
||||
|
||||
|
||||
class MeterSourceBinding(Base):
|
||||
"""Connect one source channel to one physical meter for a half-open window."""
|
||||
|
||||
__tablename__ = "meter_source_binding"
|
||||
|
||||
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
||||
uuid: Mapped[str] = mapped_column(String(36), unique=True, nullable=False, default=_uuid4_str)
|
||||
meter_id: Mapped[int] = mapped_column(
|
||||
ForeignKey("meter.id", ondelete="RESTRICT"), nullable=False, index=True
|
||||
)
|
||||
channel_id: Mapped[int] = mapped_column(
|
||||
ForeignKey("meter_source_channel.id", ondelete="RESTRICT"), nullable=False, index=True
|
||||
)
|
||||
started_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||
ended_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
|
||||
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||
|
||||
meter: Mapped["Meter"] = relationship(back_populates="source_bindings")
|
||||
channel: Mapped["MeterSourceChannel"] = relationship(back_populates="bindings")
|
||||
cost_periods: Mapped[list["EnergyCostPeriod"]] = relationship(
|
||||
back_populates="source_binding", cascade="save-update, merge", passive_deletes="all"
|
||||
)
|
||||
|
||||
|
||||
Index("ix_meter_source_kind_enabled", MeterSource.kind, MeterSource.enabled)
|
||||
+44
-7
@@ -76,6 +76,24 @@ class PricesResponse(BaseModel):
|
||||
"Null for tibber contracts and when no active contract exists."
|
||||
),
|
||||
)
|
||||
contract_version_id: int | None = Field(
|
||||
default=None,
|
||||
description="Thermal active contract version identifier; omitted for electricity.",
|
||||
)
|
||||
effective_from: datetime | None = Field(
|
||||
default=None,
|
||||
description="Thermal contract version start; omitted for electricity.",
|
||||
)
|
||||
effective_to: datetime | None = Field(
|
||||
default=None,
|
||||
description="Thermal contract version end; omitted for electricity.",
|
||||
)
|
||||
values: dict[str, dict[str, str]] | None = Field(
|
||||
default=None,
|
||||
description="Thermal normalized Decimal-string contract values; omitted for electricity.",
|
||||
)
|
||||
|
||||
model_config = {"ser_json_exclude_none": True}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -95,13 +113,18 @@ class CostPeriodSchema(BaseModel):
|
||||
export_revenue: float = Field(description="Revenue from electricity fed to grid (EUR).")
|
||||
net_cost: float = Field(description="import_cost − export_revenue (EUR).")
|
||||
currency: str = Field(description="ISO 4217 currency code.")
|
||||
degraded: bool = Field(
|
||||
description="True when the period was computed with incomplete data."
|
||||
)
|
||||
degraded: bool = Field(description="True when the period was computed with incomplete data.")
|
||||
contract_version_id: int | None = Field(
|
||||
default=None,
|
||||
description="FK to the contract version used for this billing period (null when degraded).",
|
||||
)
|
||||
source_binding_id: int | None = Field(
|
||||
default=None,
|
||||
description=(
|
||||
"FK to the source binding that supplied both cumulative endpoints "
|
||||
"(null for legacy or degraded periods)."
|
||||
),
|
||||
)
|
||||
|
||||
model_config = {"from_attributes": True}
|
||||
|
||||
@@ -121,15 +144,29 @@ class CostsResponse(BaseModel):
|
||||
class SummaryResponse(BaseModel):
|
||||
"""Response for GET /api/energy/costs/summary.
|
||||
|
||||
All monetary values are in ``currency``.
|
||||
Monetary values are in ``currency``; the ``*_kwh`` fields are energy totals
|
||||
in kWh. ``metered_import``/``metered_export`` are **money**, not energy —
|
||||
only the ``_kwh``-suffixed fields carry kWh.
|
||||
|
||||
``total_payable = metered_net + fixed_costs − credits``
|
||||
"""
|
||||
|
||||
currency: str
|
||||
metered_import: float = Field(description="Σ import_cost for non-degraded periods.")
|
||||
metered_export: float = Field(description="Σ export_revenue for non-degraded periods.")
|
||||
metered_net: float = Field(description="Σ net_cost for non-degraded periods.")
|
||||
metered_import: float = Field(
|
||||
description="Σ import_cost for non-degraded periods (money, in `currency`)."
|
||||
)
|
||||
metered_export: float = Field(
|
||||
description="Σ export_revenue for non-degraded periods (money, in `currency`)."
|
||||
)
|
||||
metered_net: float = Field(
|
||||
description="Σ net_cost for non-degraded periods (money, in `currency`)."
|
||||
)
|
||||
metered_import_kwh: float = Field(
|
||||
description="Σ (d1_kwh + d2_kwh) for non-degraded periods (energy imported, kWh)."
|
||||
)
|
||||
metered_export_kwh: float = Field(
|
||||
description="Σ (r1_kwh + r2_kwh) for non-degraded periods (energy exported, kWh)."
|
||||
)
|
||||
fixed_costs: float = Field(
|
||||
description="Standing charges (network_fee + management_fee) apportioned over the interval."
|
||||
)
|
||||
|
||||
@@ -51,6 +51,7 @@ class ContractResponse(BaseModel):
|
||||
id: int
|
||||
name: str
|
||||
kind: str
|
||||
scope: str
|
||||
active: bool
|
||||
currency: str
|
||||
created_at: datetime
|
||||
@@ -69,6 +70,7 @@ class ContractDetailResponse(BaseModel):
|
||||
id: int
|
||||
name: str
|
||||
kind: str
|
||||
scope: str
|
||||
active: bool
|
||||
currency: str
|
||||
created_at: datetime
|
||||
@@ -101,6 +103,7 @@ class ContractCreate(BaseModel):
|
||||
|
||||
name: str = Field(..., min_length=1, max_length=255)
|
||||
kind: str = Field(..., min_length=1, max_length=32)
|
||||
scope: str | None = Field(default=None, min_length=1, max_length=32)
|
||||
currency: str = Field(default="EUR", min_length=1, max_length=8)
|
||||
values: dict[str, Any]
|
||||
effective_from: datetime | None = Field(
|
||||
|
||||
@@ -49,10 +49,21 @@ class MeterResponse(BaseModel):
|
||||
reason: str
|
||||
note: str | None
|
||||
created_at: datetime
|
||||
bindings: list["MeterBindingSummary"] = Field(default_factory=list)
|
||||
|
||||
model_config = {"from_attributes": True}
|
||||
|
||||
|
||||
class MeterBindingSummary(BaseModel):
|
||||
"""Stable, non-sensitive binding identity embedded in meter responses."""
|
||||
|
||||
uuid: str
|
||||
source_channel_uuid: str
|
||||
source_uuid: str
|
||||
started_at: datetime
|
||||
ended_at: datetime | None
|
||||
|
||||
|
||||
class MeterListResponse(BaseModel):
|
||||
"""Response schema for GET /api/energy/meters.
|
||||
|
||||
@@ -107,6 +118,12 @@ class MeterDeclareRequest(BaseModel):
|
||||
max_length=32,
|
||||
description="Energy commodity this meter measures. Defaults to 'electricity'.",
|
||||
)
|
||||
source_channel_uuid: str | None = Field(
|
||||
default=None,
|
||||
min_length=1,
|
||||
max_length=36,
|
||||
description="Optional compatible source channel to bind atomically to this meter.",
|
||||
)
|
||||
|
||||
|
||||
class MeterPatchRequest(BaseModel):
|
||||
@@ -129,3 +146,9 @@ class MeterPatchRequest(BaseModel):
|
||||
"Triggers billing recompute over the affected window."
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
class MeterCloseRequest(BaseModel):
|
||||
"""Close the active meter epoch at an exclusive end boundary."""
|
||||
|
||||
ended_at: datetime
|
||||
|
||||
@@ -0,0 +1,62 @@
|
||||
"""Schemas for the commodity-scoped thermal cost ledger."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime
|
||||
from typing import Literal
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
|
||||
class MeterCostPeriodSchema(BaseModel):
|
||||
"""One auditable thermal ledger row; all Decimal values are JSON strings."""
|
||||
|
||||
commodity: Literal["heating", "hot_water"]
|
||||
period_start: datetime
|
||||
period_end: datetime
|
||||
meter_id: int | None
|
||||
source_binding_id: int | None
|
||||
contract_version_id: int | None
|
||||
quantity: str
|
||||
cost: str
|
||||
currency: str
|
||||
cost_breakdown: dict[str, str]
|
||||
pricing_snapshot: dict[str, dict[str, str]]
|
||||
quality: str
|
||||
degraded: bool
|
||||
degraded_reason: str | None
|
||||
|
||||
|
||||
class MeterCostsResponse(BaseModel):
|
||||
items: list[MeterCostPeriodSchema]
|
||||
total: int = Field(description="Total matching rows before pagination.")
|
||||
|
||||
|
||||
class ThermalFixedBreakdown(BaseModel):
|
||||
"""D11 annual-standing charges accrued per settled local day, as Decimal strings."""
|
||||
|
||||
heating_network: str
|
||||
metering: str
|
||||
delivery_set: str
|
||||
hot_water_network: str
|
||||
other: str
|
||||
|
||||
|
||||
class ThermalCostSummaryResponse(BaseModel):
|
||||
currency: str
|
||||
heating: str
|
||||
hot_water_heating: str
|
||||
hot_water: str
|
||||
hot_water_tax: str
|
||||
variable_subtotal: str
|
||||
fixed_breakdown: ThermalFixedBreakdown
|
||||
fixed_subtotal: str
|
||||
all_in: str
|
||||
period_count: int
|
||||
degraded_count: int
|
||||
|
||||
|
||||
class MeterCostRecomputeResponse(BaseModel):
|
||||
processed: int
|
||||
normal: int
|
||||
degraded: int
|
||||
@@ -0,0 +1,162 @@
|
||||
"""Public schemas for protocol-agnostic meter sources and bindings."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime
|
||||
from decimal import Decimal
|
||||
from typing import Any
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
|
||||
class SourceConfigFieldResponse(BaseModel):
|
||||
name: str
|
||||
value_type: str
|
||||
default: Any = None
|
||||
required: bool
|
||||
secret: bool
|
||||
|
||||
|
||||
class SourceProfileResponse(BaseModel):
|
||||
kind: str
|
||||
fields: list[SourceConfigFieldResponse]
|
||||
defaults: dict[str, Any]
|
||||
capabilities: list[str]
|
||||
allowed_units: list[str]
|
||||
|
||||
|
||||
class SourceProfilesResponse(BaseModel):
|
||||
items: list[SourceProfileResponse]
|
||||
|
||||
|
||||
class MeterSourceCreate(BaseModel):
|
||||
name: str = Field(..., min_length=1, max_length=255)
|
||||
kind: str = Field(..., min_length=1, max_length=64)
|
||||
config: dict[str, Any] = Field(default_factory=dict)
|
||||
enabled: bool = True
|
||||
|
||||
|
||||
class MeterSourcePatch(BaseModel):
|
||||
name: str | None = Field(default=None, min_length=1, max_length=255)
|
||||
config: dict[str, Any] | None = None
|
||||
enabled: bool | None = None
|
||||
|
||||
|
||||
class MeterSourceResponse(BaseModel):
|
||||
uuid: str
|
||||
name: str
|
||||
kind: str
|
||||
enabled: bool
|
||||
config: dict[str, Any]
|
||||
status: str
|
||||
last_seen_at: datetime | None
|
||||
last_error: str | None
|
||||
created_at: datetime
|
||||
updated_at: datetime
|
||||
|
||||
|
||||
class MeterSourceListResponse(BaseModel):
|
||||
items: list[MeterSourceResponse]
|
||||
total: int
|
||||
|
||||
|
||||
class DiscoverResponse(BaseModel):
|
||||
requested: bool
|
||||
supported: bool
|
||||
status: str
|
||||
request_id: int | None = None
|
||||
detail: str | None = None
|
||||
channels: list["DiscoverChannelResponse"] = Field(default_factory=list)
|
||||
|
||||
|
||||
class DiscoverChannelResponse(BaseModel):
|
||||
uuid: str
|
||||
label: str
|
||||
unit: str
|
||||
latest_value: Decimal | None
|
||||
latest_at: datetime | None
|
||||
latest_quality: str | None
|
||||
|
||||
|
||||
class ChannelBindingSummaryResponse(BaseModel):
|
||||
count: int
|
||||
meter_ids: list[int]
|
||||
|
||||
|
||||
class CommodityResponse(BaseModel):
|
||||
key: str
|
||||
unit: str
|
||||
capabilities: list[str]
|
||||
|
||||
|
||||
class CommoditiesResponse(BaseModel):
|
||||
items: list[CommodityResponse]
|
||||
|
||||
|
||||
class MeterSourceChannelResponse(BaseModel):
|
||||
uuid: str
|
||||
label: str
|
||||
suggested_commodity: str | None
|
||||
unit: str
|
||||
device_type: str | None
|
||||
latest_value: Decimal | None
|
||||
latest_at: datetime | None
|
||||
latest_quality: str | None
|
||||
binding_count: int
|
||||
bound_meter_ids: list[int]
|
||||
binding_summary: ChannelBindingSummaryResponse
|
||||
|
||||
|
||||
class MeterSourceChannelListResponse(BaseModel):
|
||||
items: list[MeterSourceChannelResponse]
|
||||
total: int
|
||||
source_status: str
|
||||
|
||||
|
||||
class ChannelReadingResponse(BaseModel):
|
||||
recorded_at: datetime
|
||||
value: Decimal | None = None
|
||||
quality: str | None = None
|
||||
|
||||
|
||||
class ChannelReadingsResponse(BaseModel):
|
||||
items: list[ChannelReadingResponse]
|
||||
total: int
|
||||
|
||||
|
||||
class BindingCreate(BaseModel):
|
||||
source_channel_uuid: str = Field(..., min_length=1, max_length=36)
|
||||
started_at: datetime
|
||||
ended_at: datetime | None = None
|
||||
|
||||
|
||||
class BindingPatch(BaseModel):
|
||||
started_at: datetime | None = None
|
||||
ended_at: datetime | None = None
|
||||
|
||||
|
||||
class BindingTransferRequest(BaseModel):
|
||||
from_binding_uuid: str = Field(..., min_length=1, max_length=36)
|
||||
to_source_channel_uuid: str = Field(..., min_length=1, max_length=36)
|
||||
effective_at: datetime
|
||||
|
||||
|
||||
class BindingResponse(BaseModel):
|
||||
uuid: str
|
||||
meter_id: int
|
||||
source_channel_uuid: str
|
||||
source_uuid: str
|
||||
started_at: datetime
|
||||
ended_at: datetime | None
|
||||
created_at: datetime
|
||||
updated_at: datetime
|
||||
|
||||
|
||||
class BindingListResponse(BaseModel):
|
||||
items: list[BindingResponse]
|
||||
total: int
|
||||
|
||||
|
||||
class BindingTransferResponse(BaseModel):
|
||||
closed_binding: BindingResponse
|
||||
created_binding: BindingResponse
|
||||
+13
-29
@@ -109,6 +109,7 @@ CONFIG_FIELDS: tuple[ConfigField, ...] = (
|
||||
ConfigField("MQTT", "MQTT_USERNAME", "mqtt_username", "MQTT Username"),
|
||||
ConfigField("MQTT", "MQTT_PASSWORD", "mqtt_password", "MQTT Password", secret=True),
|
||||
ConfigField("MQTT", "MQTT_TLS_ENABLED", "mqtt_tls_enabled", "MQTT TLS Enabled", input_type="checkbox"),
|
||||
ConfigField("MQTT", "MQTT_CLIENT_ID", "mqtt_client_id", "MQTT Client ID"),
|
||||
ConfigField(
|
||||
"Home Assistant Discovery",
|
||||
"HA_DISCOVERY_ENABLED",
|
||||
@@ -129,27 +130,6 @@ CONFIG_FIELDS: tuple[ConfigField, ...] = (
|
||||
"HA State Topic Prefix",
|
||||
),
|
||||
ConfigField("Modbus", "MODBUS_POLLING_ENABLED", "modbus_polling_enabled", "Modbus Polling Enabled", input_type="checkbox"),
|
||||
ConfigField(
|
||||
"DSMR",
|
||||
"DSMR_INGEST_ENABLED",
|
||||
"dsmr_ingest_enabled",
|
||||
"DSMR Ingest Enabled",
|
||||
input_type="checkbox",
|
||||
),
|
||||
ConfigField("DSMR", "DSMR_MQTT_TOPIC", "dsmr_mqtt_topic", "DSMR MQTT Topic"),
|
||||
ConfigField(
|
||||
"DSMR",
|
||||
"DSMR_SAMPLE_INTERVAL_S",
|
||||
"dsmr_sample_interval_s",
|
||||
"DSMR Sample Interval (s)",
|
||||
input_type="number",
|
||||
),
|
||||
ConfigField(
|
||||
"DSMR",
|
||||
"DSMR_TARIFF_TOPIC",
|
||||
"dsmr_tariff_topic",
|
||||
"DSMR Tariff Topic",
|
||||
),
|
||||
ConfigField(
|
||||
"Tibber",
|
||||
"TIBBER_API_TOKEN",
|
||||
@@ -243,7 +223,12 @@ def save_config_updates(session: Session, form_data: dict[str, str], bootstrap_s
|
||||
else:
|
||||
merged_values[field.env_name] = submitted_value
|
||||
|
||||
_validate_config_values(merged_values, bootstrap_settings)
|
||||
validated_settings = _validate_config_values(merged_values, bootstrap_settings)
|
||||
# Persist the canonical client identity as well as using it at runtime. A
|
||||
# whitespace-padded value must not survive in app_config and unexpectedly
|
||||
# reappear in another consumer of the stored settings.
|
||||
if "MQTT_CLIENT_ID" in merged_values:
|
||||
merged_values["MQTT_CLIENT_ID"] = validated_settings.mqtt_client_id
|
||||
_persist_config_values(session, merged_values)
|
||||
get_settings.cache_clear()
|
||||
reset_db_caches()
|
||||
@@ -258,7 +243,9 @@ def save_config_value(
|
||||
) -> None:
|
||||
current_values = _read_config_values(session)
|
||||
current_values[env_name] = value
|
||||
_validate_config_values(current_values, bootstrap_settings)
|
||||
validated_settings = _validate_config_values(current_values, bootstrap_settings)
|
||||
if env_name == "MQTT_CLIENT_ID":
|
||||
current_values[env_name] = validated_settings.mqtt_client_id
|
||||
_persist_config_values(session, current_values)
|
||||
get_settings.cache_clear()
|
||||
reset_db_caches()
|
||||
@@ -277,14 +264,14 @@ def _read_config_values(session: Session) -> dict[str, str]:
|
||||
return {row.key: row.value for row in rows}
|
||||
|
||||
|
||||
def _validate_config_values(config_values: dict[str, str], bootstrap_settings: Settings) -> None:
|
||||
def _validate_config_values(config_values: dict[str, str], bootstrap_settings: Settings) -> Settings:
|
||||
payload = _settings_payload(bootstrap_settings)
|
||||
for field in CONFIG_FIELDS:
|
||||
if field.env_name in config_values:
|
||||
payload[field.setting_attr] = config_values[field.env_name]
|
||||
|
||||
try:
|
||||
Settings(_env_file=None, **payload)
|
||||
return Settings(_env_file=None, **payload)
|
||||
except Exception as exc:
|
||||
raise ConfigSaveError("invalid config submission") from exc
|
||||
|
||||
@@ -355,13 +342,10 @@ def _settings_payload(settings: Settings) -> dict[str, Any]:
|
||||
"mqtt_username": settings.mqtt_username,
|
||||
"mqtt_password": settings.mqtt_password,
|
||||
"mqtt_tls_enabled": settings.mqtt_tls_enabled,
|
||||
"mqtt_client_id": settings.mqtt_client_id,
|
||||
"ha_discovery_enabled": settings.ha_discovery_enabled,
|
||||
"ha_discovery_prefix": settings.ha_discovery_prefix,
|
||||
"ha_state_topic_prefix": settings.ha_state_topic_prefix,
|
||||
"dsmr_ingest_enabled": settings.dsmr_ingest_enabled,
|
||||
"dsmr_mqtt_topic": settings.dsmr_mqtt_topic,
|
||||
"dsmr_sample_interval_s": settings.dsmr_sample_interval_s,
|
||||
"dsmr_tariff_topic": settings.dsmr_tariff_topic,
|
||||
"tibber_api_token": settings.tibber_api_token,
|
||||
"tibber_home_id": settings.tibber_home_id,
|
||||
}
|
||||
|
||||
+63
-18
@@ -11,8 +11,8 @@ Design decisions
|
||||
setting its ``effective_to`` to the new version's ``effective_from``; raises
|
||||
``ContractVersionError`` if the new date is strictly earlier than the previous
|
||||
version's ``effective_from``.
|
||||
- ``activate_contract``: mutual-exclusion; sets all other contracts' ``active``
|
||||
to False, then sets the given contract's ``active`` to True.
|
||||
- ``activate_contract``: scope-local mutual exclusion; sets other contracts in
|
||||
the target scope inactive, then sets the given contract active.
|
||||
- ``active_contract_version_at``: returns the single version of the currently
|
||||
active contract that covers *ts* (``effective_from ≤ ts < effective_to``,
|
||||
or open-ended when ``effective_to`` is None).
|
||||
@@ -32,7 +32,7 @@ import logging
|
||||
from datetime import UTC, datetime
|
||||
from typing import Any
|
||||
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy import select, update
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.integrations.pricing.profiles import validate_values
|
||||
@@ -41,6 +41,32 @@ from app.models.energy import EnergyContract, EnergyContractVersion
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
# This is deliberately separate from the pricing-profile loader. T12 needs to
|
||||
# reserve the thermal domain before T13 supplies its actual profile.
|
||||
CONTRACT_KIND_SCOPES: dict[str, str] = {
|
||||
"manual": "electricity",
|
||||
"tibber": "electricity",
|
||||
"district_heating": "thermal",
|
||||
}
|
||||
|
||||
|
||||
class ContractScopeError(ValueError):
|
||||
"""Raised when a contract kind is unknown or its supplied scope disagrees."""
|
||||
|
||||
|
||||
def contract_scope_for_kind(kind: str, requested_scope: str | None = None) -> str:
|
||||
"""Return the registry-owned scope for *kind*, rejecting client mismatches."""
|
||||
try:
|
||||
scope = CONTRACT_KIND_SCOPES[kind]
|
||||
except KeyError as exc:
|
||||
raise ContractScopeError(f"Unknown energy contract kind: {kind!r}") from exc
|
||||
if requested_scope is not None and requested_scope != scope:
|
||||
raise ContractScopeError(
|
||||
f"Contract kind {kind!r} belongs to scope {scope!r}, not {requested_scope!r}."
|
||||
)
|
||||
return scope
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Internal helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -86,10 +112,14 @@ def get_contract_or_none(session: Session, contract_id: int) -> EnergyContract |
|
||||
).scalar_one_or_none()
|
||||
|
||||
|
||||
def list_contracts(session: Session) -> list[EnergyContract]:
|
||||
"""Return all contracts ordered by id (ascending)."""
|
||||
def list_contracts(session: Session, *, scope: str = "electricity") -> list[EnergyContract]:
|
||||
"""Return contracts in one scope, ordered by id (ascending)."""
|
||||
return list(
|
||||
session.execute(select(EnergyContract).order_by(EnergyContract.id)).scalars().all()
|
||||
session.execute(
|
||||
select(EnergyContract)
|
||||
.where(EnergyContract.scope == scope)
|
||||
.order_by(EnergyContract.id)
|
||||
).scalars().all()
|
||||
)
|
||||
|
||||
|
||||
@@ -117,6 +147,7 @@ def create_contract(
|
||||
name: str,
|
||||
kind: str,
|
||||
currency: str = "EUR",
|
||||
scope: str | None = None,
|
||||
values: dict[str, Any],
|
||||
effective_from: datetime,
|
||||
) -> EnergyContract:
|
||||
@@ -129,11 +160,14 @@ def create_contract(
|
||||
name:
|
||||
Human-readable label for the contract.
|
||||
kind:
|
||||
Pricing strategy identifier (``"manual"`` or ``"tibber"``).
|
||||
Pricing strategy identifier (``"manual"``, ``"tibber"``, or
|
||||
``"district_heating"``).
|
||||
currency:
|
||||
ISO 4217 currency code (default ``"EUR"``).
|
||||
values:
|
||||
Pricing values dict conforming to the named profile's structure.
|
||||
Pricing values dict conforming to the named profile's structure. The
|
||||
district-heating profile normalises its Decimal-safe values to strings
|
||||
before the JSON snapshot is stored.
|
||||
Validated via ``validate_values(kind, values)`` before any writes.
|
||||
effective_from:
|
||||
UTC datetime at which the first pricing version takes effect.
|
||||
@@ -150,6 +184,7 @@ def create_contract(
|
||||
ProfileValidationError
|
||||
If *values* does not conform to the profile structure.
|
||||
"""
|
||||
resolved_scope = contract_scope_for_kind(kind, scope)
|
||||
# Validate (and fill defaults) before any DB write.
|
||||
filled_values = validate_values(kind, values)
|
||||
|
||||
@@ -157,6 +192,7 @@ def create_contract(
|
||||
contract = EnergyContract(
|
||||
name=name,
|
||||
kind=kind,
|
||||
scope=resolved_scope,
|
||||
currency=currency,
|
||||
active=False, # New contracts are inactive; caller must explicitly activate.
|
||||
created_at=now,
|
||||
@@ -262,15 +298,18 @@ def add_version(
|
||||
def activate_contract(session: Session, contract: EnergyContract) -> None:
|
||||
"""Activate a contract with mutual exclusion.
|
||||
|
||||
Sets every other contract's ``active`` flag to False, then sets the given
|
||||
contract's ``active`` to True. This guarantees at most one active contract
|
||||
at any time.
|
||||
Sets every other contract in the same scope inactive, then sets the given
|
||||
contract active. This guarantees at most one active contract per scope.
|
||||
|
||||
Caller must commit after this returns.
|
||||
"""
|
||||
# Deactivate all contracts (including the target; we re-activate below).
|
||||
for other in session.execute(select(EnergyContract)).scalars().all():
|
||||
other.active = False
|
||||
# This bulk update is a single write statement inside the caller's
|
||||
# transaction. SQLite serializes writers, and another scope is never touched.
|
||||
session.execute(
|
||||
update(EnergyContract)
|
||||
.where(EnergyContract.scope == contract.scope, EnergyContract.id != contract.id)
|
||||
.values(active=False)
|
||||
)
|
||||
contract.active = True
|
||||
contract.updated_at = datetime.now(UTC)
|
||||
logger.info("Activated contract %r (id=%d)", contract.name, contract.id)
|
||||
@@ -286,7 +325,9 @@ def deactivate_contract(session: Session, contract: EnergyContract) -> None:
|
||||
logger.info("Deactivated contract %r (id=%d)", contract.name, contract.id)
|
||||
|
||||
|
||||
def active_contract_versions(session: Session) -> list[EnergyContractVersion]:
|
||||
def active_contract_versions(
|
||||
session: Session, *, scope: str = "electricity"
|
||||
) -> list[EnergyContractVersion]:
|
||||
"""Return all versions of the currently active contract, ordered by effective_from ascending.
|
||||
|
||||
Returns an empty list when there is no active contract. The list spans the
|
||||
@@ -295,7 +336,9 @@ def active_contract_versions(session: Session) -> list[EnergyContractVersion]:
|
||||
cost / credit accumulation (Principle C).
|
||||
"""
|
||||
active = session.execute(
|
||||
select(EnergyContract).where(EnergyContract.active.is_(True)).limit(1)
|
||||
select(EnergyContract)
|
||||
.where(EnergyContract.active.is_(True), EnergyContract.scope == scope)
|
||||
.limit(1)
|
||||
).scalar_one_or_none()
|
||||
|
||||
if active is None:
|
||||
@@ -313,7 +356,7 @@ def active_contract_versions(session: Session) -> list[EnergyContractVersion]:
|
||||
|
||||
|
||||
def active_contract_version_at(
|
||||
session: Session, ts: datetime
|
||||
session: Session, ts: datetime, *, scope: str = "electricity"
|
||||
) -> EnergyContractVersion | None:
|
||||
"""Return the active contract's version that covers *ts*.
|
||||
|
||||
@@ -336,7 +379,9 @@ def active_contract_version_at(
|
||||
EnergyContractVersion | None
|
||||
"""
|
||||
active = session.execute(
|
||||
select(EnergyContract).where(EnergyContract.active.is_(True)).limit(1)
|
||||
select(EnergyContract)
|
||||
.where(EnergyContract.active.is_(True), EnergyContract.scope == scope)
|
||||
.limit(1)
|
||||
).scalar_one_or_none()
|
||||
|
||||
if active is None:
|
||||
|
||||
+380
-246
@@ -1,46 +1,21 @@
|
||||
"""DSMR telegram ingest service.
|
||||
|
||||
Subscribes to the DSMR Reader MQTT topic (``dsmr/json``) and persists
|
||||
down-sampled DSMR telegram frames to the ``dsmr_reading`` table.
|
||||
|
||||
Design decisions
|
||||
----------------
|
||||
- **Whole-frame storage**: the entire parsed telegram dict is stored as a JSON
|
||||
blob in ``DsmrReading.payload``; no field allow-list is applied. This lets
|
||||
future commodities (gas, heating, three-phase) be accommodated without a
|
||||
table-schema change.
|
||||
- **10-second down-sampling** (configurable via ``dsmr_sample_interval_s``):
|
||||
only telegrams whose ``timestamp`` second falls on an exact multiple of the
|
||||
interval are persisted. This reduces write volume from ~60 rows/min to ~6
|
||||
rows/min while guaranteeing that every 15-minute boundary (second=00) is
|
||||
captured.
|
||||
- **Idempotency**: the telegram's own ``id`` field is stored as ``source_id``
|
||||
with a UNIQUE constraint. A second delivery of the same telegram (e.g. after
|
||||
a broker reconnect) is silently skipped.
|
||||
- **Network-thread safety**: ``handle_message`` is called from paho's background
|
||||
loop thread. It opens and closes its own short-lived SQLAlchemy session and
|
||||
swallows all exceptions so that a buggy payload or transient DB error never
|
||||
crashes the paho loop or drops the MQTT connection.
|
||||
- **Numeric values kept as strings**: the DSMR Reader emits all numeric readings
|
||||
as JSON strings (e.g. ``"20915.154"``). They are stored verbatim; conversion
|
||||
to ``Decimal`` is deferred to the billing engine (T07) where precision matters.
|
||||
- **Null phases**: some telegrams omit certain phase readings (``null`` in JSON);
|
||||
these are stored as-is without special handling.
|
||||
"""
|
||||
"""DSMR MQTT ingest, keyed by durable meter-source identity."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import threading
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime, timezone
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
import sqlalchemy.exc
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.db import get_session_local
|
||||
from app.models.energy import DsmrReading
|
||||
from app.models.energy import DsmrReading, Meter
|
||||
from app.models.meter_source import MeterSource, MeterSourceBinding, MeterSourceChannel
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from app.config import Settings
|
||||
@@ -48,250 +23,409 @@ if TYPE_CHECKING:
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
# Tracks the DSMR topic currently subscribed via the MQTT manager, so a config
|
||||
# change can unsubscribe the old topic before subscribing the new one.
|
||||
_current_dsmr_topic: str | None = None
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class DsmrSourceSnapshot:
|
||||
"""The only DSMR runtime configuration a network handler may use."""
|
||||
|
||||
# Tracks the DSMR tariff topic currently subscribed via the MQTT manager.
|
||||
_current_tariff_topic: str | None = None
|
||||
source_id: int
|
||||
topic: str
|
||||
tariff_topic: str
|
||||
sample_interval_s: int
|
||||
broker_host: str = ""
|
||||
broker_port: int = 1883
|
||||
username: str = ""
|
||||
password: str = ""
|
||||
tls_enabled: bool = False
|
||||
|
||||
# Current electricity tariff: 1 = dal/off-peak, 2 = normal/peak.
|
||||
# Written by the paho network thread, read by the publish job — guarded by a lock.
|
||||
_current_tariff: int | None = None
|
||||
|
||||
_subscriptions: dict[int, DsmrSourceSnapshot] = {}
|
||||
_subscription_client_ids: dict[int, str] = {}
|
||||
_subscription_lock = threading.RLock()
|
||||
# A configuration value is not an ownership identity: disable and re-enable
|
||||
# can produce an equal snapshot. Each installed handler therefore captures a
|
||||
# fresh token and verifies object identity before it can write or update tariff.
|
||||
_subscription_tokens: dict[int, object] = {}
|
||||
_reconcile_lock = threading.RLock()
|
||||
_tariffs: dict[int, int] = {}
|
||||
_tariff_lock = threading.Lock()
|
||||
# Kept only for legacy direct test callers of set_current_tariff(value). Runtime
|
||||
# handlers never write this value; production callers resolve a binding first.
|
||||
_current_tariff: int | None = None
|
||||
_UNSET = object()
|
||||
|
||||
|
||||
def get_current_tariff() -> int | None:
|
||||
"""Return the most recently received electricity tariff (1 or 2), or None."""
|
||||
with _tariff_lock:
|
||||
def get_current_tariff(meter_source_id: int | None = None) -> int | None:
|
||||
"""Return a source tariff, or resolve the active electricity binding.
|
||||
|
||||
The no-argument form is retained for the pre-M8 expose integration. It
|
||||
opens a short session to select the current electricity binding, so a
|
||||
source's MQTT callback can never make another source's tariff current.
|
||||
``_current_tariff`` is solely a test-era fallback when no binding database
|
||||
is available; runtime MQTT handlers do not update it.
|
||||
"""
|
||||
if meter_source_id is None:
|
||||
session_local = get_session_local()
|
||||
session = session_local()
|
||||
try:
|
||||
source_id = _current_electricity_source_id(session, datetime.now(timezone.utc))
|
||||
if source_id is not None:
|
||||
return get_current_tariff(source_id)
|
||||
except Exception:
|
||||
logger.debug("DSMR legacy tariff lookup could not resolve a binding", exc_info=True)
|
||||
finally:
|
||||
session.close()
|
||||
return _current_tariff
|
||||
|
||||
|
||||
def set_current_tariff(value: int | None) -> None:
|
||||
"""Set the current electricity tariff (1 or 2), or clear it with None."""
|
||||
with _tariff_lock:
|
||||
global _current_tariff
|
||||
_current_tariff = value
|
||||
return _tariffs.get(meter_source_id)
|
||||
|
||||
|
||||
def handle_tariff_message(payload_bytes: bytes) -> None:
|
||||
"""Parse one DSMR tariff MQTT payload and update the in-memory tariff state.
|
||||
|
||||
Called from the paho network thread; *must* swallow all exceptions so that
|
||||
a bad payload never crashes the loop or drops the broker connection.
|
||||
|
||||
Accepts payload as bytes or str (paho can deliver either). Ignores
|
||||
whitespace. Only accepts integer values 1 or 2; anything else is discarded
|
||||
and the previous known tariff is preserved.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
payload_bytes:
|
||||
Raw bytes from the MQTT message (may also be a str in some paho versions).
|
||||
"""
|
||||
try:
|
||||
# Decode bytes → str if needed; strip surrounding whitespace.
|
||||
if isinstance(payload_bytes, (bytes, bytearray)):
|
||||
raw = payload_bytes.decode("utf-8", errors="replace").strip()
|
||||
def set_current_tariff(meter_source_id: int, value: int | None | object = _UNSET) -> None:
|
||||
"""Set or clear an individual source's tariff state."""
|
||||
global _current_tariff
|
||||
if value is _UNSET:
|
||||
# Compatibility with older direct callers. Do not route runtime source
|
||||
# updates through this global fallback.
|
||||
_current_tariff = meter_source_id if meter_source_id in (1, 2) else None
|
||||
return
|
||||
with _tariff_lock:
|
||||
if value is None:
|
||||
_tariffs.pop(meter_source_id, None)
|
||||
else:
|
||||
raw = str(payload_bytes).strip()
|
||||
_tariffs[meter_source_id] = value
|
||||
|
||||
|
||||
def _current_electricity_source_id(session: Session, at: datetime) -> int | None:
|
||||
"""Return the DSMR source bound to electricity at ``at``, if any."""
|
||||
return session.scalar(
|
||||
select(MeterSource.id)
|
||||
.join(MeterSourceChannel, MeterSourceChannel.source_id == MeterSource.id)
|
||||
.join(MeterSourceBinding, MeterSourceBinding.channel_id == MeterSourceChannel.id)
|
||||
.join(Meter, Meter.id == MeterSourceBinding.meter_id)
|
||||
.where(
|
||||
Meter.commodity == "electricity",
|
||||
MeterSourceBinding.started_at <= at,
|
||||
(MeterSourceBinding.ended_at.is_(None)) | (MeterSourceBinding.ended_at > at),
|
||||
)
|
||||
.order_by(MeterSourceBinding.started_at.desc())
|
||||
.limit(1)
|
||||
)
|
||||
|
||||
|
||||
def get_current_electricity_tariff(session: Session, at: datetime | None = None) -> int | None:
|
||||
"""Resolve tariff through the current electricity binding, never globally."""
|
||||
source_id = _current_electricity_source_id(session, at or datetime.now(timezone.utc))
|
||||
if source_id is None:
|
||||
return None
|
||||
return get_current_tariff(source_id)
|
||||
|
||||
|
||||
def handle_tariff_message(payload_bytes: bytes, meter_source_id: int) -> None:
|
||||
"""Parse one source's tariff payload without raising in paho's thread."""
|
||||
try:
|
||||
raw = (
|
||||
payload_bytes.decode("utf-8", errors="replace").strip()
|
||||
if isinstance(payload_bytes, (bytes, bytearray))
|
||||
else str(payload_bytes).strip()
|
||||
)
|
||||
value = int(raw)
|
||||
if value not in (1, 2):
|
||||
logger.debug(
|
||||
"dsmr_ingest.handle_tariff_message: unexpected tariff value %d (expected 1 or 2, ignored).",
|
||||
value,
|
||||
)
|
||||
return
|
||||
|
||||
set_current_tariff(value)
|
||||
logger.debug("dsmr_ingest.handle_tariff_message: tariff updated to %d.", value)
|
||||
|
||||
if value in (1, 2):
|
||||
set_current_tariff(meter_source_id, value)
|
||||
except Exception:
|
||||
# Malformed payload (e.g. non-numeric); swallow silently to protect network thread.
|
||||
logger.debug(
|
||||
"dsmr_ingest.handle_tariff_message: could not parse payload %r (ignored).",
|
||||
payload_bytes,
|
||||
)
|
||||
logger.debug("DSMR tariff payload ignored for source_id=%s", meter_source_id)
|
||||
|
||||
|
||||
def apply_dsmr_subscription(settings: "Settings") -> None:
|
||||
"""(Re)apply the DSMR MQTT subscriptions to match *settings* — restart-free.
|
||||
|
||||
Call this at startup and after every config save. It makes the live MQTT
|
||||
subscriptions reflect the current ``dsmr_ingest_enabled`` / ``dsmr_mqtt_topic``
|
||||
/ ``dsmr_sample_interval_s`` / ``dsmr_tariff_topic`` settings without an app
|
||||
restart:
|
||||
|
||||
- **Disabled** → unsubscribe any active DSMR and tariff subscriptions.
|
||||
- **Enabled** → (re)subscribe to ``dsmr_mqtt_topic`` with a handler bound to
|
||||
a *fresh* settings snapshot, so a changed sample interval also takes effect.
|
||||
Also subscribe to ``dsmr_tariff_topic`` when non-empty.
|
||||
- **Topic changed** → unsubscribe the old topic before subscribing the new one.
|
||||
|
||||
Idempotent and safe to call when MQTT is not connected (the subscription is
|
||||
queued in the manager and established on the next connect).
|
||||
"""
|
||||
# Imported here (not at module top) to avoid a circular import at app start.
|
||||
from app.integrations.mqtt import mqtt_manager
|
||||
|
||||
global _current_dsmr_topic, _current_tariff_topic
|
||||
|
||||
if not settings.dsmr_ingest_enabled:
|
||||
if _current_dsmr_topic is not None:
|
||||
mqtt_manager.unsubscribe(_current_dsmr_topic)
|
||||
logger.info("DSMR ingest disabled — unsubscribed from topic=%s.", _current_dsmr_topic)
|
||||
_current_dsmr_topic = None
|
||||
if _current_tariff_topic is not None:
|
||||
mqtt_manager.unsubscribe(_current_tariff_topic)
|
||||
logger.info(
|
||||
"DSMR ingest disabled — unsubscribed from tariff topic=%s.",
|
||||
_current_tariff_topic,
|
||||
)
|
||||
_current_tariff_topic = None
|
||||
return
|
||||
|
||||
# --- Main DSMR telegram topic ---
|
||||
topic = settings.dsmr_mqtt_topic
|
||||
if _current_dsmr_topic is not None and _current_dsmr_topic != topic:
|
||||
mqtt_manager.unsubscribe(_current_dsmr_topic)
|
||||
|
||||
# Re-subscribe (overwrites any existing handler for this topic) with a fresh
|
||||
# settings snapshot so dsmr_sample_interval_s changes take effect too.
|
||||
snapshot = settings
|
||||
mqtt_manager.subscribe(topic, lambda payload: handle_message(payload, snapshot))
|
||||
_current_dsmr_topic = topic
|
||||
logger.info("DSMR ingest enabled — subscribed to topic=%s.", topic)
|
||||
|
||||
# --- DSMR tariff topic (dual-tariff slot indicator) ---
|
||||
tariff_topic = settings.dsmr_tariff_topic if settings.dsmr_tariff_topic else ""
|
||||
if tariff_topic:
|
||||
if _current_tariff_topic is not None and _current_tariff_topic != tariff_topic:
|
||||
mqtt_manager.unsubscribe(_current_tariff_topic)
|
||||
mqtt_manager.subscribe(tariff_topic, lambda payload: handle_tariff_message(payload))
|
||||
_current_tariff_topic = tariff_topic
|
||||
logger.info("DSMR tariff topic — subscribed to topic=%s.", tariff_topic)
|
||||
else:
|
||||
# tariff_topic is empty → unsubscribe any existing tariff subscription.
|
||||
if _current_tariff_topic is not None:
|
||||
mqtt_manager.unsubscribe(_current_tariff_topic)
|
||||
logger.info(
|
||||
"DSMR tariff topic cleared — unsubscribed from topic=%s.",
|
||||
_current_tariff_topic,
|
||||
)
|
||||
_current_tariff_topic = None
|
||||
def _snapshot(source: MeterSource) -> DsmrSourceSnapshot:
|
||||
config = source.config
|
||||
return DsmrSourceSnapshot(
|
||||
source_id=source.id,
|
||||
topic=str(config.get("topic", "dsmr/json")),
|
||||
tariff_topic=str(config.get("tariff_topic", "")),
|
||||
sample_interval_s=int(config.get("sample_interval_s", 10)),
|
||||
broker_host=str(config.get("broker_host", "")),
|
||||
broker_port=int(config.get("broker_port", 1883)),
|
||||
username=str(config.get("username", "")),
|
||||
password=str(config.get("password", "")),
|
||||
tls_enabled=bool(config.get("tls_enabled", False)),
|
||||
)
|
||||
|
||||
|
||||
def handle_message(payload_bytes: bytes, settings: "Settings") -> None:
|
||||
"""Parse one DSMR MQTT payload and persist it if it passes the sample filter.
|
||||
|
||||
Called from the paho network thread; *must* swallow all exceptions so that
|
||||
a bad payload or transient error does not crash the loop or drop the broker
|
||||
connection.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
payload_bytes:
|
||||
Raw bytes from the MQTT message.
|
||||
settings:
|
||||
Runtime settings snapshot (captured at subscription time). Used for
|
||||
``dsmr_sample_interval_s``.
|
||||
"""
|
||||
try:
|
||||
_handle_message_inner(payload_bytes, settings)
|
||||
except Exception:
|
||||
logger.exception("dsmr_ingest.handle_message: unexpected error (swallowed).")
|
||||
|
||||
|
||||
def _handle_message_inner(payload_bytes: bytes, settings: "Settings") -> None:
|
||||
"""Inner implementation — may raise; caller wraps in try/except."""
|
||||
# --- 1. Parse JSON ---
|
||||
try:
|
||||
data: dict = json.loads(payload_bytes)
|
||||
except (json.JSONDecodeError, ValueError):
|
||||
logger.debug("dsmr_ingest: invalid JSON payload (skipped).")
|
||||
return
|
||||
|
||||
if not isinstance(data, dict):
|
||||
logger.debug("dsmr_ingest: payload is not a JSON object (skipped).")
|
||||
return
|
||||
|
||||
# --- 2. Parse timestamp ---
|
||||
raw_ts = data.get("timestamp")
|
||||
if raw_ts is None:
|
||||
logger.debug("dsmr_ingest: missing 'timestamp' field (skipped).")
|
||||
return
|
||||
|
||||
try:
|
||||
# Python 3.11+ accepts the trailing 'Z' directly; for 3.10 compat we
|
||||
# replace 'Z' with '+00:00' before parsing.
|
||||
ts_str = raw_ts if not isinstance(raw_ts, str) else raw_ts.replace("Z", "+00:00")
|
||||
ts_utc: datetime = datetime.fromisoformat(ts_str)
|
||||
# Ensure it is timezone-aware UTC.
|
||||
if ts_utc.tzinfo is None:
|
||||
ts_utc = ts_utc.replace(tzinfo=timezone.utc)
|
||||
except (ValueError, TypeError, AttributeError):
|
||||
logger.debug(
|
||||
"dsmr_ingest: cannot parse 'timestamp' value %r (skipped).", raw_ts
|
||||
)
|
||||
return
|
||||
|
||||
# --- 3. Down-sample: only persist if second falls on interval boundary ---
|
||||
interval = settings.dsmr_sample_interval_s
|
||||
if interval > 0 and (ts_utc.second % interval) != 0:
|
||||
# This telegram is between sample points; discard silently.
|
||||
return
|
||||
|
||||
# --- 4. Extract source_id (telegram's own id) — stored only as a reference,
|
||||
# NOT used for uniqueness/idempotency (it overflows and gets reset). ---
|
||||
source_id: int | None = data.get("id")
|
||||
if source_id is not None and not isinstance(source_id, int):
|
||||
# Unexpected type — treat as missing rather than raising.
|
||||
logger.debug(
|
||||
"dsmr_ingest: 'id' field has unexpected type %s (ignoring).",
|
||||
type(source_id).__name__,
|
||||
)
|
||||
source_id = None
|
||||
|
||||
# --- 5. Persist to database ---
|
||||
# Idempotency is keyed on recorded_at (the telegram timestamp), which is
|
||||
# telegram-id-independent: a single P1 meter emits one telegram per second,
|
||||
# and down-sampling keeps at most one per interval-aligned second. The
|
||||
# UNIQUE(recorded_at) constraint is the backstop for the IntegrityError race.
|
||||
def _enabled_snapshots() -> list[DsmrSourceSnapshot]:
|
||||
session_local = get_session_local()
|
||||
session = session_local()
|
||||
try:
|
||||
existing = session.scalar(
|
||||
select(DsmrReading).where(DsmrReading.recorded_at == ts_utc)
|
||||
)
|
||||
if existing is not None:
|
||||
logger.debug(
|
||||
"dsmr_ingest: recorded_at=%s already in DB, skipping.",
|
||||
ts_utc.isoformat(),
|
||||
sources = session.scalars(
|
||||
select(MeterSource).where(MeterSource.kind == "dsmr_mqtt", MeterSource.enabled.is_(True))
|
||||
).all()
|
||||
return [_snapshot(source) for source in sources]
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
|
||||
def apply_dsmr_subscription(settings: "Settings | None" = None) -> None:
|
||||
"""Reconcile enabled DSMR source subscriptions from the database.
|
||||
|
||||
``settings`` provides the DB-merged app-wide MQTT identity. Individual
|
||||
DSMR broker settings continue to come solely from MeterSource records.
|
||||
"""
|
||||
from app.integrations.mqtt import mqtt_manager
|
||||
from app.config import get_settings
|
||||
|
||||
base_client_id = (settings or get_settings()).mqtt_client_id
|
||||
|
||||
try:
|
||||
desired = {snapshot.source_id: snapshot for snapshot in _enabled_snapshots()}
|
||||
except Exception:
|
||||
logger.exception("DSMR subscription reconcile failed while reading sources")
|
||||
return
|
||||
|
||||
# Do not retain _subscription_lock while stopping MQTT clients: a callback
|
||||
# may currently hold it through its complete dispatch, and MqttManager
|
||||
# waits for that callback before teardown returns.
|
||||
with _reconcile_lock:
|
||||
# Every source owns a distinct MQTT client, so equal topics from
|
||||
# different sources/brokers are dispatchable. A telegram and tariff
|
||||
# topic on the *same* client would overwrite one handler, however.
|
||||
rejected_source_ids: set[int] = set()
|
||||
for source_id, snapshot in list(desired.items()):
|
||||
if snapshot.tariff_topic and snapshot.topic == snapshot.tariff_topic:
|
||||
logger.error("DSMR source_id=%s rejected: telegram/tariff topic collision", source_id)
|
||||
rejected_source_ids.add(source_id)
|
||||
desired.pop(source_id)
|
||||
|
||||
with _subscription_lock:
|
||||
stale_source_ids = [
|
||||
source_id
|
||||
for source_id, current in _subscriptions.items()
|
||||
if desired.get(source_id) != current
|
||||
or _subscription_client_ids.get(source_id) != base_client_id
|
||||
]
|
||||
for source_id in stale_source_ids:
|
||||
_subscriptions.pop(source_id, None)
|
||||
_subscription_client_ids.pop(source_id, None)
|
||||
_subscription_tokens.pop(source_id, None)
|
||||
set_current_tariff(source_id, None)
|
||||
|
||||
# A disabled source has no installed MQTT owner. Persist that fact
|
||||
# after invalidating its callback token, so a retained callback cannot
|
||||
# revive an earlier online state while teardown is in progress.
|
||||
for source_id in stale_source_ids:
|
||||
_mark_disabled_source_inactive(source_id)
|
||||
|
||||
# A topic collision is a configuration error for an enabled source,
|
||||
# not a disabled-state transition. It must therefore replace any
|
||||
# earlier online state even when this process started without a
|
||||
# matching runtime subscription to tear down.
|
||||
for source_id in rejected_source_ids:
|
||||
_mark_rejected_source_error(source_id)
|
||||
|
||||
for source_id in stale_source_ids:
|
||||
mqtt_manager.remove_source(source_id)
|
||||
|
||||
for source_id, snapshot in desired.items():
|
||||
with _subscription_lock:
|
||||
current = _subscriptions.get(source_id)
|
||||
current_client_id = _subscription_client_ids.get(source_id)
|
||||
if (
|
||||
current == snapshot
|
||||
and current_client_id == base_client_id
|
||||
and mqtt_manager.source_is_active(source_id)
|
||||
):
|
||||
continue
|
||||
if current is not None:
|
||||
# The client went inactive outside reconcile. Invalidate its
|
||||
# old token before rebuilding the same snapshot.
|
||||
with _subscription_lock:
|
||||
if _subscriptions.get(source_id) == current:
|
||||
_subscriptions.pop(source_id, None)
|
||||
_subscription_client_ids.pop(source_id, None)
|
||||
_subscription_tokens.pop(source_id, None)
|
||||
mqtt_manager.remove_source(source_id)
|
||||
token = object()
|
||||
handlers = {
|
||||
snapshot.topic: lambda payload, captured=snapshot, captured_token=token: (
|
||||
handle_captured_message(payload, captured, captured_token)
|
||||
)
|
||||
}
|
||||
if snapshot.tariff_topic:
|
||||
handlers[snapshot.tariff_topic] = (
|
||||
lambda payload, captured=snapshot, captured_token=token: (
|
||||
handle_captured_tariff_message(payload, captured, captured_token)
|
||||
)
|
||||
)
|
||||
with _subscription_lock:
|
||||
_subscriptions[source_id] = snapshot
|
||||
_subscription_client_ids[source_id] = base_client_id
|
||||
_subscription_tokens[source_id] = token
|
||||
applied = mqtt_manager.replace_source(
|
||||
source_id,
|
||||
host=snapshot.broker_host,
|
||||
port=snapshot.broker_port,
|
||||
username=snapshot.username,
|
||||
password=snapshot.password,
|
||||
tls_enabled=snapshot.tls_enabled,
|
||||
subscriptions=handlers,
|
||||
base_client_id=base_client_id,
|
||||
state_handler=lambda state, captured=snapshot, captured_token=token: (
|
||||
handle_captured_source_state(captured, captured_token, state)
|
||||
),
|
||||
)
|
||||
if not applied:
|
||||
with _subscription_lock:
|
||||
if _subscription_tokens.get(source_id) is token:
|
||||
_subscriptions.pop(source_id, None)
|
||||
_subscription_client_ids.pop(source_id, None)
|
||||
_subscription_tokens.pop(source_id, None)
|
||||
|
||||
|
||||
def handle_message(payload_bytes: bytes, snapshot: DsmrSourceSnapshot) -> None:
|
||||
"""Persist one down-sampled frame under its captured source identity."""
|
||||
try:
|
||||
_handle_message_inner(payload_bytes, snapshot)
|
||||
except Exception:
|
||||
logger.exception("DSMR ingest handler failed for source_id=%s (swallowed)", snapshot.source_id)
|
||||
|
||||
|
||||
def handle_captured_message(
|
||||
payload_bytes: bytes, snapshot: DsmrSourceSnapshot, token: object | None = None
|
||||
) -> None:
|
||||
"""Run a broker callback only while its exact source generation is active."""
|
||||
with _subscription_lock:
|
||||
if token is not None:
|
||||
if _subscription_tokens.get(snapshot.source_id) is not token:
|
||||
return
|
||||
elif _subscriptions.get(snapshot.source_id) != snapshot:
|
||||
return
|
||||
handle_message(payload_bytes, snapshot)
|
||||
|
||||
|
||||
def handle_captured_tariff_message(
|
||||
payload_bytes: bytes, snapshot: DsmrSourceSnapshot, token: object | None = None
|
||||
) -> None:
|
||||
"""Ignore tariff callbacks retained from a removed/replaced source."""
|
||||
with _subscription_lock:
|
||||
if token is not None:
|
||||
if _subscription_tokens.get(snapshot.source_id) is not token:
|
||||
return
|
||||
elif _subscriptions.get(snapshot.source_id) != snapshot:
|
||||
return
|
||||
handle_tariff_message(payload_bytes, snapshot.source_id)
|
||||
|
||||
|
||||
reading = DsmrReading(
|
||||
recorded_at=ts_utc,
|
||||
source_id=source_id,
|
||||
payload=data, # full frame, verbatim
|
||||
def handle_captured_source_state(snapshot: DsmrSourceSnapshot, token: object, state: str) -> None:
|
||||
"""Persist one active generation's connection health in a short DB session."""
|
||||
with _subscription_lock:
|
||||
if _subscription_tokens.get(snapshot.source_id) is not token:
|
||||
return
|
||||
session_local = get_session_local()
|
||||
session = session_local()
|
||||
try:
|
||||
source = session.get(MeterSource, snapshot.source_id)
|
||||
if source is None or not source.enabled or source.kind != "dsmr_mqtt":
|
||||
return
|
||||
source.status = state
|
||||
source.last_error = "MQTT connection failed." if state == "error" else None
|
||||
source.updated_at = datetime.now(timezone.utc)
|
||||
session.commit()
|
||||
except Exception:
|
||||
session.rollback()
|
||||
logger.exception("DSMR source health update failed for source_id=%s", snapshot.source_id)
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
|
||||
def _mark_disabled_source_inactive(source_id: int) -> None:
|
||||
"""Clear an obsolete online health state for a disabled DSMR source.
|
||||
|
||||
The caller has already invalidated the source's generation token. This
|
||||
helper deliberately opens its own short session so reconcile never shares
|
||||
a callback-thread transaction. Deleted sources simply have no row left
|
||||
to update.
|
||||
"""
|
||||
session_local = get_session_local()
|
||||
session = session_local()
|
||||
try:
|
||||
source = session.get(MeterSource, source_id)
|
||||
if source is None or source.enabled or source.kind != "dsmr_mqtt":
|
||||
return
|
||||
source.status = "unknown"
|
||||
source.last_error = None
|
||||
source.updated_at = datetime.now(timezone.utc)
|
||||
session.commit()
|
||||
except Exception:
|
||||
session.rollback()
|
||||
logger.exception("DSMR disabled source health update failed for source_id=%s", source_id)
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
|
||||
def _mark_rejected_source_error(source_id: int) -> None:
|
||||
"""Persist a non-sensitive error for an enabled source rejected by reconcile."""
|
||||
session_local = get_session_local()
|
||||
session = session_local()
|
||||
try:
|
||||
source = session.get(MeterSource, source_id)
|
||||
if source is None or not source.enabled or source.kind != "dsmr_mqtt":
|
||||
return
|
||||
source.status = "error"
|
||||
source.last_error = "DSMR source configuration invalid."
|
||||
source.updated_at = datetime.now(timezone.utc)
|
||||
session.commit()
|
||||
except Exception:
|
||||
session.rollback()
|
||||
logger.exception("DSMR rejected source health update failed for source_id=%s", source_id)
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
|
||||
def _handle_message_inner(payload_bytes: bytes, snapshot: DsmrSourceSnapshot) -> None:
|
||||
try:
|
||||
data = json.loads(payload_bytes)
|
||||
except (json.JSONDecodeError, ValueError):
|
||||
return
|
||||
if not isinstance(data, dict):
|
||||
return
|
||||
try:
|
||||
raw_ts = data["timestamp"]
|
||||
ts_utc = datetime.fromisoformat(raw_ts.replace("Z", "+00:00"))
|
||||
if ts_utc.tzinfo is None:
|
||||
ts_utc = ts_utc.replace(tzinfo=timezone.utc)
|
||||
except (KeyError, ValueError, TypeError, AttributeError):
|
||||
return
|
||||
if snapshot.sample_interval_s > 0 and ts_utc.second % snapshot.sample_interval_s:
|
||||
return
|
||||
telegram_id = data.get("id")
|
||||
if telegram_id is not None and not isinstance(telegram_id, int):
|
||||
telegram_id = None
|
||||
|
||||
session_local = get_session_local()
|
||||
session = session_local()
|
||||
try:
|
||||
exists = session.scalar(
|
||||
select(DsmrReading.id).where(
|
||||
DsmrReading.meter_source_id == snapshot.source_id,
|
||||
DsmrReading.recorded_at == ts_utc,
|
||||
)
|
||||
)
|
||||
session.add(reading)
|
||||
if exists is None:
|
||||
session.add(
|
||||
DsmrReading(
|
||||
meter_source_id=snapshot.source_id,
|
||||
recorded_at=ts_utc,
|
||||
telegram_id=telegram_id,
|
||||
payload=data,
|
||||
)
|
||||
)
|
||||
source = session.get(MeterSource, snapshot.source_id)
|
||||
if source is not None and source.enabled and source.kind == "dsmr_mqtt":
|
||||
source.status = "online"
|
||||
source.last_seen_at = datetime.now(timezone.utc)
|
||||
source.last_error = None
|
||||
source.updated_at = datetime.now(timezone.utc)
|
||||
session.commit()
|
||||
logger.debug(
|
||||
"dsmr_ingest: persisted reading recorded_at=%s source_id=%s.",
|
||||
ts_utc.isoformat(),
|
||||
source_id,
|
||||
)
|
||||
except sqlalchemy.exc.IntegrityError:
|
||||
# Race / duplicate: another insert beat us to the same recorded_at.
|
||||
session.rollback()
|
||||
logger.debug(
|
||||
"dsmr_ingest: IntegrityError for recorded_at=%s (duplicate, skipped).",
|
||||
ts_utc.isoformat(),
|
||||
)
|
||||
except Exception:
|
||||
session.rollback()
|
||||
logger.exception("dsmr_ingest: DB error (swallowed).")
|
||||
logger.exception("DSMR database write failed for source_id=%s", snapshot.source_id)
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
+140
-42
@@ -18,6 +18,12 @@ M6 design document, extended in M7-T03 to be meter-aware:
|
||||
÷ 30 per day) and subtracts the energy-tax credit (heffingskorting,
|
||||
apportioned at EUR/year ÷ 365 per day).
|
||||
|
||||
The summary reports **both** money and energy: ``metered_import`` /
|
||||
``metered_export`` are monetary totals (Σ import_cost / Σ export_revenue),
|
||||
while ``metered_import_kwh`` / ``metered_export_kwh`` are the corresponding
|
||||
metered energy totals in kWh. The ``_kwh`` suffix is the only thing that
|
||||
distinguishes them — always check it before labelling a value in a UI.
|
||||
|
||||
Design notes
|
||||
------------
|
||||
- **Decimal arithmetic throughout**: all monetary computations use
|
||||
@@ -31,7 +37,7 @@ Design notes
|
||||
- **Register keys**: DSMR payload uses JSON strings like ``"20915.154"``
|
||||
for cumulative kWh registers. ``register_at`` converts them to Decimal.
|
||||
- **Degraded vs skip semantics**:
|
||||
- *No meter coverage* (``meter_at`` returns None for t0): write a
|
||||
- *No unique meter coverage* (no sole electricity meter at t0): write a
|
||||
``degraded=True`` row with ``meter_id=None``.
|
||||
- *Cross-meter boundary* (m0.id != m1.id for t0/t1): write a ``degraded=True``
|
||||
row with ``meter_id=m0.id``; losing this one period at the swap boundary is
|
||||
@@ -61,8 +67,8 @@ Meter-aware compute_period ordering rationale (M7-T03)
|
||||
The order of checks inside ``compute_period`` is:
|
||||
|
||||
1. **Immutability guard** (existing non-degraded row, overwrite=False) → return False.
|
||||
2. **Meter determination** (m0 = meter_at(t0), m1 = meter_at(t1)):
|
||||
- No meter (m0 is None) → write degraded, meter_id=None.
|
||||
2. **Meter determination** (m0/m1 each resolve to one electricity Meter):
|
||||
- No unique meter (m0 is None) → write degraded, meter_id=None.
|
||||
- Cross-meter boundary (m0.id != m1.id) → write degraded, meter_id=m0.id.
|
||||
3. **Active contract version check** → skip (no write) if absent.
|
||||
4. **Boundary register readings** within m0's window → write degraded if missing.
|
||||
@@ -92,8 +98,8 @@ from app.integrations.pricing.strategies import (
|
||||
get_strategy,
|
||||
)
|
||||
from app.models.energy import DsmrReading, EnergyCostPeriod, Meter
|
||||
from app.models.meter_source import MeterSource, MeterSourceBinding, MeterSourceChannel
|
||||
from app.services.contracts import active_contract_version_at, active_contract_versions
|
||||
from app.services.meters import meter_at
|
||||
from app.services.timezone import local_date, local_now
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -128,10 +134,10 @@ _MAX_DELTA_KWH = Decimal("100")
|
||||
_SETTLEMENT_OFFSET = timedelta(hours=1, minutes=5)
|
||||
|
||||
# DSMR payload register keys (cumulative kWh, JSON string values).
|
||||
_KEY_D1 = "electricity_delivered_1" # delivered low-tariff (dal / _1)
|
||||
_KEY_D2 = "electricity_delivered_2" # delivered high-tariff (normal / _2)
|
||||
_KEY_R1 = "electricity_returned_1" # returned low-tariff
|
||||
_KEY_R2 = "electricity_returned_2" # returned high-tariff
|
||||
_KEY_D1 = "electricity_delivered_1" # delivered low-tariff (dal / _1)
|
||||
_KEY_D2 = "electricity_delivered_2" # delivered high-tariff (normal / _2)
|
||||
_KEY_R1 = "electricity_returned_1" # returned low-tariff
|
||||
_KEY_R2 = "electricity_returned_2" # returned high-tariff
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -168,6 +174,23 @@ def _existing_period(session: Session, t0: datetime) -> EnergyCostPeriod | None:
|
||||
).scalar_one_or_none()
|
||||
|
||||
|
||||
def _unique_electricity_meter_at(session: Session, boundary: datetime) -> Meter | None:
|
||||
"""Return the sole electricity meter covering *boundary*, if one exists.
|
||||
|
||||
Billing must treat overlapping meter epochs as a structural ambiguity rather
|
||||
than relying on ``meter_at``'s newest-started tie breaker. A cumulative
|
||||
delta is safe only when exactly one electricity meter covers each endpoint.
|
||||
"""
|
||||
candidates = session.execute(
|
||||
select(Meter).where(
|
||||
Meter.commodity == "electricity",
|
||||
Meter.started_at <= boundary,
|
||||
(Meter.ended_at.is_(None)) | (Meter.ended_at > boundary),
|
||||
)
|
||||
).scalars().all()
|
||||
return candidates[0] if len(candidates) == 1 else None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# register_at — boundary reading lookup (meter-aware)
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -177,6 +200,8 @@ def register_at(
|
||||
session: Session,
|
||||
boundary: datetime,
|
||||
meter: Meter,
|
||||
*,
|
||||
meter_source_id: int | None = None,
|
||||
) -> dict[str, Decimal] | None:
|
||||
"""Return the four cumulative kWh register values at *boundary*, within *meter*'s window.
|
||||
|
||||
@@ -220,7 +245,7 @@ def register_at(
|
||||
"""
|
||||
# Build the meter-window constraints: [started_at, ended_at).
|
||||
meter_lower = meter.started_at # DsmrReading.recorded_at >= meter.started_at
|
||||
meter_upper = meter.ended_at # DsmrReading.recorded_at < meter.ended_at (if set)
|
||||
meter_upper = meter.ended_at # DsmrReading.recorded_at < meter.ended_at (if set)
|
||||
|
||||
stmt = (
|
||||
select(DsmrReading)
|
||||
@@ -234,6 +259,8 @@ def register_at(
|
||||
# Apply the upper bound only when the meter is closed (ended_at is not None).
|
||||
if meter_upper is not None:
|
||||
stmt = stmt.where(DsmrReading.recorded_at < meter_upper)
|
||||
if meter_source_id is not None:
|
||||
stmt = stmt.where(DsmrReading.meter_source_id == meter_source_id)
|
||||
|
||||
row: DsmrReading | None = session.execute(stmt).scalar_one_or_none()
|
||||
|
||||
@@ -267,6 +294,33 @@ def register_at(
|
||||
}
|
||||
|
||||
|
||||
def _binding_at(
|
||||
session: Session, boundary: datetime, meter: Meter
|
||||
) -> tuple[MeterSourceBinding, int] | None:
|
||||
"""Resolve the sole DSMR binding for *meter* at one period boundary.
|
||||
|
||||
Costing must not infer a cumulative domain from whichever reading happens
|
||||
to be latest. A binding anchors both the physical meter epoch and its
|
||||
source stream. Any missing or overlapping binding is therefore
|
||||
deliberately unresolvable.
|
||||
"""
|
||||
candidates = session.execute(
|
||||
select(MeterSourceBinding, MeterSourceChannel.source_id)
|
||||
.join(MeterSourceChannel, MeterSourceChannel.id == MeterSourceBinding.channel_id)
|
||||
.join(MeterSource, MeterSource.id == MeterSourceChannel.source_id)
|
||||
.where(
|
||||
MeterSourceBinding.meter_id == meter.id,
|
||||
MeterSourceBinding.started_at <= boundary,
|
||||
(MeterSourceBinding.ended_at.is_(None)) | (MeterSourceBinding.ended_at > boundary),
|
||||
MeterSource.kind == "dsmr_mqtt",
|
||||
)
|
||||
).all()
|
||||
if len(candidates) != 1:
|
||||
return None
|
||||
binding, source_id = candidates[0]
|
||||
return binding, source_id
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# compute_period — single 15-minute period
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -296,9 +350,9 @@ def compute_period(session: Session, t0: datetime, *, overwrite: bool = False) -
|
||||
Side-effects
|
||||
------------
|
||||
- Inserts or updates an ``EnergyCostPeriod`` row keyed on ``period_start=t0``.
|
||||
- If no meter covers t0 (``meter_at`` returns None for t0): inserts/updates
|
||||
a degraded row with ``meter_id=None``.
|
||||
- If the period spans a meter boundary (``meter_at(t0).id != meter_at(t1).id``):
|
||||
- If no unique meter covers t0: inserts/updates a degraded row with
|
||||
``meter_id=None``.
|
||||
- If the period spans a meter boundary (m0.id != m1.id):
|
||||
inserts/updates a degraded row with ``meter_id=m0.id`` (D5 decision).
|
||||
- If readings are missing at either boundary within the meter window:
|
||||
inserts/updates a degraded row with ``meter_id=m0.id``.
|
||||
@@ -326,7 +380,7 @@ def compute_period(session: Session, t0: datetime, *, overwrite: bool = False) -
|
||||
# is corrected and a recompute_range is triggered.
|
||||
#
|
||||
# Ordering rationale:
|
||||
# 1. No meter (m0 is None) → degraded(meter_id=None): no epoch for t0.
|
||||
# 1. No unique meter (m0 is None) → degraded(meter_id=None): no unambiguous epoch for t0.
|
||||
# 2. Cross-meter boundary (m0.id != m1.id) → degraded(meter_id=m0.id): D5.
|
||||
# 3. (Single meter, proceed) → contract check → readings → delta guard → price.
|
||||
#
|
||||
@@ -335,13 +389,13 @@ def compute_period(session: Session, t0: datetime, *, overwrite: bool = False) -
|
||||
# first, a missing-contract skip would silently discard the cross-table
|
||||
# evidence; once a contract is added and recompute runs, the engine would
|
||||
# incorrectly use cross-table reads.
|
||||
m0 = meter_at(session, t0)
|
||||
m1 = meter_at(session, t1)
|
||||
m0 = _unique_electricity_meter_at(session, t0)
|
||||
m1 = _unique_electricity_meter_at(session, t1)
|
||||
|
||||
if m0 is None:
|
||||
# No meter epoch covers t0 — degraded with no meter attribution.
|
||||
# No unambiguous meter epoch covers t0 — degraded with no attribution.
|
||||
logger.debug(
|
||||
"compute_period(%s): no active meter at t0 — writing degraded (meter_id=None).",
|
||||
"compute_period(%s): no unique active meter at t0 — writing degraded (meter_id=None).",
|
||||
t0.isoformat(),
|
||||
)
|
||||
_upsert_degraded(session, t0, now, existing, meter_id=None)
|
||||
@@ -360,6 +414,16 @@ def compute_period(session: Session, t0: datetime, *, overwrite: bool = False) -
|
||||
_upsert_degraded(session, t0, now, existing, meter_id=m0.id)
|
||||
return True
|
||||
|
||||
# Both endpoints must resolve to the same binding and source before a
|
||||
# cumulative subtraction is permitted. This is checked before contract
|
||||
# lookup so structural inconsistencies remain visible as degraded rows.
|
||||
bound0 = _binding_at(session, t0, m0)
|
||||
bound1 = _binding_at(session, t1, m1)
|
||||
if bound0 is None or bound1 is None or bound0[0].id != bound1[0].id or bound0[1] != bound1[1]:
|
||||
_upsert_degraded(session, t0, now, existing, meter_id=m0.id)
|
||||
return True
|
||||
binding, meter_source_id = bound0
|
||||
|
||||
# --- Active contract version at t0 ---
|
||||
# If there is no active contract covering t0, skip the period entirely.
|
||||
# We do not write a degraded row — there is no meaningful state to recover
|
||||
@@ -372,8 +436,8 @@ def compute_period(session: Session, t0: datetime, *, overwrite: bool = False) -
|
||||
return False
|
||||
|
||||
# --- Boundary readings within m0's meter window ---
|
||||
start_regs = register_at(session, t0, m0)
|
||||
end_regs = register_at(session, t1, m0)
|
||||
start_regs = register_at(session, t0, m0, meter_source_id=meter_source_id)
|
||||
end_regs = register_at(session, t1, m0, meter_source_id=meter_source_id)
|
||||
|
||||
if start_regs is None or end_regs is None:
|
||||
# Missing readings within the meter window → degraded with m0 attribution.
|
||||
@@ -415,9 +479,7 @@ def compute_period(session: Session, t0: datetime, *, overwrite: bool = False) -
|
||||
except TibberPriceNotFoundError:
|
||||
# Missing Tibber price → skip the period; it will be retried once the
|
||||
# price arrives (e.g. after the next Tibber refresh job runs).
|
||||
logger.debug(
|
||||
"compute_period(%s): no Tibber price found — skipping.", t0.isoformat()
|
||||
)
|
||||
logger.debug("compute_period(%s): no Tibber price found — skipping.", t0.isoformat())
|
||||
return False
|
||||
|
||||
# --- Upsert the billing record ---
|
||||
@@ -439,6 +501,7 @@ def compute_period(session: Session, t0: datetime, *, overwrite: bool = False) -
|
||||
existing.pricing = pricing
|
||||
existing.contract_version_id = version.id
|
||||
existing.meter_id = m0.id
|
||||
existing.source_binding_id = binding.id
|
||||
existing.degraded = False
|
||||
existing.computed_at = now
|
||||
else:
|
||||
@@ -455,6 +518,7 @@ def compute_period(session: Session, t0: datetime, *, overwrite: bool = False) -
|
||||
pricing=pricing,
|
||||
contract_version_id=version.id,
|
||||
meter_id=m0.id,
|
||||
source_binding_id=binding.id,
|
||||
degraded=False,
|
||||
computed_at=now,
|
||||
)
|
||||
@@ -514,6 +578,7 @@ def _upsert_degraded(
|
||||
existing.pricing = {}
|
||||
existing.contract_version_id = None
|
||||
existing.meter_id = meter_id
|
||||
existing.source_binding_id = None
|
||||
existing.degraded = True
|
||||
existing.computed_at = now
|
||||
else:
|
||||
@@ -530,6 +595,7 @@ def _upsert_degraded(
|
||||
pricing={},
|
||||
contract_version_id=None,
|
||||
meter_id=meter_id,
|
||||
source_binding_id=None,
|
||||
degraded=True,
|
||||
computed_at=now,
|
||||
)
|
||||
@@ -599,7 +665,9 @@ def compute_closed_periods(session: Session) -> int:
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def recompute_range(session: Session, start: datetime, end: datetime) -> int:
|
||||
def recompute_range(
|
||||
session: Session, start: datetime, end: datetime, *, commit: bool = True, strict: bool = False
|
||||
) -> int:
|
||||
"""Recompute (overwrite) all 15-minute periods in ``[start, end)``.
|
||||
|
||||
This is the *explicit opt-in* path for recovering from:
|
||||
@@ -620,8 +688,16 @@ def recompute_range(session: Session, start: datetime, end: datetime) -> int:
|
||||
Parameters
|
||||
----------
|
||||
session:
|
||||
Active SQLAlchemy session. The function commits after all periods
|
||||
have been processed.
|
||||
Active SQLAlchemy session.
|
||||
commit:
|
||||
When true (the default), commit after all periods have been processed.
|
||||
Callers composing this recompute with other writes may pass false and
|
||||
own the surrounding transaction themselves.
|
||||
strict:
|
||||
When true, propagate a failed period computation to the caller. This
|
||||
is for lifecycle transactions which must roll back their meter/binding
|
||||
mutation together with the cost recompute. The default remains
|
||||
best-effort for existing background and standalone callers.
|
||||
start:
|
||||
Inclusive start datetime (floored to the nearest quarter-hour internally).
|
||||
end:
|
||||
@@ -652,13 +728,16 @@ def recompute_range(session: Session, start: datetime, end: datetime) -> int:
|
||||
if did_write:
|
||||
written += 1
|
||||
except Exception:
|
||||
if strict:
|
||||
raise
|
||||
logger.exception(
|
||||
"recompute_range: unexpected error for t0=%s — continuing.",
|
||||
t0.isoformat(),
|
||||
)
|
||||
t0 += timedelta(minutes=_PERIOD_MINUTES)
|
||||
|
||||
session.commit()
|
||||
if commit:
|
||||
session.commit()
|
||||
logger.info(
|
||||
"recompute_range(%s, %s): wrote %d period(s).",
|
||||
start.isoformat(),
|
||||
@@ -731,16 +810,18 @@ def summarize(session: Session, start: datetime, end: datetime) -> dict[str, Any
|
||||
-------
|
||||
dict with keys:
|
||||
|
||||
currency str ISO 4217 currency (from contract, or "EUR" fallback)
|
||||
metered_import float Σ import_cost from non-degraded periods
|
||||
metered_export float Σ export_revenue from non-degraded periods
|
||||
metered_net float Σ net_cost from non-degraded periods
|
||||
fixed_costs float standing charges for elapsed whole local days
|
||||
credits float energy-tax credit for elapsed whole local days
|
||||
total_payable float metered_net + fixed_costs − credits
|
||||
period_count int number of non-degraded periods in range
|
||||
degraded_count int number of degraded periods in range
|
||||
days float interval length in days (total_seconds / 86400)
|
||||
currency str ISO 4217 currency (from contract, or "EUR" fallback)
|
||||
metered_import float Σ import_cost from non-degraded periods (money)
|
||||
metered_export float Σ export_revenue from non-degraded periods (money)
|
||||
metered_net float Σ net_cost from non-degraded periods (money)
|
||||
metered_import_kwh float Σ (d1_kwh + d2_kwh) from non-degraded periods (energy)
|
||||
metered_export_kwh float Σ (r1_kwh + r2_kwh) from non-degraded periods (energy)
|
||||
fixed_costs float standing charges for elapsed whole local days
|
||||
credits float energy-tax credit for elapsed whole local days
|
||||
total_payable float metered_net + fixed_costs − credits
|
||||
period_count int number of non-degraded periods in range
|
||||
degraded_count int number of degraded periods in range
|
||||
days float interval length in days (total_seconds / 86400)
|
||||
"""
|
||||
from datetime import timedelta as _td, date as _date
|
||||
|
||||
@@ -748,12 +829,16 @@ def summarize(session: Session, start: datetime, end: datetime) -> dict[str, Any
|
||||
end_utc = _as_utc(end)
|
||||
|
||||
# --- Fetch all EnergyCostPeriod rows in [start, end) ---
|
||||
rows = session.execute(
|
||||
select(EnergyCostPeriod).where(
|
||||
EnergyCostPeriod.period_start >= start_utc,
|
||||
EnergyCostPeriod.period_start < end_utc,
|
||||
rows = (
|
||||
session.execute(
|
||||
select(EnergyCostPeriod).where(
|
||||
EnergyCostPeriod.period_start >= start_utc,
|
||||
EnergyCostPeriod.period_start < end_utc,
|
||||
)
|
||||
)
|
||||
).scalars().all()
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
|
||||
good_rows = [r for r in rows if not r.degraded]
|
||||
degraded_rows = [r for r in rows if r.degraded]
|
||||
@@ -763,6 +848,15 @@ def summarize(session: Session, start: datetime, end: datetime) -> dict[str, Any
|
||||
sum_export = sum((_to_decimal(r.export_revenue) for r in good_rows), Decimal("0"))
|
||||
sum_net = sum((_to_decimal(r.net_cost) for r in good_rows), Decimal("0"))
|
||||
|
||||
# Σ metered energy (kWh), summed across both tariff registers. Reuses the
|
||||
# already-fetched ``good_rows`` so no extra query is issued.
|
||||
sum_import_kwh = sum(
|
||||
(_to_decimal(r.d1_kwh) + _to_decimal(r.d2_kwh) for r in good_rows), Decimal("0")
|
||||
)
|
||||
sum_export_kwh = sum(
|
||||
(_to_decimal(r.r1_kwh) + _to_decimal(r.r2_kwh) for r in good_rows), Decimal("0")
|
||||
)
|
||||
|
||||
# --- Interval length in days (window, not elapsed — kept for API compat) ---
|
||||
total_seconds = (end_utc - start_utc).total_seconds()
|
||||
days = _to_decimal(str(total_seconds)) / _to_decimal("86400")
|
||||
@@ -846,7 +940,9 @@ def summarize(session: Session, start: datetime, end: datetime) -> dict[str, Any
|
||||
version_segments: list[tuple[_date, _date | None, dict]] = []
|
||||
for v in versions:
|
||||
v_start_local = local_date(_as_utc(v.effective_from))
|
||||
v_end_local = local_date(_as_utc(v.effective_to)) if v.effective_to is not None else None
|
||||
v_end_local = (
|
||||
local_date(_as_utc(v.effective_to)) if v.effective_to is not None else None
|
||||
)
|
||||
version_segments.append((v_start_local, v_end_local, v.values or {}))
|
||||
|
||||
for v_start, v_end_excl, v_values in version_segments:
|
||||
@@ -885,6 +981,8 @@ def summarize(session: Session, start: datetime, end: datetime) -> dict[str, Any
|
||||
"metered_import": float(sum_import),
|
||||
"metered_export": float(sum_export),
|
||||
"metered_net": float(sum_net),
|
||||
"metered_import_kwh": float(sum_import_kwh),
|
||||
"metered_export_kwh": float(sum_export_kwh),
|
||||
"fixed_costs": float(fixed_dec),
|
||||
"credits": float(credits_dec),
|
||||
"total_payable": float(total_payable),
|
||||
|
||||
@@ -98,6 +98,15 @@ def _availability_topic(device_uuid: str, prefix: str) -> str:
|
||||
return f"{prefix}/modbus/{node}/availability"
|
||||
|
||||
|
||||
def _availability_id(entity: ExposableEntity) -> str:
|
||||
"""Return the identity which owns this entity's liveness topic.
|
||||
|
||||
M8 meters deliberately retain their own UUID as HA node/unique identity,
|
||||
while their availability is supplied by a MeterSource UUID.
|
||||
"""
|
||||
return entity.device.availability_id or entity.device.identifiers[1]
|
||||
|
||||
|
||||
def _unique_id(entity: ExposableEntity) -> str:
|
||||
"""Stable unique_id — device uuid + metric key (never from mutable fields)."""
|
||||
device_uuid = entity.device.identifiers[1]
|
||||
@@ -139,8 +148,7 @@ def build_discovery_payload(
|
||||
if state_prefix is None:
|
||||
state_prefix = discovery_prefix
|
||||
|
||||
device_uuid = entity.device.identifiers[1]
|
||||
avail_topic = _availability_topic(device_uuid, state_prefix)
|
||||
avail_topic = _availability_topic(_availability_id(entity), state_prefix)
|
||||
state_t = _state_topic(entity, state_prefix)
|
||||
topic = _discovery_topic(entity, discovery_prefix)
|
||||
|
||||
@@ -212,6 +220,21 @@ def publish_discovery(session: Session) -> None:
|
||||
logger.exception("publish_discovery: failed to build catalog; aborting")
|
||||
return
|
||||
|
||||
# Meter UUIDs are intentionally identity-changing epochs. Discovery config
|
||||
# is retained, so clear only the precisely enumerable old M8 identities;
|
||||
# never wildcard a provider/topic and risk removing another source's card.
|
||||
try:
|
||||
stale_entities = _stale_m8_entities(session)
|
||||
except Exception:
|
||||
logger.exception("publish_discovery: unable to enumerate old M8 identities")
|
||||
stale_entities = []
|
||||
for old_entity in stale_entities:
|
||||
try:
|
||||
old_topic, _ = build_discovery_payload(old_entity, discovery_prefix, state_prefix)
|
||||
mqtt_manager.publish(old_topic, b"", retain=True)
|
||||
except Exception:
|
||||
logger.exception("publish_discovery: unable to clear old identity %r", old_entity.key)
|
||||
|
||||
for entry in catalog:
|
||||
entity = entry.entity
|
||||
try:
|
||||
@@ -236,6 +259,62 @@ def publish_discovery(session: Session) -> None:
|
||||
)
|
||||
|
||||
|
||||
def _stale_m8_entities(session: Session) -> list[ExposableEntity]:
|
||||
"""Return synthetic discovery entries for superseded thermal identities.
|
||||
|
||||
This is deliberately a narrow, best-effort cleanup: ended Meter UUIDs and
|
||||
historically possible thermal combinations only; current identities are excluded.
|
||||
"""
|
||||
from app.integrations.expose import DeviceInfo
|
||||
from app.models.energy import Meter
|
||||
from sqlalchemy import select
|
||||
|
||||
meters = session.execute(select(Meter).where(
|
||||
Meter.commodity.in_(("electricity", "heating", "hot_water"))
|
||||
)).scalars().all()
|
||||
current = {meter.commodity: meter for meter in meters if meter.ended_at is None}
|
||||
old = [meter for meter in meters if meter.ended_at is not None]
|
||||
entities: list[ExposableEntity] = []
|
||||
for meter in old:
|
||||
info = DeviceInfo(identifiers=("meter", meter.uuid), name=meter.label)
|
||||
for suffix in ("total", "today"):
|
||||
entities.append(ExposableEntity(
|
||||
key=f"meter.{meter.uuid}.{suffix}", component="sensor", device=info,
|
||||
device_class=None, unit="", name="obsolete",
|
||||
))
|
||||
heatings = [meter for meter in meters if meter.commodity == "heating"]
|
||||
waters = [meter for meter in meters if meter.commodity == "hot_water"]
|
||||
current_identity = (
|
||||
".".join(sorted((current["heating"].uuid, current["hot_water"].uuid)))
|
||||
if current.get("heating") is not None and current.get("hot_water") is not None else None
|
||||
)
|
||||
for heating in heatings:
|
||||
for water in waters:
|
||||
if heating.ended_at is None and water.ended_at is None:
|
||||
continue
|
||||
# A thermal identity can only have been published when both Meter
|
||||
# epochs were current at the same instant. Do not form a Cartesian
|
||||
# product of historical records: that would tombstone identities
|
||||
# which have never existed in HA.
|
||||
heating_start, water_start = heating.started_at, water.started_at
|
||||
heating_end, water_end = heating.ended_at, water.ended_at
|
||||
if (heating_end is not None and water_start >= heating_end) or (
|
||||
water_end is not None and heating_start >= water_end
|
||||
):
|
||||
continue
|
||||
identity = ".".join(sorted((heating.uuid, water.uuid)))
|
||||
if identity == current_identity:
|
||||
continue
|
||||
info = DeviceInfo(identifiers=("thermal-cost", identity), name="obsolete")
|
||||
for metric in ("heating", "hot_water_heating", "water", "water_tax", "fixed", "all_in"):
|
||||
for suffix in ("total", "today"):
|
||||
entities.append(ExposableEntity(
|
||||
key=f"thermal_cost.{identity}.{metric}_{suffix}", component="sensor", device=info,
|
||||
device_class=None, unit="", name="obsolete",
|
||||
))
|
||||
return entities
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Public: publish states
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -288,7 +367,19 @@ def _publish_entity_state(
|
||||
Also publishes the availability topic for ``binary_sensor`` "online" entities.
|
||||
"""
|
||||
state_t = _state_topic(entity, prefix)
|
||||
device_uuid = entity.device.identifiers[1]
|
||||
# Source-backed entities can have a different liveness identity from their
|
||||
# HA device identity. Publish it before the state; a None value below is
|
||||
# intentionally not converted to a synthetic zero.
|
||||
if entity.device.provides_availability and entity.device.availability_getter is not None:
|
||||
try:
|
||||
available = bool(entity.device.availability_getter(session))
|
||||
mqtt_manager.publish(
|
||||
_availability_topic(_availability_id(entity), prefix),
|
||||
"online" if available else "offline",
|
||||
retain=False,
|
||||
)
|
||||
except Exception:
|
||||
logger.exception("availability_getter raised for entity %r", entity.key)
|
||||
|
||||
if entity.component == "binary_sensor" and "online" in entity.key:
|
||||
# The online sensor represents device availability.
|
||||
@@ -303,7 +394,7 @@ def _publish_entity_state(
|
||||
# Default to offline when no reading is available.
|
||||
online = (raw_value == "ON")
|
||||
avail_payload = "online" if online else "offline"
|
||||
avail_topic = _availability_topic(device_uuid, prefix)
|
||||
avail_topic = _availability_topic(_availability_id(entity), prefix)
|
||||
mqtt_manager.publish(avail_topic, avail_payload, retain=False)
|
||||
# The state of the binary_sensor itself
|
||||
state_payload = "ON" if online else "OFF"
|
||||
|
||||
@@ -0,0 +1,298 @@
|
||||
"""Thermal (WarmteLink) 15-minute cost ledger.
|
||||
|
||||
This module deliberately does not share the electricity ledger: thermal has
|
||||
two independently-bound cumulative domains and Decimal database columns.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from datetime import UTC, date, datetime, time, timedelta
|
||||
from decimal import Decimal
|
||||
from typing import Any
|
||||
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.models.energy import Meter, MeterCostPeriod
|
||||
from app.models.meter_source import MeterSource, MeterSourceBinding, MeterSourceChannel, WarmteLinkReading
|
||||
from app.services.contracts import active_contract_version_at, active_contract_versions
|
||||
from app.services.energy_cost import floor_to_quarter
|
||||
from app.services import timezone as timezone_service
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_PERIOD = timedelta(minutes=15)
|
||||
_FRESHNESS = timedelta(seconds=120)
|
||||
_LIMITS = {"heating": Decimal("0.1"), "hot_water": Decimal("1")}
|
||||
_ACCEPTED_QUALITIES = {"valid", "unverifiable"}
|
||||
_SETTLEMENT_TIME = time(1, 5)
|
||||
|
||||
|
||||
def _utc(value: datetime) -> datetime:
|
||||
return value.replace(tzinfo=UTC) if value.tzinfo is None else value.astimezone(UTC)
|
||||
|
||||
|
||||
def _decimal(value: Any) -> Decimal:
|
||||
return value if isinstance(value, Decimal) else Decimal(str(value))
|
||||
|
||||
|
||||
def _existing(session: Session, commodity: str, start: datetime) -> MeterCostPeriod | None:
|
||||
return session.execute(
|
||||
select(MeterCostPeriod).where(
|
||||
MeterCostPeriod.commodity == commodity, MeterCostPeriod.period_start == start
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
|
||||
|
||||
def _meter_at(session: Session, commodity: str, instant: datetime) -> Meter | None:
|
||||
meters = session.execute(
|
||||
select(Meter).where(
|
||||
Meter.commodity == commodity,
|
||||
Meter.started_at <= instant,
|
||||
(Meter.ended_at.is_(None)) | (Meter.ended_at > instant),
|
||||
)
|
||||
).scalars().all()
|
||||
return meters[0] if len(meters) == 1 else None
|
||||
|
||||
|
||||
def _binding_at(
|
||||
session: Session, meter: Meter, instant: datetime
|
||||
) -> tuple[MeterSourceBinding, MeterSourceChannel] | None:
|
||||
rows = session.execute(
|
||||
select(MeterSourceBinding, MeterSourceChannel)
|
||||
.join(MeterSourceChannel, MeterSourceChannel.id == MeterSourceBinding.channel_id)
|
||||
.join(MeterSource, MeterSource.id == MeterSourceChannel.source_id)
|
||||
.where(
|
||||
MeterSourceBinding.meter_id == meter.id,
|
||||
MeterSourceBinding.started_at <= instant,
|
||||
(MeterSourceBinding.ended_at.is_(None)) | (MeterSourceBinding.ended_at > instant),
|
||||
MeterSource.kind == "warmtelink_serial",
|
||||
)
|
||||
).all()
|
||||
return rows[0] if len(rows) == 1 else None
|
||||
|
||||
|
||||
def _reading_at(
|
||||
session: Session,
|
||||
channel: MeterSourceChannel,
|
||||
meter: Meter,
|
||||
binding: MeterSourceBinding,
|
||||
target: datetime,
|
||||
) -> WarmteLinkReading | None:
|
||||
"""Choose the nearest accepted reading inside this cumulative domain.
|
||||
|
||||
Freshness alone is insufficient: a frame immediately before a meter or
|
||||
source hand-off belongs to a different cumulative register and must never
|
||||
be used as the other side of a delta.
|
||||
"""
|
||||
window_start, window_end = target - _FRESHNESS, target + _FRESHNESS
|
||||
rows = session.execute(
|
||||
select(WarmteLinkReading)
|
||||
.where(
|
||||
WarmteLinkReading.channel_id == channel.id,
|
||||
WarmteLinkReading.recorded_at >= window_start,
|
||||
WarmteLinkReading.recorded_at <= window_end,
|
||||
)
|
||||
).scalars().all()
|
||||
domain_start = max(_utc(meter.started_at), _utc(binding.started_at))
|
||||
domain_ends = (meter.ended_at, binding.ended_at)
|
||||
domain_end = min((_utc(value) for value in domain_ends if value is not None), default=None)
|
||||
accepted = [
|
||||
row for row in rows
|
||||
if row.quality in _ACCEPTED_QUALITIES
|
||||
and _utc(row.recorded_at) >= domain_start
|
||||
and (domain_end is None or _utc(row.recorded_at) < domain_end)
|
||||
]
|
||||
if not accepted:
|
||||
return None
|
||||
return min(
|
||||
accepted,
|
||||
key=lambda row: (abs((_utc(row.recorded_at) - target).total_seconds()), _utc(row.recorded_at)),
|
||||
)
|
||||
|
||||
|
||||
def _degrade(
|
||||
session: Session,
|
||||
commodity: str,
|
||||
start: datetime,
|
||||
end: datetime,
|
||||
existing: MeterCostPeriod | None,
|
||||
reason: str,
|
||||
meter_id: int | None = None,
|
||||
binding_id: int | None = None,
|
||||
) -> None:
|
||||
now = datetime.now(UTC)
|
||||
fields = dict(
|
||||
period_end=end, meter_id=meter_id, source_binding_id=binding_id,
|
||||
contract_version_id=None, quantity=Decimal("0"), cost=Decimal("0"), currency="EUR",
|
||||
cost_breakdown={}, pricing_snapshot={}, quality="invalid", degraded=True,
|
||||
degraded_reason=reason, updated_at=now,
|
||||
)
|
||||
if existing is None:
|
||||
session.add(MeterCostPeriod(commodity=commodity, period_start=start, created_at=now, **fields))
|
||||
else:
|
||||
for key, value in fields.items():
|
||||
setattr(existing, key, value)
|
||||
|
||||
|
||||
def compute_period(
|
||||
session: Session, commodity: str, period_start: datetime, *, overwrite: bool = False
|
||||
) -> bool:
|
||||
"""Compute one closed thermal period, recording every unsafe input as degraded."""
|
||||
if commodity not in _LIMITS:
|
||||
raise ValueError("commodity must be heating or hot_water")
|
||||
start = floor_to_quarter(_utc(period_start))
|
||||
end = start + _PERIOD
|
||||
existing = _existing(session, commodity, start)
|
||||
if existing is not None and not existing.degraded and not overwrite:
|
||||
return False
|
||||
|
||||
meter0, meter1 = _meter_at(session, commodity, start), _meter_at(session, commodity, end)
|
||||
if meter0 is None:
|
||||
_degrade(session, commodity, start, end, existing, "missing_or_ambiguous_meter")
|
||||
return True
|
||||
if meter1 is None or meter1.id != meter0.id:
|
||||
_degrade(session, commodity, start, end, existing, "cross_meter_epoch", meter0.id)
|
||||
return True
|
||||
bound0, bound1 = _binding_at(session, meter0, start), _binding_at(session, meter1, end)
|
||||
if bound0 is None or bound1 is None:
|
||||
_degrade(session, commodity, start, end, existing, "missing_or_ambiguous_binding", meter0.id)
|
||||
return True
|
||||
binding, channel = bound0
|
||||
if bound1[0].id != binding.id or bound1[1].id != channel.id:
|
||||
_degrade(session, commodity, start, end, existing, "cross_source_binding", meter0.id, binding.id)
|
||||
return True
|
||||
first = _reading_at(session, channel, meter0, binding, start)
|
||||
last = _reading_at(session, channel, meter0, binding, end)
|
||||
if first is None or last is None:
|
||||
_degrade(session, commodity, start, end, existing, "missing_stale_or_invalid_reading", meter0.id, binding.id)
|
||||
return True
|
||||
delta = _decimal(last.value) - _decimal(first.value)
|
||||
if delta < 0:
|
||||
_degrade(session, commodity, start, end, existing, "negative_delta", meter0.id, binding.id)
|
||||
return True
|
||||
if delta > _LIMITS[commodity]:
|
||||
_degrade(session, commodity, start, end, existing, "delta_limit_exceeded", meter0.id, binding.id)
|
||||
return True
|
||||
version = active_contract_version_at(session, start, scope="thermal")
|
||||
if version is None:
|
||||
_degrade(session, commodity, start, end, existing, "missing_contract", meter0.id, binding.id)
|
||||
return True
|
||||
|
||||
values = {key: _decimal(value) for key, value in version.values["variable"].items()}
|
||||
if commodity == "heating":
|
||||
breakdown = {"heating": delta * values["heating"]}
|
||||
else:
|
||||
breakdown = {
|
||||
key: delta * values[key]
|
||||
for key in ("hot_water_heating", "hot_water", "hot_water_tax")
|
||||
}
|
||||
cost = sum(breakdown.values(), Decimal("0"))
|
||||
now = datetime.now(UTC)
|
||||
fields = dict(
|
||||
period_end=end, meter_id=meter0.id, source_binding_id=binding.id,
|
||||
contract_version_id=version.id, quantity=delta, cost=cost, currency=version.contract.currency,
|
||||
cost_breakdown=breakdown, pricing_snapshot=dict(version.values),
|
||||
quality="valid" if first.quality == last.quality == "valid" else "unverifiable",
|
||||
degraded=False, degraded_reason=None, updated_at=now,
|
||||
)
|
||||
if existing is None:
|
||||
session.add(MeterCostPeriod(commodity=commodity, period_start=start, created_at=now, **fields))
|
||||
else:
|
||||
for key, value in fields.items():
|
||||
setattr(existing, key, value)
|
||||
return True
|
||||
|
||||
|
||||
def compute_closed_periods(session: Session, *, now: datetime | None = None) -> int:
|
||||
"""Retry incomplete thermal rows and fill recent closed periods without touching good rows."""
|
||||
now = _utc(now or datetime.now(UTC))
|
||||
first = floor_to_quarter(now - timedelta(days=7))
|
||||
written = 0
|
||||
cursor = first
|
||||
while cursor + _PERIOD <= now:
|
||||
for commodity in ("heating", "hot_water"):
|
||||
if compute_period(session, commodity, cursor):
|
||||
written += 1
|
||||
cursor += _PERIOD
|
||||
session.commit()
|
||||
return written
|
||||
|
||||
|
||||
def recompute_range(session: Session, start: datetime, end: datetime, *, commit: bool = True) -> int:
|
||||
"""Recompute a thermal range.
|
||||
|
||||
The historical service entry point remains self-committing for the scheduler
|
||||
and direct callers. HTTP callers pass ``commit=False`` so validation,
|
||||
recomputation, response statistics, and the single commit share one
|
||||
transaction owned by the route.
|
||||
"""
|
||||
cursor, end = floor_to_quarter(_utc(start)), _utc(end)
|
||||
now, written = datetime.now(UTC), 0
|
||||
while cursor < end:
|
||||
if cursor + _PERIOD <= now:
|
||||
for commodity in ("heating", "hot_water"):
|
||||
if compute_period(session, commodity, cursor, overwrite=True):
|
||||
written += 1
|
||||
cursor += _PERIOD
|
||||
if commit:
|
||||
session.commit()
|
||||
return written
|
||||
|
||||
|
||||
def _settled_end_date(now: datetime) -> date:
|
||||
local_now = timezone_service.to_local(now)
|
||||
return local_now.date() if local_now.timetz().replace(tzinfo=None) >= _SETTLEMENT_TIME else local_now.date() - timedelta(days=1)
|
||||
|
||||
|
||||
def summarize(session: Session, start: datetime, end: datetime, *, now: datetime | None = None) -> dict[str, Any]:
|
||||
"""Return thermal variable/fixed totals; standing is charged once per contract/day."""
|
||||
start, end = _utc(start), _utc(end)
|
||||
rows = session.execute(select(MeterCostPeriod).where(
|
||||
MeterCostPeriod.period_start >= start, MeterCostPeriod.period_start < end
|
||||
)).scalars().all()
|
||||
good = [row for row in rows if not row.degraded]
|
||||
variable = sum((_decimal(row.cost) for row in good), Decimal("0"))
|
||||
breakdown: dict[str, Decimal] = {key: Decimal("0") for key in (
|
||||
"heating", "hot_water_heating", "hot_water", "hot_water_tax")}
|
||||
for row in good:
|
||||
for key, value in row.cost_breakdown.items():
|
||||
breakdown[key] = breakdown.get(key, Decimal("0")) + _decimal(value)
|
||||
|
||||
# A summary is half-open. ``end`` at local midnight has no overlap with
|
||||
# that next local date, and an empty/reversed range owns no standing day.
|
||||
final_day = timezone_service.local_date(end - timedelta(microseconds=1))
|
||||
final_day = min(final_day, _settled_end_date(now or datetime.now(UTC)))
|
||||
day = timezone_service.local_date(start)
|
||||
fixed_breakdown: dict[str, Decimal] = {key: Decimal("0") for key in (
|
||||
"heating_network", "metering", "delivery_set", "hot_water_network", "other"
|
||||
)}
|
||||
versions = active_contract_versions(session, scope="thermal")
|
||||
while start < end and day <= final_day:
|
||||
day_start = datetime.combine(day, time.min, tzinfo=timezone_service.local_tz()).astimezone(UTC)
|
||||
next_day_start = datetime.combine(
|
||||
day + timedelta(days=1), time.min, tzinfo=timezone_service.local_tz()
|
||||
).astimezone(UTC)
|
||||
local_day_seconds = Decimal(str((next_day_start - day_start).total_seconds()))
|
||||
# A rate revision part-way through a local date is attributable only
|
||||
# to its effective interval. This preserves one contract-level daily
|
||||
# charge while correctly handling first-version and intra-day changes.
|
||||
for version in versions:
|
||||
segment_start = max(day_start, _utc(version.effective_from))
|
||||
version_end = _utc(version.effective_to) if version.effective_to is not None else next_day_start
|
||||
segment_end = min(next_day_start, version_end)
|
||||
if segment_start >= segment_end:
|
||||
continue
|
||||
values = version.values["standing"]
|
||||
fraction = Decimal(str((segment_end - segment_start).total_seconds())) / local_day_seconds
|
||||
for key in fixed_breakdown:
|
||||
fixed_breakdown[key] += _decimal(values.get(key, "0")) / Decimal("365") * fraction
|
||||
day += timedelta(days=1)
|
||||
fixed = sum(fixed_breakdown.values(), Decimal("0"))
|
||||
return {
|
||||
"currency": good[0].currency if good else "EUR", "variable_cost": variable,
|
||||
"fixed_cost": fixed, "fixed_breakdown": fixed_breakdown,
|
||||
"total_cost": variable + fixed, "breakdown": breakdown,
|
||||
"period_count": len(good), "degraded_count": len(rows) - len(good),
|
||||
}
|
||||
@@ -0,0 +1,529 @@
|
||||
"""Service layer for source/channel discovery and meter-source bindings.
|
||||
|
||||
All mutating functions receive a caller-owned :class:`~sqlalchemy.orm.Session`
|
||||
and never commit. This lets HTTP handlers compose source and meter changes in
|
||||
one transaction later without exposing any connection I/O here.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import UTC, datetime
|
||||
from typing import Any
|
||||
|
||||
from sqlalchemy import or_, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.integrations.meter_sources import (
|
||||
SourceProfileError,
|
||||
get_source_profile,
|
||||
merge_source_config,
|
||||
validate_source_config,
|
||||
)
|
||||
from app.models.energy import Meter
|
||||
from app.models.meter_source import (
|
||||
MeterSource,
|
||||
MeterSourceBinding,
|
||||
MeterSourceChannel,
|
||||
half_open_intervals_overlap,
|
||||
)
|
||||
|
||||
|
||||
class MeterSourceError(ValueError):
|
||||
"""Base class for source-domain validation errors."""
|
||||
|
||||
|
||||
class SourceNotFoundError(MeterSourceError):
|
||||
"""Raised when the requested source does not exist."""
|
||||
|
||||
|
||||
class ChannelNotFoundError(MeterSourceError):
|
||||
"""Raised when the requested source channel does not exist."""
|
||||
|
||||
|
||||
class MeterNotFoundError(MeterSourceError):
|
||||
"""Raised when the requested meter does not exist."""
|
||||
|
||||
|
||||
class BindingNotFoundError(MeterSourceError):
|
||||
"""Raised when the requested binding does not exist."""
|
||||
|
||||
|
||||
class BindingValidationError(MeterSourceError):
|
||||
"""Raised for an incompatible unit, commodity, or invalid interval."""
|
||||
|
||||
|
||||
class BindingOverlapError(BindingValidationError):
|
||||
"""Raised when a meter or channel already has an overlapping binding."""
|
||||
|
||||
|
||||
class SourceDeleteRestrictedError(MeterSourceError):
|
||||
"""Raised when a source has retained channel, binding, or reading history."""
|
||||
|
||||
|
||||
COMMODITY_UNITS = {"electricity": "kWh", "heating": "GJ", "hot_water": "m³"}
|
||||
_UNSET = object()
|
||||
|
||||
|
||||
def _utc_now() -> datetime:
|
||||
return datetime.now(UTC)
|
||||
|
||||
|
||||
def _as_utc(value: datetime) -> datetime:
|
||||
return value.replace(tzinfo=UTC) if value.tzinfo is None else value.astimezone(UTC)
|
||||
|
||||
|
||||
def get_source(session: Session, source_id: int) -> MeterSource:
|
||||
source = session.get(MeterSource, source_id)
|
||||
if source is None:
|
||||
raise SourceNotFoundError(f"Meter source {source_id} was not found.")
|
||||
return source
|
||||
|
||||
|
||||
def list_sources(session: Session, *, kind: str | None = None) -> list[MeterSource]:
|
||||
statement = select(MeterSource).order_by(MeterSource.id)
|
||||
if kind is not None:
|
||||
get_source_profile(kind)
|
||||
statement = statement.where(MeterSource.kind == kind)
|
||||
return list(session.execute(statement).scalars())
|
||||
|
||||
|
||||
def create_source(
|
||||
session: Session,
|
||||
*,
|
||||
name: str,
|
||||
kind: str,
|
||||
config: dict[str, Any],
|
||||
enabled: bool = True,
|
||||
) -> MeterSource:
|
||||
"""Add a source after validating its complete kind-specific config."""
|
||||
now = _utc_now()
|
||||
source = MeterSource(
|
||||
name=name,
|
||||
kind=kind,
|
||||
enabled=enabled,
|
||||
config=validate_source_config(kind, config),
|
||||
created_at=now,
|
||||
updated_at=now,
|
||||
)
|
||||
session.add(source)
|
||||
return source
|
||||
|
||||
|
||||
def update_source(
|
||||
session: Session,
|
||||
source_id: int,
|
||||
*,
|
||||
name: str | None = None,
|
||||
enabled: bool | None = None,
|
||||
config_patch: dict[str, Any] | None = None,
|
||||
) -> MeterSource:
|
||||
"""Update source metadata and/or merge a partial source config without commit."""
|
||||
source = get_source(session, source_id)
|
||||
if name is not None:
|
||||
source.name = name
|
||||
if enabled is not None:
|
||||
source.enabled = enabled
|
||||
if config_patch is not None:
|
||||
source.config = merge_source_config(source.kind, source.config, config_patch)
|
||||
source.updated_at = _utc_now()
|
||||
return source
|
||||
|
||||
|
||||
def delete_source(session: Session, source_id: int) -> None:
|
||||
"""Delete an entirely unused source; history is always retained instead."""
|
||||
source = get_source(session, source_id)
|
||||
has_channel = session.execute(
|
||||
select(MeterSourceChannel.id).where(MeterSourceChannel.source_id == source.id).limit(1)
|
||||
).scalar_one_or_none()
|
||||
has_binding = session.execute(
|
||||
select(MeterSourceBinding.id)
|
||||
.join(MeterSourceChannel)
|
||||
.where(MeterSourceChannel.source_id == source.id)
|
||||
.limit(1)
|
||||
).scalar_one_or_none()
|
||||
if has_channel is not None or has_binding is not None:
|
||||
raise SourceDeleteRestrictedError(
|
||||
f"Meter source {source_id} has dependent channels, bindings, or readings."
|
||||
)
|
||||
session.delete(source)
|
||||
|
||||
|
||||
def get_channel(session: Session, channel_id: int) -> MeterSourceChannel:
|
||||
channel = session.get(MeterSourceChannel, channel_id)
|
||||
if channel is None:
|
||||
raise ChannelNotFoundError(f"Meter source channel {channel_id} was not found.")
|
||||
return channel
|
||||
|
||||
|
||||
def upsert_discovered_channel(
|
||||
session: Session,
|
||||
*,
|
||||
source_id: int,
|
||||
channel_key: str,
|
||||
label: str,
|
||||
unit: str,
|
||||
suggested_commodity: str | None = None,
|
||||
device_type: str | None = None,
|
||||
fingerprint: str | None = None,
|
||||
latest_value: Any = None,
|
||||
latest_at: datetime | None = None,
|
||||
latest_quality: str | None = None,
|
||||
) -> MeterSourceChannel:
|
||||
"""Idempotently create or refresh a discovered channel's metadata.
|
||||
|
||||
``suggested_commodity`` remains metadata only; this function never creates
|
||||
a meter or a binding.
|
||||
"""
|
||||
source = get_source(session, source_id)
|
||||
if unit not in get_source_profile(source.kind).allowed_units:
|
||||
raise SourceProfileError(f"Unit {unit!r} is not allowed for source kind {source.kind!r}.")
|
||||
channel = session.execute(
|
||||
select(MeterSourceChannel).where(
|
||||
MeterSourceChannel.source_id == source.id,
|
||||
MeterSourceChannel.channel_key == channel_key,
|
||||
)
|
||||
).scalar_one_or_none()
|
||||
now = _utc_now()
|
||||
if channel is None:
|
||||
channel = MeterSourceChannel(
|
||||
source_id=source.id,
|
||||
channel_key=channel_key,
|
||||
label=label,
|
||||
unit=unit,
|
||||
suggested_commodity=suggested_commodity,
|
||||
device_type=device_type,
|
||||
fingerprint=fingerprint,
|
||||
latest_value=latest_value,
|
||||
latest_at=latest_at,
|
||||
latest_quality=latest_quality,
|
||||
created_at=now,
|
||||
updated_at=now,
|
||||
)
|
||||
session.add(channel)
|
||||
return channel
|
||||
|
||||
if channel.unit != unit:
|
||||
binding_id = session.execute(
|
||||
select(MeterSourceBinding.id)
|
||||
.where(MeterSourceBinding.channel_id == channel.id)
|
||||
.limit(1)
|
||||
).scalar_one_or_none()
|
||||
if binding_id is not None:
|
||||
raise BindingValidationError(
|
||||
f"Cannot change unit of bound channel {channel.id} from {channel.unit!r} to {unit!r}."
|
||||
)
|
||||
|
||||
channel.label = label
|
||||
channel.unit = unit
|
||||
channel.suggested_commodity = suggested_commodity
|
||||
channel.device_type = device_type
|
||||
channel.fingerprint = fingerprint
|
||||
channel.latest_value = latest_value
|
||||
channel.latest_at = latest_at
|
||||
channel.latest_quality = latest_quality
|
||||
channel.updated_at = now
|
||||
return channel
|
||||
|
||||
|
||||
def list_bindings(
|
||||
session: Session, *, meter_id: int | None = None, channel_id: int | None = None
|
||||
) -> list[MeterSourceBinding]:
|
||||
statement = select(MeterSourceBinding).order_by(MeterSourceBinding.started_at, MeterSourceBinding.id)
|
||||
if meter_id is not None:
|
||||
statement = statement.where(MeterSourceBinding.meter_id == meter_id)
|
||||
if channel_id is not None:
|
||||
statement = statement.where(MeterSourceBinding.channel_id == channel_id)
|
||||
return list(session.execute(statement).scalars())
|
||||
|
||||
|
||||
def _get_meter(session: Session, meter_id: int) -> Meter:
|
||||
meter = session.get(Meter, meter_id)
|
||||
if meter is None:
|
||||
raise MeterNotFoundError(f"Meter {meter_id} was not found.")
|
||||
return meter
|
||||
|
||||
|
||||
def _validate_binding(
|
||||
session: Session,
|
||||
*,
|
||||
meter_id: int,
|
||||
channel_id: int,
|
||||
started_at: datetime,
|
||||
ended_at: datetime | None,
|
||||
excluding_ids: set[int] | None = None,
|
||||
) -> None:
|
||||
meter = _get_meter(session, meter_id)
|
||||
channel = get_channel(session, channel_id)
|
||||
expected_unit = COMMODITY_UNITS.get(meter.commodity)
|
||||
if expected_unit is None:
|
||||
raise BindingValidationError(f"Commodity {meter.commodity!r} cannot be bound to a source channel.")
|
||||
if channel.unit != expected_unit:
|
||||
raise BindingValidationError(
|
||||
f"Meter commodity {meter.commodity!r} requires unit {expected_unit!r}, "
|
||||
f"but channel has {channel.unit!r}."
|
||||
)
|
||||
if ended_at is not None and _as_utc(ended_at) <= _as_utc(started_at):
|
||||
raise BindingValidationError("Binding ended_at must be strictly after started_at.")
|
||||
if _as_utc(started_at) < _as_utc(meter.started_at):
|
||||
raise BindingValidationError("Binding must not start before its meter epoch.")
|
||||
if meter.ended_at is None:
|
||||
if ended_at is not None:
|
||||
# A historical binding on an active epoch is valid, but it must be
|
||||
# wholly within that epoch (whose upper bound is open).
|
||||
pass
|
||||
else:
|
||||
meter_end = _as_utc(meter.ended_at)
|
||||
if ended_at is None or _as_utc(ended_at) > meter_end:
|
||||
raise BindingValidationError("Closed meter bindings must end within the meter epoch.")
|
||||
|
||||
excluded = excluding_ids or set()
|
||||
candidates = session.execute(
|
||||
select(MeterSourceBinding).where(
|
||||
or_(
|
||||
MeterSourceBinding.meter_id == meter_id,
|
||||
MeterSourceBinding.channel_id == channel_id,
|
||||
)
|
||||
)
|
||||
).scalars()
|
||||
for existing in candidates:
|
||||
if existing.id in excluded:
|
||||
continue
|
||||
if half_open_intervals_overlap(
|
||||
_as_utc(started_at),
|
||||
_as_utc(ended_at) if ended_at is not None else None,
|
||||
_as_utc(existing.started_at),
|
||||
_as_utc(existing.ended_at) if existing.ended_at is not None else None,
|
||||
):
|
||||
side = "meter" if existing.meter_id == meter_id else "channel"
|
||||
raise BindingOverlapError(f"Binding overlaps existing {side} binding {existing.id}.")
|
||||
|
||||
|
||||
def create_binding(
|
||||
session: Session,
|
||||
*,
|
||||
meter_id: int,
|
||||
channel_id: int,
|
||||
started_at: datetime,
|
||||
ended_at: datetime | None = None,
|
||||
) -> MeterSourceBinding:
|
||||
"""Create a compatible non-overlapping half-open source binding."""
|
||||
now = _utc_now()
|
||||
if _as_utc(started_at) > now or (ended_at is not None and _as_utc(ended_at) > now):
|
||||
raise BindingValidationError("Binding boundaries must not be in the future.")
|
||||
_validate_binding(
|
||||
session,
|
||||
meter_id=meter_id,
|
||||
channel_id=channel_id,
|
||||
started_at=started_at,
|
||||
ended_at=ended_at,
|
||||
)
|
||||
binding = MeterSourceBinding(
|
||||
meter_id=meter_id,
|
||||
channel_id=channel_id,
|
||||
started_at=started_at,
|
||||
ended_at=ended_at,
|
||||
created_at=now,
|
||||
updated_at=now,
|
||||
)
|
||||
session.add(binding)
|
||||
return binding
|
||||
|
||||
|
||||
def create_binding_for_meter_swap(
|
||||
session: Session,
|
||||
*,
|
||||
old_meter_id: int | None,
|
||||
new_meter_id: int,
|
||||
channel_id: int,
|
||||
started_at: datetime,
|
||||
) -> MeterSourceBinding:
|
||||
"""Create a binding during a physical meter swap, handing off one channel if safe.
|
||||
|
||||
A channel is transferable only when exactly one of its bindings covered the
|
||||
instant immediately before ``started_at`` and that binding belongs to the
|
||||
meter which this declaration just closed. All other occupied or ambiguous
|
||||
cases retain the normal fail-closed overlap behaviour.
|
||||
|
||||
This function deliberately does not commit. The caller must keep the meter
|
||||
declaration, binding handoff, and any billing recompute in one transaction.
|
||||
"""
|
||||
new_meter = _get_meter(session, new_meter_id)
|
||||
channel = get_channel(session, channel_id)
|
||||
expected_unit = COMMODITY_UNITS.get(new_meter.commodity)
|
||||
if expected_unit is None or channel.unit != expected_unit:
|
||||
raise BindingValidationError(
|
||||
f"Meter commodity {new_meter.commodity!r} requires unit {expected_unit!r}, "
|
||||
f"but channel has {channel.unit!r}."
|
||||
)
|
||||
|
||||
boundary = _as_utc(started_at)
|
||||
if _as_utc(new_meter.started_at) != boundary:
|
||||
raise BindingValidationError(
|
||||
"Meter-swap binding must start at the new meter's started_at boundary."
|
||||
)
|
||||
covering_bindings = [
|
||||
binding
|
||||
for binding in session.execute(
|
||||
select(MeterSourceBinding).where(MeterSourceBinding.channel_id == channel_id)
|
||||
).scalars()
|
||||
if _as_utc(binding.started_at) < boundary
|
||||
and (binding.ended_at is None or _as_utc(binding.ended_at) >= boundary)
|
||||
]
|
||||
|
||||
if not covering_bindings:
|
||||
return create_binding(
|
||||
session,
|
||||
meter_id=new_meter_id,
|
||||
channel_id=channel_id,
|
||||
started_at=started_at,
|
||||
)
|
||||
|
||||
if old_meter_id is None or len(covering_bindings) != 1:
|
||||
raise BindingOverlapError("Channel is occupied or has an ambiguous binding at meter swap.")
|
||||
|
||||
old_meter = _get_meter(session, old_meter_id)
|
||||
old_binding = covering_bindings[0]
|
||||
if (
|
||||
old_meter.commodity != new_meter.commodity
|
||||
or old_meter.ended_at is None
|
||||
or _as_utc(old_meter.ended_at) != boundary
|
||||
or old_binding.meter_id != old_meter.id
|
||||
):
|
||||
raise BindingOverlapError("Channel is occupied by a binding that cannot be handed off.")
|
||||
|
||||
update_binding(session, old_binding.id, ended_at=started_at)
|
||||
return create_binding(
|
||||
session,
|
||||
meter_id=new_meter_id,
|
||||
channel_id=channel_id,
|
||||
started_at=started_at,
|
||||
)
|
||||
|
||||
|
||||
def update_binding(
|
||||
session: Session,
|
||||
binding_id: int,
|
||||
*,
|
||||
meter_id: int | None = None,
|
||||
channel_id: int | None = None,
|
||||
started_at: datetime | None = None,
|
||||
ended_at: datetime | None | object = _UNSET,
|
||||
) -> MeterSourceBinding:
|
||||
"""Correct a binding while preserving half-open timeline constraints."""
|
||||
binding = session.get(MeterSourceBinding, binding_id)
|
||||
if binding is None:
|
||||
raise BindingNotFoundError(f"Meter source binding {binding_id} was not found.")
|
||||
new_meter_id = binding.meter_id if meter_id is None else meter_id
|
||||
new_channel_id = binding.channel_id if channel_id is None else channel_id
|
||||
new_started_at = binding.started_at if started_at is None else started_at
|
||||
new_ended_at = binding.ended_at if ended_at is _UNSET else ended_at
|
||||
now = _utc_now()
|
||||
if _as_utc(new_started_at) > now or (new_ended_at is not None and _as_utc(new_ended_at) > now):
|
||||
raise BindingValidationError("Binding boundaries must not be in the future.")
|
||||
_validate_binding(
|
||||
session,
|
||||
meter_id=new_meter_id,
|
||||
channel_id=new_channel_id,
|
||||
started_at=new_started_at,
|
||||
ended_at=new_ended_at,
|
||||
excluding_ids={binding.id},
|
||||
)
|
||||
binding.meter_id = new_meter_id
|
||||
binding.channel_id = new_channel_id
|
||||
binding.started_at = new_started_at
|
||||
binding.ended_at = new_ended_at
|
||||
binding.updated_at = _utc_now()
|
||||
return binding
|
||||
|
||||
|
||||
def close_binding(session: Session, binding_id: int, *, ended_at: datetime) -> MeterSourceBinding:
|
||||
"""Close an existing binding at its exclusive end boundary."""
|
||||
return update_binding(session, binding_id, ended_at=ended_at)
|
||||
|
||||
|
||||
def close_open_bindings_for_meter(session: Session, meter_id: int, *, ended_at: datetime) -> list[MeterSourceBinding]:
|
||||
"""Close every open binding on a meter at one shared boundary."""
|
||||
bindings = list(session.execute(
|
||||
select(MeterSourceBinding).where(
|
||||
MeterSourceBinding.meter_id == meter_id, MeterSourceBinding.ended_at.is_(None)
|
||||
)
|
||||
).scalars())
|
||||
for binding in bindings:
|
||||
update_binding(session, binding.id, ended_at=ended_at)
|
||||
return bindings
|
||||
|
||||
|
||||
def transfer_binding(
|
||||
session: Session, *, target_meter_id: int, from_binding_id: int, to_channel_id: int,
|
||||
effective_at: datetime,
|
||||
) -> tuple[MeterSourceBinding, MeterSourceBinding]:
|
||||
"""Atomically close a binding and open its replacement on the target meter."""
|
||||
source = session.get(MeterSourceBinding, from_binding_id)
|
||||
if source is None:
|
||||
raise BindingNotFoundError(f"Meter source binding {from_binding_id} was not found.")
|
||||
target = _get_meter(session, target_meter_id)
|
||||
old_meter = _get_meter(session, source.meter_id)
|
||||
effective_at = _as_utc(effective_at)
|
||||
now = _utc_now()
|
||||
if effective_at > now:
|
||||
raise BindingValidationError("Binding transfer effective_at must not be in the future.")
|
||||
if old_meter.commodity != target.commodity:
|
||||
raise BindingValidationError("Binding transfer meters must have the same commodity.")
|
||||
if source.ended_at is not None:
|
||||
raise BindingValidationError("Only an open binding can be transferred.")
|
||||
if old_meter.id == target.id:
|
||||
close_at = effective_at
|
||||
else:
|
||||
# Recovery is deliberately narrow: the source meter must be the one
|
||||
# and only most-recent closed predecessor in this commodity's timeline.
|
||||
# A manually closed meter may leave an intentional epoch gap before the
|
||||
# target is declared, so adjacency is not required.
|
||||
if old_meter.ended_at is None:
|
||||
raise BindingValidationError("Source binding must belong to a closed predecessor meter.")
|
||||
timeline = list(session.execute(
|
||||
select(Meter).where(Meter.commodity == target.commodity)
|
||||
).scalars())
|
||||
predecessors = [
|
||||
meter for meter in timeline
|
||||
if meter.id != target.id
|
||||
and meter.ended_at is not None
|
||||
and _as_utc(meter.ended_at) <= _as_utc(target.started_at)
|
||||
]
|
||||
if not predecessors:
|
||||
raise BindingValidationError("Source meter is not the unique immediately preceding meter.")
|
||||
latest_end = max(_as_utc(meter.ended_at) for meter in predecessors)
|
||||
latest = [meter for meter in predecessors if _as_utc(meter.ended_at) == latest_end]
|
||||
if len(latest) != 1 or latest[0].id != old_meter.id:
|
||||
raise BindingValidationError("Source meter is not the unique immediately preceding meter.")
|
||||
# Reject any overlapping epoch around either endpoint. A separate
|
||||
# meter inside the gap is already excluded by the predecessor check;
|
||||
# one extending into either endpoint is an ambiguous timeline too.
|
||||
for meter in timeline:
|
||||
if meter.id in {old_meter.id, target.id}:
|
||||
continue
|
||||
meter_end = _as_utc(meter.ended_at) if meter.ended_at is not None else None
|
||||
if (
|
||||
half_open_intervals_overlap(
|
||||
_as_utc(old_meter.started_at), _as_utc(old_meter.ended_at),
|
||||
_as_utc(meter.started_at), meter_end,
|
||||
)
|
||||
or half_open_intervals_overlap(
|
||||
_as_utc(target.started_at),
|
||||
_as_utc(target.ended_at) if target.ended_at is not None else None,
|
||||
_as_utc(meter.started_at), meter_end,
|
||||
)
|
||||
):
|
||||
raise BindingValidationError("Source meter has an ambiguous commodity timeline.")
|
||||
close_at = _as_utc(old_meter.ended_at)
|
||||
if effective_at < _as_utc(target.started_at):
|
||||
raise BindingValidationError("Transfer effective_at must be within the target meter epoch.")
|
||||
if effective_at < _as_utc(source.started_at):
|
||||
raise BindingValidationError("Transfer effective_at precedes the source binding.")
|
||||
# Validate the target before mutating the old row, then close/create in one session.
|
||||
_validate_binding(session, meter_id=target.id, channel_id=to_channel_id,
|
||||
started_at=effective_at, ended_at=None,
|
||||
excluding_ids={source.id})
|
||||
update_binding(session, source.id, ended_at=close_at)
|
||||
created = create_binding(session, meter_id=target.id, channel_id=to_channel_id,
|
||||
started_at=effective_at)
|
||||
return source, created
|
||||
+76
-2
@@ -53,6 +53,7 @@ from sqlalchemy import select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.models.energy import Meter
|
||||
from app.models.meter_source import MeterSourceBinding
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -104,6 +105,20 @@ class MeterIntervalError(MeterError):
|
||||
"""
|
||||
|
||||
|
||||
def close_meter(session: Session, meter: Meter, *, ended_at: datetime) -> Meter:
|
||||
"""Close an active meter at a valid, non-future exclusive boundary."""
|
||||
boundary = _as_utc(ended_at)
|
||||
if meter.ended_at is not None:
|
||||
raise MeterIntervalError("Only an active meter can be closed.")
|
||||
if boundary <= _as_utc(meter.started_at):
|
||||
raise MeterIntervalError("Meter ended_at must be strictly after started_at.")
|
||||
if boundary > datetime.now(UTC):
|
||||
raise MeterIntervalError("Meter ended_at must not be in the future.")
|
||||
_validate_bindings_fit_meter_end(session, meter, boundary)
|
||||
meter.ended_at = boundary
|
||||
return meter
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Internal query helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -121,6 +136,23 @@ def _active_meter(session: Session, commodity: str) -> Optional[Meter]:
|
||||
).scalar_one_or_none()
|
||||
|
||||
|
||||
def _validate_bindings_fit_meter_end(session: Session, meter: Meter, boundary: datetime) -> None:
|
||||
"""Reject an epoch close that would put any retained binding out of bounds.
|
||||
|
||||
Closed binding history is immutable here. Open bindings may subsequently
|
||||
be closed by the caller at the shared meter boundary, but only when that
|
||||
produces a non-empty interval.
|
||||
"""
|
||||
for binding in session.execute(
|
||||
select(MeterSourceBinding).where(MeterSourceBinding.meter_id == meter.id)
|
||||
).scalars():
|
||||
if binding.ended_at is None:
|
||||
if _as_utc(binding.started_at) >= boundary:
|
||||
raise MeterIntervalError("Open binding cannot be closed within the proposed meter epoch.")
|
||||
elif _as_utc(binding.ended_at) > boundary:
|
||||
raise MeterIntervalError("Closed binding extends beyond the proposed meter epoch.")
|
||||
|
||||
|
||||
def _meter_before(session: Session, meter: Meter) -> Optional[Meter]:
|
||||
"""Return the meter whose ``ended_at`` equals *meter*'s ``started_at``.
|
||||
|
||||
@@ -296,6 +328,8 @@ def declare_meter(
|
||||
If *started_at* is strictly earlier than the current active meter's
|
||||
``started_at`` (chronological backdate below the active epoch's start).
|
||||
"""
|
||||
if _as_utc(started_at) > datetime.now(UTC):
|
||||
raise MeterIntervalError("Meter started_at must not be in the future.")
|
||||
active = _active_meter(session, commodity)
|
||||
|
||||
if active is not None:
|
||||
@@ -308,6 +342,9 @@ def declare_meter(
|
||||
"Declare a started_at on or after the active meter's start to avoid "
|
||||
"a chronologically inconsistent epoch ordering."
|
||||
)
|
||||
# Validate before changing the epoch: retained closed binding history
|
||||
# must never be silently truncated by a later declaration.
|
||||
_validate_bindings_fit_meter_end(session, active, _as_utc(started_at))
|
||||
# Close the current active meter at the swap point (contiguous handoff).
|
||||
active.ended_at = started_at
|
||||
logger.info(
|
||||
@@ -400,6 +437,9 @@ def update_meter(
|
||||
invert).
|
||||
b. It must be **strictly before** this meter's ``ended_at`` (if set),
|
||||
so this meter's epoch remains non-empty.
|
||||
c. Every binding on this meter and its affected predecessor must remain
|
||||
wholly inside its proposed epoch. The service rejects the correction
|
||||
rather than rewriting binding history.
|
||||
|
||||
Note: triggering a billing recompute (``recompute_range``) after a
|
||||
retroactive ``started_at`` change is **out of scope** for this service
|
||||
@@ -429,6 +469,14 @@ def update_meter(
|
||||
If the new ``started_at`` would produce an invalid (empty or inverted)
|
||||
epoch for this meter or the immediately preceding one.
|
||||
"""
|
||||
# Validate the proposed epoch boundary before touching *any* mutable
|
||||
# field. PATCH accepts label/note together with started_at, so doing this
|
||||
# first keeps an invalid future timestamp from leaking a partial in-session
|
||||
# update before the API's rollback boundary is reached.
|
||||
proposed_started_at = _as_utc(started_at) if started_at is not None else None
|
||||
if proposed_started_at is not None and proposed_started_at > datetime.now(UTC):
|
||||
raise MeterIntervalError("Meter started_at must not be in the future.")
|
||||
|
||||
if label is not None:
|
||||
meter.label = label
|
||||
logger.info("Updated meter id=%d label=%r", meter.id, label)
|
||||
@@ -439,10 +487,11 @@ def update_meter(
|
||||
|
||||
if started_at is not None:
|
||||
old_started_at = meter.started_at
|
||||
assert proposed_started_at is not None
|
||||
|
||||
# --- Validate upper bound: new started_at must be < this meter's ended_at (if set).
|
||||
if meter.ended_at is not None:
|
||||
if _as_utc(started_at) >= _as_utc(meter.ended_at):
|
||||
if proposed_started_at >= _as_utc(meter.ended_at):
|
||||
raise MeterIntervalError(
|
||||
f"New started_at ({started_at.isoformat()}) must be strictly before "
|
||||
f"this meter's ended_at ({meter.ended_at.isoformat()}). "
|
||||
@@ -454,12 +503,37 @@ def update_meter(
|
||||
|
||||
# --- Validate lower bound: new started_at must be strictly after prev's started_at.
|
||||
if prev is not None:
|
||||
if _as_utc(started_at) <= _as_utc(prev.started_at):
|
||||
if proposed_started_at <= _as_utc(prev.started_at):
|
||||
raise MeterIntervalError(
|
||||
f"New started_at ({started_at.isoformat()}) must be strictly after "
|
||||
f"the previous meter's started_at ({prev.started_at.isoformat()}). "
|
||||
"Moving the boundary that far back would collapse the previous meter's epoch."
|
||||
)
|
||||
# A boundary correction changes both adjacent meter epochs. Fail closed
|
||||
# rather than silently rewriting binding history: every existing binding
|
||||
# must still fit in its proposed epoch before either Meter is mutated.
|
||||
affected_meters = [
|
||||
(meter, proposed_started_at, _as_utc(meter.ended_at) if meter.ended_at is not None else None)
|
||||
]
|
||||
if prev is not None:
|
||||
affected_meters.append((prev, _as_utc(prev.started_at), proposed_started_at))
|
||||
for affected_meter, proposed_start, proposed_end in affected_meters:
|
||||
bindings = session.execute(
|
||||
select(MeterSourceBinding).where(MeterSourceBinding.meter_id == affected_meter.id)
|
||||
).scalars()
|
||||
for binding in bindings:
|
||||
if _as_utc(binding.started_at) < proposed_start:
|
||||
raise MeterIntervalError(
|
||||
f"Binding {binding.id} starts before meter {affected_meter.id}'s epoch."
|
||||
)
|
||||
if proposed_end is not None and (
|
||||
binding.ended_at is None or _as_utc(binding.ended_at) > proposed_end
|
||||
):
|
||||
raise MeterIntervalError(
|
||||
f"Binding {binding.id} would fall outside meter {affected_meter.id}'s epoch."
|
||||
)
|
||||
|
||||
if prev is not None:
|
||||
# Maintain continuity: update the previous meter's ended_at to match the new start.
|
||||
prev.ended_at = started_at
|
||||
logger.info(
|
||||
|
||||
@@ -64,6 +64,7 @@ def poll_device(session: Session, device: ModbusDevice) -> ModbusReading | None:
|
||||
device.port,
|
||||
device.unit_id,
|
||||
[{"start": b.start, "count": b.count} for b in profile.blocks],
|
||||
function_code=profile.function_code,
|
||||
)
|
||||
payload = profiles.decode(profile, registers)
|
||||
|
||||
|
||||
@@ -30,6 +30,7 @@ Design decisions
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from threading import Lock, Thread
|
||||
from datetime import UTC, datetime
|
||||
|
||||
from sqlalchemy import select
|
||||
@@ -41,7 +42,10 @@ from app.models.energy import EnergyContract, TibberPrice
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _active_tibber_contract_exists(session: Session) -> bool:
|
||||
_background_refresh_lock = Lock()
|
||||
|
||||
|
||||
def active_tibber_contract_exists(session: Session) -> bool:
|
||||
"""Return True if there is an active contract with kind='tibber'."""
|
||||
row = session.execute(
|
||||
select(EnergyContract).where(
|
||||
@@ -84,14 +88,17 @@ def refresh_prices(session: Session, settings: object) -> int:
|
||||
logger.debug("refresh_prices: tibber_api_token is empty — no-op")
|
||||
return 0
|
||||
|
||||
if not _active_tibber_contract_exists(session):
|
||||
if not active_tibber_contract_exists(session):
|
||||
logger.debug("refresh_prices: no active tibber contract — no-op")
|
||||
return 0
|
||||
|
||||
home_id: str = getattr(settings, "tibber_home_id", "") or ""
|
||||
home_id_or_none: str | None = home_id.strip() or None
|
||||
|
||||
logger.info("refresh_prices: fetching Tibber price range (home_id=%r)", home_id_or_none)
|
||||
# Neither the API token nor the selected home identifier is safe to emit in
|
||||
# diagnostics. The fetch client receives them, but logs only describe the
|
||||
# operation itself.
|
||||
logger.info("refresh_prices: fetching Tibber price range")
|
||||
|
||||
# May raise TibberError or TibberAuthError — let them propagate.
|
||||
price_points = fetch_price_range(token, home_id_or_none)
|
||||
@@ -137,3 +144,59 @@ def refresh_prices(session: Session, settings: object) -> int:
|
||||
|
||||
logger.info("refresh_prices: upserted %d price points", upserted)
|
||||
return upserted
|
||||
|
||||
|
||||
def run_tibber_refresh_best_effort() -> bool:
|
||||
"""Run one refresh with an isolated session, skipping concurrent requests.
|
||||
|
||||
This is shared by the hourly scheduler and immediate post-commit triggers.
|
||||
It intentionally catches all failures: refresh is advisory and must never
|
||||
make app startup or a successfully committed configuration/contract update
|
||||
appear to have failed. The boolean reports whether this invocation owned
|
||||
the work; it is primarily useful for tests and diagnostics.
|
||||
"""
|
||||
if not _background_refresh_lock.acquire(blocking=False):
|
||||
logger.debug("Tibber refresh already running; skipping duplicate request")
|
||||
return False
|
||||
|
||||
session: Session | None = None
|
||||
try:
|
||||
# Local imports keep the pure refresh service free of app startup import
|
||||
# cycles, while every background invocation gets a fresh DB session.
|
||||
from app.config import get_settings
|
||||
from app.db import get_session_local
|
||||
from app.services.config_page import build_runtime_settings
|
||||
|
||||
session = get_session_local()()
|
||||
refresh_prices(session, build_runtime_settings(session, get_settings()))
|
||||
except Exception as exc:
|
||||
# Exception text can contain remote request details. Keep diagnostics
|
||||
# useful without allowing a token or home id to escape through logging.
|
||||
logger.warning("Tibber price refresh failed (%s)", type(exc).__name__)
|
||||
if session is not None:
|
||||
try:
|
||||
session.rollback()
|
||||
except Exception:
|
||||
logger.warning("Tibber price refresh rollback failed")
|
||||
finally:
|
||||
if session is not None:
|
||||
try:
|
||||
session.close()
|
||||
except Exception:
|
||||
logger.warning("Tibber price refresh session close failed")
|
||||
_background_refresh_lock.release()
|
||||
return True
|
||||
|
||||
|
||||
def trigger_tibber_refresh() -> None:
|
||||
"""Request a non-blocking, best-effort Tibber refresh after a DB commit."""
|
||||
try:
|
||||
Thread(
|
||||
target=run_tibber_refresh_best_effort,
|
||||
name="tibber-price-refresh",
|
||||
daemon=True,
|
||||
).start()
|
||||
except Exception as exc:
|
||||
# Starting the optional worker must not turn an already committed API
|
||||
# operation into a failure; avoid logging exception text for secrecy.
|
||||
logger.warning("Unable to start Tibber price refresh (%s)", type(exc).__name__)
|
||||
|
||||
@@ -19,8 +19,8 @@ Priority for resolving the local timezone
|
||||
-----------------------------------------
|
||||
1. ``TZ`` environment variable — ``ZoneInfo(os.environ["TZ"])``.
|
||||
Set ``TZ=Europe/Amsterdam`` in the deployment env for correct NL handling.
|
||||
2. System local timezone fallback: ``datetime.now().astimezone().tzinfo``.
|
||||
This matches the behaviour callers already relied on implicitly.
|
||||
2. The DST-aware ``Europe/Amsterdam`` business timezone. This keeps local-day
|
||||
calculations deterministic when a deployment does not set ``TZ``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -40,7 +40,7 @@ def local_tz() -> "tzinfo":
|
||||
Resolution order:
|
||||
1. ``TZ`` environment variable (``ZoneInfo(TZ)``). Set
|
||||
``TZ=Europe/Amsterdam`` in production for correct NL/DST handling.
|
||||
2. System local timezone via ``datetime.now().astimezone().tzinfo``.
|
||||
2. The DST-aware ``Europe/Amsterdam`` business timezone.
|
||||
|
||||
**Monkeypatch this function in tests** to get deterministic timezone
|
||||
behaviour regardless of CI host configuration::
|
||||
@@ -51,9 +51,7 @@ def local_tz() -> "tzinfo":
|
||||
tz_env = os.environ.get("TZ", "").strip()
|
||||
if tz_env:
|
||||
return ZoneInfo(tz_env)
|
||||
# System fallback — identical to the .astimezone() pattern already used
|
||||
# in homeassistant_inbound.py and poo.py.
|
||||
return datetime.now().astimezone().tzinfo # type: ignore[return-value]
|
||||
return ZoneInfo("Europe/Amsterdam")
|
||||
|
||||
|
||||
def to_local(dt: datetime) -> datetime:
|
||||
|
||||
@@ -0,0 +1,401 @@
|
||||
"""Privacy-preserving WarmteLink frame admission and minute sampling.
|
||||
|
||||
The serial worker added later owns I/O. This module deliberately only accepts
|
||||
already parsed :class:`P1Telegram` instances (or a parser callable at its
|
||||
small convenience entry point), so rejected telegram bytes never enter the
|
||||
database or an exception message.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Callable
|
||||
from dataclasses import dataclass
|
||||
from datetime import UTC, datetime, timedelta
|
||||
from decimal import Decimal
|
||||
import re
|
||||
from zoneinfo import ZoneInfo
|
||||
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.db import get_session_local
|
||||
from app.integrations.p1 import IntegrityStatus, P1Channel, P1Telegram, parse_telegram
|
||||
from app.models.meter_source import MeterSource, WarmteLinkReading
|
||||
from app.services.meter_sources import upsert_discovered_channel
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class _ChannelSample:
|
||||
key: str
|
||||
label: str
|
||||
value: Decimal
|
||||
unit: str
|
||||
device_type: str | None
|
||||
fingerprint: str | None
|
||||
identity: tuple[object, ...]
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class _FrameSnapshot:
|
||||
recorded_at: datetime
|
||||
fingerprint: str | None
|
||||
samples: tuple[_ChannelSample, ...]
|
||||
|
||||
|
||||
class WarmteLinkIngestor:
|
||||
"""Keep unverifiable candidates isolated by source for one worker lifetime."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
*,
|
||||
clock: Callable[[], datetime] | None = None,
|
||||
) -> None:
|
||||
self._clock = clock or (lambda: datetime.now(UTC))
|
||||
self._previous: dict[int, _FrameSnapshot] = {}
|
||||
|
||||
def ingest(
|
||||
self,
|
||||
session: Session,
|
||||
*,
|
||||
source_id: int,
|
||||
telegram: P1Telegram,
|
||||
received_at: datetime | None = None,
|
||||
) -> bool:
|
||||
"""Apply one parsed telegram in the caller's transaction.
|
||||
|
||||
Returns whether the frame was admitted. Callers that own a session
|
||||
must commit on success; :func:`handle_frame` is the failure-contained
|
||||
worker-facing entry point.
|
||||
"""
|
||||
source = session.get(MeterSource, source_id)
|
||||
if source is None:
|
||||
raise ValueError("WarmteLink source was not found")
|
||||
if source.kind != "warmtelink_serial":
|
||||
raise ValueError("Source is not a WarmteLink serial source")
|
||||
|
||||
received_at = _utc(self._clock()) if received_at is None else _utc(received_at)
|
||||
try:
|
||||
integrity = _integrity_status(telegram.integrity)
|
||||
except Exception:
|
||||
self._previous.pop(source_id, None)
|
||||
self._diagnose(source, "WarmteLink frame could not be normalized", now=received_at)
|
||||
return False
|
||||
|
||||
if integrity is IntegrityStatus.INVALID:
|
||||
self._previous.pop(source_id, None)
|
||||
self._diagnose(source, "WarmteLink frame checksum is invalid", now=received_at)
|
||||
return False
|
||||
|
||||
try:
|
||||
snapshot = _snapshot(telegram, received_at)
|
||||
fingerprints = _final_fingerprints(snapshot)
|
||||
except Exception:
|
||||
# A parser DTO remains an untrusted boundary. Do not let an
|
||||
# unnormalizable DTO bridge two otherwise matching candidates.
|
||||
self._previous.pop(source_id, None)
|
||||
self._diagnose(source, "WarmteLink frame could not be normalized", now=received_at)
|
||||
return False
|
||||
if fingerprints is None:
|
||||
self._previous.pop(source_id, None)
|
||||
self._diagnose(source, "WarmteLink frame fingerprint is invalid", now=received_at)
|
||||
return False
|
||||
if integrity is IntegrityStatus.UNVERIFIABLE:
|
||||
previous = self._previous.get(source_id)
|
||||
if previous is None:
|
||||
self._previous[source_id] = snapshot
|
||||
self._diagnose(
|
||||
source, "Awaiting a second matching unverifiable WarmteLink frame", now=received_at
|
||||
)
|
||||
return False
|
||||
reason = _continuity_problem(previous, snapshot)
|
||||
if reason is not None:
|
||||
self._previous[source_id] = snapshot
|
||||
self._diagnose(source, reason, now=received_at)
|
||||
return False
|
||||
try:
|
||||
self._admit(session, source, snapshot, integrity.value, fingerprints, received_at)
|
||||
except Exception:
|
||||
# ``ingest`` owns flushes and may be used directly by tests or
|
||||
# future callers. A failed write must never become a speculative
|
||||
# predecessor for this source.
|
||||
self._previous.pop(source_id, None)
|
||||
raise
|
||||
if integrity is IntegrityStatus.UNVERIFIABLE:
|
||||
# Advance the sliding predecessor only once every database write
|
||||
# for this frame has succeeded. ``handle_frame`` also clears it
|
||||
# if the caller's later commit fails.
|
||||
self._previous[source_id] = snapshot
|
||||
else:
|
||||
# A verified frame has no need for a speculative predecessor.
|
||||
self._previous.pop(source_id, None)
|
||||
return True
|
||||
|
||||
def handle_frame(
|
||||
self,
|
||||
source_id: int,
|
||||
frame: bytes,
|
||||
*,
|
||||
session_factory: Callable[[], Session] = get_session_local,
|
||||
parser: Callable[[bytes], P1Telegram] = parse_telegram,
|
||||
) -> bool:
|
||||
"""Parse and persist one frame, containing both parse and DB failures.
|
||||
|
||||
A failed write is rolled back before a fresh transaction records only
|
||||
a generic source error. Consequently no partial latest/history update
|
||||
survives and the following frame may recover normally.
|
||||
"""
|
||||
received_at = _utc(self._clock())
|
||||
try:
|
||||
telegram = parser(frame)
|
||||
except Exception:
|
||||
self._previous.pop(source_id, None)
|
||||
self._record_error(
|
||||
session_factory, source_id, "WarmteLink frame could not be parsed", now=received_at
|
||||
)
|
||||
return False
|
||||
try:
|
||||
with session_factory() as session:
|
||||
admitted = self.ingest(
|
||||
session, source_id=source_id, telegram=telegram, received_at=received_at
|
||||
)
|
||||
session.commit()
|
||||
return admitted
|
||||
except Exception:
|
||||
self._previous.pop(source_id, None)
|
||||
self._record_error(session_factory, source_id, "WarmteLink ingest failed", now=received_at)
|
||||
return False
|
||||
|
||||
def _admit(
|
||||
self,
|
||||
session: Session,
|
||||
source: MeterSource,
|
||||
snapshot: _FrameSnapshot,
|
||||
quality: str,
|
||||
fingerprints: tuple[str, ...],
|
||||
received_at: datetime,
|
||||
) -> None:
|
||||
for sample, fingerprint in zip(snapshot.samples, fingerprints, strict=True):
|
||||
channel = upsert_discovered_channel(
|
||||
session,
|
||||
source_id=source.id,
|
||||
channel_key=sample.key,
|
||||
label=sample.label,
|
||||
unit=sample.unit,
|
||||
suggested_commodity=_suggestion(sample.unit),
|
||||
device_type=sample.device_type,
|
||||
fingerprint=fingerprint,
|
||||
latest_value=sample.value,
|
||||
latest_at=snapshot.recorded_at,
|
||||
latest_quality=quality,
|
||||
)
|
||||
session.flush()
|
||||
bucket = snapshot.recorded_at.replace(second=0, microsecond=0)
|
||||
exists = session.scalar(
|
||||
select(WarmteLinkReading.id)
|
||||
.where(
|
||||
WarmteLinkReading.channel_id == channel.id,
|
||||
WarmteLinkReading.recorded_at >= bucket,
|
||||
WarmteLinkReading.recorded_at < bucket + timedelta(minutes=1),
|
||||
)
|
||||
.limit(1)
|
||||
)
|
||||
if exists is None:
|
||||
session.add(
|
||||
WarmteLinkReading(
|
||||
channel_id=channel.id,
|
||||
recorded_at=snapshot.recorded_at,
|
||||
received_at=received_at,
|
||||
value=sample.value,
|
||||
unit=sample.unit,
|
||||
quality=quality,
|
||||
equipment_fingerprint=fingerprint,
|
||||
)
|
||||
)
|
||||
source.status = "online"
|
||||
source.last_seen_at = received_at
|
||||
source.last_error = None
|
||||
source.updated_at = received_at
|
||||
|
||||
def _diagnose(self, source: MeterSource, reason: str, *, now: datetime | None = None) -> None:
|
||||
now = _utc(self._clock()) if now is None else now
|
||||
source.status = "error"
|
||||
source.last_error = reason
|
||||
source.updated_at = now
|
||||
|
||||
def _record_error(
|
||||
self,
|
||||
session_factory: Callable[[], Session],
|
||||
source_id: int,
|
||||
message: str,
|
||||
*,
|
||||
now: datetime,
|
||||
) -> None:
|
||||
try:
|
||||
with session_factory() as session:
|
||||
source = session.get(MeterSource, source_id)
|
||||
if source is not None:
|
||||
self._diagnose(source, message, now=now)
|
||||
session.commit()
|
||||
except Exception:
|
||||
# Error reporting itself must not kill another source worker.
|
||||
return
|
||||
|
||||
|
||||
def _snapshot(telegram: P1Telegram, received_at: datetime) -> _FrameSnapshot:
|
||||
recorded_at = _parse_timestamp(telegram.timestamp, received_at)
|
||||
samples = tuple(_sample(channel) for channel in telegram.channels)
|
||||
if not samples:
|
||||
raise ValueError("WarmteLink telegram contains no cumulative channels")
|
||||
return _FrameSnapshot(recorded_at, telegram.equipment_fingerprint, samples)
|
||||
|
||||
|
||||
def _sample(channel: P1Channel) -> _ChannelSample:
|
||||
profile = _canonical_channel_profile(channel.number)
|
||||
if channel.device_type != profile.device_type:
|
||||
raise ValueError("WarmteLink channel device type is not canonical")
|
||||
if len(channel.readings) != 1:
|
||||
raise ValueError("WarmteLink channel has no unambiguous cumulative reading")
|
||||
reading = channel.readings[0]
|
||||
if reading.code != profile.reading_code or reading.value is None or reading.unit != profile.raw_unit:
|
||||
raise ValueError("WarmteLink cumulative reading is incomplete")
|
||||
# Parser annotations are not a trust boundary: fake or future parser DTOs
|
||||
# must not put a float (including NaN) or a non-finite Decimal into the
|
||||
# per-source unverifiable candidate state. Do not coerce here: accepting
|
||||
# another numeric type would make the persistence and continuity paths
|
||||
# disagree about the cumulative-value contract.
|
||||
if not isinstance(reading.value, Decimal) or not reading.value.is_finite():
|
||||
raise ValueError("WarmteLink cumulative reading is not a finite Decimal")
|
||||
fingerprint = channel.equipment_fingerprint
|
||||
return _ChannelSample(
|
||||
key=profile.key,
|
||||
label=profile.label,
|
||||
value=reading.value,
|
||||
unit=profile.unit,
|
||||
device_type=profile.device_type,
|
||||
fingerprint=fingerprint,
|
||||
identity=(channel.number, profile.device_type, fingerprint, profile.reading_code, profile.unit),
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class _CanonicalChannelProfile:
|
||||
key: str
|
||||
label: str
|
||||
device_type: str
|
||||
reading_code: str
|
||||
raw_unit: str
|
||||
unit: str
|
||||
|
||||
|
||||
_CANONICAL_CHANNELS = {
|
||||
1: _CanonicalChannelProfile(
|
||||
key="channel-1",
|
||||
label="WarmteLink channel 1",
|
||||
device_type="006",
|
||||
reading_code="0-1:24.2.1",
|
||||
raw_unit="m3",
|
||||
unit="m³",
|
||||
),
|
||||
2: _CanonicalChannelProfile(
|
||||
key="channel-2",
|
||||
label="WarmteLink channel 2",
|
||||
device_type="012",
|
||||
reading_code="0-2:24.2.1",
|
||||
raw_unit="GJ",
|
||||
unit="GJ",
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
def _canonical_channel_profile(number: object) -> _CanonicalChannelProfile:
|
||||
# Do not format or coerce parser supplied channel numbers: that could turn
|
||||
# an arbitrary object into an identity key before it is rejected.
|
||||
if type(number) is not int:
|
||||
raise ValueError("WarmteLink channel number is not canonical")
|
||||
try:
|
||||
return _CANONICAL_CHANNELS[number]
|
||||
except KeyError as exc:
|
||||
raise ValueError("WarmteLink channel is not supported by the profile") from exc
|
||||
|
||||
|
||||
def _integrity_status(value: object) -> IntegrityStatus:
|
||||
"""Require a real parser integrity enum, never a look-alike value object."""
|
||||
if not isinstance(value, IntegrityStatus):
|
||||
raise ValueError("WarmteLink integrity status is not canonical")
|
||||
return value
|
||||
|
||||
|
||||
def _continuity_problem(previous: _FrameSnapshot, current: _FrameSnapshot) -> str | None:
|
||||
if current.recorded_at <= previous.recorded_at:
|
||||
return "Unverifiable WarmteLink timestamp is not strictly increasing"
|
||||
if current.recorded_at - previous.recorded_at != timedelta(seconds=10):
|
||||
return "Unverifiable WarmteLink frame cadence is not 10 seconds"
|
||||
if current.fingerprint != previous.fingerprint:
|
||||
return "WarmteLink equipment metadata changed"
|
||||
if tuple(sample.identity for sample in current.samples) != tuple(sample.identity for sample in previous.samples):
|
||||
return "WarmteLink channel metadata changed or channel set changed"
|
||||
old_values = {sample.key: sample.value for sample in previous.samples}
|
||||
if any(sample.value < old_values[sample.key] for sample in current.samples):
|
||||
return "WarmteLink cumulative value decreased"
|
||||
return None
|
||||
|
||||
|
||||
_FINGERPRINT_PATTERN = re.compile(r"[0-9a-f]{64}")
|
||||
|
||||
|
||||
def _final_fingerprints(snapshot: _FrameSnapshot) -> tuple[str, ...] | None:
|
||||
"""Return only canonical SHA-256 hexdigests safe to persist.
|
||||
|
||||
A parser DTO is an untrusted boundary: even a field named ``fingerprint``
|
||||
can contain a raw equipment identifier. Validate the complete DTO before
|
||||
choosing persisted values: the top-level value and every channel value
|
||||
must independently be canonical hashes. This deliberately does not use
|
||||
a top-level fallback for an absent or malformed channel fingerprint.
|
||||
"""
|
||||
values = (snapshot.fingerprint, *(sample.fingerprint for sample in snapshot.samples))
|
||||
if any(not _is_canonical_fingerprint(value) for value in values):
|
||||
return None
|
||||
return tuple(sample.fingerprint for sample in snapshot.samples if sample.fingerprint is not None)
|
||||
|
||||
|
||||
def _is_canonical_fingerprint(value: str | None) -> bool:
|
||||
return value is not None and _FINGERPRINT_PATTERN.fullmatch(value) is not None
|
||||
|
||||
|
||||
_AMSTERDAM = ZoneInfo("Europe/Amsterdam")
|
||||
_MAX_CLOCK_SKEW = timedelta(minutes=5)
|
||||
|
||||
|
||||
def _parse_timestamp(value: str | None, received_at: datetime) -> datetime:
|
||||
if value is None or len(value) != 13 or value[-1] not in {"S", "W"} or not value[:-1].isdigit():
|
||||
raise ValueError("WarmteLink timestamp is unavailable")
|
||||
naive = datetime.strptime(value[:-1], "%y%m%d%H%M%S")
|
||||
# The S/W marker is only advisory: deployed devices have emitted W while
|
||||
# on CEST. Validate both folds by a UTC round trip, which rejects spring
|
||||
# gaps and leaves one (ordinary) or two (fall-back) real instants.
|
||||
candidates: list[datetime] = []
|
||||
for fold in (0, 1):
|
||||
candidate = naive.replace(tzinfo=_AMSTERDAM, fold=fold).astimezone(UTC)
|
||||
local = candidate.astimezone(_AMSTERDAM)
|
||||
if local.replace(tzinfo=None) == naive and local.fold == fold and candidate not in candidates:
|
||||
candidates.append(candidate)
|
||||
if not candidates:
|
||||
raise ValueError("WarmteLink timestamp is unavailable")
|
||||
received_at = _utc(received_at)
|
||||
recorded_at = min(candidates, key=lambda candidate: abs(candidate - received_at))
|
||||
if abs(recorded_at - received_at) > _MAX_CLOCK_SKEW:
|
||||
raise ValueError("WarmteLink timestamp is unavailable")
|
||||
return recorded_at
|
||||
|
||||
|
||||
def _suggestion(unit: str) -> str | None:
|
||||
return {"GJ": "heating", "m³": "hot_water"}.get(unit)
|
||||
|
||||
|
||||
def _canonical_unit(unit: str) -> str:
|
||||
"""Map the P1 spelling of cubic metres to the source-profile unit."""
|
||||
return "m³" if unit == "m3" else unit
|
||||
|
||||
|
||||
def _utc(value: datetime) -> datetime:
|
||||
return value.replace(tzinfo=UTC) if value.tzinfo is None else value.astimezone(UTC)
|
||||
@@ -0,0 +1,436 @@
|
||||
"""Read-only WarmteLink serial workers and their lifecycle manager.
|
||||
|
||||
The worker deliberately owns no long-lived SQLAlchemy session and never
|
||||
retains telegram bytes after handing a complete frame to the ingestor.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Callable
|
||||
from contextlib import suppress
|
||||
from dataclasses import dataclass, field
|
||||
from datetime import UTC, datetime, timedelta
|
||||
import logging
|
||||
import threading
|
||||
from typing import Protocol
|
||||
|
||||
import serial
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.db import get_session_local
|
||||
from app.integrations.p1 import TelegramFramer
|
||||
from app.models.meter_source import MeterSource
|
||||
from app.services.warmtelink_ingest import WarmteLinkIngestor
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_BACKOFF_SECONDS = (1, 2, 4, 8, 16, 32, 60)
|
||||
_JOIN_TIMEOUT_SECONDS = 5
|
||||
_DISCOVERY_LOCK_TIMEOUT_SECONDS = 0.05
|
||||
_DISCOVERY_WAIT_SECONDS = 0.1
|
||||
_DISCOVERY_TIMEOUT_SECONDS = 5
|
||||
|
||||
|
||||
class ReadOnlySerial(Protocol):
|
||||
def read(self, size: int = 1) -> bytes: ...
|
||||
|
||||
def close(self) -> None: ...
|
||||
|
||||
|
||||
SerialFactory = Callable[[dict], ReadOnlySerial]
|
||||
SessionFactory = Callable[[], Session]
|
||||
|
||||
|
||||
def _default_session_factory() -> Session:
|
||||
"""Resolve the cached sessionmaker at call time, then open one session."""
|
||||
return get_session_local()()
|
||||
|
||||
|
||||
class WorkerClock(Protocol):
|
||||
"""Injectable interruptible clock, keeping retry tests deterministic."""
|
||||
|
||||
def wait(self, stop_event: threading.Event, seconds: float) -> bool: ...
|
||||
|
||||
|
||||
class _EventClock:
|
||||
def wait(self, stop_event: threading.Event, seconds: float) -> bool:
|
||||
return stop_event.wait(seconds)
|
||||
|
||||
|
||||
def open_warmtelink_serial(config: dict) -> ReadOnlySerial:
|
||||
"""Open the fixed WarmteLink P1 profile; no write-capable API is exposed."""
|
||||
return serial.Serial(
|
||||
port=config["path"], baudrate=115200, bytesize=serial.SEVENBITS,
|
||||
parity=serial.PARITY_NONE, stopbits=serial.STOPBITS_ONE, timeout=1,
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class _WorkerConfig:
|
||||
source_id: int
|
||||
config: dict
|
||||
|
||||
|
||||
@dataclass
|
||||
class DiscoveryRequest:
|
||||
"""One source-scoped request, completed only by its serial owner."""
|
||||
|
||||
request_id: int
|
||||
source_id: int
|
||||
deadline: datetime
|
||||
status: str = "pending"
|
||||
detail: str | None = None
|
||||
completed: threading.Event = field(default_factory=threading.Event)
|
||||
|
||||
|
||||
class WarmteLinkWorker:
|
||||
"""One interruptible, read-only serial loop for one meter source."""
|
||||
|
||||
def __init__(
|
||||
self, source_id: int, config: dict, *, session_factory: SessionFactory = _default_session_factory,
|
||||
serial_factory: SerialFactory = open_warmtelink_serial,
|
||||
stop_event: threading.Event | None = None,
|
||||
ingestor: WarmteLinkIngestor | None = None,
|
||||
clock: WorkerClock | None = None,
|
||||
) -> None:
|
||||
self.source_id = source_id
|
||||
self.config = dict(config)
|
||||
self._session_factory = session_factory
|
||||
self._serial_factory = serial_factory
|
||||
self._stop_event = stop_event or threading.Event()
|
||||
self._ingestor = ingestor or WarmteLinkIngestor()
|
||||
self._clock = clock or _EventClock()
|
||||
self._serial: ReadOnlySerial | None = None
|
||||
self._serial_lock = threading.Lock()
|
||||
self._discovery_lock = threading.Lock()
|
||||
self._discoveries: list[DiscoveryRequest] = []
|
||||
# Never inherit a daemon flag from a caller's background thread: a serial
|
||||
# descriptor and its orderly shutdown must remain visible to the process.
|
||||
self._thread = threading.Thread(
|
||||
target=self._run, name=f"warmtelink-{source_id}", daemon=False
|
||||
)
|
||||
|
||||
@property
|
||||
def thread(self) -> threading.Thread:
|
||||
return self._thread
|
||||
|
||||
def start(self) -> None:
|
||||
self._thread.start()
|
||||
|
||||
def stop(self) -> None:
|
||||
self._stop_event.set()
|
||||
self._close_serial()
|
||||
|
||||
def request_discovery(self, request: DiscoveryRequest) -> None:
|
||||
"""Queue a read request; this worker remains the sole serial owner."""
|
||||
with self._discovery_lock:
|
||||
self._discoveries.append(request)
|
||||
|
||||
def _finish_discoveries(self, status: str, detail: str | None = None) -> None:
|
||||
now = datetime.now(UTC)
|
||||
with self._discovery_lock:
|
||||
pending, self._discoveries = self._discoveries, []
|
||||
for request in pending:
|
||||
if request.completed.is_set():
|
||||
continue
|
||||
if request.deadline <= now and status == "completed":
|
||||
request.status, request.detail = "error", "Discovery timed out."
|
||||
else:
|
||||
request.status, request.detail = status, detail
|
||||
request.completed.set()
|
||||
|
||||
def _expire_discoveries(self) -> None:
|
||||
now = datetime.now(UTC)
|
||||
with self._discovery_lock:
|
||||
expired = [request for request in self._discoveries if request.deadline <= now]
|
||||
self._discoveries = [request for request in self._discoveries if request.deadline > now]
|
||||
for request in expired:
|
||||
if request.completed.is_set():
|
||||
continue
|
||||
request.status, request.detail = "error", "Discovery timed out."
|
||||
request.completed.set()
|
||||
|
||||
def join(self, timeout: float = _JOIN_TIMEOUT_SECONDS) -> bool:
|
||||
self._thread.join(timeout)
|
||||
return not self._thread.is_alive()
|
||||
|
||||
def _close_serial(self) -> None:
|
||||
with self._serial_lock:
|
||||
device, self._serial = self._serial, None
|
||||
if device is not None:
|
||||
with suppress(Exception):
|
||||
device.close()
|
||||
|
||||
def _record_error(self, message: str) -> None:
|
||||
try:
|
||||
with self._session_factory() as session:
|
||||
source = session.get(MeterSource, self.source_id)
|
||||
if source is not None:
|
||||
source.status = "error"
|
||||
source.last_error = message
|
||||
source.updated_at = datetime.now(UTC)
|
||||
session.commit()
|
||||
except Exception:
|
||||
# A source-status failure must not end another source's worker.
|
||||
return
|
||||
|
||||
def _run(self) -> None:
|
||||
backoff_index = 0
|
||||
while not self._stop_event.is_set():
|
||||
try:
|
||||
device = self._serial_factory(self.config)
|
||||
with self._serial_lock:
|
||||
if self._stop_event.is_set():
|
||||
with suppress(Exception):
|
||||
device.close()
|
||||
return
|
||||
self._serial = device
|
||||
# A disconnect makes any bytes buffered from the previous
|
||||
# descriptor untrustworthy. In particular, never let a
|
||||
# trailing partial telegram be completed by a newly opened
|
||||
# device.
|
||||
framer = TelegramFramer()
|
||||
while not self._stop_event.is_set():
|
||||
self._expire_discoveries()
|
||||
chunk = device.read(1024)
|
||||
if not chunk:
|
||||
# ``timeout`` reads are normal, but still yield so a bad
|
||||
# fake/device cannot turn an empty read into a busy spin.
|
||||
self._clock.wait(self._stop_event, 0.05)
|
||||
continue
|
||||
frames = framer.feed(chunk)
|
||||
for frame in frames:
|
||||
if self._stop_event.is_set():
|
||||
break
|
||||
admitted = self._ingestor.handle_frame(
|
||||
self.source_id, frame, session_factory=self._session_factory
|
||||
)
|
||||
if admitted:
|
||||
self._finish_discoveries("completed")
|
||||
# A complete frame proves transport recovery even if its
|
||||
# contents are rejected by the privacy/admission layer.
|
||||
backoff_index = 0
|
||||
except Exception:
|
||||
self._finish_discoveries("error", "WarmteLink discovery failed.")
|
||||
self._record_error("WarmteLink serial connection failed")
|
||||
delay = _BACKOFF_SECONDS[min(backoff_index, len(_BACKOFF_SECONDS) - 1)]
|
||||
backoff_index += 1
|
||||
self._clock.wait(self._stop_event, delay)
|
||||
finally:
|
||||
self._close_serial()
|
||||
|
||||
|
||||
class WarmteLinkWorkerManager:
|
||||
"""Reconcile enabled serial sources into exactly one worker each."""
|
||||
|
||||
def __init__(
|
||||
self, *, session_factory: SessionFactory = _default_session_factory,
|
||||
serial_factory: SerialFactory = open_warmtelink_serial,
|
||||
worker_factory: Callable[..., WarmteLinkWorker] = WarmteLinkWorker,
|
||||
) -> None:
|
||||
self._session_factory = session_factory
|
||||
self._serial_factory = serial_factory
|
||||
self._worker_factory = worker_factory
|
||||
self._workers: dict[int, tuple[_WorkerConfig, WarmteLinkWorker]] = {}
|
||||
self._lock = threading.Lock()
|
||||
self._reapers: set[int] = set()
|
||||
self._next_discovery_id = 0
|
||||
self._shutting_down = False
|
||||
|
||||
@property
|
||||
def worker_count(self) -> int:
|
||||
with self._lock:
|
||||
return len(self._workers)
|
||||
|
||||
def reconcile(self) -> None:
|
||||
# Reading desired state under the same lock which applies it prevents a
|
||||
# delayed pre-commit snapshot from rolling a newer commit backwards.
|
||||
with self._lock:
|
||||
if self._shutting_down:
|
||||
return
|
||||
desired = self._read_desired()
|
||||
self._reconcile_locked(desired)
|
||||
|
||||
def start(self) -> None:
|
||||
"""Enable reconciliation for a newly entered application lifespan."""
|
||||
with self._lock:
|
||||
self._shutting_down = False
|
||||
self.reconcile()
|
||||
|
||||
def request_discovery(self, source_id: int) -> DiscoveryRequest:
|
||||
"""Ask the current source worker for one bounded read/discovery attempt.
|
||||
|
||||
This intentionally does not reconcile or open a descriptor. Lifecycle
|
||||
convergence remains separate; a request can neither replace nor stop a
|
||||
worker when an HTTP client times out or disconnects.
|
||||
"""
|
||||
now = datetime.now(UTC)
|
||||
request = DiscoveryRequest(0, source_id, now)
|
||||
if not self._lock.acquire(timeout=_DISCOVERY_LOCK_TIMEOUT_SECONDS):
|
||||
request.status, request.detail = "error", "Discovery queue is busy."
|
||||
request.completed.set()
|
||||
return request
|
||||
try:
|
||||
self._next_discovery_id += 1
|
||||
request.request_id = self._next_discovery_id
|
||||
request.deadline = now + timedelta(seconds=_DISCOVERY_TIMEOUT_SECONDS)
|
||||
if self._shutting_down:
|
||||
request.status, request.detail = "error", "WarmteLink manager is stopped."
|
||||
request.completed.set()
|
||||
elif (entry := self._workers.get(source_id)) is None:
|
||||
request.status, request.detail = "error", "WarmteLink worker is not running."
|
||||
request.completed.set()
|
||||
else:
|
||||
entry[1].request_discovery(request)
|
||||
timer = threading.Timer(_DISCOVERY_TIMEOUT_SECONDS, self._timeout_discovery, args=(request,))
|
||||
timer.daemon = True
|
||||
timer.start()
|
||||
finally:
|
||||
self._lock.release()
|
||||
# A tiny bounded wait makes an immediately available frame observable,
|
||||
# without turning an HTTP call into serial I/O or an unbounded wait.
|
||||
request.completed.wait(_DISCOVERY_WAIT_SECONDS)
|
||||
return request
|
||||
|
||||
@staticmethod
|
||||
def _timeout_discovery(request: DiscoveryRequest) -> None:
|
||||
"""Resolve a stale HTTP request without touching its healthy worker."""
|
||||
if not request.completed.is_set():
|
||||
request.status, request.detail = "error", "Discovery timed out."
|
||||
request.completed.set()
|
||||
|
||||
def _read_desired(self) -> dict[int, _WorkerConfig]:
|
||||
with self._session_factory() as session:
|
||||
return {
|
||||
source.id: _WorkerConfig(source.id, dict(source.config))
|
||||
for source in session.execute(
|
||||
select(MeterSource).where(
|
||||
MeterSource.kind == "warmtelink_serial", MeterSource.enabled.is_(True)
|
||||
)
|
||||
).scalars()
|
||||
}
|
||||
|
||||
def _record_manager_error(self, source_id: int) -> None:
|
||||
"""Best-effort, deliberately non-sensitive lifecycle failure status."""
|
||||
try:
|
||||
with self._session_factory() as session:
|
||||
source = session.get(MeterSource, source_id)
|
||||
if source is not None:
|
||||
source.status = "error"
|
||||
source.last_error = "WarmteLink worker failed"
|
||||
source.updated_at = datetime.now(UTC)
|
||||
session.commit()
|
||||
except Exception:
|
||||
return
|
||||
|
||||
def _reconcile_locked(self, desired: dict[int, _WorkerConfig]) -> None:
|
||||
stale = [
|
||||
source_id for source_id, (config, _) in self._workers.items()
|
||||
if source_id not in desired or desired[source_id] != config
|
||||
]
|
||||
blocked: set[int] = set()
|
||||
for source_id in stale:
|
||||
_, worker = self._workers[source_id]
|
||||
try:
|
||||
worker.stop()
|
||||
stopped = worker.join()
|
||||
except Exception:
|
||||
self._record_manager_error(source_id)
|
||||
blocked.add(source_id)
|
||||
continue
|
||||
if stopped:
|
||||
self._workers.pop(source_id, None)
|
||||
else:
|
||||
logger.error("WarmteLink worker did not stop for source %s", source_id)
|
||||
blocked.add(source_id)
|
||||
self._schedule_reaper_locked(source_id, worker)
|
||||
for source_id, config in desired.items():
|
||||
if source_id in self._workers or source_id in blocked:
|
||||
continue
|
||||
worker: WarmteLinkWorker | None = None
|
||||
try:
|
||||
worker = self._worker_factory(
|
||||
source_id, config.config, session_factory=self._session_factory,
|
||||
serial_factory=self._serial_factory,
|
||||
)
|
||||
self._workers[source_id] = (config, worker)
|
||||
worker.start()
|
||||
except Exception:
|
||||
self._record_manager_error(source_id)
|
||||
# A failed start normally has no thread. If an unusual worker
|
||||
# did start before raising, keep it tracked until it is reaped.
|
||||
if not self._worker_is_alive(worker):
|
||||
self._workers.pop(source_id, None)
|
||||
else:
|
||||
self._schedule_reaper_locked(source_id, worker)
|
||||
|
||||
@staticmethod
|
||||
def _worker_is_alive(worker: object | None) -> bool:
|
||||
thread = getattr(worker, "thread", None)
|
||||
return bool(thread is not None and thread.is_alive())
|
||||
|
||||
def _schedule_reaper_locked(self, source_id: int, worker: WarmteLinkWorker) -> None:
|
||||
if source_id in self._reapers:
|
||||
return
|
||||
self._reapers.add(source_id)
|
||||
threading.Thread(
|
||||
target=self._reap_worker, args=(source_id, worker),
|
||||
# This bookkeeping watcher must not turn a deliberately bounded
|
||||
# application shutdown into an unbounded process wait. The actual
|
||||
# serial worker itself is explicitly non-daemon.
|
||||
name=f"warmtelink-reaper-{source_id}", daemon=True,
|
||||
).start()
|
||||
|
||||
def _reap_worker(self, source_id: int, worker: WarmteLinkWorker) -> None:
|
||||
"""Wait for one timed-out worker, then converge without another API call."""
|
||||
try:
|
||||
while True:
|
||||
try:
|
||||
if worker.join():
|
||||
break
|
||||
except Exception:
|
||||
self._record_manager_error(source_id)
|
||||
return
|
||||
# A custom worker can report a bounded join timeout immediately;
|
||||
# yield before asking again so its reaper cannot busy-spin.
|
||||
threading.Event().wait(0.05)
|
||||
with self._lock:
|
||||
current = self._workers.get(source_id)
|
||||
if current is not None and current[1] is worker:
|
||||
self._workers.pop(source_id)
|
||||
self._reapers.discard(source_id)
|
||||
should_reconcile = not self._shutting_down
|
||||
if should_reconcile:
|
||||
self.reconcile()
|
||||
finally:
|
||||
with self._lock:
|
||||
self._reapers.discard(source_id)
|
||||
|
||||
def shutdown(self) -> None:
|
||||
with self._lock:
|
||||
self._shutting_down = True
|
||||
workers = list(self._workers.items())
|
||||
for source_id, (_, worker) in workers:
|
||||
try:
|
||||
worker.stop()
|
||||
except Exception:
|
||||
self._record_manager_error(source_id)
|
||||
for source_id, (_, worker) in workers:
|
||||
try:
|
||||
stopped = worker.join()
|
||||
except Exception:
|
||||
self._record_manager_error(source_id)
|
||||
continue
|
||||
if not stopped:
|
||||
logger.error("WarmteLink worker did not stop during shutdown for source %s", source_id)
|
||||
with self._lock:
|
||||
self._schedule_reaper_locked(source_id, worker)
|
||||
else:
|
||||
with self._lock:
|
||||
current = self._workers.get(source_id)
|
||||
if current is not None and current[1] is worker:
|
||||
self._workers.pop(source_id)
|
||||
|
||||
|
||||
warmtelink_worker_manager = WarmteLinkWorkerManager()
|
||||
@@ -88,6 +88,8 @@ pyproject-hooks==1.2.0
|
||||
# via
|
||||
# build
|
||||
# pip-tools
|
||||
pyserial==3.5
|
||||
# via -r requirements.in
|
||||
pytest==8.4.2
|
||||
# via -r dev-requirements.in
|
||||
python-dotenv==1.2.2
|
||||
|
||||
@@ -26,3 +26,7 @@ services:
|
||||
- "127.0.0.1:8002:8000"
|
||||
environment:
|
||||
APP_DATABASE_URL: "sqlite:////app/data/app.db"
|
||||
devices: !override
|
||||
- "${WARMTELINK_DEVICE_PATH:?Set a stable /dev/serial/by-id path}:/dev/warmtelink:rw"
|
||||
group_add: !override
|
||||
- "${WARMTELINK_SERIAL_GID:?Set the host serial device GID}"
|
||||
|
||||
+8
-1
@@ -6,6 +6,8 @@ services:
|
||||
restart: "no"
|
||||
init: true
|
||||
command: ["python", "-m", "scripts.run_migrations"]
|
||||
environment:
|
||||
TZ: "${TZ:-Europe/Amsterdam}"
|
||||
volumes:
|
||||
- ./data:/app/data
|
||||
- ./.env:/app/.env:ro
|
||||
@@ -17,13 +19,18 @@ services:
|
||||
user: "1000:1000"
|
||||
restart: unless-stopped
|
||||
init: true
|
||||
environment:
|
||||
TZ: "${TZ:-Europe/Amsterdam}"
|
||||
depends_on:
|
||||
migration:
|
||||
condition: service_completed_successfully
|
||||
ports:
|
||||
- "127.0.0.1:8881:8000"
|
||||
devices:
|
||||
- "${WARMTELINK_DEVICE_PATH:?Set a stable /dev/serial/by-id path}:/dev/warmtelink:rw"
|
||||
group_add:
|
||||
- "${WARMTELINK_SERIAL_GID:?Set the host serial device GID}"
|
||||
volumes:
|
||||
- ./data:/app/data
|
||||
- ./.env:/app/.env:ro
|
||||
- /etc/localtime:/etc/localtime:ro
|
||||
|
||||
|
||||
@@ -19,7 +19,7 @@
|
||||
|
||||
- `main.py`
|
||||
- FastAPI app factory
|
||||
- lifespan(APScheduler 启停、MQTT 客户端起停、连接后触发 HA Discovery 发布;M6 新增 `tibber-refresh` 抓价 job + `energy-cost` 1 分钟计费 tick job;M6 启用时注册 DSMR MQTT 订阅)
|
||||
- lifespan(APScheduler 启停、MQTT 客户端起停、连接后触发 HA Discovery 发布;注册 DSMR MQTT source,并启动/关闭每个 enabled WarmteLink 业务只读 serial worker;`tibber-refresh`、electricity 与 thermal cost tick 均由 scheduler 驱动)
|
||||
- 基础路由注册
|
||||
- `config.py`
|
||||
- 环境变量驱动的 settings(含 M5 新增的 MQTT/HA Discovery/Modbus 配置项;M6 新增 `dsmr_ingest_enabled`、`dsmr_mqtt_topic`、`dsmr_sample_interval_s`、`tibber_api_token`(secret)、`tibber_home_id`)
|
||||
@@ -29,22 +29,22 @@
|
||||
- 通用依赖注入
|
||||
- `api/`
|
||||
- HTTP routes
|
||||
- `api/routes/api/`:JSON API(`/api/*` 前缀),供 React SPA 调用:会话/鉴权、配置读写、数据查询、记录 CRUD、Modbus 设备 CRUD + readings + metrics + test(`/api/modbus/*`)、Expose 勾选 + 重发 discovery(`/api/expose`)、MQTT 测试连接(`/api/config/mqtt/test`)、M6 新增合同 CRUD + 版本(`/api/energy/contracts*`)、pricing profile 列表(`/api/energy/profiles`)、价格/费用/汇总/DSMR 最新/重算/Tibber 测试(`/api/energy/prices`、`/api/energy/costs`、`/api/energy/costs/summary`、`/api/energy/dsmr/latest`、`/api/energy/costs/recompute`、`/api/energy/tibber/test`)
|
||||
- `api/routes/api/`:JSON API(`/api/*` 前缀),供 React SPA 调用:会话/鉴权、配置读写、记录 CRUD、Modbus、Expose 与 MQTT 测试;Energy 包含 source profile/source/channel/history/discover/binding/Meter API、scope-aware contracts/prices/costs,以及兼容的 DSMR latest API
|
||||
- 裸 ingestion 端点:`GET /public-ip/check`、`POST /homeassistant/publish`、`POST /poo/record`、`GET /poo/latest`、TickTick OAuth 等
|
||||
- `models/`
|
||||
- SQLAlchemy models
|
||||
- 所有模型(auth / config / public_ip / location / poo / modbus / expose / energy)共用同一个 `Base`,均落在单一 `app.db` 中
|
||||
- 所有模型(auth / config / public_ip / location / poo / modbus / expose / energy / meter_source)共用同一个 `Base`,均落在单一 `app.db` 中
|
||||
- M5 新增:`ModbusDevice`(设备部署层)、`ModbusReading`(通用遥测,JSON payload)、`ExposedEntityToggle`(HA 实体暴露开关)
|
||||
- M6 新增:`DsmrReading`(整帧 DSMR telegram,10s 降采样)、`EnergyContract`(合同头,含 active 标记)、`EnergyContractVersion`(版本/时段,values JSON,只增不改)、`TibberPrice`(15 分钟价缓存,不可变)、`EnergyCostPeriod`(每 15 分钟计量电费,快照价,不可变)
|
||||
- Energy:`MeterSource` / `MeterSourceChannel` / `MeterSourceBinding` 将协议连接、稳定测量 channel 与 Meter epoch 分离;`DsmrReading`、`WarmteLinkReading` 分别保存 JSON 与 Decimal scalar 历史;electricity `EnergyCostPeriod` 绑定 source binding,thermal `MeterCostPeriod` 保存审计账本;合同按 electricity / thermal scope 共存
|
||||
- `schemas/`
|
||||
- Pydantic schemas(M5 新增 `modbus.py`、`expose.py`;M6 新增 `energy_contract.py`、`energy.py`)
|
||||
- Pydantic schemas(包括 `modbus.py`、`expose.py`、`energy_contract.py`、`energy.py`、`meter_source.py`)
|
||||
- `services/`
|
||||
- 业务服务层
|
||||
- 当前已迁入 config page 的 DB 持久化逻辑
|
||||
- 当前已迁入 public IPv4 检查、状态持久化与变化通知逻辑
|
||||
- 当前已迁入 SMTP 发信与测试发信逻辑
|
||||
- M5 新增:`modbus_poll.py`(采集 service,逐设备 poll + 落库 + 推 MQTT state)、`ha_discovery.py`(构建 HA Discovery payload、发布 retained config、发布 state)
|
||||
- M6 新增:`tibber_prices.py`(httpx GraphQL 抓 15 分钟价,upsert `tibber_price`,幂等;仅 active=tibber 且 token 存在时运行)、`dsmr_ingest.py`(MQTT handler,整帧 JSON blob + 10s 降采样落库,`source_id` 幂等)、`energy_cost.py`(计费引擎:每 15 分钟寄存器差 × strategy 出价 → `energy_cost_period` 不可变快照;汇总 Σnet + 固定费 − heffingskorting;重算显式 opt-in)
|
||||
- Energy:`dsmr_ingest.py` 按 source 入库;`warmtelink_ingest.py` 接纳连续确认的业务只读 P1 scalar,`warmtelink_worker.py` 管理 interruptible serial reconnect(pyserial 的 POSIX `O_RDWR` 打开由非 root、非 privileged、无 `m` 的 Docker `rw` device rule 支持;worker 只 read/close,绝不 write);`energy_cost.py` 与 `meter_cost.py` 分别计算 electricity/thermal 账本,均拒绝跨 Meter/binding 相减
|
||||
- `integrations/`
|
||||
- 外部系统适配层
|
||||
- Home Assistant outbound adapter(REST 通道,原有)
|
||||
|
||||
@@ -9,6 +9,8 @@
|
||||
- [`m5-iot-energy.md`](./m5-iot-energy.md) — IoT 集成与能耗采集(Modbus/Energy + MQTT/HA Discovery + 前端侧边栏)
|
||||
- [`m6-tibber-dynamic-energy.md`](./m6-tibber-dynamic-energy.md) — 通用电价层 + DSMR 实时电表接入 + 实时买卖电费计算 + HA Energy 反哺
|
||||
- [`m7-meter-epochs-archival.md`](./m7-meter-epochs-archival.md) — 电表生命周期 / 换表归档(Meter epochs)
|
||||
- [`pre-m8-warmtelink-p1-poc.md`](./pre-m8-warmtelink-p1-poc.md) — WarmteLink P1 真机概念验证(已完成;正式 CLI 长测与供暖变化均经物理表复核)
|
||||
- [`m8-warmtelink-energy.md`](./m8-warmtelink-energy.md) — WarmteLink P1、多数据源 Meter 与热力计费(M8-T01~T20 自动化技术验收已完成;交付后用户人工 walkthrough 待验收)
|
||||
|
||||
本文件定义**所有任务共用的格式与协作规则**,各个里程碑文档不再重复这些约定。
|
||||
|
||||
|
||||
@@ -118,8 +118,9 @@ credits:
|
||||
kind: tibber
|
||||
label: Tibber 动态电价(15 分钟)
|
||||
energy:
|
||||
source: tibber_api # buy = total; sell = total − energy_tax − sell_adjust
|
||||
source: tibber_api # buy = total; sell = total − energy_tax − sell_fee − sell_adjust
|
||||
energy_tax: { unit: EUR/kWh }
|
||||
sell_fee: { unit: EUR/kWh, default: 0.0248 } # verkoopvergoeding, always subtracted
|
||||
sell_adjust: { unit: EUR/kWh, default: 0 }
|
||||
standing:
|
||||
management_fee: { unit: EUR/month, default: 5.99 }
|
||||
@@ -164,7 +165,7 @@ credits:
|
||||
1. 取各寄存器在 `t0`/`t1` 的值(`recorded_at ≤ 边界` 的最后一行,Decimal),算 **per-register 差**:`Δd1,Δd2,Δr1,Δr2`。
|
||||
2. 取 active 合同**在 t0 生效的版本** + 其 strategy:
|
||||
- `manual`:`import_cost = Δd1×(buy_dal) + Δd2×(buy_normal)`(`buy_x = energy_buy_x + energy_tax + ode`);`export_revenue = Δr1×sell_dal + Δr2×sell_normal`。
|
||||
- `tibber`:取覆盖 t0 的 `tibber_price`(`starts_at ≤ t0` 最近一条);`buy = total`、`sell = total − energy_tax − sell_adjust`;`import_cost = (Δd1+Δd2)×buy`、`export_revenue = (Δr1+Δr2)×sell`。
|
||||
- `tibber`:取覆盖 t0 的 `tibber_price`(`starts_at ≤ t0` 最近一条);`buy = total`、`sell = total − energy_tax − sell_fee − sell_adjust`(`sell_fee`=verkoopvergoeding,默认 0.0248,见下修正说明);`import_cost = (Δd1+Δd2)×buy`、`export_revenue = (Δr1+Δr2)×sell`。
|
||||
3. `net_cost = import_cost − export_revenue`;**upsert** `energy_cost_period`,**快照**当时用的价 + `contract_version_id`。
|
||||
- 缺价/缺数据:跳过或标 `degraded`,留待重算。**不做净计量**(进出口分开累加)。
|
||||
|
||||
@@ -240,7 +241,7 @@ credits:
|
||||
|
||||
1. **两层电价模型**:profile YAML 定结构(仓库、固定、UI 不可编辑)+ `EnergyContract`(+版本) 存数值(UI 填、版本化)+ strategy 按 kind 出价。仿 M5。
|
||||
2. **kind 不叫 "fixed"**:`manual`(人工填、可双费率、可带时段)/ `tibber`(API 动态);合同 `name` UI 自由填。
|
||||
3. **买价**:tibber = API `total`(全包,已证 total=energy+tax);manual = `energy_buy_档 + energy_tax`。**卖价**:tibber = `total − energy_tax − sell_adjust`;manual = `sell_档`(回送价,无能源税)。均含 VAT。
|
||||
3. **买价**:tibber = API `total`(全包,已证 total=energy+tax;含 inkoopvergoeding);manual = `energy_buy_档 + energy_tax`。**卖价**:tibber = `total − energy_tax − sell_fee − sell_adjust`(`sell_fee`=verkoopvergoeding 默认 0.0248);manual = `sell_档`(回送价,无能源税)。均含 VAT。
|
||||
4. **双费率**:manual 用 `delivered_1/2`、`returned_1/2` 分 dal/normal 计价(`_1`=dal/低、`_2`=normal/高);tibber 求和、15min 价不分档。
|
||||
5. **两层费用**:每 15min `energy_cost_period` 只算计量电费(不可变、快照价);日/月/年汇总再加固定费(按月→天)− heffingskorting(按年→天)。
|
||||
6. **回送阶梯罚金(terugleverkosten)不做**:按自然年累计、用户住不到年底算不准——不算、不记、不加功能(留痕见 §10)。
|
||||
@@ -475,7 +476,7 @@ Phase D(API + 前端)
|
||||
## 13. 待确认 / TODO(拿到真实 token + 账单后钉死,均已落成配置/默认值,不阻塞实现)
|
||||
|
||||
1. **买价**:✅ tibber = API `total`(demo 已证 total=energy+tax);manual = energy_buy_档 + energy_tax。无待办。
|
||||
2. **卖价残差(tibber)**:`sell = total − energy_tax − sell_adjust`,`sell_adjust` 默认 0(买卖费相等抵消)。真实账单确认后若有残差再调。
|
||||
2. **卖价残差(tibber)**:~~`sell = total − energy_tax − sell_adjust`,`sell_adjust` 默认 0(买卖费相等抵消)~~ → **已修正(2026-07,见 references §3.1)**:`total` 含 inkoopvergoeding,净计量回送 = `total − verkoopvergoeding`,两费**不抵消**。公式改为 `sell = total − energy_tax − sell_fee − sell_adjust`,新增 `sell_fee`(默认 0.0248,始终扣除);`sell_adjust` 净计量期设 `−energy_tax`。真实账单确认后若有残差再调 `sell_fee`。
|
||||
3. **双费率寄存器映射**:`_1`=dal/低、`_2`=normal/高(NL 惯例)——接价前用真实数据确认别接反(差价小但要对)。
|
||||
4. **能源税年值**:manual/tibber 的 `energy_tax` 默认 ~0.1108(2026 第一档含 VAT),按当年实际值核。
|
||||
5. **Tibber 15min + 币种**:✅ 查询/分辨率已 demo 证实;仍需合同生效后用**真实 token** 确认 NL 返回真 15 分钟价 + 币种 EUR。
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,333 @@
|
||||
# Pre-M8 — WarmteLink P1 真机概念验证
|
||||
|
||||
> **状态:已完成。** 2026-08-22 已完成临时脚本真机 bring-up;随后仓库内正式 probe 以
|
||||
> `115200 7N1` 连续运行 10 分钟,形成脱敏的可重复验收证据;最终人工走查开启供暖后又观察到
|
||||
> 区域供暖累计量从 `0.017 GJ` 增至 `0.018 GJ`,并与物理表一致。Pre-M8 现已解除
|
||||
> **M8 Planning** 的入口限制;M8 的架构和实现范围仍未锁定。
|
||||
|
||||
## 1. 目的
|
||||
|
||||
在进入 M8 正式设计和实现前,用新家的 Vattenfall WarmteLink 建立一条只读 P1 验证链,回答:
|
||||
|
||||
1. USB→P1 线、主机串口权限和 WarmteLink P1 端口能否稳定输出可解析数据。
|
||||
2. 真机使用什么串口参数,telegram 的 framing、CRC、时间戳和 OBIS/M-Bus channel 有何特征。
|
||||
3. 实际能够读取哪些累计量,以及单位、精度、更新频率和累计语义。
|
||||
4. P1 值是否与热量表/生活热水表面板一致。
|
||||
|
||||
Pre-M8 是 M8 的证据门:它只交付真机事实和可重复 probe,不决定数据库结构、Meter 数据源
|
||||
绑定、后台采集服务或前端布局。
|
||||
|
||||
## 2. 执行边界
|
||||
|
||||
本阶段只建立下面这条最短链路:
|
||||
|
||||
```text
|
||||
WarmteLink P1 → USB serial → 原始 telegram → 完整性状态 → 字段解析 → 终端输出
|
||||
```
|
||||
|
||||
明确不做:
|
||||
|
||||
- 不写入 `app.db`,不新增 Alembic migration。
|
||||
- 不新增 FastAPI API、后台常驻 worker、配置页面或 Energy 前端。
|
||||
- 不发布 MQTT / Home Assistant Discovery。
|
||||
- 不修改现有 DSMR Reader MQTT、电费计算或 Meter 逻辑。
|
||||
- 不在本阶段决定 WarmteLink 应落在哪个正式 Device/Source 模型中。
|
||||
- 不写串口、不修改 FTDI EEPROM;probe 必须严格只读。
|
||||
|
||||
## 3. 2026-08-22 真机事实
|
||||
|
||||
### 3.1 USB、权限与线材
|
||||
|
||||
- 新 USB→P1 线是 FTDI FT232R,稳定路径形态为
|
||||
`/dev/serial/by-id/usb-FTDI_FT232R_USB_UART_<redacted>-if00-port0`;正式配置不得依赖
|
||||
`/dev/ttyUSB1`。
|
||||
- 运行用户加入 `dialout` 后可以直接读取 tty,无需 root,也不应把容器作为硬件 bring-up
|
||||
的中间层。
|
||||
- 已用另一根已知正常的 DSMR 线在电表上读到连续、CRC 正确的 DSMR telegram,证明宿主机
|
||||
串口读取方法本身可用。
|
||||
- 曾临时清除新 FTDI 线 EEPROM 的 `INVERT_RXD` 做对照;输出发生变化但仍不可解析,随后已将
|
||||
EEPROM 逐字节恢复为原厂镜像并回读校验。正式方案不得依赖 EEPROM 修改。
|
||||
|
||||
### 3.2 串口参数矩阵
|
||||
|
||||
真机不是按最初假设的 `115200 8N1` 得到可读正文;当前线材与 WarmteLink 的实测最佳组合为:
|
||||
|
||||
```text
|
||||
115200 baud, 7 data bits, no parity, 1 stop bit(7N1)
|
||||
```
|
||||
|
||||
| 参数 | 实测结果 |
|
||||
| --- | --- |
|
||||
| `115200 7N1` | 正文稳定可读,可枚举 9 个 OBIS 字段 |
|
||||
| `115200 7N2` | 同样可读;没有理由增加停止位,正式默认仍用 `7N1` |
|
||||
| `115200 8N1/8E1/8O1` | 乱码,无有效 OBIS/CRC |
|
||||
| `115200 7E1/7O1` | 乱码,无有效 OBIS/CRC |
|
||||
| `120000–3000000`,分别用 `8N1`、`7N1` | 无完整报文;`120000` 仅残留少量可辨识文本 |
|
||||
| XON/XOFF 开/关 | 不改变 `7N1` 的字段解析结果 |
|
||||
|
||||
[DSMR 5.0.2 P1 Companion Standard](https://www.netbeheernederland.nl/sites/default/files/2024-02/dsmr_5.0.2_p1_companion_standard.pdf)
|
||||
规定 `115200 8N1`。因此 `7N1` 是当前设备/线材组合的实测事实,不应被文档或代码包装成
|
||||
标准 DSMR framing;CLI 必须允许显式覆盖数据位、校验位和停止位。
|
||||
|
||||
### 3.3 Framing 与 CRC 异常
|
||||
|
||||
- 临时 bring-up 的一次 8 帧盘点中每帧均为 256 字节;正式 10 分钟采样显示长度并不固定,
|
||||
详见 §3.6。
|
||||
- 正文结构稳定,版本、时间戳、两个 M-Bus channel、单位和值均可重复解析。
|
||||
- 实测头部为 `)TU)2NWA-MYRSKY`,偶见 `)TU{2NWA-MYRSKY`;没有标准要求的 `/` 起始符。
|
||||
- 帧尾存在 `!`,但其后的字符并非每帧都稳定为四位十六进制;即使恰好是四位,也无法从
|
||||
缺失的 `/` 起点完成标准 DSMR CRC16 验证。
|
||||
|
||||
因此正式 probe 必须把完整性明确表示为 `valid`、`invalid` 或 `unverifiable`,保留原始字节并
|
||||
输出原因。它可以在 `unverifiable` 状态下枚举字段用于 PoC,但绝不能把该帧报告为 CRC 已通过。
|
||||
M8 若要持久化这些读数,必须在 Planning 中单独锁定异常帧的接纳、重复确认和告警策略。
|
||||
|
||||
### 3.4 实际字段清单
|
||||
|
||||
连续 8 帧只出现以下 9 个字段;除 capture timestamp 外,字段集合和值均稳定:
|
||||
|
||||
| OBIS | 实测结构/值 | 结论 |
|
||||
| --- | --- | --- |
|
||||
| `1-3:0.2.8` | `(50)` | DSMR P1 输出版本 5.0 |
|
||||
| `0-0:1.0.0` | `(YYMMDDhhmmssX)` | telegram/capture timestamp,每 10 秒变化 |
|
||||
| `0-0:96.1.1` | `<redacted>` | WarmteLink/gateway equipment identifier |
|
||||
| `0-1:24.1.0` | `(006)` | channel 1,M-Bus device type `0x06`,生活热水 |
|
||||
| `0-1:96.1.0` | `<redacted>` | channel 1 equipment identifier;样本制造商可解码为 `KAM` |
|
||||
| `0-1:24.2.1` | `(<timestamp>)(5.900*m3)` | 生活热水累计体积,输出到 `0.001 m³` 小数位 |
|
||||
| `0-2:24.1.0` | `(012)` | channel 2,M-Bus device type `0x0C`,热量表 |
|
||||
| `0-2:96.1.0` | `<redacted>` | channel 2 equipment identifier;样本制造商可解码为 `KAM` |
|
||||
| `0-2:24.2.1` | `(<timestamp>)(0.017*GJ)` | 区域供暖累计热量,输出到 `0.001 GJ` 小数位 |
|
||||
|
||||
人工面板在同一时间显示 `5.900 m³` 和 `0.017 GJ`,与 P1 值完全一致;用户已确认这两个累计量
|
||||
足以作为后续 Home Assistant Energy 展示的数据基础。
|
||||
|
||||
没有发现瞬时流量、当前热功率、供水温度或回水温度字段。完整 OMS/M-Bus 模型允许这些可选
|
||||
量,但 WarmteLink 当前 P1 telegram 没有导出它们;Pre-M8 和 M8 不得假设它们可用。参考
|
||||
[OMS Specification Vol. 2 Annex A](https://oms-group.org/wp-content/uploads/2024/05/OMS-Spec_Vol2_AnnexA_F121.pdf)。
|
||||
|
||||
### 3.5 对 M8 已经成立的事实
|
||||
|
||||
- 一个 WarmteLink serial source 同时暴露两个独立累计 measurement channel。
|
||||
- channel 1 是生活热水 `m³`;channel 2 是区域供暖 `GJ`,不能按论坛样例固定 channel。
|
||||
- 两个累计量都与物理表一致,可进入 M8 的 Meter/source 映射讨论。
|
||||
- 当前没有瞬时流量、温度或热功率;M8 只承诺累计量采集与展示。
|
||||
- 串口 framing 和 CRC 异常尚未消失,必须作为正式 ingestion 的显式质量状态处理。
|
||||
|
||||
### 3.6 正式 CLI 10 分钟复验(脱敏)
|
||||
|
||||
2026-08-22,仓库内 `python -m scripts.p1_probe` 在真实
|
||||
`/dev/serial/by-id/<redacted>` 上以 `115200 7N1` 运行 `--duration 600 --show-changes`,原始
|
||||
bytes 仅写入 `/tmp`。CLI 正常以 exit code 0 结束;期间没有 I/O error、未处理异常或断连。
|
||||
|
||||
- 读取到 60 个完整 frame、15,362 raw bytes;最后 2 bytes 是下一帧的不完整残片,60 个完整帧
|
||||
合计 15,360 bytes。
|
||||
- 设备 timestamp 从 `18:16:10` 至 `18:26:00`,严格每 10 秒递进。CLI 处理的 59 个相邻间隔中,
|
||||
53 个为 10.0 s、1 个为 9.0 s、3 个为 0.0 s、2 个为 20.0 s,均值 9.81 s;0/20 s 配对来自
|
||||
serial chunk 的批量交付,不代表设备 cadence 改变。
|
||||
- 完整 frame 长度分布为 237 bytes × 2、239 × 1、254 × 12、256 × 31、258 × 11、275 × 3,
|
||||
平均 256 bytes、范围 237–275。不能再把临时样本的「固定 256 bytes」视为帧格式契约;变长与
|
||||
非标准 footer/分块边界一致。
|
||||
- 60/60 的完整性均为 `unverifiable`,因为 60/60 缺少标准 `/` 起始符。footer 长度为 4 × 41、
|
||||
6 × 16、23 × 3;仅 16/60 的 footer 恰为四位十六进制,但仍不能在缺失 `/` 时完成标准 CRC16
|
||||
验证。**没有任何 frame 被报告为 CRC valid。**
|
||||
- 60/60 帧均解析到同一组 9 个 OBIS code:`1-3:0.2.8`、`0-0:1.0.0`、`0-0:96.1.1`、
|
||||
`0-1:24.1.0`、`0-1:96.1.0`、`0-1:24.2.1`、`0-2:24.1.0`、`0-2:96.1.0`、`0-2:24.2.1`。
|
||||
equipment identifier 已脱敏,不进入 Git。
|
||||
- channel 1 的 device type 是 `006`,累计值稳定为 `5.900 m³`;channel 2 的 device type 是
|
||||
`012`,累计值稳定为 `0.017 GJ`。同日人工表盘复核也显示 `5.900 m³` / `0.017 GJ`,正式 CLI
|
||||
因而复现了该基线。
|
||||
- 60 帧中未见瞬时流量、热功率、供水温度或回水温度字段。10 分钟内累计值未变化只能证明这段
|
||||
时间的累计值稳定;它不能证明发生消费时的更新频率,也不能证明 reset 或 wrap 行为。
|
||||
|
||||
### 3.7 最终人工走查:供暖累计量变化
|
||||
|
||||
正式长测交付后,用户又在终端直接运行只读 probe 十几分钟并开启供暖。channel 2 的区域供暖
|
||||
累计量在本次运行中从 `0.017 GJ` 增至 `0.018 GJ`,同一时刻物理热量表也显示 `0.018 GJ`;
|
||||
因此可以确认当前 P1 输出会在实际供暖消费下更新累计量,且 `0.001 GJ` 的变化与物理表一致。
|
||||
|
||||
这次人工走查没有改变完整性结论:telegram 仍缺少标准 `/`,数值与物理表一致不能替代 CRC
|
||||
验证。走查也没有覆盖 reset、wrap 或精确更新延迟;这些仍须由 M8 的接纳、重复确认和告警策略
|
||||
处理,而不能从一次累计量递增外推。
|
||||
|
||||
## 4. 正式 probe 的预期操作方式
|
||||
|
||||
实现后,在 workspace virtual environment 中运行只读 probe:
|
||||
|
||||
```bash
|
||||
source .venv/bin/activate
|
||||
python -m scripts.p1_probe \
|
||||
--device /dev/serial/by-id/<usb-p1-device> \
|
||||
--baudrate 115200 \
|
||||
--bytesize 7 \
|
||||
--parity N \
|
||||
--stopbits 1 \
|
||||
--duration 600 \
|
||||
--show-changes \
|
||||
--raw-output /tmp/warmtelink-p1-telegram.bin
|
||||
```
|
||||
|
||||
参数名可在实现时小幅调整,但必须保留这些能力:
|
||||
|
||||
- 设备路径由用户显式传入,文档推荐 `/dev/serial/by-id/...`。
|
||||
- 串口默认采用本机实测 `115200 7N1`,同时允许显式覆盖 framing。
|
||||
- 连续读取多帧,输出 telegram cadence、帧长度和读取/重连错误。
|
||||
- 同时显示完整性/CRC 状态、原始 OBIS 字段和解析后的 channel、设备类型、值与单位。
|
||||
- parser 不依赖字段固定顺序,也不把 GJ/m³ 固定到 channel 1 或 2。
|
||||
- `--show-changes` 可只显示发生变化的字段,原始捕获可写入用户指定的 `/tmp` 路径。
|
||||
- 捕获文件默认按原始 bytes 保存;未经脱敏不得提交到 Git。
|
||||
- permission denied 时给出 `dialout` 指引;不得建议以 root 常驻运行。
|
||||
|
||||
Home Assistant Community 的
|
||||
[最小 WarmteLink Bash 读取方法](https://community.home-assistant.io/t/solved-dsmr-add-warmtelink-as-data-source/485255/2)
|
||||
仍可作为快速可见性检查,但论坛样例的 header、channel 和 device type 与本机均不同,不能作为
|
||||
parser 契约。是否引入 [`dsmr_parser`](https://github.com/ndokter/dsmr_parser) 也由 T01 的实测
|
||||
兼容性决定;若它要求标准 `/...!CRC` framing,则应保留小型专用 parser,而不是绕过其校验。
|
||||
|
||||
## 5. 实现任务
|
||||
|
||||
### PRE-M8-T01 — 纯函数 telegram framing、CRC 与 OBIS parser
|
||||
|
||||
- **Status**: `done`
|
||||
- **Depends**: `none`
|
||||
- **Context**: 先把串口 I/O 与解析分开,用脱敏 fixture 固定标准 DSMR 帧和本机异常帧行为。
|
||||
|
||||
**Files**
|
||||
|
||||
- `create scripts/p1_probe.py`
|
||||
- `create tests/fixtures/dsmr_p1_valid.txt`
|
||||
- `create tests/fixtures/warmtelink_p1_7n1.txt`
|
||||
- `create tests/test_p1_probe.py`
|
||||
|
||||
**Steps**
|
||||
|
||||
1. 实现不依赖串口的增量 framing、DSMR CRC16 和通用 OBIS 行解析函数。
|
||||
2. 用数据结构表达原始 header/footer、完整性状态、timestamp、channel、device type、equipment id、
|
||||
value 和 unit;数值使用 `Decimal`,不使用二进制浮点保存累计量。
|
||||
3. 加入一份完全脱敏的本机结构 fixture,并另造一份 CRC 正确的标准 DSMR fixture。
|
||||
4. 对字段重排、分块输入、缺失 `/`、非十六进制 footer、CRC mismatch、未知字段和两个 channel
|
||||
写单元测试。
|
||||
|
||||
**Out of scope / 不要碰**
|
||||
|
||||
- 不打开真实 serial device,不增加依赖,不写数据库/API/MQTT。
|
||||
- 不因本机正文可读而伪造 `/` header 或把 CRC 状态升级为 valid。
|
||||
|
||||
**Acceptance criteria**
|
||||
|
||||
- [x] 标准 fixture 的 frame boundary 与 CRC 可验证为 `valid`。
|
||||
- [x] 脱敏本机 fixture 被标为 `unverifiable`,但能按字段而非位置解析 `m³` 和 `GJ` channel。
|
||||
- [x] 任意 chunk boundary 和字段顺序不影响结果,未知字段原样保留。
|
||||
- [x] 累计量以 `Decimal` + 原单位返回。
|
||||
- [x] `pytest tests/test_p1_probe.py`、`pytest`、`ruff check .` 全绿。
|
||||
|
||||
**Reviewer checklist**
|
||||
|
||||
- CRC 覆盖范围必须严格从 `/` 到 `!`(包含二者),不得对缺失字节做猜测性修补。
|
||||
- fixture 必须脱敏且保留足以复现 framing 异常的字节结构。
|
||||
- parser 不得硬编码 channel 1=GJ 或 channel 2=m³。
|
||||
|
||||
### PRE-M8-T02 — 只读 serial probe CLI
|
||||
|
||||
- **Status**: `done`
|
||||
- **Depends**: `PRE-M8-T01`
|
||||
- **Context**: 在纯 parser 通过后增加最小 serial I/O,使真机验证可以从仓库稳定复现。
|
||||
|
||||
**Files**
|
||||
|
||||
- `modify requirements.in`
|
||||
- `modify requirements.txt`
|
||||
- `modify dev-requirements.txt`
|
||||
- `modify scripts/p1_probe.py`
|
||||
- `modify tests/test_p1_probe.py`
|
||||
|
||||
**Steps**
|
||||
|
||||
1. 增加受约束的 `pyserial` runtime 依赖并用仓库既有 pip-compile 流程同步生成 requirements。
|
||||
2. 实现 `--device`、framing 参数、`--duration`、`--show-changes` 和 `--raw-output`。
|
||||
3. 默认使用 `115200 7N1`;串口只读,禁止 write、EEPROM 或自动修改设备配置。
|
||||
4. 输出每帧完整性状态、全部字段、值变化、cadence 和错误;SIGINT/超时后关闭串口并正常退出。
|
||||
5. 用 fake serial/chunk stream 测试 CLI,不要求 CI 存在 `/dev/ttyUSB*`。
|
||||
|
||||
**Out of scope / 不要碰**
|
||||
|
||||
- 不做 daemon、自动重连 worker、Docker device mapping、数据库、API、MQTT 或 HA Discovery。
|
||||
- 不内置本机 FTDI 序列号,不自动扫描或改写任意 USB 设备。
|
||||
|
||||
**Acceptance criteria**
|
||||
|
||||
- [x] CLI 可用 `/dev/serial/by-id/...` 读取,且所有 framing 参数都可显式覆盖。
|
||||
- [x] 默认参数准确反映本机 `115200 7N1`,帮助文本说明它是实测值而非 DSMR 标准默认。
|
||||
- [x] raw output 保留原始 bytes;终端清楚区分 `valid`、`invalid`、`unverifiable`。
|
||||
- [x] permission/busy/disconnect 错误非零退出并给出可执行诊断,绝不建议常驻 root。
|
||||
- [x] 依赖输入与两个生成 requirements 文件同步。
|
||||
- [x] `pytest tests/test_p1_probe.py`、`pytest`、`ruff check .` 全绿。
|
||||
|
||||
**Reviewer checklist**
|
||||
|
||||
- 确认所有 serial write path 均不存在。
|
||||
- 确认测试完全 mock 硬件、没有 CI timing flake,退出路径总会关闭文件描述符。
|
||||
- 确认 requirements 是生成结果而非仅手改 lock file。
|
||||
|
||||
### PRE-M8-T03 — 正式真机验收与 M8 交接
|
||||
|
||||
- **Status**: `done`
|
||||
- **Depends**: `PRE-M8-T02`
|
||||
- **Context**: 用仓库内 probe 替代本轮临时脚本,形成可重复、脱敏且能支撑 M8 Planning 的证据。
|
||||
|
||||
**Files**
|
||||
|
||||
- `modify docs/design/pre-m8-warmtelink-p1-poc.md`
|
||||
- `modify docs/design/m8-warmtelink-energy.md`
|
||||
- `modify docs/design/README.md`
|
||||
- `modify docs/roadmap.md`
|
||||
|
||||
**Steps**
|
||||
|
||||
1. 在真实 `/dev/serial/by-id/...` 上运行 probe 至少 10 分钟,并保留原始捕获在 `/tmp`。
|
||||
2. 汇总帧数、cadence、长度、完整性状态、全部字段和读数变化;不得提交原始设备标识。
|
||||
3. 再次与物理表对照 GJ 和 m³,并记录累计/重置行为中本次能证实和不能证实的部分。
|
||||
4. 更新本节事实、通过条件和 M8 入口;只有证据齐全后才把 Pre-M8 标记为完成。
|
||||
|
||||
**Out of scope / 不要碰**
|
||||
|
||||
- 不为完成 checklist 而修补原始字节或放宽 CRC 结果。
|
||||
- 不进入 M8 schema/API/worker/frontend 实现。
|
||||
|
||||
**Acceptance criteria**
|
||||
|
||||
- [x] 仓库内 probe 在真机连续运行至少 10 分钟,无未处理异常退出。
|
||||
- [x] 脱敏摘要列出两个累计 channel、单位、精度、cadence 和完整性异常。
|
||||
- [x] `0.017 GJ`、`5.900 m³` 的基线或运行时新值与物理表再次对照。
|
||||
- [x] 明确没有从当前 P1 输出读取到瞬时流量、功率或温度。
|
||||
- [x] Pre-M8 状态与 roadmap/M8 入口同步;代码闸门保持全绿。
|
||||
|
||||
**Reviewer checklist**
|
||||
|
||||
- 证据必须来自正式 CLI,不得只复述本轮临时脚本结果。
|
||||
- 任何 equipment id、FTDI serial 和未脱敏 raw capture 都不得进入 Git。
|
||||
- CRC/framing 风险必须原样交给 M8,不能用“数值看起来正确”替代完整性判断。
|
||||
|
||||
## 6. 当前通过条件
|
||||
|
||||
- [x] USB、tty 权限和稳定 `/dev/serial/by-id/...` 路径已验证。
|
||||
- [x] 串口参数矩阵已完成,实测正文可读参数为 `115200 7N1`。
|
||||
- [x] 已枚举全部实际 channel/OBIS 字段,并确认没有轮换出现的额外测量量。
|
||||
- [x] GJ 和生活热水 m³ 均与物理表面板完全一致。
|
||||
- [x] 已记录精度、约 10 秒 telegram cadence 和当前字段集合。
|
||||
- [x] framing/CRC 失败已保留为显式异常,没有误报为校验通过。
|
||||
- [x] 仓库内 parser、fixtures、probe CLI 和自动化测试完成。
|
||||
- [x] 正式 probe 完成至少 10 分钟真机复验并产出脱敏摘要。
|
||||
- [x] 最终人工走查在供暖开启后观察到 `0.017 → 0.018 GJ`,并再次与物理表核对一致。
|
||||
|
||||
Pre-M8 已完成并向 M8 解锁 Planning;它只交付下列真机事实,不锁定正式架构或实现任务。
|
||||
|
||||
## 7. 向 M8 的交付物
|
||||
|
||||
Pre-M8 完成后只向 M8 交付事实,不交付正式架构:
|
||||
|
||||
- 脱敏 telegram 结构、fixture 和全部字段清单。
|
||||
- GJ 与 m³ 的实际 channel、device type、单位、精度和更新时间。
|
||||
- 串口参数、稳定设备路径形态与 `dialout` 权限要求。
|
||||
- parser 适配结论和 `unverifiable` framing/CRC 异常样本。
|
||||
- “一个 serial source 包含两个独立累计计量 channel”的实测结论。
|
||||
- 当前 P1 不提供瞬时流量、热功率或温度的明确边界。
|
||||
@@ -49,3 +49,14 @@
|
||||
- 更复杂的 backoff 策略
|
||||
|
||||
这一轮重点是先把 app -> Home Assistant 的出站契约和可复用结构迁进来。
|
||||
|
||||
## Energy、Source 与热力实体
|
||||
|
||||
Expose 框架还可以把已勾选的 Energy 实体通过 MQTT Home Assistant Discovery 发布;开关位于应用 Config 页的 HA Expose 面板,默认均为关闭。M8 增加了 source online、按 Meter UUID 锚定的累计量/today,以及 heating、hot-water-heating、water、water-tax、fixed、all-in total/today 等 thermal 实体。
|
||||
|
||||
- source 和 Meter identity 不依赖可变 label;换表会产生新 Meter UUID identity。
|
||||
- thermal 组合成本 identity 由当前 heating/hot_water Meter UUID 的有序组合锚定,任一换表都会产生新 identity,避免不同累计域拼接。
|
||||
- availability、unit、device/state class 与 today reset 由 provider 声明;operator 应在 HA 中核对,而不应假设同名实体可跨换表连续。
|
||||
- 关闭 toggle 后 retained discovery 会被清理;关闭暴露不删除 source、Meter、合同、读数或成本历史。
|
||||
|
||||
WarmteLink 的 P1 质量会原样保留为 `unverifiable`(若适用),不因发布到 HA 而提升为 `valid`。不要将 raw telegram、equipment id、串口路径、合同金额或 API secret 作为 HA entity/state/attribute 发布。
|
||||
|
||||
+54
-1
@@ -104,9 +104,62 @@ MQTT 上报的累计成本实体(`import_cost_total` / `export_revenue_total`
|
||||
- 若是全新空库,初始表不创建(无历史数据)。
|
||||
- 回填幂等:重复跑迁移不会创建多条初始表;回填后对账(非降级周期 `meter_id IS NULL` 数必须为 0)。
|
||||
|
||||
## M8:Source binding 与热力 Meter
|
||||
|
||||
M8 将“协议连接”和物理 Meter 分开:`MeterSource` 产生稳定 channel,`MeterSourceBinding` 在 `[started_at, ended_at)` 内把 channel 接到一个 Meter epoch。DSMR electricity 与 WarmteLink heating `GJ`、hot_water `m³` 都使用这条链。
|
||||
|
||||
- 正常成本周期的两端必须解析到**同一** Meter 和 binding;跨 epoch、跨 binding、无 binding、读数陈旧或质量不可接纳时一律 degraded,不跨累计域相减。
|
||||
- source switch 是关闭/新建 binding,不创建假 Meter swap;实际换表才创建新的 Meter epoch。对于 thermal,heating 与 hot_water 独立换表,热力组合 HA identity 因任一 UUID 改变而更新。
|
||||
- `meter_cost_period` 为 heating 与 hot_water 保存 15 分钟 Decimal quantity/cost、binding、合同版本、price snapshot 和 degraded reason。固定费是合同级日汇总,只计一次。
|
||||
|
||||
部署与回滚串口 source 参见 [`warmtelink-energy.md`](./warmtelink-energy.md);保留旧 source/binding/history 可使审计和重算可重复,不能通过删除历史来“修复”边界周期。
|
||||
|
||||
## 生命周期操作与 stranded binding 恢复
|
||||
|
||||
M8-R08/R09 为 Meter 与 binding 增加了显式的生命周期操作。所有时间都由前端按本地日期时间输入,再按既有
|
||||
Principle-A 约定交给后端;未来时间会被拒绝。
|
||||
|
||||
- **Close Meter**:只能关闭 active Meter。该 Meter 与它的所有 open binding 在同一个 `ended_at`、同一
|
||||
事务中关闭;关闭后该 commodity 没有 active Meter。
|
||||
- **Unbind**:可对任意 open binding 执行,只写 binding 的 `ended_at`,绝不删除历史。已关闭 Meter 上残留的
|
||||
open binding 也可在 UI 中解绑,默认关闭时间为该 Meter 的 `ended_at`。
|
||||
- **Transfer**:以单个原子请求切换 source channel,不采用浏览器端“先关闭再新建”的两步操作。同一 Meter
|
||||
的 source switch 在同一 `effective_at` 关闭旧 binding、开启新 binding。失败时 binding、Meter 与受影响的
|
||||
成本重算全部回滚,不会显示或留下部分成功。
|
||||
|
||||
新建或更新 binding 必须完整落在所属 Meter epoch 内:
|
||||
`meter.started_at <= binding.started_at < binding.ended_at <= meter.ended_at`(Meter 已关闭时);
|
||||
open-ended binding 只允许属于 active Meter。Close、Unbind、Transfer 与声明 Meter 都从最早受影响边界重算到
|
||||
当前时间;electricity 与 heating/hot-water 分别使用对应的成本引擎,重算失败时整笔生命周期变更回滚。数据库
|
||||
提交成功后才会 best-effort 重新发布 HA discovery;发布失败不会伪装成持久化失败。
|
||||
|
||||
### 换表自动交接与人工恢复
|
||||
|
||||
声明 `reason=meter_swap` 的新 Meter 时,若未选择 channel 且旧 active Meter 恰好有一条唯一、单位兼容的
|
||||
open binding,系统会在新 Meter 起点自动把该 channel 原子交接过去。存在多个候选或时间线歧义时,声明会
|
||||
fail closed 并整体回滚。不是这种唯一自动交接的声明,也会在新边界关闭旧 Meter 的 open binding,避免产生新的
|
||||
“closed Meter + open binding”。
|
||||
|
||||
旧版本或历史异常可能已经留下 stranded binding:前一块 closed Meter 仍有 open DSMR/WarmteLink binding,
|
||||
而当前 active Meter 没有 binding。不要修改数据库、不要跑 migration,也不会在启动时自动修复。请在
|
||||
**Energy → Meters** 使用该 stranded binding 的 **Recover binding** 操作:选择当前 active Meter 与兼容
|
||||
channel,提交一次 Transfer。来源必须是同 commodity、唯一且紧邻的前一块 closed Meter;旧 binding 固定在旧
|
||||
Meter 的 `ended_at` 结束,新 binding 从选择的 `effective_at` 开始。默认是新 Meter 的 `started_at`;选择更晚
|
||||
时间是允许的,但 UI 会提示这段明确的 unbound gap。非前序 Meter、单位不匹配、channel 在无关区间被占用或歧义
|
||||
都会被拒绝,不会误关历史。
|
||||
|
||||
人工 walkthrough 只能使用开发库里**已经存在且由用户报告的** stranded row。不得为了演示通过 SQL、API、
|
||||
脚本、migration 或直接改数据库制造这种历史异常;隔离开发库中没有该 row 时,记录此项为 `N/A/blocked` 即可。
|
||||
Close、Unbind、same-Meter Transfer、Recover binding 与 meter-swap handoff 都会改变 Meter 或 binding 状态,
|
||||
因此应作为彼此独立的验收场景:每个场景使用各自满足前置条件的 Meter/binding,或在执行前恢复独立前置状态,
|
||||
不能把它们串成会互相破坏前提的单一故事。
|
||||
|
||||
上述恢复不新增 Alembic migration、不执行启动修复,也不删除 Meter、binding、reading 或成本历史;它只把用户
|
||||
确认的时间线修正为可审计的闭区间。
|
||||
|
||||
## 非目标(本里程碑不做)
|
||||
|
||||
- Gas / 区域供暖的计费(`commodity != "electricity"` 的 strategy)。
|
||||
- Gas 计费 strategy。
|
||||
- "家庭(home)"分组实体;多合同时间线积分。
|
||||
- 自动识别换表(DSMR 帧无电表序列号,无法自动识别,靠用户显式声明)。
|
||||
- `last_reset` 信号(消除 HA 长期统计 blip)。
|
||||
|
||||
Binary file not shown.
@@ -0,0 +1,198 @@
|
||||
# DDSU666 Modbus 协议(从官方 PDF 提取)
|
||||
|
||||
> 来源:`docs/references/DDSU666 Single phase Smart Meter.pdf`
|
||||
> (CHINT / 正泰仪表 **DDSU666 Single phase Smart Meter — Operation Manual**,文档号 `ZTY0.464.1224`,版本 **V2**,2020 年 8 月;厂商 Zhejiang Chint Instrument & Meter Co., Ltd.)
|
||||
> 本文件是 PDF 的可读化提取,供本项目的 Modbus 采集驱动设计参考。**以官方 PDF 为准**,本文件如有出入以 PDF 为准。
|
||||
> 同类文档见 SDM120 的 `SDM120-Modbus-Protocol.md`;两表差异较大,见下方 §6「与 SDM120 的关键差异」。
|
||||
|
||||
## 设备速览(来自手册 Table 1 / Table 5)
|
||||
|
||||
| 项 | DDSU666(直接接入) | DDSU666-CT(经互感器) |
|
||||
| --- | --- | --- |
|
||||
| 精度等级 | Active Class B | Active Class C |
|
||||
| 参考电压 | 230 V | 230 V |
|
||||
| 电流规格 | 0.25–5(80) A | 0.015–1.5(6) A |
|
||||
| 表常数 | 800 imp/kWh | 6400 imp/kWh |
|
||||
| 接入方式 | 直接接入 | 经电流互感器 |
|
||||
|
||||
- 单相电子式电能表,DIN35mm 导轨安装;测量电压、电流、有功/无功功率、频率、功率因数、正/反向有功电能。
|
||||
- 电能测量范围 `0~999999.99 kWh`(LCD 只显示 6 位,自动移动小数点)。
|
||||
- 通信:RS485,**Modbus-RTU**(也支持 DL/T 645-2007,可切换,见 §5 `0005H ChangeProtocol`)。
|
||||
- 手册 Table 1 标注 Frequency Reference = 60Hz,但 LCD 示例又写 `F=50.00Hz`(手册自身不一致);**实际频率以寄存器 `200EH` 读数为准**,不要把 50/60 写死。
|
||||
|
||||
## 0. 本项目的接入方式(重要)
|
||||
|
||||
DDSU666 物理层是 **Modbus RTU(RS-485 串口)**,半双工。和 SDM120 一样,本项目通过一个 **Modbus-TCP 网关**接入:
|
||||
|
||||
- 后端用 **Modbus TCP**(`IP:port`)连到网关,网关在串口侧转成 RTU 与电表通信。
|
||||
- TCP 帧用 MBAP header、**无 CRC**(CRC 由网关在 RTU 侧处理)。本文档里 RTU 帧的 `CRC (Lo/Hi)` 字段在 TCP 模式下不需要我们关心。
|
||||
- **Slave Address / Unit ID = 电表的通信地址 Addr**(范围 1–247;面板按键只能设 1–99;见 §5 `0006H`),在 TCP 请求里作为 unit id 传入。
|
||||
- 若以后直连串口(RTU),才需要管波特率 / 数据格式 / CRC:**默认串口格式是 8 数据位、无校验、2 停止位(8N2)**,与 SDM120 的 8N1 不同——直连时务必对齐。
|
||||
- 电表必须处于 **Modbus 协议模式**(而非 DL/T 645)才能用本协议;可经面板长按切换,或写 `0005H = 2`(见 §5)。
|
||||
|
||||
## 1. 协议帧格式(Appendix A)
|
||||
|
||||
异步传输,按字节为单位。一帧 10 位字符 = **1 起始位(0) + 8 数据位(无校验) + 2 停止位(1)**(其它格式可定制)。
|
||||
|
||||
### 信息帧结构(Table A.1)
|
||||
|
||||
| 字段 | 长度 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| Start(起始) | >3.5 字符静默 | 帧间至少 3.5 字符空闲时间作为分隔 |
|
||||
| Address code(地址码) | 1 字节 | 目标从机地址 1–247;每个从机在总线上地址唯一 |
|
||||
| Function code(功能码) | 1 字节 | 仅支持 **03H / 10H**(见 §2) |
|
||||
| Data(数据域) | n 字节 | 随功能码不同而不同(起始地址、寄存器数、寄存器数据等) |
|
||||
| CRC check code(CRC 校验) | 2 字节 | 16-bit CRC(**低字节在前、高字节在后**;多项式 `A001`) |
|
||||
| End(结束) | >3.5 字符静默 | 帧间静默 |
|
||||
|
||||
> TCP 网关模式下 Start/End 静默与 CRC 由网关处理,本项目不关心。
|
||||
|
||||
### 功能码 03H 示例(读寄存器,Table A.3/A.4)
|
||||
|
||||
读从机 `01H`、起始地址 `0CH`、2 个寄存器:
|
||||
|
||||
- 主机发送:`01 03 00 0C 00 02 04 08`(最后 `04 08` 是 CRC,低字节 `04` 在前)。
|
||||
- 从机返回(设 `0CH/0DH` 内容为 `0000H`、`1388H`):`01 03 04 00 00 13 88 F7 65`
|
||||
- `04` = 字节数;`00 00` = `0CH` 数据;`13 88` = `0DH` 数据;`F7 65` = CRC(低字节 `F7` 在前)。
|
||||
|
||||
> **注意**:单个 16-bit 寄存器内是「高字节在前、低字节在后」(Table A.4 里 `0DH` 数据返回 `13 88` = `0x1388`)。这一点对解码浮点的字节序很关键,见 §3。
|
||||
|
||||
### 功能码 10H 示例(写多个寄存器,Table A.5/A.6)
|
||||
|
||||
向从机 `01H`、起始地址 `00H` 连续写 3 个寄存器 `0002H,1388H,000AH`:
|
||||
|
||||
- 主机发送:`01 10 00 00 00 03 06 00 02 13 88 00 0A 9B E9`
|
||||
- `06` = 写入字节数(3 寄存器 × 2 字节);随后是 3 个寄存器的数据;末尾 `9B E9` CRC。
|
||||
- 从机返回:`01 10 00 00 00 03 80 08`(回显起始地址 + 寄存器数 + CRC `80 08`)。
|
||||
|
||||
### 异常响应(Table A.7/A.8)
|
||||
|
||||
- 异常时返回的 Function Code = **原功能码 + 128**(即最高位置 1,`03H→83H`、`10H→90H`)。
|
||||
- 数据为单字节 Error Code:
|
||||
|
||||
| Error Code | 含义 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| `01H` | Illegal function code | 收到的功能码本表不支持 |
|
||||
| `02H` | Illegal register address | 寄存器地址超出有效范围 |
|
||||
| `03H` | Illegal data value | 数据值超出对应地址的取值范围 |
|
||||
|
||||
## 2. 功能码
|
||||
|
||||
DDSU666 **只支持两个功能码**(Table A.2):
|
||||
|
||||
| 功能码 | 作用 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| **03H** | Read register(读寄存器) | 读一个或多个寄存器——**测量值、电量、配置全部走它** |
|
||||
| **10H** | Write multiple registers(写多个寄存器) | 向 n 个连续寄存器写 n 个 16-bit 数据(改配置 / 清电量) |
|
||||
|
||||
> ⚠️ **DDSU666 没有「输入寄存器 / FC04」概念**——所有数据(包括电压电流功率)都用 **FC 03H** 读保持寄存器。这是它和 SDM120(测量值走 FC04)最大的踩坑差异,见 §6。
|
||||
|
||||
## 3. 数据编码
|
||||
|
||||
DDSU666 有两类数据:
|
||||
|
||||
1. **配置 / 参数寄存器(§5,`0000H`–`0010H`)**:每个 1 个寄存器、`16-bit with symbols`(**16 位有符号整数**)。
|
||||
2. **测量 / 电量寄存器(§4,`2000H`+ / `4000H`+)**:每个参数 = **32-bit IEEE-754 单精度浮点**(手册写 “single precision floating decimal”),占 **2 个相邻寄存器**(Length = 2 Word)。
|
||||
|
||||
**浮点字节序 / 字序**
|
||||
|
||||
- **字节序(byte order)= 大端:寄存器内高字节在前** —— 由 Table A.4 的 `0x1388` 返回为 `13 88` 确认。
|
||||
- **字序(word order,两个寄存器谁是高 16 位)**:手册**没有给出浮点解码的实例**,未明确标注。按标准 Modbus 浮点惯例应为**大端字序(高寄存器在前,`ABCD`)**,与本项目 SDM120 驱动一致(`registers_to_float` 用 `>f`,高寄存器在前)。
|
||||
- ⚠️ **需上机实测确认**:读 `2000H`(电压)应解出 ~230V 这样的合理值;若解出乱数,多半是字序相反,改成「低寄存器在前」再试。CHINT 同系列(DTSU/DDSU666)现场固件偶有字序差异,**首次接入务必用一个已知量(电压)校验**,不要凭手册想当然。
|
||||
|
||||
> Python 解码(大端、高寄存器在前):`struct.unpack('>f', struct.pack('>HH', hi_reg, lo_reg))[0]`。
|
||||
> pymodbus:`BinaryPayloadDecoder.fromRegisters(regs, byteorder=Endian.BIG, wordorder=Endian.BIG)`。
|
||||
|
||||
## 4. 测量 / 电量寄存器表(FC 03H 读)
|
||||
|
||||
全部为只读、`Float`(32-bit),每项占 **2 个寄存器**。地址为 Modbus 协议原始地址(即帧里的 Start Register Address Hi/Lo),手册用十六进制。
|
||||
|
||||
### 4.1 瞬时量(“Electric quantity of the secondary side”,`2000H` 段)
|
||||
|
||||
| 地址(hex) | 参数 | 代号 | 单位 | 备注 |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| `2000H` | 电压 Voltage | U | V | |
|
||||
| `2002H` | 电流 Current | I | A | |
|
||||
| `2004H` | 有功功率 Active power | P | **kW** | 手册标注 “the unit is KW”——**不是 W** |
|
||||
| `2006H` | 无功功率 Reactive power | Q | **kvar** | |
|
||||
| `2008H` | (保留 RESERVED) | — | — | 占 2 寄存器,跳过 |
|
||||
| `200AH` | 功率因数 Power factor | PF | — | 无量纲 |
|
||||
| `200CH` | (保留 RESERVED) | — | — | 占 2 寄存器,跳过 |
|
||||
| `200EH` | 频率 Frequency | Freq | Hz | |
|
||||
|
||||
### 4.2 电量(“Electrical data of the secondary side”,`4000H` 段)
|
||||
|
||||
| 地址(hex) | 参数 | 代号 | 单位 | 备注 |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| `4000H` | 正向(导入)有功电能 Active in electricity | Ep | kWh | 正向 / forward active energy |
|
||||
| `400AH` | 反向(导出)有功电能 Reverse in electricity | -Ep | kWh | 反向 / reverse active energy |
|
||||
|
||||
> 手册里 `4000H` 与 `400AH` 之间(`4002H`–`4009H`)未列出,视为保留/未文档化。
|
||||
|
||||
**读取分块建议**
|
||||
|
||||
- 瞬时量:`2000H`–`200FH` 共 **16 个寄存器连续**,一次块读即可覆盖 U…Freq(含两段 RESERVED,解码时跳过)。
|
||||
- 电量:`4000H`(2 寄存器)与 `400AH`(2 寄存器)相距较远,分两小块读,或读 `4000H`–`400BH`(12 寄存器)一次取出后挑用——以网关 / 电表是否允许跨保留地址块读为准,谨慎起见分开读更稳。
|
||||
|
||||
### 常用核心子集(日常监控够用)
|
||||
|
||||
电压 `2000H`、电流 `2002H`、有功功率 `2004H`(kW)、功率因数 `200AH`、频率 `200EH`、正向有功电能 `4000H`、反向有功电能 `400AH`。
|
||||
|
||||
## 5. 配置 / 参数寄存器表(FC 03H 读 / FC 10H 写)(Table 9)
|
||||
|
||||
每个 1 个寄存器、**16-bit 有符号整数**。`R/W` 列来自手册。
|
||||
|
||||
| 地址(hex) | 代号 | 含义 | R/W | 取值 / 说明 |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| `0000H` | UCode | 编程密码 Programming password code | R/W | 写配置前的密码字 |
|
||||
| `0001H` | REV. | 保留;**实际读出的是版本号** | R | |
|
||||
| `0002H` | ClrE | 电能清零 CLr.E | R/W | **写 `1` 清除总电量**(不可逆,慎用) |
|
||||
| `0003H`–`0004H` | RESERVED | 保留 | — | |
|
||||
| `0005H` | ChangeProtocol | 协议切换 | R/W | **`2` = Modbus-RTU**,`1` = DL/T 645-2007 |
|
||||
| `0006H` | Addr | 通信地址 | R/W | 1–247(面板按键仅 1–99) |
|
||||
| `0007H`–`000AH` | RESERVED | 保留 | — | |
|
||||
| `000BH` | Meter type | 表型 Meter type | R | 只读设备类型标识 |
|
||||
| `000CH` | BAud | 通信波特率 | R/W | **`1`=2400bps,`2`=4800bps,`3`=9600bps**(手册寄存器仅列这三档;通信章另提到也支持 1200bps) |
|
||||
| `000DH`–`0010H` | RESERVED | 保留 | — | |
|
||||
|
||||
> ⚠️ 写 `0002H`(清电量)、`0006H`(改地址)、`000CH`(改波特率)、`0005H`(切协议)都会改变电表状态或通信参数,配错可能**清空累计电量**或**导致通信中断**。本项目默认**只读采集**,不在自动化链路里写电表配置寄存器。
|
||||
> 写配置通常需先经 `0000H UCode` 密码字校验;具体密码值手册正文未给出,需向厂商确认或经面板操作。
|
||||
|
||||
## 6. 与 SDM120 的关键差异(迁移 / 复用驱动时必看)
|
||||
|
||||
本项目已有 SDM120 profile(`app/integrations/modbus/profiles/sdm120.yaml`)。DDSU666 **不能照搬**,主要差异:
|
||||
|
||||
| 维度 | SDM120 (Eastron) | DDSU666 (CHINT) |
|
||||
| --- | --- | --- |
|
||||
| 读测量值功能码 | **FC 04**(输入寄存器 3X) | **FC 03**(保持寄存器,无 FC04) |
|
||||
| 测量值起始地址 | `0x0000` 起(30001) | 瞬时量 `0x2000` 起;电量 `0x4000`/`0x400A` |
|
||||
| 有功功率单位 | **W**(瓦) | **kW(千瓦)** —— 入库前注意换算 / 单位标注 |
|
||||
| 无功功率单位 | VAr | kvar |
|
||||
| 配置寄存器格式 | Float(FC03/16) | **16-bit 有符号整数**(FC03/10) |
|
||||
| 写功能码 | 16 / 0x10 | 10H(同 0x10) |
|
||||
| 串口默认格式 | 8N1(1 停止位) | **8N2(2 停止位)** |
|
||||
| 多协议 | 仅 Modbus | Modbus **与 DL/T 645-2007 可切换**(需确保在 Modbus 模式) |
|
||||
| 浮点字序 | 大端、高寄存器在前(手册有实例佐证) | 字节序大端已确认;**字序手册无实例,需上机实测** |
|
||||
|
||||
## 7. 给本项目采集驱动的要点小结
|
||||
|
||||
1. 走 **Modbus TCP 网关**:`ModbusTcpClient(host, port)`,`slave=<Addr>`;电表须在 **Modbus 协议模式**。
|
||||
2. 所有读取(测量 + 电量 + 配置)都用 **FC 03H**——**没有 FC04**。
|
||||
3. 测量值在 `0x2000` 段、电量在 `0x4000`/`0x400A`,均为 **float32**;解码大端字节序,**字序默认高寄存器在前但务必用电压实测校验**。
|
||||
4. **有功功率单位是 kW、无功是 kvar**——与 SDM120 的 W/VAr 不同,新建 profile / 入库映射时单位别抄错。
|
||||
5. 配置寄存器(`0x0000`–`0x0010`)是 **16-bit 有符号整数**,不是 float。
|
||||
6. 默认**只读**;`0002H` 写 1 会**清空累计电量**、`0006H/000CH/0005H` 会改通信参数,自动化链路里一律不写。
|
||||
7. 新建 profile 时这是 `ddsu666` 这一个 register profile 的定义;建议 `function_code: 3`、`word_order: big`(先按大端字序,接入后用电压读数验证)、瞬时量与电量分块读。
|
||||
|
||||
## 8. 实测记录(真机验证,2026-06-30)
|
||||
|
||||
首次接入一台 **DDSU666 直接接入版(5(80)A)** 实测,确认以下几点:
|
||||
|
||||
- **字序大端,确认无误**:电压 / 电流 / 频率 / 电能用「大端、高寄存器在前」解码全部得到合理值(如 233.9 V / 0.055 A / 49.99 Hz),与 §3 的假设一致。`ddsu666.yaml` 的 `word_order: big` / `byte_order: big` **无需修改**,§3 里「字序需上机实测」一项可视为已关闭。
|
||||
- **FC03 读通**:所有量走 FC03,profile `ddsu666` 在采集链路(CLI `read` / 设备 `/test` / 后台轮询)中工作正常。
|
||||
- **低电流下「瞬时功率读 0、但电能照常累加」**:测试负载仅为一台 PoE 交换机(≈230 V / 0.05 A,真实有功仅几瓦),**远低于本表测量量程下限 Imin≈0.25 A**。此工况下:
|
||||
- 瞬时 `active_power`(`0x2004`)与 `power_factor`(`0x200A`)寄存器返回**全零**(原始 hex `0x0000 0x0000`);电表 LCD 上功率在 0~3.3 W、PF 在 0~0.6 之间抖动。
|
||||
- 抖动成因 = **低电流测量噪声 + 开关电源(SMPS,无 PFC)畸变电流**(电流为电压峰值附近的窄脉冲、谐波重 → 畸变功率因数天然偏低)。**不是**「采样率与开关频率拍频」:计量芯片 SH79F7019 采样在 kHz 量级,远低于开关电源 50–200 kHz 的开关频率,两者不在一个频段。
|
||||
- 但累计电能 `import_energy`(`0x4000`)**正常累加**(实测 0 → 0.01 kWh)——电表内部积分器在防潜动起始电流(`0.004·Ib`≈0.02 A)之上照常计量。
|
||||
- **结论**:低于量程下限时**瞬时功率 / PF 不可信,但电能计量不丢**;电流进入量程(正常负载)后瞬时量即稳定可信。这是 5(80)A 大量程表对极小负载的固有特性,**非缺陷、非解码问题**。
|
||||
- **排错提示**:若日后看到 DDSU666「功率一直 0」,先确认负载电流是否在 Imin 以上——多半是负载太轻而非链路故障;可用 `scripts/modbus_cli probe --fc 3 --address 0x2000 --count 16` 看 `0x2004/0x2005` 原始寄存器是否真为全零佐证。
|
||||
@@ -113,16 +113,26 @@ curl -s -X POST https://api.tibber.com/v1-beta/gql \
|
||||
> "De verkoopvergoeding van 2,48 cent is gelijk aan de inkoopvergoeding die je bij je afgenomen stroom betaalt."
|
||||
> (卖侧 verkoopvergoeding 2.48 分 = 买侧 inkoopvergoeding。)
|
||||
|
||||
→ **买卖服务费相等(均 €0.0248/kWh)**,在买卖里一进一出**相互抵消**。
|
||||
→ **买卖服务费金额相等(均 €0.0248/kWh),但两者对住户都是成本、不互相抵消**:
|
||||
- 买侧 inkoopvergoeding 已经**包含在 Tibber API 的 `total` 里**(见下 §3.1 的实证拆解),买电按 `total` 计价即已含它。
|
||||
- 卖侧 verkoopvergoeding 则是从回送价里**额外扣掉**的一笔——所以回送价 = `total − 0.0248`,比买价低 0.0248/kWh。
|
||||
- ⚠️ **早期版本误判为"一进一出抵消 → 回送=total"**,这是错的:`total` 里那笔 inkoopvergoeding 不会退回来充抵 verkoopvergoeding。代码里用 `energy.sell_fee`(默认 0.0248)建模这笔卖侧费用。
|
||||
|
||||
---
|
||||
|
||||
## 3. 净计量(saldering)、回送(teruglevering)、负电价、2027
|
||||
|
||||
### 3.1 回送价(净计量期内,文档原文)
|
||||
### 3.1 回送价(净计量期内,文档原文 + 实证)
|
||||
> "Op het moment dat je teruglevert geven we je per kWh de beursprijs die op dat moment geldt …, inclusief energiebelasting en inkoopvergoeding plus de btw minus de verkoopvergoeding."
|
||||
|
||||
即净计量期内回送价 = `beursprijs + energiebelasting + inkoopvergoeding + btw − verkoopvergoeding`。因 inkoopvergoeding = verkoopvergoeding 抵消 → **= 全额零售价**(spot+能源税+VAT),正是 saldering "回送 1 度 = 用 1 度"的本质。
|
||||
> **Worked example(Tibber NL 原文)**:"Stel dat tussen 14:00 en 14:15 de totale stroomprijs €0,28 per kWh incl. is, dan krijg je €0,28 − €0,0248 verkoopvergoeding = **€0,2552** per teruggeleverde kWh terug."
|
||||
|
||||
即净计量期内回送价 = `beursprijs + energiebelasting + inkoopvergoeding + btw − verkoopvergoeding`,而官方例子直接写成 **`回送价 = totale stroomprijs − verkoopvergoeding = total − 0.0248`**。能源税**退回**(留在 total 里没动),只有 verkoopvergoeding 这 0.0248 被扣。
|
||||
|
||||
**✅ 实证(本项目生产库,2026-07-20 三个刻钟)**:按 21% VAT 拆 `total`:`total = 现货×1.21 + energiebelasting(0.11085) + inkoopvergoeding(0.0248)`,三段解出的 inkoop 都精确等于 **0.0248**。→ **我们存的 `tibber_price.total` 就是官方 "totale stroomprijs"(含 inkoopvergoeding 的买价)**,因此:
|
||||
- 买价 `buy = total`(已含 inkoopvergoeding,正确)。
|
||||
- 净计量回送价 `sell = total − verkoopvergoeding = total − 0.0248`。
|
||||
- ⚠️ 所以 saldering 下"回送 1 度"仍比"用 1 度"少 0.0248——**不是完全 1:1**。代码用 `sell_fee` 建模这笔扣减,`sell_adjust` 只负责在净计量期把能源税补回(`sell_adjust = −energy_tax`)。
|
||||
|
||||
### 3.2 年末盈余 / 取消净计量后(文档原文,Scenario 2)
|
||||
> "Voor de overproductie van 500 kWh heb je recht op de beursprijs en de inkoopvergoeding, maar heb je geen recht op de energiebelasting. … ontvang je nog een factuur van ons voor de te veel uitgekeerde belastingen …"
|
||||
@@ -146,9 +156,11 @@ curl -s -X POST https://api.tibber.com/v1-beta/gql \
|
||||
|
||||
> spot 取 API `energy`;`total = energy + tax`(全包)。**买价直接用 `total`**,卖价从 `total` 扣掉卖电不交的能源税。
|
||||
|
||||
- **Tibber 动态合同**(post-2027 口径):
|
||||
- 买价 `buy = price.total`
|
||||
- 卖价 `sell = price.total − energy_tax_per_kwh − sell_adjust`(`sell_adjust` 默认 0;含 VAT 归己;买卖费抵消已隐含在 total 里)
|
||||
- **Tibber 动态合同**:
|
||||
- 买价 `buy = price.total`(含 energy_tax + VAT + inkoopvergoeding)
|
||||
- 卖价 `sell = price.total − energy_tax − sell_fee − sell_adjust`
|
||||
- `sell_fee`:verkoopvergoeding(卖侧上网费,默认 **0.0248**,含 VAT),**始终扣除**——即使净计量期也扣(见 §3.1)。
|
||||
- `sell_adjust`:手动修正项(默认 0)。**净计量期**设为 `−energy_tax`(把能源税补回),得 `sell = total − sell_fee`;**2027 取消净计量后**设为 0,得 `sell = total − energy_tax − sell_fee`(无能源税、纯市场价再扣上网费)。
|
||||
- **固定合同(manual,双费率)**:
|
||||
- 买价 `buy_档 = energy_buy_档 + energy_tax`(档 ∈ {normal, dal})
|
||||
- 卖价 `sell_档 = sell_档`(回送价,**无能源税**)
|
||||
@@ -242,7 +254,7 @@ extra_device_timestamp, extra_device_delivered # 燃气表(m³,每
|
||||
## 8. 待真实数据核对(合同生效后用真实 token / 账单)
|
||||
|
||||
1. **真实 token 复核**:跑 §1.4 的 15 分钟 curl,确认 NL 返回**真** 15 分钟价(非重复小时价)+ 币种 EUR。
|
||||
2. **卖价残差**:确认 `total` 里 purchase fee 是否被卖侧 sales fee 完全抵掉、回送 VAT 口径 → 调 `sell_adjust`(默认 0)。
|
||||
2. ~~**卖价残差**:确认 `total` 里 purchase fee 是否被卖侧 sales fee 完全抵掉~~ → **已核实(2026-07)**:`total` 含 inkoopvergoeding(0.0248),净计量回送价 = `total − verkoopvergoeding(0.0248)`,两费**不抵消**;代码以 `sell_fee`(默认 0.0248)建模。仍待真实账单核对 `sell_fee` / VAT 口径的最终残差。
|
||||
3. **双费率寄存器映射**:确认 `_1`=dal/`_2`=normal 没接反(差价小但要对)。
|
||||
4. **能源税年值**:按当年实际值与年用电档位核 `energy_tax`。
|
||||
5. **固定合同数值**:回送两档价、电网费、heffingskorting 待用户从账单填。
|
||||
|
||||
+95
-2
@@ -2,7 +2,7 @@
|
||||
|
||||
本文档记录 `home-automation` 在 `v1.0.3` 之后的下一阶段规划。这一阶段不是小修补,而是几次较大的结构性改动:单库化、前端重写、以及远期的移动端试水。
|
||||
|
||||
> 每个里程碑的**可执行原子任务**展开在 [`docs/design/`](./design/README.md):M1 [`m1-db-consolidation.md`](./design/m1-db-consolidation.md)、M2 [`m2-frontend-v2.md`](./design/m2-frontend-v2.md)、M3 [`m3-token-mobile.md`](./design/m3-token-mobile.md)、M4 [`m4-login-hardening.md`](./design/m4-login-hardening.md)、M5 [`m5-iot-energy.md`](./design/m5-iot-energy.md)、M6 [`m6-tibber-dynamic-energy.md`](./design/m6-tibber-dynamic-energy.md)、M7 [`m7-meter-epochs-archival.md`](./design/m7-meter-epochs-archival.md)。这些文档为 Orchestrator→Implementer→Reviewer 的多模型流水线设计。
|
||||
> 每个里程碑的设计与**可执行原子任务**展开在 [`docs/design/`](./design/README.md):M1 [`m1-db-consolidation.md`](./design/m1-db-consolidation.md)、M2 [`m2-frontend-v2.md`](./design/m2-frontend-v2.md)、M3 [`m3-token-mobile.md`](./design/m3-token-mobile.md)、M4 [`m4-login-hardening.md`](./design/m4-login-hardening.md)、M5 [`m5-iot-energy.md`](./design/m5-iot-energy.md)、M6 [`m6-tibber-dynamic-energy.md`](./design/m6-tibber-dynamic-energy.md)、M7 [`m7-meter-epochs-archival.md`](./design/m7-meter-epochs-archival.md)、Pre-M8 [`pre-m8-warmtelink-p1-poc.md`](./design/pre-m8-warmtelink-p1-poc.md)、M8 [`m8-warmtelink-energy.md`](./design/m8-warmtelink-energy.md)。Pre-M8 与 M8-T01~T20 的自动化技术验收均已完成;M8 交付后用户人工 walkthrough 待验收。
|
||||
|
||||
## 当前基线(v1.0.3)
|
||||
|
||||
@@ -40,6 +40,8 @@
|
||||
| **M5** ✅ | IoT / 能耗采集 | 通用 Modbus 采集(YAML profile + JSON readings)+ MQTT/HA Discovery + 前端侧边栏 + Energy 视图 |
|
||||
| **M6** ✅ | 通用电价层 + DSMR 接入 + 实时电费计算 | 通用电价层(manual/tibber profile + 合同版本)+ DSMR 实时电表接入 + 每 15min 寄存器差×价计量电费(不可变快照)+ 日/月/年汇总 + 反哺 HA Energy + 前端合同/价格/费用视图 |
|
||||
| **M7** ✅ | 电表生命周期 / 换表归档 | 引入 Meter epoch,计费永不跨表算 delta,跨表/无表/异常 delta 一律降级,累计按当前表归零,追溯换表可重算,Meter CRUD API + 前端管理 UI |
|
||||
| **Pre-M8** ✅ | WarmteLink P1 真机概念验证 | 正式只读 CLI 长测通过;人工开启供暖后累计量 `0.017 → 0.018 GJ` 且与物理表一致,所有 frame 的 CRC 状态仍为 `unverifiable` |
|
||||
| **M8** ✅ | WarmteLink P1、多数据源 Meter 与热力计费 | M8-T01~T20 自动化技术验收完成:Source/Channel/Binding、WarmteLink、DSMR 迁移、thermal 合同/成本、HA/UI、部署与文档闭环;真实 serial/HA walkthrough 由用户交付后验收 |
|
||||
| **M3** | 开放与移动端(远期试水) | token 鉴权 + React Native 移动端 |
|
||||
|
||||
排序原则:**先清地基,再在干净结构上盖楼。** M2 的新 API 和 React 必须建立在合并后的单库之上;M4 是公网安全加固,在 M5 IoT 集成之前先堵住裸密码这个洞;M5 在安全基座就绪后再做 IoT 接入。
|
||||
@@ -257,6 +259,72 @@ httpx / paho-mqtt / pyyaml / apscheduler 均为 M5 已有依赖,M6 复用,
|
||||
|
||||
---
|
||||
|
||||
## Pre-M8 — WarmteLink P1 真机概念验证(✅ 已完成)
|
||||
|
||||
### 目标
|
||||
|
||||
2026-08-22 的正式仓库 CLI 以 `115200 7N1` 连续运行 10 分钟,正常退出且没有 I/O error、未处理
|
||||
异常或断连。一个 WarmteLink source 在 60/60 完整 frame 中暴露 channel 1 的生活热水累计量
|
||||
`5.900 m³` 和 channel 2 的区域供暖累计量 `0.017 GJ`;二者再次与同日物理表一致。没有流量、
|
||||
热功率或温度字段。
|
||||
|
||||
所有完整 frame 都缺少标准 `/`,CRC 因而为 `unverifiable`;这项异常由正式 probe 原样报告,
|
||||
没有伪装成校验通过。frame 平均 256 bytes、范围 237–275,设备 timestamp 严格每 10 秒递进。
|
||||
10 分钟内累计值无变化只能证明稳定基线,不能证明消费时更新频率或 reset/wrap 行为。
|
||||
|
||||
最终人工走查又在终端连续运行 probe 十几分钟并开启供暖;区域供暖累计量从 `0.017 GJ` 增至
|
||||
`0.018 GJ`,物理热量表同步显示 `0.018 GJ`。这确认了实际消费时的累计递增与 `0.001 GJ`
|
||||
精度,但不改变 CRC `unverifiable` 结论,也不证明精确更新延迟或 reset/wrap 行为。
|
||||
|
||||
本阶段不落库、不接 API/前端/HA,也不决定正式 Device/Source/Meter 关系。它只向 M8 提供脱敏的真机事实,避免在未知 firmware/字段语义上提前设计。
|
||||
|
||||
> 验证计划与通过条件:[`docs/design/pre-m8-warmtelink-p1-poc.md`](./design/pre-m8-warmtelink-p1-poc.md)
|
||||
|
||||
---
|
||||
|
||||
## M8 — WarmteLink P1、多数据源 Meter 与热力计费(✅ 自动化技术验收已完成)
|
||||
|
||||
### 目标
|
||||
|
||||
以统一的 `MeterSource → MeterSourceChannel → MeterSourceBinding → Meter epoch` 链路承载现有
|
||||
DSMR MQTT 与新的 WarmteLink serial source;从一个 WarmteLink source 只读采集已由真机确认的
|
||||
区域供暖累计 `GJ` 和生活热水累计 `m³`,并完成历史、计费、API、前端、HA 与部署闭环。
|
||||
|
||||
### 已锁定范围
|
||||
|
||||
- Source、Modbus Device、Meter 分离;channel 先发现、再由用户确认绑定,source 切换与物理换表
|
||||
使用两条独立半开时间线。
|
||||
- DSMR JSON 和 WarmteLink Decimal scalar 分表存储,但都显式关联 source/binding;既有 DSMR
|
||||
历史和电费安全回填,正常电费数字保持不变,跨 source/binding 周期降级。
|
||||
- WarmteLink `unverifiable` frame 需要连续双帧确认后接纳,质量标签不漂白;约 10 秒更新 latest、
|
||||
每分钟保存 history,不保存 raw telegram 或 raw equipment identifier。
|
||||
- 串口 worker 使用 `115200 7N1`、只读、短 DB session、可热更新和 `1…60s` 退避重连;现有
|
||||
production base 与 base+dev Compose 通过 `.env` 必填的 `WARMTELINK_DEVICE_PATH`(stable by-id)
|
||||
和 `WARMTELINK_SERIAL_GID` 直配,将设备固定映射为 `/dev/warmtelink:rw`,保持非 root 最小权限;
|
||||
同一物理串口只能有一个 owner。
|
||||
- 合同增加 electricity/thermal scope;一份 district-heating 合同覆盖 heating GJ 与 hot-water m³,
|
||||
生成按 commodity 的 15 分钟成本和 contract-level 每日固定费。仓库不硬编码真实 tariff。
|
||||
- UI 分成 Sources / Modbus Devices / Meters,并让 Contracts / Prices / Costs 按 scope 切换;新增
|
||||
source、meter 与 thermal cost HA entities,开关默认关闭,旧 DSMR latest API 保持兼容。
|
||||
|
||||
### 原子实施链
|
||||
|
||||
- **M8-T01~T06**:已完成统一 source/channel/binding schema、DSMR 历史/runtime/电费迁移和管理 API。
|
||||
- **M8-T07~T11**:已完成共享 P1 parser、WarmteLink 标量存储、质量接纳、serial worker、发现与历史 API。
|
||||
- **M8-T12~T16**:已完成合同 scope、district-heating profile、thermal cost 账本/引擎/API。
|
||||
- **M8-T17~T19**:已完成 HA、Sources/Meters UI 与 scope-aware 计费 UI;**M8-T20** 已完成 compose、文档与隔离自动化技术收尾。
|
||||
|
||||
完成判据不仅是单元闸门全绿,还包括以 mock/fake、合成数据库和隔离 Docker 完成的历史迁移对账、
|
||||
OpenAPI/codegen、全部前端闸门、真实 `docker build` 与非 root 串口部署技术验收;并须交付完整的
|
||||
九步用户 walkthrough 和证据模板。真实 serial/HA 的观察由用户在交付后自行验收,不是 agent/Reviewer
|
||||
技术 PASS、T20 状态更新或 M8 autosquash/收尾的前置条件,也不得写成已执行。任何任务都不得删除旧
|
||||
数据库、历史读数、旧 config 行或 volume;push/tag 仍需用户单独授权。
|
||||
|
||||
> 完整架构、HTTP 契约、质量/计费规则、依赖图与 M8-T01~M8-T20 任务卡:
|
||||
> [`docs/design/m8-warmtelink-energy.md`](./design/m8-warmtelink-energy.md)
|
||||
|
||||
---
|
||||
|
||||
## M3 — 开放与移动端(远期试水)
|
||||
|
||||
### 目标
|
||||
@@ -288,11 +356,36 @@ httpx / paho-mqtt / pyyaml / apscheduler 均为 M5 已有依赖,M6 复用,
|
||||
|
||||
**动机**:浏览器端走 session cookie 即可,但**脚本 / 设备 / 外部程序调用 API** 需要一种长期有效、可随身携带的凭据。在设置页加一组功能,由 admin **手动签发 long-lived token**,之后用它来调 API。
|
||||
|
||||
**本次明确的首要目标 = 给现在裸奔的 ingestion 端点上鉴权**(2026-06-27 与用户确认):
|
||||
|
||||
- `POST /location/record`(`app/api/routes/location.py:18`)——位置记录上报。**目前无任何鉴权**。当前数据经 Home Assistant 转发进来,上 token 后 **HA 侧需携带该 token**;也可由其他客户端直接上报。
|
||||
- `POST /poo/record`(`app/api/routes/poo.py:21`)+ `GET /poo/latest`(`poo.py:57`)——小狗排便记录上报 / 最新查询。**目前无任何鉴权**。
|
||||
- 这些是设备 / 脚本(非浏览器)端点,session cookie 不适用,正是 long-lived token 的用武之地。(浏览器 CRUD `/api/data/*` 已由 session 保护,不在此列。)
|
||||
|
||||
**范围(粗略,待细化)**:
|
||||
|
||||
- 设置页新增「API Token」区:生成 / 命名 / 吊销 long-lived token;明文只在**生成时展示一次**,此后只存哈希。
|
||||
- 后端支持用该 token 鉴权访问 API(与现有 session cookie 并存,互不影响)。
|
||||
- 后端支持用该 token 鉴权访问 API(与现有 session cookie 并存,互不影响);给上述 ingestion 端点加 token 鉴权依赖。
|
||||
- 与 [M3](#m3--开放与移动端远期试水) 的 token 主题相关,但**这条是 Web 设置页手动签发的 PAT 风格**,不依赖移动端 OAuth 流程;两者实现时可复用同一套 token 存储 / 校验。
|
||||
- 与下面第 3 条「Session 滑动续期」同属 Authentication 主题(一个是设备/脚本的长期凭据,一个是浏览器短会话体验),实现时鉴权层可一并梳理。
|
||||
|
||||
### 3. Session 滑动自动续期(Authentication)
|
||||
|
||||
**动机**(2026-06-27 与用户确认):当前 session 是**绝对过期**——登录即定死、活动不续期,满 TTL 必须重新登录,体验割裂。希望改成**滑动续期(sliding / rolling)**:只要用户还在活动就自动延长,提供"在用就不掉线"的体验。
|
||||
|
||||
**现状(实现起点,便于快速拾起)**:
|
||||
|
||||
- TTL 默认 **12 小时**:`auth_session_ttl_hours`(`app/config.py:38`;配置页 `app/services/config_page.py:45` 可运行时改)。
|
||||
- 登录时**一次性写死**:`create_session` 设 `expires_at = now + ttl`(`app/services/auth.py:94`)+ cookie `max_age = ttl`(`app/api/routes/api/session.py:153`)。
|
||||
- 每请求**只读校验、从不延长**:`get_authenticated_session`(`app/services/auth.py:103`)只判断 `expires_at <= now`,过期时仅顺手标 `revoked`。`set_cookie` 只在登录路由调用一次,**无 per-request 中间件**。→ 所以是绝对过期,不是滑动。
|
||||
|
||||
**设计要点(待写设计文档时展开)**:
|
||||
|
||||
- 校验通过时 bump `expires_at = now + ttl` 并**重发 cookie**(滑动窗口)。
|
||||
- **写节流**:不要每个请求都写 DB——仅当剩余寿命已过半(或距上次续期 > N 分钟)才续期,避免高频写放大。
|
||||
- **绝对寿命硬顶**:除滑动 TTL 外再设 `created_at + max_lifetime` 上限,防止"永不过期"的会话(安全考量)。
|
||||
- 新增配置项:滑动 TTL、绝对寿命上限、续期节流阈值。
|
||||
- 注意:改动只对**新逻辑生效**,已存在 session 的 `expires_at` 行为按新校验路径走即可;上线前过校验闸门。
|
||||
|
||||
## Future Ideas(暂不排期,想到先记下)
|
||||
|
||||
|
||||
@@ -0,0 +1,110 @@
|
||||
# WarmteLink、数据源与热力计费运维手册
|
||||
|
||||
本手册说明如何在不改变既有 DSMR、Modbus 和 electricity 功能的前提下,部署只读的 WarmteLink P1 采集、绑定热量/生活热水 Meter、配置热力合同,并按需暴露给 Home Assistant。所有示例均使用占位符;不要把设备标识、GID、数据库路径、token 或真实合同金额提交到仓库。
|
||||
|
||||
## 安全边界与开始前备份
|
||||
|
||||
- 应用容器继续使用基础 `docker-compose.yml` 中的非 root `user: "1000:1000"`;基础文件和 dev 合并配置都不设置 `privileged` 或额外 capability。
|
||||
- Docker device cgroup 必须以 `rw` 映射,容器内固定为 `/dev/warmtelink`:这是 pyserial 3.5 在 POSIX 上以 `O_RDWR` 打开串口所需的最小系统权限,并不表示业务可写。规则绝不包含 `m`,且不授予 root、`privileged` 或额外 capability。WarmteLink worker 仍只调用 serial `read` / `close`,绝不调用 `write` 或发送写命令。
|
||||
- 不删除或覆盖 `app_config`、`app.db`、旧数据库、Docker volume 或既有 source。禁用/解绑/回滚配置不是删除历史的替代方式。
|
||||
- 维护前停止写入窗口,使用宿主机的备份流程复制 `./data/app.db` 到受保护的备份位置;确认备份可用后才运行 migration。不要把生产库复制到开发机或用于测试。
|
||||
|
||||
## 识别稳定串口并配置 Compose
|
||||
|
||||
在宿主机(不是容器)找出稳定 symlink;不要使用会在重启后变化的 `/dev/ttyUSB*` 名称:
|
||||
|
||||
```bash
|
||||
ls -l /dev/serial/by-id/
|
||||
stable_path=/dev/serial/by-id/<stable-by-id-name>
|
||||
stat -c '%g %n' "$stable_path"
|
||||
```
|
||||
|
||||
记录输出的数字 GID,而不是猜测 `dialout` 的数值。确认 path 指向预期的字符设备后,在部署机本地 `.env`(不提交)设置:
|
||||
|
||||
```bash
|
||||
WARMTELINK_DEVICE_PATH=/dev/serial/by-id/<stable-by-id-name>
|
||||
WARMTELINK_SERIAL_GID=<host-serial-gid>
|
||||
```
|
||||
|
||||
production 使用基础 compose;local dev 使用 base 与 dev 合并文件。两种环境都会从本地 `.env` 读取这两个变量:
|
||||
|
||||
```bash
|
||||
docker compose -f docker-compose.yml up -d
|
||||
docker compose -f docker-compose.yml -f docker-compose.dev.yml up -d --build
|
||||
```
|
||||
|
||||
启动前可用相同文件组合运行 `docker compose ... config`。结果的 `app` 必须仍显示 `user: "1000:1000"`,并只出现从 stable by-id path 到 `/dev/warmtelink` 的 `rw` device mapping 与 `group_add` GID;migration 不得有 device 或 group。不得出现 `privileged`、root user 或 `m` device permission。`rw` 仅满足 pyserial 的 POSIX `O_RDWR` 打开,不改变 worker 的只读业务行为。不得把设备路径或 GID 写入仓库的 `.env.example` 或文档。
|
||||
|
||||
同一物理串口在任意时刻只能有一个 owner。运行 Pre-M8 `p1_probe.py` 前,必须先停止 app(包括 dev stack),并在 probe 结束后再启动 app;不要让 probe 与 worker 同时打开该串口。
|
||||
|
||||
## Migration 与 source 配置
|
||||
|
||||
先在维护窗口运行 migration;它只升级 schema,绝不删除历史表或配置:
|
||||
|
||||
```bash
|
||||
docker compose -f docker-compose.yml run --rm migration
|
||||
```
|
||||
|
||||
登录 Energy 页面,在 **Sources** 创建 `warmtelink_serial` source:
|
||||
|
||||
| 字段 | 值 |
|
||||
| --- | --- |
|
||||
| Path | `/dev/warmtelink` |
|
||||
| Baud rate | `115200` |
|
||||
| Data bits / parity / stop bits | `7` / `N` / `1` |
|
||||
| Enabled | 先关闭,保存并检查配置后再开启 |
|
||||
|
||||
这些参数是固定 P1 profile。不要录入 telegram 或 equipment id;应用不会把原始 frame、设备身份或 serial 设置以外的协议标识保存到 source 配置中。开启后选择 **Discover**。成功发现后应有两个只读 channel:heating(`GJ`)和 hot water(`m³`);受 P1 CRC 限制,正常可接纳的质量可能显示为 `unverifiable`,这不是被错误提升为 `valid`。
|
||||
|
||||
在 **Meters** 分别创建或选择 `heating` 与 `hot_water` Meter,并各自选择与单位相符的 channel 创建 binding。source switch 只关闭旧 binding、在相同 Meter 上创建新 binding,不是换表;真正的物理换表才使用 Meter swap。跨 binding 或 Meter 边界的 15 分钟成本周期会明确标为 degraded,不能相减伪造成本。
|
||||
|
||||
## 运行检查与排障
|
||||
|
||||
启用 source 后,latest 通常约每 10 秒更新,持久化 history 最多每分钟一条。检查 source 状态、channel latest/quality 和绑定时间线,而不是从日志中寻找原始 telegram。
|
||||
|
||||
| 现象 | 安全排查 |
|
||||
| --- | --- |
|
||||
| `offline` 或无法 open serial | 核对 stable symlink 是否仍存在、`stat` 的 GID 是否等于 `WARMTELINK_SERIAL_GID`,再用 `docker compose ... config` 检查 non-root `rw`(非 `r`、非 `m`)device rule;不要用 root/privileged 绕过权限。 |
|
||||
| 无 channels / discover 超时 | 确认 source 启用、`/dev/warmtelink` path 和固定 `115200 7N1`;检查电缆供电后重试 Discover。 |
|
||||
| 短暂拔线 | worker 标记 offline 并以退避重连;插回后应恢复。检查 history 的 `(channel, recorded_at)` 唯一性,不能手工补重复行。 |
|
||||
| 成本 degraded | 查两端读数 freshness(120 秒)、quality、Meter epoch 与 binding;不要通过修改累计值清除 degraded。 |
|
||||
|
||||
若需要停采集,在 UI 禁用该 source,确认 worker 关闭串口后再维护电缆。删除有 channel、binding 或历史的 source 会被 API 拒绝;保留记录以保证审计和成本重算。
|
||||
|
||||
## 热力合同、成本与 Home Assistant
|
||||
|
||||
在 **Contracts** 选择 `Thermal` scope,创建 `district_heating` 合同及版本。费率由 operator 按合同人工录入,字段为 heating(EUR/GJ)、hot-water heating / water / tax(EUR/m³)与五个年固定费字段;仓库不含任何真实默认金额。thermal 和 electricity 各可有一个 active 合同,彼此不互斥。
|
||||
|
||||
成本页的 15 分钟 ledger 分开显示 heating 与 hot-water 三项 variable breakdown;fixed 费只在合同级 summary 按本地自然日计提一次,all-in = variable + fixed。用显式 recompute 来验证测试时间窗时,应手算并核对 Decimal 金额,保留原有 electricity 合同和数字不变。
|
||||
|
||||
在 Config 的 HA Expose 中只开启需要的 source、Meter 与 thermal entities。核对 unit、state class、availability、today reset 和换表后 identity;关闭 toggle 后应用会清理 retained discovery。不要把 source secret、设备 identity 或合同金额放进 HA entity 名称、日志或截图。
|
||||
|
||||
## 安全回滚
|
||||
|
||||
1. 在 UI 禁用 WarmteLink source,确认状态离线且 worker 已停止;保留 channels、bindings、history、contracts 与成本账本。
|
||||
2. 停止 app 后,在 UI 保持 source 禁用;这不会删除 `./data`、数据库、配置或 volumes。需要恢复 WarmteLink 时,确认本地 `.env` 的 stable by-id/GID 后再重新启用 source。
|
||||
3. 确认 DSMR、Modbus、电价、既有 electricity 成本和前端正常。
|
||||
4. schema migration 不应以 production downgrade 回滚;只有经过验证的备份恢复流程才处理灾难恢复,且必须由 operator 在隔离维护窗口执行。
|
||||
|
||||
## 上线验收清单
|
||||
|
||||
- production 与 base+dev Compose 都从本地 `.env` 获取 serial path/GID;app/migration 均为非 root,只有 app 有 `/dev/warmtelink:rw` 和 serial GID,绝无 `m`、root 或 privileged。该 `rw` 仅为 pyserial 的 `O_RDWR` 打开,worker 业务仍只读,且没有真实设备/GID 被记录在仓库。
|
||||
- 下列项目是交付后由用户在备份数据库、可回滚部署、真实 serial 设备和真实 HA 环境执行的人工验收;自动化技术验收不能替代这些观察,也不得把它们伪称为已执行。
|
||||
|
||||
### 用户人工验收记录(交付后填写)
|
||||
|
||||
本模板在交付后由用户填写;它不描述当前宿主环境,也不代表任何项目已通过。每项均填写日期、隔离部署
|
||||
标识、§12 项号、预期观察、实际观察、脱敏日志/截图引用、回滚结果和结果状态。不得记录 stable device id、
|
||||
GID、secret、数据库路径或合同金额;交付时所有项目默认均为“未执行”,不得填造真实环境结果。
|
||||
|
||||
| §12 项号 | 日期 | 隔离部署标识 | 预期观察 | 实际观察 | 脱敏日志/截图引用 | 回滚结果 | 结果状态 |
|
||||
| --- | --- | --- | --- | --- | --- | --- | --- |
|
||||
| 1. 默认 stack 回归 | 待填写 | 待填写 | DSMR、Modbus、电价/电费、前端均无回归 | 待填写 | 待填写 | 待填写 | 未执行 |
|
||||
| 2. stable by-id 与权限 | 待填写 | 待填写 | 非 root、非 privileged、`rw` 无 `m`;pyserial 可打开,worker 无 `write` | 待填写 | 待填写 | 待填写 | 未执行 |
|
||||
| 3. Discover 与隐私 | 待填写 | 待填写 | 两 channel/unit/quality;日志、DB、API、UI 均完成脱敏检查 | 待填写 | 待填写 | 待填写 | 未执行 |
|
||||
| 4. history/拔插重连 | 待填写 | 待填写 | latest、分钟 history、拔插恢复且无重复记录 | 待填写 | 待填写 | 待填写 | 未执行 |
|
||||
| 5. source switch / Meter swap | 待填写 | 待填写 | 两条时间线正确、边界 degraded、HA identity 按设计变化 | 待填写 | 待填写 | 待填写 | 未执行 |
|
||||
| 6. thermal 合同/成本 | 待填写 | 待填写 | 脱敏测试费率手算一致;15 分钟与 01:05 fixed 正确 | 待填写 | 待填写 | 待填写 | 未执行 |
|
||||
| 7. 双 active scope | 待填写 | 待填写 | electricity 数字对照一致;UI/成本/HA 不串 scope | 待填写 | 待填写 | 待填写 | 未执行 |
|
||||
| 8. HA toggles | 待填写 | 待填写 | unit、state class、availability、today、identity、retained cleanup 正确 | 待填写 | 待填写 | 待填写 | 未执行 |
|
||||
| 9. 重启与默认 compose 回滚 | 待填写 | 待填写 | 历史恢复;回默认 compose 后数据完整 | 待填写 | 待填写 | 待填写 | 未执行 |
|
||||
Vendored
+1317
-10
File diff suppressed because it is too large
Load Diff
@@ -65,6 +65,11 @@ const PROFILES_RESPONSE = {
|
||||
heffingskorting: { unit: 'EUR/year' },
|
||||
},
|
||||
},
|
||||
{
|
||||
kind: 'district_heating', label: 'District heating',
|
||||
variable: { heating: { unit: 'EUR/GJ', default: 0 } },
|
||||
standing: { delivery_set: { unit: 'EUR/year', default: 0 } },
|
||||
},
|
||||
],
|
||||
}
|
||||
|
||||
@@ -79,6 +84,21 @@ const CREATED_CONTRACT = {
|
||||
versions: [],
|
||||
}
|
||||
|
||||
const D11_PROFILE_RESPONSE = {
|
||||
profiles: [{
|
||||
kind: 'district_heating', label: 'District heating',
|
||||
variable: {
|
||||
heating: { unit: 'EUR/GJ' }, hot_water_heating: { unit: 'EUR/m³' },
|
||||
hot_water: { unit: 'EUR/m³' }, hot_water_tax: { unit: 'EUR/m³' },
|
||||
},
|
||||
standing: {
|
||||
heating_network: { unit: 'EUR/year' }, metering: { unit: 'EUR/year' },
|
||||
delivery_set: { unit: 'EUR/year' }, hot_water_network: { unit: 'EUR/year' },
|
||||
other: { unit: 'EUR/year' },
|
||||
},
|
||||
}],
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Import component
|
||||
// ---------------------------------------------------------------------------
|
||||
@@ -174,6 +194,49 @@ describe('ContractForm', () => {
|
||||
}, { timeout: 3000 })
|
||||
})
|
||||
|
||||
it('limits a thermal create form to its compatible profile and posts its scope', async () => {
|
||||
const user = userEvent.setup()
|
||||
mockGet.mockResolvedValue({ data: PROFILES_RESPONSE })
|
||||
mockPost.mockResolvedValue({ data: CREATED_CONTRACT })
|
||||
renderWithProviders(<ContractForm scope="thermal" defaultKind="district_heating" onClose={vi.fn()} onSaved={vi.fn()} />)
|
||||
await waitFor(() => expect(screen.getByTestId('contract-field-variable.heating')).toBeInTheDocument())
|
||||
await user.type(screen.getByTestId('contract-name'), 'Heat')
|
||||
await user.click(screen.getByTestId('contract-form-submit'))
|
||||
await waitFor(() => expect(mockPost).toHaveBeenCalledWith('/api/energy/contracts', expect.objectContaining({ body: expect.objectContaining({ kind: 'district_heating', scope: 'thermal' }) })))
|
||||
})
|
||||
|
||||
it('posts all nine D11 values as unrounded Decimal strings, including zero standing fees', async () => {
|
||||
const user = userEvent.setup()
|
||||
mockGet.mockResolvedValue({ data: D11_PROFILE_RESPONSE })
|
||||
mockPost.mockResolvedValue({ data: CREATED_CONTRACT })
|
||||
renderWithProviders(<ContractForm scope="thermal" defaultKind="district_heating" onClose={vi.fn()} onSaved={vi.fn()} />)
|
||||
await waitFor(() => expect(screen.getByTestId('contract-field-variable.heating')).toBeInTheDocument())
|
||||
await user.type(screen.getByTestId('contract-name'), 'Precise heat')
|
||||
const values: Record<string, string> = {
|
||||
'variable.heating': '20.123456789123456789', 'variable.hot_water_heating': '8.200000000000000001',
|
||||
'variable.hot_water': '1.234567890123456789', 'variable.hot_water_tax': '0.456789012345678901',
|
||||
'standing.heating_network': '0', 'standing.metering': '0', 'standing.delivery_set': '0',
|
||||
'standing.hot_water_network': '0', 'standing.other': '0',
|
||||
}
|
||||
for (const [key, value] of Object.entries(values)) {
|
||||
const input = screen.getByTestId(`contract-field-${key}`)
|
||||
await user.clear(input)
|
||||
await user.type(input, value)
|
||||
}
|
||||
await user.click(screen.getByTestId('contract-form-submit'))
|
||||
await waitFor(() => expect(mockPost).toHaveBeenCalled())
|
||||
const body = mockPost.mock.calls[0][1].body
|
||||
expect(body).toMatchObject({ scope: 'thermal', values: {
|
||||
variable: {
|
||||
heating: values['variable.heating'], hot_water_heating: values['variable.hot_water_heating'],
|
||||
hot_water: values['variable.hot_water'], hot_water_tax: values['variable.hot_water_tax'],
|
||||
},
|
||||
standing: {
|
||||
heating_network: '0', metering: '0', delivery_set: '0', hot_water_network: '0', other: '0',
|
||||
},
|
||||
} })
|
||||
})
|
||||
|
||||
it('calls POST /api/energy/contracts/{id}/versions in add-version mode', async () => {
|
||||
const user = userEvent.setup()
|
||||
|
||||
|
||||
@@ -38,6 +38,8 @@ export interface ContractFormProps {
|
||||
contractId?: number
|
||||
/** Existing contract kind (for add-version mode or edit). */
|
||||
defaultKind?: string
|
||||
/** The list/create scope currently selected by the parent. */
|
||||
scope?: 'electricity' | 'thermal'
|
||||
onClose: () => void
|
||||
onSaved: () => void
|
||||
}
|
||||
@@ -64,7 +66,7 @@ interface LeafField {
|
||||
/** Dot-separated path within the section, e.g. "buy.normal" */
|
||||
fieldPath: string
|
||||
unit: string
|
||||
defaultValue?: number
|
||||
defaultValue?: number | string
|
||||
}
|
||||
|
||||
function extractLeafFields(obj: Record<string, unknown>, prefix = ''): LeafField[] {
|
||||
@@ -77,7 +79,9 @@ function extractLeafFields(obj: Record<string, unknown>, prefix = ''): LeafField
|
||||
fields.push({
|
||||
fieldPath: path,
|
||||
unit: val.unit,
|
||||
defaultValue: typeof val.default === 'number' ? val.default : undefined,
|
||||
defaultValue: typeof val.default === 'number' || typeof val.default === 'string'
|
||||
? val.default
|
||||
: undefined,
|
||||
})
|
||||
} else {
|
||||
fields.push(...extractLeafFields(val as Record<string, unknown>, path))
|
||||
@@ -93,6 +97,7 @@ function extractLeafFields(obj: Record<string, unknown>, prefix = ''): LeafField
|
||||
function buildNestedValues(
|
||||
sectionFields: Record<string, LeafField[]>,
|
||||
fieldValues: Record<string, number | string>,
|
||||
decimalStrings: boolean,
|
||||
): Record<string, unknown> {
|
||||
const result: Record<string, unknown> = {}
|
||||
|
||||
@@ -100,7 +105,6 @@ function buildNestedValues(
|
||||
const sectionObj: Record<string, unknown> = {}
|
||||
for (const field of fields) {
|
||||
const raw = fieldValues[`${section}.${field.fieldPath}`]
|
||||
const numVal = typeof raw === 'number' ? raw : parseFloat(String(raw))
|
||||
// Set nested path
|
||||
const parts = field.fieldPath.split('.')
|
||||
let current = sectionObj
|
||||
@@ -108,7 +112,10 @@ function buildNestedValues(
|
||||
if (!(parts[i] in current)) current[parts[i]] = {}
|
||||
current = current[parts[i]] as Record<string, unknown>
|
||||
}
|
||||
current[parts[parts.length - 1]] = isNaN(numVal) ? 0 : numVal
|
||||
// Values are Decimal JSON strings. Do not round-trip user money through JS Number.
|
||||
current[parts[parts.length - 1]] = decimalStrings
|
||||
? (raw === undefined || raw === '' ? '0' : String(raw))
|
||||
: (Number.isFinite(Number(raw)) ? Number(raw) : 0)
|
||||
}
|
||||
result[section] = sectionObj
|
||||
}
|
||||
@@ -130,7 +137,7 @@ function formatLabel(path: string): string {
|
||||
// Component
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export function ContractForm({ contractId, defaultKind, onClose, onSaved }: ContractFormProps) {
|
||||
export function ContractForm({ contractId, defaultKind, scope, onClose, onSaved }: ContractFormProps) {
|
||||
const isAddVersion = contractId != null
|
||||
|
||||
// Profiles query
|
||||
@@ -161,7 +168,9 @@ export function ContractForm({ contractId, defaultKind, onClose, onSaved }: Cont
|
||||
const effectiveKind = isAddVersion ? (defaultKind ?? null) : selectedKind
|
||||
|
||||
// Build profile options from API response
|
||||
const profiles = profilesQuery.data?.profiles ?? []
|
||||
const profiles = (profilesQuery.data?.profiles ?? []).filter((p: Record<string, unknown>) =>
|
||||
scope === 'thermal' ? p.kind === 'district_heating' : p.kind !== 'district_heating',
|
||||
)
|
||||
const profileOptions = profiles.map((p: Record<string, unknown>) => ({
|
||||
value: p.kind as string,
|
||||
label: (p.label as string | undefined) ?? (p.kind as string),
|
||||
@@ -219,8 +228,7 @@ export function ContractForm({ contractId, defaultKind, onClose, onSaved }: Cont
|
||||
cursor = (cursor as Record<string, unknown>)[part]
|
||||
}
|
||||
if (cursor != null && (typeof cursor === 'number' || typeof cursor === 'string')) {
|
||||
const numVal = typeof cursor === 'number' ? cursor : parseFloat(String(cursor))
|
||||
seeded[`${section}.${leaf.fieldPath}`] = isNaN(numVal) ? 0 : numVal
|
||||
seeded[`${section}.${leaf.fieldPath}`] = effectiveKind === 'district_heating' ? String(cursor) : Number(cursor)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -231,6 +239,7 @@ export function ContractForm({ contractId, defaultKind, onClose, onSaved }: Cont
|
||||
contractDetailQuery.isError,
|
||||
contractDetailQuery.data,
|
||||
sectionFields,
|
||||
effectiveKind,
|
||||
])
|
||||
|
||||
// The effective field values: user edits override prefill; prefill is the base.
|
||||
@@ -285,7 +294,7 @@ export function ContractForm({ contractId, defaultKind, onClose, onSaved }: Cont
|
||||
return
|
||||
}
|
||||
|
||||
const values = buildNestedValues(sectionFields, fieldValues)
|
||||
const values = buildNestedValues(sectionFields, fieldValues, effectiveKind === 'district_heating')
|
||||
|
||||
try {
|
||||
// Convert local date string to a naive local-midnight datetime string (no Z).
|
||||
@@ -307,6 +316,7 @@ export function ContractForm({ contractId, defaultKind, onClose, onSaved }: Cont
|
||||
const body = {
|
||||
name: name.trim(),
|
||||
kind: effectiveKind,
|
||||
...(scope ? { scope } : {}),
|
||||
currency,
|
||||
values,
|
||||
...(effectiveFromISO ? { effective_from: effectiveFromISO } : {}),
|
||||
@@ -409,19 +419,21 @@ export function ContractForm({ contractId, defaultKind, onClose, onSaved }: Cont
|
||||
</Title>
|
||||
{fields.map((field) => {
|
||||
const key = `${section}.${field.fieldPath}`
|
||||
const raw = fieldValues[key]
|
||||
const numVal = typeof raw === 'number' ? raw : parseFloat(String(raw))
|
||||
return (
|
||||
<NumberInput
|
||||
return effectiveKind === 'district_heating' ? (
|
||||
<TextInput
|
||||
key={key}
|
||||
label={formatLabel(field.fieldPath)}
|
||||
description={field.unit}
|
||||
value={isNaN(numVal) ? 0 : numVal}
|
||||
onChange={(val) => handleFieldChange(key, val)}
|
||||
decimalScale={6}
|
||||
step={0.001}
|
||||
inputMode="decimal"
|
||||
value={String(fieldValues[key] ?? '0')}
|
||||
onChange={(event) => handleFieldChange(key, event.currentTarget.value)}
|
||||
data-testid={`contract-field-${key}`}
|
||||
/>
|
||||
) : (
|
||||
<NumberInput key={key} label={formatLabel(field.fieldPath)} description={field.unit}
|
||||
value={typeof fieldValues[key] === 'number' ? fieldValues[key] : Number(fieldValues[key] ?? 0)}
|
||||
onChange={(value) => handleFieldChange(key, value)} decimalScale={6} step={0.001}
|
||||
data-testid={`contract-field-${key}`} />
|
||||
)
|
||||
})}
|
||||
</Stack>
|
||||
|
||||
@@ -66,6 +66,11 @@ const INACTIVE_CONTRACT = {
|
||||
updated_at: '2026-06-02T00:00:00Z',
|
||||
}
|
||||
|
||||
const ACTIVE_THERMAL_CONTRACT = {
|
||||
id: 3, name: 'Active Heat Contract', kind: 'district_heating', active: true, currency: 'EUR',
|
||||
created_at: '2026-06-03T00:00:00Z', updated_at: '2026-06-03T00:00:00Z',
|
||||
}
|
||||
|
||||
const PROFILES_RESPONSE = {
|
||||
profiles: [
|
||||
{
|
||||
@@ -202,4 +207,39 @@ describe('ContractManager', () => {
|
||||
expect(screen.getByTestId('contract-form-modal')).toBeInTheDocument()
|
||||
})
|
||||
})
|
||||
|
||||
it('keeps the selector available while thermal data loads and requests each scope separately', async () => {
|
||||
const user = userEvent.setup()
|
||||
mockGet.mockImplementation((path: string, options?: { params?: { query?: { scope?: string } } }) => {
|
||||
if (path === '/api/energy/contracts' && options?.params?.query?.scope === 'thermal') return new Promise(() => {})
|
||||
if (path === '/api/energy/contracts') return Promise.resolve({ data: { items: [ACTIVE_CONTRACT], total: 1 } })
|
||||
return Promise.resolve({ data: PROFILES_RESPONSE })
|
||||
})
|
||||
renderWithProviders(<ContractManager />)
|
||||
await waitFor(() => expect(screen.getByTestId('contracts-scope-selector')).toBeInTheDocument())
|
||||
await user.click(screen.getByText('Thermal'))
|
||||
expect(screen.getByTestId('contracts-loading')).toBeInTheDocument()
|
||||
expect(screen.getByTestId('contracts-scope-selector')).toBeInTheDocument()
|
||||
await user.click(screen.getByText('Electricity'))
|
||||
await waitFor(() => expect(screen.getByTestId('contracts-table')).toBeInTheDocument())
|
||||
expect(mockGet).toHaveBeenCalledWith('/api/energy/contracts', { params: { query: { scope: 'thermal' } } })
|
||||
})
|
||||
|
||||
it('keeps simultaneous active electricity and thermal contracts isolated across repeated switches', async () => {
|
||||
const user = userEvent.setup()
|
||||
mockGet.mockImplementation((_path: string, options?: { params?: { query?: { scope?: string } } }) =>
|
||||
Promise.resolve({ data: { items: options?.params?.query?.scope === 'thermal'
|
||||
? [ACTIVE_THERMAL_CONTRACT] : [ACTIVE_CONTRACT], total: 1 } }),
|
||||
)
|
||||
renderWithProviders(<ContractManager />)
|
||||
await waitFor(() => expect(screen.getByText('My Active Contract')).toBeInTheDocument())
|
||||
for (let i = 0; i < 2; i += 1) {
|
||||
await user.click(screen.getByText('Thermal'))
|
||||
await waitFor(() => expect(screen.getByText('Active Heat Contract')).toBeInTheDocument())
|
||||
expect(screen.queryByText('My Active Contract')).not.toBeInTheDocument()
|
||||
await user.click(screen.getByText('Electricity'))
|
||||
await waitFor(() => expect(screen.getByText('My Active Contract')).toBeInTheDocument())
|
||||
expect(screen.queryByText('Active Heat Contract')).not.toBeInTheDocument()
|
||||
}
|
||||
})
|
||||
})
|
||||
|
||||
@@ -10,6 +10,7 @@
|
||||
*/
|
||||
|
||||
import { useState } from 'react'
|
||||
import { useQuery } from '@tanstack/react-query'
|
||||
import {
|
||||
Table,
|
||||
Button,
|
||||
@@ -24,9 +25,9 @@ import {
|
||||
Modal,
|
||||
Accordion,
|
||||
Code,
|
||||
SegmentedControl,
|
||||
} from '@mantine/core'
|
||||
import {
|
||||
useContracts,
|
||||
useUpdateContract,
|
||||
type ContractResponse,
|
||||
type ContractDetailResponse,
|
||||
@@ -244,7 +245,14 @@ function ContractTable({
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export function ContractManager() {
|
||||
const contractsQuery = useContracts()
|
||||
const [scope, setScope] = useState<'electricity' | 'thermal'>('electricity')
|
||||
const contractsQuery = useQuery({
|
||||
queryKey: ['energy-contracts', scope],
|
||||
queryFn: async () => {
|
||||
const res = await apiClient.GET('/api/energy/contracts', { params: { query: { scope } } })
|
||||
return res.data
|
||||
},
|
||||
})
|
||||
const updateMutation = useUpdateContract()
|
||||
|
||||
const [showCreateForm, setShowCreateForm] = useState(false)
|
||||
@@ -252,6 +260,13 @@ export function ContractManager() {
|
||||
const [historyContract, setHistoryContract] = useState<ContractResponse | null>(null)
|
||||
const [activatingId, setActivatingId] = useState<number | null>(null)
|
||||
|
||||
function handleScopeChange(nextScope: 'electricity' | 'thermal') {
|
||||
setScope(nextScope)
|
||||
setShowCreateForm(false)
|
||||
setAddVersionContract(null)
|
||||
setHistoryContract(null)
|
||||
}
|
||||
|
||||
async function handleActivate(id: number) {
|
||||
setActivatingId(id)
|
||||
try {
|
||||
@@ -265,44 +280,40 @@ export function ContractManager() {
|
||||
// Render states
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
if (contractsQuery.isLoading) {
|
||||
return (
|
||||
<Center py="xl" data-testid="contracts-loading">
|
||||
<Loader />
|
||||
</Center>
|
||||
)
|
||||
}
|
||||
|
||||
if (contractsQuery.isError || !contractsQuery.data) {
|
||||
return (
|
||||
<Alert color="red" data-testid="contracts-load-error">
|
||||
Failed to load contracts. Please refresh.
|
||||
</Alert>
|
||||
)
|
||||
}
|
||||
|
||||
const contracts = contractsQuery.data.items
|
||||
const contracts = contractsQuery.data?.items ?? []
|
||||
|
||||
return (
|
||||
<Stack gap="lg" data-testid="contract-manager">
|
||||
<Group justify="space-between" align="center">
|
||||
<Text fw={500}>Energy Contracts</Text>
|
||||
<Group gap="sm">
|
||||
<Text fw={500}>Energy Contracts</Text>
|
||||
<SegmentedControl
|
||||
value={scope}
|
||||
onChange={(value) => handleScopeChange(value as 'electricity' | 'thermal')}
|
||||
data={[{ label: 'Electricity', value: 'electricity' }, { label: 'Thermal', value: 'thermal' }]}
|
||||
data-testid="contracts-scope-selector"
|
||||
/>
|
||||
</Group>
|
||||
<Button onClick={() => setShowCreateForm(true)} data-testid="contract-new-button">
|
||||
New Contract
|
||||
</Button>
|
||||
</Group>
|
||||
|
||||
<ContractTable
|
||||
{contractsQuery.isLoading && <Center py="xl" data-testid="contracts-loading"><Loader /></Center>}
|
||||
{contractsQuery.isError && <Alert color="red" data-testid="contracts-load-error">Failed to load contracts. Please refresh.</Alert>}
|
||||
{!contractsQuery.isLoading && !contractsQuery.isError && <ContractTable
|
||||
contracts={contracts}
|
||||
onActivate={handleActivate}
|
||||
onAddVersion={(c) => setAddVersionContract(c)}
|
||||
onViewHistory={(c) => setHistoryContract(c)}
|
||||
activatingId={activatingId}
|
||||
/>
|
||||
/>}
|
||||
|
||||
{/* Create new contract */}
|
||||
{showCreateForm && (
|
||||
<ContractForm
|
||||
defaultKind={scope === 'thermal' ? 'district_heating' : undefined}
|
||||
scope={scope}
|
||||
onClose={() => setShowCreateForm(false)}
|
||||
onSaved={() => setShowCreateForm(false)}
|
||||
/>
|
||||
@@ -313,6 +324,7 @@ export function ContractManager() {
|
||||
<ContractForm
|
||||
contractId={addVersionContract.id}
|
||||
defaultKind={addVersionContract.kind}
|
||||
scope={scope}
|
||||
onClose={() => setAddVersionContract(null)}
|
||||
onSaved={() => setAddVersionContract(null)}
|
||||
/>
|
||||
|
||||
@@ -61,14 +61,89 @@ const COST_PERIOD = {
|
||||
|
||||
const SUMMARY = {
|
||||
currency: 'EUR',
|
||||
// Money totals and kWh totals are deliberately distinct so the assertions
|
||||
// below prove the cards read the *_kwh fields, not the monetary ones.
|
||||
metered_import: 10.5,
|
||||
metered_export: 2.3,
|
||||
metered_net: 8.2,
|
||||
metered_import_kwh: 33.3,
|
||||
metered_export_kwh: 44.4,
|
||||
fixed_costs: 5.0,
|
||||
credits: 50.0,
|
||||
total_payable: 12.5,
|
||||
}
|
||||
|
||||
const THERMAL_SUMMARY = {
|
||||
currency: 'EUR', heating: '1.10', hot_water_heating: '2.20', hot_water: '3.30',
|
||||
hot_water_tax: '0.40', variable_subtotal: '7.00', fixed_subtotal: '0.50', all_in: '7.50',
|
||||
period_count: 4, degraded_count: 2,
|
||||
fixed_breakdown: { heating_network: '0.1', metering: '0.1', delivery_set: '0', hot_water_network: '0.1', other: '0.2' },
|
||||
}
|
||||
const ROUNDED_THERMAL_SUMMARY = {
|
||||
...THERMAL_SUMMARY,
|
||||
heating: '20.12344',
|
||||
hot_water_heating: '8.20005',
|
||||
hot_water: '1.99995',
|
||||
hot_water_tax: '0.40000',
|
||||
variable_subtotal: '7.0000',
|
||||
fixed_subtotal: '0.00004',
|
||||
all_in: '9.99995',
|
||||
fixed_breakdown: {
|
||||
heating_network: '100.000000000000000001',
|
||||
metering: '0.0000',
|
||||
delivery_set: '20.20000',
|
||||
hot_water_network: '30.30004',
|
||||
other: '40.40005',
|
||||
},
|
||||
}
|
||||
const THERMAL_VALUES = {
|
||||
variable: {
|
||||
heating: '20.123456789123456789', hot_water_heating: '8.200000000000000001',
|
||||
hot_water: '1.234567890123456789', hot_water_tax: '0.456789012345678901',
|
||||
},
|
||||
standing: {
|
||||
heating_network: '100.000000000000000001', metering: '0', delivery_set: '20.2',
|
||||
hot_water_network: '30.3', other: '40.4',
|
||||
},
|
||||
}
|
||||
const THERMAL_ROW = {
|
||||
commodity: 'heating', period_start: '2026-06-22T10:00:00Z', period_end: '2026-06-22T10:15:00Z',
|
||||
meter_id: 1, source_binding_id: 2, contract_version_id: 99, quantity: '1.2', cost: '0.123456789', currency: 'EUR',
|
||||
cost_breakdown: { heating: '0.123456789' }, pricing_snapshot: THERMAL_VALUES,
|
||||
quality: 'unverifiable', degraded: false, degraded_reason: null,
|
||||
}
|
||||
const ROUNDED_THERMAL_ROW = {
|
||||
...THERMAL_ROW,
|
||||
quantity: '1.23456',
|
||||
cost: '0.12344',
|
||||
cost_breakdown: { heating: '0.12345', hot_water: '0.10000' },
|
||||
}
|
||||
const THERMAL_ROW_OTHER_VERSION = {
|
||||
...THERMAL_ROW, commodity: 'hot_water', period_start: '2026-06-22T10:15:00Z', period_end: '2026-06-22T10:30:00Z',
|
||||
contract_version_id: 100, quantity: '2.3', cost: '4.339506172839506170',
|
||||
cost_breakdown: { hot_water_heating: '1.2', hot_water: '2.8', hot_water_tax: '0.339506172839506170' },
|
||||
pricing_snapshot: { ...THERMAL_VALUES, variable: { ...THERMAL_VALUES.variable, hot_water: '1.234567890123456789' } },
|
||||
}
|
||||
const THERMAL_DEGRADED_MISSING_CONTRACT = {
|
||||
commodity: 'heating', period_start: '2026-06-22T10:30:00Z', period_end: '2026-06-22T10:45:00Z',
|
||||
meter_id: 1, source_binding_id: 2, contract_version_id: null, quantity: '0', cost: '0', currency: 'EUR',
|
||||
cost_breakdown: {}, pricing_snapshot: {}, quality: 'invalid', degraded: true, degraded_reason: 'missing_contract',
|
||||
}
|
||||
const THERMAL_DEGRADED_CROSS_EPOCH = {
|
||||
commodity: 'hot_water', period_start: '2026-06-22T10:45:00Z', period_end: '2026-06-22T11:00:00Z',
|
||||
meter_id: 2, source_binding_id: null, contract_version_id: null, quantity: '0', cost: '0', currency: 'EUR',
|
||||
cost_breakdown: {}, pricing_snapshot: {}, quality: 'invalid', degraded: true, degraded_reason: 'cross_meter_epoch',
|
||||
}
|
||||
const ACTIVE_HEATING_METER = { id: 10, commodity: 'heating', ended_at: null }
|
||||
const ACTIVE_HOT_WATER_METER = { id: 11, commodity: 'hot_water', ended_at: null }
|
||||
const ENDED_HEATING_METER = { id: 9, commodity: 'heating', ended_at: '2026-06-01T00:00:00Z' }
|
||||
const CLOSED_THERMAL_RANGE = {
|
||||
startDate: '2020-01-01',
|
||||
endDate: '2020-01-02',
|
||||
start: '2020-01-01T00:00:00.000Z',
|
||||
end: '2020-01-02T00:00:00.000Z',
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Import component
|
||||
// ---------------------------------------------------------------------------
|
||||
@@ -145,9 +220,14 @@ describe('CostView', () => {
|
||||
expect(screen.getByTestId('summary-import')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
expect(screen.getByTestId('summary-import')).toHaveTextContent('10.500')
|
||||
expect(screen.getByTestId('summary-export')).toHaveTextContent('2.300')
|
||||
// Main figure is energy (kWh), taken from the *_kwh fields.
|
||||
expect(screen.getByTestId('summary-import')).toHaveTextContent('33.300')
|
||||
expect(screen.getByTestId('summary-export')).toHaveTextContent('44.400')
|
||||
expect(screen.getByTestId('summary-total')).toHaveTextContent('12.50')
|
||||
|
||||
// Sub-line carries the monetary equivalent, so money is still visible.
|
||||
expect(screen.getByTestId('summary-import-sub')).toHaveTextContent('10.50 EUR')
|
||||
expect(screen.getByTestId('summary-export-sub')).toHaveTextContent('2.30 EUR')
|
||||
})
|
||||
|
||||
it('shows recompute confirmation modal on button click', async () => {
|
||||
@@ -176,6 +256,32 @@ describe('CostView', () => {
|
||||
})
|
||||
})
|
||||
|
||||
it('uses the same bottom-aligned toolbar structure in both scopes', async () => {
|
||||
const user = userEvent.setup()
|
||||
mockGet.mockImplementation((path: string) => {
|
||||
if (path === '/api/energy/costs/summary') return Promise.resolve({ data: SUMMARY })
|
||||
if (path === '/api/energy/meter-costs/summary') return Promise.resolve({ data: THERMAL_SUMMARY })
|
||||
return Promise.resolve({ data: { items: [], total: 0 } })
|
||||
})
|
||||
|
||||
renderWithProviders(<CostView />)
|
||||
|
||||
const electricityToolbar = screen.getByTestId('cost-toolbar')
|
||||
expect(electricityToolbar).toContainElement(screen.getByTestId('costs-scope-selector'))
|
||||
expect(electricityToolbar).toContainElement(screen.getByTestId('cost-range-control'))
|
||||
expect(electricityToolbar).toContainElement(screen.getByTestId('cost-recompute-button'))
|
||||
expect(screen.getByTestId('cost-recompute-button')).toHaveStyle({ marginLeft: 'auto' })
|
||||
|
||||
await user.click(screen.getByTestId('costs-scope-selector'))
|
||||
await user.click(screen.getByText('Thermal'))
|
||||
await waitFor(() => expect(screen.getByTestId('thermal-cost-toolbar')).toBeInTheDocument())
|
||||
const thermalToolbar = screen.getByTestId('thermal-cost-toolbar')
|
||||
expect(thermalToolbar).toContainElement(screen.getByTestId('costs-scope-selector'))
|
||||
expect(thermalToolbar).toContainElement(screen.getByTestId('thermal-cost-range-control'))
|
||||
expect(thermalToolbar).toContainElement(screen.getByTestId('thermal-recompute-button'))
|
||||
expect(screen.getByTestId('thermal-recompute-button')).toHaveStyle({ marginLeft: 'auto' })
|
||||
})
|
||||
|
||||
it('calls recompute mutation when confirmed', async () => {
|
||||
const user = userEvent.setup()
|
||||
|
||||
@@ -211,4 +317,180 @@ describe('CostView', () => {
|
||||
)
|
||||
})
|
||||
})
|
||||
|
||||
it('audits thermal rows and recomputes only a closed UTC quarter through the typed client', async () => {
|
||||
const user = userEvent.setup()
|
||||
mockGet.mockImplementation((path: string) => {
|
||||
if (path === '/api/energy/meter-costs') return Promise.resolve({ data: { items: [THERMAL_ROW, THERMAL_ROW_OTHER_VERSION, THERMAL_DEGRADED_MISSING_CONTRACT, THERMAL_DEGRADED_CROSS_EPOCH], total: 4 } })
|
||||
if (path === '/api/energy/meter-costs/summary') return Promise.resolve({ data: THERMAL_SUMMARY })
|
||||
if (path === '/api/energy/costs') return Promise.resolve({ data: { items: [], total: 0 } })
|
||||
if (path === '/api/energy/costs/summary') return Promise.resolve({ data: SUMMARY })
|
||||
return Promise.resolve({ data: null })
|
||||
})
|
||||
mockPost.mockResolvedValue({ data: { processed: 1, normal: 0, degraded: 1 } })
|
||||
renderWithProviders(<CostView />)
|
||||
await user.click(screen.getByTestId('costs-scope-selector'))
|
||||
await user.click(screen.getByText('Thermal'))
|
||||
await waitFor(() => expect(screen.getByTestId('thermal-cost-summary')).toBeInTheDocument())
|
||||
expect(screen.getByTestId('thermal-period-count')).toHaveTextContent('4 periods; 2 degraded')
|
||||
expect(screen.getByTestId('thermal-summary-degraded')).toHaveTextContent('Expand a row to see its recorded reason')
|
||||
expect(screen.queryByTestId('thermal-degraded-0')).not.toBeInTheDocument()
|
||||
expect(screen.queryByTestId('thermal-degraded-1')).not.toBeInTheDocument()
|
||||
expect(screen.getByTestId('thermal-degraded-2')).toHaveTextContent('missing_contract')
|
||||
expect(screen.getByTestId('thermal-degraded-3')).toHaveTextContent('cross_meter_epoch')
|
||||
await user.click(screen.getByTestId('thermal-cost-expand-0'))
|
||||
expect(screen.getByTestId('thermal-cost-audit-0')).toHaveTextContent('Contract version: 99')
|
||||
expect(screen.getByTestId('thermal-cost-audit-0')).toHaveTextContent('20.123456789123456789')
|
||||
expect(screen.getByTestId('thermal-cost-audit-0')).toHaveTextContent('heating_network')
|
||||
await user.click(screen.getByTestId('thermal-cost-expand-1'))
|
||||
expect(screen.getByTestId('thermal-cost-audit-1')).toHaveTextContent('Contract version: 100')
|
||||
expect(screen.getByTestId('thermal-cost-audit-1')).toHaveTextContent('1.234567890123456789')
|
||||
expect(screen.getByTestId('thermal-cost-audit-1')).toHaveTextContent('hot_water_tax')
|
||||
await user.click(screen.getByTestId('thermal-cost-expand-2'))
|
||||
expect(screen.getByTestId('thermal-cost-audit-2')).toHaveTextContent('Contract version: none')
|
||||
expect(screen.getByTestId('thermal-cost-audit-2')).toHaveTextContent('Pricing snapshot: {}')
|
||||
expect(screen.getByTestId('thermal-fixed-breakdown')).toHaveTextContent('summary only')
|
||||
expect(screen.getAllByText(/Fixed subtotal|All-in total/)).toHaveLength(2)
|
||||
expect(screen.getByTestId('thermal-costs-table')).not.toHaveTextContent('Fixed')
|
||||
await user.click(screen.getByTestId('thermal-cost-range-control'))
|
||||
await user.click(screen.getByText('Custom'))
|
||||
expect(screen.getByTestId('thermal-recompute-button')).toBeDisabled()
|
||||
await user.type(screen.getByTestId('thermal-cost-custom-start'), CLOSED_THERMAL_RANGE.startDate)
|
||||
await user.type(screen.getByTestId('thermal-cost-custom-end'), CLOSED_THERMAL_RANGE.endDate)
|
||||
await waitFor(() => expect(screen.getByTestId('thermal-recompute-button')).toBeEnabled())
|
||||
await user.click(screen.getByTestId('thermal-recompute-button'))
|
||||
expect(screen.getByTestId('thermal-recompute-confirm-modal')).toHaveTextContent('closed 15-minute')
|
||||
await user.click(screen.getByTestId('thermal-recompute-confirm'))
|
||||
await waitFor(() => expect(mockPost).toHaveBeenCalledWith(
|
||||
'/api/energy/meter-costs/recompute',
|
||||
{ params: { query: { scope: 'thermal', start: CLOSED_THERMAL_RANGE.start, end: CLOSED_THERMAL_RANGE.end } } },
|
||||
))
|
||||
expect(new Date(CLOSED_THERMAL_RANGE.end).getUTCMinutes() % 15).toBe(0)
|
||||
})
|
||||
|
||||
it('keeps the thermal confirmation cancellable and surfaces a 422 recompute failure', async () => {
|
||||
const user = userEvent.setup()
|
||||
mockGet.mockImplementation((path: string) => {
|
||||
if (path === '/api/energy/meter-costs') return Promise.resolve({ data: { items: [THERMAL_ROW], total: 1 } })
|
||||
if (path === '/api/energy/meter-costs/summary') return Promise.resolve({ data: THERMAL_SUMMARY })
|
||||
if (path === '/api/energy/costs/summary') return Promise.resolve({ data: SUMMARY })
|
||||
return Promise.resolve({ data: { items: [], total: 0 } })
|
||||
})
|
||||
mockPost.mockRejectedValue(new Error('422'))
|
||||
renderWithProviders(<CostView />)
|
||||
await user.click(screen.getByTestId('costs-scope-selector')); await user.click(screen.getByText('Thermal'))
|
||||
await waitFor(() => expect(screen.getByTestId('thermal-recompute-button')).toBeInTheDocument())
|
||||
await user.click(screen.getByTestId('thermal-cost-range-control')); await user.click(screen.getByText('Custom'))
|
||||
expect(screen.getByTestId('thermal-recompute-button')).toBeDisabled()
|
||||
await user.type(screen.getByTestId('thermal-cost-custom-start'), CLOSED_THERMAL_RANGE.startDate)
|
||||
await user.type(screen.getByTestId('thermal-cost-custom-end'), CLOSED_THERMAL_RANGE.endDate)
|
||||
await waitFor(() => expect(screen.getByTestId('thermal-recompute-button')).toBeEnabled())
|
||||
await user.click(screen.getByTestId('thermal-recompute-button')); await user.click(screen.getByTestId('thermal-recompute-cancel'))
|
||||
expect(screen.queryByTestId('thermal-recompute-confirm-modal')).not.toBeInTheDocument()
|
||||
await user.click(screen.getByTestId('thermal-recompute-button')); await user.click(screen.getByTestId('thermal-recompute-confirm'))
|
||||
await waitFor(() => expect(mockPost).toHaveBeenCalledWith(
|
||||
'/api/energy/meter-costs/recompute',
|
||||
{ params: { query: { scope: 'thermal', start: CLOSED_THERMAL_RANGE.start, end: CLOSED_THERMAL_RANGE.end } } },
|
||||
))
|
||||
await waitFor(() => expect(screen.getByTestId('thermal-recompute-error')).toBeInTheDocument())
|
||||
})
|
||||
|
||||
it.each([
|
||||
['only heating', [ACTIVE_HEATING_METER], 'hot-water meter is not configured', 'Not configured'],
|
||||
['only hot water', [ACTIVE_HOT_WATER_METER], 'heating meter is not configured', 'Not configured'],
|
||||
['both current meters', [ACTIVE_HEATING_METER, ACTIVE_HOT_WATER_METER], null, '1.1'],
|
||||
['a replaced heating meter plus its current epoch', [ENDED_HEATING_METER, ACTIVE_HEATING_METER, ACTIVE_HOT_WATER_METER], null, '1.1'],
|
||||
])('uses active meter epochs for %s without treating zero amounts as missing', async (_name, meterItems, missingText, heatingValue) => {
|
||||
mockGet.mockImplementation((path: string) => {
|
||||
if (path === '/api/energy/meters') return Promise.resolve({ data: { items: meterItems, total: meterItems.length } })
|
||||
if (path === '/api/energy/meter-costs') return Promise.resolve({ data: { items: [THERMAL_ROW], total: 1 } })
|
||||
if (path === '/api/energy/meter-costs/summary') return Promise.resolve({ data: { ...THERMAL_SUMMARY, heating: heatingValue === 'Not configured' ? '0' : THERMAL_SUMMARY.heating } })
|
||||
if (path === '/api/energy/costs/summary') return Promise.resolve({ data: SUMMARY })
|
||||
return Promise.resolve({ data: { items: [], total: 0 } })
|
||||
})
|
||||
const user = userEvent.setup()
|
||||
renderWithProviders(<CostView />)
|
||||
await user.click(screen.getByTestId('costs-scope-selector'))
|
||||
await user.click(screen.getByText('Thermal'))
|
||||
await waitFor(() => expect(screen.getByTestId('thermal-cost-summary')).toBeInTheDocument())
|
||||
if (missingText) {
|
||||
expect(screen.getByTestId('thermal-missing-current-meter')).toHaveTextContent(missingText)
|
||||
expect(screen.getByTestId('thermal-cost-summary')).toHaveTextContent('Not configured')
|
||||
} else {
|
||||
expect(screen.queryByTestId('thermal-missing-current-meter')).not.toBeInTheDocument()
|
||||
expect(screen.getByTestId('thermal-cost-summary')).toHaveTextContent(heatingValue)
|
||||
}
|
||||
})
|
||||
|
||||
it('rounds ordinary thermal Decimal strings without changing the audit snapshot', async () => {
|
||||
const user = userEvent.setup()
|
||||
mockGet.mockImplementation((path: string) => {
|
||||
if (path === '/api/energy/meters') {
|
||||
return Promise.resolve({ data: { items: [ACTIVE_HEATING_METER, ACTIVE_HOT_WATER_METER], total: 2 } })
|
||||
}
|
||||
if (path === '/api/energy/meter-costs') {
|
||||
return Promise.resolve({ data: { items: [ROUNDED_THERMAL_ROW], total: 1 } })
|
||||
}
|
||||
if (path === '/api/energy/meter-costs/summary') {
|
||||
return Promise.resolve({ data: ROUNDED_THERMAL_SUMMARY })
|
||||
}
|
||||
if (path === '/api/energy/costs/summary') return Promise.resolve({ data: SUMMARY })
|
||||
return Promise.resolve({ data: { items: [], total: 0 } })
|
||||
})
|
||||
|
||||
renderWithProviders(<CostView />)
|
||||
await user.click(screen.getByTestId('costs-scope-selector'))
|
||||
await user.click(screen.getByText('Thermal'))
|
||||
await waitFor(() => expect(screen.getByTestId('thermal-cost-summary')).toBeInTheDocument())
|
||||
|
||||
expect(screen.getByTestId('thermal-cost-summary')).toHaveTextContent('Heating20.1234')
|
||||
expect(screen.getByTestId('thermal-cost-summary')).toHaveTextContent('Hot-water heating8.2001')
|
||||
expect(screen.getByTestId('thermal-cost-summary')).toHaveTextContent('Hot water2')
|
||||
expect(screen.getByTestId('thermal-cost-summary')).toHaveTextContent('Hot-water tax0.4')
|
||||
expect(screen.getByTestId('thermal-cost-summary')).toHaveTextContent('Variable subtotal7')
|
||||
expect(screen.getByTestId('thermal-cost-summary')).toHaveTextContent('Fixed subtotal0')
|
||||
expect(screen.getByTestId('thermal-cost-summary')).toHaveTextContent('All-in total10')
|
||||
expect(screen.getByTestId('thermal-fixed-breakdown')).toHaveTextContent('heating_network 100')
|
||||
expect(screen.getByTestId('thermal-fixed-breakdown')).toHaveTextContent('other 40.4001')
|
||||
expect(screen.getByTestId('thermal-cost-row-0')).toHaveTextContent('1.2346')
|
||||
expect(screen.getByTestId('thermal-cost-row-0')).toHaveTextContent('0.1234 EUR')
|
||||
expect(screen.getByTestId('thermal-cost-row-0')).toHaveTextContent('heating 0.1235, hot_water 0.1')
|
||||
|
||||
await user.click(screen.getByTestId('thermal-cost-expand-0'))
|
||||
expect(screen.getByTestId('thermal-cost-audit-0')).toHaveTextContent('20.123456789123456789')
|
||||
})
|
||||
|
||||
it('paginates the complete thermal ledger and resets offset when its range or scope changes', async () => {
|
||||
const user = userEvent.setup()
|
||||
const lastRow = { ...THERMAL_ROW, period_start: '2026-06-22T12:00:00Z', quantity: '501' }
|
||||
mockGet.mockImplementation((path: string, options?: { params?: { query?: { offset?: number } } }) => {
|
||||
if (path === '/api/energy/meters') return Promise.resolve({ data: { items: [ACTIVE_HEATING_METER, ACTIVE_HOT_WATER_METER], total: 2 } })
|
||||
if (path === '/api/energy/meter-costs') {
|
||||
const offset = options?.params?.query?.offset ?? 0
|
||||
return Promise.resolve({ data: offset === 0 ? { items: Array.from({ length: 500 }, () => THERMAL_ROW), total: 501 } : { items: [lastRow], total: 501 } })
|
||||
}
|
||||
if (path === '/api/energy/meter-costs/summary') return Promise.resolve({ data: THERMAL_SUMMARY })
|
||||
if (path === '/api/energy/costs/summary') return Promise.resolve({ data: SUMMARY })
|
||||
return Promise.resolve({ data: { items: [], total: 0 } })
|
||||
})
|
||||
renderWithProviders(<CostView />)
|
||||
await user.click(screen.getByTestId('costs-scope-selector'))
|
||||
await user.click(screen.getByText('Thermal'))
|
||||
await waitFor(() => expect(screen.getByTestId('thermal-ledger-count')).toHaveTextContent('Showing 1-500 of 501'))
|
||||
expect(screen.getByTestId('thermal-ledger-prev')).toBeDisabled()
|
||||
expect(screen.getByTestId('thermal-ledger-next')).toBeEnabled()
|
||||
await user.click(screen.getByTestId('thermal-ledger-next'))
|
||||
await waitFor(() => expect(screen.getByTestId('thermal-ledger-count')).toHaveTextContent('Showing 501-501 of 501'))
|
||||
expect(screen.getByTestId('thermal-ledger-prev')).toBeEnabled()
|
||||
expect(screen.getByTestId('thermal-ledger-next')).toBeDisabled()
|
||||
expect(mockGet).toHaveBeenCalledWith('/api/energy/meter-costs', expect.objectContaining({ params: { query: expect.objectContaining({ scope: 'thermal', offset: 500, limit: 500 }) } }))
|
||||
await user.click(screen.getByTestId('thermal-cost-range-control'))
|
||||
await user.click(screen.getByText('This month'))
|
||||
await waitFor(() => expect(screen.getByTestId('thermal-ledger-count')).toHaveTextContent('Showing 1-500 of 501'))
|
||||
await user.click(screen.getByTestId('costs-scope-selector'))
|
||||
await user.click(screen.getByText('Electricity'))
|
||||
await user.click(screen.getByTestId('costs-scope-selector'))
|
||||
await user.click(screen.getByText('Thermal'))
|
||||
await waitFor(() => expect(screen.getByTestId('thermal-ledger-count')).toHaveTextContent('Showing 1-500 of 501'))
|
||||
})
|
||||
})
|
||||
|
||||
@@ -13,6 +13,7 @@
|
||||
*/
|
||||
|
||||
import { useState } from 'react'
|
||||
import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query'
|
||||
import {
|
||||
Stack,
|
||||
Text,
|
||||
@@ -43,6 +44,7 @@ import {
|
||||
} from 'recharts'
|
||||
import { useEnergyCosts, useEnergyCostSummary, useRecomputeCosts } from './hooks'
|
||||
import { formatLocalTime } from '../utils/datetime'
|
||||
import apiClient from '../api/client'
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Cost limit — prevent accidental full-table pulls
|
||||
@@ -77,10 +79,12 @@ function getThisMonthRange(): { start: string; end: string } {
|
||||
interface SummaryCardProps {
|
||||
label: string
|
||||
value: string
|
||||
/** Optional secondary line, e.g. the monetary equivalent of an energy figure. */
|
||||
sub?: string
|
||||
testId?: string
|
||||
}
|
||||
|
||||
function SummaryCard({ label, value, testId }: SummaryCardProps) {
|
||||
function SummaryCard({ label, value, sub, testId }: SummaryCardProps) {
|
||||
return (
|
||||
<Paper withBorder p="sm" data-testid={testId}>
|
||||
<Stack gap={4}>
|
||||
@@ -90,6 +94,11 @@ function SummaryCard({ label, value, testId }: SummaryCardProps) {
|
||||
<Text fw={600} size="lg">
|
||||
{value}
|
||||
</Text>
|
||||
{sub !== undefined && (
|
||||
<Text size="xs" c="dimmed" data-testid={testId ? `${testId}-sub` : undefined}>
|
||||
{sub}
|
||||
</Text>
|
||||
)}
|
||||
</Stack>
|
||||
</Paper>
|
||||
)
|
||||
@@ -101,7 +110,60 @@ function SummaryCard({ label, value, testId }: SummaryCardProps) {
|
||||
|
||||
type RangePreset = 'today' | 'month' | 'custom'
|
||||
|
||||
/** Format an API Decimal string without converting it through binary floating point. */
|
||||
function formatDecimal(value: string): string {
|
||||
const negative = value.startsWith('-')
|
||||
const unsigned = negative ? value.slice(1) : value
|
||||
const [rawWhole = '0', rawFraction = ''] = unsigned.split('.')
|
||||
let whole = rawWhole.replace(/^0+(?=\d)/, '') || '0'
|
||||
let fraction = rawFraction.slice(0, 4)
|
||||
|
||||
if (rawFraction.length > 4 && rawFraction[4] >= '5') {
|
||||
const digits = '0123456789'
|
||||
const fractionDigits = fraction.split('')
|
||||
let carry = true
|
||||
for (let index = fractionDigits.length - 1; index >= 0 && carry; index -= 1) {
|
||||
const digit = fractionDigits[index]
|
||||
if (digit === '9') {
|
||||
fractionDigits[index] = '0'
|
||||
} else {
|
||||
fractionDigits[index] = digits[digits.indexOf(digit) + 1]
|
||||
carry = false
|
||||
}
|
||||
}
|
||||
fraction = fractionDigits.join('')
|
||||
|
||||
if (carry) {
|
||||
const wholeDigits = whole.split('')
|
||||
for (let index = wholeDigits.length - 1; index >= 0 && carry; index -= 1) {
|
||||
const digit = wholeDigits[index]
|
||||
if (digit === '9') {
|
||||
wholeDigits[index] = '0'
|
||||
} else {
|
||||
wholeDigits[index] = digits[digits.indexOf(digit) + 1]
|
||||
carry = false
|
||||
}
|
||||
}
|
||||
whole = `${carry ? '1' : ''}${wholeDigits.join('')}`
|
||||
}
|
||||
}
|
||||
|
||||
fraction = fraction.replace(/0+$/, '')
|
||||
const formatted = fraction ? `${whole}.${fraction}` : whole
|
||||
return negative && formatted !== '0' ? `-${formatted}` : formatted
|
||||
}
|
||||
|
||||
export function CostView() {
|
||||
const [scope, setScope] = useState<'electricity' | 'thermal'>('electricity')
|
||||
if (scope === 'thermal') return <ThermalCostView onScopeChange={setScope} />
|
||||
return <ElectricityCostView onScopeChange={setScope} />
|
||||
}
|
||||
|
||||
function ScopeSelector({ scope, onScopeChange }: { scope: 'electricity' | 'thermal'; onScopeChange: (scope: 'electricity' | 'thermal') => void }) {
|
||||
return <SegmentedControl value={scope} onChange={(value) => onScopeChange(value as 'electricity' | 'thermal')} data={[{ label: 'Electricity', value: 'electricity' }, { label: 'Thermal', value: 'thermal' }]} data-testid="costs-scope-selector" />
|
||||
}
|
||||
|
||||
function ElectricityCostView({ onScopeChange }: { onScopeChange: (scope: 'electricity' | 'thermal') => void }) {
|
||||
const [rangePreset, setRangePreset] = useState<RangePreset>('today')
|
||||
// Date strings in YYYY-MM-DD format for custom range
|
||||
const [customStartStr, setCustomStartStr] = useState('')
|
||||
@@ -136,7 +198,8 @@ export function CostView() {
|
||||
return (
|
||||
<Stack gap="lg" data-testid="cost-view">
|
||||
{/* Date range selector */}
|
||||
<Group align="flex-start" gap="md" wrap="wrap">
|
||||
<Group align="flex-end" gap="md" wrap="wrap" data-testid="cost-toolbar">
|
||||
<ScopeSelector scope="electricity" onScopeChange={onScopeChange} />
|
||||
<Stack gap="xs">
|
||||
<Text size="sm" fw={500}>
|
||||
Date range
|
||||
@@ -172,18 +235,17 @@ export function CostView() {
|
||||
</Group>
|
||||
)}
|
||||
|
||||
<Group gap="sm" style={{ marginLeft: 'auto' }} align="flex-end">
|
||||
<Button
|
||||
variant="outline"
|
||||
color="orange"
|
||||
size="sm"
|
||||
onClick={() => setShowRecomputeConfirm(true)}
|
||||
loading={recomputeMutation.isPending}
|
||||
data-testid="cost-recompute-button"
|
||||
>
|
||||
Recompute
|
||||
</Button>
|
||||
</Group>
|
||||
<Button
|
||||
variant="outline"
|
||||
color="orange"
|
||||
size="sm"
|
||||
onClick={() => setShowRecomputeConfirm(true)}
|
||||
loading={recomputeMutation.isPending}
|
||||
style={{ marginLeft: 'auto' }}
|
||||
data-testid="cost-recompute-button"
|
||||
>
|
||||
Recompute
|
||||
</Button>
|
||||
</Group>
|
||||
|
||||
{/* Summary cards */}
|
||||
@@ -207,12 +269,14 @@ export function CostView() {
|
||||
<SimpleGrid cols={{ base: 2, sm: 3 }} spacing="sm" data-testid="cost-summary">
|
||||
<SummaryCard
|
||||
label="Import (kWh)"
|
||||
value={summaryQuery.data.metered_import.toFixed(3)}
|
||||
value={summaryQuery.data.metered_import_kwh.toFixed(3)}
|
||||
sub={`${summaryQuery.data.metered_import.toFixed(2)} ${currency}`}
|
||||
testId="summary-import"
|
||||
/>
|
||||
<SummaryCard
|
||||
label="Export (kWh)"
|
||||
value={summaryQuery.data.metered_export.toFixed(3)}
|
||||
value={summaryQuery.data.metered_export_kwh.toFixed(3)}
|
||||
sub={`${summaryQuery.data.metered_export.toFixed(2)} ${currency}`}
|
||||
testId="summary-export"
|
||||
/>
|
||||
<SummaryCard
|
||||
@@ -401,3 +465,76 @@ export function CostView() {
|
||||
</Stack>
|
||||
)
|
||||
}
|
||||
|
||||
function ThermalCostView({ onScopeChange }: { onScopeChange: (scope: 'electricity' | 'thermal') => void }) {
|
||||
const [rangePreset, setRangePreset] = useState<RangePreset>('today')
|
||||
const [customStartStr, setCustomStartStr] = useState('')
|
||||
const [customEndStr, setCustomEndStr] = useState('')
|
||||
const [showConfirm, setShowConfirm] = useState(false)
|
||||
const [recomputeError, setRecomputeError] = useState<string | null>(null)
|
||||
const [recomputeSuccess, setRecomputeSuccess] = useState<string | null>(null)
|
||||
const [expandedRows, setExpandedRows] = useState<Set<number>>(() => new Set())
|
||||
const [ledgerOffset, setLedgerOffset] = useState(0)
|
||||
const { start, end } = (() => {
|
||||
if (rangePreset === 'today') return getTodayRange()
|
||||
if (rangePreset === 'month') return getThisMonthRange()
|
||||
return {
|
||||
start: customStartStr ? new Date(customStartStr).toISOString() : undefined,
|
||||
end: customEndStr ? new Date(customEndStr).toISOString() : undefined,
|
||||
}
|
||||
})()
|
||||
// The server only accepts complete UTC quarters. Never send a future end.
|
||||
const closedEnd = (() => {
|
||||
const now = new Date()
|
||||
now.setUTCMinutes(Math.floor(now.getUTCMinutes() / 15) * 15, 0, 0)
|
||||
const selectedEnd = end ? new Date(end) : now
|
||||
return new Date(Math.min(selectedEnd.getTime(), now.getTime())).toISOString()
|
||||
})()
|
||||
const recomputeStart = start
|
||||
const recomputeAvailable = !!recomputeStart && new Date(recomputeStart) < new Date(closedEnd)
|
||||
const qc = useQueryClient()
|
||||
const resetLedgerPage = () => {
|
||||
setLedgerOffset(0)
|
||||
setExpandedRows(new Set())
|
||||
}
|
||||
const rows = useQuery({ queryKey: ['meter-costs', 'thermal', start, end, ledgerOffset], queryFn: async () => {
|
||||
const result = await apiClient.GET('/api/energy/meter-costs', { params: { query: { scope: 'thermal', start, end, limit: COSTS_MAX_LIMIT, offset: ledgerOffset } } })
|
||||
return result.data
|
||||
} })
|
||||
const meters = useQuery({ queryKey: ['energy-meters', 'thermal'], queryFn: async () => {
|
||||
const result = await apiClient.GET('/api/energy/meters')
|
||||
return result.data
|
||||
} })
|
||||
const summary = useQuery({ queryKey: ['meter-cost-summary', 'thermal', start, end], queryFn: async () => {
|
||||
const result = await apiClient.GET('/api/energy/meter-costs/summary', { params: { query: { scope: 'thermal', start, end } } })
|
||||
return result.data
|
||||
} })
|
||||
const recompute = useMutation({ mutationFn: () => apiClient.POST('/api/energy/meter-costs/recompute', { params: { query: { scope: 'thermal', start: recomputeStart!, end: closedEnd } } }), onSuccess: (result) => {
|
||||
void qc.invalidateQueries({ queryKey: ['meter-costs', 'thermal'] }); void qc.invalidateQueries({ queryKey: ['meter-cost-summary', 'thermal'] })
|
||||
setRecomputeSuccess(`Recomputed ${result.data?.processed ?? 0} closed periods.`)
|
||||
} })
|
||||
const currency = summary.data?.currency ?? rows.data?.items[0]?.currency ?? 'EUR'
|
||||
const fixed = summary.data?.fixed_breakdown
|
||||
const hasCurrentHeatingMeter = meters.data?.items.some((meter) => meter.commodity === 'heating' && meter.ended_at === null)
|
||||
const hasCurrentHotWaterMeter = meters.data?.items.some((meter) => meter.commodity === 'hot_water' && meter.ended_at === null)
|
||||
const missingCurrentMeters = [
|
||||
...(hasCurrentHeatingMeter === false ? ['heating'] : []),
|
||||
...(hasCurrentHotWaterMeter === false ? ['hot-water'] : []),
|
||||
]
|
||||
const totalRows = rows.data?.total ?? 0
|
||||
const shownStart = totalRows === 0 ? 0 : ledgerOffset + 1
|
||||
const shownEnd = Math.min(ledgerOffset + (rows.data?.items.length ?? 0), totalRows)
|
||||
return <Stack gap="lg" data-testid="thermal-cost-view">
|
||||
<Group align="flex-end" gap="md" wrap="wrap" data-testid="thermal-cost-toolbar"><ScopeSelector scope="thermal" onScopeChange={onScopeChange} /><Stack gap="xs"><Text size="sm" fw={500}>Date range</Text><SegmentedControl value={rangePreset} onChange={(value) => { resetLedgerPage(); setRangePreset(value as RangePreset) }} data={[{ label: 'Today', value: 'today' }, { label: 'This month', value: 'month' }, { label: 'Custom', value: 'custom' }]} data-testid="thermal-cost-range-control" /></Stack>{rangePreset === 'custom' && <Group gap="sm" align="flex-end"><TextInput label="From" type="date" value={customStartStr} onChange={(event) => { resetLedgerPage(); setCustomStartStr(event.currentTarget.value) }} data-testid="thermal-cost-custom-start" /><TextInput label="To" type="date" value={customEndStr} onChange={(event) => { resetLedgerPage(); setCustomEndStr(event.currentTarget.value) }} data-testid="thermal-cost-custom-end" /></Group>}<Button variant="outline" color="orange" onClick={() => { setRecomputeError(null); setRecomputeSuccess(null); setShowConfirm(true) }} disabled={!recomputeAvailable} style={{ marginLeft: 'auto' }} data-testid="thermal-recompute-button">Recompute</Button></Group>
|
||||
{(rows.isLoading || summary.isLoading) && <Center><Loader size="sm" /></Center>}
|
||||
{(rows.isError || summary.isError) && <Alert color="red">Failed to load thermal costs.</Alert>}
|
||||
{recomputeError && <Alert color="red" data-testid="thermal-recompute-error">{recomputeError}</Alert>}
|
||||
{recomputeSuccess && <Alert color="green" data-testid="thermal-recompute-success">{recomputeSuccess}</Alert>}
|
||||
{summary.data && <Stack gap="xs" data-testid="thermal-cost-summary"><Title order={6}>{rangePreset === 'today' ? 'Today' : rangePreset === 'month' ? 'This month' : 'Custom range'} ({currency})</Title><Text size="sm" data-testid="thermal-cost-range">{start ?? 'Select a start date'} — {end ?? 'Select an end date'}</Text><SimpleGrid cols={{ base: 2, sm: 3 }}>
|
||||
<SummaryCard label="Heating" value={hasCurrentHeatingMeter === false ? 'Not configured' : formatDecimal(summary.data.heating)} /><SummaryCard label="Hot-water heating" value={hasCurrentHotWaterMeter === false ? 'Not configured' : formatDecimal(summary.data.hot_water_heating)} /><SummaryCard label="Hot water" value={hasCurrentHotWaterMeter === false ? 'Not configured' : formatDecimal(summary.data.hot_water)} /><SummaryCard label="Hot-water tax" value={hasCurrentHotWaterMeter === false ? 'Not configured' : formatDecimal(summary.data.hot_water_tax)} /><SummaryCard label="Variable subtotal" value={formatDecimal(summary.data.variable_subtotal)} /><SummaryCard label="Fixed subtotal" value={formatDecimal(summary.data.fixed_subtotal)} /><SummaryCard label="All-in total" value={formatDecimal(summary.data.all_in)} />
|
||||
</SimpleGrid>{missingCurrentMeters.length > 0 && <Alert color="yellow" data-testid="thermal-missing-current-meter">Current {missingCurrentMeters.join(' and ')} meter{missingCurrentMeters.length > 1 ? 's are' : ' is'} not configured. Historical ledger rows do not establish a current meter.</Alert>}<Text size="sm" data-testid="thermal-period-count">{summary.data.period_count} periods; {summary.data.degraded_count} degraded</Text>{summary.data.degraded_count > 0 && <Alert color="orange" data-testid="thermal-summary-degraded">Some totals include degraded periods. Expand a row to see its recorded reason.</Alert>}{fixed && <Text size="sm" data-testid="thermal-fixed-breakdown">Fixed once per settled local day (summary only): {Object.entries(fixed).map(([key, value]) => `${key} ${formatDecimal(value)}`).join(' · ')}</Text>}</Stack>}
|
||||
{rows.data?.items.length === 0 && <Alert color="gray" data-testid="thermal-costs-empty">No thermal cost data for this range. Check that heating or hot-water meters are bound and have settled readings.</Alert>}
|
||||
{rows.data && <Stack gap="xs"><Text size="sm" c="dimmed" data-testid="thermal-ledger-count">Showing {shownStart}-{shownEnd} of {totalRows}</Text>{rows.data.items.length > 0 && <ScrollArea><Table striped withTableBorder data-testid="thermal-costs-table"><Table.Thead><Table.Tr><Table.Th>Time</Table.Th><Table.Th>Commodity</Table.Th><Table.Th>Quantity</Table.Th><Table.Th>Cost</Table.Th><Table.Th>Breakdown</Table.Th><Table.Th>Status</Table.Th><Table.Th></Table.Th></Table.Tr></Table.Thead><Table.Tbody>{rows.data.items.flatMap((item, index) => [<Table.Tr key={`${item.commodity}-${item.period_start}`} data-testid={`thermal-cost-row-${index}`}><Table.Td>{formatLocalTime(item.period_start)}</Table.Td><Table.Td>{item.commodity}</Table.Td><Table.Td>{formatDecimal(item.quantity)}</Table.Td><Table.Td>{formatDecimal(item.cost)} {item.currency}</Table.Td><Table.Td>{Object.entries(item.cost_breakdown).map(([key, value]) => `${key} ${formatDecimal(value)}`).join(', ')}</Table.Td><Table.Td>{item.degraded ? <Badge color="orange" data-testid={`thermal-degraded-${index}`}>{item.degraded_reason ?? 'degraded'}</Badge> : 'normal'}</Table.Td><Table.Td><Button size="xs" variant="subtle" onClick={() => setExpandedRows((current) => { const next = new Set(current); if (next.has(index)) next.delete(index); else next.add(index); return next })} data-testid={`thermal-cost-expand-${index}`}>{expandedRows.has(index) ? 'Hide audit' : 'Audit'}</Button></Table.Td></Table.Tr>, ...(expandedRows.has(index) ? [<Table.Tr key={`audit-${item.commodity}-${item.period_start}`} data-testid={`thermal-cost-audit-${index}`}><Table.Td colSpan={7}><Text size="xs">Contract version: {item.contract_version_id ?? 'none'}</Text><Text size="xs">Pricing snapshot: {JSON.stringify(item.pricing_snapshot)}</Text></Table.Td></Table.Tr>] : [])])}</Table.Tbody></Table></ScrollArea>}<Group justify="flex-end"><Button size="xs" variant="default" disabled={ledgerOffset === 0} onClick={() => { setExpandedRows(new Set()); setLedgerOffset((current) => Math.max(0, current - COSTS_MAX_LIMIT)) }} data-testid="thermal-ledger-prev">Previous</Button><Button size="xs" variant="default" disabled={ledgerOffset + (rows.data.items.length ?? 0) >= totalRows} onClick={() => { setExpandedRows(new Set()); setLedgerOffset((current) => current + COSTS_MAX_LIMIT) }} data-testid="thermal-ledger-next">Next</Button></Group></Stack>}
|
||||
{showConfirm && <Modal opened onClose={() => setShowConfirm(false)} title="Recompute thermal costs?" data-testid="thermal-recompute-confirm-modal"><Stack><Text>This explicitly overwrites closed 15-minute thermal ledger rows for {recomputeStart ?? 'the selected start'} — {closedEnd}. Continue?</Text>{!recomputeAvailable && <Alert color="yellow">Select a range containing at least one closed UTC quarter.</Alert>}<Group justify="flex-end"><Button variant="default" onClick={() => setShowConfirm(false)} data-testid="thermal-recompute-cancel">Cancel</Button><Button color="orange" loading={recompute.isPending} disabled={!recomputeAvailable} onClick={async () => { try { await recompute.mutateAsync(); setShowConfirm(false) } catch { setRecomputeError('Failed to recompute thermal costs. Please try again.'); setShowConfirm(false) } }} data-testid="thermal-recompute-confirm">Recompute</Button></Group></Stack></Modal>}
|
||||
</Stack>
|
||||
}
|
||||
|
||||
@@ -50,6 +50,9 @@ describe('DsmrPanel', () => {
|
||||
renderWithProviders(<DsmrPanel />)
|
||||
await waitFor(() => expect(screen.getByTestId('dsmr-empty')).toBeInTheDocument())
|
||||
expect(screen.queryByTestId('dsmr-table')).not.toBeInTheDocument()
|
||||
expect(screen.getByText(/In this DSMR Source, enable or edit the broker, topic, and profile/)).toBeInTheDocument()
|
||||
expect(screen.getByText(/confirm the publisher is sending/)).toBeInTheDocument()
|
||||
expect(screen.queryByText(/Enable DSMR ingest.*Config/)).not.toBeInTheDocument()
|
||||
})
|
||||
|
||||
it('renders the latest telegram as a key/value table; null shown as dash', async () => {
|
||||
@@ -79,4 +82,11 @@ describe('DsmrPanel', () => {
|
||||
renderWithProviders(<DsmrPanel />)
|
||||
await waitFor(() => expect(screen.getByTestId('dsmr-error')).toBeInTheDocument())
|
||||
})
|
||||
|
||||
it('keeps the compatibility endpoint available for DSMR source detail', async () => {
|
||||
mockGet.mockResolvedValue({ data: { found: true, recorded_at: '2026-06-23T12:16:00Z', payload: { tariff: 'low' } } })
|
||||
renderWithProviders(<DsmrPanel />)
|
||||
await waitFor(() => expect(mockGet).toHaveBeenCalledWith('/api/energy/dsmr/latest'))
|
||||
expect(await screen.findByText('Latest DSMR reading (compatibility view)')).toBeInTheDocument()
|
||||
})
|
||||
})
|
||||
|
||||
@@ -47,7 +47,7 @@ export function DsmrPanel() {
|
||||
<Stack gap="md" data-testid="dsmr-panel">
|
||||
<Group justify="space-between" align="center">
|
||||
<div>
|
||||
<Text fw={600}>Latest DSMR reading</Text>
|
||||
<Text fw={600}>Latest DSMR reading (compatibility view)</Text>
|
||||
<Text size="xs" c="dimmed">
|
||||
The most recent parsed telegram persisted to <code>dsmr_reading</code>.
|
||||
</Text>
|
||||
@@ -100,9 +100,9 @@ function DsmrContent({ isLoading, isError, data }: DsmrContentProps) {
|
||||
if (!data.found || !data.payload) {
|
||||
return (
|
||||
<Alert color="gray" data-testid="dsmr-empty">
|
||||
No DSMR data yet. Enable <strong>DSMR ingest</strong> in Config, make sure MQTT
|
||||
is connected, and confirm the DSMR Reader is publishing to the configured topic
|
||||
(default <code>dsmr/json</code>). Rows are stored about once every 10 seconds.
|
||||
No DSMR data yet. In this DSMR Source, enable or edit the broker, topic, and profile
|
||||
configuration, then confirm the publisher is sending to the configured topic (default
|
||||
<code>dsmr/json</code>). Rows are stored about once every 10 seconds.
|
||||
</Alert>
|
||||
)
|
||||
}
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -34,11 +34,17 @@ import {
|
||||
useMeters,
|
||||
useDeclareMeter,
|
||||
useUpdateMeter,
|
||||
useSources,
|
||||
useSourceChannels,
|
||||
useCreateBinding,
|
||||
useCloseBinding,
|
||||
useCloseMeter,
|
||||
useTransferBinding,
|
||||
type MeterResponse,
|
||||
type MeterReason,
|
||||
} from './hooks'
|
||||
import { ApiError } from '../api/client'
|
||||
import { formatLocalDate, parseBackendTimestamp } from '../utils/datetime'
|
||||
import { formatLocalDate, formatLocalDateTime, parseBackendTimestamp } from '../utils/datetime'
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Helpers
|
||||
@@ -73,23 +79,202 @@ function toLocalDateInputString(d: Date): string {
|
||||
return `${y}-${m}-${day}`
|
||||
}
|
||||
|
||||
function toLocalDateTimeInputString(d = new Date()): string {
|
||||
const pad = (value: number) => String(value).padStart(2, '0')
|
||||
return `${toLocalDateInputString(d)}T${pad(d.getHours())}:${pad(d.getMinutes())}`
|
||||
}
|
||||
|
||||
function expectedUnit(commodity: string): string {
|
||||
return ({ electricity: 'kWh', heating: 'GJ', hot_water: 'm³' } as Record<string, string>)[commodity] ?? ''
|
||||
}
|
||||
|
||||
function apiErrorMessage(err: unknown, fallback: string): string {
|
||||
if (err instanceof ApiError) {
|
||||
const body = err.body
|
||||
if (typeof body === 'string') return body
|
||||
if (body && typeof body === 'object' && 'detail' in body) {
|
||||
const detail = (body as { detail?: unknown }).detail
|
||||
if (typeof detail === 'string') return detail
|
||||
if (Array.isArray(detail)) {
|
||||
const messages = detail.map((item) => item && typeof item === 'object' && typeof item.msg === 'string'
|
||||
? item.msg : String(item)).filter(Boolean)
|
||||
if (messages.length) return messages.join('; ')
|
||||
}
|
||||
if (detail != null) return typeof detail === 'object' ? JSON.stringify(detail) : String(detail)
|
||||
}
|
||||
return `${fallback} (error ${err.status}).`
|
||||
}
|
||||
return fallback
|
||||
}
|
||||
|
||||
function isValidLocalDateTime(value: string): boolean {
|
||||
return value.trim() !== '' && !Number.isNaN(new Date(value).getTime())
|
||||
}
|
||||
|
||||
type Eligibility = { eligible: boolean; reason?: string }
|
||||
|
||||
function localInstant(value: string): number | null {
|
||||
const instant = new Date(value).getTime()
|
||||
return Number.isNaN(instant) ? null : instant
|
||||
}
|
||||
|
||||
function intervalsOverlap(start: number, end: number | null, otherStart: number, otherEnd: number | null): boolean {
|
||||
return (end === null || otherStart < end) && (otherEnd === null || start < otherEnd)
|
||||
}
|
||||
|
||||
function channelIntervalEligibility(
|
||||
meters: MeterResponse[], channelUuid: string, startedAt: string, excludeBindingUuids: string[] = [],
|
||||
): Eligibility {
|
||||
const start = localInstant(startedAt)
|
||||
if (start === null) return { eligible: false, reason: 'choose a valid start time first' }
|
||||
const conflicts = meters.flatMap((meter) => (meter.bindings ?? []).filter((binding) =>
|
||||
binding.source_channel_uuid === channelUuid && !excludeBindingUuids.includes(binding.uuid) &&
|
||||
intervalsOverlap(start, null, parseBackendTimestamp(binding.started_at).getTime(), binding.ended_at ? parseBackendTimestamp(binding.ended_at).getTime() : null),
|
||||
))
|
||||
if (!conflicts.length) return { eligible: true }
|
||||
if (conflicts.length > 1) return { eligible: false, reason: 'ambiguous: channel has overlapping binding history; resolve it first' }
|
||||
return conflicts[0].ended_at === null
|
||||
? { eligible: false, reason: 'occupied by an open binding; close or transfer it first' }
|
||||
: { eligible: false, reason: 'overlaps closed binding history; choose a time at or after it ends' }
|
||||
}
|
||||
|
||||
function meterContainsInstant(meter: MeterResponse, instant: number): boolean {
|
||||
const start = parseBackendTimestamp(meter.started_at).getTime()
|
||||
const end = meter.ended_at ? parseBackendTimestamp(meter.ended_at).getTime() : null
|
||||
return instant >= start && (end === null || instant < end)
|
||||
}
|
||||
|
||||
function recoveryTargetFor(meter: MeterResponse, meters: MeterResponse[]): MeterResponse | null {
|
||||
if (meter.ended_at === null) return null
|
||||
const active = meters.filter((candidate) => candidate.commodity === meter.commodity && candidate.ended_at === null)
|
||||
if (active.length !== 1) return null
|
||||
const target = active[0]
|
||||
const targetStart = parseBackendTimestamp(target.started_at).getTime()
|
||||
const predecessors = meters.filter((candidate) => candidate.commodity === meter.commodity && candidate.ended_at !== null &&
|
||||
parseBackendTimestamp(candidate.ended_at).getTime() <= targetStart)
|
||||
const latestEnd = Math.max(...predecessors.map((candidate) => parseBackendTimestamp(candidate.ended_at!).getTime()))
|
||||
const immediate = predecessors.filter((candidate) => parseBackendTimestamp(candidate.ended_at!).getTime() === latestEnd)
|
||||
if (immediate.length !== 1 || immediate[0].id !== meter.id) return null
|
||||
const sourceStart = parseBackendTimestamp(meter.started_at).getTime()
|
||||
const sourceEnd = parseBackendTimestamp(meter.ended_at).getTime()
|
||||
for (const candidate of meters) {
|
||||
if (candidate.id === meter.id || candidate.id === target.id || candidate.commodity !== meter.commodity) continue
|
||||
const candidateStart = parseBackendTimestamp(candidate.started_at).getTime()
|
||||
const candidateEnd = candidate.ended_at ? parseBackendTimestamp(candidate.ended_at).getTime() : null
|
||||
if (intervalsOverlap(sourceStart, sourceEnd, candidateStart, candidateEnd) ||
|
||||
intervalsOverlap(targetStart, null, candidateStart, candidateEnd)) return null
|
||||
}
|
||||
return target
|
||||
}
|
||||
|
||||
function BindingTimeline({ binding, sources }: {
|
||||
binding: NonNullable<MeterResponse['bindings']>[number]
|
||||
sources: ReturnType<typeof useSources>
|
||||
}) {
|
||||
const source = sources.data?.items.find((item) => item.uuid === binding.source_uuid)
|
||||
const channels = useSourceChannels(source?.uuid ?? null)
|
||||
const channel = channels.data?.items.find((item) => item.uuid === binding.source_channel_uuid)
|
||||
let endpoint = 'Source details unavailable'
|
||||
if (sources.isLoading) endpoint = 'Loading source details…'
|
||||
else if (!sources.isError && source) endpoint = source.name
|
||||
else if (!sources.isError) endpoint = 'Source unavailable'
|
||||
|
||||
let channelName = 'Channel details unavailable'
|
||||
if (source && channels.isLoading) channelName = 'Loading channel details…'
|
||||
else if (source && !channels.isError && channel) channelName = channel.label
|
||||
else if (source && !channels.isError) channelName = 'Channel unavailable'
|
||||
|
||||
return (
|
||||
<Stack gap={0} mb="xs" data-testid={`binding-timeline-${binding.uuid}`}>
|
||||
<Text size="xs">{endpoint} → {channelName}</Text>
|
||||
<Text size="xs" c="dimmed">
|
||||
[{formatLocalDateTime(binding.started_at)}, {binding.ended_at ? formatLocalDateTime(binding.ended_at) : 'open-ended'})
|
||||
{' '}({binding.ended_at ? 'closed' : 'active'})
|
||||
</Text>
|
||||
</Stack>
|
||||
)
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Declare meter form (modal)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
interface DeclareMeterFormProps {
|
||||
meters: MeterResponse[]
|
||||
onClose: () => void
|
||||
onSaved: () => void
|
||||
}
|
||||
|
||||
function DeclareMeterForm({ onClose, onSaved }: DeclareMeterFormProps) {
|
||||
function DeclareMeterForm({ meters, onClose, onSaved }: DeclareMeterFormProps) {
|
||||
const [label, setLabel] = useState('')
|
||||
const [dateStr, setDateStr] = useState('')
|
||||
const [reason, setReason] = useState<string | null>(null)
|
||||
const [note, setNote] = useState('')
|
||||
const [error, setError] = useState<string | null>(null)
|
||||
const [commodity, setCommodity] = useState<string | null>('electricity')
|
||||
const [sourceUuid, setSourceUuid] = useState<string | null>(null)
|
||||
const [channelUuid, setChannelUuid] = useState<string | null>(null)
|
||||
const sources = useSources()
|
||||
const channels = useSourceChannels(sourceUuid)
|
||||
|
||||
const declareMutation = useDeclareMeter()
|
||||
const unit = expectedUnit(commodity ?? 'electricity')
|
||||
const oldMeter = meters.find((meter) => meter.commodity === commodity && meter.ended_at === null)
|
||||
const channelEligibility = (uuid: string, startedAt: string): Eligibility => {
|
||||
const channel = channels.data?.items.find((item) => item.uuid === uuid)
|
||||
if (!channel) return { eligible: false, reason: 'channel is unavailable; reload the source' }
|
||||
if (channel.unit !== unit) return { eligible: false, reason: `unit mismatch: ${channel.unit}; this meter needs ${unit}` }
|
||||
if (!startedAt) return { eligible: false, reason: 'choose a start date first' }
|
||||
const start = localInstant(toLocalMidnightNaive(startedAt))
|
||||
if (start === null) return { eligible: false, reason: 'choose a valid start date first' }
|
||||
if (startedAt > toLocalDateInputString(new Date())) return { eligible: false, reason: 'future start dates cannot bind a channel' }
|
||||
|
||||
// Channel aggregates include closed binding history. For a meter swap, only
|
||||
// currently open bindings determine whether this channel can be handed off.
|
||||
const openBindings = meters.flatMap((meter) =>
|
||||
(meter.bindings ?? [])
|
||||
.filter((binding) => binding.source_channel_uuid === channel.uuid && binding.ended_at === null)
|
||||
.map((binding) => ({ meter, binding })),
|
||||
)
|
||||
const oldBinding = openBindings[0]
|
||||
const isSingleOldBinding = reason === 'meter_swap' && oldMeter !== undefined &&
|
||||
openBindings.length === 1 && oldBinding !== undefined &&
|
||||
oldBinding.meter.id === oldMeter.id && oldBinding.meter.commodity === commodity
|
||||
// A handoff must leave a non-empty interval on the old Meter. The backend
|
||||
// rejects equality too, so surface it here instead of offering a request
|
||||
// that is guaranteed to fail.
|
||||
if (isSingleOldBinding && start <= parseBackendTimestamp(oldBinding.binding.started_at).getTime()) {
|
||||
return { eligible: false, reason: 'handoff boundary must be strictly after the current binding start' }
|
||||
}
|
||||
const canHandoff = isSingleOldBinding
|
||||
const interval = channelIntervalEligibility(
|
||||
meters, channel.uuid, toLocalMidnightNaive(startedAt), canHandoff ? [oldBinding.binding.uuid] : [],
|
||||
)
|
||||
if (!interval.eligible) return interval
|
||||
if (openBindings.length && !canHandoff) {
|
||||
return openBindings.length > 1
|
||||
? { eligible: false, reason: 'ambiguous: multiple open bindings; resolve them first' }
|
||||
: { eligible: false, reason: 'occupied by an open binding; choose a different channel or close/transfer it first' }
|
||||
}
|
||||
return { eligible: true }
|
||||
}
|
||||
const channelOptions = channels.data?.items.map((channel) => {
|
||||
const result = channelEligibility(channel.uuid, dateStr)
|
||||
const canHandoff = !isUnboundChannel(channel.uuid) && result.eligible
|
||||
return {
|
||||
value: channel.uuid,
|
||||
label: `${channel.label} (${channel.unit})${canHandoff ? ' — hand off from current meter' : result.reason ? ` — ${result.reason}` : ''}`,
|
||||
disabled: !result.eligible,
|
||||
}
|
||||
}) ?? []
|
||||
const selectedChannelUuid = channelUuid && channelEligibility(channelUuid, dateStr).eligible
|
||||
? channelUuid
|
||||
: null
|
||||
function isUnboundChannel(uuid: string): boolean {
|
||||
return !meters.some((meter) => meter.bindings?.some(
|
||||
(binding) => binding.source_channel_uuid === uuid && binding.ended_at === null,
|
||||
))
|
||||
}
|
||||
|
||||
async function handleSubmit(e: React.FormEvent) {
|
||||
e.preventDefault()
|
||||
@@ -107,6 +292,29 @@ function DeclareMeterForm({ onClose, onSaved }: DeclareMeterFormProps) {
|
||||
setError('Reason is required.')
|
||||
return
|
||||
}
|
||||
if (sourceUuid && !selectedChannelUuid) {
|
||||
const unavailable = channelOptions.length === 1 && channelOptions[0].disabled
|
||||
? channelEligibility(channelOptions[0].value, dateStr).reason
|
||||
: undefined
|
||||
setError(unavailable ? `${unavailable[0].toUpperCase()}${unavailable.slice(1)}` : 'Select an eligible source channel or clear the optional source.')
|
||||
return
|
||||
}
|
||||
// Omitting a channel for meter_swap asks the backend to auto-handoff the
|
||||
// sole open binding. Keep that implicit path subject to the same strict
|
||||
// boundary rule as an explicitly selected channel.
|
||||
const start = localInstant(toLocalMidnightNaive(dateStr))
|
||||
const oldMeter = meters.find((meter) => meter.commodity === commodity && meter.ended_at === null)
|
||||
// The implicit backend handoff only considers bindings on the current
|
||||
// commodity's old meter. Bindings are unit-compatible with their meter
|
||||
// by the binding contract, so an unrelated heating/hot-water binding must
|
||||
// neither make this ambiguous nor bypass this strict boundary check.
|
||||
const autoHandoffCandidates = (oldMeter?.bindings ?? []).filter((binding) => binding.ended_at === null)
|
||||
const oldBinding = autoHandoffCandidates[0]
|
||||
if (reason === 'meter_swap' && start !== null && oldMeter !== undefined && autoHandoffCandidates.length === 1 &&
|
||||
oldBinding !== undefined && start <= parseBackendTimestamp(oldBinding.started_at).getTime()) {
|
||||
setError('Handoff boundary must be strictly after the current binding start.')
|
||||
return
|
||||
}
|
||||
|
||||
try {
|
||||
await declareMutation.mutateAsync({
|
||||
@@ -114,18 +322,12 @@ function DeclareMeterForm({ onClose, onSaved }: DeclareMeterFormProps) {
|
||||
started_at: toLocalMidnightNaive(dateStr),
|
||||
reason: reason as MeterReason,
|
||||
note: note.trim() || undefined,
|
||||
commodity: 'electricity',
|
||||
commodity: commodity ?? 'electricity',
|
||||
...(selectedChannelUuid ? { source_channel_uuid: selectedChannelUuid } : {}),
|
||||
})
|
||||
onSaved()
|
||||
onClose()
|
||||
} catch (err) {
|
||||
if (err instanceof ApiError) {
|
||||
const detail = (err.body as { detail?: string } | null)?.detail
|
||||
setError(detail ?? `Error ${err.status}: failed to declare meter.`)
|
||||
} else {
|
||||
setError('Failed to declare meter. Please try again.')
|
||||
}
|
||||
}
|
||||
} catch (err) { setError(apiErrorMessage(err, 'Failed to declare meter. Please try again.')) }
|
||||
}
|
||||
|
||||
return (
|
||||
@@ -153,7 +355,11 @@ function DeclareMeterForm({ onClose, onSaved }: DeclareMeterFormProps) {
|
||||
type="date"
|
||||
required
|
||||
value={dateStr}
|
||||
onChange={(e) => setDateStr(e.currentTarget.value)}
|
||||
onChange={(e) => {
|
||||
const nextDateStr = e.currentTarget.value
|
||||
setDateStr(nextDateStr)
|
||||
setChannelUuid((uuid) => uuid && !channelEligibility(uuid, nextDateStr).eligible ? null : uuid)
|
||||
}}
|
||||
data-testid="meter-started-at"
|
||||
/>
|
||||
|
||||
@@ -162,10 +368,21 @@ function DeclareMeterForm({ onClose, onSaved }: DeclareMeterFormProps) {
|
||||
required
|
||||
data={REASON_OPTIONS}
|
||||
value={reason}
|
||||
onChange={setReason}
|
||||
onChange={(value) => { setReason(value); setChannelUuid(null) }}
|
||||
data-testid="meter-reason"
|
||||
/>
|
||||
|
||||
<Select label="Commodity" value={commodity} onChange={(value) => { setCommodity(value); setChannelUuid(null) }} data={[
|
||||
{ value: 'electricity', label: 'Electricity' },
|
||||
{ value: 'heating', label: 'Heating' },
|
||||
{ value: 'hot_water', label: 'Hot water' },
|
||||
]} />
|
||||
<Select label="Bind source (optional)" value={sourceUuid} onChange={(value) => { setSourceUuid(value); setChannelUuid(null) }} data={sources.data?.items.filter((source) => typeof source.uuid === 'string').map((source) => ({ value: source.uuid, label: source.name })) ?? []} />
|
||||
{reason === 'meter_swap' && <Alert color="blue">If the previous meter has exactly one compatible open binding, declaring this meter automatically hands that channel over atomically. Ambiguous bindings remain unavailable.</Alert>}
|
||||
{sources.isLoading && <Text size="sm">Loading sources…</Text>}{sources.isError && <Alert color="red">Could not load sources. Retry after the connection recovers.</Alert>}
|
||||
{sourceUuid && channels.isLoading && <Text size="sm">Loading source channels…</Text>}{sourceUuid && channels.isError && <Alert color="red">Could not load source channels. Choose another source or retry.</Alert>}
|
||||
{sourceUuid && <Select label="Compatible source channel (optional)" value={selectedChannelUuid} onChange={setChannelUuid} description="Disabled channels explain unit, current interval, ambiguity, or required time. Closed history remains reusable." data={channelOptions} />}
|
||||
|
||||
<Textarea
|
||||
label="Note (optional)"
|
||||
value={note}
|
||||
@@ -252,14 +469,7 @@ function EditMeterForm({ meter, onClose, onSaved }: EditMeterFormProps) {
|
||||
await updateMutation.mutateAsync({ id: meter.id, body: patchBody })
|
||||
onSaved(startedAtChanged)
|
||||
onClose()
|
||||
} catch (err) {
|
||||
if (err instanceof ApiError) {
|
||||
const detail = (err.body as { detail?: string } | null)?.detail
|
||||
setError(detail ?? `Error ${err.status}: failed to update meter.`)
|
||||
} else {
|
||||
setError('Failed to update meter. Please try again.')
|
||||
}
|
||||
}
|
||||
} catch (err) { setError(apiErrorMessage(err, 'Failed to update meter. Please try again.')) }
|
||||
}
|
||||
|
||||
return (
|
||||
@@ -333,10 +543,12 @@ function EditMeterForm({ meter, onClose, onSaved }: EditMeterFormProps) {
|
||||
|
||||
interface MeterTableProps {
|
||||
meters: MeterResponse[]
|
||||
sources: ReturnType<typeof useSources>
|
||||
onEdit: (meter: MeterResponse) => void
|
||||
onClose: (meter: MeterResponse) => void
|
||||
}
|
||||
|
||||
function MeterTable({ meters, onEdit }: MeterTableProps) {
|
||||
function MeterTable({ meters, sources, onEdit, onClose }: MeterTableProps) {
|
||||
if (meters.length === 0) {
|
||||
return (
|
||||
<Text c="dimmed" ta="center" size="sm" data-testid="meters-empty">
|
||||
@@ -356,6 +568,7 @@ function MeterTable({ meters, onEdit }: MeterTableProps) {
|
||||
<Table.Th>To</Table.Th>
|
||||
<Table.Th>Status</Table.Th>
|
||||
<Table.Th>Reason</Table.Th>
|
||||
<Table.Th>Binding timeline</Table.Th>
|
||||
<Table.Th style={{ textAlign: 'right' }}>Actions</Table.Th>
|
||||
</Table.Tr>
|
||||
</Table.Thead>
|
||||
@@ -399,16 +612,26 @@ function MeterTable({ meters, onEdit }: MeterTableProps) {
|
||||
{meter.reason}
|
||||
</Badge>
|
||||
</Table.Td>
|
||||
<Table.Td>
|
||||
{meter.bindings?.length ? meter.bindings.map((binding) => (
|
||||
<BindingTimeline key={binding.uuid} binding={binding} sources={sources} />
|
||||
)) : <Text size="xs" c="dimmed">Unbound</Text>}
|
||||
</Table.Td>
|
||||
<Table.Td>
|
||||
<Group justify="flex-end" gap="xs">
|
||||
<Button
|
||||
size="xs"
|
||||
variant="outline"
|
||||
variant="light"
|
||||
onClick={() => onEdit(meter)}
|
||||
data-testid={`meter-edit-${meter.id}`}
|
||||
>
|
||||
Edit
|
||||
</Button>
|
||||
{meter.bindings?.filter((binding) => binding.ended_at === null).map((binding) => (
|
||||
<BindingActions key={binding.uuid} meter={meter} binding={binding} meters={meters} />
|
||||
))}
|
||||
{isActive && !meter.bindings?.some((binding) => binding.ended_at === null) && <DirectBindButton meter={meter} meters={meters} />}
|
||||
{isActive && <Button size="xs" color="red" variant="light" onClick={() => onClose(meter)}>Close meter</Button>}
|
||||
</Group>
|
||||
</Table.Td>
|
||||
</Table.Tr>
|
||||
@@ -420,15 +643,184 @@ function MeterTable({ meters, onEdit }: MeterTableProps) {
|
||||
)
|
||||
}
|
||||
|
||||
function DirectBindButton({ meter, meters }: { meter: MeterResponse; meters: MeterResponse[] }) {
|
||||
const [opened, setOpened] = useState(false)
|
||||
return <>{<Button size="xs" variant="light" onClick={() => setOpened(true)}>Bind source</Button>}{opened && <DirectBindModal meter={meter} meters={meters} onClose={() => setOpened(false)} />}</>
|
||||
}
|
||||
|
||||
function DirectBindModal({ meter, meters, onClose }: { meter: MeterResponse; meters: MeterResponse[]; onClose: () => void }) {
|
||||
const [sourceUuid, setSourceUuid] = useState<string | null>(null)
|
||||
const [channelUuid, setChannelUuid] = useState<string | null>(null)
|
||||
const [startedAt, setStartedAt] = useState(() => toLocalDateTimeInputString())
|
||||
const [error, setError] = useState<string | null>(null)
|
||||
const sources = useSources(); const channels = useSourceChannels(sourceUuid); const create = useCreateBinding()
|
||||
const eligibilityAt = (channel: { uuid: string; unit: string }, value: string): Eligibility => {
|
||||
if (channel.unit !== expectedUnit(meter.commodity)) return { eligible: false, reason: `unit mismatch: ${channel.unit}; this meter needs ${expectedUnit(meter.commodity)}` }
|
||||
const start = localInstant(value)
|
||||
if (start === null) return { eligible: false, reason: 'choose a valid binding start time first' }
|
||||
if (value > toLocalDateTimeInputString()) return { eligible: false, reason: 'future binding start times are not allowed' }
|
||||
if (!meterContainsInstant(meter, start)) return { eligible: false, reason: 'binding start must be within this meter epoch' }
|
||||
return channelIntervalEligibility(meters, channel.uuid, value)
|
||||
}
|
||||
const eligibility = (channel: { uuid: string; unit: string }) => eligibilityAt(channel, startedAt)
|
||||
const options = channels.data?.items.map((channel) => {
|
||||
const result = eligibility(channel)
|
||||
return { value: channel.uuid, label: `${channel.label} (${channel.unit})${result.reason ? ` — ${result.reason}` : ''}`, disabled: !result.eligible }
|
||||
}) ?? []
|
||||
const selectedChannel = channels.data?.items.find((channel) => channel.uuid === channelUuid)
|
||||
const selectedEligible = selectedChannel !== undefined && eligibility(selectedChannel).eligible
|
||||
async function save() {
|
||||
if (create.isPending) return
|
||||
if (!channelUuid) return setError('Select a genuinely unbound, unit-compatible channel.')
|
||||
if (!isValidLocalDateTime(startedAt)) return setError('Choose a valid binding start time.')
|
||||
if (!selectedEligible) return setError('The selected channel is no longer eligible. Choose an available channel.')
|
||||
setError(null)
|
||||
try { await create.mutateAsync({ id: meter.id, body: { source_channel_uuid: channelUuid, started_at: startedAt } }); onClose() } catch (err) { setError(apiErrorMessage(err, 'Could not bind this source.')) }
|
||||
}
|
||||
return <Modal opened onClose={onClose} title="Bind source" data-testid={`direct-bind-modal-${meter.id}`}><form onSubmit={(event) => { event.preventDefault(); void save() }}><Stack>
|
||||
<Alert color="blue">This active meter has no open binding. Choose a currently unoccupied compatible channel.</Alert>
|
||||
{sources.isLoading && <Text size="sm">Loading sources…</Text>}{sources.isError && <Alert color="red">Could not load sources. Retry after the connection recovers.</Alert>}
|
||||
<Select label="Source" value={sourceUuid} onChange={(value) => { setSourceUuid(value); setChannelUuid(null) }} data={sources.data?.items.map((source) => ({ value: source.uuid, label: source.name })) ?? []} />
|
||||
{sourceUuid && channels.isLoading && <Text size="sm">Loading source channels…</Text>}{sourceUuid && channels.isError && <Alert color="red">Could not load source channels. Choose another source or retry.</Alert>}
|
||||
<Select label="Source channel" value={channelUuid} onChange={setChannelUuid} description="Disabled channels explain the unit, interval, ambiguity, or time constraint." data={options} />
|
||||
<TextInput label="Binding start time" type="datetime-local" value={startedAt} onChange={(event) => { const value = event.currentTarget.value; setStartedAt(value); setChannelUuid((uuid) => { if (!isValidLocalDateTime(value)) return uuid; const channel = channels.data?.items.find((item) => item.uuid === uuid); return channel && !eligibilityAt(channel, value).eligible ? null : uuid }) }} required />
|
||||
{error && <Alert color="red">{error}</Alert>}
|
||||
<Group justify="flex-end"><Button type="button" variant="default" onClick={onClose}>Cancel</Button><Button type="submit" loading={create.isPending} disabled={create.isPending || (!!channelUuid && !selectedEligible)}>Bind source</Button></Group>
|
||||
</Stack></form></Modal>
|
||||
}
|
||||
|
||||
function BindingActions({ meter, binding, meters }: { meter: MeterResponse; binding: NonNullable<MeterResponse['bindings']>[number]; meters: MeterResponse[] }) {
|
||||
const [unbindOpened, setUnbindOpened] = useState(false)
|
||||
const [transferOpened, setTransferOpened] = useState(false)
|
||||
const recoveryTarget = recoveryTargetFor(meter, meters)
|
||||
// Cross-Meter recovery closes at the old Meter boundary. An anomalous
|
||||
// retained binding beginning at or after that boundary would create a
|
||||
// zero-length/negative source interval that the server correctly rejects.
|
||||
const recoverySourceIsClosable = meter.ended_at === null ||
|
||||
parseBackendTimestamp(binding.started_at).getTime() < parseBackendTimestamp(meter.ended_at).getTime()
|
||||
return <>
|
||||
{meter.ended_at === null ? <Button size="xs" variant="light" onClick={() => setTransferOpened(true)}>Transfer source</Button> : recoveryTarget && recoverySourceIsClosable && <Button size="xs" variant="light" onClick={() => setTransferOpened(true)}>Recover binding</Button>}
|
||||
<Button size="xs" variant="light" onClick={() => setUnbindOpened(true)}>Unbind</Button>
|
||||
{meter.ended_at !== null && recoveryTarget && !recoverySourceIsClosable && <Text size="xs" c="red">Cannot recover: the source binding starts at or after this Meter ended.</Text>}
|
||||
{unbindOpened && <UnbindModal meter={meter} binding={binding} onClose={() => setUnbindOpened(false)} />}
|
||||
{transferOpened && <TransferModal target={recoveryTarget ?? meter} sourceBinding={binding} meters={meters} recovery={!!recoveryTarget} onClose={() => setTransferOpened(false)} />}
|
||||
</>
|
||||
}
|
||||
|
||||
function UnbindModal({ meter, binding, onClose }: { meter: MeterResponse; binding: NonNullable<MeterResponse['bindings']>[number]; onClose: () => void }) {
|
||||
const [endedAt, setEndedAt] = useState(() => meter.ended_at ? toLocalDateTimeInputString(parseBackendTimestamp(meter.ended_at)) : toLocalDateTimeInputString())
|
||||
const [error, setError] = useState<string | null>(null)
|
||||
const close = useCloseBinding()
|
||||
async function save() {
|
||||
if (close.isPending) return
|
||||
if (!isValidLocalDateTime(endedAt)) return setError('Choose a valid unbind time.')
|
||||
const instant = localInstant(endedAt)
|
||||
if (instant === null || instant <= parseBackendTimestamp(binding.started_at).getTime()) {
|
||||
return setError('Unbind time must be strictly after the binding start.')
|
||||
}
|
||||
if (endedAt > toLocalDateTimeInputString()) return setError('A future unbind time is not allowed.')
|
||||
if (meter.ended_at && instant > parseBackendTimestamp(meter.ended_at).getTime()) {
|
||||
return setError('Unbind time must not be after the meter end.')
|
||||
}
|
||||
setError(null)
|
||||
try { await close.mutateAsync({ uuid: binding.uuid, ended_at: endedAt }); onClose() } catch (err) { setError(apiErrorMessage(err, 'Could not unbind this source. History was not deleted.')) }
|
||||
}
|
||||
return <Modal opened onClose={onClose} title="Unbind source" data-testid={`unbind-modal-${binding.uuid}`}><form onSubmit={(event) => { event.preventDefault(); void save() }}><Stack>
|
||||
<Alert color="blue">Unbinding closes this binding at the selected time. It never deletes binding history.</Alert>
|
||||
<TextInput label="Unbind time" type="datetime-local" value={endedAt} onChange={(event) => setEndedAt(event.currentTarget.value)} required />
|
||||
{error && <Alert color="red">{error}</Alert>}
|
||||
<Group justify="flex-end"><Button type="button" variant="default" onClick={onClose}>Cancel</Button><Button type="submit" loading={close.isPending} disabled={close.isPending}>Unbind</Button></Group>
|
||||
</Stack></form></Modal>
|
||||
}
|
||||
|
||||
function TransferModal({ target, sourceBinding, meters, recovery, onClose }: { target: MeterResponse; sourceBinding: NonNullable<MeterResponse['bindings']>[number]; meters: MeterResponse[]; recovery: boolean; onClose: () => void }) {
|
||||
const [sourceUuid, setSourceUuid] = useState<string | null>(null)
|
||||
const [channelUuid, setChannelUuid] = useState<string | null>(null)
|
||||
const [effectiveAt, setEffectiveAt] = useState(() => recovery ? toLocalDateTimeInputString(parseBackendTimestamp(target.started_at)) : toLocalDateTimeInputString())
|
||||
const [error, setError] = useState<string | null>(null)
|
||||
const sources = useSources(); const channels = useSourceChannels(sourceUuid)
|
||||
const transfer = useTransferBinding()
|
||||
const channelEligibilityAt = (channel: { uuid: string; unit: string }, value: string): Eligibility => {
|
||||
if (channel.unit !== expectedUnit(target.commodity)) return { eligible: false, reason: `unit mismatch: ${channel.unit}; target needs ${expectedUnit(target.commodity)}` }
|
||||
const start = localInstant(value)
|
||||
if (start === null) return { eligible: false, reason: 'choose a valid effective time first' }
|
||||
if (value > toLocalDateTimeInputString()) return { eligible: false, reason: 'future effective times are not allowed' }
|
||||
if (!meterContainsInstant(target, start)) return { eligible: false, reason: 'effective time must be within the target meter epoch' }
|
||||
// A same-meter transfer closes its source at `effective_at`. Equality
|
||||
// would therefore create the forbidden zero-length [start, start)
|
||||
// interval. Recovery closes at the old meter boundary instead, so it
|
||||
// deliberately keeps the normal target-epoch rule and may be equal to
|
||||
// the source binding's (much earlier) start.
|
||||
if (!recovery && start <= parseBackendTimestamp(sourceBinding.started_at).getTime()) {
|
||||
return { eligible: false, reason: 'same-meter transfer must be strictly after the source binding start' }
|
||||
}
|
||||
return channelIntervalEligibility(meters, channel.uuid, value, [sourceBinding.uuid])
|
||||
}
|
||||
const channelEligibility = (channel: { uuid: string; unit: string }) => channelEligibilityAt(channel, effectiveAt)
|
||||
const options = channels.data?.items.map((channel) => {
|
||||
const result = channelEligibility(channel)
|
||||
return { value: channel.uuid, label: `${channel.label} (${channel.unit})${result.reason ? ` — ${result.reason}` : ''}`, disabled: !result.eligible }
|
||||
}) ?? []
|
||||
const selectedChannel = channels.data?.items.find((channel) => channel.uuid === channelUuid)
|
||||
const selectedEligible = selectedChannel !== undefined && channelEligibility(selectedChannel).eligible
|
||||
const isFuture = effectiveAt > toLocalDateTimeInputString()
|
||||
async function save() {
|
||||
if (transfer.isPending) return
|
||||
if (!channelUuid) return setError('Select a unit-compatible source channel.')
|
||||
if (!isValidLocalDateTime(effectiveAt) || isFuture) return setError('Choose a valid non-future effective time.')
|
||||
if (!selectedEligible) return setError('The selected channel is no longer eligible. Choose an available channel.')
|
||||
setError(null)
|
||||
try { await transfer.mutateAsync({ id: target.id, body: { from_binding_uuid: sourceBinding.uuid, to_source_channel_uuid: channelUuid, effective_at: effectiveAt } }); onClose() } catch (err) { setError(apiErrorMessage(err, 'Transfer failed. No partial source switch was saved.')) }
|
||||
}
|
||||
return <Modal opened onClose={onClose} title={recovery ? 'Recover stranded binding' : 'Transfer source binding'} data-testid={`transfer-modal-${sourceBinding.uuid}`}><form onSubmit={(event) => { event.preventDefault(); void save() }}><Stack>
|
||||
<Alert color="blue">This is one atomic Transfer request: either the old binding closes and the new one opens together, or neither change is saved.</Alert>
|
||||
{recovery && <Alert color="yellow">This binding is stranded on a closed meter. Recovery defaults to the new meter start. A later time is allowed, but creates an unbound gap before it.</Alert>}
|
||||
{sources.isLoading && <Text size="sm">Loading sources…</Text>}{sources.isError && <Alert color="red">Could not load sources. Retry after the connection recovers.</Alert>}
|
||||
<Select label="Source" value={sourceUuid} onChange={(value) => { setSourceUuid(value); setChannelUuid(null) }} data={sources.data?.items.map((source) => ({ value: source.uuid, label: source.name })) ?? []} />
|
||||
{sourceUuid && channels.isLoading && <Text size="sm">Loading source channels…</Text>}{sourceUuid && channels.isError && <Alert color="red">Could not load source channels. Choose another source or retry.</Alert>}
|
||||
<Select label="Source channel" value={channelUuid} onChange={setChannelUuid} description="Disabled channels name the specific unit or unrelated-open-interval conflict. The server also rejects ambiguity atomically." data={options} />
|
||||
<TextInput label="Effective time" type="datetime-local" value={effectiveAt} onChange={(event) => { const value = event.currentTarget.value; setEffectiveAt(value); setChannelUuid((uuid) => { if (!isValidLocalDateTime(value)) return uuid; const channel = channels.data?.items.find((item) => item.uuid === uuid); return channel && !channelEligibilityAt(channel, value).eligible ? null : uuid }) }} required data-testid="transfer-effective-at" />
|
||||
{recovery && effectiveAt && effectiveAt > toLocalDateTimeInputString(parseBackendTimestamp(target.started_at)) && <Alert color="yellow">Warning: this later time leaves an unbound gap from the new meter start until this transfer.</Alert>}
|
||||
{isFuture && <Alert color="red">A future effective time is not allowed.</Alert>}
|
||||
{error && <Alert color="red">{error}</Alert>}
|
||||
<Group justify="flex-end"><Button type="button" variant="default" onClick={onClose}>Cancel</Button><Button type="submit" loading={transfer.isPending} disabled={transfer.isPending || (!!channelUuid && !selectedEligible)}>Transfer binding</Button></Group>
|
||||
</Stack></form></Modal>
|
||||
}
|
||||
|
||||
function CloseMeterModal({ meter, onClose }: { meter: MeterResponse; onClose: () => void }) {
|
||||
const [endedAt, setEndedAt] = useState(() => toLocalDateTimeInputString())
|
||||
const [error, setError] = useState<string | null>(null)
|
||||
const close = useCloseMeter()
|
||||
async function save() {
|
||||
if (close.isPending) return
|
||||
if (!isValidLocalDateTime(endedAt)) return setError('Choose a valid close time.')
|
||||
const instant = localInstant(endedAt)
|
||||
if (instant === null || instant <= parseBackendTimestamp(meter.started_at).getTime()) {
|
||||
return setError('Close time must be strictly after the meter start.')
|
||||
}
|
||||
if (endedAt > toLocalDateTimeInputString()) return setError('A future close time is not allowed.')
|
||||
setError(null)
|
||||
try { await close.mutateAsync({ id: meter.id, ended_at: endedAt }); onClose() } catch (err) { setError(apiErrorMessage(err, 'Could not close this meter.')) }
|
||||
}
|
||||
return <Modal opened onClose={onClose} title={`Close Meter — ${meter.label}`} data-testid={`close-meter-modal-${meter.id}`}><form onSubmit={(event) => { event.preventDefault(); void save() }}><Stack>
|
||||
<Alert color="yellow">Closing leaves no active {meter.commodity} meter. Every open binding on this meter closes at the same boundary.</Alert>
|
||||
<TextInput label="Close time" type="datetime-local" value={endedAt} onChange={(event) => setEndedAt(event.currentTarget.value)} required />
|
||||
{error && <Alert color="red">{error}</Alert>}
|
||||
<Group justify="flex-end"><Button type="button" variant="default" onClick={onClose}>Cancel</Button><Button type="submit" color="red" loading={close.isPending} disabled={close.isPending}>Close meter</Button></Group>
|
||||
</Stack></form></Modal>
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// MeterManager — top-level
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export function MeterManager() {
|
||||
const metersQuery = useMeters()
|
||||
const sources = useSources()
|
||||
|
||||
const [showDeclareForm, setShowDeclareForm] = useState(false)
|
||||
const [editMeter, setEditMeter] = useState<MeterResponse | null>(null)
|
||||
const [closeMeter, setCloseMeter] = useState<MeterResponse | null>(null)
|
||||
const [recomputeNotice, setRecomputeNotice] = useState(false)
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
@@ -456,7 +848,7 @@ export function MeterManager() {
|
||||
return (
|
||||
<Stack gap="lg" data-testid="meter-manager">
|
||||
<Group justify="space-between" align="center">
|
||||
<Text fw={500}>Electricity Meters</Text>
|
||||
<Text fw={500}>Meters</Text>
|
||||
<Button
|
||||
onClick={() => setShowDeclareForm(true)}
|
||||
data-testid="meter-declare-button"
|
||||
@@ -477,11 +869,12 @@ export function MeterManager() {
|
||||
</Notification>
|
||||
)}
|
||||
|
||||
<MeterTable meters={meters} onEdit={(m) => setEditMeter(m)} />
|
||||
<MeterTable meters={meters} sources={sources} onEdit={(m) => setEditMeter(m)} onClose={(m) => setCloseMeter(m)} />
|
||||
|
||||
{/* Declare new meter */}
|
||||
{showDeclareForm && (
|
||||
<DeclareMeterForm
|
||||
meters={meters}
|
||||
onClose={() => setShowDeclareForm(false)}
|
||||
onSaved={() => setShowDeclareForm(false)}
|
||||
/>
|
||||
@@ -498,6 +891,8 @@ export function MeterManager() {
|
||||
}}
|
||||
/>
|
||||
)}
|
||||
|
||||
{closeMeter && <CloseMeterModal meter={closeMeter} onClose={() => setCloseMeter(null)} />}
|
||||
</Stack>
|
||||
)
|
||||
}
|
||||
|
||||
@@ -0,0 +1,15 @@
|
||||
import { describe, expect, it, vi, beforeEach } from 'vitest'
|
||||
import { screen, waitFor } from '@testing-library/react'
|
||||
import userEvent from '@testing-library/user-event'
|
||||
import { renderWithProviders } from '../test-utils'
|
||||
import { SourceForm } from './SourceForm'
|
||||
|
||||
const mockGet = vi.fn(); const mockPatch = vi.fn()
|
||||
vi.mock('../api/client', () => ({ default: { GET: (...a: unknown[]) => mockGet(...a), POST: vi.fn(), PATCH: (...a: unknown[]) => mockPatch(...a), DELETE: vi.fn() }, ApiError: class ApiError extends Error { constructor(public status: number, public body: unknown) { super(`API error ${status}`) } }, registerLoginRedirect: vi.fn() }))
|
||||
const profile = { kind: 'dsmr_mqtt', fields: [{ name: 'tls_enabled', value_type: 'bool', default: false }, { name: 'port', value_type: 'int', default: 1883 }, { name: 'topic', value_type: 'string', default: 'telegram' }, { name: 'password', value_type: 'string', secret: true }] }
|
||||
const source = { uuid: 'source-1', name: 'DSMR', kind: 'dsmr_mqtt', enabled: true, config: { tls_enabled: true, port: 8883, topic: 'old', password: '********' } }
|
||||
describe('SourceForm typed PATCH and secrets', () => {
|
||||
beforeEach(() => { vi.clearAllMocks(); mockGet.mockResolvedValue({ data: { items: [profile] } }) })
|
||||
it('preserves native bool/number/string and omits untouched masked secret', async () => { const user = userEvent.setup(); mockPatch.mockResolvedValue({ data: source }); renderWithProviders(<SourceForm source={source as never} onClose={vi.fn()} />); await user.click(await screen.findByRole('button', { name: 'Save Source' })); await waitFor(() => expect(mockPatch).toHaveBeenCalled()); expect(mockPatch).toHaveBeenCalledWith('/api/energy/sources/{source_uuid}', { params: { path: { source_uuid: 'source-1' } }, body: { name: 'DSMR', enabled: true, config: { tls_enabled: true, port: 8883, topic: 'old' } } }) })
|
||||
it('sends a replacement secret and shows 422 detail', async () => { const user = userEvent.setup(); mockPatch.mockRejectedValue(new (await import('../api/client')).ApiError(422, { detail: 'invalid broker' })); renderWithProviders(<SourceForm source={source as never} onClose={vi.fn()} />); await user.type(await screen.findByLabelText('password'), 'new-secret'); await user.click(screen.getByRole('button', { name: 'Save Source' })); await waitFor(() => expect(mockPatch).toHaveBeenCalled()); expect(mockPatch.mock.calls[0][1].body.config).toMatchObject({ password: 'new-secret', tls_enabled: true, port: 8883 }); expect(await screen.findByText('invalid broker')).toBeInTheDocument() })
|
||||
})
|
||||
@@ -0,0 +1,33 @@
|
||||
import { useState } from 'react'
|
||||
import { Alert, Button, Checkbox, Group, Modal, Select, Stack, TextInput } from '@mantine/core'
|
||||
import { ApiError } from '../api/client'
|
||||
import { useCreateSource, useSourceProfiles, useUpdateSource, type MeterSourceResponse } from './hooks'
|
||||
|
||||
export function SourceForm({ source, onClose }: { source?: MeterSourceResponse; onClose: () => void }) {
|
||||
const profiles = useSourceProfiles(); const create = useCreateSource(); const update = useUpdateSource()
|
||||
const [name, setName] = useState(source?.name ?? ''); const [kind, setKind] = useState<string | null>(source?.kind ?? null)
|
||||
const [enabled, setEnabled] = useState(source?.enabled ?? true); const [config, setConfig] = useState<Record<string, string | boolean | number>>({}); const [error, setError] = useState<string | null>(null)
|
||||
const profile = profiles.data?.items.find((item) => item.kind === kind)
|
||||
async function submit(e: React.FormEvent) { e.preventDefault(); setError(null); if (!name.trim() || !kind) return setError('Name and source type are required.')
|
||||
const values: Record<string, unknown> = {}; profile?.fields.forEach((field) => {
|
||||
const changed = Object.prototype.hasOwnProperty.call(config, field.name)
|
||||
const raw = changed ? config[field.name] : (source?.config[field.name] ?? field.default ?? '')
|
||||
// A masked secret is deliberately absent from edit PATCHes until the user
|
||||
// explicitly enters a replacement; sending an empty/masked value is unsafe.
|
||||
if (field.secret && source && (!changed || raw === '')) return
|
||||
if (field.value_type === 'bool' || field.value_type === 'boolean') values[field.name] = typeof raw === 'boolean' ? raw : raw === 'true'
|
||||
else if (field.value_type === 'int' || field.value_type === 'integer') values[field.name] = typeof raw === 'number' ? raw : Number(raw)
|
||||
else values[field.name] = typeof raw === 'string' ? raw : String(raw)
|
||||
})
|
||||
try { if (source) await update.mutateAsync({ uuid: source.uuid, body: { name: name.trim(), enabled, config: values } }); else await create.mutateAsync({ name: name.trim(), kind, enabled, config: values }); onClose() } catch (err) { setError(err instanceof ApiError ? String((err.body as { detail?: string })?.detail ?? `Error ${err.status}`) : 'Could not save source.') }
|
||||
}
|
||||
return <Modal opened onClose={onClose} title={source ? 'Edit Source' : 'New Source'}><form onSubmit={submit}><Stack>
|
||||
<TextInput label="Name" required value={name} onChange={(e) => setName(e.currentTarget.value)} />
|
||||
{profiles.isLoading && <Alert color="blue">Loading source profiles…</Alert>}{profiles.isError && <Alert color="red">Failed to load source profiles.</Alert>}
|
||||
<Select label="Source type" required data={profiles.data?.items.map((p) => ({ value: p.kind, label: p.kind })) ?? []} value={kind} onChange={setKind} disabled={!!source} />
|
||||
{kind === 'warmtelink_serial' && <Alert color="blue">Serial sources use <code>/dev/serial/by-id/…</code>; 115200 7N1.</Alert>}
|
||||
{profile?.fields.map((field) => field.value_type === 'bool' || field.value_type === 'boolean' ? <Checkbox key={field.name} label={field.name} checked={Boolean(config[field.name] ?? source?.config[field.name] ?? field.default ?? false)} onChange={(e) => setConfig({ ...config, [field.name]: e.currentTarget.checked })} /> : <TextInput key={field.name} label={field.name} required={field.required} type={field.secret ? 'password' : (field.value_type === 'int' || field.value_type === 'integer' ? 'number' : 'text')} placeholder={field.secret && source ? 'Stored secret unchanged when blank' : undefined} value={String(config[field.name] ?? (field.secret ? '' : source?.config[field.name] ?? field.default ?? ''))} onChange={(e) => setConfig({ ...config, [field.name]: e.currentTarget.value })} />)}
|
||||
<Checkbox label="Enabled" checked={enabled} onChange={(e) => setEnabled(e.currentTarget.checked)} />
|
||||
{error && <Alert color="red">{error}</Alert>}<Group justify="flex-end"><Button variant="default" onClick={onClose}>Cancel</Button><Button type="submit" loading={create.isPending || update.isPending}>Save Source</Button></Group>
|
||||
</Stack></form></Modal>
|
||||
}
|
||||
@@ -0,0 +1,17 @@
|
||||
import { describe, expect, it, vi, beforeEach } from 'vitest'
|
||||
import { screen, waitFor } from '@testing-library/react'
|
||||
import userEvent from '@testing-library/user-event'
|
||||
import { renderWithProviders } from '../test-utils'
|
||||
import { SourceManager } from './SourceManager'
|
||||
const mockGet = vi.fn(); const mockPost = vi.fn(); const mockDelete = vi.fn()
|
||||
vi.mock('../api/client', () => ({ default: { GET: (...a: unknown[]) => mockGet(...a), POST: (...a: unknown[]) => mockPost(...a), PATCH: vi.fn(), DELETE: (...a: unknown[]) => mockDelete(...a) }, ApiError: class ApiError extends Error { constructor(public status: number, public body: unknown) { super(`API error ${status}`) } }, registerLoginRedirect: vi.fn() }))
|
||||
const source = { uuid: 's1', name: 'WarmteLink', kind: 'warmtelink_serial', enabled: true, status: 'online', config: {} }
|
||||
describe('SourceManager API states', () => {
|
||||
beforeEach(() => vi.clearAllMocks())
|
||||
it('renders source list, detail and a completed bounded discovery refresh', async () => { const user = userEvent.setup(); mockGet.mockImplementation((path: string) => Promise.resolve({ data: path === '/api/energy/sources' ? { items: [source] } : path.includes('channels') ? { items: [] } : source })); mockPost.mockResolvedValue({ data: { status: 'completed' } }); renderWithProviders(<SourceManager />); await user.click(await screen.findByText('WarmteLink')); expect(await screen.findByText(/No channels discovered yet/)).toBeInTheDocument(); await user.click(screen.getByRole('button', { name: /Refresh discovered channels/ })); expect(await screen.findByText(/Discovery completed/)).toBeInTheDocument() })
|
||||
it('explains that an enabled serial source reconnects automatically', async () => { const user = userEvent.setup(); mockGet.mockImplementation((path: string) => Promise.resolve({ data: path === '/api/energy/sources' ? { items: [{ ...source, status: 'error', last_error: 'serial unavailable' }] } : path.includes('channels') ? { items: [] } : { ...source, status: 'error', last_error: 'serial unavailable' } })); renderWithProviders(<SourceManager />); await user.click(await screen.findByText('WarmteLink')); expect(await screen.findAllByText(/reconnecting automatically/i)).not.toHaveLength(0); expect(screen.getByRole('button', { name: /Refresh discovered channels/ })).toBeInTheDocument() })
|
||||
it('renders source list empty and error states', async () => { mockGet.mockResolvedValueOnce({ data: { items: [] } }); const { unmount } = renderWithProviders(<SourceManager />); expect(await screen.findByText(/No sources configured/)).toBeInTheDocument(); unmount(); mockGet.mockRejectedValueOnce(new Error('offline')); renderWithProviders(<SourceManager />); expect(await screen.findByText(/Failed to load sources/)).toBeInTheDocument() })
|
||||
it('contains the wide four-column source table in a scroll area', async () => { mockGet.mockResolvedValue({ data: { items: [source] } }); renderWithProviders(<SourceManager />); expect(await screen.findByTestId('sources-table-scrollarea')).toBeInTheDocument(); expect(screen.getByTestId('sources-table')).toHaveStyle({ minWidth: '640px' }) })
|
||||
it('safely deletes an unreferenced source and clears its selection', async () => { const user = userEvent.setup(); let items = [source]; mockGet.mockImplementation((path: string) => Promise.resolve({ data: path === '/api/energy/sources' ? { items } : path.includes('channels') ? { items: [] } : source })); mockDelete.mockImplementation(async () => { items = []; return { data: undefined } }); renderWithProviders(<SourceManager />); await user.click(await screen.findByText('WarmteLink')); await user.click(screen.getByRole('button', { name: 'Delete source' })); await waitFor(() => expect(mockDelete).toHaveBeenCalledWith('/api/energy/sources/{source_uuid}', { params: { path: { source_uuid: 's1' } } })); expect(await screen.findByText(/No sources configured/)).toBeInTheDocument(); expect(screen.queryByRole('button', { name: 'Delete source' })).not.toBeInTheDocument() })
|
||||
it('keeps source visible and explains dependencies when deletion returns 409', async () => { const user = userEvent.setup(); mockGet.mockImplementation((path: string) => Promise.resolve({ data: path === '/api/energy/sources' ? { items: [source] } : path.includes('channels') ? { items: [] } : source })); const { ApiError } = await import('../api/client'); mockDelete.mockRejectedValue(new ApiError(409, { detail: 'dependent readings' })); renderWithProviders(<SourceManager />); await user.click(await screen.findByText('WarmteLink')); await user.click(screen.getByRole('button', { name: 'Delete source' })); expect(await screen.findByText(/dependent channels, readings, or meter bindings/)).toBeInTheDocument(); expect(screen.getByRole('button', { name: 'WarmteLink' })).toBeInTheDocument() })
|
||||
})
|
||||
@@ -0,0 +1,34 @@
|
||||
import { useState } from 'react'
|
||||
import { Alert, Badge, Button, Center, Group, Loader, Paper, ScrollArea, Stack, Table, Text } from '@mantine/core'
|
||||
import { DsmrPanel } from './DsmrPanel'
|
||||
import { SourceForm } from './SourceForm'
|
||||
import { SourceReadings } from './SourceReadings'
|
||||
import { ApiError } from '../api/client'
|
||||
import { useDeleteSource, useDiscoverSource, useSource, useSourceChannels, useSources, type MeterSourceResponse } from './hooks'
|
||||
import { formatLocalDateTime } from '../utils/datetime'
|
||||
|
||||
export function SourceManager() {
|
||||
const sources = useSources(); const [selected, setSelected] = useState<string | null>(null); const [form, setForm] = useState<MeterSourceResponse | undefined | null>(null)
|
||||
if (sources.isLoading) return <Center data-testid="sources-loading"><Loader /></Center>
|
||||
if (sources.isError || !sources.data) return <Alert color="red">Failed to load sources.</Alert>
|
||||
return <Stack data-testid="source-manager"><Group justify="space-between"><Text fw={600}>Sources</Text><Button onClick={() => setForm(undefined)}>New Source</Button></Group>
|
||||
{sources.data.items.length === 0 ? <Text c="dimmed">No sources configured yet.</Text> : <ScrollArea data-testid="sources-table-scrollarea"><Table data-testid="sources-table" style={{ minWidth: 640 }}><Table.Thead><Table.Tr><Table.Th>Name</Table.Th><Table.Th>Type</Table.Th><Table.Th>Status</Table.Th><Table.Th>Last seen</Table.Th></Table.Tr></Table.Thead><Table.Tbody>{sources.data.items.map((source) => <Table.Tr key={source.uuid}><Table.Td><Button variant="subtle" onClick={() => setSelected(source.uuid)}>{source.name}</Button></Table.Td><Table.Td>{source.kind}</Table.Td><Table.Td><Badge color={source.enabled && source.status === 'online' ? 'green' : 'gray'}>{source.enabled ? source.status : 'disabled'}</Badge>{source.enabled && source.status === 'error' && <Text c="orange" size="xs">Reconnecting automatically.</Text>}{source.last_error && <Text c="red" size="xs">{source.last_error}</Text>}</Table.Td><Table.Td>{source.last_seen_at ? formatLocalDateTime(source.last_seen_at) : '—'}</Table.Td></Table.Tr>)}</Table.Tbody></Table></ScrollArea>}
|
||||
{selected && <SourceDetail uuid={selected} onEdit={setForm} onDeleted={() => setSelected(null)} />}{form !== null && <SourceForm source={form} onClose={() => setForm(null)} />}
|
||||
</Stack>
|
||||
}
|
||||
function SourceDetail({ uuid, onEdit, onDeleted }: { uuid: string; onEdit: (source: MeterSourceResponse) => void; onDeleted: () => void }) {
|
||||
const source = useSource(uuid); const channels = useSourceChannels(uuid); const discover = useDiscoverSource(); const remove = useDeleteSource(); const [deleteError, setDeleteError] = useState<string | null>(null)
|
||||
async function deleteSource() { setDeleteError(null); try { await remove.mutateAsync(uuid); onDeleted() } catch (err) { if (err instanceof ApiError && err.status === 409) setDeleteError('This source cannot be deleted because it still has dependent channels, readings, or meter bindings. Remove those dependencies first; no data was deleted.'); else setDeleteError('Could not delete source. No data was deleted.') } }
|
||||
if (source.isLoading) return <Loader />; if (source.isError || !source.data) return <Alert color="red">Failed to load source.</Alert>
|
||||
const detail = source.data!
|
||||
const result = discover.data?.data
|
||||
return <Paper withBorder p="md"><Stack><Group justify="space-between"><Text fw={600}>{detail.name}</Text><Group><Button variant="default" onClick={() => onEdit(detail)}>Edit</Button><Button loading={discover.isPending} onClick={() => discover.mutate(uuid)}>Refresh discovered channels</Button><Button color="red" variant="outline" loading={remove.isPending} onClick={deleteSource}>Delete source</Button></Group></Group>
|
||||
{deleteError && <Alert color="red">{deleteError}</Alert>}
|
||||
{detail.enabled && detail.status === 'error' && <Alert color="orange">The worker is reconnecting automatically. Refresh only requests a bounded status update; it does not start ingestion.</Alert>}
|
||||
{discover.isPending && <Alert color="blue">Discovery pending…</Alert>}{discover.isError && <Alert color="red">Discovery request failed. Check the source and try again.</Alert>}
|
||||
{result && <Alert color={result.status === 'error' || result.status === 'timeout' ? 'red' : 'blue'}>Discovery {result.status}: {result.detail ?? (result.status === 'completed' ? 'Channels refreshed.' : 'Waiting for discovery.')}</Alert>}
|
||||
{detail.kind.includes('dsmr') && <DsmrPanel />}
|
||||
{channels.isLoading && <Loader />}{channels.isError && <Alert color="red">Failed to load channels.</Alert>}{channels.data?.items.length === 0 && <Text c="dimmed">No channels discovered yet.</Text>}
|
||||
{channels.data?.items.map((channel) => <Stack key={channel.uuid} gap="xs"><Text>{channel.label} ({channel.unit}) — suggestion: {channel.suggested_commodity ?? 'none'} (review before binding)</Text><Text size="sm">Latest: {channel.latest_value ?? '—'}; quality: {channel.latest_quality ?? 'unknown'}; bindings: {channel.binding_count}; meter IDs: {channel.bound_meter_ids.length ? channel.bound_meter_ids.join(', ') : 'none'}</Text><SourceReadings sourceUuid={uuid} channel={channel} /></Stack>)}
|
||||
</Stack></Paper>
|
||||
}
|
||||
@@ -0,0 +1,13 @@
|
||||
import { describe, expect, it, vi, beforeEach } from 'vitest'
|
||||
import { screen, waitFor } from '@testing-library/react'
|
||||
import { renderWithProviders } from '../test-utils'
|
||||
import { SourceReadings } from './SourceReadings'
|
||||
const mockGet = vi.fn()
|
||||
vi.mock('../api/client', () => ({ default: { GET: (...a: unknown[]) => mockGet(...a), POST: vi.fn(), PATCH: vi.fn(), DELETE: vi.fn() }, ApiError: class extends Error {}, registerLoginRedirect: vi.fn() }))
|
||||
const channel = { uuid: 'channel-1', label: 'Heat', unit: 'GJ', latest_value: '1.2', latest_quality: 'unverifiable' }
|
||||
describe('SourceReadings', () => {
|
||||
beforeEach(() => vi.clearAllMocks())
|
||||
it('renders quality and history from the mocked channel API', async () => { mockGet.mockResolvedValue({ data: { items: [{ recorded_at: '2026-08-01T10:00:00Z', value: '1.1', quality: 'unverifiable' }] } }); renderWithProviders(<SourceReadings sourceUuid="s1" channel={channel as never} />); expect(await screen.findByText(/shown for review, not marked verified/)).toBeInTheDocument(); await waitFor(() => expect(screen.getByText('1.1')).toBeInTheDocument()) })
|
||||
it('renders an API error and empty history', async () => { mockGet.mockRejectedValueOnce(new Error('down')); const { unmount } = renderWithProviders(<SourceReadings sourceUuid="s1" channel={channel as never} />); expect(await screen.findByText('Failed to load channel history.')).toBeInTheDocument(); unmount(); mockGet.mockResolvedValue({ data: { items: [] } }); renderWithProviders(<SourceReadings sourceUuid="s1" channel={channel as never} />); await waitFor(() => expect(screen.getByText('No channel history yet.')).toBeInTheDocument()) })
|
||||
it('keeps the latest quality explanation visible while history is loading or fails', async () => { mockGet.mockImplementationOnce(() => new Promise(() => {})); const { unmount } = renderWithProviders(<SourceReadings sourceUuid="s1" channel={channel as never} />); expect(screen.getByText('Quality: unverifiable')).toBeInTheDocument(); expect(screen.getByText(/shown for review, not marked verified/)).toBeInTheDocument(); unmount(); mockGet.mockRejectedValueOnce(new Error('down')); renderWithProviders(<SourceReadings sourceUuid="s1" channel={channel as never} />); expect(await screen.findByText('Failed to load channel history.')).toBeInTheDocument(); expect(screen.getByText('Quality: unverifiable')).toBeInTheDocument(); expect(screen.getByText(/shown for review, not marked verified/)).toBeInTheDocument() })
|
||||
})
|
||||
@@ -0,0 +1,12 @@
|
||||
import { Alert, Badge, Center, Loader, Stack, Table, Text } from '@mantine/core'
|
||||
import { useChannelReadings, type MeterSourceChannelResponse } from './hooks'
|
||||
import { formatLocalDateTime } from '../utils/datetime'
|
||||
|
||||
export function SourceReadings({ sourceUuid, channel }: { sourceUuid: string; channel: MeterSourceChannelResponse }) {
|
||||
const query = useChannelReadings(sourceUuid, channel.uuid)
|
||||
return <Stack gap="xs"><Text fw={500}>{channel.label} — latest {channel.latest_value ?? '—'} {channel.unit}</Text>
|
||||
<Text size="sm">Quality: {channel.latest_quality ?? 'unknown'}</Text>
|
||||
{channel.latest_quality === 'unverifiable' && <Alert color="yellow">This reading is unverifiable: it is shown for review, not marked verified.</Alert>}
|
||||
{query.isLoading ? <Center><Loader /></Center> : query.isError || !query.data ? <Alert color="red">Failed to load channel history.</Alert> : query.data.items.length === 0 ? <Text c="dimmed">No channel history yet.</Text> : <Table><Table.Thead><Table.Tr><Table.Th>Recorded</Table.Th><Table.Th>Value</Table.Th><Table.Th>Quality</Table.Th></Table.Tr></Table.Thead><Table.Tbody>{query.data.items.map((row) => <Table.Tr key={row.recorded_at}><Table.Td>{formatLocalDateTime(row.recorded_at)}</Table.Td><Table.Td>{row.value ?? '—'}</Table.Td><Table.Td><Badge color={row.quality === 'unverifiable' ? 'yellow' : 'gray'}>{row.quality ?? 'unknown'}</Badge></Table.Td></Table.Tr>)}</Table.Tbody></Table>}
|
||||
</Stack>
|
||||
}
|
||||
@@ -6,10 +6,15 @@
|
||||
* 2. Empty state (no active contract / no kind).
|
||||
* 3. Renders tibber chart when tibber kind data is available.
|
||||
* 4. Shows tariff table for manual kind.
|
||||
* 5. Marks the currently active price slot (dot + caption).
|
||||
* 6. Hovering past midnight resolves tomorrow's slot, not today's (regression).
|
||||
* 7. buildChartRows: unique X keys across midnight.
|
||||
* 8. findActiveSlotIndex: which slot is currently active.
|
||||
*/
|
||||
|
||||
import { describe, it, expect, vi, beforeEach } from 'vitest'
|
||||
import { screen, waitFor } from '@testing-library/react'
|
||||
import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest'
|
||||
import { screen, waitFor, fireEvent } from '@testing-library/react'
|
||||
import userEvent from '@testing-library/user-event'
|
||||
import { renderWithProviders } from '../test-utils'
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
@@ -42,7 +47,107 @@ vi.mock('../api/client', () => ({
|
||||
// Import component
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
import { TibberPrices } from './TibberPrices'
|
||||
import { TibberPrices, buildChartRows, findActiveSlotIndex } from './TibberPrices'
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Chart size harness
|
||||
//
|
||||
// jsdom reports every element as 0x0, so Recharts renders an empty plot and no
|
||||
// pointer interaction is possible. These helpers hand the chart a fixed size:
|
||||
// - the ResponsiveContainer gets 800x300 from its bounding rect + a ResizeObserver
|
||||
// that reports the same size,
|
||||
// - the chart wrapper reports 800x260 (the height the component asks for), which
|
||||
// is what Recharts uses to translate clientX/clientY into chart coordinates,
|
||||
// - everything else stays 0x0 so the legend does not eat the whole plot area.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const CHART_W = 800
|
||||
const CONTAINER_H = 300
|
||||
const CHART_H = 260
|
||||
|
||||
function fakeRect(width: number, height: number): DOMRect {
|
||||
return {
|
||||
x: 0,
|
||||
y: 0,
|
||||
left: 0,
|
||||
top: 0,
|
||||
right: width,
|
||||
bottom: height,
|
||||
width,
|
||||
height,
|
||||
toJSON: () => {},
|
||||
} as DOMRect
|
||||
}
|
||||
|
||||
const originalResizeObserver = globalThis.ResizeObserver
|
||||
const offsetWidthDescriptor = Object.getOwnPropertyDescriptor(
|
||||
HTMLElement.prototype,
|
||||
'offsetWidth',
|
||||
)
|
||||
const offsetHeightDescriptor = Object.getOwnPropertyDescriptor(
|
||||
HTMLElement.prototype,
|
||||
'offsetHeight',
|
||||
)
|
||||
|
||||
function installChartSize() {
|
||||
vi.spyOn(Element.prototype, 'getBoundingClientRect').mockImplementation(function (this: Element) {
|
||||
if (this.classList.contains('recharts-responsive-container')) {
|
||||
return fakeRect(CHART_W, CONTAINER_H)
|
||||
}
|
||||
if (this.classList.contains('recharts-wrapper')) return fakeRect(CHART_W, CHART_H)
|
||||
return fakeRect(0, 0)
|
||||
})
|
||||
|
||||
// Recharts divides rect size by offset size to undo CSS transform scaling;
|
||||
// matching them keeps the scale factor at 1.
|
||||
Object.defineProperty(HTMLElement.prototype, 'offsetWidth', {
|
||||
configurable: true,
|
||||
value: CHART_W,
|
||||
})
|
||||
Object.defineProperty(HTMLElement.prototype, 'offsetHeight', {
|
||||
configurable: true,
|
||||
value: CHART_H,
|
||||
})
|
||||
|
||||
globalThis.ResizeObserver = class implements ResizeObserver {
|
||||
private readonly cb: ResizeObserverCallback
|
||||
constructor(cb: ResizeObserverCallback) {
|
||||
this.cb = cb
|
||||
}
|
||||
observe() {
|
||||
this.cb(
|
||||
[{ contentRect: { width: CHART_W, height: CONTAINER_H } } as ResizeObserverEntry],
|
||||
this,
|
||||
)
|
||||
}
|
||||
unobserve() {}
|
||||
disconnect() {}
|
||||
}
|
||||
}
|
||||
|
||||
function restoreChartSize() {
|
||||
globalThis.ResizeObserver = originalResizeObserver
|
||||
if (offsetWidthDescriptor) {
|
||||
Object.defineProperty(HTMLElement.prototype, 'offsetWidth', offsetWidthDescriptor)
|
||||
}
|
||||
if (offsetHeightDescriptor) {
|
||||
Object.defineProperty(HTMLElement.prototype, 'offsetHeight', offsetHeightDescriptor)
|
||||
}
|
||||
}
|
||||
|
||||
/** Hourly price points, one per hour starting at `startUtc`, with unique prices. */
|
||||
function hourlyPoints(startUtc: number, count: number) {
|
||||
return Array.from({ length: count }, (_, i) => ({
|
||||
starts_at: new Date(startUtc + i * 3600_000).toISOString(),
|
||||
buy: 0.1 + i / 1000,
|
||||
sell: 0.05 + i / 1000,
|
||||
level: 'NORMAL',
|
||||
}))
|
||||
}
|
||||
|
||||
function tooltipText(): string {
|
||||
return document.querySelector('.recharts-tooltip-wrapper')?.textContent ?? ''
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Tests
|
||||
@@ -50,6 +155,10 @@ import { TibberPrices } from './TibberPrices'
|
||||
|
||||
describe('TibberPrices', () => {
|
||||
beforeEach(() => vi.clearAllMocks())
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks()
|
||||
restoreChartSize()
|
||||
})
|
||||
|
||||
it('renders loading state initially', () => {
|
||||
mockGet.mockImplementation(() => new Promise(() => {}))
|
||||
@@ -135,4 +244,210 @@ describe('TibberPrices', () => {
|
||||
expect(screen.getByTestId('tariff-sell-normal')).toHaveTextContent('0.0900')
|
||||
expect(screen.getByTestId('tariff-sell-dal')).toHaveTextContent('0.0900')
|
||||
})
|
||||
|
||||
it('keeps thermal prices scoped and renders the complete D11 Decimal snapshot as a table', async () => {
|
||||
const user = userEvent.setup()
|
||||
const thermal = {
|
||||
kind: 'district_heating', currency: 'EUR', points: [], tariff: null,
|
||||
contract_version_id: 42, effective_from: '2026-01-01T00:00:00Z', effective_to: '2026-12-31T00:00:00Z',
|
||||
values: {
|
||||
variable: {
|
||||
heating: '20.123456789123456789', hot_water_heating: '8.200000000000000001',
|
||||
hot_water: '1.234567890123456789', hot_water_tax: '0.456789012345678901',
|
||||
},
|
||||
standing: {
|
||||
heating_network: '100.000000000000000001', metering: '0', delivery_set: '20.2',
|
||||
hot_water_network: '30.3', other: '40.4',
|
||||
},
|
||||
},
|
||||
}
|
||||
mockGet.mockImplementation((_path: string, options?: { params?: { query?: { scope?: string } } }) =>
|
||||
Promise.resolve({ data: options?.params?.query?.scope === 'thermal'
|
||||
? thermal
|
||||
: { kind: 'manual', currency: 'EUR', points: [], tariff: { buy_dal: 0.1, buy_normal: 0.2, sell_dal: 0.03, sell_normal: 0.04 } } }),
|
||||
)
|
||||
|
||||
renderWithProviders(<TibberPrices />)
|
||||
await waitFor(() => expect(screen.getByTestId('manual-tariff-table')).toBeInTheDocument())
|
||||
await user.click(screen.getByText('Thermal'))
|
||||
await waitFor(() => expect(screen.getByTestId('thermal-price-snapshot')).toBeInTheDocument())
|
||||
|
||||
const snapshot = screen.getByTestId('thermal-price-snapshot')
|
||||
const table = screen.getByTestId('thermal-price-table')
|
||||
expect(snapshot).toHaveTextContent('Version 42')
|
||||
expect(snapshot).toHaveTextContent('effective 2026-01-01T00:00:00Z to 2026-12-31T00:00:00Z')
|
||||
expect(snapshot).toHaveTextContent('not a 15-minute market spot price')
|
||||
expect(screen.getAllByRole('columnheader').map((header) => header.textContent)).toEqual([
|
||||
'Category', 'Charge', 'Rate', 'Unit',
|
||||
])
|
||||
expect(table.querySelectorAll('tbody tr')).toHaveLength(9)
|
||||
expect(screen.getByTestId('thermal-price-row-variable-heating')).toHaveTextContent('VariableHeating20.123456789123456789EUR/GJ')
|
||||
expect(screen.getByTestId('thermal-price-row-variable-hot_water_heating')).toHaveTextContent('Hot Water Heating8.200000000000000001EUR/m³')
|
||||
expect(screen.getByTestId('thermal-price-row-standing-heating_network')).toHaveTextContent('StandingHeating Network100.000000000000000001EUR/year')
|
||||
for (const value of Object.values(thermal.values.standing)) {
|
||||
expect(table).toHaveTextContent(value)
|
||||
}
|
||||
expect(mockGet).toHaveBeenCalledWith('/api/energy/prices', expect.objectContaining({
|
||||
params: { query: expect.objectContaining({ scope: 'thermal' }) },
|
||||
}))
|
||||
|
||||
await user.click(screen.getByText('Electricity'))
|
||||
await waitFor(() => expect(screen.getByTestId('manual-tariff-table')).toBeInTheDocument())
|
||||
expect(screen.queryByTestId('thermal-price-snapshot')).not.toBeInTheDocument()
|
||||
})
|
||||
|
||||
it('keeps unknown thermal snapshot fields with safe human-readable labels', async () => {
|
||||
const user = userEvent.setup()
|
||||
mockGet.mockImplementation((_path: string, options?: { params?: { query?: { scope?: string } } }) => Promise.resolve({
|
||||
data: options?.params?.query?.scope === 'thermal'
|
||||
? { kind: 'district_heating', currency: 'EUR', values: { future_fee: { experimental_charge: '1.000000000000000001' } } }
|
||||
: { kind: 'manual', currency: 'EUR', points: [], tariff: { buy_dal: 0.1, buy_normal: 0.2, sell_dal: 0.03, sell_normal: 0.04 } },
|
||||
}))
|
||||
renderWithProviders(<TibberPrices />)
|
||||
await user.click(await screen.findByText('Thermal'))
|
||||
const row = await screen.findByTestId('thermal-price-row-future_fee-experimental_charge')
|
||||
expect(row).toHaveTextContent('Future FeeExperimental Charge1.000000000000000001EUR')
|
||||
})
|
||||
|
||||
it('marks the currently active price slot with a dot and a caption', async () => {
|
||||
installChartSize()
|
||||
|
||||
const SLOT_MS = 15 * 60 * 1000
|
||||
// Start of the quarter-hour slot that contains "now".
|
||||
const currentSlot = Math.floor(Date.now() / SLOT_MS) * SLOT_MS
|
||||
|
||||
mockGet.mockResolvedValue({
|
||||
data: {
|
||||
kind: 'tibber',
|
||||
currency: 'EUR',
|
||||
points: [
|
||||
{ starts_at: new Date(currentSlot - SLOT_MS).toISOString(), buy: 0.11, sell: 0.05 },
|
||||
{ starts_at: new Date(currentSlot).toISOString(), buy: 0.2431, sell: 0.1102 },
|
||||
{ starts_at: new Date(currentSlot + SLOT_MS).toISOString(), buy: 0.31, sell: 0.15 },
|
||||
],
|
||||
tariff: null,
|
||||
},
|
||||
})
|
||||
|
||||
renderWithProviders(<TibberPrices />)
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId('tibber-current-price')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
const marker = screen.getByTestId('tibber-current-price')
|
||||
expect(marker).toHaveTextContent('0.2431')
|
||||
expect(marker).toHaveTextContent('0.1102')
|
||||
|
||||
// One dot on the buy line, one on the sell line — visible without hovering.
|
||||
await waitFor(() => {
|
||||
expect(document.querySelectorAll('.recharts-reference-dot')).toHaveLength(2)
|
||||
})
|
||||
})
|
||||
|
||||
it('resolves the hovered slot past midnight to tomorrow, not today', async () => {
|
||||
installChartSize()
|
||||
|
||||
// 26 hourly points starting at 2020-01-01T00:00Z, so "00:00" and "01:00"
|
||||
// each appear twice. Fixed past dates keep the "now" marker out of range.
|
||||
const points = hourlyPoints(Date.UTC(2020, 0, 1), 26)
|
||||
|
||||
mockGet.mockResolvedValue({
|
||||
data: { kind: 'tibber', currency: 'EUR', points, tariff: null },
|
||||
})
|
||||
|
||||
renderWithProviders(<TibberPrices />)
|
||||
|
||||
await waitFor(() => expect(screen.getByTestId('tibber-chart')).toBeInTheDocument())
|
||||
expect(screen.queryByTestId('tibber-current-price')).not.toBeInTheDocument()
|
||||
|
||||
const wrapper = document.querySelector('.recharts-wrapper')
|
||||
expect(wrapper).not.toBeNull()
|
||||
|
||||
// Right edge of the plot area = the last slot (day 2, 01:00, buy 0.1250).
|
||||
fireEvent.mouseMove(wrapper!, { clientX: 770, clientY: CHART_H / 2 })
|
||||
|
||||
await waitFor(() => expect(tooltipText()).toContain('0.1250'))
|
||||
|
||||
// Label carries the date, so day 2 is distinguishable from day 1.
|
||||
expect(tooltipText()).toContain('1/2/2020')
|
||||
expect(tooltipText()).toContain('0.0750')
|
||||
|
||||
// The active dots must sit on the hovered point (right half of the plot).
|
||||
// The bug put them on day 1's identically-labelled slot near the left edge.
|
||||
const dots = Array.from(document.querySelectorAll('.recharts-active-dot circle'))
|
||||
expect(dots).toHaveLength(2)
|
||||
for (const dot of dots) {
|
||||
expect(Number(dot.getAttribute('cx'))).toBeGreaterThan(CHART_W / 2)
|
||||
}
|
||||
})
|
||||
})
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// buildChartRows
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('buildChartRows', () => {
|
||||
it('keeps X-axis keys unique across midnight', () => {
|
||||
// Same local time-of-day on two consecutive days: as "HH:mm" labels these
|
||||
// collided, which made Recharts resolve the hovered point to the first match
|
||||
// (today) instead of the hovered one (tomorrow).
|
||||
const rows = buildChartRows([
|
||||
{ starts_at: '2026-07-26T22:00:00Z', buy: 0.1, sell: 0.05 },
|
||||
{ starts_at: '2026-07-27T22:00:00Z', buy: 0.2, sell: 0.06 },
|
||||
])
|
||||
|
||||
expect(rows).toHaveLength(2)
|
||||
expect(new Set(rows.map((r) => r.ts)).size).toBe(2)
|
||||
})
|
||||
|
||||
it('sorts rows by slot start and parses naive timestamps as UTC', () => {
|
||||
const rows = buildChartRows([
|
||||
{ starts_at: '2026-07-27T02:00:00', buy: 0.3, sell: 0.07 },
|
||||
{ starts_at: '2026-07-27T01:00:00Z', buy: 0.2, sell: 0.06 },
|
||||
{ starts_at: '2026-07-27T00:00:00Z', buy: 0.1, sell: 0.05 },
|
||||
])
|
||||
|
||||
expect(rows.map((r) => r.buy)).toEqual([0.1, 0.2, 0.3])
|
||||
expect(rows.map((r) => r.ts)).toEqual([
|
||||
'2026-07-27T00:00:00.000Z',
|
||||
'2026-07-27T01:00:00.000Z',
|
||||
'2026-07-27T02:00:00.000Z',
|
||||
])
|
||||
})
|
||||
})
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// findActiveSlotIndex
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('findActiveSlotIndex', () => {
|
||||
const rows = buildChartRows([
|
||||
{ starts_at: '2026-07-27T00:00:00Z', buy: 0.1, sell: 0.05 },
|
||||
{ starts_at: '2026-07-27T00:15:00Z', buy: 0.2, sell: 0.06 },
|
||||
{ starts_at: '2026-07-27T00:30:00Z', buy: 0.3, sell: 0.07 },
|
||||
])
|
||||
|
||||
const at = (iso: string) => new Date(iso).getTime()
|
||||
|
||||
it('returns the slot containing now', () => {
|
||||
expect(findActiveSlotIndex(rows, at('2026-07-27T00:20:00Z'))).toBe(1)
|
||||
})
|
||||
|
||||
it('returns the slot at its exact start boundary', () => {
|
||||
expect(findActiveSlotIndex(rows, at('2026-07-27T00:15:00Z'))).toBe(1)
|
||||
})
|
||||
|
||||
it('returns null before the first slot', () => {
|
||||
expect(findActiveSlotIndex(rows, at('2026-07-26T23:59:00Z'))).toBeNull()
|
||||
})
|
||||
|
||||
it('stays on the last slot until its inferred end, then returns null', () => {
|
||||
expect(findActiveSlotIndex(rows, at('2026-07-27T00:44:00Z'))).toBe(2)
|
||||
expect(findActiveSlotIndex(rows, at('2026-07-27T00:45:00Z'))).toBeNull()
|
||||
})
|
||||
|
||||
it('returns null for empty data', () => {
|
||||
expect(findActiveSlotIndex([], Date.now())).toBeNull()
|
||||
})
|
||||
})
|
||||
|
||||
@@ -2,13 +2,15 @@
|
||||
* TibberPrices — price curve visualization.
|
||||
*
|
||||
* - Fetches today + tomorrow price range using useEnergyPrices.
|
||||
* - For tibber kind: Recharts LineChart showing buy/sell prices over time.
|
||||
* - For tibber kind: Recharts LineChart showing buy/sell prices over time,
|
||||
* with the currently active price slot marked by a dot.
|
||||
* - For manual kind: shows tariff table (buy_dal, buy_normal, sell_dal, sell_normal).
|
||||
* - Handles: no active contract, empty data, loading, error.
|
||||
*
|
||||
* Recharts imports are isolated to this file only.
|
||||
*/
|
||||
|
||||
import { useEffect, useMemo, useState } from 'react'
|
||||
import {
|
||||
Stack,
|
||||
Text,
|
||||
@@ -20,6 +22,8 @@ import {
|
||||
Badge,
|
||||
Group,
|
||||
Paper,
|
||||
SegmentedControl,
|
||||
ScrollArea,
|
||||
} from '@mantine/core'
|
||||
import {
|
||||
LineChart,
|
||||
@@ -29,10 +33,46 @@ import {
|
||||
CartesianGrid,
|
||||
Tooltip,
|
||||
Legend,
|
||||
ReferenceDot,
|
||||
ResponsiveContainer,
|
||||
} from 'recharts'
|
||||
import { useEnergyPrices } from './hooks'
|
||||
import { formatLocalTime } from '../utils/datetime'
|
||||
import { useQuery } from '@tanstack/react-query'
|
||||
import apiClient from '../api/client'
|
||||
import { formatLocalDate, formatLocalTime, parseBackendTimestamp } from '../utils/datetime'
|
||||
|
||||
const BUY_COLOR = '#2196f3'
|
||||
const SELL_COLOR = '#4caf50'
|
||||
|
||||
/** Slot length assumed for the very last point, when no next point bounds it. */
|
||||
const FALLBACK_SLOT_MS = 60 * 60 * 1000
|
||||
|
||||
/** How often the "current price" marker re-evaluates which slot is active. */
|
||||
const NOW_TICK_MS = 30 * 1000
|
||||
|
||||
/** D11 thermal profile units. The API snapshot is Decimal strings, while the
|
||||
* currency comes from its contract metadata. Keep this display-only: no rates
|
||||
* are derived or prefilled in the browser. */
|
||||
function thermalUnit(section: string, key: string, currency: string): string {
|
||||
if (section === 'standing') return `${currency}/year`
|
||||
if (key === 'heating') return `${currency}/GJ`
|
||||
if (key === 'hot_water_heating' || key === 'hot_water' || key === 'hot_water_tax') {
|
||||
return `${currency}/m³`
|
||||
}
|
||||
return currency
|
||||
}
|
||||
|
||||
function humanizeThermalField(value: string): string {
|
||||
return value
|
||||
.replace(/_/g, ' ')
|
||||
.replace(/\b\w/g, (letter) => letter.toUpperCase())
|
||||
}
|
||||
|
||||
function thermalSections(values: Record<string, Record<string, string>>): string[] {
|
||||
const preferred = ['variable', 'standing']
|
||||
return [...preferred.filter((section) => section in values), ...Object.keys(values)
|
||||
.filter((section) => !preferred.includes(section))
|
||||
.sort()]
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Time range helpers
|
||||
@@ -51,34 +91,121 @@ function getTomorrowEnd(): string {
|
||||
return d.toISOString()
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Chart data helpers
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface PricePoint {
|
||||
starts_at: string
|
||||
buy: number
|
||||
sell: number
|
||||
level?: string | null
|
||||
}
|
||||
|
||||
export interface ChartRow {
|
||||
/**
|
||||
* X-axis category key — the full instant, NOT a "HH:mm" label.
|
||||
*
|
||||
* Must be unique per slot: Recharts resolves the hovered point by *value*
|
||||
* (findEntryInArray on the axis dataKey), so a repeated key makes the tooltip
|
||||
* and the active dot snap back to the first match. With "HH:mm" labels, every
|
||||
* time of day appears twice in a today+tomorrow range, which pinned the dot on
|
||||
* today once the cursor passed midnight. Formatting to HH:mm happens in the
|
||||
* tick / tooltip formatters instead.
|
||||
*/
|
||||
ts: string
|
||||
/** Slot start as epoch ms; NaN when starts_at is unparseable. */
|
||||
tsMs: number
|
||||
buy: number
|
||||
sell: number
|
||||
}
|
||||
|
||||
/** Map API price points to chart rows with unique X keys, sorted by slot start. */
|
||||
export function buildChartRows(points: PricePoint[]): ChartRow[] {
|
||||
return points
|
||||
.map((p) => {
|
||||
const d = parseBackendTimestamp(p.starts_at)
|
||||
const tsMs = d.getTime()
|
||||
return {
|
||||
ts: Number.isFinite(tsMs) ? d.toISOString() : p.starts_at,
|
||||
tsMs,
|
||||
buy: p.buy,
|
||||
sell: p.sell,
|
||||
}
|
||||
})
|
||||
.sort((a, b) => {
|
||||
// Unparseable timestamps sort last so the ascending scan below can stop early.
|
||||
if (!Number.isFinite(a.tsMs)) return Number.isFinite(b.tsMs) ? 1 : 0
|
||||
if (!Number.isFinite(b.tsMs)) return -1
|
||||
return a.tsMs - b.tsMs
|
||||
})
|
||||
}
|
||||
|
||||
/**
|
||||
* Index of the row whose slot contains `nowMs`, or null when now is outside the
|
||||
* fetched range. A slot ends where the next one starts; the last row has no next
|
||||
* slot, so it falls back to the series spacing (quarter-hourly for Tibber).
|
||||
*/
|
||||
export function findActiveSlotIndex(rows: ChartRow[], nowMs: number): number | null {
|
||||
let idx = -1
|
||||
for (let i = 0; i < rows.length; i += 1) {
|
||||
if (!Number.isFinite(rows[i].tsMs) || rows[i].tsMs > nowMs) break
|
||||
idx = i
|
||||
}
|
||||
if (idx < 0) return null
|
||||
|
||||
const spacing = rows.length > 1 ? rows[1].tsMs - rows[0].tsMs : NaN
|
||||
const slotMs = Number.isFinite(spacing) && spacing > 0 ? spacing : FALLBACK_SLOT_MS
|
||||
const slotEnd = idx + 1 < rows.length ? rows[idx + 1].tsMs : rows[idx].tsMs + slotMs
|
||||
return nowMs < slotEnd ? idx : null
|
||||
}
|
||||
|
||||
/** Ticking clock so the active-slot marker follows slot boundaries while open. */
|
||||
function useNowMs(intervalMs = NOW_TICK_MS): number {
|
||||
const [now, setNow] = useState(() => Date.now())
|
||||
useEffect(() => {
|
||||
const id = setInterval(() => setNow(Date.now()), intervalMs)
|
||||
return () => clearInterval(id)
|
||||
}, [intervalMs])
|
||||
return now
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Tibber chart
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
interface TibberChartProps {
|
||||
points: Array<{ starts_at: string; buy: number; sell: number; level?: string | null }>
|
||||
points: PricePoint[]
|
||||
currency: string
|
||||
}
|
||||
|
||||
function TibberChart({ points, currency }: TibberChartProps) {
|
||||
const data = points.map((p) => ({
|
||||
time: formatLocalTime(p.starts_at),
|
||||
buy: p.buy,
|
||||
sell: p.sell,
|
||||
}))
|
||||
const data = useMemo(() => buildChartRows(points), [points])
|
||||
const nowMs = useNowMs()
|
||||
const activeIndex = findActiveSlotIndex(data, nowMs)
|
||||
const activeRow = activeIndex == null ? null : data[activeIndex]
|
||||
|
||||
return (
|
||||
<Stack gap="xs" data-testid="tibber-chart">
|
||||
<Title order={6} c="dimmed">
|
||||
Price curve ({currency})
|
||||
</Title>
|
||||
<Group gap="xs" justify="space-between" align="baseline">
|
||||
<Title order={6} c="dimmed">
|
||||
Price curve ({currency})
|
||||
</Title>
|
||||
{activeRow && (
|
||||
<Text size="xs" c="dimmed" data-testid="tibber-current-price">
|
||||
Now {formatLocalTime(activeRow.ts)} · buy {activeRow.buy.toFixed(4)} · sell{' '}
|
||||
{activeRow.sell.toFixed(4)}
|
||||
</Text>
|
||||
)}
|
||||
</Group>
|
||||
<ResponsiveContainer width="100%" height={260}>
|
||||
<LineChart data={data} margin={{ top: 4, right: 16, left: 0, bottom: 4 }}>
|
||||
<CartesianGrid strokeDasharray="3 3" />
|
||||
<XAxis
|
||||
dataKey="time"
|
||||
dataKey="ts"
|
||||
tick={{ fontSize: 10 }}
|
||||
interval="preserveStartEnd"
|
||||
tickFormatter={(v: string) => formatLocalTime(v)}
|
||||
/>
|
||||
<YAxis
|
||||
tick={{ fontSize: 10 }}
|
||||
@@ -89,12 +216,17 @@ function TibberChart({ points, currency }: TibberChartProps) {
|
||||
formatter={(val: any) =>
|
||||
[`${typeof val === 'number' ? val.toFixed(4) : String(val)} ${currency}`, undefined]
|
||||
}
|
||||
labelFormatter={(label) =>
|
||||
typeof label === 'string'
|
||||
? `${formatLocalDate(label)} ${formatLocalTime(label)}`
|
||||
: label
|
||||
}
|
||||
/>
|
||||
<Legend />
|
||||
<Line
|
||||
type="monotone"
|
||||
dataKey="buy"
|
||||
stroke="#2196f3"
|
||||
stroke={BUY_COLOR}
|
||||
dot={false}
|
||||
strokeWidth={2}
|
||||
name="Buy"
|
||||
@@ -102,11 +234,32 @@ function TibberChart({ points, currency }: TibberChartProps) {
|
||||
<Line
|
||||
type="monotone"
|
||||
dataKey="sell"
|
||||
stroke="#4caf50"
|
||||
stroke={SELL_COLOR}
|
||||
dot={false}
|
||||
strokeWidth={2}
|
||||
name="Sell"
|
||||
/>
|
||||
{/* Currently active price slot, marked by default (no hover needed). */}
|
||||
{activeRow && (
|
||||
<ReferenceDot
|
||||
x={activeRow.ts}
|
||||
y={activeRow.buy}
|
||||
r={4}
|
||||
fill={BUY_COLOR}
|
||||
stroke="#fff"
|
||||
strokeWidth={2}
|
||||
/>
|
||||
)}
|
||||
{activeRow && (
|
||||
<ReferenceDot
|
||||
x={activeRow.ts}
|
||||
y={activeRow.sell}
|
||||
r={4}
|
||||
fill={SELL_COLOR}
|
||||
stroke="#fff"
|
||||
strokeWidth={2}
|
||||
/>
|
||||
)}
|
||||
</LineChart>
|
||||
</ResponsiveContainer>
|
||||
</Stack>
|
||||
@@ -163,46 +316,59 @@ function ManualTariffTable({ tariff, currency }: ManualTariffTableProps) {
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export function TibberPrices() {
|
||||
const [scope, setScope] = useState<'electricity' | 'thermal'>('electricity')
|
||||
const start = getTodayStart()
|
||||
const end = getTomorrowEnd()
|
||||
const { data, isLoading, isError } = useQuery({
|
||||
queryKey: ['energy-prices', scope, start, end],
|
||||
queryFn: async () => {
|
||||
const res = await apiClient.GET('/api/energy/prices', { params: { query: { scope, start, end } } })
|
||||
return res.data
|
||||
},
|
||||
})
|
||||
|
||||
const { data, isLoading, isError } = useEnergyPrices(start, end)
|
||||
const selector = (
|
||||
<SegmentedControl
|
||||
value={scope}
|
||||
onChange={(value) => setScope(value as 'electricity' | 'thermal')}
|
||||
data={[{ label: 'Electricity', value: 'electricity' }, { label: 'Thermal', value: 'thermal' }]}
|
||||
data-testid="prices-scope-selector"
|
||||
/>
|
||||
)
|
||||
|
||||
if (isLoading) {
|
||||
return (
|
||||
<Center py="xl" data-testid="prices-loading">
|
||||
<Loader />
|
||||
</Center>
|
||||
<Stack><Group><Text fw={500}>Energy Prices</Text>{selector}</Group><Center py="xl" data-testid="prices-loading"><Loader /></Center></Stack>
|
||||
)
|
||||
}
|
||||
|
||||
if (isError) {
|
||||
return (
|
||||
<Alert color="red" data-testid="prices-error">
|
||||
<Stack><Group><Text fw={500}>Energy Prices</Text>{selector}</Group><Alert color="red" data-testid="prices-error">
|
||||
Failed to load energy prices. Please refresh.
|
||||
</Alert>
|
||||
</Alert></Stack>
|
||||
)
|
||||
}
|
||||
|
||||
if (!data) {
|
||||
return (
|
||||
<Alert color="gray" data-testid="prices-no-data">
|
||||
<Stack><Group><Text fw={500}>Energy Prices</Text>{selector}</Group><Alert color="gray" data-testid="prices-no-data">
|
||||
No pricing data available.
|
||||
</Alert>
|
||||
</Alert></Stack>
|
||||
)
|
||||
}
|
||||
|
||||
// No active contract
|
||||
if (!data.kind) {
|
||||
return (
|
||||
<Paper withBorder p="md" data-testid="prices-no-contract">
|
||||
<Stack><Group><Text fw={500}>Energy Prices</Text>{selector}</Group><Paper withBorder p="md" data-testid="prices-no-contract">
|
||||
<Stack gap="xs">
|
||||
<Text fw={500}>No active contract</Text>
|
||||
<Text size="sm" c="dimmed">
|
||||
Activate an energy contract on the Contracts tab to see pricing data.
|
||||
</Text>
|
||||
</Stack>
|
||||
</Paper>
|
||||
</Paper></Stack>
|
||||
)
|
||||
}
|
||||
|
||||
@@ -212,6 +378,7 @@ export function TibberPrices() {
|
||||
<Stack gap="lg" data-testid="tibber-prices">
|
||||
<Group gap="sm" align="center">
|
||||
<Text fw={500}>Energy Prices</Text>
|
||||
{selector}
|
||||
<Badge variant="outline" size="sm">
|
||||
{data.kind}
|
||||
</Badge>
|
||||
@@ -239,6 +406,38 @@ export function TibberPrices() {
|
||||
Manual tariff data not available.
|
||||
</Alert>
|
||||
)}
|
||||
|
||||
{scope === 'thermal' && data.values && (
|
||||
<Paper withBorder p="md" data-testid="thermal-price-snapshot">
|
||||
<Stack gap="xs">
|
||||
<Text fw={500}>Thermal contract snapshot</Text>
|
||||
<Text size="sm">Version {data.contract_version_id ?? '—'} · effective {data.effective_from ?? '—'} to {data.effective_to ?? 'open'}</Text>
|
||||
<Text size="sm" c="dimmed">This is a contract snapshot, not a 15-minute market spot price.</Text>
|
||||
<ScrollArea type="auto">
|
||||
<Table withTableBorder withColumnBorders data-testid="thermal-price-table">
|
||||
<Table.Thead>
|
||||
<Table.Tr>
|
||||
<Table.Th>Category</Table.Th>
|
||||
<Table.Th>Charge</Table.Th>
|
||||
<Table.Th>Rate</Table.Th>
|
||||
<Table.Th>Unit</Table.Th>
|
||||
</Table.Tr>
|
||||
</Table.Thead>
|
||||
<Table.Tbody>
|
||||
{thermalSections(data.values!).flatMap((section) => Object.entries(data.values![section]).map(([key, rate]) => (
|
||||
<Table.Tr key={`${section}-${key}`} data-testid={`thermal-price-row-${section}-${key}`}>
|
||||
<Table.Td>{humanizeThermalField(section)}</Table.Td>
|
||||
<Table.Td>{humanizeThermalField(key)}</Table.Td>
|
||||
<Table.Td>{rate}</Table.Td>
|
||||
<Table.Td>{thermalUnit(section, key, currency)}</Table.Td>
|
||||
</Table.Tr>
|
||||
)))}
|
||||
</Table.Tbody>
|
||||
</Table>
|
||||
</ScrollArea>
|
||||
</Stack>
|
||||
</Paper>
|
||||
)}
|
||||
</Stack>
|
||||
)
|
||||
}
|
||||
|
||||
@@ -83,6 +83,8 @@ const SUMMARY = {
|
||||
metered_import: 10.5,
|
||||
metered_export: 2.3,
|
||||
metered_net: 8.2,
|
||||
metered_import_kwh: 33.3,
|
||||
metered_export_kwh: 44.4,
|
||||
fixed_costs: 5.0,
|
||||
credits: 50.0,
|
||||
total_payable: 12.5,
|
||||
@@ -148,6 +150,17 @@ describe('useCreateContract', () => {
|
||||
|
||||
expect(mockPost).toHaveBeenCalledWith('/api/energy/contracts', { body })
|
||||
})
|
||||
|
||||
it('preserves the caller-selected thermal scope in the typed create payload', async () => {
|
||||
mockPost.mockResolvedValue({ data: { id: 9 } })
|
||||
const { Wrapper } = makeWrapper()
|
||||
const { useCreateContract } = await import('./hooks')
|
||||
const { result } = renderHook(() => useCreateContract(), { wrapper: Wrapper })
|
||||
await act(async () => {
|
||||
await result.current.mutateAsync({ name: 'Heat', kind: 'district_heating', scope: 'thermal', currency: 'EUR', values: {} })
|
||||
})
|
||||
expect(mockPost).toHaveBeenCalledWith('/api/energy/contracts', expect.objectContaining({ body: expect.objectContaining({ scope: 'thermal' }) }))
|
||||
})
|
||||
})
|
||||
|
||||
describe('useEnergyPrices', () => {
|
||||
|
||||
@@ -81,6 +81,7 @@ function makeWrapper() {
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('useDevices', () => {
|
||||
// Source hooks use the same typed client and QueryClient invalidation boundary.
|
||||
beforeEach(() => vi.clearAllMocks())
|
||||
|
||||
it('calls GET /api/modbus/devices and returns device list', async () => {
|
||||
@@ -98,6 +99,57 @@ describe('useDevices', () => {
|
||||
})
|
||||
})
|
||||
|
||||
describe('useDeclareMeter source binding invalidation', () => {
|
||||
beforeEach(() => vi.clearAllMocks())
|
||||
it('invalidates every cache affected by atomic meter and binding creation', async () => {
|
||||
mockPost.mockResolvedValue({ data: { id: 9 } })
|
||||
const { qc, Wrapper } = makeWrapper()
|
||||
const affectedKeys = [
|
||||
['energy-meters'], ['energy-source-channels'], ['energy-meter-bindings', 9],
|
||||
['energy-sources'], ['energy-source', 'source-1'], ['energy-channel-readings', 'source-1', 'channel-1'],
|
||||
['expose-catalog'], ['energy-costs', 'electricity'], ['energy-costs-summary', 'electricity'],
|
||||
['meter-costs', 'thermal', 'month'], ['meter-cost-summary', 'thermal'],
|
||||
]
|
||||
for (const queryKey of affectedKeys) qc.setQueryData(queryKey, { cached: true })
|
||||
const { useDeclareMeter } = await import('./hooks')
|
||||
const { result } = renderHook(() => useDeclareMeter(), { wrapper: Wrapper })
|
||||
await act(async () => { await result.current.mutateAsync({ label: 'Heat', commodity: 'heating', started_at: '2026-08-01T00:00:00Z', reason: 'initial', source_channel_uuid: 'channel-1' } as never) })
|
||||
expect(mockPost).toHaveBeenCalledWith('/api/energy/meters', expect.objectContaining({ body: expect.objectContaining({ source_channel_uuid: 'channel-1' }) }))
|
||||
for (const queryKey of affectedKeys) expect(qc.getQueryState(queryKey)?.isInvalidated).toBe(true)
|
||||
})
|
||||
})
|
||||
|
||||
describe('meter lifecycle mutations', () => {
|
||||
beforeEach(() => vi.clearAllMocks())
|
||||
|
||||
it.each([
|
||||
['Declare auto-handoff', 'post', async (hooks: typeof import('./hooks')) => hooks.useDeclareMeter, { label: 'swap', commodity: 'electricity', started_at: '2026-08-24T12:34', reason: 'meter_swap' }],
|
||||
['Close meter', 'post', async (hooks: typeof import('./hooks')) => hooks.useCloseMeter, { id: 9, ended_at: '2026-08-24T12:34' }],
|
||||
['Unbind', 'patch', async (hooks: typeof import('./hooks')) => hooks.useCloseBinding, { uuid: 'old', ended_at: '2026-08-24T12:34' }],
|
||||
['Transfer', 'post', async (hooks: typeof import('./hooks')) => hooks.useTransferBinding, { id: 10, body: { from_binding_uuid: 'old', to_source_channel_uuid: 'new', effective_at: '2026-08-24T12:34' } }],
|
||||
['Direct bind', 'post', async (hooks: typeof import('./hooks')) => hooks.useCreateBinding, { id: 10, body: { source_channel_uuid: 'new', started_at: '2026-08-24T12:34' } }],
|
||||
['Update electricity start', 'patch', async (hooks: typeof import('./hooks')) => hooks.useUpdateMeter, { id: 10, body: { started_at: '2026-08-24T12:34' } }],
|
||||
['Update heating label', 'patch', async (hooks: typeof import('./hooks')) => hooks.useUpdateMeter, { id: 11, body: { label: 'Heating meter' } }],
|
||||
['Update hot water start', 'patch', async (hooks: typeof import('./hooks')) => hooks.useUpdateMeter, { id: 12, body: { started_at: '2026-08-24T12:34' } }],
|
||||
])('%s invalidates every lifecycle view using a fresh QueryClient', async (_name, method, getHook, payload) => {
|
||||
mockPost.mockResolvedValue({ data: {} })
|
||||
mockPatch.mockResolvedValue({ data: {} })
|
||||
const { qc, Wrapper } = makeWrapper()
|
||||
const affectedKeys = [
|
||||
['energy-meters'], ['energy-sources'], ['energy-source', 'source-1'], ['energy-source-channels'], ['energy-meter-bindings'],
|
||||
['energy-channel-readings', 'source-1', 'channel-1'], ['energy-costs', 'electricity'], ['energy-costs-summary', 'electricity'],
|
||||
['meter-costs', 'thermal', 'month'], ['meter-cost-summary', 'thermal'], ['expose-catalog'],
|
||||
]
|
||||
for (const key of affectedKeys) qc.setQueryData(key, { cached: true })
|
||||
const hooks = await import('./hooks')
|
||||
const useHook = await getHook(hooks)
|
||||
const result = renderHook(() => useHook(), { wrapper: Wrapper })
|
||||
await act(async () => { await result.result.current.mutateAsync(payload as never) })
|
||||
expect(method === 'post' ? mockPost : mockPatch).toHaveBeenCalled()
|
||||
for (const key of affectedKeys) expect(qc.getQueryState(key)?.isInvalidated).toBe(true)
|
||||
})
|
||||
})
|
||||
|
||||
describe('useProfiles', () => {
|
||||
beforeEach(() => vi.clearAllMocks())
|
||||
|
||||
|
||||
@@ -237,6 +237,15 @@ export type SummaryResponse = components['schemas']['SummaryResponse']
|
||||
export type DsmrLatestResponse = components['schemas']['DsmrLatestResponse']
|
||||
export type TibberTestResponse = components['schemas']['TibberTestResponse']
|
||||
export type TibberTestPriceSchema = components['schemas']['TibberTestPriceSchema']
|
||||
export type SourceProfileResponse = components['schemas']['SourceProfileResponse']
|
||||
export type MeterSourceResponse = components['schemas']['MeterSourceResponse']
|
||||
export type MeterSourceCreate = components['schemas']['MeterSourceCreate']
|
||||
export type MeterSourcePatch = components['schemas']['MeterSourcePatch']
|
||||
export type MeterSourceChannelResponse = components['schemas']['MeterSourceChannelResponse']
|
||||
export type BindingResponse = components['schemas']['BindingResponse']
|
||||
export type BindingCreate = components['schemas']['BindingCreate']
|
||||
export type BindingTransferRequest = components['schemas']['BindingTransferRequest']
|
||||
export type MeterCloseRequest = components['schemas']['MeterCloseRequest']
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Query: list all energy contracts
|
||||
@@ -470,12 +479,9 @@ export function useDeclareMeter() {
|
||||
return useMutation({
|
||||
mutationFn: (body: MeterDeclareRequest) =>
|
||||
apiClient.POST('/api/energy/meters', { body }),
|
||||
onSuccess: () => {
|
||||
void qc.invalidateQueries({ queryKey: ['energy-meters'] })
|
||||
// Invalidate cost-related queries: a new meter may trigger recompute server-side.
|
||||
void qc.invalidateQueries({ queryKey: ['energy-costs'] })
|
||||
void qc.invalidateQueries({ queryKey: ['energy-costs-summary'] })
|
||||
},
|
||||
// A meter_swap can hand a binding over even when the optional channel was
|
||||
// omitted from this request, so every lifecycle write shares this boundary.
|
||||
onSuccess: () => invalidateLifecycleQueries(qc),
|
||||
})
|
||||
}
|
||||
|
||||
@@ -491,15 +497,48 @@ export function useUpdateMeter() {
|
||||
params: { path: { meter_id: id } },
|
||||
body,
|
||||
}),
|
||||
onSuccess: () => {
|
||||
void qc.invalidateQueries({ queryKey: ['energy-meters'] })
|
||||
// Retroactive started_at correction triggers recompute server-side.
|
||||
void qc.invalidateQueries({ queryKey: ['energy-costs'] })
|
||||
void qc.invalidateQueries({ queryKey: ['energy-costs-summary'] })
|
||||
},
|
||||
onSuccess: () => invalidateLifecycleQueries(qc),
|
||||
})
|
||||
}
|
||||
|
||||
// Source → channel → binding hooks. These deliberately use the generated
|
||||
// OpenAPI types; UI suggestions remain just suggestions until a user binds one.
|
||||
export function useSourceProfiles() {
|
||||
return useQuery({ queryKey: ['energy-source-profiles'], queryFn: async () => {
|
||||
const res = await apiClient.GET('/api/energy/source-profiles'); return res.data
|
||||
}, staleTime: 5 * 60 * 1000 })
|
||||
}
|
||||
export function useSources() {
|
||||
return useQuery({ queryKey: ['energy-sources'], queryFn: async () => {
|
||||
const res = await apiClient.GET('/api/energy/sources'); return res.data
|
||||
}, refetchInterval: 5_000 })
|
||||
}
|
||||
export function useSource(uuid: string | null) {
|
||||
return useQuery({ queryKey: ['energy-source', uuid], enabled: !!uuid, queryFn: async () => {
|
||||
const res = await apiClient.GET('/api/energy/sources/{source_uuid}', { params: { path: { source_uuid: uuid! } } }); return res.data
|
||||
}, refetchInterval: 3_000 })
|
||||
}
|
||||
function invalidateLifecycleQueries(qc: ReturnType<typeof useQueryClient>) {
|
||||
void qc.invalidateQueries({ queryKey: ['energy-sources'] }); void qc.invalidateQueries({ queryKey: ['energy-source'] });
|
||||
void qc.invalidateQueries({ queryKey: ['energy-source-channels'] }); void qc.invalidateQueries({ queryKey: ['energy-meters'] });
|
||||
void qc.invalidateQueries({ queryKey: ['energy-channel-readings'] }); void qc.invalidateQueries({ queryKey: ['energy-meter-bindings'] })
|
||||
void qc.invalidateQueries({ queryKey: ['expose-catalog'] })
|
||||
void qc.invalidateQueries({ queryKey: ['energy-costs'] }); void qc.invalidateQueries({ queryKey: ['energy-costs-summary'] })
|
||||
void qc.invalidateQueries({ queryKey: ['meter-costs', 'thermal'] })
|
||||
void qc.invalidateQueries({ queryKey: ['meter-cost-summary', 'thermal'] })
|
||||
}
|
||||
export function useCreateSource() { const qc = useQueryClient(); return useMutation({ mutationFn: (body: MeterSourceCreate) => apiClient.POST('/api/energy/sources', { body }), onSuccess: () => invalidateLifecycleQueries(qc) }) }
|
||||
export function useUpdateSource() { const qc = useQueryClient(); return useMutation({ mutationFn: ({ uuid, body }: { uuid: string; body: MeterSourcePatch }) => apiClient.PATCH('/api/energy/sources/{source_uuid}', { params: { path: { source_uuid: uuid } }, body }), onSuccess: () => invalidateLifecycleQueries(qc) }) }
|
||||
export function useDeleteSource() { const qc = useQueryClient(); return useMutation({ mutationFn: (uuid: string) => apiClient.DELETE('/api/energy/sources/{source_uuid}', { params: { path: { source_uuid: uuid } } }), onSuccess: () => invalidateLifecycleQueries(qc) }) }
|
||||
export function useDiscoverSource() { const qc = useQueryClient(); return useMutation({ mutationFn: (uuid: string) => apiClient.POST('/api/energy/sources/{source_uuid}/discover', { params: { path: { source_uuid: uuid } } }), onSuccess: () => invalidateLifecycleQueries(qc) }) }
|
||||
export function useSourceChannels(uuid: string | null) { return useQuery({ queryKey: ['energy-source-channels', uuid], enabled: !!uuid, queryFn: async () => { const res = await apiClient.GET('/api/energy/sources/{source_uuid}/channels', { params: { path: { source_uuid: uuid! } } }); return res.data }, refetchInterval: 3_000 }) }
|
||||
export function useChannelReadings(sourceUuid: string | null, channelUuid: string | null) { return useQuery({ queryKey: ['energy-channel-readings', sourceUuid, channelUuid], enabled: !!sourceUuid && !!channelUuid, queryFn: async () => { const res = await apiClient.GET('/api/energy/sources/{source_uuid}/channels/{channel_uuid}/readings', { params: { path: { source_uuid: sourceUuid!, channel_uuid: channelUuid! }, query: { limit: 60 } } }); return res.data }, refetchInterval: 5_000 }) }
|
||||
export function useMeterBindings(id: number | null) { return useQuery({ queryKey: ['energy-meter-bindings', id], enabled: id != null, queryFn: async () => { const res = await apiClient.GET('/api/energy/meters/{meter_id}/bindings', { params: { path: { meter_id: id! } } }); return res.data } }) }
|
||||
export function useCreateBinding() { const qc = useQueryClient(); return useMutation({ mutationFn: ({ id, body }: { id: number; body: BindingCreate }) => apiClient.POST('/api/energy/meters/{meter_id}/bindings', { params: { path: { meter_id: id } }, body }), onSuccess: () => invalidateLifecycleQueries(qc) }) }
|
||||
export function useCloseBinding() { const qc = useQueryClient(); return useMutation({ mutationFn: ({ uuid, ended_at }: { uuid: string; ended_at: string }) => apiClient.PATCH('/api/energy/bindings/{binding_uuid}', { params: { path: { binding_uuid: uuid } }, body: { ended_at } }), onSuccess: () => invalidateLifecycleQueries(qc) }) }
|
||||
export function useCloseMeter() { const qc = useQueryClient(); return useMutation({ mutationFn: ({ id, ended_at }: { id: number; ended_at: string }) => apiClient.POST('/api/energy/meters/{meter_id}/close', { params: { path: { meter_id: id } }, body: { ended_at } satisfies MeterCloseRequest }), onSuccess: () => invalidateLifecycleQueries(qc) }) }
|
||||
export function useTransferBinding() { const qc = useQueryClient(); return useMutation({ mutationFn: ({ id, body }: { id: number; body: BindingTransferRequest }) => apiClient.POST('/api/energy/meters/{meter_id}/bindings/transfer', { params: { path: { meter_id: id } }, body }), onSuccess: () => invalidateLifecycleQueries(qc) }) }
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Query: time-range readings for a device (window + limit — never full-table)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
@@ -15,6 +15,7 @@
|
||||
|
||||
import { describe, it, expect, vi, beforeEach } from 'vitest'
|
||||
import { screen, waitFor, fireEvent } from '@testing-library/react'
|
||||
import userEvent from '@testing-library/user-event'
|
||||
import { renderWithProviders } from '../test-utils'
|
||||
import { EnergyPage } from './EnergyPage'
|
||||
|
||||
@@ -135,6 +136,19 @@ describe('EnergyPage — device list', () => {
|
||||
})
|
||||
})
|
||||
|
||||
describe('EnergyPage — M8 navigation labels', () => {
|
||||
beforeEach(() => { vi.clearAllMocks(); setupDefaultMocks() })
|
||||
it('has keyboard-operable Sources, Modbus Devices and Meters tabs, without a DSMR tab', async () => {
|
||||
const user = userEvent.setup(); renderEnergy()
|
||||
const sources = screen.getByRole('tab', { name: 'Sources' })
|
||||
expect(screen.getByRole('tab', { name: 'Modbus Devices' })).toBeInTheDocument()
|
||||
expect(screen.getByRole('tab', { name: 'Meters' })).toBeInTheDocument()
|
||||
expect(screen.queryByRole('tab', { name: /^DSMR$/ })).not.toBeInTheDocument()
|
||||
sources.focus(); await user.keyboard('{Enter}')
|
||||
expect(await screen.findByTestId('panel-sources')).toBeInTheDocument()
|
||||
})
|
||||
})
|
||||
|
||||
describe('EnergyPage — create device', () => {
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks()
|
||||
|
||||
@@ -45,7 +45,7 @@ import { ContractManager } from '../energy/ContractManager'
|
||||
import { MeterManager } from '../energy/MeterManager'
|
||||
import { TibberPrices } from '../energy/TibberPrices'
|
||||
import { CostView } from '../energy/CostView'
|
||||
import { DsmrPanel } from '../energy/DsmrPanel'
|
||||
import { SourceManager } from '../energy/SourceManager'
|
||||
import type { ModbusDevice, ModbusTestReadResponse, MetricInfo } from '../energy/hooks'
|
||||
import { ApiError } from '../api/client'
|
||||
import { formatMetricValue } from '../energy/format'
|
||||
@@ -603,7 +603,7 @@ function DevicesTab() {
|
||||
return (
|
||||
<Stack gap="lg">
|
||||
<Group justify="space-between" align="center">
|
||||
<Title order={2}>Energy — Devices</Title>
|
||||
<Title order={2}>Energy — Modbus Devices</Title>
|
||||
<Button onClick={openCreate} data-testid="device-new-button">
|
||||
New Device
|
||||
</Button>
|
||||
@@ -659,8 +659,11 @@ export function EnergyPage() {
|
||||
<Container size="xl" pt="xl" pb="xl" data-testid="energy-page">
|
||||
<Tabs defaultValue="devices">
|
||||
<Tabs.List mb="lg">
|
||||
<Tabs.Tab value="sources" data-testid="tab-sources">
|
||||
Sources
|
||||
</Tabs.Tab>
|
||||
<Tabs.Tab value="devices" data-testid="tab-devices">
|
||||
Devices
|
||||
Modbus Devices
|
||||
</Tabs.Tab>
|
||||
<Tabs.Tab value="meters" data-testid="tab-meters">
|
||||
Meters
|
||||
@@ -674,11 +677,12 @@ export function EnergyPage() {
|
||||
<Tabs.Tab value="costs" data-testid="tab-costs">
|
||||
Costs
|
||||
</Tabs.Tab>
|
||||
<Tabs.Tab value="dsmr" data-testid="tab-dsmr">
|
||||
DSMR
|
||||
</Tabs.Tab>
|
||||
</Tabs.List>
|
||||
|
||||
<Tabs.Panel value="sources" data-testid="panel-sources">
|
||||
<SourceManager />
|
||||
</Tabs.Panel>
|
||||
|
||||
<Tabs.Panel value="devices" data-testid="panel-devices">
|
||||
<DevicesTab />
|
||||
</Tabs.Panel>
|
||||
@@ -699,9 +703,6 @@ export function EnergyPage() {
|
||||
<CostView />
|
||||
</Tabs.Panel>
|
||||
|
||||
<Tabs.Panel value="dsmr" data-testid="panel-dsmr">
|
||||
<DsmrPanel />
|
||||
</Tabs.Panel>
|
||||
</Tabs>
|
||||
</Container>
|
||||
)
|
||||
|
||||
+2367
-7
File diff suppressed because it is too large
Load Diff
+1536
-11
File diff suppressed because it is too large
Load Diff
@@ -7,6 +7,7 @@ paho-mqtt>=2.0,<3.0
|
||||
pymodbus>=3.6,<4.0
|
||||
pydantic-settings>=2.6,<3.0
|
||||
pyotp>=2.9,<3.0
|
||||
pyserial>=3.5,<4.0
|
||||
python-multipart>=0.0.12,<1.0
|
||||
pyyaml>=6.0,<7.0
|
||||
sqlalchemy>=2.0,<3.0
|
||||
|
||||
@@ -65,6 +65,8 @@ pymodbus==3.13.1
|
||||
# via -r requirements.in
|
||||
pyotp==2.10.0
|
||||
# via -r requirements.in
|
||||
pyserial==3.5
|
||||
# via -r requirements.in
|
||||
python-dotenv==1.2.2
|
||||
# via
|
||||
# pydantic-settings
|
||||
|
||||
@@ -15,7 +15,7 @@ if str(PROJECT_ROOT) not in sys.path:
|
||||
|
||||
from app.config import get_settings
|
||||
|
||||
APP_BASELINE_REVISION = "20260625_14_meter_uuid"
|
||||
APP_BASELINE_REVISION = "20260822_19_meter_cost_periods"
|
||||
|
||||
|
||||
class AppDatabaseAdoptionError(RuntimeError):
|
||||
|
||||
@@ -131,6 +131,7 @@ def cmd_read(args: argparse.Namespace) -> None:
|
||||
|
||||
print(f"Profile : {profile.name} — {profile.description}")
|
||||
print(f"Gateway : {host}:{port} unit_id={unit_id}")
|
||||
print(f"Function : FC{profile.function_code:02d}")
|
||||
print(f"Blocks : {[(b.start, b.count) for b in profile.blocks]}")
|
||||
print()
|
||||
|
||||
@@ -141,7 +142,7 @@ def cmd_read(args: argparse.Namespace) -> None:
|
||||
from app.integrations.modbus.driver import read_blocks
|
||||
|
||||
try:
|
||||
registers = read_blocks(host, port, unit_id, blocks)
|
||||
registers = read_blocks(host, port, unit_id, blocks, function_code=profile.function_code)
|
||||
except ModbusConnectionError as exc:
|
||||
print(f"Connection error: {exc}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
@@ -0,0 +1,214 @@
|
||||
"""Read-only parser and command-line probe for DSMR and WarmteLink P1 telegrams."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import errno
|
||||
import sys
|
||||
import time
|
||||
from typing import BinaryIO, Callable, TextIO
|
||||
|
||||
import serial
|
||||
from app.integrations.p1 import IntegrityStatus, ObisField, P1Telegram, TelegramFramer, parse_telegram
|
||||
|
||||
__all__ = ["IntegrityStatus", "TelegramFramer", "build_parser", "parse_telegram", "run_probe"]
|
||||
|
||||
|
||||
def build_parser() -> argparse.ArgumentParser:
|
||||
"""Build the CLI parser for an explicitly selected, read-only serial device."""
|
||||
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Read-only WarmteLink P1 serial probe; it never writes to the device.",
|
||||
epilog="Defaults are the locally measured WarmteLink 115200 7N1 framing, not DSMR defaults.",
|
||||
)
|
||||
parser.add_argument("--device", required=True, help="Explicit serial path, preferably /dev/serial/by-id/..."
|
||||
)
|
||||
parser.add_argument(
|
||||
"--baudrate",
|
||||
type=int,
|
||||
default=115200,
|
||||
help="Baud rate (default: 115200, the locally measured WarmteLink value).",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--bytesize",
|
||||
type=int,
|
||||
choices=(5, 6, 7, 8),
|
||||
default=7,
|
||||
help="Data bits (default: 7, locally measured; not the DSMR standard default).",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--parity",
|
||||
choices=("N", "E", "O", "M", "S"),
|
||||
type=lambda value: value.upper(),
|
||||
default="N",
|
||||
help="Parity (default: N, locally measured; not the DSMR standard default).",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--stopbits",
|
||||
type=float,
|
||||
choices=(1, 1.5, 2),
|
||||
default=1,
|
||||
help="Stop bits (default: 1, locally measured; not the DSMR standard default).",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--duration",
|
||||
type=_positive_duration,
|
||||
default=600.0,
|
||||
help="Maximum capture time in seconds (default: 600).",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--show-changes",
|
||||
action="store_true",
|
||||
help="After the first frame, print only OBIS fields whose values changed.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--raw-output",
|
||||
type=argparse.FileType("wb"),
|
||||
help="Optional path for the exact raw bytes read from the serial device.",
|
||||
)
|
||||
return parser
|
||||
|
||||
|
||||
def _positive_duration(value: str) -> float:
|
||||
try:
|
||||
duration = float(value)
|
||||
except ValueError as exc:
|
||||
raise argparse.ArgumentTypeError("duration must be a positive number") from exc
|
||||
if duration <= 0:
|
||||
raise argparse.ArgumentTypeError("duration must be greater than zero")
|
||||
return duration
|
||||
|
||||
|
||||
def _serial_error_message(exc: BaseException) -> str:
|
||||
"""Return a practical, non-root diagnostic for a serial open/read failure."""
|
||||
|
||||
error_number = getattr(exc, "errno", None)
|
||||
message = str(exc)
|
||||
if error_number == errno.EACCES or "permission denied" in message.lower():
|
||||
return f"serial permission denied: {message}. Add your user to the dialout group; do not run it as root."
|
||||
if error_number == errno.EBUSY or "resource busy" in message.lower():
|
||||
return f"serial device is busy: {message}. Close the program currently using this device and retry."
|
||||
if error_number in {errno.ENODEV, errno.ENOENT, errno.EIO}:
|
||||
return f"serial device disconnected or unavailable: {message}. Check the cable and --device path."
|
||||
return f"serial I/O failed: {message}. Check the cable, device path, and serial framing settings."
|
||||
|
||||
|
||||
def _format_field(field: ObisField) -> str:
|
||||
raw_values = ", ".join(field.raw_values) or "<no values>"
|
||||
value = f" parsed={field.value} {field.unit or ''}" if field.value is not None else ""
|
||||
return f" {field.code}: {raw_values}{value}".rstrip()
|
||||
|
||||
|
||||
def _comparison_values(field: ObisField) -> tuple[str, ...]:
|
||||
"""Return a local change-detection token without exposing identifiers."""
|
||||
|
||||
return (field.comparison_token,) if field.comparison_token is not None else field.raw_values
|
||||
|
||||
|
||||
def _print_telegram(
|
||||
telegram: P1Telegram,
|
||||
frame_number: int,
|
||||
cadence: float | None,
|
||||
previous_fields: dict[str, tuple[str, ...]],
|
||||
show_changes: bool,
|
||||
output: TextIO,
|
||||
) -> dict[str, tuple[str, ...]]:
|
||||
cadence_text = "first frame" if cadence is None else f"cadence={cadence:.1f}s"
|
||||
print(
|
||||
f"frame {frame_number}: {telegram.integrity.value}; bytes={telegram.frame_length}; {cadence_text}",
|
||||
file=output,
|
||||
)
|
||||
print(f" integrity: {telegram.integrity_reason}", file=output)
|
||||
current_fields = {field.code: _comparison_values(field) for field in telegram.fields}
|
||||
fields = telegram.fields
|
||||
if show_changes and previous_fields:
|
||||
fields = tuple(field for field in fields if previous_fields.get(field.code) != _comparison_values(field))
|
||||
print(f" changed fields: {len(fields)}", file=output)
|
||||
for field in fields:
|
||||
print(_format_field(field), file=output)
|
||||
for channel in telegram.channels:
|
||||
print(
|
||||
f" channel {channel.number}: device_type={channel.device_type or '<unknown>'}; "
|
||||
f"readings={len(channel.readings)}",
|
||||
file=output,
|
||||
)
|
||||
return current_fields
|
||||
|
||||
|
||||
SerialFactory = Callable[..., serial.Serial]
|
||||
|
||||
|
||||
def run_probe(
|
||||
args: argparse.Namespace,
|
||||
*,
|
||||
serial_factory: SerialFactory = serial.Serial,
|
||||
clock: Callable[[], float] = time.monotonic,
|
||||
output: TextIO = sys.stdout,
|
||||
error_output: TextIO = sys.stderr,
|
||||
) -> int:
|
||||
"""Capture and report frames until duration elapses or the user interrupts.
|
||||
|
||||
The only operation on ``serial_port`` is ``read``. It is always closed,
|
||||
including after an interrupt, timeout, or a read error.
|
||||
"""
|
||||
|
||||
raw_output: BinaryIO | None = args.raw_output
|
||||
serial_port: serial.Serial | None = None
|
||||
try:
|
||||
serial_port = serial_factory(
|
||||
port=args.device,
|
||||
baudrate=args.baudrate,
|
||||
bytesize=args.bytesize,
|
||||
parity=args.parity,
|
||||
stopbits=args.stopbits,
|
||||
timeout=1,
|
||||
)
|
||||
framer = TelegramFramer()
|
||||
deadline = clock() + args.duration
|
||||
frame_number = 0
|
||||
last_frame_at: float | None = None
|
||||
previous_fields: dict[str, tuple[str, ...]] = {}
|
||||
while clock() < deadline:
|
||||
chunk = serial_port.read(4096)
|
||||
if not chunk:
|
||||
continue
|
||||
if raw_output is not None:
|
||||
raw_output.write(chunk)
|
||||
raw_output.flush()
|
||||
for frame in framer.feed(chunk):
|
||||
now = clock()
|
||||
telegram = parse_telegram(frame)
|
||||
frame_number += 1
|
||||
cadence = None if last_frame_at is None else now - last_frame_at
|
||||
previous_fields = _print_telegram(
|
||||
telegram,
|
||||
frame_number,
|
||||
cadence,
|
||||
previous_fields,
|
||||
args.show_changes,
|
||||
output,
|
||||
)
|
||||
last_frame_at = now
|
||||
return 0
|
||||
except KeyboardInterrupt:
|
||||
print("capture interrupted; serial device closed", file=output)
|
||||
return 0
|
||||
except (serial.SerialException, OSError) as exc:
|
||||
print(_serial_error_message(exc), file=error_output)
|
||||
return 1
|
||||
finally:
|
||||
if serial_port is not None:
|
||||
serial_port.close()
|
||||
if raw_output is not None:
|
||||
raw_output.close()
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
"""Run the command-line probe."""
|
||||
|
||||
args = build_parser().parse_args(argv)
|
||||
return run_probe(args)
|
||||
|
||||
|
||||
if __name__ == "__main__": # pragma: no cover - exercised through main()
|
||||
raise SystemExit(main())
|
||||
Vendored
+12
@@ -0,0 +1,12 @@
|
||||
/ISk5\2MT382-1000
|
||||
|
||||
1-3:0.2.8(50)
|
||||
0-0:1.0.0(240822120000S)
|
||||
0-0:96.1.1(TEST-GATEWAY)
|
||||
0-1:24.1.0(006)
|
||||
0-1:96.1.0(TEST-DHW)
|
||||
0-1:24.2.1(240822120000S)(5.900*m3)
|
||||
0-2:24.1.0(012)
|
||||
0-2:96.1.0(TEST-HEAT)
|
||||
0-2:24.2.1(240822120000S)(0.017*GJ)
|
||||
!D18A
|
||||
Vendored
+11
@@ -0,0 +1,11 @@
|
||||
)TU)2NWA-MYRSKY
|
||||
0-2:24.2.1(240822120000S)(0.017*GJ)
|
||||
0-1:96.1.0(KAM-REDACTED)
|
||||
0-0:1.0.0(240822120000S)
|
||||
0-2:24.1.0(012)
|
||||
0-1:24.2.1(240822120000S)(5.900*m3)
|
||||
0-0:96.1.1(WARMTE-REDACTED)
|
||||
0-1:24.1.0(006)
|
||||
0-2:96.1.0(KAM-REDACTED)
|
||||
1-3:0.2.8(50)
|
||||
!x7z?
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user