Compare commits
85
Commits
bf7fd71a21
..
v1.4.2
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
b65f700d56 | ||
|
|
f4cea3874b | ||
|
|
134f0abb5f | ||
|
|
3e04b15656 | ||
|
|
f2e8f6a8e7 | ||
|
|
90a03e7fd6 | ||
|
|
d3fc90b320 | ||
|
|
d4acfc438a | ||
|
|
c26160b10b | ||
|
|
efbe36d7c0 | ||
|
|
f663981cdb | ||
|
|
da05fd2f09 | ||
|
|
188168b16a | ||
|
|
f1e7ce3133 | ||
|
|
e3d0d17cac | ||
|
|
33f5de0591 | ||
|
|
839faaf95b | ||
|
|
82bb329500 | ||
|
|
17057ed41e | ||
|
|
227b146c2c | ||
|
|
32a20785ee | ||
|
|
2544514f52 | ||
|
|
96e85fb48e | ||
|
|
b71009620a | ||
|
|
8b91146c29 | ||
|
|
682e06d256 | ||
|
|
d7f0731d1c | ||
|
|
9bc46f8d2d | ||
|
|
6f8cb05eab | ||
|
|
effe434637 | ||
|
|
9d3e28a529 | ||
|
|
7e61b0a938 | ||
|
|
e8521351d7 | ||
|
|
8d4f496ff4 | ||
|
|
78f3cc4776 | ||
|
|
96e88861d4 | ||
|
|
57f2459f60 | ||
|
|
c61fc2d4ba | ||
|
|
d35ac7d752 | ||
|
|
ef48032adb | ||
|
|
7e266a21eb | ||
|
|
4284bd40a7 | ||
|
|
bedea196c3 | ||
|
|
df54f5518b | ||
|
|
18fe18281b | ||
|
|
355c8eb8b8 | ||
|
|
1ffdaebbbd | ||
|
|
c64693b8e5 | ||
|
|
c89d083942 | ||
|
|
6090b98ab0 | ||
|
|
489cd7df3d | ||
|
|
35cdc7f22a | ||
|
|
75d685f04d | ||
|
|
935e68846c | ||
|
|
4767a7af60 | ||
|
|
45d87f36f7 | ||
|
|
35b95f5c88 | ||
|
|
9d05f5ea62 | ||
|
|
360cddc05b | ||
|
|
14b846549e | ||
|
|
68165f0e01 | ||
|
|
2dc469274a | ||
|
|
702a1cec00 | ||
|
|
49d15d8ffe | ||
|
|
c89cea4953 | ||
|
|
d9c82e39fb | ||
|
|
eac7860b51 | ||
|
|
da7d0f9d62 | ||
|
|
e49a6ad963 | ||
|
|
d70b985eb0 | ||
|
|
e3c47b2762 | ||
|
|
dbf17f3592 | ||
|
|
ae75e4582d | ||
|
|
ee1264b66b | ||
|
|
51da8c5716 | ||
|
|
81bd32f613 | ||
|
|
ee3031013e | ||
|
|
5026967cee | ||
|
|
5481fbc99c | ||
|
|
e480a84a44 | ||
|
|
64d3882c16 | ||
|
|
6b67b8e0f8 | ||
|
|
0cb94d85ec | ||
|
|
9da88db221 | ||
|
|
1a3aaea933 |
@@ -28,3 +28,38 @@ TICKTICK_CLIENT_ID=
|
||||
TICKTICK_CLIENT_SECRET=
|
||||
TICKTICK_TOKEN=
|
||||
HOME_ASSISTANT_ACTION_TASK_PROJECT_ID=
|
||||
|
||||
# Optional: Modbus polling (global kill-switch; default false — opt-in).
|
||||
# MODBUS_POLLING_ENABLED=false
|
||||
|
||||
# Optional: MQTT broker connection.
|
||||
# Leave MQTT_ENABLED=false (or unset) when MQTT is not needed.
|
||||
MQTT_ENABLED=false
|
||||
MQTT_BROKER_HOST=
|
||||
MQTT_BROKER_PORT=1883
|
||||
MQTT_USERNAME=
|
||||
MQTT_PASSWORD=
|
||||
MQTT_TLS_ENABLED=false
|
||||
|
||||
# Optional: Home Assistant MQTT Discovery.
|
||||
# Requires MQTT_ENABLED=true and a running MQTT broker.
|
||||
# HA_DISCOVERY_PREFIX is the config topic prefix (must be "homeassistant" for HA auto-discovery).
|
||||
# HA_STATE_TOPIC_PREFIX is the prefix for state/availability topics (separate from discovery).
|
||||
HA_DISCOVERY_ENABLED=false
|
||||
HA_DISCOVERY_PREFIX=homeassistant
|
||||
HA_STATE_TOPIC_PREFIX=home_automation
|
||||
|
||||
# Optional: DSMR smart-meter ingest via MQTT (M6; default off — opt-in).
|
||||
# Set DSMR_INGEST_ENABLED=true to subscribe to the DSMR MQTT topic and
|
||||
# store 10-second downsampled telegram frames in dsmr_reading.
|
||||
# DSMR_INGEST_ENABLED=false
|
||||
# DSMR_MQTT_TOPIC=dsmr/json
|
||||
# DSMR_SAMPLE_INTERVAL_S=10
|
||||
# DSMR tariff topic: publishes "1" (dal/off-peak) or "2" (normal/peak).
|
||||
# Empty string disables tariff-aware pricing (buy/sell_price_now always show normal rate).
|
||||
# DSMR_TARIFF_TOPIC=dsmr/meter-stats/electricity_tariff
|
||||
|
||||
# Optional: Tibber dynamic pricing credentials (M6).
|
||||
# Only used when an active energy contract with kind=tibber is configured.
|
||||
# TIBBER_API_TOKEN=
|
||||
# TIBBER_HOME_ID=
|
||||
|
||||
@@ -36,14 +36,29 @@
|
||||
|
||||
- **默认逐步**:给一个 milestone 文档,按其中原子任务**一步一步**实现。
|
||||
- **(a) 只实现一步**:用户说"只实现一步 / 这一个任务"时,**只做那一个任务卡**,跑完校验闸门后停下,等用户确认,不要顺手往下做。
|
||||
- **(b) 完成整个 milestone**:仅当用户在提示词里**显式要求启用 sub-agent 并指定模型**时,才用指定模型起 implementer sub-agent,按任务依赖顺序跑完整条链。
|
||||
- **Sub-agent 纪律**:只在用户显式要求时才 spawn sub-agent;单步/小改动在主线内联完成。起 sub-agent 时用用户**指定的模型**(Agent 工具的 `model` 覆盖)。
|
||||
- **(b) 完成整个 milestone**:仅当用户在提示词里**显式要求启用 sub-agent**时,才起 implementer / reviewer / fixer sub-agent(模型用下方**『默认模型档位』**,用户人工指定则覆盖),按任务依赖顺序跑完整条链。
|
||||
- **Sub-agent 纪律**:只在用户显式要求时才 spawn sub-agent;单步/小改动在主线内联完成。起 sub-agent 时按下方**『默认模型档位』**选择模型(用户人工指定则以人工指定为准),用 Agent 工具的 `model` 字段落实。
|
||||
|
||||
### 角色(Orchestrator → Implementer → Reviewer)
|
||||
### 默认模型档位(实现模式 sub-agent;可被人工指定覆盖)
|
||||
|
||||
起 implementer / reviewer / fixer sub-agent 时,**默认**用下列模型档位,无需用户每次人工指定:
|
||||
|
||||
| 角色 | 默认模型 | 推理档位 |
|
||||
| --- | --- | --- |
|
||||
| **Implementer** | **Sonnet** | high reasoning effort |
|
||||
| **Fixer**(返工) | **Sonnet** | high reasoning effort |
|
||||
| **Reviewer** | **Opus** | extra-high reasoning effort |
|
||||
|
||||
- **人工指定覆盖**:若用户在提示词里**显式指定了其他模型**(针对任一角色),则**以用户人工指定为准**,覆盖上述默认。
|
||||
- 用 Agent 工具的 `model` 字段落实模型选择;该字段当前仅支持 `sonnet` / `opus` / `haiku` / `fable`。
|
||||
- **推理档位说明**:Agent 工具未暴露独立的 reasoning-effort 旋钮,"high / extra-high reasoning" 通过 spawn prompt 里的显式指令传达(要求 implementer/fixer 动手前充分推理边界条件;要求 reviewer 以对抗性外部审计心态最高强度复核)。若所在 harness 提供真正的 effort 设置,则一并按此档位设置。
|
||||
|
||||
### 角色(Orchestrator → Implementer → Reviewer → Fixer)
|
||||
|
||||
- 我(主线)= **Orchestrator**:挑依赖已满足的下一个任务、派发、转述结果、维护任务 `Status`。
|
||||
- **Implementer**(便宜模型,用户指定):一次一个任务,严格按任务卡,不扩范围。
|
||||
- **Reviewer**(强模型,用户指定):实现完成后起 Reviewer sub-agent,按任务卡 `Acceptance criteria` + `Reviewer checklist` 复核、**独立重跑校验闸门**,驱动 implementer 返工直到本轮 PASS。
|
||||
- **Implementer**(默认 **Sonnet**,high reasoning;见上方『默认模型档位』):一次一个任务,严格按任务卡,不扩范围。
|
||||
- **Reviewer**(默认 **Opus**,extra-high reasoning):实现完成后起 Reviewer sub-agent,按任务卡 `Acceptance criteria` + `Reviewer checklist` 复核、**独立重跑校验闸门**,驱动返工直到本轮 PASS。
|
||||
- **Fixer**(默认 **Sonnet**,high reasoning):按 reviewer 的编号返工清单返工;**每轮返工起一个干净的 Fixer**(与首次实现的 Implementer 分开冷启动),先读对应 `review-notes/<task>-review-<n>.md` 再改。
|
||||
|
||||
#### Reviewer 盲审纪律(M1 教训)
|
||||
|
||||
@@ -101,7 +116,9 @@ python scripts/export_openapi.py && git diff --exit-code openapi/ # 改了路
|
||||
## Commit 规范(重点)
|
||||
|
||||
### 分支
|
||||
- 每个 milestone/feature 一个分支(如 `feature/m1-db-consolidation`),**不在 `main` 上直接提交**。
|
||||
- **本仓库是个人单用户项目:默认直接在 `main` 上开发**,不强制 feature 分支,**直接提交并 push 到 `main` 是允许的(无需开 PR)**。
|
||||
- 仍保持**每个任务一个干净 commit**(message 前缀任务/里程碑 ID)。改动较大想隔离时可临时开分支,用完**快进合并**回 `main`(保持线性历史),非必需。
|
||||
- 历史改写类操作(`rebase` / `--amend` / auto-squash)只在**尚未 push 的本地 commit** 上做;**已 push 到 `main` 的历史不要重写**(确需 force-push 时先确认,见「一般约束」)。
|
||||
|
||||
### 一轮实现完成(用户确认「实现完成」后)
|
||||
- 准备好**这一轮的 commit message** 并提交,作为本轮的 **base commit**。
|
||||
@@ -130,7 +147,8 @@ python scripts/export_openapi.py && git diff --exit-code openapi/ # 改了路
|
||||
- autosquash **改写历史**:仅在 push / 开 PR **之前**做。若该分支已 push,需要 force-push——属对外操作,**先取得用户确认再做**。
|
||||
|
||||
### 一般约束
|
||||
- commit / push 只在用户要求时进行;push、force-push、开/改 PR 等对外操作先确认。
|
||||
- **个人单用户仓库:直接 commit 并 push 到 `main` 已获授权**——在校验闸门全绿、一轮工作完成时即可提交 / 推送,无需逐次征求同意。
|
||||
- 仍需**先取得用户确认**的操作:**force-push / 改写已推送历史**,以及**打 tag**(会触发镜像 CI / 对外发布;且打 tag 前须按下方「发版前置走查」真跑一次 `docker build`)。
|
||||
|
||||
## 发版前置走查(打 tag 前必做)
|
||||
|
||||
|
||||
+7
-2
@@ -1,5 +1,10 @@
|
||||
# Stage 1: build the React SPA
|
||||
FROM node:22-slim AS frontend-build
|
||||
# Stage 1: build the React SPA.
|
||||
# Pin to the native build host ($BUILDPLATFORM) so the Node/V8 build never runs
|
||||
# under QEMU during multi-arch builds — emulated Node crashes V8's baseline JIT
|
||||
# (SIGTRAP / exit 133 on `npm ci`). The dist/ output is static JS/CSS, i.e.
|
||||
# architecture-independent, so building it once and COPYing it into each
|
||||
# target-arch runtime stage is both correct and avoids the emulator entirely.
|
||||
FROM --platform=$BUILDPLATFORM node:22-slim AS frontend-build
|
||||
|
||||
WORKDIR /frontend
|
||||
|
||||
|
||||
@@ -6,14 +6,21 @@
|
||||
|
||||
- FastAPI Web 应用(React SPA 前端 + JSON API)
|
||||
- SQLite + SQLAlchemy + Alembic 的单库结构
|
||||
- username/password + server-side session 鉴权
|
||||
- username/password + server-side session 鉴权(含登录加固,见下文)
|
||||
- runtime config 页面与 app DB 持久化
|
||||
- public IPv4 monitor、历史持久化与定时检查
|
||||
- SMTP 配置、测试发信与 public IPv4 changed 邮件通知
|
||||
- location recorder
|
||||
- poo recorder
|
||||
- Home Assistant inbound / outbound integration
|
||||
- Home Assistant inbound / outbound integration(REST 通道)
|
||||
- TickTick OAuth 与 action task 集成
|
||||
- **Modbus 设备采集**:通过 YAML profile(首个:SDM120 电表)按设备周期轮询 Modbus-TCP 网关,解码工程量并落通用读数表(`modbus_device` + `modbus_reading`)
|
||||
- **MQTT + Home Assistant Discovery**:以可勾选方式把 Modbus 设备/工程量注册为 HA device/entity(含 binary_sensor online),state 周期发布;配置变更可重连重发
|
||||
- **前端侧边栏 + Energy 视图**:侧边导航替换顶栏;Energy 页管理 Modbus 设备、展示最新读数与 Recharts 走势图;Config 页 Accordion 分区展开;Expose 设置勾选 HA 可暴露实体
|
||||
- **DSMR 实时电表接入**:订阅 DSMR Reader 的 `dsmr/json`(每秒一帧)、整帧 JSON blob 按 10 秒降采样落库(`dsmr_reading`)
|
||||
- **通用电价合同层**:YAML profile 定合同结构(manual 固定/双费率 / tibber 动态电价);`EnergyContract`+`EnergyContractVersion` 存 UI 可填的数值,改价加新版本旧版本保留;price strategy 按 kind 出价
|
||||
- **实时买卖电费计算**:每 15 分钟按寄存器差值(`_1`=dal/低、`_2`=normal/高)× 买/卖价算计量电费,快照不可变;日/月/年汇总加固定费减 heffingskorting
|
||||
- **反哺 Home Assistant Energy**:当前买/卖价 + 累计买电支出/卖电收入(`total_increasing`)发成 HA 实体,可直接挂 HA Energy 仪表盘
|
||||
- pytest 测试与 OpenAPI 导出脚本
|
||||
- Docker / Compose 部署入口
|
||||
|
||||
@@ -30,6 +37,13 @@
|
||||
- public IPv4 当前状态与变化历史
|
||||
- location 记录(`location` 表)
|
||||
- poo 记录(`poo_records` 表)
|
||||
- Modbus 设备定义(`modbus_device` 表)
|
||||
- Modbus 通用读数(`modbus_reading` 表,JSON payload)
|
||||
- HA 实体暴露开关(`exposed_entity_toggle` 表)
|
||||
- DSMR 电表实时读数(`dsmr_reading` 表,整帧 JSON blob,10s 降采样)
|
||||
- 电价合同(`energy_contract` 表)与版本(`energy_contract_version` 表,values JSON)
|
||||
- Tibber 15 分钟电价缓存(`tibber_price` 表,不可变)
|
||||
- 每 15 分钟计量电费(`energy_cost_period` 表,快照价,不可变)
|
||||
|
||||
配置层只保留一个数据库环境变量:
|
||||
|
||||
@@ -41,7 +55,7 @@
|
||||
python -m scripts.run_migrations
|
||||
```
|
||||
|
||||
该命令会通过 Alembic 将 `app.db` 初始化或升级到最新 head(含 `location` / `poo_records` 表)。
|
||||
该命令会通过 Alembic 将 `app.db` 初始化或升级到最新 head(含全部表,包括 M5 新增的 `modbus_device`、`modbus_reading`、`exposed_entity_toggle`,以及 M6 新增的 `dsmr_reading`、`energy_contract`、`energy_contract_version`、`tibber_price`、`energy_cost_period`)。
|
||||
|
||||
## 当前目录
|
||||
|
||||
@@ -49,7 +63,7 @@ python -m scripts.run_migrations
|
||||
|
||||
- `app/`: FastAPI 应用代码(包含 JSON API、业务服务、数据模型)
|
||||
- `frontend/`: React SPA 前端(Vite + React + TypeScript + Mantine)
|
||||
- `alembic_app/`: App DB 的 Alembic migration 环境(同时管理 `location` / `poo_records` 表)
|
||||
- `alembic_app/`: App DB 的 Alembic migration 环境(管理所有表,含 M5 新增的 `modbus_device`、`modbus_reading`、`exposed_entity_toggle`,以及 M6 新增的 `dsmr_reading`、`energy_contract`、`energy_contract_version`、`tibber_price`、`energy_cost_period`)
|
||||
- `tests/`: pytest 测试
|
||||
- `docs/`: 当前系统说明文档
|
||||
- `scripts/`: 辅助脚本,例如 OpenAPI 导出
|
||||
@@ -128,6 +142,7 @@ M2 用 React SPA 取代了原有 Jinja 服务端模板,由 FastAPI 同源托
|
||||
- **Vite + React + TypeScript + Mantine**(组件库)
|
||||
- **TanStack Query**(数据请求/缓存)
|
||||
- **Leaflet / react-leaflet**(地图与热力图)
|
||||
- **Recharts**(Energy 视图走势图,M5 引入)
|
||||
- **openapi-typescript + openapi-fetch**(类型化 API client,由 `openapi/openapi.json` 生成)
|
||||
|
||||
### 本地开发(前端)
|
||||
@@ -177,7 +192,7 @@ npm run build # 构建,确认产出 dist
|
||||
- App DB:`sqlite:///./data/app.db`
|
||||
- 数据目录:`./data/`
|
||||
|
||||
所有模型(auth / config / public_ip / location / poo)共用同一个 `Base`,均通过单一 Alembic 链管理:
|
||||
所有模型(auth / config / public_ip / location / poo / modbus / expose)共用同一个 `Base`,均通过单一 Alembic 链管理:
|
||||
|
||||
- Alembic 环境:`alembic_app.ini` + `alembic_app/`
|
||||
- 统一 migration job:`python -m scripts.run_migrations`
|
||||
@@ -228,6 +243,195 @@ React SPA 主要页面路由(客户端路由,均由 FastAPI fallback 到 `in
|
||||
|
||||
无论是本地 `host:port` 还是反向代理后的域名访问,登录成功后进入 SPA 首页(`/`)。
|
||||
|
||||
## M4 登录加固
|
||||
|
||||
M4 在基础鉴权之上叠加了三层防御,详细说明见 [`docs/auth.md`](./docs/auth.md)。
|
||||
|
||||
### 防爆破 / 指数退避
|
||||
|
||||
登录失败超过 3 次后进入指数退避(`wait = min(900s, 1s × 2^(failures-3))`),期间请求返回 `429 Too Many Requests`(含 `Retry-After` 响应头);成功登录后自动清零。退避按 **client IP** 与 **username** 双键取较大值,不会因此永久锁定账号(只是延迟,不是封号)。
|
||||
|
||||
- 全局开关:`AUTH_LOGIN_THROTTLE_ENABLED`(CONFIG_FIELDS,默认 `true`)
|
||||
- 反代后需要 `AUTH_TRUST_FORWARDED_FOR=true` 才会读 `X-Forwarded-For`(`.env` 部署级配置,默认 `false`)
|
||||
|
||||
### CLI 逃生通道
|
||||
|
||||
拿到服务器 CLI 权限时,可以在**不依赖任何已存凭据**(无需密码、恢复码)的情况下重置密码、解锁退避、关停 TOTP:
|
||||
|
||||
```bash
|
||||
# 重置密码(不加 --password 则交互式输入,不回显)
|
||||
python -m scripts.admin_cli reset-password admin
|
||||
|
||||
# 解锁退避(被 429 挡住时使用)
|
||||
python -m scripts.admin_cli unlock --all # 清所有退避行
|
||||
python -m scripts.admin_cli unlock --ip 1.2.3.4 # 按 IP 清
|
||||
python -m scripts.admin_cli unlock --username admin # 按 username 清
|
||||
|
||||
# TOTP 相关(需要先启用,见下文)
|
||||
python -m scripts.admin_cli disable-totp admin # 关停 TOTP(零凭据,逃生口)
|
||||
python -m scripts.admin_cli reissue-totp admin # 重新发放 TOTP secret(打印新 URI)
|
||||
|
||||
# 查看用户列表
|
||||
python -m scripts.admin_cli list-admin
|
||||
```
|
||||
|
||||
在 Docker 容器内执行时:
|
||||
|
||||
```bash
|
||||
docker compose exec app python -m scripts.admin_cli <command>
|
||||
```
|
||||
|
||||
### 可选 TOTP 二次验证
|
||||
|
||||
admin 可在 React SPA 设置页(`/config`)自选启用 RFC 6238 TOTP:
|
||||
|
||||
1. 设置页点「启用 TOTP」→ 后端生成 `otpauth://` URI,前端渲染二维码(`qrcode.react`)
|
||||
2. 用 Authenticator App(如 Google Authenticator、Authy)扫码
|
||||
3. 输入当前 6 位动态码确认 → TOTP 启用
|
||||
4. 妥善保存一次性展示的 10 个恢复码(格式 `xxxx-xxxx`)
|
||||
|
||||
启用后,登录需要两步:密码 → 6 位动态码(或恢复码,一次性)。不启用则维持纯密码登录,行为不变。
|
||||
|
||||
恢复码丢失时,可用 CLI 逃生:`python -m scripts.admin_cli disable-totp admin`,随后即可纯密码登录。
|
||||
|
||||
TOTP issuer 标签(显示在 Authenticator 里)通过 `AUTH_TOTP_ISSUER` 环境变量配置(`.env` 部署级),默认回退 `app_name`。
|
||||
|
||||
## M5 Modbus 设备采集 / Energy / MQTT + HA Discovery
|
||||
|
||||
M5 给后端接入家庭 IoT 生态,新增通用 Modbus 采集链路(首个领域:能耗)、MQTT + Home Assistant Discovery 发布,以及前端侧边栏与 Energy 视图。
|
||||
|
||||
### 依赖
|
||||
|
||||
后端新增:
|
||||
- `pymodbus`:Modbus-TCP 客户端(轮询电表等 slave 设备)
|
||||
- `paho-mqtt`:MQTT 客户端(HA Discovery 与 state 发布)
|
||||
- `pyyaml`:YAML profile 加载(设备协议声明式描述)
|
||||
|
||||
前端新增:
|
||||
- `recharts`:Energy 视图走势图
|
||||
|
||||
### Modbus 设备采集
|
||||
|
||||
采集链路采用两层分离:**YAML profile**(协议知识,随代码走)+ **`modbus_device` 数据库行**(部署/可配置信息)+ **`modbus_reading` 通用读数表**(JSON payload 遥测)。
|
||||
|
||||
- **profile**(如 `sdm120.yaml`)描述:读哪些寄存器(FC04 块读)、每个量的 key/unit/device_class/ha_component。纯协议知识,不含 unit_id / friendly_name 等部署项。
|
||||
- **`modbus_device` 行**:friendly_name、网关 host/port、Modbus slave `unit_id`(电表 Meter ID,设备面板可改故落 DB)、选用哪个 profile、采样周期、是否启用。
|
||||
- **`modbus_reading` 行**:device_id FK、recorded_at、payload(JSON,如 `{"voltage": 230.2, "current": 1.3, ...}`)。
|
||||
- 多设备可共享同一 profile(如两块 SDM120 共用 `sdm120` profile,各自独立 unit_id 和 friendly_name)。
|
||||
- APScheduler 后台 job 周期轮询所有 `enabled` 设备,更新 `last_poll_at` / `last_poll_ok`。全局开关 `MODBUS_POLLING_ENABLED`(CONFIG_FIELDS)。
|
||||
|
||||
**手工命令行试读**(不依赖 DB,最快验证网关连通性):
|
||||
|
||||
```bash
|
||||
# 按 profile 解码读一次(验证整套解码链路)
|
||||
python -m scripts.modbus_cli read --host <网关IP> --port 502 --unit 1 --profile sdm120
|
||||
|
||||
# 手工指定请求内容(first-contact 验证,不依赖 profile)
|
||||
python -m scripts.modbus_cli probe --host <网关IP> --port 502 --unit 1 --fc 4 --address 0x0000 --count 2 --decode float32
|
||||
```
|
||||
|
||||
CLI 工具为受控手工验证而设(设备需接市电),仅暴露读功能码(FC03/04),无写寄存器子命令。
|
||||
|
||||
### MQTT + Home Assistant Discovery
|
||||
|
||||
后端作为 MQTT 发布方,把 Modbus 设备/工程量以 HA Discovery 协议自动注册为 device/entity:
|
||||
|
||||
- 每个 Modbus 设备 = 一个 HA device(`unique_id` 锚定于设备 `uuid`,不随改名变)
|
||||
- 各工程量 = sensor entity(device_class/unit 取自 YAML profile);另有 binary_sensor `online`(取 `last_poll_ok`)
|
||||
- 每次轮询成功后推 state topic;连接成功或勾选变更时重发 retained discovery config
|
||||
- `POST /api/config/mqtt/test`:试连 broker 并发布一条测试消息(可用 MQTT Explorer 验证链路)
|
||||
|
||||
**Config 页配置流程**:
|
||||
1. 在 `/config` 的「MQTT」section 填写 broker host/port/username/password,启用 `MQTT_ENABLED`
|
||||
2. 点「发送测试消息」确认 broker 链路通
|
||||
3. 启用 `HA_DISCOVERY_ENABLED`
|
||||
4. 在「Home Assistant Expose」面板勾选要暴露的实体,点「重新发布 discovery」
|
||||
5. 在 Home Assistant 确认对应 device/entity 出现
|
||||
|
||||
### API 端点(M5 新增)
|
||||
|
||||
| 端点 | 用途 |
|
||||
| --- | --- |
|
||||
| `GET /api/modbus/devices` | 列出 Modbus 设备 |
|
||||
| `POST /api/modbus/devices` | 新建设备 |
|
||||
| `GET /api/modbus/devices/{uuid}` | 单个设备 |
|
||||
| `PATCH /api/modbus/devices/{uuid}` | 修改设备(含 enable/disable)|
|
||||
| `DELETE /api/modbus/devices/{uuid}` | 删除设备;有读数时 409 |
|
||||
| `GET /api/modbus/devices/{uuid}/metrics` | 该设备 profile 的量目录(key/unit/device_class)|
|
||||
| `GET /api/modbus/devices/{uuid}/latest` | 最新一条读数 payload |
|
||||
| `GET /api/modbus/devices/{uuid}/readings` | 时间范围读数(start/end/limit),供走势图 |
|
||||
| `POST /api/modbus/devices/{uuid}/test` | 即时试读(不落库) |
|
||||
| `GET /api/modbus/profiles` | 列出可用 profile 名 + 描述 |
|
||||
| `GET /api/expose` | 可暴露实体目录 + 勾选状态 + MQTT/Discovery 状态 |
|
||||
| `PUT /api/expose` | 设置逐 key 暴露开关 |
|
||||
| `POST /api/expose/republish` | 手动重发 discovery |
|
||||
| `POST /api/config/mqtt/test` | 试连 broker 并发布测试消息 |
|
||||
|
||||
### 前端视图(M5 新增)
|
||||
|
||||
- **侧边栏**:把顶栏改为侧边导航(Home / Records / Energy / Config + 主题切换 + 注销),当前路由高亮,移动端可折叠。
|
||||
- **`/energy`(Energy 视图)**:设备 CRUD(新建/编辑/删除,删除有二次确认;有读数时引导改用禁用);最新读数卡片(字段标签/单位取自 profile metrics);时间序列走势图(Recharts,支持电压/电流/功率/电能,带时间范围选择)。
|
||||
- **Config 页 Accordion**:各大 config section 可独立折叠/展开;「Home Assistant Expose」面板按设备分组勾选可暴露实体、显示 MQTT/Discovery 连接状态、「重新发布 discovery」按钮。
|
||||
- SPA 路由新增 `/energy`。
|
||||
|
||||
## M6 DSMR 接入 / 电价合同 / 实时电费计算 / HA Energy 反哺
|
||||
|
||||
M6 在 M5 IoT 基建之上接入 DSMR 实时智能电表数据,建立通用电价合同层,按每 15 分钟算出实际买卖电费并反哺 HA Energy。
|
||||
|
||||
### 依赖
|
||||
|
||||
M6 **不新增任何 Python 依赖**,复用 M5 已有的 `httpx`(Tibber GraphQL)、`paho-mqtt`(DSMR 订阅)、`pyyaml`(pricing profile 加载)、`apscheduler`(抓价 job、计费 job)。
|
||||
|
||||
### DSMR 实时电表接入
|
||||
|
||||
订阅 DSMR Reader 的 `dsmr/json` topic(每秒一帧完整 telegram),整帧存为 JSON blob、按 `dsmr_sample_interval_s`(默认 10 秒)降采样落 `dsmr_reading`(`source_id` 幂等去重)。`dsmr_ingest_enabled`(默认 false,opt-in)。
|
||||
|
||||
### 电价合同层
|
||||
|
||||
- **YAML profile 定结构**(仓库内,不放数值):`manual.yaml`(固定/双费率:buy_normal/dal、sell_normal/dal、energy_tax、ode、固定费、heffingskorting);`tibber.yaml`(动态:source=tibber_api,energy_tax、sell_adjust)
|
||||
- **`EnergyContract` + `EnergyContractVersion`**(UI 填数值):改价 = 加新版本行(带 `effective_from`),旧版本保留(审计链);一次只有一个 active 合同
|
||||
- **price strategy**:`manual` 用双费率常数(`buy = energy_buy_档 + energy_tax`,`sell = sell_档`);`tibber` 用 `tibber_price.total` 作买价(已含税,demo 确认 `total=energy+tax`),`total − energy_tax − sell_adjust` 作卖价(卖价残差 `sell_adjust` 默认 0,待真实账单核定)
|
||||
|
||||
### 每 15 分钟计量电费(不可变)
|
||||
|
||||
APScheduler 1 分钟 tick,取每个闭合 15 分钟窗口的 DSMR 寄存器差值(`delivered_1/2`,`returned_1/2`;`_1`=dal/低,`_2`=normal/高,NL 惯例)× 当时合同版本的 strategy 出价,upsert `energy_cost_period`(快照当时价 + `contract_version_id`)。缺价/缺数据时标 `degraded`。`POST /api/energy/costs/recompute` 显式重算。
|
||||
|
||||
日/月/年汇总 = Σnet + 固定费(network_fee + management_fee 按月→天 × 天数)- heffingskorting(按年→天 × 天数),读时计算、不落表。能源税 `energy_tax` 参考值约 0.1108 EUR/kWh(2026 第一档含 VAT,待真实账单核定;该值由 UI 填入合同版本,YAML profile 仅声明字段 unit,代码无写死默认数值)。
|
||||
|
||||
### Tibber 动态电价
|
||||
|
||||
`app/integrations/tibber/client.py` httpx POST GraphQL(`priceInfoRange(QUARTER_HOURLY, first=96)`),解析 `startsAt`/`total`/`energy`/`tax`/`level`,按 `starts_at` upsert `tibber_price`(幂等)。启动 + 每小时抓取今明两天 15 分钟价、幂等 upsert(hourly trigger,确保每日刷新且可补重试);仅当 active 合同 kind=tibber 且 `tibber_api_token` 存在时运行。`POST /api/energy/tibber/test` 试连三态(success 带当前价 / config-error / failed)。
|
||||
|
||||
### 反哺 Home Assistant Energy
|
||||
|
||||
`_energy_cost_provider` 向 expose 框架注册 4 个实体:`buy_price_now`、`sell_price_now`(€/kWh sensor)、`import_cost_total`、`export_revenue_total`(`total_increasing` monetary,可直接挂 HA Energy 仪表盘)。默认未勾选,在 Expose 面板启用。
|
||||
|
||||
### API 端点(M6 新增)
|
||||
|
||||
| 端点 | 用途 |
|
||||
| --- | --- |
|
||||
| `GET /api/energy/contracts` | 列出合同 + active 标记 |
|
||||
| `POST /api/energy/contracts` | 新建合同(kind + 首版本值,按 profile 校验)|
|
||||
| `GET /api/energy/contracts/{id}` | 单个合同 + 版本历史 |
|
||||
| `PATCH /api/energy/contracts/{id}` | 改名 / 激活 |
|
||||
| `POST /api/energy/contracts/{id}/versions` | 加新版本(改价,带生效日期)|
|
||||
| `GET /api/energy/profiles` | 列出 pricing profile 结构(前端按它渲染表单)|
|
||||
| `GET /api/energy/prices` | 区间价格点(曲线)|
|
||||
| `GET /api/energy/costs` | 区间 `energy_cost_period`(走势/明细)|
|
||||
| `GET /api/energy/costs/summary` | 区间汇总(计量电费 + 固定费 − 抵扣)|
|
||||
| `POST /api/energy/costs/recompute` | 幂等重算 |
|
||||
| `GET /api/energy/dsmr/latest` | 最新 `dsmr_reading` |
|
||||
| `POST /api/energy/tibber/test` | 试连 Tibber + 拉当前价,三态 |
|
||||
|
||||
DSMR/Tibber 标量配置复用现有 `GET/PUT /api/config`(新增 `dsmr_ingest_enabled`、`dsmr_mqtt_topic`、`dsmr_sample_interval_s`、`tibber_api_token`(secret)、`tibber_home_id`)。
|
||||
|
||||
### 前端视图(M6 新增,并入 Energy 视图)
|
||||
|
||||
- **Contracts Tab**:合同列表 + 新建/编辑(表单按 `/api/energy/profiles` 结构渲染,不 hardcode 字段)+ 激活 + 改价加版本 + 版本历史只读。
|
||||
- **Prices Tab**:15 分钟价格曲线(tibber 动态或 manual 档位),复用 Recharts。
|
||||
- **Costs Tab**:费用走势/明细 + 汇总卡片(含固定费/抵扣)。
|
||||
- **Config 页 Tibber 测试**:三态(success/config-error/failed)。
|
||||
|
||||
## Config 持久化
|
||||
|
||||
当前 config 页面不会把修改写回 `.env`。
|
||||
@@ -253,6 +457,11 @@ React SPA 主要页面路由(客户端路由,均由 FastAPI fallback 到 `in
|
||||
- SMTP 基础配置
|
||||
- TickTick OAuth 配置
|
||||
- Home Assistant 配置
|
||||
- MQTT broker 配置(`MQTT_ENABLED`、`MQTT_BROKER_HOST/PORT/USERNAME/PASSWORD`、`MQTT_TLS_ENABLED`)
|
||||
- Home Assistant Discovery 配置(`HA_DISCOVERY_ENABLED`、`HA_DISCOVERY_PREFIX`)
|
||||
- Modbus 采集配置(`MODBUS_POLLING_ENABLED`)
|
||||
- DSMR 接入配置(`DSMR_INGEST_ENABLED`、`DSMR_MQTT_TOPIC`、`DSMR_SAMPLE_INTERVAL_S`)
|
||||
- Tibber 凭据(`TIBBER_API_TOKEN`(secret)、`TIBBER_HOME_ID`)
|
||||
|
||||
其中 SMTP password 与其他 secret 字段一致:
|
||||
|
||||
|
||||
+11
-1
@@ -6,10 +6,20 @@ from sqlalchemy import engine_from_config, pool
|
||||
from app.config import get_settings
|
||||
from app.db import Base
|
||||
from app.models.config import AppConfigEntry # noqa: F401
|
||||
from app.models.auth import AuthSession, AuthUser # noqa: F401
|
||||
from app.models.auth import AuthSession, AuthUser, RecoveryCode # noqa: F401
|
||||
from app.models.auth_throttle import LoginThrottle # noqa: F401
|
||||
from app.models.public_ip import PublicIPHistory, PublicIPState # noqa: F401
|
||||
from app.models.location import Location # noqa: F401
|
||||
from app.models.poo import PooRecord # noqa: F401
|
||||
from app.models.modbus import ModbusDevice, ModbusReading # noqa: F401
|
||||
from app.models.expose import ExposedEntityToggle # noqa: F401
|
||||
from app.models.energy import ( # noqa: F401
|
||||
DsmrReading,
|
||||
EnergyContract,
|
||||
EnergyContractVersion,
|
||||
TibberPrice,
|
||||
EnergyCostPeriod,
|
||||
)
|
||||
|
||||
config = context.config
|
||||
|
||||
|
||||
@@ -0,0 +1,43 @@
|
||||
"""add auth_login_throttle table for exponential back-off throttling
|
||||
|
||||
Revision ID: 20260621_07_auth_login_throttle
|
||||
Revises: 20260611_06_merge_location_poo_tables
|
||||
Create Date: 2026-06-21 00:00:00.000000
|
||||
"""
|
||||
|
||||
from typing import Sequence, Union
|
||||
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
|
||||
revision: str = "20260621_07_auth_login_throttle"
|
||||
down_revision: Union[str, None] = "20260611_06_merge_location_poo_tables"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.create_table(
|
||||
"auth_login_throttle",
|
||||
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
|
||||
sa.Column("key", sa.String(length=255), nullable=False),
|
||||
sa.Column("scope", sa.String(length=16), nullable=False),
|
||||
sa.Column("failures", sa.Integer(), nullable=False),
|
||||
sa.Column("first_failed_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.Column("last_failed_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.Column("next_allowed_at", sa.DateTime(timezone=True), nullable=True),
|
||||
sa.PrimaryKeyConstraint("id"),
|
||||
sa.UniqueConstraint("scope", "key", name="uq_auth_login_throttle_scope_key"),
|
||||
)
|
||||
op.create_index(
|
||||
"ix_auth_login_throttle_scope_key",
|
||||
"auth_login_throttle",
|
||||
["scope", "key"],
|
||||
unique=False,
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_index("ix_auth_login_throttle_scope_key", table_name="auth_login_throttle")
|
||||
op.drop_table("auth_login_throttle")
|
||||
@@ -0,0 +1,61 @@
|
||||
"""add TOTP fields to auth_users and create auth_recovery_code table
|
||||
|
||||
Revision ID: 20260621_08_totp
|
||||
Revises: 20260621_07_auth_login_throttle
|
||||
Create Date: 2026-06-21 00:00:00.000000
|
||||
"""
|
||||
|
||||
from typing import Sequence, Union
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
|
||||
|
||||
revision: str = "20260621_08_totp"
|
||||
down_revision: Union[str, None] = "20260621_07_auth_login_throttle"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
# Add totp_secret (nullable) to auth_users — existing rows get NULL, which is correct.
|
||||
op.add_column("auth_users", sa.Column("totp_secret", sa.String(length=64), nullable=True))
|
||||
# Add totp_enabled (NOT NULL) with server_default="0" so existing rows default to false.
|
||||
op.add_column(
|
||||
"auth_users",
|
||||
sa.Column(
|
||||
"totp_enabled",
|
||||
sa.Boolean(),
|
||||
nullable=False,
|
||||
server_default="0",
|
||||
),
|
||||
)
|
||||
|
||||
# Create auth_recovery_code table for one-time TOTP recovery codes.
|
||||
op.create_table(
|
||||
"auth_recovery_code",
|
||||
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
|
||||
sa.Column("user_id", sa.Integer(), nullable=False),
|
||||
sa.Column("code_hash", sa.String(length=255), nullable=False),
|
||||
sa.Column("used_at", sa.DateTime(timezone=True), nullable=True),
|
||||
sa.ForeignKeyConstraint(["user_id"], ["auth_users.id"]),
|
||||
sa.PrimaryKeyConstraint("id"),
|
||||
)
|
||||
op.create_index(
|
||||
op.f("ix_auth_recovery_code_user_id"),
|
||||
"auth_recovery_code",
|
||||
["user_id"],
|
||||
unique=False,
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
# Drop auth_recovery_code table first (has FK to auth_users).
|
||||
op.drop_index(op.f("ix_auth_recovery_code_user_id"), table_name="auth_recovery_code")
|
||||
op.drop_table("auth_recovery_code")
|
||||
|
||||
# Drop the two TOTP columns from auth_users.
|
||||
# Use batch_alter_table for SQLite compatibility (alembic's portable column-drop path).
|
||||
with op.batch_alter_table("auth_users") as batch_op:
|
||||
batch_op.drop_column("totp_enabled")
|
||||
batch_op.drop_column("totp_secret")
|
||||
@@ -0,0 +1,81 @@
|
||||
"""add modbus_device and modbus_reading tables
|
||||
|
||||
Revision ID: 20260622_09_modbus_tables
|
||||
Revises: 20260621_08_totp
|
||||
Create Date: 2026-06-22 00:00:00.000000
|
||||
"""
|
||||
|
||||
from typing import Sequence, Union
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
|
||||
|
||||
revision: str = "20260622_09_modbus_tables"
|
||||
down_revision: Union[str, None] = "20260621_08_totp"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
# modbus_device — deployment/configurable metadata for each polled device.
|
||||
op.create_table(
|
||||
"modbus_device",
|
||||
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
|
||||
sa.Column("uuid", sa.String(length=36), nullable=False),
|
||||
sa.Column("friendly_name", sa.String(length=255), nullable=False),
|
||||
sa.Column("transport", sa.String(length=16), nullable=False),
|
||||
sa.Column("host", sa.String(length=255), nullable=False),
|
||||
sa.Column("port", sa.Integer(), nullable=False),
|
||||
sa.Column("unit_id", sa.Integer(), nullable=False),
|
||||
sa.Column("profile", sa.String(length=64), nullable=False),
|
||||
sa.Column("poll_interval_s", sa.Integer(), nullable=False),
|
||||
sa.Column("enabled", sa.Boolean(), nullable=False),
|
||||
sa.Column("last_poll_at", sa.DateTime(timezone=True), nullable=True),
|
||||
sa.Column("last_poll_ok", sa.Boolean(), nullable=True),
|
||||
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.PrimaryKeyConstraint("id"),
|
||||
sa.UniqueConstraint("uuid", name="uq_modbus_device_uuid"),
|
||||
)
|
||||
|
||||
# modbus_reading — generic telemetry, one row per device per poll cycle.
|
||||
op.create_table(
|
||||
"modbus_reading",
|
||||
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
|
||||
sa.Column("device_id", sa.Integer(), nullable=False),
|
||||
sa.Column("recorded_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.Column("payload", sa.JSON(), nullable=False),
|
||||
sa.ForeignKeyConstraint(
|
||||
["device_id"],
|
||||
["modbus_device.id"],
|
||||
ondelete="RESTRICT",
|
||||
),
|
||||
sa.PrimaryKeyConstraint("id"),
|
||||
)
|
||||
|
||||
# Individual index on recorded_at (from the ORM-level index=True).
|
||||
op.create_index(
|
||||
"ix_modbus_reading_recorded_at",
|
||||
"modbus_reading",
|
||||
["recorded_at"],
|
||||
unique=False,
|
||||
)
|
||||
|
||||
# Composite index for efficient time-range queries per device.
|
||||
op.create_index(
|
||||
"ix_modbus_reading_device_recorded",
|
||||
"modbus_reading",
|
||||
["device_id", "recorded_at"],
|
||||
unique=False,
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
# Drop the reading table first (it has a FK referencing modbus_device).
|
||||
op.drop_index("ix_modbus_reading_device_recorded", table_name="modbus_reading")
|
||||
op.drop_index("ix_modbus_reading_recorded_at", table_name="modbus_reading")
|
||||
op.drop_table("modbus_reading")
|
||||
|
||||
# Drop the device table.
|
||||
op.drop_table("modbus_device")
|
||||
@@ -0,0 +1,44 @@
|
||||
"""add exposed_entity_toggle table
|
||||
|
||||
Revision ID: 20260622_10_exposed_entities
|
||||
Revises: 20260622_09_modbus_tables
|
||||
Create Date: 2026-06-22 00:00:00.000000
|
||||
"""
|
||||
|
||||
from typing import Sequence, Union
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
|
||||
|
||||
revision: str = "20260622_10_exposed_entities"
|
||||
down_revision: Union[str, None] = "20260622_09_modbus_tables"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
# exposed_entity_toggle — per-entity on/off switch for MQTT / HA Discovery.
|
||||
op.create_table(
|
||||
"exposed_entity_toggle",
|
||||
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
|
||||
sa.Column("key", sa.String(length=255), nullable=False),
|
||||
sa.Column("enabled", sa.Boolean(), nullable=False),
|
||||
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.PrimaryKeyConstraint("id"),
|
||||
sa.UniqueConstraint("key", name="uq_exposed_entity_toggle_key"),
|
||||
)
|
||||
|
||||
# Index on key for fast single-key lookups (toggle by key).
|
||||
op.create_index(
|
||||
"ix_exposed_entity_toggle_key",
|
||||
"exposed_entity_toggle",
|
||||
["key"],
|
||||
unique=True,
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
# Drop index then table — only removes what this revision created.
|
||||
op.drop_index("ix_exposed_entity_toggle_key", table_name="exposed_entity_toggle")
|
||||
op.drop_table("exposed_entity_toggle")
|
||||
@@ -0,0 +1,132 @@
|
||||
"""add energy pricing and DSMR metering tables
|
||||
|
||||
Revision ID: 20260623_11_energy_tables
|
||||
Revises: 20260622_10_exposed_entities
|
||||
Create Date: 2026-06-23 00:00:00.000000
|
||||
"""
|
||||
|
||||
from typing import Sequence, Union
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
|
||||
|
||||
revision: str = "20260623_11_energy_tables"
|
||||
down_revision: Union[str, None] = "20260622_10_exposed_entities"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
# dsmr_reading — raw DSMR telegram blobs (10-second down-sampled).
|
||||
# No device table: single P1 smart meter; a ``source`` column can be added later
|
||||
# if a second meter is introduced.
|
||||
op.create_table(
|
||||
"dsmr_reading",
|
||||
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
|
||||
sa.Column("recorded_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.Column("source_id", sa.Integer(), nullable=True),
|
||||
sa.Column("payload", sa.JSON(), nullable=False),
|
||||
sa.PrimaryKeyConstraint("id"),
|
||||
sa.UniqueConstraint("source_id", name="uq_dsmr_reading_source_id"),
|
||||
)
|
||||
op.create_index(
|
||||
"ix_dsmr_reading_recorded_at",
|
||||
"dsmr_reading",
|
||||
["recorded_at"],
|
||||
unique=False,
|
||||
)
|
||||
|
||||
# energy_contract — contract head (manual or tibber, one active at a time).
|
||||
op.create_table(
|
||||
"energy_contract",
|
||||
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
|
||||
sa.Column("name", sa.String(length=255), nullable=False),
|
||||
sa.Column("kind", sa.String(length=32), nullable=False),
|
||||
sa.Column("active", sa.Boolean(), nullable=False),
|
||||
sa.Column("currency", sa.String(length=8), nullable=False),
|
||||
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.PrimaryKeyConstraint("id"),
|
||||
)
|
||||
|
||||
# energy_contract_version — versioned pricing values; append-only for auditability.
|
||||
# Must be created after energy_contract because of the FK dependency.
|
||||
op.create_table(
|
||||
"energy_contract_version",
|
||||
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
|
||||
sa.Column("contract_id", sa.Integer(), nullable=False),
|
||||
sa.Column("effective_from", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.Column("effective_to", sa.DateTime(timezone=True), nullable=True),
|
||||
sa.Column("values", sa.JSON(), nullable=False),
|
||||
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.ForeignKeyConstraint(
|
||||
["contract_id"],
|
||||
["energy_contract.id"],
|
||||
ondelete="RESTRICT",
|
||||
),
|
||||
sa.PrimaryKeyConstraint("id"),
|
||||
)
|
||||
|
||||
# tibber_price — cached Tibber 15-minute spot prices (immutable once fetched).
|
||||
op.create_table(
|
||||
"tibber_price",
|
||||
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
|
||||
sa.Column("starts_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.Column("resolution", sa.String(length=32), nullable=False),
|
||||
sa.Column("energy", sa.Float(), nullable=False),
|
||||
sa.Column("tax", sa.Float(), nullable=False),
|
||||
sa.Column("total", sa.Float(), nullable=False),
|
||||
sa.Column("level", sa.String(length=32), nullable=True),
|
||||
sa.Column("currency", sa.String(length=8), nullable=False),
|
||||
sa.Column("fetched_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.PrimaryKeyConstraint("id"),
|
||||
sa.UniqueConstraint("starts_at", name="uq_tibber_price_starts_at"),
|
||||
)
|
||||
|
||||
# energy_cost_period — computed 15-minute billing periods (immutable snapshot).
|
||||
# Must be created after energy_contract_version because of the FK dependency.
|
||||
op.create_table(
|
||||
"energy_cost_period",
|
||||
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
|
||||
sa.Column("period_start", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.Column("d1_kwh", sa.Float(), nullable=False),
|
||||
sa.Column("d2_kwh", sa.Float(), nullable=False),
|
||||
sa.Column("r1_kwh", sa.Float(), nullable=False),
|
||||
sa.Column("r2_kwh", sa.Float(), nullable=False),
|
||||
sa.Column("import_cost", sa.Float(), nullable=False),
|
||||
sa.Column("export_revenue", sa.Float(), nullable=False),
|
||||
sa.Column("net_cost", sa.Float(), nullable=False),
|
||||
sa.Column("currency", sa.String(length=8), nullable=False),
|
||||
sa.Column("pricing", sa.JSON(), nullable=False),
|
||||
sa.Column("contract_version_id", sa.Integer(), nullable=True),
|
||||
sa.Column("degraded", sa.Boolean(), nullable=False),
|
||||
sa.Column("computed_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.ForeignKeyConstraint(
|
||||
["contract_version_id"],
|
||||
["energy_contract_version.id"],
|
||||
ondelete="RESTRICT",
|
||||
),
|
||||
sa.PrimaryKeyConstraint("id"),
|
||||
sa.UniqueConstraint("period_start", name="uq_energy_cost_period_period_start"),
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
# Drop tables in reverse dependency order: tables with FKs first.
|
||||
|
||||
# energy_cost_period has a FK to energy_contract_version — drop it first.
|
||||
op.drop_table("energy_cost_period")
|
||||
|
||||
# tibber_price has no FK dependencies on our new tables.
|
||||
op.drop_table("tibber_price")
|
||||
|
||||
# energy_contract_version has a FK to energy_contract — drop it before energy_contract.
|
||||
op.drop_table("energy_contract_version")
|
||||
|
||||
# energy_contract has no FK dependencies on our new tables.
|
||||
op.drop_table("energy_contract")
|
||||
|
||||
# dsmr_reading has no FK dependencies.
|
||||
op.drop_index("ix_dsmr_reading_recorded_at", table_name="dsmr_reading")
|
||||
op.drop_table("dsmr_reading")
|
||||
@@ -0,0 +1,58 @@
|
||||
"""decouple dsmr_reading from the telegram id
|
||||
|
||||
The DSMR Reader's own ``id`` field is auto-incrementing but is known to overflow
|
||||
and require a manual reset to zero (a long-standing DSMR firmware quirk). If we
|
||||
keep a UNIQUE constraint on ``source_id`` (the telegram id) and use it for
|
||||
idempotency, an overflow/reset would make legitimately-new telegrams collide with
|
||||
old ids and be silently dropped as "duplicates" — data loss.
|
||||
|
||||
This migration removes that coupling:
|
||||
|
||||
- Drops the UNIQUE constraint on ``source_id`` (the column is kept as a plain,
|
||||
nullable reference value; it is no longer relied upon for uniqueness/dedup).
|
||||
- Makes ``recorded_at`` (the telegram timestamp) UNIQUE instead — a single P1
|
||||
meter emits one telegram per timestamp, so this is a robust, telegram-id-
|
||||
independent idempotency key. The table's own autoincrement ``id`` PK remains
|
||||
the stable internal identity.
|
||||
|
||||
Revision ID: 20260624_12_dsmr_decouple_telegram_id
|
||||
Revises: 20260623_11_energy_tables
|
||||
Create Date: 2026-06-24 00:00:00.000000
|
||||
"""
|
||||
|
||||
from typing import Sequence, Union
|
||||
|
||||
from alembic import op
|
||||
|
||||
|
||||
revision: str = "20260624_12_dsmr_decouple_telegram_id"
|
||||
down_revision: Union[str, None] = "20260623_11_energy_tables"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
# SQLite cannot ALTER away a constraint in place, so recreate the table via
|
||||
# Alembic's batch mode. The table is recreated and rows are copied; existing
|
||||
# data (if any) is preserved. recorded_at must be unique for this to succeed
|
||||
# — a single meter never emits two telegrams at the exact same timestamp.
|
||||
with op.batch_alter_table("dsmr_reading", schema=None) as batch_op:
|
||||
# Old non-unique index on recorded_at is replaced by the unique constraint.
|
||||
batch_op.drop_index("ix_dsmr_reading_recorded_at")
|
||||
# Telegram id is no longer a uniqueness/idempotency key.
|
||||
batch_op.drop_constraint("uq_dsmr_reading_source_id", type_="unique")
|
||||
# Timestamp becomes the telegram-id-independent dedup key.
|
||||
batch_op.create_unique_constraint(
|
||||
"uq_dsmr_reading_recorded_at", ["recorded_at"]
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
with op.batch_alter_table("dsmr_reading", schema=None) as batch_op:
|
||||
batch_op.drop_constraint("uq_dsmr_reading_recorded_at", type_="unique")
|
||||
batch_op.create_unique_constraint(
|
||||
"uq_dsmr_reading_source_id", ["source_id"]
|
||||
)
|
||||
batch_op.create_index(
|
||||
"ix_dsmr_reading_recorded_at", ["recorded_at"], unique=False
|
||||
)
|
||||
@@ -0,0 +1,238 @@
|
||||
"""add meter table and energy_cost_period.meter_id
|
||||
|
||||
Introduces the ``meter`` table (one row per physical meter installation epoch)
|
||||
and a nullable FK column ``energy_cost_period.meter_id`` that attributes each
|
||||
billing period to a specific physical meter.
|
||||
|
||||
**Backfill logic (§3.7 of the M7 design doc)**:
|
||||
|
||||
If the database already contains any ``dsmr_reading`` or
|
||||
``energy_cost_period`` rows, one initial ``meter`` row is created:
|
||||
|
||||
label = "Initial meter"
|
||||
commodity = "electricity"
|
||||
started_at = earliest dsmr_reading.recorded_at
|
||||
(or, if none, earliest energy_cost_period.period_start,
|
||||
or, if still none, the migration timestamp)
|
||||
ended_at = NULL (still active)
|
||||
reason = "initial"
|
||||
|
||||
All existing ``energy_cost_period`` rows are then back-filled with that
|
||||
initial meter's id.
|
||||
|
||||
**Idempotency**: the backfill is guarded with a check for any existing
|
||||
``meter`` row whose ``reason = 'initial'`` and ``commodity = 'electricity'``
|
||||
and ``ended_at IS NULL``, so repeating the upgrade does not create duplicate
|
||||
meters or overwrite already-filled meter_id values.
|
||||
|
||||
**Audit (on-non-degraded periods only)**: after backfilling, the number of
|
||||
non-degraded ``energy_cost_period`` rows with ``meter_id IS NULL`` must be
|
||||
zero; if it is not, the migration raises a ``RuntimeError`` and rolls back.
|
||||
|
||||
**Data safety**: this migration is additive only — no existing rows are
|
||||
deleted or overwritten; it only creates a new table, adds a nullable column,
|
||||
and back-fills that column.
|
||||
|
||||
Revision ID: 20260625_13_meter_table
|
||||
Revises: 20260624_12_dsmr_decouple_telegram_id
|
||||
Create Date: 2026-06-25 00:00:00.000000
|
||||
"""
|
||||
|
||||
from datetime import datetime, timezone
|
||||
from typing import Sequence, Union
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
|
||||
revision: str = "20260625_13_meter_table"
|
||||
down_revision: Union[str, None] = "20260624_12_dsmr_decouple_telegram_id"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
# ------------------------------------------------------------------ #
|
||||
# 1. Create the meter table. #
|
||||
# ------------------------------------------------------------------ #
|
||||
op.create_table(
|
||||
"meter",
|
||||
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
|
||||
sa.Column("label", sa.String(length=255), nullable=False),
|
||||
sa.Column("commodity", sa.String(length=32), nullable=False),
|
||||
sa.Column("started_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.Column("ended_at", sa.DateTime(timezone=True), nullable=True),
|
||||
sa.Column("reason", sa.String(length=64), nullable=False),
|
||||
sa.Column("note", sa.String(length=1024), nullable=True),
|
||||
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
|
||||
sa.PrimaryKeyConstraint("id"),
|
||||
)
|
||||
|
||||
# ------------------------------------------------------------------ #
|
||||
# 2. Add meter_id column to energy_cost_period (nullable FK). #
|
||||
# ------------------------------------------------------------------ #
|
||||
with op.batch_alter_table("energy_cost_period", schema=None) as batch_op:
|
||||
batch_op.add_column(
|
||||
sa.Column("meter_id", sa.Integer(), nullable=True)
|
||||
)
|
||||
batch_op.create_foreign_key(
|
||||
"fk_energy_cost_period_meter_id",
|
||||
"meter",
|
||||
["meter_id"],
|
||||
["id"],
|
||||
ondelete="RESTRICT",
|
||||
)
|
||||
|
||||
# ------------------------------------------------------------------ #
|
||||
# 3. Backfill initial meter (idempotent). #
|
||||
# ------------------------------------------------------------------ #
|
||||
conn = op.get_bind()
|
||||
|
||||
# Check if there is already an initial meter (idempotency guard).
|
||||
existing_initial = conn.execute(
|
||||
sa.text(
|
||||
"SELECT id FROM meter "
|
||||
"WHERE reason = 'initial' AND commodity = 'electricity' AND ended_at IS NULL "
|
||||
"LIMIT 1"
|
||||
)
|
||||
).fetchone()
|
||||
|
||||
if existing_initial is not None:
|
||||
# Already backfilled — nothing to do.
|
||||
return
|
||||
|
||||
# Determine whether there is any historical data to create a meter for.
|
||||
has_readings = conn.execute(
|
||||
sa.text("SELECT 1 FROM dsmr_reading LIMIT 1")
|
||||
).fetchone()
|
||||
has_periods = conn.execute(
|
||||
sa.text("SELECT 1 FROM energy_cost_period LIMIT 1")
|
||||
).fetchone()
|
||||
|
||||
if not has_readings and not has_periods:
|
||||
# Empty database: no historical data, so no initial meter is needed.
|
||||
# meter_id will remain NULL on any future rows until T02 service layer
|
||||
# starts populating it.
|
||||
return
|
||||
|
||||
# Determine started_at: earliest dsmr_reading.recorded_at, falling back to
|
||||
# earliest energy_cost_period.period_start, and finally to now().
|
||||
earliest_reading_row = conn.execute(
|
||||
sa.text("SELECT MIN(recorded_at) AS ts FROM dsmr_reading")
|
||||
).fetchone()
|
||||
earliest_period_row = conn.execute(
|
||||
sa.text("SELECT MIN(period_start) AS ts FROM energy_cost_period")
|
||||
).fetchone()
|
||||
|
||||
started_at_value: datetime | None = None
|
||||
if earliest_reading_row and earliest_reading_row[0] is not None:
|
||||
# SQLite returns ISO strings for datetime columns; parse to datetime.
|
||||
raw = earliest_reading_row[0]
|
||||
started_at_value = _parse_sqlite_datetime(raw)
|
||||
if started_at_value is None and earliest_period_row and earliest_period_row[0] is not None:
|
||||
raw = earliest_period_row[0]
|
||||
started_at_value = _parse_sqlite_datetime(raw)
|
||||
if started_at_value is None:
|
||||
started_at_value = datetime.now(tz=timezone.utc)
|
||||
|
||||
now_utc = datetime.now(tz=timezone.utc)
|
||||
|
||||
# Insert the initial meter row.
|
||||
conn.execute(
|
||||
sa.text(
|
||||
"INSERT INTO meter (label, commodity, started_at, ended_at, reason, note, created_at) "
|
||||
"VALUES (:label, :commodity, :started_at, NULL, :reason, NULL, :created_at)"
|
||||
),
|
||||
{
|
||||
"label": "Initial meter",
|
||||
"commodity": "electricity",
|
||||
"started_at": _iso(started_at_value),
|
||||
"reason": "initial",
|
||||
"created_at": _iso(now_utc),
|
||||
},
|
||||
)
|
||||
|
||||
# Retrieve the newly created meter id.
|
||||
meter_row = conn.execute(
|
||||
sa.text(
|
||||
"SELECT id FROM meter "
|
||||
"WHERE reason = 'initial' AND commodity = 'electricity' AND ended_at IS NULL "
|
||||
"LIMIT 1"
|
||||
)
|
||||
).fetchone()
|
||||
assert meter_row is not None, "Initial meter row not found after insert"
|
||||
meter_id: int = meter_row[0]
|
||||
|
||||
# Back-fill all existing energy_cost_period rows that have meter_id IS NULL.
|
||||
conn.execute(
|
||||
sa.text(
|
||||
"UPDATE energy_cost_period SET meter_id = :mid WHERE meter_id IS NULL"
|
||||
),
|
||||
{"mid": meter_id},
|
||||
)
|
||||
|
||||
# ------------------------------------------------------------------ #
|
||||
# 4. Audit: verify all non-degraded periods have a meter_id. #
|
||||
# ------------------------------------------------------------------ #
|
||||
unmatched_row = conn.execute(
|
||||
sa.text(
|
||||
"SELECT COUNT(*) FROM energy_cost_period "
|
||||
"WHERE meter_id IS NULL AND degraded = 0"
|
||||
)
|
||||
).fetchone()
|
||||
unmatched_count: int = unmatched_row[0] if unmatched_row else 0
|
||||
|
||||
if unmatched_count != 0:
|
||||
raise RuntimeError(
|
||||
f"Meter backfill audit failed: {unmatched_count} non-degraded "
|
||||
"energy_cost_period row(s) still have meter_id IS NULL after backfill. "
|
||||
"Migration aborted to protect data integrity."
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
# Remove the FK column from energy_cost_period first (references meter).
|
||||
with op.batch_alter_table("energy_cost_period", schema=None) as batch_op:
|
||||
batch_op.drop_constraint("fk_energy_cost_period_meter_id", type_="foreignkey")
|
||||
batch_op.drop_column("meter_id")
|
||||
|
||||
# Drop the meter table.
|
||||
op.drop_table("meter")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _parse_sqlite_datetime(value: str | datetime) -> datetime:
|
||||
"""Parse a SQLite datetime value into an aware UTC datetime.
|
||||
|
||||
SQLite stores datetimes as ISO 8601 strings. SQLAlchemy may return them
|
||||
as plain strings or as naive datetimes (no tzinfo) depending on the driver
|
||||
and column declaration. This helper normalises both forms to an aware UTC
|
||||
``datetime``.
|
||||
"""
|
||||
if isinstance(value, datetime):
|
||||
if value.tzinfo is None:
|
||||
return value.replace(tzinfo=timezone.utc)
|
||||
return value
|
||||
# String form — strip trailing Z or +00:00 variants, then attach UTC.
|
||||
s = str(value).strip()
|
||||
for suffix in ("+00:00", "Z", " UTC"):
|
||||
if s.endswith(suffix):
|
||||
s = s[: -len(suffix)]
|
||||
# SQLite uses space as the T separator in some formats.
|
||||
s = s.replace(" ", "T")
|
||||
try:
|
||||
dt = datetime.fromisoformat(s)
|
||||
except ValueError:
|
||||
# Fallback: strip subseconds if present to handle unusual formats.
|
||||
dt = datetime.strptime(s[:19], "%Y-%m-%dT%H:%M:%S")
|
||||
return dt.replace(tzinfo=timezone.utc)
|
||||
|
||||
|
||||
def _iso(dt: datetime) -> str:
|
||||
"""Serialise a datetime to an ISO 8601 string for SQLite storage."""
|
||||
if dt.tzinfo is not None:
|
||||
dt = dt.astimezone(timezone.utc).replace(tzinfo=None)
|
||||
return dt.strftime("%Y-%m-%dT%H:%M:%S")
|
||||
@@ -0,0 +1,104 @@
|
||||
"""add uuid column to meter table
|
||||
|
||||
Adds a stable ``uuid`` (UUID v4 string) column to the ``meter`` table so that
|
||||
each meter epoch has a durable identity anchor suitable for use as an HA
|
||||
Discovery ``unique_id``.
|
||||
|
||||
**Migration strategy (SQLite-safe)**:
|
||||
|
||||
SQLite does not support adding a NOT NULL + UNIQUE column to a non-empty table
|
||||
in a single ``ALTER TABLE ADD COLUMN`` statement (adding a NOT NULL column
|
||||
without a default value is rejected if the table already has rows). The
|
||||
safe approach used here is:
|
||||
|
||||
1. Add ``uuid`` as a **nullable** column (SQLite allows this).
|
||||
2. **Back-fill** every existing ``meter`` row with a distinct ``str(uuid4())``
|
||||
value. Each row gets its *own* random UUID — not a shared value — so the
|
||||
subsequent UNIQUE constraint is satisfied.
|
||||
3. Use ``batch_alter_table`` (which re-creates the table under the hood in
|
||||
SQLite) to alter the column to ``NOT NULL`` and add a UNIQUE constraint.
|
||||
|
||||
**Idempotency**: only rows where ``uuid IS NULL`` are back-filled; rows that
|
||||
already have a uuid (e.g. from a repeated upgrade after a partial failure) are
|
||||
left untouched.
|
||||
|
||||
**Audit**: after back-fill, the count of rows with ``uuid IS NULL`` must be
|
||||
exactly zero; if not, the migration raises ``RuntimeError`` and rolls back.
|
||||
|
||||
**Data safety**: this migration is additive only — no existing rows are deleted
|
||||
or overwritten; it only adds a new column and fills it in.
|
||||
|
||||
Revision ID: 20260625_14_meter_uuid
|
||||
Revises: 20260625_13_meter_table
|
||||
Create Date: 2026-06-25 00:00:00.000000
|
||||
"""
|
||||
|
||||
import uuid as _uuid
|
||||
from typing import Sequence, Union
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
|
||||
revision: str = "20260625_14_meter_uuid"
|
||||
down_revision: Union[str, None] = "20260625_13_meter_table"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
conn = op.get_bind()
|
||||
|
||||
# ------------------------------------------------------------------ #
|
||||
# 1. Add uuid as a nullable column. #
|
||||
# ------------------------------------------------------------------ #
|
||||
with op.batch_alter_table("meter", schema=None) as batch_op:
|
||||
batch_op.add_column(
|
||||
sa.Column("uuid", sa.String(length=36), nullable=True)
|
||||
)
|
||||
|
||||
# ------------------------------------------------------------------ #
|
||||
# 2. Back-fill: assign a distinct UUID to every row that has #
|
||||
# uuid IS NULL. Each row gets its own random value so that the #
|
||||
# subsequent UNIQUE constraint is satisfied. #
|
||||
# ------------------------------------------------------------------ #
|
||||
rows = conn.execute(sa.text("SELECT id FROM meter WHERE uuid IS NULL")).fetchall()
|
||||
for (meter_id,) in rows:
|
||||
new_uuid = str(_uuid.uuid4())
|
||||
conn.execute(
|
||||
sa.text("UPDATE meter SET uuid = :uuid WHERE id = :mid"),
|
||||
{"uuid": new_uuid, "mid": meter_id},
|
||||
)
|
||||
|
||||
# ------------------------------------------------------------------ #
|
||||
# 3. Audit: verify no rows remain with uuid IS NULL. #
|
||||
# ------------------------------------------------------------------ #
|
||||
null_count_row = conn.execute(
|
||||
sa.text("SELECT COUNT(*) FROM meter WHERE uuid IS NULL")
|
||||
).fetchone()
|
||||
null_count: int = null_count_row[0] if null_count_row else 0
|
||||
|
||||
if null_count != 0:
|
||||
raise RuntimeError(
|
||||
f"meter.uuid back-fill audit failed: {null_count} meter row(s) still have "
|
||||
"uuid IS NULL after back-fill. Migration aborted to protect data integrity."
|
||||
)
|
||||
|
||||
# ------------------------------------------------------------------ #
|
||||
# 4. Alter column to NOT NULL + UNIQUE (requires batch on SQLite). #
|
||||
# batch_alter_table re-creates the table, so the UNIQUE constraint #
|
||||
# and NOT NULL are applied atomically. #
|
||||
# ------------------------------------------------------------------ #
|
||||
with op.batch_alter_table("meter", schema=None) as batch_op:
|
||||
batch_op.alter_column(
|
||||
"uuid",
|
||||
existing_type=sa.String(length=36),
|
||||
nullable=False,
|
||||
)
|
||||
batch_op.create_unique_constraint("uq_meter_uuid", ["uuid"])
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
# Drop the UNIQUE constraint and the uuid column (batch on SQLite).
|
||||
with op.batch_alter_table("meter", schema=None) as batch_op:
|
||||
batch_op.drop_constraint("uq_meter_uuid", type_="unique")
|
||||
batch_op.drop_column("uuid")
|
||||
@@ -9,12 +9,14 @@ from sqlalchemy.orm import Session
|
||||
from app.api.routes.api.deps import require_csrf, require_session
|
||||
from app.config import Settings, get_settings
|
||||
from app.dependencies import get_app_settings, get_db
|
||||
from app.integrations.mqtt import MQTT_SETTINGS_KEYS, mqtt_manager
|
||||
from app.schemas.config import (
|
||||
ConfigField,
|
||||
ConfigResponse,
|
||||
ConfigSection,
|
||||
ConfigUpdateRequest,
|
||||
ConfigUpdateResponse,
|
||||
MqttTestResponse,
|
||||
SmtpTestResponse,
|
||||
)
|
||||
from app.services.auth import AuthenticatedSession
|
||||
@@ -58,7 +60,12 @@ def put_config(
|
||||
|
||||
- Blank secret value keeps the existing stored value (no change).
|
||||
- Invalid values return 422 and nothing is written to the database.
|
||||
- If MQTT-related settings changed, the MQTT client reconnects automatically.
|
||||
"""
|
||||
# Detect whether any MQTT-related key is being submitted (non-secret change
|
||||
# or non-blank secret change) so we know to reconnect after saving.
|
||||
mqtt_keys_submitted = any(k.lower() in MQTT_SETTINGS_KEYS for k in body.updates)
|
||||
|
||||
try:
|
||||
save_config_updates(db, body.updates, settings)
|
||||
except ConfigSaveError as exc:
|
||||
@@ -68,8 +75,23 @@ def put_config(
|
||||
detail="invalid config submission",
|
||||
) from exc
|
||||
|
||||
# Re-read settings after save (save_config_updates clears the settings cache)
|
||||
refreshed_settings = get_settings()
|
||||
# Re-read settings after save (save_config_updates clears the settings cache).
|
||||
# Use build_runtime_settings so the reconnect picks up DB-stored values, not
|
||||
# just the bootstrap env (otherwise a broker configured via the UI is ignored).
|
||||
from app.services.config_page import build_runtime_settings
|
||||
refreshed_settings = build_runtime_settings(db, get_settings())
|
||||
|
||||
# Reconnect MQTT client if any MQTT setting was updated.
|
||||
if mqtt_keys_submitted:
|
||||
logger.info("MQTT settings changed — triggering reconnect.")
|
||||
mqtt_manager.reconnect(refreshed_settings)
|
||||
|
||||
# Re-apply the DSMR subscription so enabling/disabling DSMR ingest (or changing
|
||||
# its topic / sample interval) takes effect immediately, without an app restart.
|
||||
# Done after any MQTT reconnect so it operates on the current client.
|
||||
from app.services.dsmr_ingest import apply_dsmr_subscription
|
||||
apply_dsmr_subscription(refreshed_settings)
|
||||
|
||||
sections_raw = build_config_sections(db, refreshed_settings)
|
||||
return ConfigUpdateResponse(sections=_sections_from_raw(sections_raw))
|
||||
|
||||
@@ -117,3 +139,174 @@ def post_smtp_test(
|
||||
status_code=status.HTTP_200_OK,
|
||||
content={"result": "success", "message": "Test email sent successfully."},
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# POST /api/config/mqtt/test — M5-T10
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class _MqttConfigurationError(ValueError):
|
||||
"""Raised when MQTT settings are incomplete or disabled."""
|
||||
|
||||
|
||||
class _MqttConnectionError(RuntimeError):
|
||||
"""Raised when MQTT broker connection or publish fails."""
|
||||
|
||||
|
||||
@router.post(
|
||||
"/config/mqtt/test",
|
||||
responses={
|
||||
200: {"model": MqttTestResponse},
|
||||
400: {"model": MqttTestResponse},
|
||||
502: {"model": MqttTestResponse},
|
||||
},
|
||||
)
|
||||
def post_mqtt_test(
|
||||
settings: Settings = Depends(get_app_settings),
|
||||
_auth: AuthenticatedSession = Depends(require_session),
|
||||
_csrf: None = Depends(require_csrf),
|
||||
) -> JSONResponse:
|
||||
"""
|
||||
Test MQTT broker connectivity by attempting to connect and publishing a
|
||||
test message to ``<ha_discovery_prefix>/home-automation/test``.
|
||||
|
||||
The message is visible in MQTT Explorer (or any subscriber) so users can
|
||||
confirm the full broker publish path is working.
|
||||
|
||||
Three possible outcomes:
|
||||
- 200 { "result": "success", "message": ... }
|
||||
- 400 { "result": "config-error", "message": ... } (not configured)
|
||||
- 502 { "result": "failed", "message": ... } (connection/publish error)
|
||||
|
||||
MQTT credentials are never echoed in the response.
|
||||
"""
|
||||
try:
|
||||
_run_mqtt_test(settings)
|
||||
except _MqttConfigurationError as exc:
|
||||
logger.warning("MQTT test rejected due to configuration: %s", exc)
|
||||
return JSONResponse(
|
||||
status_code=status.HTTP_400_BAD_REQUEST,
|
||||
content={"result": "config-error", "message": str(exc)},
|
||||
)
|
||||
except _MqttConnectionError as exc:
|
||||
logger.warning("MQTT test connection/publish failed: %s", exc)
|
||||
return JSONResponse(
|
||||
status_code=status.HTTP_502_BAD_GATEWAY,
|
||||
content={"result": "failed", "message": str(exc)},
|
||||
)
|
||||
|
||||
return JSONResponse(
|
||||
status_code=status.HTTP_200_OK,
|
||||
content={
|
||||
"result": "success",
|
||||
"message": (
|
||||
f"Test message published to "
|
||||
f"{settings.ha_discovery_prefix}/home-automation/test."
|
||||
),
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
def _run_mqtt_test(settings: Settings) -> None:
|
||||
"""Attempt a transient MQTT connection and publish a test message.
|
||||
|
||||
Raises
|
||||
------
|
||||
_MqttConfigurationError
|
||||
When MQTT is not enabled or broker host is not configured.
|
||||
_MqttConnectionError
|
||||
When the broker is unreachable or the publish fails.
|
||||
"""
|
||||
import socket
|
||||
|
||||
import paho.mqtt.client as mqtt
|
||||
|
||||
if not settings.mqtt_broker_host:
|
||||
raise _MqttConfigurationError("MQTT broker host is not configured.")
|
||||
|
||||
test_topic = f"{settings.ha_discovery_prefix}/home-automation/test"
|
||||
test_payload = '{"source": "home-automation", "event": "mqtt_test"}'
|
||||
|
||||
connected_event = __import__("threading").Event()
|
||||
published_event = __import__("threading").Event()
|
||||
connect_error: list[str] = []
|
||||
|
||||
client = mqtt.Client(
|
||||
callback_api_version=mqtt.CallbackAPIVersion.VERSION2,
|
||||
client_id="home-automation-test",
|
||||
)
|
||||
|
||||
def _on_connect(
|
||||
_client: mqtt.Client,
|
||||
_userdata: object,
|
||||
_flags: mqtt.ConnectFlags,
|
||||
reason_code: mqtt.ReasonCode,
|
||||
_properties: mqtt.Properties | None,
|
||||
) -> None:
|
||||
if reason_code.is_failure:
|
||||
connect_error.append(f"Broker refused connection: {reason_code}")
|
||||
connected_event.set()
|
||||
|
||||
def _on_publish(
|
||||
_client: mqtt.Client,
|
||||
_userdata: object,
|
||||
_mid: int,
|
||||
_reason_code: mqtt.ReasonCode,
|
||||
_properties: mqtt.Properties | None,
|
||||
) -> None:
|
||||
published_event.set()
|
||||
|
||||
client.on_connect = _on_connect
|
||||
client.on_publish = _on_publish
|
||||
|
||||
if settings.mqtt_tls_enabled:
|
||||
try:
|
||||
client.tls_set()
|
||||
except Exception as exc:
|
||||
raise _MqttConnectionError(f"TLS setup failed: {exc}") from exc
|
||||
|
||||
if settings.mqtt_username:
|
||||
client.username_pw_set(
|
||||
username=settings.mqtt_username,
|
||||
password=settings.mqtt_password or None,
|
||||
)
|
||||
|
||||
client.loop_start()
|
||||
try:
|
||||
try:
|
||||
client.connect(
|
||||
host=settings.mqtt_broker_host,
|
||||
port=settings.mqtt_broker_port,
|
||||
keepalive=10,
|
||||
)
|
||||
except (OSError, socket.error) as exc:
|
||||
# Sanitise: ensure password never leaks into the error message.
|
||||
msg = _sanitize_mqtt_error(str(exc), settings.mqtt_password)
|
||||
raise _MqttConnectionError(f"Cannot reach broker: {msg}") from exc
|
||||
|
||||
# Wait up to 5 s for connection acknowledgement.
|
||||
if not connected_event.wait(timeout=5):
|
||||
raise _MqttConnectionError("Broker connection timed out (5 s).")
|
||||
|
||||
if connect_error:
|
||||
raise _MqttConnectionError(connect_error[0])
|
||||
|
||||
# Publish test message.
|
||||
client.publish(test_topic, payload=test_payload, qos=1, retain=False)
|
||||
# Wait up to 5 s for the publish ACK (QoS 1).
|
||||
if not published_event.wait(timeout=5):
|
||||
raise _MqttConnectionError("Publish timed out (5 s) — broker reachable but no ACK.")
|
||||
|
||||
finally:
|
||||
try:
|
||||
client.disconnect()
|
||||
except Exception:
|
||||
pass
|
||||
client.loop_stop()
|
||||
|
||||
|
||||
def _sanitize_mqtt_error(message: str, password: str | None) -> str:
|
||||
"""Replace *password* in *message* with ``[redacted]``."""
|
||||
if password:
|
||||
return message.replace(password, "[redacted]")
|
||||
return message
|
||||
|
||||
@@ -0,0 +1,584 @@
|
||||
"""Energy data API: prices, costs, summary, DSMR, recompute, Tibber test (M6-T09).
|
||||
|
||||
All endpoints are under /api/energy, require an authenticated session, and
|
||||
write endpoints (POST) additionally require a non-empty X-CSRF-Token header.
|
||||
|
||||
Route prefix note
|
||||
-----------------
|
||||
This router shares the ``/api/energy`` prefix with ``energy_contracts.py``
|
||||
(which handles contract CRUD at /contracts/* and /profiles). The sub-paths
|
||||
used here (/prices, /costs, /costs/summary, /costs/recompute, /dsmr/latest,
|
||||
/tibber/test) are disjoint from the contract router's paths, so there is no
|
||||
conflict.
|
||||
|
||||
Tibber token security
|
||||
---------------------
|
||||
``POST /api/energy/tibber/test`` calls the Tibber API but **never** echoes the
|
||||
token in the response body or in log messages. Three-state logic mirrors the
|
||||
MQTT test endpoint (M5-T10, app/api/routes/api/config.py::post_mqtt_test):
|
||||
|
||||
200 { result: "success", message: ..., price: {...} }
|
||||
400 { result: "config-error", message: ... }
|
||||
502 { result: "failed", message: ... }
|
||||
|
||||
Recompute safety
|
||||
----------------
|
||||
``POST /api/energy/costs/recompute`` is idempotent: it calls
|
||||
``energy_cost.recompute_range`` which upserts existing rows without deleting
|
||||
anything. The endpoint enforces a maximum time-window of 366 days to avoid
|
||||
unbounded recomputation triggered by erroneous client requests.
|
||||
|
||||
Prices endpoint behaviour
|
||||
-------------------------
|
||||
``GET /api/energy/prices`` queries the ``tibber_price`` table for tibber
|
||||
contracts, or derives the effective fixed-tariff prices for manual contracts
|
||||
using the same formula as the billing engine (_manual_strategy in strategies.py):
|
||||
|
||||
buy_dal = energy.buy.dal + energy.energy_tax + energy.ode
|
||||
buy_normal = energy.buy.normal + energy.energy_tax + energy.ode
|
||||
sell_dal = energy.sell.dal (no tax added to sell price)
|
||||
sell_normal = energy.sell.normal
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from datetime import UTC, datetime, timedelta
|
||||
from typing import Any
|
||||
|
||||
from fastapi import APIRouter, Depends, Query, status
|
||||
from fastapi.responses import JSONResponse
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.api.routes.api.deps import require_csrf, require_session
|
||||
from app.dependencies import get_app_settings, get_db
|
||||
from app.config import Settings
|
||||
from app.integrations.tibber.client import (
|
||||
TibberAuthError,
|
||||
TibberError,
|
||||
fetch_current_price,
|
||||
)
|
||||
from app.models.energy import DsmrReading, EnergyCostPeriod, TibberPrice
|
||||
from app.schemas.energy import (
|
||||
CostPeriodSchema,
|
||||
CostsResponse,
|
||||
DsmrLatestResponse,
|
||||
ManualTariffSchema,
|
||||
PricePointSchema,
|
||||
PricesResponse,
|
||||
RecomputeResponse,
|
||||
SummaryResponse,
|
||||
TibberTestPriceSchema,
|
||||
TibberTestResponse,
|
||||
)
|
||||
from app.services.auth import AuthenticatedSession
|
||||
from app.services.contracts import active_contract_version_at
|
||||
from app.services.energy_cost import recompute_range, summarize
|
||||
from app.services.timezone import local_midnight_utc, local_now
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
router = APIRouter(prefix="/api/energy", tags=["api-energy"])
|
||||
|
||||
# Maximum number of cost periods returned per request (mirrors modbus readings cap).
|
||||
_COSTS_LIMIT_MAX = 5000
|
||||
|
||||
# Maximum allowed time-window for recompute to prevent unbounded computation.
|
||||
_RECOMPUTE_MAX_DAYS = 366
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Internal helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _as_utc(dt: datetime) -> datetime:
|
||||
"""Attach UTC tzinfo to a naive datetime (SQLite read-back workaround)."""
|
||||
if dt.tzinfo is None:
|
||||
return dt.replace(tzinfo=UTC)
|
||||
return dt
|
||||
|
||||
|
||||
def _manual_tariff_from_values(values: dict[str, Any]) -> ManualTariffSchema:
|
||||
"""Derive the effective fixed tariff from a manual contract version's values dict.
|
||||
|
||||
Mirrors the _manual_strategy formula (strategies.py):
|
||||
buy_dal = energy.buy.dal + energy.energy_tax + energy.ode
|
||||
buy_normal = energy.buy.normal + energy.energy_tax + energy.ode
|
||||
sell_dal = energy.sell.dal (no tax added)
|
||||
sell_normal = energy.sell.normal
|
||||
"""
|
||||
from decimal import Decimal
|
||||
|
||||
def _d(v: Any) -> Decimal:
|
||||
return Decimal(str(v or 0))
|
||||
|
||||
energy = values.get("energy", {})
|
||||
buy = energy.get("buy", {})
|
||||
sell = energy.get("sell", {})
|
||||
energy_tax = _d(energy.get("energy_tax", 0))
|
||||
ode = _d(energy.get("ode", 0))
|
||||
|
||||
buy_dal = _d(buy.get("dal", 0)) + energy_tax + ode
|
||||
buy_normal = _d(buy.get("normal", 0)) + energy_tax + ode
|
||||
sell_dal = _d(sell.get("dal", 0))
|
||||
sell_normal = _d(sell.get("normal", 0))
|
||||
|
||||
return ManualTariffSchema(
|
||||
buy_dal=float(buy_dal),
|
||||
buy_normal=float(buy_normal),
|
||||
sell_dal=float(sell_dal),
|
||||
sell_normal=float(sell_normal),
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# GET /api/energy/prices
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@router.get("/prices", response_model=PricesResponse)
|
||||
def get_prices(
|
||||
start: datetime | None = Query(
|
||||
default=None,
|
||||
description="Inclusive start of the time window (ISO 8601). "
|
||||
"Defaults to the start of today UTC when omitted.",
|
||||
),
|
||||
end: datetime | None = Query(
|
||||
default=None,
|
||||
description="Inclusive end of the time window (ISO 8601). "
|
||||
"Defaults to the end of tomorrow UTC when omitted.",
|
||||
),
|
||||
limit: int = Query(
|
||||
default=500,
|
||||
ge=1,
|
||||
le=_COSTS_LIMIT_MAX,
|
||||
description="Maximum number of Tibber price points to return.",
|
||||
),
|
||||
db: Session = Depends(get_db),
|
||||
_auth: AuthenticatedSession = Depends(require_session),
|
||||
) -> PricesResponse:
|
||||
"""Return the price curve for the active contract.
|
||||
|
||||
**Tibber contracts** (kind="tibber"):
|
||||
Fetches ``tibber_price`` rows within ``[start, end]``, ordered ascending
|
||||
by ``starts_at``. At most ``limit`` rows are returned (most recent first
|
||||
within the window, then reversed to ascending order — identical to the
|
||||
modbus readings pattern).
|
||||
|
||||
Response ``points`` carries per-slot:
|
||||
- ``buy = total`` (Tibber all-inclusive price)
|
||||
- ``sell = total − energy_tax − sell_adjust`` (from active version values)
|
||||
- ``level`` (Tibber price level, may be null)
|
||||
|
||||
``tariff`` is null.
|
||||
|
||||
**Manual contracts** (kind="manual"):
|
||||
``points`` is empty. ``tariff`` carries the four effective prices
|
||||
derived using the billing engine formula:
|
||||
- ``buy_dal = energy.buy.dal + energy_tax + ode``
|
||||
- ``buy_normal = energy.buy.normal + energy_tax + ode``
|
||||
- ``sell_dal = energy.sell.dal``
|
||||
- ``sell_normal = energy.sell.normal``
|
||||
|
||||
**No active contract**: returns kind=null, currency="EUR", points=[], tariff=null (200).
|
||||
"""
|
||||
now = datetime.now(UTC)
|
||||
|
||||
# Default window: today + tomorrow.
|
||||
if start is None:
|
||||
start = now.replace(hour=0, minute=0, second=0, microsecond=0)
|
||||
if end is None:
|
||||
end = (start + timedelta(days=2)).replace(hour=0, minute=0, second=0, microsecond=0)
|
||||
|
||||
start_utc = _as_utc(start)
|
||||
end_utc = _as_utc(end)
|
||||
|
||||
# Resolve the active contract version at the start of the window.
|
||||
version = active_contract_version_at(db, start_utc)
|
||||
|
||||
if version is None:
|
||||
return PricesResponse(
|
||||
kind=None,
|
||||
currency="EUR",
|
||||
points=[],
|
||||
tariff=None,
|
||||
)
|
||||
|
||||
contract = version.contract
|
||||
currency = contract.currency
|
||||
|
||||
if contract.kind == "tibber":
|
||||
# Fetch tibber_price rows in the window.
|
||||
stmt = (
|
||||
select(TibberPrice)
|
||||
.where(
|
||||
TibberPrice.starts_at >= start_utc,
|
||||
TibberPrice.starts_at <= end_utc,
|
||||
)
|
||||
.order_by(TibberPrice.starts_at.desc())
|
||||
.limit(limit)
|
||||
)
|
||||
rows = list(reversed(db.execute(stmt).scalars().all()))
|
||||
|
||||
# Derive sell price per-point using version values (energy_tax + sell_adjust).
|
||||
from decimal import Decimal
|
||||
|
||||
def _d(v: Any) -> Decimal:
|
||||
return Decimal(str(v or 0))
|
||||
|
||||
energy = version.values.get("energy", {}) if version.values else {}
|
||||
energy_tax = _d(energy.get("energy_tax", 0))
|
||||
sell_adjust = _d(energy.get("sell_adjust", 0))
|
||||
|
||||
points = []
|
||||
for row in rows:
|
||||
total = _d(row.total)
|
||||
sell = float(total - energy_tax - sell_adjust)
|
||||
points.append(
|
||||
PricePointSchema(
|
||||
starts_at=_as_utc(row.starts_at),
|
||||
buy=row.total,
|
||||
sell=sell,
|
||||
level=row.level,
|
||||
)
|
||||
)
|
||||
|
||||
return PricesResponse(
|
||||
kind="tibber",
|
||||
currency=currency,
|
||||
points=points,
|
||||
tariff=None,
|
||||
)
|
||||
|
||||
elif contract.kind == "manual":
|
||||
tariff = _manual_tariff_from_values(version.values or {})
|
||||
return PricesResponse(
|
||||
kind="manual",
|
||||
currency=currency,
|
||||
points=[],
|
||||
tariff=tariff,
|
||||
)
|
||||
|
||||
else:
|
||||
# Unknown kind — return empty response gracefully.
|
||||
return PricesResponse(
|
||||
kind=contract.kind,
|
||||
currency=currency,
|
||||
points=[],
|
||||
tariff=None,
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# GET /api/energy/costs
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@router.get("/costs", response_model=CostsResponse)
|
||||
def get_costs(
|
||||
start: datetime | None = Query(
|
||||
default=None,
|
||||
description="Inclusive lower bound for period_start (ISO 8601).",
|
||||
),
|
||||
end: datetime | None = Query(
|
||||
default=None,
|
||||
description="Inclusive upper bound for period_start (ISO 8601).",
|
||||
),
|
||||
limit: int = Query(
|
||||
default=500,
|
||||
ge=1,
|
||||
le=_COSTS_LIMIT_MAX,
|
||||
description=f"Maximum number of cost periods to return (default 500, max {_COSTS_LIMIT_MAX}).",
|
||||
),
|
||||
db: Session = Depends(get_db),
|
||||
_auth: AuthenticatedSession = Depends(require_session),
|
||||
) -> CostsResponse:
|
||||
"""Return energy_cost_period rows within a time window.
|
||||
|
||||
Rows are ordered by ``period_start`` ascending. When the window contains
|
||||
more rows than ``limit``, the **most recent** N rows are returned (DESC LIMIT),
|
||||
then reversed to ascending order — identical to the modbus readings pattern.
|
||||
|
||||
Query parameters:
|
||||
- ``start``: inclusive lower bound on ``period_start`` (ISO 8601 datetime).
|
||||
- ``end``: inclusive upper bound on ``period_start`` (ISO 8601 datetime).
|
||||
- ``limit``: max rows to return (default 500, max {_COSTS_LIMIT_MAX}).
|
||||
"""
|
||||
stmt = (
|
||||
select(EnergyCostPeriod)
|
||||
.order_by(EnergyCostPeriod.period_start.desc())
|
||||
.limit(limit)
|
||||
)
|
||||
|
||||
if start is not None:
|
||||
stmt = stmt.where(EnergyCostPeriod.period_start >= _as_utc(start))
|
||||
if end is not None:
|
||||
stmt = stmt.where(EnergyCostPeriod.period_start <= _as_utc(end))
|
||||
|
||||
rows = list(reversed(db.execute(stmt).scalars().all()))
|
||||
items = [CostPeriodSchema.model_validate(r) for r in rows]
|
||||
return CostsResponse(items=items, total=len(items))
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# GET /api/energy/costs/summary
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@router.get("/costs/summary", response_model=SummaryResponse)
|
||||
def get_costs_summary(
|
||||
start: datetime | None = Query(
|
||||
default=None,
|
||||
description=(
|
||||
"Inclusive start of the summary interval (ISO 8601). "
|
||||
"Defaults to the start of the current UTC day."
|
||||
),
|
||||
),
|
||||
end: datetime | None = Query(
|
||||
default=None,
|
||||
description=(
|
||||
"Exclusive end of the summary interval (ISO 8601). "
|
||||
"Defaults to the start of the next UTC day (i.e. today's full data)."
|
||||
),
|
||||
),
|
||||
db: Session = Depends(get_db),
|
||||
_auth: AuthenticatedSession = Depends(require_session),
|
||||
) -> SummaryResponse:
|
||||
"""Aggregate billing for a time interval.
|
||||
|
||||
Calls ``energy_cost.summarize(session, start, end)`` which computes:
|
||||
|
||||
total_payable = Σ(net_cost) + fixed_costs − credits
|
||||
|
||||
where ``fixed_costs`` is (network_fee + management_fee) apportioned to the
|
||||
interval length in days (÷30 per month), and ``credits`` is heffingskorting
|
||||
apportioned similarly (÷365 per year).
|
||||
|
||||
Both ``fixed_costs`` and ``credits`` are derived from the **currently active
|
||||
contract version at ``end``**. When no active contract exists they are 0.
|
||||
"""
|
||||
if start is None or end is None:
|
||||
# Default to the server's local today: [local_midnight, next_local_midnight).
|
||||
# This ensures "today" aligns with the local calendar day (NL time) rather
|
||||
# than UTC midnight.
|
||||
local_today = local_now().date()
|
||||
if start is None:
|
||||
start = local_midnight_utc(local_today)
|
||||
if end is None:
|
||||
end = local_midnight_utc(local_today + timedelta(days=1))
|
||||
|
||||
result = summarize(db, _as_utc(start), _as_utc(end))
|
||||
return SummaryResponse(**result)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# GET /api/energy/dsmr/latest
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@router.get("/dsmr/latest", response_model=DsmrLatestResponse)
|
||||
def get_dsmr_latest(
|
||||
db: Session = Depends(get_db),
|
||||
_auth: AuthenticatedSession = Depends(require_session),
|
||||
) -> DsmrLatestResponse:
|
||||
"""Return the most recent dsmr_reading row.
|
||||
|
||||
Returns ``{"found": false, "recorded_at": null, "payload": null}`` (200, not
|
||||
404) when no rows exist yet, so the front-end can distinguish "no data" from
|
||||
a server error.
|
||||
"""
|
||||
row = db.execute(
|
||||
select(DsmrReading)
|
||||
.order_by(DsmrReading.recorded_at.desc())
|
||||
.limit(1)
|
||||
).scalar_one_or_none()
|
||||
|
||||
if row is None:
|
||||
return DsmrLatestResponse(found=False)
|
||||
|
||||
return DsmrLatestResponse(
|
||||
found=True,
|
||||
recorded_at=_as_utc(row.recorded_at),
|
||||
payload=row.payload,
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# POST /api/energy/costs/recompute
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@router.post(
|
||||
"/costs/recompute",
|
||||
responses={
|
||||
200: {"model": RecomputeResponse},
|
||||
422: {"description": "Validation error (missing window or range too large)"},
|
||||
},
|
||||
)
|
||||
def post_recompute(
|
||||
start: datetime = Query(
|
||||
...,
|
||||
description="Inclusive start of the recompute window (ISO 8601). Required.",
|
||||
),
|
||||
end: datetime = Query(
|
||||
...,
|
||||
description="Exclusive end of the recompute window (ISO 8601). Required.",
|
||||
),
|
||||
db: Session = Depends(get_db),
|
||||
_auth: AuthenticatedSession = Depends(require_session),
|
||||
_csrf: None = Depends(require_csrf),
|
||||
) -> RecomputeResponse:
|
||||
"""Idempotently recompute billing records in a time window.
|
||||
|
||||
Calls ``energy_cost.recompute_range(session, start, end)`` which overwrites
|
||||
existing rows (including successful ones) for every UTC quarter-hour boundary
|
||||
in ``[start, end)``.
|
||||
|
||||
**Idempotency**: repeated calls with the same window produce the same
|
||||
outcome. No rows are deleted; only upserted.
|
||||
|
||||
**Window constraint**: the maximum allowed range is {_RECOMPUTE_MAX_DAYS} days.
|
||||
Requests exceeding this return 422.
|
||||
|
||||
Returns the number of periods for which a billing record was written.
|
||||
Periods skipped due to missing contract or missing Tibber price are not counted.
|
||||
"""
|
||||
start_utc = _as_utc(start)
|
||||
end_utc = _as_utc(end)
|
||||
|
||||
if end_utc <= start_utc:
|
||||
from fastapi import HTTPException
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
|
||||
detail="'end' must be strictly after 'start'.",
|
||||
)
|
||||
|
||||
span_days = (end_utc - start_utc).total_seconds() / 86400
|
||||
if span_days > _RECOMPUTE_MAX_DAYS:
|
||||
from fastapi import HTTPException
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
|
||||
detail=(
|
||||
f"Time window is {span_days:.1f} days which exceeds the maximum of "
|
||||
f"{_RECOMPUTE_MAX_DAYS} days. Use a smaller window."
|
||||
),
|
||||
)
|
||||
|
||||
n = recompute_range(db, start_utc, end_utc)
|
||||
logger.info(
|
||||
"POST /api/energy/costs/recompute [%s, %s): wrote %d period(s).",
|
||||
start_utc.isoformat(),
|
||||
end_utc.isoformat(),
|
||||
n,
|
||||
)
|
||||
return RecomputeResponse(recomputed=n)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# POST /api/energy/tibber/test
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@router.post(
|
||||
"/tibber/test",
|
||||
responses={
|
||||
200: {"model": TibberTestResponse},
|
||||
400: {"model": TibberTestResponse},
|
||||
502: {"model": TibberTestResponse},
|
||||
},
|
||||
)
|
||||
def post_tibber_test(
|
||||
settings: Settings = Depends(get_app_settings),
|
||||
_auth: AuthenticatedSession = Depends(require_session),
|
||||
_csrf: None = Depends(require_csrf),
|
||||
) -> JSONResponse:
|
||||
"""Test Tibber API connectivity by fetching the current price point.
|
||||
|
||||
Three possible outcomes:
|
||||
|
||||
- **200** ``{ result: "success", message: ..., price: {...} }``
|
||||
The Tibber API responded with a valid current price. ``price`` contains
|
||||
starts_at, total, energy, tax, currency, and level.
|
||||
|
||||
- **400** ``{ result: "config-error", message: ... }``
|
||||
The Tibber API token is empty or not configured.
|
||||
|
||||
- **502** ``{ result: "failed", message: ... }``
|
||||
The API call failed (authentication rejected, network error, timeout,
|
||||
unexpected response, etc.).
|
||||
|
||||
The API token is **never** included in the response body or logged.
|
||||
"""
|
||||
token = settings.tibber_api_token
|
||||
home_id = settings.tibber_home_id or None # treat empty string as None
|
||||
|
||||
if not token:
|
||||
logger.info("POST /api/energy/tibber/test: no token configured.")
|
||||
return JSONResponse(
|
||||
status_code=status.HTTP_400_BAD_REQUEST,
|
||||
content=TibberTestResponse(
|
||||
result="config-error",
|
||||
message=(
|
||||
"Tibber API token is not configured. "
|
||||
"Set TIBBER_API_TOKEN in the Config page."
|
||||
),
|
||||
price=None,
|
||||
).model_dump(mode="json"),
|
||||
)
|
||||
|
||||
try:
|
||||
price_point = fetch_current_price(token, home_id)
|
||||
except TibberAuthError:
|
||||
logger.warning("POST /api/energy/tibber/test: authentication failed.")
|
||||
return JSONResponse(
|
||||
status_code=status.HTTP_502_BAD_GATEWAY,
|
||||
content=TibberTestResponse(
|
||||
result="failed",
|
||||
message=(
|
||||
"Tibber API authentication failed. "
|
||||
"Check that your API token is correct."
|
||||
),
|
||||
price=None,
|
||||
).model_dump(mode="json"),
|
||||
)
|
||||
except TibberError as exc:
|
||||
logger.warning("POST /api/energy/tibber/test: API call failed — %s", exc)
|
||||
return JSONResponse(
|
||||
status_code=status.HTTP_502_BAD_GATEWAY,
|
||||
content=TibberTestResponse(
|
||||
result="failed",
|
||||
message=f"Tibber API call failed: {exc}",
|
||||
price=None,
|
||||
).model_dump(mode="json"),
|
||||
)
|
||||
|
||||
price_schema = TibberTestPriceSchema(
|
||||
starts_at=price_point.starts_at,
|
||||
total=price_point.total,
|
||||
energy=price_point.energy,
|
||||
tax=price_point.tax,
|
||||
currency=price_point.currency,
|
||||
level=price_point.level,
|
||||
)
|
||||
|
||||
logger.info(
|
||||
"POST /api/energy/tibber/test: success (starts_at=%s, total=%s %s).",
|
||||
price_point.starts_at.isoformat(),
|
||||
price_point.total,
|
||||
price_point.currency,
|
||||
)
|
||||
|
||||
return JSONResponse(
|
||||
status_code=status.HTTP_200_OK,
|
||||
content=TibberTestResponse(
|
||||
result="success",
|
||||
message=(
|
||||
f"Tibber API connected. Current price: "
|
||||
f"{price_point.total} {price_point.currency}/kWh "
|
||||
f"(starts {price_point.starts_at.isoformat()})."
|
||||
),
|
||||
price=price_schema,
|
||||
).model_dump(mode="json"),
|
||||
)
|
||||
@@ -0,0 +1,331 @@
|
||||
"""EnergyContract CRUD, versioning, and pricing-profile API (M6-T04).
|
||||
|
||||
All endpoints are under /api/energy, require an authenticated session, and
|
||||
write endpoints (POST/PATCH) additionally require a non-empty X-CSRF-Token
|
||||
header.
|
||||
|
||||
There is deliberately no DELETE endpoint for contracts: the FK RESTRICT
|
||||
constraint on energy_contract_version.contract_id and
|
||||
energy_cost_period.contract_version_id prevents accidental deletion of
|
||||
contracts that have billing history. Users should instead deactivate a
|
||||
contract (PATCH active=false) to stop it from being used.
|
||||
|
||||
Route ordering note
|
||||
-------------------
|
||||
GET /api/energy/profiles and GET /api/energy/contracts/{id} are on separate
|
||||
path prefixes (/energy/profiles vs /energy/contracts/{id}) so there is no
|
||||
ambiguity even without extra ordering gymnastics.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from datetime import UTC, datetime
|
||||
from typing import Any
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, status
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.api.routes.api.deps import require_csrf, require_session
|
||||
from app.dependencies import get_db
|
||||
from app.integrations.pricing.profiles import (
|
||||
ProfileNotFoundError,
|
||||
ProfileValidationError,
|
||||
list_profiles,
|
||||
)
|
||||
from app.schemas.energy_contract import (
|
||||
ContractCreate,
|
||||
ContractDetailResponse,
|
||||
ContractListResponse,
|
||||
ContractPatch,
|
||||
ContractResponse,
|
||||
ContractVersionResponse,
|
||||
ProfilesResponse,
|
||||
VersionCreate,
|
||||
)
|
||||
from app.services.auth import AuthenticatedSession
|
||||
from app.services import timezone as _tz_mod
|
||||
from app.services.contracts import (
|
||||
ContractVersionError,
|
||||
activate_contract,
|
||||
add_version,
|
||||
create_contract,
|
||||
deactivate_contract,
|
||||
get_contract_or_none,
|
||||
list_contracts,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
router = APIRouter(prefix="/api/energy", tags=["api-energy-contracts"])
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _get_contract_or_404(db: Session, contract_id: int):
|
||||
"""Return the contract with the given id or raise 404."""
|
||||
contract = get_contract_or_none(db, contract_id)
|
||||
if contract is None:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_404_NOT_FOUND,
|
||||
detail=f"Energy contract {contract_id!r} not found.",
|
||||
)
|
||||
return contract
|
||||
|
||||
|
||||
def _contract_detail(db: Session, contract) -> ContractDetailResponse:
|
||||
"""Build a ContractDetailResponse for *contract*, loading versions."""
|
||||
# Eager-load versions ordered by effective_from for consistent display.
|
||||
from sqlalchemy import select
|
||||
|
||||
from app.models.energy import EnergyContractVersion
|
||||
|
||||
versions = list(
|
||||
db.execute(
|
||||
select(EnergyContractVersion)
|
||||
.where(EnergyContractVersion.contract_id == contract.id)
|
||||
.order_by(EnergyContractVersion.effective_from)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
|
||||
version_schemas = [ContractVersionResponse.model_validate(v) for v in versions]
|
||||
return ContractDetailResponse(
|
||||
id=contract.id,
|
||||
name=contract.name,
|
||||
kind=contract.kind,
|
||||
active=contract.active,
|
||||
currency=contract.currency,
|
||||
created_at=contract.created_at,
|
||||
updated_at=contract.updated_at,
|
||||
versions=version_schemas,
|
||||
)
|
||||
|
||||
|
||||
def _raise_422_for_profile_error(exc: Exception) -> Any:
|
||||
"""Convert a ProfileValidationError or ProfileNotFoundError into HTTP 422."""
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
|
||||
detail=str(exc),
|
||||
)
|
||||
|
||||
|
||||
def _localize_effective_from(dt: datetime | None) -> datetime:
|
||||
"""Resolve *dt* to an aware UTC datetime for storage.
|
||||
|
||||
Rules (Principle A):
|
||||
- If *dt* is None → use ``datetime.now(UTC)`` (unchanged from before).
|
||||
- If *dt* is timezone-aware → convert to UTC as-is.
|
||||
- If *dt* is timezone-naive → interpret as server local wall-clock time,
|
||||
localize with ``local_tz()``, then convert to UTC.
|
||||
|
||||
This means a front-end that sends ``"2026-06-25T00:00:00"`` (no Z) has it
|
||||
interpreted as local midnight (e.g. CEST = UTC+2 → stored as 2026-06-24T22:00:00Z),
|
||||
not as UTC midnight.
|
||||
"""
|
||||
if dt is None:
|
||||
return datetime.now(UTC)
|
||||
if dt.tzinfo is not None:
|
||||
# Already aware: convert to UTC.
|
||||
return dt.astimezone(UTC)
|
||||
# Naive: assume server local wall-clock.
|
||||
tz = _tz_mod.local_tz()
|
||||
local_dt = dt.replace(tzinfo=tz)
|
||||
return local_dt.astimezone(UTC)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# GET /api/energy/profiles
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@router.get("/profiles", response_model=ProfilesResponse)
|
||||
def get_profiles(
|
||||
_auth: AuthenticatedSession = Depends(require_session),
|
||||
) -> ProfilesResponse:
|
||||
"""List all available pricing profile structures.
|
||||
|
||||
Returns the full profile structure for each supported contract kind
|
||||
(``manual`` and ``tibber``). The front-end uses this to dynamically
|
||||
render the correct fields and labels for the contract creation/editing form.
|
||||
"""
|
||||
return ProfilesResponse(profiles=list_profiles())
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# GET /api/energy/contracts
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@router.get("/contracts", response_model=ContractListResponse)
|
||||
def list_energy_contracts(
|
||||
db: Session = Depends(get_db),
|
||||
_auth: AuthenticatedSession = Depends(require_session),
|
||||
) -> ContractListResponse:
|
||||
"""List all energy contracts with their active status.
|
||||
|
||||
Returns a flat list (no embedded version history); use
|
||||
GET /api/energy/contracts/{id} to fetch the full version history for a
|
||||
specific contract.
|
||||
"""
|
||||
contracts = list_contracts(db)
|
||||
items = [ContractResponse.model_validate(c) for c in contracts]
|
||||
return ContractListResponse(items=items, total=len(items))
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# POST /api/energy/contracts
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@router.post(
|
||||
"/contracts",
|
||||
response_model=ContractDetailResponse,
|
||||
status_code=status.HTTP_201_CREATED,
|
||||
)
|
||||
def create_energy_contract(
|
||||
body: ContractCreate,
|
||||
db: Session = Depends(get_db),
|
||||
_auth: AuthenticatedSession = Depends(require_session),
|
||||
_csrf: None = Depends(require_csrf),
|
||||
) -> ContractDetailResponse:
|
||||
"""Create a new energy contract with an initial pricing version.
|
||||
|
||||
The ``values`` dict is validated against the YAML profile for the given
|
||||
``kind``; non-conforming values result in 422 Unprocessable Entity.
|
||||
The new contract is created with ``active=False``; use
|
||||
PATCH /api/energy/contracts/{id} with ``active=true`` to activate it.
|
||||
"""
|
||||
effective_from = _localize_effective_from(body.effective_from)
|
||||
|
||||
try:
|
||||
contract = create_contract(
|
||||
db,
|
||||
name=body.name,
|
||||
kind=body.kind,
|
||||
currency=body.currency,
|
||||
values=body.values,
|
||||
effective_from=effective_from,
|
||||
)
|
||||
except (ProfileNotFoundError, ProfileValidationError) as exc:
|
||||
_raise_422_for_profile_error(exc)
|
||||
|
||||
db.commit()
|
||||
db.refresh(contract)
|
||||
logger.info("Created energy contract id=%d name=%r kind=%s", contract.id, contract.name, contract.kind)
|
||||
return _contract_detail(db, contract)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# GET /api/energy/contracts/{id}
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@router.get("/contracts/{contract_id}", response_model=ContractDetailResponse)
|
||||
def get_energy_contract(
|
||||
contract_id: int,
|
||||
db: Session = Depends(get_db),
|
||||
_auth: AuthenticatedSession = Depends(require_session),
|
||||
) -> ContractDetailResponse:
|
||||
"""Return a single energy contract with its full version history.
|
||||
|
||||
Versions are ordered by ``effective_from`` ascending so the caller can
|
||||
easily inspect the pricing timeline.
|
||||
"""
|
||||
contract = _get_contract_or_404(db, contract_id)
|
||||
return _contract_detail(db, contract)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# PATCH /api/energy/contracts/{id}
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@router.patch("/contracts/{contract_id}", response_model=ContractDetailResponse)
|
||||
def patch_energy_contract(
|
||||
contract_id: int,
|
||||
body: ContractPatch,
|
||||
db: Session = Depends(get_db),
|
||||
_auth: AuthenticatedSession = Depends(require_session),
|
||||
_csrf: None = Depends(require_csrf),
|
||||
) -> ContractDetailResponse:
|
||||
"""Partially update a contract: rename or change activation status.
|
||||
|
||||
- ``name``: updates the human-readable label.
|
||||
- ``active=true``: activates this contract (all others are deactivated).
|
||||
- ``active=false``: deactivates this contract (no effect on others).
|
||||
|
||||
At most one contract may be active at any time; the service layer enforces
|
||||
mutual exclusion.
|
||||
"""
|
||||
contract = _get_contract_or_404(db, contract_id)
|
||||
|
||||
if body.name is not None:
|
||||
contract.name = body.name
|
||||
contract.updated_at = datetime.now(UTC)
|
||||
|
||||
if body.active is True:
|
||||
activate_contract(db, contract)
|
||||
elif body.active is False:
|
||||
deactivate_contract(db, contract)
|
||||
|
||||
db.commit()
|
||||
db.refresh(contract)
|
||||
return _contract_detail(db, contract)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# POST /api/energy/contracts/{id}/versions
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@router.post(
|
||||
"/contracts/{contract_id}/versions",
|
||||
response_model=ContractDetailResponse,
|
||||
status_code=status.HTTP_201_CREATED,
|
||||
)
|
||||
def add_contract_version(
|
||||
contract_id: int,
|
||||
body: VersionCreate,
|
||||
db: Session = Depends(get_db),
|
||||
_auth: AuthenticatedSession = Depends(require_session),
|
||||
_csrf: None = Depends(require_csrf),
|
||||
) -> ContractDetailResponse:
|
||||
"""Add a new pricing version to an existing contract.
|
||||
|
||||
This is how price changes are recorded: the current open version is
|
||||
automatically closed (its ``effective_to`` is set to ``body.effective_from``)
|
||||
and a new version is created starting at ``body.effective_from``.
|
||||
|
||||
The ``values`` dict must conform to the contract's pricing profile.
|
||||
Non-conforming values return 422. If ``effective_from`` is not strictly
|
||||
after the previous version's ``effective_from``, 422 is returned without
|
||||
writing any rows.
|
||||
|
||||
Historical versions are never modified; this endpoint is append-only.
|
||||
"""
|
||||
contract = _get_contract_or_404(db, contract_id)
|
||||
effective_from = _localize_effective_from(body.effective_from)
|
||||
|
||||
try:
|
||||
add_version(
|
||||
db,
|
||||
contract,
|
||||
effective_from=effective_from,
|
||||
values=body.values,
|
||||
)
|
||||
except ContractVersionError as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
|
||||
detail=str(exc),
|
||||
)
|
||||
except (ProfileNotFoundError, ProfileValidationError) as exc:
|
||||
_raise_422_for_profile_error(exc)
|
||||
|
||||
db.commit()
|
||||
db.refresh(contract)
|
||||
return _contract_detail(db, contract)
|
||||
@@ -0,0 +1,221 @@
|
||||
"""Expose API routes (M5-T12).
|
||||
|
||||
Three endpoints:
|
||||
GET /api/expose — Return catalog of exposable entities, per-key
|
||||
toggle state, and MQTT/Discovery connection status.
|
||||
PUT /api/expose — Set per-key toggle state (map key → bool);
|
||||
triggers a HA Discovery re-publish on success.
|
||||
POST /api/expose/republish — Manually trigger a full HA Discovery re-publish.
|
||||
|
||||
Auth model:
|
||||
- GET: session required (no CSRF — read-only).
|
||||
- PUT, POST: session + CSRF required.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from datetime import UTC, datetime
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, status
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.api.routes.api.deps import require_csrf, require_session
|
||||
from app.config import get_settings
|
||||
from app.dependencies import get_db
|
||||
from app.services.config_page import build_runtime_settings
|
||||
from app.integrations.expose import CatalogEntry, build_catalog
|
||||
from app.integrations.mqtt import mqtt_manager
|
||||
from app.models.expose import ExposedEntityToggle
|
||||
from app.schemas.expose import (
|
||||
CatalogEntrySchema,
|
||||
DeviceInfoSchema,
|
||||
ExposeResponse,
|
||||
ExposeUpdateRequest,
|
||||
ExposeUpdateResponse,
|
||||
ExposableEntitySchema,
|
||||
MqttStatusSchema,
|
||||
RepublishResponse,
|
||||
)
|
||||
from app.services.auth import AuthenticatedSession
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
router = APIRouter(prefix="/api", tags=["api-expose"])
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Internal helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _get_mqtt_status(db: Session) -> MqttStatusSchema:
|
||||
"""Build MQTT/Discovery status from live runtime state (DB-merged settings)."""
|
||||
settings = build_runtime_settings(db, get_settings())
|
||||
return MqttStatusSchema(
|
||||
mqtt_configured=mqtt_manager.is_configured(settings),
|
||||
mqtt_connected=mqtt_manager.is_connected,
|
||||
discovery_enabled=bool(settings.ha_discovery_enabled),
|
||||
)
|
||||
|
||||
|
||||
def _entry_to_schema(entry: CatalogEntry) -> CatalogEntrySchema:
|
||||
"""Convert a CatalogEntry to its Pydantic schema form.
|
||||
|
||||
Excludes ``value_getter`` because it is a non-serialisable callable.
|
||||
"""
|
||||
entity = entry.entity
|
||||
return CatalogEntrySchema(
|
||||
entity=ExposableEntitySchema(
|
||||
key=entity.key,
|
||||
component=entity.component,
|
||||
device=DeviceInfoSchema(
|
||||
identifiers=list(entity.device.identifiers),
|
||||
name=entity.device.name,
|
||||
),
|
||||
device_class=entity.device_class,
|
||||
unit=entity.unit,
|
||||
name=entity.name,
|
||||
state_class=entity.state_class,
|
||||
),
|
||||
enabled=entry.enabled,
|
||||
)
|
||||
|
||||
|
||||
def _build_response_data(
|
||||
session: Session,
|
||||
) -> tuple[list[CatalogEntrySchema], MqttStatusSchema]:
|
||||
"""Return (catalog_entries, mqtt_status) for building GET / PUT responses."""
|
||||
catalog = build_catalog(session)
|
||||
catalog_schema = [_entry_to_schema(e) for e in catalog]
|
||||
mqtt_status = _get_mqtt_status(session)
|
||||
return catalog_schema, mqtt_status
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# GET /api/expose
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@router.get("/expose", response_model=ExposeResponse)
|
||||
def get_expose(
|
||||
db: Session = Depends(get_db),
|
||||
_auth: AuthenticatedSession = Depends(require_session),
|
||||
) -> ExposeResponse:
|
||||
"""Return the full exposable-entity catalog with toggle states and MQTT status.
|
||||
|
||||
The catalog is computed dynamically from registered providers (e.g. the
|
||||
Modbus provider enumerates all enabled devices and their metric entities).
|
||||
Toggle states come from the ``exposed_entity_toggle`` table; entities with
|
||||
no row default to ``enabled=False``.
|
||||
"""
|
||||
catalog_schema, mqtt_status = _build_response_data(db)
|
||||
return ExposeResponse(catalog=catalog_schema, mqtt_status=mqtt_status)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# PUT /api/expose
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@router.put("/expose", response_model=ExposeUpdateResponse)
|
||||
def put_expose(
|
||||
body: ExposeUpdateRequest,
|
||||
db: Session = Depends(get_db),
|
||||
_auth: AuthenticatedSession = Depends(require_session),
|
||||
_csrf: None = Depends(require_csrf),
|
||||
) -> ExposeUpdateResponse:
|
||||
"""Set per-entity toggle state.
|
||||
|
||||
Accepts a map of ``{key: bool}`` and upserts rows in the
|
||||
``exposed_entity_toggle`` table. Only keys present in ``body.toggles``
|
||||
are touched; other entities' toggles are left unchanged.
|
||||
|
||||
After writing the toggles, triggers a HA Discovery re-publish so any
|
||||
changes (enabled ↔ disabled) are reflected in Home Assistant immediately.
|
||||
"""
|
||||
now = datetime.now(UTC)
|
||||
|
||||
for key, enabled in body.toggles.items():
|
||||
existing = (
|
||||
db.query(ExposedEntityToggle)
|
||||
.filter(ExposedEntityToggle.key == key)
|
||||
.first()
|
||||
)
|
||||
if existing is not None:
|
||||
existing.enabled = enabled
|
||||
existing.updated_at = now
|
||||
else:
|
||||
db.add(
|
||||
ExposedEntityToggle(
|
||||
key=key,
|
||||
enabled=enabled,
|
||||
updated_at=now,
|
||||
)
|
||||
)
|
||||
|
||||
db.commit()
|
||||
|
||||
# Trigger discovery re-publish after toggle change.
|
||||
_trigger_republish(db)
|
||||
|
||||
catalog_schema, mqtt_status = _build_response_data(db)
|
||||
return ExposeUpdateResponse(catalog=catalog_schema, mqtt_status=mqtt_status)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# POST /api/expose/republish
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@router.post("/expose/republish", response_model=RepublishResponse)
|
||||
def post_expose_republish(
|
||||
db: Session = Depends(get_db),
|
||||
_auth: AuthenticatedSession = Depends(require_session),
|
||||
_csrf: None = Depends(require_csrf),
|
||||
) -> RepublishResponse:
|
||||
"""Manually trigger a full HA Discovery re-publish.
|
||||
|
||||
Calls ``publish_discovery(session)`` from the HA discovery service (M5-T11).
|
||||
Returns a status indicating whether the publish was attempted (or skipped
|
||||
because MQTT / discovery is not enabled / connected).
|
||||
"""
|
||||
settings = build_runtime_settings(db, get_settings())
|
||||
if not (settings.mqtt_enabled and settings.ha_discovery_enabled and mqtt_manager.is_connected):
|
||||
return RepublishResponse(
|
||||
ok=False,
|
||||
message="MQTT / HA Discovery not enabled or broker not connected.",
|
||||
)
|
||||
|
||||
try:
|
||||
_trigger_republish(db)
|
||||
return RepublishResponse(
|
||||
ok=True,
|
||||
message="HA Discovery re-published successfully.",
|
||||
)
|
||||
except Exception as exc:
|
||||
logger.exception("post_expose_republish: unexpected error during publish")
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
|
||||
detail="Discovery re-publish failed.",
|
||||
) from exc
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Internal: trigger discovery re-publish
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _trigger_republish(session: Session) -> None:
|
||||
"""Call publish_discovery from the HA discovery service.
|
||||
|
||||
No-op if MQTT / discovery is not enabled or broker is not connected
|
||||
(publish_discovery guards internally). All errors are swallowed to avoid
|
||||
breaking the API response.
|
||||
"""
|
||||
try:
|
||||
from app.services.ha_discovery import publish_discovery
|
||||
|
||||
publish_discovery(session)
|
||||
except Exception:
|
||||
logger.exception("_trigger_republish: publish_discovery raised an error")
|
||||
@@ -0,0 +1,331 @@
|
||||
"""Meter CRUD, swap declaration, and retroactive recompute API (M7-T05).
|
||||
|
||||
All endpoints are under /api/energy/meters, require an authenticated session,
|
||||
and write endpoints (POST/PATCH) additionally require a non-empty X-CSRF-Token
|
||||
header.
|
||||
|
||||
Route semantics
|
||||
---------------
|
||||
GET /api/energy/meters — list all meter epochs (ascending started_at)
|
||||
POST /api/energy/meters — declare a meter swap / initial meter epoch
|
||||
PATCH /api/energy/meters/{id} — edit label / note, or correct started_at (retroactive)
|
||||
|
||||
Retroactive recompute
|
||||
---------------------
|
||||
Whenever a write operation changes a meter's ``started_at`` (new declaration
|
||||
or PATCH correction), the affected billing window is re-judged via
|
||||
``recompute_range``:
|
||||
|
||||
- **POST** (new meter, possibly retroactive):
|
||||
window = [new_meter.started_at, now)
|
||||
Rationale: the new meter's ``started_at`` closes the previous meter at that
|
||||
point; all periods from that boundary forward may have a different meter
|
||||
attribution. Using ``now`` as the upper bound is safe because
|
||||
``recompute_range`` only processes closed quarters and the operation is
|
||||
idempotent.
|
||||
|
||||
- **PATCH started_at** (retroactive correction):
|
||||
window = [min(old_started_at, new_started_at), now)
|
||||
Rationale: shifting the boundary in either direction affects all periods
|
||||
between the old and new boundary (and potentially beyond if re-attribution
|
||||
cascades). Using the minimum of the two timestamps guarantees the entire
|
||||
affected range is covered; using ``now`` as the upper bound is safe and
|
||||
idempotent.
|
||||
|
||||
``started_at`` localisation (Principle A, FU10 convention)
|
||||
----------------------------------------------------------
|
||||
If the client sends a timezone-naive ``started_at`` value, it is interpreted as
|
||||
the **server's local wall-clock time** and converted to UTC before storage.
|
||||
Timezone-aware values are converted to UTC as-is. This is identical to the
|
||||
``_localize_effective_from`` convention used in ``energy_contracts.py``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from datetime import UTC, datetime
|
||||
from typing import Optional
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, status
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.api.routes.api.deps import require_csrf, require_session
|
||||
from app.dependencies import get_db
|
||||
from app.models.energy import Meter
|
||||
from app.schemas.meter import (
|
||||
MeterDeclareRequest,
|
||||
MeterListResponse,
|
||||
MeterPatchRequest,
|
||||
MeterResponse,
|
||||
)
|
||||
from app.services import timezone as _tz_mod
|
||||
from app.services.auth import AuthenticatedSession
|
||||
from app.services.energy_cost import recompute_range
|
||||
from app.services.meters import (
|
||||
MeterIntervalError,
|
||||
MeterOverlapError,
|
||||
declare_meter,
|
||||
list_meters,
|
||||
update_meter,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
router = APIRouter(prefix="/api/energy", tags=["api-energy-meters"])
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _trigger_discovery_republish(session: Session) -> None:
|
||||
"""Call publish_discovery after a meter write operation (best-effort).
|
||||
|
||||
No-op if MQTT / discovery is not enabled or the broker is not connected
|
||||
(publish_discovery guards internally). All errors are swallowed so that a
|
||||
discovery failure never breaks the API response.
|
||||
|
||||
Must be called **after** db.commit() so that publish_discovery sees the
|
||||
final committed state of the meter table when it rebuilds the catalog.
|
||||
"""
|
||||
try:
|
||||
from app.services.ha_discovery import publish_discovery
|
||||
|
||||
publish_discovery(session)
|
||||
except Exception:
|
||||
logger.exception("_trigger_discovery_republish: publish_discovery raised an error")
|
||||
|
||||
|
||||
def _get_meter_or_404(db: Session, meter_id: int) -> Meter:
|
||||
"""Return the meter with the given id or raise 404."""
|
||||
meter: Optional[Meter] = db.get(Meter, meter_id)
|
||||
if meter is None:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_404_NOT_FOUND,
|
||||
detail=f"Meter {meter_id!r} not found.",
|
||||
)
|
||||
return meter
|
||||
|
||||
|
||||
def _localize_started_at(dt: datetime) -> datetime:
|
||||
"""Resolve *dt* to an aware UTC datetime for storage.
|
||||
|
||||
Follows the same Principle-A convention as ``_localize_effective_from``
|
||||
in ``energy_contracts.py`` (FU10):
|
||||
|
||||
- Timezone-aware → convert to UTC as-is.
|
||||
- Timezone-naive → interpret as server local wall-clock time, localize
|
||||
with ``local_tz()``, then convert to UTC.
|
||||
|
||||
A front-end sending ``"2026-06-25T00:00:00"`` (no Z) has it interpreted
|
||||
as local midnight (e.g. CEST = UTC+2 → stored as 2026-06-24T22:00:00Z),
|
||||
not as UTC midnight.
|
||||
"""
|
||||
if dt.tzinfo is not None:
|
||||
return dt.astimezone(UTC)
|
||||
tz = _tz_mod.local_tz()
|
||||
local_dt = dt.replace(tzinfo=tz)
|
||||
return local_dt.astimezone(UTC)
|
||||
|
||||
|
||||
def _trigger_recompute(db: Session, start: datetime, label: str) -> int:
|
||||
"""Trigger recompute_range from *start* to now (UTC).
|
||||
|
||||
This is the standard "retroactive window" call: everything from the
|
||||
affected boundary up to the current moment needs re-attribution.
|
||||
Using ``now`` as the upper bound is safe because ``recompute_range``
|
||||
only touches closed quarter-hour periods and the operation is idempotent.
|
||||
"""
|
||||
end = datetime.now(UTC)
|
||||
if start >= end:
|
||||
# started_at is in the future — nothing to recompute.
|
||||
logger.info("%s: started_at (%s) is in the future, skipping recompute.", label, start)
|
||||
return 0
|
||||
n = recompute_range(db, start, end)
|
||||
logger.info(
|
||||
"%s: recomputed %d period(s) in window [%s, %s).",
|
||||
label,
|
||||
n,
|
||||
start.isoformat(),
|
||||
end.isoformat(),
|
||||
)
|
||||
return n
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# GET /api/energy/meters
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@router.get("/meters", response_model=MeterListResponse)
|
||||
def list_energy_meters(
|
||||
db: Session = Depends(get_db),
|
||||
_auth: AuthenticatedSession = Depends(require_session),
|
||||
) -> MeterListResponse:
|
||||
"""List all meter epochs in ascending ``started_at`` order.
|
||||
|
||||
Returns the full historical sequence of meter installations across all
|
||||
commodities. The active meter (``ended_at=null``) appears last because it
|
||||
has the latest ``started_at``.
|
||||
"""
|
||||
meters = list_meters(db)
|
||||
items = [MeterResponse.model_validate(m) for m in meters]
|
||||
return MeterListResponse(items=items, total=len(items))
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# POST /api/energy/meters
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@router.post(
|
||||
"/meters",
|
||||
response_model=MeterResponse,
|
||||
status_code=status.HTTP_201_CREATED,
|
||||
)
|
||||
def declare_energy_meter(
|
||||
body: MeterDeclareRequest,
|
||||
db: Session = Depends(get_db),
|
||||
_auth: AuthenticatedSession = Depends(require_session),
|
||||
_csrf: None = Depends(require_csrf),
|
||||
) -> MeterResponse:
|
||||
"""Declare a new meter epoch (swap, home move, or initial declaration).
|
||||
|
||||
Closes the current active meter for the given commodity at ``started_at``
|
||||
and opens a new active meter. If no active meter exists, the new meter is
|
||||
simply created without closing anything.
|
||||
|
||||
**Validation**: ``started_at`` must be **≥** the current active meter's
|
||||
own ``started_at`` (no chronological backdate below the active epoch's
|
||||
start). Equal timestamps are allowed (replaces the current meter at the
|
||||
same logical moment). Violation → 422.
|
||||
|
||||
**Retroactive recompute**: if ``started_at`` is in the past, billing
|
||||
records from that point forward are re-judged via ``recompute_range`` to
|
||||
reflect the new meter attribution. The recompute is transparent — the
|
||||
response body is the created meter (``MeterResponse``) only and does **not**
|
||||
include a recompute count; callers should re-fetch costs if they need the
|
||||
updated totals.
|
||||
"""
|
||||
started_at_utc = _localize_started_at(body.started_at)
|
||||
|
||||
try:
|
||||
new_meter = declare_meter(
|
||||
db,
|
||||
label=body.label,
|
||||
started_at=started_at_utc,
|
||||
reason=body.reason.value,
|
||||
commodity=body.commodity,
|
||||
note=body.note,
|
||||
)
|
||||
except MeterOverlapError as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
|
||||
detail=str(exc),
|
||||
)
|
||||
|
||||
db.flush() # assign PK before recompute (recompute uses session, needs meter in DB)
|
||||
|
||||
# Retroactive recompute: re-judge attribution from the new boundary onward.
|
||||
now = datetime.now(UTC)
|
||||
if started_at_utc < now:
|
||||
_trigger_recompute(db, started_at_utc, "POST /api/energy/meters")
|
||||
|
||||
db.commit()
|
||||
db.refresh(new_meter)
|
||||
|
||||
# Trigger HA discovery re-publish so the new active meter's energy-cost
|
||||
# device/sensor configuration is pushed to Home Assistant. Best-effort:
|
||||
# failures are logged and swallowed; the API response is not affected.
|
||||
_trigger_discovery_republish(db)
|
||||
|
||||
logger.info(
|
||||
"POST /api/energy/meters: declared %r meter id=%d label=%r started_at=%s",
|
||||
body.commodity,
|
||||
new_meter.id,
|
||||
new_meter.label,
|
||||
started_at_utc.isoformat(),
|
||||
)
|
||||
return MeterResponse.model_validate(new_meter)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# PATCH /api/energy/meters/{id}
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@router.patch("/meters/{meter_id}", response_model=MeterResponse)
|
||||
def patch_energy_meter(
|
||||
meter_id: int,
|
||||
body: MeterPatchRequest,
|
||||
db: Session = Depends(get_db),
|
||||
_auth: AuthenticatedSession = Depends(require_session),
|
||||
_csrf: None = Depends(require_csrf),
|
||||
) -> MeterResponse:
|
||||
"""Partially update a meter epoch: rename, edit note, or correct started_at.
|
||||
|
||||
- ``label``: updates the human-readable label.
|
||||
- ``note``: updates the free-form note.
|
||||
- ``started_at``: **retroactive correction** — shifts this meter's start
|
||||
boundary. The service layer maintains timeline continuity by also
|
||||
updating the preceding meter's ``ended_at``. Validation:
|
||||
* Must be strictly after the previous meter's own ``started_at``.
|
||||
* Must be strictly before this meter's ``ended_at`` (if closed).
|
||||
Violation → 422.
|
||||
|
||||
**Retroactive recompute when ``started_at`` changes**: billing records in
|
||||
the window ``[min(old, new), now)`` are re-judged to reflect the corrected
|
||||
meter attribution.
|
||||
|
||||
Not found → 404.
|
||||
"""
|
||||
meter = _get_meter_or_404(db, meter_id)
|
||||
|
||||
# Capture old started_at before mutation (needed for recompute window).
|
||||
old_started_at: Optional[datetime] = meter.started_at
|
||||
|
||||
# Localise started_at if provided.
|
||||
new_started_at_utc: Optional[datetime] = None
|
||||
if body.started_at is not None:
|
||||
new_started_at_utc = _localize_started_at(body.started_at)
|
||||
|
||||
try:
|
||||
update_meter(
|
||||
db,
|
||||
meter,
|
||||
label=body.label,
|
||||
note=body.note,
|
||||
started_at=new_started_at_utc,
|
||||
)
|
||||
except MeterIntervalError as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
|
||||
detail=str(exc),
|
||||
)
|
||||
|
||||
# Retroactive recompute if started_at was changed.
|
||||
if new_started_at_utc is not None and old_started_at is not None:
|
||||
# Normalise old_started_at to UTC-aware for comparison.
|
||||
if old_started_at.tzinfo is None:
|
||||
old_started_at = old_started_at.replace(tzinfo=UTC)
|
||||
# Window = [min(old, new), now) — covers all periods whose attribution
|
||||
# may have changed due to the boundary shift in either direction.
|
||||
window_start = min(old_started_at, new_started_at_utc)
|
||||
_trigger_recompute(db, window_start, f"PATCH /api/energy/meters/{meter_id}")
|
||||
|
||||
db.commit()
|
||||
db.refresh(meter)
|
||||
|
||||
# Trigger HA discovery re-publish so label renames on the active meter
|
||||
# propagate to the HA device name. Best-effort: failures are logged and
|
||||
# swallowed; the API response is not affected.
|
||||
_trigger_discovery_republish(db)
|
||||
|
||||
logger.info(
|
||||
"PATCH /api/energy/meters/%d: updated meter label=%r started_at=%s",
|
||||
meter_id,
|
||||
meter.label,
|
||||
meter.started_at,
|
||||
)
|
||||
return MeterResponse.model_validate(meter)
|
||||
@@ -0,0 +1,511 @@
|
||||
"""Modbus device CRUD, readings, metrics, and test-read API (M5-T05).
|
||||
|
||||
All endpoints are under /api/modbus, require an authenticated session, and
|
||||
write endpoints (POST/PATCH/DELETE + test-read) additionally require a
|
||||
non-empty X-CSRF-Token header.
|
||||
|
||||
Deletion safety:
|
||||
DELETE /api/modbus/devices/{uuid} queries the application layer for
|
||||
existing modbus_reading rows before deleting. Without ``cascade`` it
|
||||
returns 409 and suggests disabling the device instead — a friendly guard
|
||||
rather than letting the DB raise. With ``cascade=true`` it removes the
|
||||
readings (and expose toggles) first, then the device. SQLite FK RESTRICT
|
||||
*is* enforced at runtime here (the app sets PRAGMA foreign_keys=ON; see
|
||||
app/db.py), so the cascade delete order matters.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from datetime import UTC, datetime
|
||||
from typing import Any
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Query, status
|
||||
from sqlalchemy import delete as sa_delete, func, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.api.routes.api.deps import require_csrf, require_session
|
||||
from app.dependencies import get_db
|
||||
from app.integrations.modbus import driver as modbus_driver
|
||||
from app.integrations.modbus.driver import ModbusDriverError
|
||||
from app.integrations.modbus.profiles import (
|
||||
ProfileNotFoundError,
|
||||
decode as decode_profile,
|
||||
list_profiles,
|
||||
load_profile,
|
||||
)
|
||||
from app.models.expose import ExposedEntityToggle
|
||||
from app.models.modbus import ModbusDevice, ModbusReading
|
||||
from app.schemas.modbus import (
|
||||
MetricInfo,
|
||||
ModbusDeleteResponse,
|
||||
ModbusDeviceCreate,
|
||||
ModbusDeviceListResponse,
|
||||
ModbusDeviceResponse,
|
||||
ModbusDeviceUpdate,
|
||||
ModbusLatestResponse,
|
||||
ModbusMetricsResponse,
|
||||
ModbusProfilesResponse,
|
||||
ModbusReadingResponse,
|
||||
ModbusReadingsResponse,
|
||||
ModbusTestReadResponse,
|
||||
ProfileSummary,
|
||||
)
|
||||
from app.services.auth import AuthenticatedSession
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
router = APIRouter(prefix="/api/modbus", tags=["api-modbus"])
|
||||
|
||||
# Maximum number of readings that can be returned per request.
|
||||
_READINGS_LIMIT_MAX = 5000
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _get_device_or_404(db: Session, uuid: str) -> ModbusDevice:
|
||||
"""Return the device with the given UUID or raise 404."""
|
||||
device = db.execute(
|
||||
select(ModbusDevice).where(ModbusDevice.uuid == uuid)
|
||||
).scalar_one_or_none()
|
||||
if device is None:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_404_NOT_FOUND,
|
||||
detail=f"Device '{uuid}' not found.",
|
||||
)
|
||||
return device
|
||||
|
||||
|
||||
def _label_from_key(key: str) -> str:
|
||||
"""Derive a human-readable label from a metric key.
|
||||
|
||||
Example: ``"active_power"`` → ``"Active Power"``
|
||||
"""
|
||||
return key.replace("_", " ").title()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# GET /api/modbus/profiles
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@router.get("/profiles", response_model=ModbusProfilesResponse)
|
||||
def get_profiles(
|
||||
_auth: AuthenticatedSession = Depends(require_session),
|
||||
) -> ModbusProfilesResponse:
|
||||
"""List all available Modbus YAML profiles (name + description).
|
||||
|
||||
Intended for the front-end's device-creation profile drop-down.
|
||||
"""
|
||||
pairs = list_profiles()
|
||||
return ModbusProfilesResponse(
|
||||
profiles=[ProfileSummary(name=name, description=desc) for name, desc in pairs]
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Device CRUD
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@router.get("/devices", response_model=ModbusDeviceListResponse)
|
||||
def list_devices(
|
||||
db: Session = Depends(get_db),
|
||||
_auth: AuthenticatedSession = Depends(require_session),
|
||||
) -> ModbusDeviceListResponse:
|
||||
"""Return all Modbus devices (no pagination — device counts stay small)."""
|
||||
devices = db.execute(select(ModbusDevice)).scalars().all()
|
||||
items = [ModbusDeviceResponse.model_validate(d) for d in devices]
|
||||
return ModbusDeviceListResponse(items=items, total=len(items))
|
||||
|
||||
|
||||
@router.post(
|
||||
"/devices",
|
||||
response_model=ModbusDeviceResponse,
|
||||
status_code=status.HTTP_201_CREATED,
|
||||
)
|
||||
def create_device(
|
||||
body: ModbusDeviceCreate,
|
||||
db: Session = Depends(get_db),
|
||||
_auth: AuthenticatedSession = Depends(require_session),
|
||||
_csrf: None = Depends(require_csrf),
|
||||
) -> ModbusDeviceResponse:
|
||||
"""Create a new Modbus device.
|
||||
|
||||
- Validates that the referenced ``profile`` exists; returns 422 if not.
|
||||
- Returns 201 with the created device on success.
|
||||
"""
|
||||
# Validate profile exists (422 if not)
|
||||
try:
|
||||
load_profile(body.profile)
|
||||
except ProfileNotFoundError:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
|
||||
detail=f"Profile '{body.profile}' does not exist.",
|
||||
)
|
||||
|
||||
now = datetime.now(UTC)
|
||||
device = ModbusDevice(
|
||||
friendly_name=body.friendly_name,
|
||||
transport=body.transport,
|
||||
host=body.host,
|
||||
port=body.port,
|
||||
unit_id=body.unit_id,
|
||||
profile=body.profile,
|
||||
poll_interval_s=body.poll_interval_s,
|
||||
enabled=body.enabled,
|
||||
created_at=now,
|
||||
updated_at=now,
|
||||
)
|
||||
db.add(device)
|
||||
db.commit()
|
||||
db.refresh(device)
|
||||
logger.info("Created Modbus device %r (uuid=%s)", device.friendly_name, device.uuid)
|
||||
return ModbusDeviceResponse.model_validate(device)
|
||||
|
||||
|
||||
@router.get("/devices/{uuid}", response_model=ModbusDeviceResponse)
|
||||
def get_device(
|
||||
uuid: str,
|
||||
db: Session = Depends(get_db),
|
||||
_auth: AuthenticatedSession = Depends(require_session),
|
||||
) -> ModbusDeviceResponse:
|
||||
"""Return a single Modbus device by UUID."""
|
||||
device = _get_device_or_404(db, uuid)
|
||||
return ModbusDeviceResponse.model_validate(device)
|
||||
|
||||
|
||||
@router.patch("/devices/{uuid}", response_model=ModbusDeviceResponse)
|
||||
def patch_device(
|
||||
uuid: str,
|
||||
body: ModbusDeviceUpdate,
|
||||
db: Session = Depends(get_db),
|
||||
_auth: AuthenticatedSession = Depends(require_session),
|
||||
_csrf: None = Depends(require_csrf),
|
||||
) -> ModbusDeviceResponse:
|
||||
"""Partially update a Modbus device (including enable/disable).
|
||||
|
||||
Only fields explicitly provided in the request body are updated.
|
||||
Providing ``profile`` triggers a profile-existence check (422 if unknown).
|
||||
"""
|
||||
device = _get_device_or_404(db, uuid)
|
||||
|
||||
update_data = body.model_dump(exclude_none=True)
|
||||
|
||||
# If profile is being changed, validate it exists.
|
||||
if "profile" in update_data:
|
||||
try:
|
||||
load_profile(update_data["profile"])
|
||||
except ProfileNotFoundError:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
|
||||
detail=f"Profile '{update_data['profile']}' does not exist.",
|
||||
)
|
||||
|
||||
for field, value in update_data.items():
|
||||
setattr(device, field, value)
|
||||
|
||||
device.updated_at = datetime.now(UTC)
|
||||
db.commit()
|
||||
db.refresh(device)
|
||||
logger.info("Updated Modbus device %r (uuid=%s)", device.friendly_name, device.uuid)
|
||||
return ModbusDeviceResponse.model_validate(device)
|
||||
|
||||
|
||||
@router.delete(
|
||||
"/devices/{uuid}",
|
||||
status_code=status.HTTP_204_NO_CONTENT,
|
||||
response_model=None,
|
||||
)
|
||||
def delete_device(
|
||||
uuid: str,
|
||||
cascade: bool = Query(default=False),
|
||||
db: Session = Depends(get_db),
|
||||
_auth: AuthenticatedSession = Depends(require_session),
|
||||
_csrf: None = Depends(require_csrf),
|
||||
) -> None | ModbusDeleteResponse:
|
||||
"""Delete a Modbus device.
|
||||
|
||||
**Default behaviour (cascade=false)**:
|
||||
Returns 409 Conflict if the device has any associated readings; use
|
||||
``enabled=false`` to disable it instead.
|
||||
|
||||
**Cascade delete (cascade=true)**:
|
||||
Permanently deletes the device together with all its readings and any
|
||||
``ExposedEntityToggle`` rows whose key matches ``modbus.<uuid>.*``.
|
||||
Also makes a best-effort attempt to clear the device's HA Discovery
|
||||
config topics from MQTT (empty retained payload) before the DB rows
|
||||
are removed. MQTT failures are swallowed — the DB deletion proceeds
|
||||
regardless.
|
||||
Returns HTTP 200 with a ``ModbusDeleteResponse`` JSON body on success.
|
||||
|
||||
**Application-layer safety**: the 409 guard uses an explicit SELECT COUNT
|
||||
query to return a friendly message. FK RESTRICT is enforced at runtime
|
||||
(the app sets ``PRAGMA foreign_keys=ON``), so the cascade path deletes
|
||||
readings before the device.
|
||||
"""
|
||||
from app.services.ha_discovery import clear_device_discovery
|
||||
|
||||
device = _get_device_or_404(db, uuid)
|
||||
|
||||
# Application-layer guard: refuse deletion if any readings exist.
|
||||
reading_count: int = db.execute(
|
||||
select(func.count(ModbusReading.id)).where(ModbusReading.device_id == device.id)
|
||||
).scalar_one()
|
||||
|
||||
if reading_count > 0 and not cascade:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_409_CONFLICT,
|
||||
detail=(
|
||||
f"Device '{uuid}' has {reading_count} associated reading(s) and cannot be "
|
||||
"deleted. Disable the device (set enabled=false) instead to stop polling "
|
||||
"without losing historical data."
|
||||
),
|
||||
)
|
||||
|
||||
if cascade:
|
||||
# --- Cascade deletion path ---
|
||||
logger.info(
|
||||
"Cascade-deleting Modbus device %r (uuid=%s): %d reading(s)",
|
||||
device.friendly_name,
|
||||
uuid,
|
||||
reading_count,
|
||||
)
|
||||
|
||||
# 1. Best-effort HA Discovery cleanup (before DB rows are gone).
|
||||
try:
|
||||
clear_device_discovery(db, uuid)
|
||||
except Exception: # noqa: BLE001
|
||||
logger.exception(
|
||||
"delete_device(cascade): HA discovery cleanup raised for uuid=%s; continuing",
|
||||
uuid,
|
||||
)
|
||||
|
||||
# 2. Delete all readings for this device (must happen before device row).
|
||||
deleted_readings_result = db.execute(
|
||||
sa_delete(ModbusReading).where(ModbusReading.device_id == device.id)
|
||||
)
|
||||
readings_deleted: int = deleted_readings_result.rowcount
|
||||
|
||||
# 3. Delete all ExposedEntityToggle rows for this device.
|
||||
# Keys follow the pattern "modbus.<uuid>.<metric>".
|
||||
toggle_prefix = f"modbus.{uuid}.%"
|
||||
deleted_toggles_result = db.execute(
|
||||
sa_delete(ExposedEntityToggle).where(ExposedEntityToggle.key.like(toggle_prefix))
|
||||
)
|
||||
toggles_deleted: int = deleted_toggles_result.rowcount
|
||||
|
||||
# 4. Delete the device itself.
|
||||
db.delete(device)
|
||||
db.commit()
|
||||
|
||||
logger.info(
|
||||
"Cascade-delete complete for device uuid=%s: %d reading(s), %d toggle(s) removed",
|
||||
uuid,
|
||||
readings_deleted,
|
||||
toggles_deleted,
|
||||
)
|
||||
|
||||
from fastapi.responses import JSONResponse
|
||||
return JSONResponse(
|
||||
status_code=status.HTTP_200_OK,
|
||||
content={
|
||||
"deleted": True,
|
||||
"readings_deleted": readings_deleted,
|
||||
"toggles_deleted": toggles_deleted,
|
||||
},
|
||||
)
|
||||
|
||||
# --- Non-cascade path (no readings exist at this point) ---
|
||||
logger.info("Deleting Modbus device %r (uuid=%s)", device.friendly_name, uuid)
|
||||
db.delete(device)
|
||||
db.commit()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Readings endpoints
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@router.get("/devices/{uuid}/latest", response_model=ModbusLatestResponse)
|
||||
def get_latest_reading(
|
||||
uuid: str,
|
||||
db: Session = Depends(get_db),
|
||||
_auth: AuthenticatedSession = Depends(require_session),
|
||||
) -> ModbusLatestResponse:
|
||||
"""Return the most recent reading for a device.
|
||||
|
||||
If no readings exist yet, returns ``{"found": false, "recorded_at": null,
|
||||
"payload": null}`` (200, not 404) so the front-end can distinguish
|
||||
"device exists but has no data" from "device not found".
|
||||
"""
|
||||
device = _get_device_or_404(db, uuid)
|
||||
|
||||
row = db.execute(
|
||||
select(ModbusReading)
|
||||
.where(ModbusReading.device_id == device.id)
|
||||
.order_by(ModbusReading.recorded_at.desc())
|
||||
.limit(1)
|
||||
).scalar_one_or_none()
|
||||
|
||||
if row is None:
|
||||
return ModbusLatestResponse(found=False, recorded_at=None, payload=None)
|
||||
|
||||
return ModbusLatestResponse(
|
||||
found=True,
|
||||
recorded_at=row.recorded_at,
|
||||
payload=row.payload,
|
||||
)
|
||||
|
||||
|
||||
@router.get("/devices/{uuid}/readings", response_model=ModbusReadingsResponse)
|
||||
def get_readings(
|
||||
uuid: str,
|
||||
start: datetime | None = Query(default=None),
|
||||
end: datetime | None = Query(default=None),
|
||||
limit: int = Query(default=500, ge=1, le=_READINGS_LIMIT_MAX),
|
||||
db: Session = Depends(get_db),
|
||||
_auth: AuthenticatedSession = Depends(require_session),
|
||||
) -> ModbusReadingsResponse:
|
||||
"""Return time-range readings for a device.
|
||||
|
||||
When the window contains more rows than ``limit``, the **most recent** N rows
|
||||
are returned (``ORDER BY recorded_at DESC LIMIT n``), then reversed to
|
||||
ascending order before being sent to the client. This ensures that for long
|
||||
time-range requests (e.g. 6 h / 24 h) the caller always sees the latest data
|
||||
rather than the oldest segment of the window.
|
||||
|
||||
When the window has fewer rows than ``limit`` the full window is returned in
|
||||
ascending order — behaviour is identical to a plain ascending query.
|
||||
|
||||
The response schema is unchanged: items are always ``recorded_at`` ascending.
|
||||
|
||||
The query uses the ``(device_id, recorded_at)`` composite index for
|
||||
efficient time-window scans.
|
||||
|
||||
Query parameters:
|
||||
- ``start``: inclusive lower bound (ISO8601 datetime)
|
||||
- ``end``: inclusive upper bound (ISO8601 datetime)
|
||||
- ``limit``: max rows to return (default 500, max {_READINGS_LIMIT_MAX})
|
||||
"""
|
||||
device = _get_device_or_404(db, uuid)
|
||||
|
||||
stmt = (
|
||||
select(ModbusReading)
|
||||
.where(ModbusReading.device_id == device.id)
|
||||
.order_by(ModbusReading.recorded_at.desc())
|
||||
.limit(limit)
|
||||
)
|
||||
|
||||
if start is not None:
|
||||
stmt = stmt.where(ModbusReading.recorded_at >= start)
|
||||
if end is not None:
|
||||
stmt = stmt.where(ModbusReading.recorded_at <= end)
|
||||
|
||||
# Fetch most-recent N rows, then reverse to restore ascending order for the response.
|
||||
rows = list(reversed(db.execute(stmt).scalars().all()))
|
||||
items = [ModbusReadingResponse(recorded_at=r.recorded_at, payload=r.payload) for r in rows]
|
||||
return ModbusReadingsResponse(items=items)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Metrics endpoint
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@router.get("/devices/{uuid}/metrics", response_model=ModbusMetricsResponse)
|
||||
def get_metrics(
|
||||
uuid: str,
|
||||
db: Session = Depends(get_db),
|
||||
_auth: AuthenticatedSession = Depends(require_session),
|
||||
) -> ModbusMetricsResponse:
|
||||
"""Return the metric catalogue for a device's profile.
|
||||
|
||||
Each entry has ``key``, ``label`` (derived from key if the profile has
|
||||
none — underscores → spaces, title-cased), ``unit``, and ``device_class``.
|
||||
This is the authoritative metadata source for front-end card labels and
|
||||
chart axis labels.
|
||||
"""
|
||||
device = _get_device_or_404(db, uuid)
|
||||
|
||||
try:
|
||||
profile = load_profile(device.profile)
|
||||
except ProfileNotFoundError:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
|
||||
detail=f"Device profile '{device.profile}' could not be loaded.",
|
||||
)
|
||||
|
||||
metrics = [
|
||||
MetricInfo(
|
||||
key=m.key,
|
||||
label=_label_from_key(m.key),
|
||||
unit=m.unit,
|
||||
device_class=m.device_class,
|
||||
)
|
||||
for m in profile.metrics
|
||||
]
|
||||
|
||||
return ModbusMetricsResponse(profile=profile.name, metrics=metrics)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Test-read endpoint
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@router.post("/devices/{uuid}/test", response_model=ModbusTestReadResponse)
|
||||
def test_read(
|
||||
uuid: str,
|
||||
db: Session = Depends(get_db),
|
||||
_auth: AuthenticatedSession = Depends(require_session),
|
||||
_csrf: None = Depends(require_csrf),
|
||||
) -> ModbusTestReadResponse:
|
||||
"""Immediately read and decode the device once without persisting any data.
|
||||
|
||||
Useful for validating gateway connectivity and Modbus address settings
|
||||
before relying on the background polling job.
|
||||
|
||||
This is a write-class endpoint (it initiates network I/O on demand) and
|
||||
therefore requires a CSRF token. The response payload is never stored.
|
||||
"""
|
||||
device = _get_device_or_404(db, uuid)
|
||||
|
||||
try:
|
||||
profile = load_profile(device.profile)
|
||||
except ProfileNotFoundError:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
|
||||
detail=f"Device profile '{device.profile}' could not be loaded.",
|
||||
)
|
||||
|
||||
try:
|
||||
registers = modbus_driver.read_blocks(
|
||||
device.host,
|
||||
device.port,
|
||||
device.unit_id,
|
||||
[{"start": b.start, "count": b.count} for b in profile.blocks],
|
||||
function_code=profile.function_code,
|
||||
)
|
||||
payload: dict[str, Any] = decode_profile(profile, registers)
|
||||
return ModbusTestReadResponse(ok=True, payload=payload)
|
||||
|
||||
except ModbusDriverError as exc:
|
||||
logger.warning(
|
||||
"Test read failed for device %r (uuid=%s): %s",
|
||||
device.friendly_name,
|
||||
uuid,
|
||||
exc,
|
||||
)
|
||||
return ModbusTestReadResponse(ok=False, error=str(exc))
|
||||
except Exception as exc: # noqa: BLE001
|
||||
logger.warning(
|
||||
"Unexpected error during test read for device %r (uuid=%s): %s",
|
||||
device.friendly_name,
|
||||
uuid,
|
||||
exc,
|
||||
)
|
||||
return ModbusTestReadResponse(ok=False, error=f"Unexpected error: {exc}")
|
||||
@@ -2,7 +2,7 @@ from __future__ import annotations
|
||||
|
||||
import logging
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Response, status
|
||||
from fastapi import APIRouter, Depends, HTTPException, Request, Response, status
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.api.routes.api.deps import require_csrf, require_session
|
||||
@@ -14,6 +14,12 @@ from app.schemas.session import (
|
||||
SessionResponse,
|
||||
SessionUser,
|
||||
)
|
||||
from app.schemas.totp import (
|
||||
TotpDisableRequest,
|
||||
TotpEnableRequest,
|
||||
TotpSetupResponse,
|
||||
TotpStatusResponse,
|
||||
)
|
||||
from app.services.auth import (
|
||||
AuthPasswordChangeError,
|
||||
AuthenticatedSession,
|
||||
@@ -22,6 +28,8 @@ from app.services.auth import (
|
||||
create_session,
|
||||
revoke_session,
|
||||
)
|
||||
from app.services import login_throttle
|
||||
from app.services import totp as totp_service
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -46,9 +54,26 @@ def get_session(
|
||||
return _build_session_response(auth)
|
||||
|
||||
|
||||
def _get_client_ip(request: Request, *, trust_forwarded_for: bool) -> str:
|
||||
"""Extract the client IP address from the request.
|
||||
|
||||
When ``trust_forwarded_for`` is True (reverse-proxy deployments) the
|
||||
left-most value from the ``X-Forwarded-For`` header is used. Otherwise the
|
||||
direct socket IP (``request.client.host``) is used.
|
||||
"""
|
||||
if trust_forwarded_for:
|
||||
xff = request.headers.get("X-Forwarded-For", "")
|
||||
if xff:
|
||||
return xff.split(",")[0].strip()
|
||||
if request.client is not None:
|
||||
return request.client.host
|
||||
return "unknown"
|
||||
|
||||
|
||||
@router.post("/auth/login", response_model=SessionResponse)
|
||||
def post_login(
|
||||
body: LoginRequest,
|
||||
request: Request,
|
||||
response: Response,
|
||||
db: Session = Depends(get_db),
|
||||
settings: Settings = Depends(get_app_settings),
|
||||
@@ -58,15 +83,67 @@ def post_login(
|
||||
|
||||
On success, sets an HttpOnly session cookie and returns the session user + CSRF token.
|
||||
On failure, returns 401 with no cookie set.
|
||||
Repeated failures trigger exponential back-off (429 + Retry-After).
|
||||
No X-CSRF-Token required (unauthenticated endpoint).
|
||||
"""
|
||||
client_ip = _get_client_ip(request, trust_forwarded_for=settings.auth_trust_forwarded_for)
|
||||
|
||||
# --- Throttle check (before any password verification) ---
|
||||
if settings.auth_login_throttle_enabled:
|
||||
wait_seconds = login_throttle.check_and_get_wait(
|
||||
db, ip=client_ip, username=body.username
|
||||
)
|
||||
if wait_seconds > 0:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_429_TOO_MANY_REQUESTS,
|
||||
detail="too many failed login attempts; please try again later",
|
||||
headers={"Retry-After": str(wait_seconds)},
|
||||
)
|
||||
|
||||
# --- Password verification ---
|
||||
user = authenticate_user(db, username=body.username, password=body.password)
|
||||
if user is None:
|
||||
if settings.auth_login_throttle_enabled:
|
||||
login_throttle.register_failure(db, ip=client_ip, username=body.username)
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||
detail="invalid username or password",
|
||||
)
|
||||
|
||||
# --- TOTP second-factor check (only when TOTP is enabled for the user) ---
|
||||
if user.totp_enabled:
|
||||
if not body.totp_code:
|
||||
# Password correct but no TOTP code supplied: signal the front-end to
|
||||
# prompt for the second factor. Do NOT issue a session.
|
||||
# Deliberate: this is a normal two-step protocol step from a legitimate
|
||||
# user — we do NOT register a throttle failure here. The attacker must
|
||||
# already have the correct password to reach this branch, and counting
|
||||
# each normal first-step as a failure would risk locking out the real
|
||||
# admin on every login attempt.
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||
detail={"totp_required": True},
|
||||
)
|
||||
|
||||
# TOTP code supplied — verify against time-based code or a recovery code.
|
||||
totp_ok = totp_service.verify_totp_code(user, body.totp_code)
|
||||
if not totp_ok:
|
||||
totp_ok = totp_service.verify_recovery_code(db, user=user, code=body.totp_code)
|
||||
|
||||
if not totp_ok:
|
||||
# Second-factor failure is an active attack signal — register failure
|
||||
# so that repeated wrong TOTP/recovery-code guesses trigger back-off.
|
||||
if settings.auth_login_throttle_enabled:
|
||||
login_throttle.register_failure(db, ip=client_ip, username=body.username)
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||
detail="invalid username or password",
|
||||
)
|
||||
|
||||
# --- Success: clear back-off state and issue session ---
|
||||
if settings.auth_login_throttle_enabled:
|
||||
login_throttle.clear(db, ip=client_ip, username=body.username)
|
||||
|
||||
auth_session, raw_token = create_session(db, user=user, settings=settings)
|
||||
logger.info("Created API authenticated session for user '%s'", user.username)
|
||||
|
||||
@@ -139,3 +216,109 @@ def post_change_password(
|
||||
|
||||
logger.info("Password updated for user '%s'", auth.user.username)
|
||||
return Response(status_code=status.HTTP_204_NO_CONTENT)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# TOTP endpoints (M4-T05)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@router.post("/auth/totp/setup", response_model=TotpSetupResponse)
|
||||
def post_totp_setup(
|
||||
db: Session = Depends(get_db),
|
||||
settings: Settings = Depends(get_app_settings),
|
||||
auth: AuthenticatedSession = Depends(require_session),
|
||||
_csrf: None = Depends(require_csrf),
|
||||
) -> TotpSetupResponse:
|
||||
"""
|
||||
Generate a new pending TOTP secret, otpauth URI, and one-time recovery codes.
|
||||
|
||||
The secret is stored in the DB but TOTP is NOT yet enabled (totp_enabled stays
|
||||
False until the user confirms with POST /api/auth/totp/enable).
|
||||
|
||||
Recovery codes are returned here as plaintext exactly once; their Argon2 hashes
|
||||
are persisted immediately so enable only needs to flip the enabled flag.
|
||||
|
||||
Repeating this call replaces any prior pending secret and regenerates codes.
|
||||
|
||||
Requires: session cookie + X-CSRF-Token.
|
||||
"""
|
||||
secret, otpauth_uri, recovery_codes = totp_service.setup(
|
||||
db,
|
||||
user=auth.user,
|
||||
issuer=settings.effective_totp_issuer,
|
||||
)
|
||||
return TotpSetupResponse(
|
||||
secret=secret,
|
||||
otpauth_uri=otpauth_uri,
|
||||
recovery_codes=recovery_codes,
|
||||
)
|
||||
|
||||
|
||||
@router.post("/auth/totp/enable", status_code=status.HTTP_204_NO_CONTENT)
|
||||
def post_totp_enable(
|
||||
body: TotpEnableRequest,
|
||||
db: Session = Depends(get_db),
|
||||
auth: AuthenticatedSession = Depends(require_session),
|
||||
_csrf: None = Depends(require_csrf),
|
||||
) -> Response:
|
||||
"""
|
||||
Enable TOTP by confirming with the current 6-digit code from the authenticator app.
|
||||
|
||||
Requires a prior call to POST /api/auth/totp/setup (so that a pending secret
|
||||
exists). On success, totp_enabled becomes True.
|
||||
|
||||
Returns 400 if the code is wrong or there is no pending secret.
|
||||
Requires: session cookie + X-CSRF-Token.
|
||||
"""
|
||||
ok = totp_service.enable(db, user=auth.user, code=body.code)
|
||||
if not ok:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_400_BAD_REQUEST,
|
||||
detail="invalid TOTP code or no pending setup",
|
||||
)
|
||||
return Response(status_code=status.HTTP_204_NO_CONTENT)
|
||||
|
||||
|
||||
@router.post("/auth/totp/disable", status_code=status.HTTP_204_NO_CONTENT)
|
||||
def post_totp_disable(
|
||||
body: TotpDisableRequest,
|
||||
db: Session = Depends(get_db),
|
||||
auth: AuthenticatedSession = Depends(require_session),
|
||||
_csrf: None = Depends(require_csrf),
|
||||
) -> Response:
|
||||
"""
|
||||
Disable TOTP. The caller must provide exactly one of:
|
||||
- ``password``: the user's current login password, OR
|
||||
- ``code``: the current 6-digit TOTP code.
|
||||
|
||||
On success: totp_enabled=False, totp_secret cleared, all recovery codes deleted.
|
||||
Returns 400 if neither credential matches or neither is provided.
|
||||
Requires: session cookie + X-CSRF-Token.
|
||||
"""
|
||||
ok = totp_service.disable(
|
||||
db,
|
||||
user=auth.user,
|
||||
password=body.password,
|
||||
code=body.code,
|
||||
)
|
||||
if not ok:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_400_BAD_REQUEST,
|
||||
detail="invalid credential; provide a valid password or TOTP code",
|
||||
)
|
||||
return Response(status_code=status.HTTP_204_NO_CONTENT)
|
||||
|
||||
|
||||
@router.get("/auth/totp", response_model=TotpStatusResponse)
|
||||
def get_totp_status(
|
||||
auth: AuthenticatedSession = Depends(require_session),
|
||||
) -> TotpStatusResponse:
|
||||
"""
|
||||
Return the current TOTP status for the authenticated user.
|
||||
|
||||
Response contains only ``{"enabled": bool}``.
|
||||
Secret and recovery codes are NEVER returned here.
|
||||
Requires: session cookie only (no CSRF — read-only).
|
||||
"""
|
||||
return TotpStatusResponse(enabled=auth.user.totp_enabled)
|
||||
|
||||
@@ -37,6 +37,42 @@ class Settings(BaseSettings):
|
||||
auth_session_cookie_name: str = "home_automation_session"
|
||||
auth_session_ttl_hours: int = 12
|
||||
auth_cookie_secure_override: bool | None = True
|
||||
auth_login_throttle_enabled: bool = True
|
||||
auth_trust_forwarded_for: bool = False
|
||||
auth_totp_issuer: str = "" # defaults to app_name when empty
|
||||
|
||||
# Modbus polling — global kill-switch (CONFIG_FIELDS registered in T08).
|
||||
# Off by default: polling is opt-in so a fresh deploy writes no modbus_reading
|
||||
# rows until the admin explicitly enables it (matches mqtt/discovery defaults).
|
||||
modbus_polling_enabled: bool = False
|
||||
|
||||
# MQTT broker connection (T08 wires into CONFIG_FIELDS/UI; T10 builds the client).
|
||||
mqtt_enabled: bool = False
|
||||
mqtt_broker_host: str = ""
|
||||
mqtt_broker_port: int = 1883
|
||||
mqtt_username: str = ""
|
||||
mqtt_password: str = ""
|
||||
mqtt_tls_enabled: bool = False
|
||||
|
||||
# Home Assistant MQTT Discovery (T08 wires into CONFIG_FIELDS/UI; T11 does publishing).
|
||||
ha_discovery_enabled: bool = False
|
||||
ha_discovery_prefix: str = "homeassistant"
|
||||
# State/availability topics use a separate prefix so they live outside the HA discovery
|
||||
# namespace. HA still receives state via the state_topic declared in the discovery config.
|
||||
ha_state_topic_prefix: str = "home_automation"
|
||||
|
||||
# DSMR smart-meter ingest via MQTT (M6; default off — opt-in).
|
||||
# Subscribe to dsmr_mqtt_topic when dsmr_ingest_enabled=True.
|
||||
dsmr_ingest_enabled: bool = False
|
||||
dsmr_mqtt_topic: str = "dsmr/json"
|
||||
dsmr_sample_interval_s: int = 10
|
||||
# DSMR dual-tariff topic: publishes "1" (dal/off-peak) or "2" (normal/peak).
|
||||
# Empty string = do not subscribe (tariff-aware pricing disabled).
|
||||
dsmr_tariff_topic: str = "dsmr/meter-stats/electricity_tariff"
|
||||
|
||||
# Tibber dynamic pricing credentials (M6; only used when active contract kind=tibber).
|
||||
tibber_api_token: str = ""
|
||||
tibber_home_id: str = ""
|
||||
|
||||
model_config = SettingsConfigDict(
|
||||
env_file=".env",
|
||||
@@ -86,6 +122,12 @@ class Settings(BaseSettings):
|
||||
return self.auth_cookie_secure_override
|
||||
return not self.is_development
|
||||
|
||||
@computed_field
|
||||
@property
|
||||
def effective_totp_issuer(self) -> str:
|
||||
"""The issuer label shown in Authenticator apps. Falls back to app_name."""
|
||||
return self.auth_totp_issuer.strip() or self.app_name
|
||||
|
||||
|
||||
@lru_cache
|
||||
def get_settings() -> Settings:
|
||||
|
||||
@@ -28,6 +28,7 @@ def _get_engine(database_url: str) -> Engine:
|
||||
def _enable_sqlite_wal(dbapi_connection, _connection_record):
|
||||
cursor = dbapi_connection.cursor()
|
||||
cursor.execute("PRAGMA journal_mode=WAL")
|
||||
cursor.execute("PRAGMA foreign_keys=ON")
|
||||
cursor.close()
|
||||
|
||||
return engine
|
||||
|
||||
@@ -0,0 +1,879 @@
|
||||
"""Expose framework: ExposableEntity, provider protocol, and build_catalog.
|
||||
|
||||
This module defines:
|
||||
|
||||
- ``ExposableEntity`` — a dataclass describing one publishable MQTT/HA entity.
|
||||
- Provider protocol — a simple callable ``(session) -> list[ExposableEntity]``.
|
||||
- A provider registry and registration helper.
|
||||
- ``build_catalog(session)`` — merges all registered providers' entities with
|
||||
per-key toggle state from the ``exposed_entity_toggle`` table.
|
||||
|
||||
Design notes
|
||||
------------
|
||||
- **No over-engineering**: providers are plain callables, not abstract base
|
||||
classes. This project is personal / single-user; keep it flat.
|
||||
- **Key stability**: entity keys MUST be derived from device uuid + metric key
|
||||
(e.g. ``"modbus.<uuid>.voltage"``), never from auto-increment IDs. Stable
|
||||
keys mean the toggle table survives device removal/re-addition without drift.
|
||||
- **Metadata from profile**: the modbus provider reads ``device_class``,
|
||||
``unit``, and ``component`` directly from the YAML profile's ``metrics[]``
|
||||
rather than maintaining a separate mapping.
|
||||
- **value_getter**: a callable ``() -> object | None`` or ``None`` if not yet
|
||||
implemented (T11 will wire real getters). Presence of the field lets T11
|
||||
call it without changing the dataclass interface.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from dataclasses import dataclass, field
|
||||
from datetime import timedelta
|
||||
from typing import Any, Callable, Optional, Protocol
|
||||
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# ExposableEntity
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@dataclass
|
||||
class DeviceInfo:
|
||||
"""Grouping metadata that maps an entity to a logical HA device.
|
||||
|
||||
``identifiers`` corresponds to ``device.identifiers`` in HA Discovery
|
||||
config — a stable tuple used to group entities under one device card.
|
||||
``name`` is the human-readable device name (friendly_name).
|
||||
"""
|
||||
|
||||
identifiers: tuple[str, ...]
|
||||
"""Stable identifiers for HA device grouping (e.g. ``("modbus", "<uuid>")``).
|
||||
|
||||
These must not change over the device lifetime; they anchor the HA device
|
||||
card even when friendly_name changes.
|
||||
"""
|
||||
|
||||
name: str
|
||||
"""Human-readable device name (may change; triggers HA discovery re-publish)."""
|
||||
|
||||
provides_availability: bool = True
|
||||
"""Whether this device publishes an availability ("online"/"offline") heartbeat.
|
||||
|
||||
When True (the default — e.g. Modbus devices, which have an ``online``
|
||||
binary_sensor driving availability), the HA Discovery config for each entity
|
||||
declares an ``availability`` topic and HA marks the entity unavailable until
|
||||
"online" is published.
|
||||
|
||||
When False (e.g. the energy-cost device, which has only sensors and no
|
||||
heartbeat source), the config omits ``availability`` so HA treats the entity
|
||||
as *always available* and shows its state as soon as one arrives. Without
|
||||
this, such entities would stay perpetually ``unavailable`` in HA even though
|
||||
their state is being published.
|
||||
"""
|
||||
|
||||
|
||||
@dataclass
|
||||
class ExposableEntity:
|
||||
"""One publishable entity in the MQTT / HA Discovery expose framework.
|
||||
|
||||
Fields
|
||||
------
|
||||
key
|
||||
Stable unique identifier for this entity. Used as the toggle table
|
||||
key and as part of the HA Discovery ``unique_id`` derivation.
|
||||
Convention: ``"<provider>.<device_uuid>.<metric_key>"``,
|
||||
e.g. ``"modbus.abc123.voltage"`` or ``"modbus.abc123.online"``.
|
||||
|
||||
component
|
||||
HA MQTT component type: ``"sensor"``, ``"binary_sensor"``, ``"switch"``, etc.
|
||||
|
||||
device
|
||||
Grouping info (DeviceInfo) that determines which HA device card this
|
||||
entity belongs to.
|
||||
|
||||
device_class
|
||||
HA ``device_class`` string (e.g. ``"voltage"``, ``"power"``, ``"None"``).
|
||||
Empty string or ``None`` means no device_class (e.g. for the online sensor).
|
||||
|
||||
unit
|
||||
Physical unit string (e.g. ``"V"``, ``"A"``, ``"kWh"``).
|
||||
Empty string for dimensionless.
|
||||
|
||||
name
|
||||
Human-readable entity label (friendly display name, may include device
|
||||
context, e.g. ``"SDM120 Voltage"``).
|
||||
|
||||
value_getter
|
||||
Optional callable ``(session) -> object | None`` that returns the current
|
||||
entity value given an open SQLAlchemy session. ``None`` means "not yet
|
||||
wired". T11 calls this to obtain the current state before publishing.
|
||||
Signature: ``Callable[[Session], Any]``.
|
||||
|
||||
state_class
|
||||
HA ``state_class`` string (e.g. ``"measurement"``, ``"total_increasing"``).
|
||||
``None`` means not set.
|
||||
"""
|
||||
|
||||
key: str
|
||||
component: str
|
||||
device: DeviceInfo
|
||||
device_class: Optional[str]
|
||||
unit: str
|
||||
name: str
|
||||
value_getter: Optional[Callable[["Session"], Any]] = field(default=None, repr=False)
|
||||
state_class: Optional[str] = None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Provider protocol
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class EntityProvider(Protocol):
|
||||
"""Protocol for entity provider callables.
|
||||
|
||||
A provider is any callable that accepts a ``Session`` and returns a list
|
||||
of ``ExposableEntity`` objects. Using a Protocol (not ABC) keeps the
|
||||
implementation lightweight — any matching callable qualifies.
|
||||
"""
|
||||
|
||||
def __call__(self, session: Session) -> list[ExposableEntity]: ...
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Provider registry
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_REGISTRY: list[EntityProvider] = []
|
||||
|
||||
|
||||
def register_provider(provider: EntityProvider) -> EntityProvider:
|
||||
"""Register an entity provider.
|
||||
|
||||
Can be used as a decorator::
|
||||
|
||||
@register_provider
|
||||
def my_provider(session: Session) -> list[ExposableEntity]:
|
||||
...
|
||||
|
||||
Or called directly::
|
||||
|
||||
register_provider(my_provider)
|
||||
|
||||
Returns the provider unchanged (for decorator compatibility).
|
||||
"""
|
||||
_REGISTRY.append(provider)
|
||||
return provider
|
||||
|
||||
|
||||
def get_providers() -> list[EntityProvider]:
|
||||
"""Return a snapshot of the currently registered providers."""
|
||||
return list(_REGISTRY)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Catalog builder
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@dataclass
|
||||
class CatalogEntry:
|
||||
"""An entity from the catalog, enriched with its current toggle state."""
|
||||
|
||||
entity: ExposableEntity
|
||||
enabled: bool
|
||||
"""True if this entity is currently enabled in the toggle table."""
|
||||
|
||||
|
||||
def build_catalog(session: Session) -> list[CatalogEntry]:
|
||||
"""Enumerate all entities from registered providers and attach toggle state.
|
||||
|
||||
For each entity key, the toggle state is looked up in ``exposed_entity_toggle``.
|
||||
Entities with no toggle row default to ``enabled=False``.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
session:
|
||||
Active SQLAlchemy session for DB access.
|
||||
|
||||
Returns
|
||||
-------
|
||||
list[CatalogEntry]
|
||||
All entities from all registered providers, each paired with its
|
||||
current ``enabled`` state.
|
||||
"""
|
||||
from app.models.expose import ExposedEntityToggle # local import to avoid circular
|
||||
|
||||
# Collect all entities from all providers.
|
||||
all_entities: list[ExposableEntity] = []
|
||||
for provider in _REGISTRY:
|
||||
try:
|
||||
entities = provider(session)
|
||||
all_entities.extend(entities)
|
||||
except Exception:
|
||||
logger.exception("Provider %r raised an error; skipping its entities", provider)
|
||||
|
||||
if not all_entities:
|
||||
return []
|
||||
|
||||
# Load all toggle rows in one query for efficiency.
|
||||
keys = [e.key for e in all_entities]
|
||||
toggle_rows = session.query(ExposedEntityToggle).filter(
|
||||
ExposedEntityToggle.key.in_(keys)
|
||||
).all()
|
||||
toggle_map: dict[str, bool] = {row.key: row.enabled for row in toggle_rows}
|
||||
|
||||
return [
|
||||
CatalogEntry(entity=entity, enabled=toggle_map.get(entity.key, False))
|
||||
for entity in all_entities
|
||||
]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Modbus provider
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _modbus_provider(session: Session) -> list[ExposableEntity]:
|
||||
"""Enumerate ExposableEntity objects for all enabled Modbus devices.
|
||||
|
||||
For each enabled ``ModbusDevice``:
|
||||
- Loads its YAML profile.
|
||||
- Produces one ``sensor`` entity per metric in ``profile.metrics``
|
||||
(device_class / unit / component taken from the metric spec).
|
||||
- Produces one ``binary_sensor`` entity named "online" (derived from
|
||||
``device.last_poll_ok``).
|
||||
|
||||
Entity keys follow the pattern ``"modbus.<uuid>.<metric_key>"``, which is
|
||||
stable across device renames and DB rebuilds.
|
||||
"""
|
||||
from app.models.modbus import ModbusDevice # local import
|
||||
from app.integrations.modbus.profiles import (
|
||||
load_profile,
|
||||
ProfileNotFoundError,
|
||||
ProfileValidationError,
|
||||
)
|
||||
|
||||
devices: list[ModbusDevice] = session.query(ModbusDevice).filter(
|
||||
ModbusDevice.enabled.is_(True)
|
||||
).all()
|
||||
|
||||
entities: list[ExposableEntity] = []
|
||||
|
||||
for device in devices:
|
||||
device_info = DeviceInfo(
|
||||
identifiers=("modbus", device.uuid),
|
||||
name=device.friendly_name,
|
||||
)
|
||||
|
||||
# Load the profile to get metric metadata.
|
||||
try:
|
||||
profile = load_profile(device.profile)
|
||||
except (ProfileNotFoundError, ProfileValidationError) as exc:
|
||||
logger.warning(
|
||||
"Skipping device %r (uuid=%s): cannot load profile %r: %s",
|
||||
device.friendly_name,
|
||||
device.uuid,
|
||||
device.profile,
|
||||
exc,
|
||||
)
|
||||
continue
|
||||
|
||||
# One sensor entity per profile metric.
|
||||
for metric in profile.metrics:
|
||||
entity_key = f"modbus.{device.uuid}.{metric.key}"
|
||||
|
||||
# Capture device id and metric key for the value_getter closure.
|
||||
# The getter queries the most recent ModbusReading for this device
|
||||
# and extracts the metric value from its payload.
|
||||
_device_id = device.id
|
||||
_metric_key = metric.key
|
||||
|
||||
def _make_value_getter(
|
||||
dev_id: int, m_key: str
|
||||
) -> Callable[["Session"], Any]:
|
||||
"""Return a getter that fetches the latest reading value from DB."""
|
||||
def _getter(sess: "Session") -> Any:
|
||||
from app.models.modbus import ModbusReading
|
||||
from sqlalchemy import desc
|
||||
|
||||
reading = (
|
||||
sess.query(ModbusReading)
|
||||
.filter(ModbusReading.device_id == dev_id)
|
||||
.order_by(desc(ModbusReading.recorded_at))
|
||||
.first()
|
||||
)
|
||||
if reading is None:
|
||||
return None
|
||||
return reading.payload.get(m_key)
|
||||
|
||||
return _getter
|
||||
|
||||
entities.append(
|
||||
ExposableEntity(
|
||||
key=entity_key,
|
||||
component=metric.ha_component,
|
||||
device=device_info,
|
||||
device_class=metric.device_class or None,
|
||||
unit=metric.unit,
|
||||
name=f"{device.friendly_name} {metric.key.replace('_', ' ').title()}",
|
||||
value_getter=_make_value_getter(_device_id, _metric_key),
|
||||
state_class=metric.state_class,
|
||||
)
|
||||
)
|
||||
|
||||
# One binary_sensor entity for device "online" status.
|
||||
online_key = f"modbus.{device.uuid}.online"
|
||||
_dev_id_online = device.id
|
||||
|
||||
def _make_online_getter(dev_id: int) -> Callable[["Session"], Any]:
|
||||
"""Return a getter that reports the device online status from DB."""
|
||||
def _getter(sess: "Session") -> Any:
|
||||
from app.models.modbus import ModbusDevice
|
||||
|
||||
dev = sess.query(ModbusDevice).filter(
|
||||
ModbusDevice.id == dev_id
|
||||
).first()
|
||||
if dev is None:
|
||||
return None
|
||||
# Map last_poll_ok to HA binary_sensor payload strings.
|
||||
if dev.last_poll_ok is True:
|
||||
return "ON"
|
||||
return "OFF"
|
||||
|
||||
return _getter
|
||||
|
||||
entities.append(
|
||||
ExposableEntity(
|
||||
key=online_key,
|
||||
component="binary_sensor",
|
||||
device=device_info,
|
||||
device_class="connectivity",
|
||||
unit="",
|
||||
name=f"{device.friendly_name} Online",
|
||||
value_getter=_make_online_getter(_dev_id_online),
|
||||
state_class=None,
|
||||
)
|
||||
)
|
||||
|
||||
return entities
|
||||
|
||||
|
||||
# Register the modbus provider at module load time.
|
||||
register_provider(_modbus_provider)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Energy Cost provider
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
# *_today 当日窗口翻天的宽限:本地午夜后 5 秒才切到新的一天,避免在 00:00:0x 把
|
||||
# 归零后的值发出去、被慢几秒的 HA 钟记成前一天的 23:59:59(归错小时桶)。
|
||||
_TODAY_RESET_GRACE = timedelta(seconds=5)
|
||||
|
||||
|
||||
def _energy_cost_provider(session: Session) -> list[ExposableEntity]:
|
||||
"""Enumerate ExposableEntity objects for the energy cost subsystem.
|
||||
|
||||
Produces 6 sensor entities grouped under a single HA device whose identity
|
||||
is anchored to the **current active electricity meter**:
|
||||
|
||||
- ``buy_price_now`` — current effective buy price (EUR/kWh or local currency).
|
||||
- ``sell_price_now`` — current effective sell price (EUR/kWh or local currency).
|
||||
- ``import_cost_total`` — cumulative import cost (total, monetary).
|
||||
- ``export_revenue_total`` — cumulative export revenue (total, monetary).
|
||||
- ``import_cost_today`` — today's import cost (total_increasing, monetary).
|
||||
- ``export_revenue_today`` — today's export revenue (total_increasing, monetary).
|
||||
|
||||
Active meter requirement
|
||||
------------------------
|
||||
**If no active electricity meter exists, the provider returns ``[]``.**
|
||||
No energy-cost entities are exposed to HA until a meter has been declared.
|
||||
This prevents spurious sensor creation with an undefined device identity.
|
||||
|
||||
HA device identity (换表 → 新 sensor)
|
||||
--------------------------------------
|
||||
``identifiers[1]`` is set to the active meter's **uuid** (not the fixed
|
||||
string ``"energy-cost"``). ``ha_discovery.py`` uses ``identifiers[1]`` as
|
||||
the MQTT node_id and as part of the ``unique_id`` for every entity.
|
||||
Declaring a new active electricity meter produces a new uuid → new node_id /
|
||||
unique_id → HA creates a brand-new sensor, cleanly isolating post-swap data.
|
||||
|
||||
Entity key stability
|
||||
--------------------
|
||||
Entity keys remain the fixed stable strings (``"energy.buy_price_now"`` etc.),
|
||||
**not** derived from the meter uuid. The ``exposed_entity_toggle`` table uses
|
||||
keys as its primary handle; keeping them stable means toggled-on entities stay
|
||||
enabled after a meter swap without requiring the user to re-tick them.
|
||||
|
||||
Current-price algorithm (source-agnostic, with fallback)
|
||||
---------------------------------------------------------
|
||||
Strategy A: read the ``pricing`` snapshot from the most recent non-degraded
|
||||
``energy_cost_period`` row.
|
||||
|
||||
- For ``kind="tibber"``: the snapshot contains ``"buy"`` and ``"sell"`` keys
|
||||
(per-unit prices in the contract currency).
|
||||
- For ``kind="manual"``: ``"buy_normal"`` and ``"sell_normal"`` are used as
|
||||
representative effective per-unit prices. (Both tariff-slot prices differ
|
||||
only by the base rate; energy_tax and ODE are the same for both, so
|
||||
buy_normal is the higher/conservative single representative value.)
|
||||
|
||||
When no non-degraded period exists or the pricing snapshot lacks the
|
||||
expected keys, ``value_getter`` returns ``None`` (``publish_states``
|
||||
skips None values automatically).
|
||||
|
||||
Cumulative totals
|
||||
-----------------
|
||||
``SUM(import_cost)`` and ``SUM(export_revenue)`` over **all non-degraded**
|
||||
``energy_cost_period`` rows within the current meter's window. Degraded rows
|
||||
carry 0 costs and are excluded to avoid double-counting when they are later
|
||||
overwritten by real values.
|
||||
|
||||
Currency
|
||||
--------
|
||||
Taken from the most recent non-degraded period's ``currency`` column.
|
||||
Falls back to ``"EUR"`` when no such row exists.
|
||||
|
||||
Key convention
|
||||
--------------
|
||||
Fixed stable string keys (not derived from any mutable field or DB id):
|
||||
- ``"energy.buy_price_now"``
|
||||
- ``"energy.sell_price_now"``
|
||||
- ``"energy.import_cost_total"``
|
||||
- ``"energy.export_revenue_total"``
|
||||
- ``"energy.import_cost_today"``
|
||||
- ``"energy.export_revenue_today"``
|
||||
|
||||
DeviceInfo identifiers
|
||||
----------------------
|
||||
**Two-element tuple** ``("energy-cost", meter.uuid)`` so that
|
||||
``ha_discovery.py``'s ``entity.device.identifiers[1]`` resolves to the
|
||||
meter uuid (used as the MQTT node_id and unique_id seed throughout).
|
||||
"""
|
||||
from app.models.energy import EnergyCostPeriod, Meter # local import to avoid circular
|
||||
from sqlalchemy import select
|
||||
|
||||
# --- Require an active electricity meter; return [] if none exists ---
|
||||
# Using an inline query (ended_at IS NULL) rather than a service-layer helper
|
||||
# to avoid a new public dependency and remain consistent with the value_getter
|
||||
# implementations below (which use the same inline pattern).
|
||||
active_meter: Meter | None = session.execute(
|
||||
select(Meter)
|
||||
.where(
|
||||
Meter.commodity == "electricity",
|
||||
Meter.ended_at.is_(None),
|
||||
)
|
||||
.limit(1)
|
||||
).scalar_one_or_none()
|
||||
|
||||
if active_meter is None:
|
||||
# No active electricity meter → do not expose any energy-cost entities.
|
||||
# HA will not see these sensors until a meter is declared.
|
||||
return []
|
||||
|
||||
# --- Determine currency from the latest non-degraded row ---
|
||||
|
||||
latest_period: EnergyCostPeriod | None = (
|
||||
session.query(EnergyCostPeriod)
|
||||
.filter(EnergyCostPeriod.degraded.is_(False))
|
||||
.order_by(EnergyCostPeriod.period_start.desc())
|
||||
.first()
|
||||
)
|
||||
|
||||
currency: str = "EUR"
|
||||
if latest_period is not None and latest_period.currency:
|
||||
currency = latest_period.currency
|
||||
|
||||
# --- Shared DeviceInfo anchored to the active meter's uuid ---
|
||||
# identifiers[1] = meter.uuid drives the MQTT node_id and unique_id in
|
||||
# ha_discovery.py. Swapping the meter produces a new uuid → new HA sensor.
|
||||
# provides_availability=False: the energy-cost device has only sensors and no
|
||||
# online/offline heartbeat, so its entities must be "always available" in HA.
|
||||
# (Otherwise HA shows them unavailable despite state being published.)
|
||||
device_info = DeviceInfo(
|
||||
identifiers=("energy-cost", active_meter.uuid),
|
||||
name=active_meter.label,
|
||||
provides_availability=False,
|
||||
)
|
||||
|
||||
# --- value_getter: current buy price ---
|
||||
|
||||
def _make_buy_price_getter() -> Callable[["Session"], Any]:
|
||||
"""Return a getter for the current buy price per kWh.
|
||||
|
||||
For ``kind="manual"`` contracts the getter selects the price slot that
|
||||
matches the current DSMR electricity tariff:
|
||||
|
||||
- tariff == 1 (dal / off-peak) → ``buy_dal``
|
||||
- tariff == 2 (normal / peak) → ``buy_normal``
|
||||
- tariff is None / unknown → ``buy_normal`` (safe fallback = current status quo)
|
||||
|
||||
For ``kind="tibber"`` (hourly dynamic pricing) the tariff slot is not
|
||||
applicable; the getter always uses the ``buy`` key from the snapshot.
|
||||
"""
|
||||
def _getter(sess: "Session") -> Any:
|
||||
from app.models.energy import EnergyCostPeriod as _ECP
|
||||
from app.services.dsmr_ingest import get_current_tariff
|
||||
|
||||
period = (
|
||||
sess.query(_ECP)
|
||||
.filter(_ECP.degraded.is_(False))
|
||||
.order_by(_ECP.period_start.desc())
|
||||
.first()
|
||||
)
|
||||
if period is None or not period.pricing:
|
||||
return None
|
||||
snap = period.pricing
|
||||
kind = snap.get("kind")
|
||||
if kind == "tibber":
|
||||
raw = snap.get("buy")
|
||||
elif kind == "manual":
|
||||
tariff = get_current_tariff()
|
||||
if tariff == 1:
|
||||
raw = snap.get("buy_dal")
|
||||
else:
|
||||
# tariff == 2, None, or any unexpected value → normal (peak) rate.
|
||||
raw = snap.get("buy_normal")
|
||||
else:
|
||||
# Unknown kind — attempt common keys gracefully.
|
||||
raw = snap.get("buy") or snap.get("buy_normal")
|
||||
if raw is None:
|
||||
return None
|
||||
try:
|
||||
return float(raw)
|
||||
except (TypeError, ValueError):
|
||||
return None
|
||||
|
||||
return _getter
|
||||
|
||||
# --- value_getter: current sell price ---
|
||||
|
||||
def _make_sell_price_getter() -> Callable[["Session"], Any]:
|
||||
"""Return a getter for the current sell price per kWh.
|
||||
|
||||
For ``kind="manual"`` contracts the getter selects the price slot that
|
||||
matches the current DSMR electricity tariff:
|
||||
|
||||
- tariff == 1 (dal / off-peak) → ``sell_dal``
|
||||
- tariff == 2 (normal / peak) → ``sell_normal``
|
||||
- tariff is None / unknown → ``sell_normal`` (safe fallback = current status quo)
|
||||
|
||||
For ``kind="tibber"`` the tariff slot is not applicable; always uses ``sell``.
|
||||
"""
|
||||
def _getter(sess: "Session") -> Any:
|
||||
from app.models.energy import EnergyCostPeriod as _ECP
|
||||
from app.services.dsmr_ingest import get_current_tariff
|
||||
|
||||
period = (
|
||||
sess.query(_ECP)
|
||||
.filter(_ECP.degraded.is_(False))
|
||||
.order_by(_ECP.period_start.desc())
|
||||
.first()
|
||||
)
|
||||
if period is None or not period.pricing:
|
||||
return None
|
||||
snap = period.pricing
|
||||
kind = snap.get("kind")
|
||||
if kind == "tibber":
|
||||
raw = snap.get("sell")
|
||||
elif kind == "manual":
|
||||
tariff = get_current_tariff()
|
||||
if tariff == 1:
|
||||
raw = snap.get("sell_dal")
|
||||
else:
|
||||
# tariff == 2, None, or any unexpected value → normal (peak) rate.
|
||||
raw = snap.get("sell_normal")
|
||||
else:
|
||||
raw = snap.get("sell") or snap.get("sell_normal")
|
||||
if raw is None:
|
||||
return None
|
||||
try:
|
||||
return float(raw)
|
||||
except (TypeError, ValueError):
|
||||
return None
|
||||
|
||||
return _getter
|
||||
|
||||
# --- value_getter: cumulative import cost (incl. standing charges) ---
|
||||
|
||||
def _make_import_cost_getter() -> Callable[["Session"], Any]:
|
||||
"""Return a getter for the cumulative import cost plus prorated standing charges.
|
||||
|
||||
D2 (M7): anchor = current active electricity meter's ``started_at``.
|
||||
After a meter swap the cumulative resets to zero for the new meter —
|
||||
old-meter periods have ``period_start < new_meter.started_at`` and fall
|
||||
outside the [anchor, now) window, so they are naturally excluded.
|
||||
Cross-meter boundary periods are already degraded and also excluded.
|
||||
|
||||
No active electricity meter → returns None (cannot anchor; safer than
|
||||
returning a stale or wrong value). The check is done via an inline
|
||||
query (``ended_at IS NULL``) rather than ``meter_at(now)`` to be robust
|
||||
against the edge case where the active meter's ``started_at`` is in the
|
||||
future (``meter_at(now)`` would return None in that scenario).
|
||||
|
||||
No non-degraded periods at all → returns None (has_data guard).
|
||||
|
||||
No active contract → ``summarize`` still runs but fixed_costs = 0;
|
||||
the return value is the pure metered sum within the current meter window.
|
||||
This is consistent with D2: the anchor is the meter, not the contract.
|
||||
|
||||
value = summarize(anchor → now).metered_import + summarize(anchor → now).fixed_costs
|
||||
"""
|
||||
def _getter(sess: "Session") -> Any:
|
||||
from datetime import UTC, datetime as _dt
|
||||
from app.models.energy import EnergyCostPeriod as _ECP, Meter as _Meter
|
||||
from app.services.energy_cost import summarize as _summarize
|
||||
from sqlalchemy import func as _func, select as _select
|
||||
|
||||
# Quick check: any non-degraded period at all?
|
||||
has_data = sess.query(_func.sum(_ECP.import_cost)).filter(
|
||||
_ECP.degraded.is_(False)
|
||||
).scalar()
|
||||
if has_data is None:
|
||||
return None
|
||||
|
||||
# D2: anchor = active electricity meter's started_at.
|
||||
# Inline query (ended_at IS NULL) is more robust than meter_at(now)
|
||||
# because it avoids the edge case where started_at is in the future.
|
||||
active_meter = sess.execute(
|
||||
_select(_Meter)
|
||||
.where(
|
||||
_Meter.commodity == "electricity",
|
||||
_Meter.ended_at.is_(None),
|
||||
)
|
||||
.limit(1)
|
||||
).scalar_one_or_none()
|
||||
|
||||
if active_meter is None:
|
||||
# No active electricity meter — cannot anchor; return None.
|
||||
return None
|
||||
|
||||
from app.services.contracts import _as_utc as _cu
|
||||
anchor_utc = _cu(active_meter.started_at)
|
||||
now_utc = _dt.now(UTC)
|
||||
|
||||
result = _summarize(sess, anchor_utc, now_utc)
|
||||
return result["metered_import"] + result["fixed_costs"]
|
||||
|
||||
return _getter
|
||||
|
||||
# --- value_getter: cumulative export revenue (incl. tax credit) ---
|
||||
|
||||
def _make_export_revenue_getter() -> Callable[["Session"], Any]:
|
||||
"""Return a getter for the cumulative export revenue plus prorated tax credit.
|
||||
|
||||
D2 (M7): anchor = current active electricity meter's ``started_at``.
|
||||
Same reasoning as the import cost getter — see its docstring.
|
||||
|
||||
value = summarize(anchor → now).metered_export + summarize(anchor → now).credits
|
||||
|
||||
Returns None when no non-degraded period exists or no active electricity meter.
|
||||
"""
|
||||
def _getter(sess: "Session") -> Any:
|
||||
from datetime import UTC, datetime as _dt
|
||||
from app.models.energy import EnergyCostPeriod as _ECP, Meter as _Meter
|
||||
from app.services.energy_cost import summarize as _summarize
|
||||
from sqlalchemy import func as _func, select as _select
|
||||
|
||||
# Quick check: any non-degraded period at all?
|
||||
has_data = sess.query(_func.sum(_ECP.export_revenue)).filter(
|
||||
_ECP.degraded.is_(False)
|
||||
).scalar()
|
||||
if has_data is None:
|
||||
return None
|
||||
|
||||
# D2: anchor = active electricity meter's started_at.
|
||||
active_meter = sess.execute(
|
||||
_select(_Meter)
|
||||
.where(
|
||||
_Meter.commodity == "electricity",
|
||||
_Meter.ended_at.is_(None),
|
||||
)
|
||||
.limit(1)
|
||||
).scalar_one_or_none()
|
||||
|
||||
if active_meter is None:
|
||||
# No active electricity meter — cannot anchor; return None.
|
||||
return None
|
||||
|
||||
from app.services.contracts import _as_utc as _cu
|
||||
anchor_utc = _cu(active_meter.started_at)
|
||||
now_utc = _dt.now(UTC)
|
||||
|
||||
result = _summarize(sess, anchor_utc, now_utc)
|
||||
return result["metered_export"] + result["credits"]
|
||||
|
||||
return _getter
|
||||
|
||||
# --- value_getter: today's import cost (local-day window, resets at local midnight) ---
|
||||
|
||||
def _make_import_cost_today_getter() -> Callable[["Session"], Any]:
|
||||
"""Return a getter for today's import cost (metered + standing for today).
|
||||
|
||||
Window: [local today 00:00, local tomorrow 00:00) in UTC.
|
||||
After local midnight the window rolls to the new day → value resets to that
|
||||
day's running cost, implementing "resets at local midnight" semantics for HA.
|
||||
|
||||
Uses timezone module via module-attribute access to preserve monkeypatch safety.
|
||||
Returns None when no non-degraded period exists or no active contract.
|
||||
"""
|
||||
def _getter(sess: "Session") -> Any:
|
||||
from app.services.contracts import active_contract_versions
|
||||
from app.models.energy import EnergyCostPeriod as _ECP
|
||||
from app.services.energy_cost import summarize as _summarize
|
||||
from app.services import timezone as _tz_mod
|
||||
from sqlalchemy import func as _func
|
||||
from datetime import timedelta as _td
|
||||
|
||||
# Quick check: any non-degraded period at all?
|
||||
has_data = sess.query(_func.sum(_ECP.import_cost)).filter(
|
||||
_ECP.degraded.is_(False)
|
||||
).scalar()
|
||||
if has_data is None:
|
||||
return None
|
||||
|
||||
versions = active_contract_versions(sess)
|
||||
if not versions:
|
||||
return None
|
||||
|
||||
# Today's window in UTC, using monkeypatch-safe module attribute calls.
|
||||
# Grace: subtract _TODAY_RESET_GRACE so that in the first 5 seconds after
|
||||
# local midnight the getter still returns yesterday's window. This prevents
|
||||
# a "归零后的值" from being published while HA's clock (which may lag a few
|
||||
# seconds) would stamp it as 23:59:59 of the previous day.
|
||||
today_local = (_tz_mod.local_now() - _TODAY_RESET_GRACE).date()
|
||||
tomorrow_local = today_local + _td(days=1)
|
||||
today_start_utc = _tz_mod.local_midnight_utc(today_local)
|
||||
tomorrow_start_utc = _tz_mod.local_midnight_utc(tomorrow_local)
|
||||
|
||||
result = _summarize(sess, today_start_utc, tomorrow_start_utc)
|
||||
return result["metered_import"] + result["fixed_costs"]
|
||||
|
||||
return _getter
|
||||
|
||||
# --- value_getter: today's export revenue (local-day window, resets at local midnight) ---
|
||||
|
||||
def _make_export_revenue_today_getter() -> Callable[["Session"], Any]:
|
||||
"""Return a getter for today's export revenue (metered + tax credit for today).
|
||||
|
||||
Same window semantics as import_cost_today.
|
||||
Returns None when no non-degraded period exists or no active contract.
|
||||
"""
|
||||
def _getter(sess: "Session") -> Any:
|
||||
from app.services.contracts import active_contract_versions
|
||||
from app.models.energy import EnergyCostPeriod as _ECP
|
||||
from app.services.energy_cost import summarize as _summarize
|
||||
from app.services import timezone as _tz_mod
|
||||
from sqlalchemy import func as _func
|
||||
from datetime import timedelta as _td
|
||||
|
||||
# Quick check: any non-degraded period at all?
|
||||
has_data = sess.query(_func.sum(_ECP.export_revenue)).filter(
|
||||
_ECP.degraded.is_(False)
|
||||
).scalar()
|
||||
if has_data is None:
|
||||
return None
|
||||
|
||||
versions = active_contract_versions(sess)
|
||||
if not versions:
|
||||
return None
|
||||
|
||||
# Grace: same logic as import_cost_today — see that getter's comment.
|
||||
today_local = (_tz_mod.local_now() - _TODAY_RESET_GRACE).date()
|
||||
tomorrow_local = today_local + _td(days=1)
|
||||
today_start_utc = _tz_mod.local_midnight_utc(today_local)
|
||||
tomorrow_start_utc = _tz_mod.local_midnight_utc(tomorrow_local)
|
||||
|
||||
result = _summarize(sess, today_start_utc, tomorrow_start_utc)
|
||||
return result["metered_export"] + result["credits"]
|
||||
|
||||
return _getter
|
||||
|
||||
# Price unit string: "<currency>/kWh"
|
||||
price_unit = f"{currency}/kWh"
|
||||
|
||||
entities: list[ExposableEntity] = [
|
||||
ExposableEntity(
|
||||
key="energy.buy_price_now",
|
||||
component="sensor",
|
||||
device=device_info,
|
||||
device_class=None,
|
||||
unit=price_unit,
|
||||
name="Energy Buy Price Now",
|
||||
value_getter=_make_buy_price_getter(),
|
||||
state_class="measurement",
|
||||
),
|
||||
ExposableEntity(
|
||||
key="energy.sell_price_now",
|
||||
component="sensor",
|
||||
device=device_info,
|
||||
device_class=None,
|
||||
unit=price_unit,
|
||||
name="Energy Sell Price Now",
|
||||
value_getter=_make_sell_price_getter(),
|
||||
state_class="measurement",
|
||||
),
|
||||
ExposableEntity(
|
||||
key="energy.import_cost_total",
|
||||
component="sensor",
|
||||
device=device_info,
|
||||
device_class="monetary",
|
||||
unit=currency,
|
||||
name="Energy Import Cost (incl. standing)",
|
||||
value_getter=_make_import_cost_getter(),
|
||||
state_class="total",
|
||||
),
|
||||
ExposableEntity(
|
||||
key="energy.export_revenue_total",
|
||||
component="sensor",
|
||||
device=device_info,
|
||||
device_class="monetary",
|
||||
unit=currency,
|
||||
name="Energy Export Revenue (incl. tax credit)",
|
||||
value_getter=_make_export_revenue_getter(),
|
||||
state_class="total",
|
||||
),
|
||||
# Daily entities: reset at local midnight, state_class=total_increasing.
|
||||
#
|
||||
# device_class=monetary is correct here. The HA developer docs (long-term
|
||||
# statistics section) prohibit MONETARY from being combined with
|
||||
# state_class=measurement — they do NOT prohibit it with total or
|
||||
# total_increasing. monetary+total_increasing is valid (the cumulative
|
||||
# entities above use monetary+total with no issue).
|
||||
#
|
||||
# total_increasing is appropriate because these daily quantities are
|
||||
# non-negative (import/export metered cost ≥ 0, fixed standing fee/tax
|
||||
# credit ≥ 0) and monotonically non-decreasing within any given local day.
|
||||
# When the local midnight rolls over the value drops back to ~1 day's fixed
|
||||
# fee; HA interprets that decrease as a new cycle, giving correct daily
|
||||
# aggregation without any explicit last_reset pipeline.
|
||||
ExposableEntity(
|
||||
key="energy.import_cost_today",
|
||||
component="sensor",
|
||||
device=device_info,
|
||||
device_class="monetary",
|
||||
unit=currency,
|
||||
name="Energy Import Cost Today (incl. standing)",
|
||||
value_getter=_make_import_cost_today_getter(),
|
||||
state_class="total_increasing",
|
||||
),
|
||||
ExposableEntity(
|
||||
key="energy.export_revenue_today",
|
||||
component="sensor",
|
||||
device=device_info,
|
||||
device_class="monetary",
|
||||
unit=currency,
|
||||
name="Energy Export Revenue Today (incl. tax credit)",
|
||||
value_getter=_make_export_revenue_today_getter(),
|
||||
state_class="total_increasing",
|
||||
),
|
||||
]
|
||||
|
||||
return entities
|
||||
|
||||
|
||||
# Register the energy cost provider at module load time.
|
||||
register_provider(_energy_cost_provider)
|
||||
@@ -0,0 +1 @@
|
||||
"""Modbus integration package — driver, profile loader, and decoder."""
|
||||
@@ -0,0 +1,230 @@
|
||||
"""Modbus TCP driver — thin wrapper around pymodbus.
|
||||
|
||||
This module provides a single public function ``read_blocks`` that performs
|
||||
one or more block reads against a Modbus TCP gateway — using either FC04
|
||||
(Read Input Registers) or FC03 (Read Holding Registers), selected per call
|
||||
via the ``function_code`` argument — and returns a flat
|
||||
``dict[register_address -> 16-bit_value]`` map. The function code comes from
|
||||
the device profile (e.g. SDM120 uses FC04, DDSU666 uses FC03).
|
||||
|
||||
Design decisions
|
||||
----------------
|
||||
- **Only read function codes** (FC03/FC04) are exposed. There is no write path.
|
||||
- float32 decoding uses ``struct.unpack('>f', ...)`` directly rather than the
|
||||
pymodbus ``BinaryPayloadDecoder`` helper, which has had API churn across 3.x
|
||||
releases. Big-endian word order + big-endian byte order means the 4 raw bytes
|
||||
are already in standard network (big-endian) order.
|
||||
- ``ModbusTcpClient`` is used in synchronous mode (``connect()`` / ``close()``).
|
||||
The client is created fresh per call; this keeps the driver stateless and
|
||||
avoids threading concerns at the cost of one TCP handshake per read cycle.
|
||||
APScheduler jobs call this from a thread pool, so stateless is safer.
|
||||
- pymodbus 3.13.x uses ``device_id=`` as the slave-address keyword argument
|
||||
(renamed from ``slave=`` in earlier 3.x releases).
|
||||
|
||||
Exceptions
|
||||
----------
|
||||
``ModbusDriverError``
|
||||
Base exception for all driver-level errors.
|
||||
``ModbusConnectionError``
|
||||
Raised when the TCP connection to the gateway cannot be established or is
|
||||
lost during the read.
|
||||
``ModbusResponseError``
|
||||
Raised when the gateway responds with a Modbus exception frame or when the
|
||||
returned register count does not match the requested count.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import struct
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from pymodbus.client import ModbusTcpClient
|
||||
from pymodbus.exceptions import ConnectionException, ModbusException
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from collections.abc import Sequence
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Custom exceptions
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class ModbusDriverError(RuntimeError):
|
||||
"""Base class for all Modbus driver errors."""
|
||||
|
||||
|
||||
class ModbusConnectionError(ModbusDriverError):
|
||||
"""Cannot connect to (or lost connection with) the Modbus TCP gateway."""
|
||||
|
||||
|
||||
class ModbusResponseError(ModbusDriverError):
|
||||
"""Modbus gateway returned an exception frame or an unexpected response."""
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Block definition type alias
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
#: A single read block: ``{"start": int, "count": int}``.
|
||||
Block = dict[str, int]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Public API
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def read_blocks(
|
||||
host: str,
|
||||
port: int,
|
||||
unit_id: int,
|
||||
blocks: Sequence[Block],
|
||||
*,
|
||||
function_code: int = 4,
|
||||
timeout: float = 3.0,
|
||||
) -> dict[int, int]:
|
||||
"""Read one or more contiguous register blocks via FC03 or FC04.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
host:
|
||||
Hostname or IP address of the Modbus TCP gateway.
|
||||
port:
|
||||
TCP port (typically 502).
|
||||
unit_id:
|
||||
Modbus slave / unit address (the device's Meter ID for SDM120).
|
||||
blocks:
|
||||
Sequence of ``{"start": int, "count": int}`` dicts describing the
|
||||
contiguous register ranges to read. ``count`` is the number of
|
||||
16-bit registers (not bytes).
|
||||
function_code:
|
||||
Modbus read function code: ``4`` for FC04 (Read Input Registers,
|
||||
default — SDM120) or ``3`` for FC03 (Read Holding Registers —
|
||||
DDSU666 and other devices that expose measurements as holding
|
||||
registers). Comes from the device profile's ``function_code`` field.
|
||||
timeout:
|
||||
TCP connect/read timeout in seconds (default 3 s).
|
||||
|
||||
Returns
|
||||
-------
|
||||
dict[int, int]
|
||||
Flat mapping of ``{register_address: 16-bit_register_value}`` for
|
||||
every register returned across all blocks.
|
||||
|
||||
Raises
|
||||
------
|
||||
ModbusDriverError
|
||||
If ``function_code`` is neither 3 nor 4 (validated before any
|
||||
connection is attempted).
|
||||
ModbusConnectionError
|
||||
If the TCP connection to the gateway fails.
|
||||
ModbusResponseError
|
||||
If the gateway returns a Modbus exception frame or an unexpected
|
||||
number of registers.
|
||||
"""
|
||||
if function_code not in (3, 4):
|
||||
raise ModbusDriverError(
|
||||
f"Unsupported read function code FC{function_code:02d} "
|
||||
f"(only FC03 holding-register and FC04 input-register reads are supported)"
|
||||
)
|
||||
|
||||
client = ModbusTcpClient(host, port=port, timeout=timeout)
|
||||
try:
|
||||
connected = client.connect()
|
||||
if not connected:
|
||||
raise ModbusConnectionError(
|
||||
f"Could not connect to Modbus gateway at {host}:{port}"
|
||||
)
|
||||
|
||||
registers: dict[int, int] = {}
|
||||
for block in blocks:
|
||||
start: int = block["start"]
|
||||
count: int = block["count"]
|
||||
_read_block(client, unit_id, start, count, registers, function_code=function_code)
|
||||
|
||||
return registers
|
||||
|
||||
except ConnectionException as exc:
|
||||
raise ModbusConnectionError(
|
||||
f"Connection to Modbus gateway {host}:{port} failed: {exc}"
|
||||
) from exc
|
||||
except ModbusException as exc:
|
||||
raise ModbusResponseError(f"Modbus protocol error: {exc}") from exc
|
||||
finally:
|
||||
client.close()
|
||||
|
||||
|
||||
def _read_block(
|
||||
client: ModbusTcpClient,
|
||||
unit_id: int,
|
||||
start: int,
|
||||
count: int,
|
||||
result: dict[int, int],
|
||||
*,
|
||||
function_code: int,
|
||||
) -> None:
|
||||
"""Read one block and merge into *result*. Raises on any error.
|
||||
|
||||
``function_code`` is assumed already validated to be 3 or 4 by the caller
|
||||
(``read_blocks``); 3 dispatches FC03 (holding) and 4 dispatches FC04 (input).
|
||||
"""
|
||||
try:
|
||||
if function_code == 3:
|
||||
response = client.read_holding_registers(start, count=count, device_id=unit_id)
|
||||
else: # function_code == 4 (input registers)
|
||||
response = client.read_input_registers(start, count=count, device_id=unit_id)
|
||||
except ConnectionException as exc:
|
||||
raise ModbusConnectionError(
|
||||
f"Lost connection while reading registers 0x{start:04X}+{count}: {exc}"
|
||||
) from exc
|
||||
|
||||
if response.isError():
|
||||
raise ModbusResponseError(
|
||||
f"Modbus exception reading registers 0x{start:04X}+{count}: {response}"
|
||||
)
|
||||
|
||||
regs = response.registers
|
||||
if len(regs) != count:
|
||||
raise ModbusResponseError(
|
||||
f"Expected {count} registers from 0x{start:04X}, got {len(regs)}"
|
||||
)
|
||||
|
||||
for offset, value in enumerate(regs):
|
||||
result[start + offset] = value
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Decoding helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def registers_to_float(hi_reg: int, lo_reg: int) -> float:
|
||||
"""Decode two 16-bit registers to a big-endian IEEE-754 float32.
|
||||
|
||||
The SDM120 (and most Modbus float devices) use big-endian word order:
|
||||
the high 16-bit register comes first, then the low register. Combined
|
||||
with big-endian byte order within each register, the raw 4-byte sequence
|
||||
is standard network-byte-order (big-endian) float32.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
hi_reg:
|
||||
The first (higher-address-range, most-significant) 16-bit register.
|
||||
lo_reg:
|
||||
The second (lower-address-range, least-significant) 16-bit register.
|
||||
|
||||
Returns
|
||||
-------
|
||||
float
|
||||
The decoded IEEE-754 single-precision floating-point value.
|
||||
|
||||
Example
|
||||
-------
|
||||
>>> registers_to_float(0x4366, 0x3334)
|
||||
230.20001220703125
|
||||
"""
|
||||
raw = struct.pack(">HH", hi_reg, lo_reg)
|
||||
return struct.unpack(">f", raw)[0]
|
||||
@@ -0,0 +1,231 @@
|
||||
"""Modbus profile loader, validator, and decoder.
|
||||
|
||||
A *profile* is a YAML file that describes the protocol-level knowledge for
|
||||
a particular Modbus device model:
|
||||
|
||||
- Which registers to read (``blocks`` for efficient bulk reads).
|
||||
- How to decode each quantity (``metrics``: register address, type, unit, …).
|
||||
- Home Assistant metadata per metric (``device_class``, ``ha_component``, …).
|
||||
|
||||
Profiles live in ``app/integrations/modbus/profiles/<name>.yaml`` and are
|
||||
located at runtime relative to *this file* (not CWD), so they work whether
|
||||
the package is installed as a wheel, run in-process, or invoked via
|
||||
``python -m scripts.modbus_cli``.
|
||||
|
||||
Design notes
|
||||
------------
|
||||
- **Purely data + functions** — no abstract base classes or inheritance.
|
||||
- ``ModbusProfile`` and ``MetricSpec`` are Pydantic models: they validate on
|
||||
construction and raise ``pydantic.ValidationError`` for malformed YAML.
|
||||
- ``decode`` uses ``driver.registers_to_float`` for float32 quantities.
|
||||
Unsupported types are silently skipped with a warning (future-proofing for
|
||||
int16/uint16 etc.).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
import yaml
|
||||
from pydantic import BaseModel, ValidationError
|
||||
|
||||
from app.integrations.modbus.driver import registers_to_float
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Directory containing the YAML profiles, located relative to *this* file.
|
||||
_PROFILES_DIR = Path(__file__).parent / "profiles"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Pydantic models for profile validation
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class MetricSpec(BaseModel):
|
||||
"""Specification for a single measurable quantity within a profile."""
|
||||
|
||||
key: str
|
||||
"""Stable identifier used as the key in decoded payloads (e.g. ``"voltage"``)."""
|
||||
|
||||
address: int
|
||||
"""Starting Modbus register address (0-based, e.g. ``0x0000`` for voltage)."""
|
||||
|
||||
type: str
|
||||
"""Encoding type — currently only ``"float32"`` is supported."""
|
||||
|
||||
unit: str
|
||||
"""Physical unit string (e.g. ``"V"``, ``"A"``, ``"kWh"``). Empty string for dimensionless."""
|
||||
|
||||
device_class: str
|
||||
"""Home Assistant device class (e.g. ``"voltage"``, ``"power"``, ``"energy"``)."""
|
||||
|
||||
ha_component: str
|
||||
"""Home Assistant component type (e.g. ``"sensor"``)."""
|
||||
|
||||
state_class: Optional[str] = None
|
||||
"""Home Assistant state class, if applicable (e.g. ``"total_increasing"``)."""
|
||||
|
||||
|
||||
class BlockSpec(BaseModel):
|
||||
"""A contiguous register block for bulk reading."""
|
||||
|
||||
start: int
|
||||
"""First register address in the block."""
|
||||
|
||||
count: int
|
||||
"""Number of 16-bit registers to read."""
|
||||
|
||||
|
||||
class ModbusProfile(BaseModel):
|
||||
"""Complete description of a Modbus device's protocol-level knowledge."""
|
||||
|
||||
name: str
|
||||
"""Short identifier matching the YAML file name (e.g. ``"sdm120"``)."""
|
||||
|
||||
description: str
|
||||
"""Human-readable description of the device."""
|
||||
|
||||
function_code: int
|
||||
"""Modbus function code for reading measurements (4 = FC04 input registers)."""
|
||||
|
||||
word_order: str
|
||||
"""Word (register) order: ``"big"`` means high register first."""
|
||||
|
||||
byte_order: str
|
||||
"""Byte order within each register: ``"big"`` means standard network order."""
|
||||
|
||||
blocks: list[BlockSpec]
|
||||
"""Register blocks to read in bulk, reducing Modbus transaction count."""
|
||||
|
||||
metrics: list[MetricSpec]
|
||||
"""List of individual measurable quantities and their decoding rules."""
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Public API
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class ProfileNotFoundError(FileNotFoundError):
|
||||
"""Raised when the requested profile YAML file does not exist."""
|
||||
|
||||
|
||||
class ProfileValidationError(ValueError):
|
||||
"""Raised when a profile YAML file fails Pydantic validation."""
|
||||
|
||||
|
||||
def load_profile(name: str) -> ModbusProfile:
|
||||
"""Load and validate a Modbus device profile by name.
|
||||
|
||||
The profile file is expected at
|
||||
``app/integrations/modbus/profiles/<name>.yaml``.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
name:
|
||||
Profile name without extension (e.g. ``"sdm120"``).
|
||||
|
||||
Returns
|
||||
-------
|
||||
ModbusProfile
|
||||
Validated profile model.
|
||||
|
||||
Raises
|
||||
------
|
||||
ProfileNotFoundError
|
||||
If ``profiles/<name>.yaml`` does not exist.
|
||||
ProfileValidationError
|
||||
If the YAML is syntactically valid but the content fails schema
|
||||
validation (missing required fields, wrong types, etc.).
|
||||
"""
|
||||
path = _PROFILES_DIR / f"{name}.yaml"
|
||||
if not path.exists():
|
||||
raise ProfileNotFoundError(
|
||||
f"Modbus profile '{name}' not found (looked for {path})"
|
||||
)
|
||||
|
||||
with path.open("r", encoding="utf-8") as fh:
|
||||
raw = yaml.safe_load(fh)
|
||||
|
||||
try:
|
||||
profile = ModbusProfile.model_validate(raw)
|
||||
except ValidationError as exc:
|
||||
raise ProfileValidationError(
|
||||
f"Profile '{name}' failed validation: {exc}"
|
||||
) from exc
|
||||
|
||||
return profile
|
||||
|
||||
|
||||
def list_profiles() -> list[tuple[str, str]]:
|
||||
"""Return ``(name, description)`` pairs for all available profiles.
|
||||
|
||||
Scans ``app/integrations/modbus/profiles/*.yaml``, loads each, and
|
||||
returns a sorted list of ``(name, description)`` tuples. Profiles that
|
||||
fail to load are skipped with a warning.
|
||||
"""
|
||||
results: list[tuple[str, str]] = []
|
||||
if not _PROFILES_DIR.exists():
|
||||
return results
|
||||
|
||||
for path in sorted(_PROFILES_DIR.glob("*.yaml")):
|
||||
name = path.stem
|
||||
try:
|
||||
profile = load_profile(name)
|
||||
results.append((profile.name, profile.description))
|
||||
except (ProfileNotFoundError, ProfileValidationError, Exception) as exc:
|
||||
logger.warning("Skipping malformed profile '%s': %s", name, exc)
|
||||
|
||||
return results
|
||||
|
||||
|
||||
def decode(profile: ModbusProfile, registers: dict[int, int]) -> dict[str, float]:
|
||||
"""Decode raw register values into a mapping of ``{key: float_value}``.
|
||||
|
||||
For each metric in *profile*, the function looks up the two registers at
|
||||
``address`` and ``address + 1`` in *registers* and decodes them as a
|
||||
big-endian float32 (high register first).
|
||||
|
||||
Metrics whose registers are absent from *registers* are skipped (not
|
||||
an error — this can happen if a block was not requested or a device does
|
||||
not populate that register). Metrics with unsupported ``type`` values
|
||||
are also skipped.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
profile:
|
||||
Validated ``ModbusProfile`` describing the device.
|
||||
registers:
|
||||
Mapping of ``{register_address: 16-bit_value}`` as returned by
|
||||
``driver.read_blocks``.
|
||||
|
||||
Returns
|
||||
-------
|
||||
dict[str, float]
|
||||
Decoded engineering values keyed by each metric's ``key`` field.
|
||||
"""
|
||||
result: dict[str, float] = {}
|
||||
for metric in profile.metrics:
|
||||
if metric.type == "float32":
|
||||
hi_addr = metric.address
|
||||
lo_addr = metric.address + 1
|
||||
if hi_addr not in registers or lo_addr not in registers:
|
||||
logger.debug(
|
||||
"Skipping metric '%s': registers 0x%04X/0x%04X not available",
|
||||
metric.key,
|
||||
hi_addr,
|
||||
lo_addr,
|
||||
)
|
||||
continue
|
||||
value = registers_to_float(registers[hi_addr], registers[lo_addr])
|
||||
result[metric.key] = value
|
||||
else:
|
||||
logger.warning(
|
||||
"Metric '%s' has unsupported type '%s', skipping",
|
||||
metric.key,
|
||||
metric.type,
|
||||
)
|
||||
return result
|
||||
@@ -0,0 +1,82 @@
|
||||
name: ddsu666
|
||||
description: CHINT DDSU666 single-phase smart meter
|
||||
function_code: 3 # holding registers (FC03) — DDSU666 has NO input registers (no FC04)
|
||||
word_order: big # high register first — ASSUMED; verify with a known voltage reading
|
||||
byte_order: big # high byte first within each register (confirmed by manual CRC example)
|
||||
|
||||
# NOTE: the manual gives no worked float-decode example, so word_order is a best-guess
|
||||
# (standard big-endian, high register first, matching sdm120). After wiring the meter,
|
||||
# read 0x2000 (voltage) — it should decode to ~230 V. If it decodes to garbage, the
|
||||
# device uses the opposite word order and this profile (and the decoder) need adjusting.
|
||||
# Byte order is confirmed big-endian from the manual (Appendix A, Table A.4: 0x1388 -> 13 88).
|
||||
|
||||
blocks:
|
||||
# Instantaneous quantities 0x2000–0x200F: voltage, current, P, Q, (rsv), PF, (rsv), Freq.
|
||||
# 16 contiguous registers — single bulk read. (DDSU666 manual Table 9.)
|
||||
- { start: 0x2000, count: 0x0010 }
|
||||
# Active energy — import (0x4000) and export (0x400A) read as two small blocks rather
|
||||
# than one span, to avoid touching the undocumented/reserved 0x4002–0x4009 gap.
|
||||
- { start: 0x4000, count: 0x0002 }
|
||||
- { start: 0x400A, count: 0x0002 }
|
||||
|
||||
metrics:
|
||||
# Addresses are the raw Modbus protocol addresses (hex) from DDSU666 manual Table 9,
|
||||
# read via FC03. Each float32 occupies two consecutive 16-bit registers.
|
||||
|
||||
- key: voltage
|
||||
address: 0x2000 # U — Voltage (V)
|
||||
type: float32
|
||||
unit: "V"
|
||||
device_class: voltage
|
||||
ha_component: sensor
|
||||
|
||||
- key: current
|
||||
address: 0x2002 # I — Current (A)
|
||||
type: float32
|
||||
unit: "A"
|
||||
device_class: current
|
||||
ha_component: sensor
|
||||
|
||||
- key: active_power
|
||||
address: 0x2004 # P — Active power. Manual unit is kW (NOT W like sdm120).
|
||||
type: float32
|
||||
unit: "kW"
|
||||
device_class: power
|
||||
ha_component: sensor
|
||||
|
||||
- key: reactive_power
|
||||
address: 0x2006 # Q — Reactive power (kvar)
|
||||
type: float32
|
||||
unit: "kvar"
|
||||
device_class: reactive_power
|
||||
ha_component: sensor
|
||||
|
||||
- key: power_factor
|
||||
address: 0x200A # PF — Power factor (dimensionless)
|
||||
type: float32
|
||||
unit: ""
|
||||
device_class: power_factor
|
||||
ha_component: sensor
|
||||
|
||||
- key: frequency
|
||||
address: 0x200E # Freq — Frequency (Hz)
|
||||
type: float32
|
||||
unit: "Hz"
|
||||
device_class: frequency
|
||||
ha_component: sensor
|
||||
|
||||
- key: import_energy
|
||||
address: 0x4000 # Ep — positive/forward active energy (kWh)
|
||||
type: float32
|
||||
unit: "kWh"
|
||||
device_class: energy
|
||||
state_class: total_increasing
|
||||
ha_component: sensor
|
||||
|
||||
- key: export_energy
|
||||
address: 0x400A # -Ep — reverse active energy (kWh)
|
||||
type: float32
|
||||
unit: "kWh"
|
||||
device_class: energy
|
||||
state_class: total_increasing
|
||||
ha_component: sensor
|
||||
@@ -0,0 +1,74 @@
|
||||
name: sdm120
|
||||
description: Eastron SDM120 single-phase energy meter
|
||||
function_code: 4 # input registers (FC04)
|
||||
word_order: big # high register first (big-endian word order)
|
||||
byte_order: big # high byte first within each register
|
||||
|
||||
blocks:
|
||||
# Registers 30001–30095 (0x0000–0x005F): voltage through max export demand
|
||||
- { start: 0x0000, count: 0x0060 }
|
||||
# Registers 30343–30346 (0x0156–0x0159): total active/reactive energy
|
||||
- { start: 0x0156, count: 0x0004 }
|
||||
|
||||
metrics:
|
||||
# Core measurements — addresses from SDM120 Modbus Protocol §4 (FC04 input registers)
|
||||
# Each float32 occupies two consecutive 16-bit registers; address = Hex start in §4.
|
||||
|
||||
- key: voltage
|
||||
address: 0x0000 # register 30001 — Voltage (V)
|
||||
type: float32
|
||||
unit: "V"
|
||||
device_class: voltage
|
||||
ha_component: sensor
|
||||
|
||||
- key: current
|
||||
address: 0x0006 # register 30007 — Current (A)
|
||||
type: float32
|
||||
unit: "A"
|
||||
device_class: current
|
||||
ha_component: sensor
|
||||
|
||||
- key: active_power
|
||||
address: 0x000C # register 30013 — Active power (W)
|
||||
type: float32
|
||||
unit: "W"
|
||||
device_class: power
|
||||
ha_component: sensor
|
||||
|
||||
- key: power_factor
|
||||
address: 0x001E # register 30031 — Power factor (dimensionless)
|
||||
type: float32
|
||||
unit: ""
|
||||
device_class: power_factor
|
||||
ha_component: sensor
|
||||
|
||||
- key: frequency
|
||||
address: 0x0046 # register 30071 — Frequency (Hz)
|
||||
type: float32
|
||||
unit: "Hz"
|
||||
device_class: frequency
|
||||
ha_component: sensor
|
||||
|
||||
- key: import_energy
|
||||
address: 0x0048 # register 30073 — Import active energy (kWh)
|
||||
type: float32
|
||||
unit: "kWh"
|
||||
device_class: energy
|
||||
state_class: total_increasing
|
||||
ha_component: sensor
|
||||
|
||||
- key: export_energy
|
||||
address: 0x004A # register 30075 — Export active energy (kWh)
|
||||
type: float32
|
||||
unit: "kWh"
|
||||
device_class: energy
|
||||
state_class: total_increasing
|
||||
ha_component: sensor
|
||||
|
||||
- key: total_energy
|
||||
address: 0x0156 # register 30343 — Total active energy (kWh)
|
||||
type: float32
|
||||
unit: "kWh"
|
||||
device_class: energy
|
||||
state_class: total_increasing
|
||||
ha_component: sensor
|
||||
@@ -0,0 +1,341 @@
|
||||
"""MQTT client integration (paho-mqtt 2.x).
|
||||
|
||||
Provides :class:`MqttManager`: a long-lived MQTT client that runs paho's
|
||||
loop in a background thread. Designed to be started in FastAPI's lifespan
|
||||
and shared across the application as a module-level singleton (``mqtt_manager``).
|
||||
|
||||
Key design decisions
|
||||
--------------------
|
||||
- **paho 2.x API**: ``Client`` requires ``CallbackAPIVersion.VERSION2`` as the
|
||||
first positional argument. VERSION2 on_connect callback receives
|
||||
``(client, userdata, connect_flags, reason_code, properties)`` — not the
|
||||
older RC int.
|
||||
- **Background thread**: ``loop_start()`` spawns a daemon thread; ``loop_stop()``
|
||||
joins it cleanly on shutdown.
|
||||
- **Never crash the process**: all paho operations are wrapped in try/except;
|
||||
connection failure is logged but does not propagate to the caller.
|
||||
- **Password safety**: the MQTT password is never passed to ``logger`` calls.
|
||||
- **Reconnect**: ``reconnect(settings)`` tears down the old client and
|
||||
establishes a fresh connection with the new settings. Callers (e.g. PUT
|
||||
/api/config) call this after saving MQTT-related config values.
|
||||
- **Subscribe**: ``subscribe(topic, handler)`` registers a topic → handler
|
||||
mapping. Subscriptions are re-established automatically on reconnect.
|
||||
The ``on_message`` callback dispatches incoming payloads to the registered
|
||||
handler; handler exceptions are caught and logged so they never crash the
|
||||
paho loop thread or the network connection.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import threading
|
||||
from collections.abc import Callable
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
import paho.mqtt.client as mqtt
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from app.config import Settings
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# MQTT settings keys that, when changed, should trigger a reconnect.
|
||||
MQTT_SETTINGS_KEYS = {
|
||||
"mqtt_enabled",
|
||||
"mqtt_broker_host",
|
||||
"mqtt_broker_port",
|
||||
"mqtt_username",
|
||||
"mqtt_password",
|
||||
"mqtt_tls_enabled",
|
||||
}
|
||||
|
||||
|
||||
def _is_configured(settings: Settings) -> bool:
|
||||
"""Return True if MQTT is enabled *and* the broker host is set."""
|
||||
return bool(settings.mqtt_enabled and settings.mqtt_broker_host)
|
||||
|
||||
|
||||
class MqttManager:
|
||||
"""Long-lived MQTT client wrapper.
|
||||
|
||||
Lifecycle
|
||||
---------
|
||||
1. ``connect(settings)`` — start background loop + connect to broker.
|
||||
2. ``publish(...)`` — publish messages while connected.
|
||||
3. ``disconnect()`` — graceful teardown (called on app shutdown).
|
||||
4. ``reconnect(settings)`` — disconnect then re-connect with new settings
|
||||
(called after saving updated MQTT configuration).
|
||||
|
||||
When MQTT is not enabled or not configured, all methods are no-ops.
|
||||
Connection failures are caught and logged; they do not raise.
|
||||
"""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self._client: mqtt.Client | None = None
|
||||
self._lock = threading.Lock()
|
||||
self._connected = False
|
||||
# topic → handler registry; persists across reconnects so subscriptions
|
||||
# are automatically re-established when the client reconnects.
|
||||
self._subscriptions: dict[str, Callable[[bytes], None]] = {}
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Public properties
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
@property
|
||||
def is_connected(self) -> bool:
|
||||
"""True if the underlying paho client is currently connected."""
|
||||
return self._connected
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Public interface
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def is_configured(self, settings: Settings) -> bool:
|
||||
"""Return True if MQTT is enabled and broker host is configured."""
|
||||
return _is_configured(settings)
|
||||
|
||||
def connect(self, settings: Settings) -> None:
|
||||
"""Connect to the MQTT broker and start the background loop thread.
|
||||
|
||||
No-op if MQTT is not enabled or broker host is not set.
|
||||
Connection errors are logged but do not raise.
|
||||
"""
|
||||
if not _is_configured(settings):
|
||||
logger.debug("MQTT not configured or not enabled — skipping connect.")
|
||||
return
|
||||
|
||||
with self._lock:
|
||||
self._start_client(settings)
|
||||
|
||||
def disconnect(self) -> None:
|
||||
"""Disconnect from the broker and stop the background loop thread.
|
||||
|
||||
No-op if no client is active.
|
||||
"""
|
||||
with self._lock:
|
||||
self._stop_client()
|
||||
|
||||
def reconnect(self, settings: Settings) -> None:
|
||||
"""Disconnect the current client (if any) and reconnect with *settings*.
|
||||
|
||||
Call this after saving updated MQTT configuration values.
|
||||
No-op if MQTT is not enabled or broker host is not set.
|
||||
"""
|
||||
with self._lock:
|
||||
self._stop_client()
|
||||
self.connect(settings)
|
||||
|
||||
def publish(
|
||||
self,
|
||||
topic: str,
|
||||
payload: str | bytes | None,
|
||||
*,
|
||||
retain: bool = False,
|
||||
qos: int = 0,
|
||||
) -> None:
|
||||
"""Publish a message to *topic*.
|
||||
|
||||
If the client is not connected the call is silently skipped.
|
||||
Errors are logged but do not raise.
|
||||
"""
|
||||
with self._lock:
|
||||
client = self._client
|
||||
|
||||
if client is None or not self._connected:
|
||||
logger.debug("MQTT publish skipped — not connected (topic=%s).", topic)
|
||||
return
|
||||
|
||||
try:
|
||||
client.publish(topic, payload=payload, qos=qos, retain=retain)
|
||||
except Exception:
|
||||
logger.exception("MQTT publish error (topic=%s).", topic)
|
||||
|
||||
def subscribe(self, topic: str, handler: Callable[[bytes], None]) -> None:
|
||||
"""Register *handler* to be called when a message arrives on *topic*.
|
||||
|
||||
If the client is already connected, the subscription is sent to the
|
||||
broker immediately. Otherwise it is queued and will be established
|
||||
the next time ``_on_connect`` fires (including after a reconnect).
|
||||
|
||||
Handler exceptions are swallowed by the ``on_message`` dispatcher so
|
||||
that a buggy handler can never crash the paho loop thread.
|
||||
"""
|
||||
self._subscriptions[topic] = handler
|
||||
# Grab the client reference outside the lock to avoid holding the lock
|
||||
# while calling back into paho (which may itself acquire internal locks).
|
||||
with self._lock:
|
||||
client = self._client
|
||||
connected = self._connected
|
||||
|
||||
if client is not None and connected:
|
||||
try:
|
||||
client.subscribe(topic)
|
||||
logger.debug("MQTT subscribed to topic=%s (immediate).", topic)
|
||||
except Exception:
|
||||
logger.exception("MQTT subscribe error (topic=%s).", topic)
|
||||
|
||||
def unsubscribe(self, topic: str) -> None:
|
||||
"""Remove the handler for *topic* and unsubscribe from the broker.
|
||||
|
||||
Idempotent: unknown topics are ignored. Used to apply config changes
|
||||
without a restart (e.g. when DSMR ingest is turned off or its topic
|
||||
changes). If the client is connected, the broker unsubscribe is sent
|
||||
immediately; either way the handler is removed from the registry so it
|
||||
will not be re-subscribed on the next ``_on_connect``.
|
||||
"""
|
||||
self._subscriptions.pop(topic, None)
|
||||
with self._lock:
|
||||
client = self._client
|
||||
connected = self._connected
|
||||
|
||||
if client is not None and connected:
|
||||
try:
|
||||
client.unsubscribe(topic)
|
||||
logger.debug("MQTT unsubscribed from topic=%s.", topic)
|
||||
except Exception:
|
||||
logger.exception("MQTT unsubscribe error (topic=%s).", topic)
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Internal helpers — must be called with self._lock held
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _start_client(self, settings: Settings) -> None:
|
||||
"""Build a fresh paho Client, configure it, and call loop_start + connect."""
|
||||
client = mqtt.Client(
|
||||
callback_api_version=mqtt.CallbackAPIVersion.VERSION2,
|
||||
client_id="home-automation",
|
||||
)
|
||||
|
||||
# Callbacks — VERSION2 on_connect signature:
|
||||
# (client, userdata, connect_flags, reason_code, properties)
|
||||
def _on_connect(
|
||||
_client: mqtt.Client,
|
||||
_userdata: object,
|
||||
_flags: mqtt.ConnectFlags,
|
||||
reason_code: mqtt.ReasonCode,
|
||||
_properties: mqtt.Properties | None,
|
||||
) -> None:
|
||||
if reason_code.is_failure:
|
||||
logger.warning("MQTT connection refused: %s", reason_code)
|
||||
self._connected = False
|
||||
else:
|
||||
logger.info("MQTT connected (reason_code=%s).", reason_code)
|
||||
self._connected = True
|
||||
# Re-establish all registered subscriptions after (re-)connect.
|
||||
for sub_topic in self._subscriptions:
|
||||
try:
|
||||
_client.subscribe(sub_topic)
|
||||
logger.debug("MQTT re-subscribed to topic=%s.", sub_topic)
|
||||
except Exception:
|
||||
logger.exception(
|
||||
"MQTT re-subscribe failed (topic=%s).", sub_topic
|
||||
)
|
||||
|
||||
# VERSION2 on_disconnect signature:
|
||||
# (client, userdata, disconnect_flags, reason_code, properties)
|
||||
def _on_disconnect(
|
||||
_client: mqtt.Client,
|
||||
_userdata: object,
|
||||
_disconnect_flags: mqtt.DisconnectFlags,
|
||||
reason_code: mqtt.ReasonCode,
|
||||
_properties: mqtt.Properties | None,
|
||||
) -> None:
|
||||
self._connected = False
|
||||
if reason_code.is_failure:
|
||||
logger.warning("MQTT disconnected unexpectedly (reason_code=%s).", reason_code)
|
||||
else:
|
||||
logger.info("MQTT disconnected cleanly.")
|
||||
|
||||
# VERSION2 on_message signature: (client, userdata, message)
|
||||
def _on_message(
|
||||
_client: mqtt.Client,
|
||||
_userdata: object,
|
||||
message: mqtt.MQTTMessage,
|
||||
) -> None:
|
||||
handler = self._subscriptions.get(message.topic)
|
||||
if handler is None:
|
||||
logger.debug(
|
||||
"MQTT on_message: no handler for topic=%s.", message.topic
|
||||
)
|
||||
return
|
||||
try:
|
||||
handler(message.payload)
|
||||
except Exception:
|
||||
logger.exception(
|
||||
"MQTT message handler raised for topic=%s (swallowed).",
|
||||
message.topic,
|
||||
)
|
||||
|
||||
client.on_connect = _on_connect
|
||||
client.on_disconnect = _on_disconnect
|
||||
client.on_message = _on_message
|
||||
|
||||
# TLS
|
||||
if settings.mqtt_tls_enabled:
|
||||
try:
|
||||
client.tls_set()
|
||||
except Exception:
|
||||
logger.exception("MQTT TLS setup failed.")
|
||||
return
|
||||
|
||||
# Credentials — never log the password
|
||||
if settings.mqtt_username:
|
||||
client.username_pw_set(
|
||||
username=settings.mqtt_username,
|
||||
password=settings.mqtt_password or None,
|
||||
)
|
||||
|
||||
# Start background loop thread *before* connect so paho can handle
|
||||
# the connection handshake asynchronously.
|
||||
client.loop_start()
|
||||
|
||||
try:
|
||||
client.connect(
|
||||
host=settings.mqtt_broker_host,
|
||||
port=settings.mqtt_broker_port,
|
||||
keepalive=60,
|
||||
)
|
||||
except Exception:
|
||||
# Log without leaking the password
|
||||
logger.exception(
|
||||
"MQTT connect failed (host=%s, port=%s). Stopping loop.",
|
||||
settings.mqtt_broker_host,
|
||||
settings.mqtt_broker_port,
|
||||
)
|
||||
try:
|
||||
client.loop_stop()
|
||||
except Exception:
|
||||
pass
|
||||
return
|
||||
|
||||
self._client = client
|
||||
logger.info(
|
||||
"MQTT client started (host=%s, port=%s).",
|
||||
settings.mqtt_broker_host,
|
||||
settings.mqtt_broker_port,
|
||||
)
|
||||
|
||||
def _stop_client(self) -> None:
|
||||
"""Disconnect and stop the background loop. Idempotent."""
|
||||
client = self._client
|
||||
if client is None:
|
||||
return
|
||||
self._client = None
|
||||
self._connected = False
|
||||
try:
|
||||
client.disconnect()
|
||||
except Exception:
|
||||
logger.debug("MQTT disconnect raised (ignoring).", exc_info=True)
|
||||
try:
|
||||
client.loop_stop()
|
||||
except Exception:
|
||||
logger.debug("MQTT loop_stop raised (ignoring).", exc_info=True)
|
||||
logger.info("MQTT client stopped.")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Module-level singleton — shared across lifespan and route handlers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
mqtt_manager = MqttManager()
|
||||
@@ -0,0 +1,11 @@
|
||||
"""Pricing profile framework and strategy registry.
|
||||
|
||||
This package provides:
|
||||
|
||||
- ``profiles``: Pydantic models describing the structure of pricing profiles
|
||||
(``ManualProfile`` / ``TibberProfile``), YAML loaders, and contract-values
|
||||
validation (``load_profile`` / ``list_profiles`` / ``validate_values``).
|
||||
- ``strategies``: A lightweight registry of price-calculation strategies
|
||||
(``register_strategy`` / ``get_strategy``), with built-in implementations for
|
||||
the ``manual`` (fixed dual-tariff) and ``tibber`` (dynamic API) kinds.
|
||||
"""
|
||||
@@ -0,0 +1,393 @@
|
||||
"""Pricing profile loader, validator, and contract-values checker.
|
||||
|
||||
A *pricing profile* is a YAML file that describes the **structure** of an energy
|
||||
contract — which fields exist, their units, and which have defaults. Actual
|
||||
pricing values (the numbers the user fills in via the UI) are stored in the DB
|
||||
as ``EnergyContractVersion.values`` (a JSON blob) and must conform to the
|
||||
corresponding profile's structure.
|
||||
|
||||
Profiles live in ``app/integrations/pricing/profiles/<kind>.yaml`` and are
|
||||
located at runtime relative to *this file* (not CWD), matching the pattern
|
||||
established by the Modbus profile loader.
|
||||
|
||||
Design notes
|
||||
------------
|
||||
- **Purely data + functions** — no abstract base classes or inheritance.
|
||||
- Two Pydantic models — ``ManualProfile`` and ``TibberProfile`` — capture the
|
||||
different structures of the two supported kinds. A thin union dispatcher in
|
||||
``load_profile`` picks the right one.
|
||||
- ``validate_values(kind, values)`` fills in fields that carry a ``default`` and
|
||||
raises ``ProfileValidationError`` for missing required fields or wrong types.
|
||||
- All errors are subclasses of built-ins so callers need not import this module
|
||||
just to catch them.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from pathlib import Path
|
||||
from typing import Any, Optional
|
||||
|
||||
import yaml
|
||||
from pydantic import BaseModel, ValidationError, model_validator
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Directory containing the YAML profiles, located relative to *this* file.
|
||||
_PROFILES_DIR = Path(__file__).parent / "profiles"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Custom exceptions
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class ProfileNotFoundError(FileNotFoundError):
|
||||
"""Raised when the requested profile YAML file does not exist."""
|
||||
|
||||
|
||||
class ProfileValidationError(ValueError):
|
||||
"""Raised when a profile YAML fails Pydantic validation or when contract
|
||||
values do not conform to the profile structure."""
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Leaf-node models (a single pricing field: unit + optional default)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class FieldSpec(BaseModel):
|
||||
"""Specification for a single numeric pricing field."""
|
||||
|
||||
unit: str
|
||||
"""Physical / monetary unit string (e.g. ``"EUR/kWh"``, ``"EUR/month"``)."""
|
||||
|
||||
default: Optional[float] = None
|
||||
"""Default value used when the field is absent from contract values.
|
||||
``None`` means the field is required (no default)."""
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# ManualProfile — fixed / variable dual-tariff contract structure
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class ManualBuySpec(BaseModel):
|
||||
normal: FieldSpec # high-tariff buy price (delivered_2 registers)
|
||||
dal: FieldSpec # low-tariff buy price (delivered_1 registers)
|
||||
|
||||
|
||||
class ManualSellSpec(BaseModel):
|
||||
normal: FieldSpec # high-tariff sell / return price
|
||||
dal: FieldSpec # low-tariff sell / return price
|
||||
|
||||
|
||||
class ManualEnergySpec(BaseModel):
|
||||
dual_tariff: bool # always True for manual profiles
|
||||
buy: ManualBuySpec
|
||||
sell: ManualSellSpec
|
||||
energy_tax: FieldSpec # added to buy price; includes VAT
|
||||
ode: FieldSpec # currently merged into energy_tax; default 0
|
||||
|
||||
|
||||
class ManualStandingSpec(BaseModel):
|
||||
network_fee: FieldSpec # EUR/month
|
||||
management_fee: FieldSpec # EUR/month
|
||||
|
||||
|
||||
class ManualCreditsSpec(BaseModel):
|
||||
heffingskorting: FieldSpec # EUR/year — energy-tax credit deducted at summary
|
||||
|
||||
|
||||
class ManualProfile(BaseModel):
|
||||
"""Complete structure description for a ``manual`` pricing contract."""
|
||||
|
||||
kind: str
|
||||
label: str
|
||||
energy: ManualEnergySpec
|
||||
standing: ManualStandingSpec
|
||||
credits: ManualCreditsSpec
|
||||
|
||||
@model_validator(mode="after")
|
||||
def _check_kind(self) -> "ManualProfile":
|
||||
if self.kind != "manual":
|
||||
raise ValueError(f"ManualProfile requires kind='manual', got {self.kind!r}")
|
||||
return self
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# TibberProfile — dynamic Tibber API contract structure
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TibberEnergySpec(BaseModel):
|
||||
source: str # must be "tibber_api"
|
||||
energy_tax: FieldSpec # subtracted from total to derive sell price
|
||||
sell_adjust: FieldSpec # additional sell-price adjustment; default 0
|
||||
|
||||
|
||||
class TibberStandingSpec(BaseModel):
|
||||
management_fee: FieldSpec # EUR/month; has a default
|
||||
network_fee: FieldSpec # EUR/month
|
||||
|
||||
|
||||
class TibberCreditsSpec(BaseModel):
|
||||
heffingskorting: FieldSpec # EUR/year
|
||||
|
||||
|
||||
class TibberProfile(BaseModel):
|
||||
"""Complete structure description for a ``tibber`` pricing contract."""
|
||||
|
||||
kind: str
|
||||
label: str
|
||||
energy: TibberEnergySpec
|
||||
standing: TibberStandingSpec
|
||||
credits: TibberCreditsSpec
|
||||
|
||||
@model_validator(mode="after")
|
||||
def _check_kind(self) -> "TibberProfile":
|
||||
if self.kind != "tibber":
|
||||
raise ValueError(f"TibberProfile requires kind='tibber', got {self.kind!r}")
|
||||
return self
|
||||
|
||||
|
||||
# A union type for type hints where either profile is acceptable.
|
||||
AnyProfile = ManualProfile | TibberProfile
|
||||
|
||||
# Map kind → Pydantic model class used for validation.
|
||||
_PROFILE_MODELS: dict[str, type[ManualProfile] | type[TibberProfile]] = {
|
||||
"manual": ManualProfile,
|
||||
"tibber": TibberProfile,
|
||||
}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Public API
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def load_profile(kind: str) -> AnyProfile:
|
||||
"""Load and validate a pricing profile by kind name.
|
||||
|
||||
The profile file is expected at
|
||||
``app/integrations/pricing/profiles/<kind>.yaml``.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
kind:
|
||||
Profile kind without extension (``"manual"`` or ``"tibber"``).
|
||||
|
||||
Returns
|
||||
-------
|
||||
ManualProfile | TibberProfile
|
||||
Validated profile model.
|
||||
|
||||
Raises
|
||||
------
|
||||
ProfileNotFoundError
|
||||
If ``profiles/<kind>.yaml`` does not exist.
|
||||
ProfileValidationError
|
||||
If the YAML is syntactically valid but fails schema validation.
|
||||
"""
|
||||
path = _PROFILES_DIR / f"{kind}.yaml"
|
||||
if not path.exists():
|
||||
raise ProfileNotFoundError(
|
||||
f"Pricing profile '{kind}' not found (looked for {path})"
|
||||
)
|
||||
|
||||
with path.open("r", encoding="utf-8") as fh:
|
||||
raw = yaml.safe_load(fh)
|
||||
|
||||
if not isinstance(raw, dict):
|
||||
raise ProfileValidationError(
|
||||
f"Profile '{kind}': expected a YAML mapping, got {type(raw).__name__}"
|
||||
)
|
||||
|
||||
# Choose the right Pydantic model based on the ``kind`` field in the YAML.
|
||||
yaml_kind = raw.get("kind", kind)
|
||||
model_cls = _PROFILE_MODELS.get(yaml_kind)
|
||||
if model_cls is None:
|
||||
raise ProfileValidationError(
|
||||
f"Profile '{kind}' has unknown kind={yaml_kind!r}; "
|
||||
f"supported: {list(_PROFILE_MODELS)}"
|
||||
)
|
||||
|
||||
try:
|
||||
return model_cls.model_validate(raw)
|
||||
except ValidationError as exc:
|
||||
raise ProfileValidationError(
|
||||
f"Profile '{kind}' failed validation: {exc}"
|
||||
) from exc
|
||||
|
||||
|
||||
def list_profiles() -> list[dict[str, Any]]:
|
||||
"""Return structural data for all available pricing profiles.
|
||||
|
||||
Scans ``app/integrations/pricing/profiles/*.yaml``, loads each, and returns
|
||||
a list of dicts (Pydantic model dumps). Profiles that fail to load are
|
||||
skipped with a warning.
|
||||
|
||||
Returns
|
||||
-------
|
||||
list[dict[str, Any]]
|
||||
One entry per valid profile, suitable for JSON serialisation and
|
||||
front-end form rendering.
|
||||
"""
|
||||
results: list[dict[str, Any]] = []
|
||||
if not _PROFILES_DIR.exists():
|
||||
return results
|
||||
|
||||
for path in sorted(_PROFILES_DIR.glob("*.yaml")):
|
||||
kind = path.stem
|
||||
try:
|
||||
profile = load_profile(kind)
|
||||
results.append(profile.model_dump())
|
||||
except (ProfileNotFoundError, ProfileValidationError, Exception) as exc:
|
||||
logger.warning("Skipping malformed pricing profile '%s': %s", kind, exc)
|
||||
|
||||
return results
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Contract-values validation
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _fill_defaults_manual(values: dict[str, Any], profile: ManualProfile) -> dict[str, Any]:
|
||||
"""Return a copy of *values* with any ``ode`` default applied if missing."""
|
||||
filled = dict(values)
|
||||
|
||||
energy = dict(filled.get("energy", {}))
|
||||
|
||||
# Apply default for ode (default=0) if absent.
|
||||
if "ode" not in energy and profile.energy.ode.default is not None:
|
||||
energy["ode"] = profile.energy.ode.default
|
||||
|
||||
filled["energy"] = energy
|
||||
return filled
|
||||
|
||||
|
||||
def _fill_defaults_tibber(values: dict[str, Any], profile: TibberProfile) -> dict[str, Any]:
|
||||
"""Return a copy of *values* with sell_adjust and management_fee defaults applied."""
|
||||
filled = dict(values)
|
||||
|
||||
energy = dict(filled.get("energy", {}))
|
||||
# Apply default for sell_adjust (default=0) if absent.
|
||||
if "sell_adjust" not in energy and profile.energy.sell_adjust.default is not None:
|
||||
energy["sell_adjust"] = profile.energy.sell_adjust.default
|
||||
filled["energy"] = energy
|
||||
|
||||
standing = dict(filled.get("standing", {}))
|
||||
# Apply default for management_fee if absent.
|
||||
if (
|
||||
"management_fee" not in standing
|
||||
and profile.standing.management_fee.default is not None
|
||||
):
|
||||
standing["management_fee"] = profile.standing.management_fee.default
|
||||
filled["standing"] = standing
|
||||
|
||||
return filled
|
||||
|
||||
|
||||
def _require_numeric(section: str, key: str, container: dict[str, Any]) -> None:
|
||||
"""Assert that *container[key]* exists and is a number; raise ProfileValidationError."""
|
||||
if key not in container:
|
||||
raise ProfileValidationError(
|
||||
f"Contract values missing required field '{section}.{key}'"
|
||||
)
|
||||
val = container[key]
|
||||
if not isinstance(val, (int, float)):
|
||||
raise ProfileValidationError(
|
||||
f"Contract values field '{section}.{key}' must be a number, "
|
||||
f"got {type(val).__name__!r}"
|
||||
)
|
||||
|
||||
|
||||
def _validate_manual_values(values: dict[str, Any], profile: ManualProfile) -> dict[str, Any]:
|
||||
"""Validate and fill-defaults for manual contract values.
|
||||
|
||||
Returns the filled values dict on success. Raises ProfileValidationError
|
||||
on missing required fields or wrong types.
|
||||
"""
|
||||
filled = _fill_defaults_manual(values, profile)
|
||||
energy = filled.get("energy", {})
|
||||
buy = energy.get("buy", {})
|
||||
sell = energy.get("sell", {})
|
||||
standing = filled.get("standing", {})
|
||||
credits = filled.get("credits", {})
|
||||
|
||||
# Required energy.buy fields.
|
||||
_require_numeric("energy.buy", "normal", buy)
|
||||
_require_numeric("energy.buy", "dal", buy)
|
||||
# Required energy.sell fields.
|
||||
_require_numeric("energy.sell", "normal", sell)
|
||||
_require_numeric("energy.sell", "dal", sell)
|
||||
# Required energy fields.
|
||||
_require_numeric("energy", "energy_tax", energy)
|
||||
_require_numeric("energy", "ode", energy)
|
||||
# Required standing fields.
|
||||
_require_numeric("standing", "network_fee", standing)
|
||||
_require_numeric("standing", "management_fee", standing)
|
||||
# Required credits fields.
|
||||
_require_numeric("credits", "heffingskorting", credits)
|
||||
|
||||
return filled
|
||||
|
||||
|
||||
def _validate_tibber_values(values: dict[str, Any], profile: TibberProfile) -> dict[str, Any]:
|
||||
"""Validate and fill-defaults for tibber contract values.
|
||||
|
||||
Returns the filled values dict on success. Raises ProfileValidationError
|
||||
on missing required fields or wrong types.
|
||||
"""
|
||||
filled = _fill_defaults_tibber(values, profile)
|
||||
energy = filled.get("energy", {})
|
||||
standing = filled.get("standing", {})
|
||||
credits = filled.get("credits", {})
|
||||
|
||||
# Required energy fields.
|
||||
_require_numeric("energy", "energy_tax", energy)
|
||||
_require_numeric("energy", "sell_adjust", energy)
|
||||
# Required standing fields.
|
||||
_require_numeric("standing", "management_fee", standing)
|
||||
_require_numeric("standing", "network_fee", standing)
|
||||
# Required credits fields.
|
||||
_require_numeric("credits", "heffingskorting", credits)
|
||||
|
||||
return filled
|
||||
|
||||
|
||||
def validate_values(kind: str, values: dict[str, Any]) -> dict[str, Any]:
|
||||
"""Validate a contract-values dict against the named profile structure.
|
||||
|
||||
Fields that carry a ``default`` in the profile (e.g. ``ode``,
|
||||
``sell_adjust``, tibber ``management_fee``) are silently filled in when
|
||||
absent from *values*. Fields with no default that are absent, or fields
|
||||
whose value is not a number, cause a ``ProfileValidationError``.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
kind:
|
||||
Profile kind (``"manual"`` or ``"tibber"``).
|
||||
values:
|
||||
Contract values dict as stored in ``EnergyContractVersion.values``.
|
||||
|
||||
Returns
|
||||
-------
|
||||
dict[str, Any]
|
||||
A (possibly mutated) copy of *values* with defaults applied.
|
||||
|
||||
Raises
|
||||
------
|
||||
ProfileNotFoundError
|
||||
If the profile YAML for *kind* does not exist.
|
||||
ProfileValidationError
|
||||
If required fields are missing or have wrong types.
|
||||
"""
|
||||
profile = load_profile(kind)
|
||||
if isinstance(profile, ManualProfile):
|
||||
return _validate_manual_values(values, profile)
|
||||
if isinstance(profile, TibberProfile):
|
||||
return _validate_tibber_values(values, profile)
|
||||
# Unreachable with current kinds, but guard for future extensions.
|
||||
raise ProfileValidationError(f"No validator implemented for kind={kind!r}")
|
||||
@@ -0,0 +1,20 @@
|
||||
kind: manual
|
||||
label: 固定 / 可变费率(NL,双费率)
|
||||
|
||||
energy:
|
||||
dual_tariff: true # use delivered_1/2, returned_1/2 for low/high tariffs
|
||||
buy:
|
||||
normal: { unit: EUR/kWh } # high-tariff buy price (delivered_2 register)
|
||||
dal: { unit: EUR/kWh } # low-tariff buy price (delivered_1 register)
|
||||
sell:
|
||||
normal: { unit: EUR/kWh } # high-tariff sell / return price
|
||||
dal: { unit: EUR/kWh } # low-tariff sell / return price (currently equal to normal)
|
||||
energy_tax: { unit: EUR/kWh } # energy tax added to buy price (incl. VAT)
|
||||
ode: { unit: EUR/kWh, default: 0 } # currently merged into energy_tax
|
||||
|
||||
standing: # fixed charges; UI fills per month, engine prorates to days
|
||||
network_fee: { unit: EUR/month }
|
||||
management_fee: { unit: EUR/month }
|
||||
|
||||
credits:
|
||||
heffingskorting: { unit: EUR/year } # energy-tax credit; deducted at summary layer
|
||||
@@ -0,0 +1,14 @@
|
||||
kind: tibber
|
||||
label: Tibber 动态电价(15 分钟)
|
||||
|
||||
energy:
|
||||
source: tibber_api # buy = total (from API); sell = total − energy_tax − sell_adjust
|
||||
energy_tax: { unit: EUR/kWh } # subtracted from total to derive sell price (incl. VAT)
|
||||
sell_adjust: { unit: EUR/kWh, default: 0 } # additional sell-price adjustment (residual spread)
|
||||
|
||||
standing: # fixed charges; UI fills per month, engine prorates to days
|
||||
management_fee: { unit: EUR/month, default: 5.99 }
|
||||
network_fee: { unit: EUR/month }
|
||||
|
||||
credits:
|
||||
heffingskorting: { unit: EUR/year } # energy-tax credit; deducted at summary layer
|
||||
@@ -0,0 +1,288 @@
|
||||
"""Price-calculation strategy registry and built-in implementations.
|
||||
|
||||
A *strategy* is a plain function registered under a ``kind`` string. Given
|
||||
per-register kWh deltas, the period start time, the active contract-version
|
||||
values, and a DB session, it returns a result dict with:
|
||||
|
||||
{
|
||||
"import_cost": Decimal, # total cost of electricity drawn from grid
|
||||
"export_revenue": Decimal, # total revenue from electricity fed to grid
|
||||
"net_cost": Decimal, # import_cost − export_revenue
|
||||
"pricing": dict, # snapshot of the price inputs used (for auditing)
|
||||
}
|
||||
|
||||
**Import and export are kept separate throughout** — net_cost is only derived
|
||||
at the end. This supports the Dutch no-netting rule (saldering afgebouwd).
|
||||
|
||||
**All monetary arithmetic uses Decimal** converted via ``Decimal(str(x))`` to
|
||||
avoid float binary rounding errors.
|
||||
|
||||
Design notes
|
||||
------------
|
||||
- **Purely data + functions**: no ABC, no class hierarchy. A ``dict``-based
|
||||
registry is the simplest structure that supports the two current kinds and
|
||||
leaves the door open for future additions (e.g. ``octopus``, ``frank``).
|
||||
- ``deltas`` is a plain dataclass ``PeriodDeltas`` — typed, but no ORM.
|
||||
- Tibber strategy queries ``TibberPrice`` directly from the session; it does not
|
||||
call the Tibber API (that is T05's job).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime
|
||||
from decimal import Decimal
|
||||
from typing import Any, Callable
|
||||
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Shared data types
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@dataclass
|
||||
class PeriodDeltas:
|
||||
"""Per-register kWh deltas for one 15-minute billing period.
|
||||
|
||||
All values are ``Decimal`` and represent ``end_reading − start_reading``
|
||||
for the corresponding cumulative energy register.
|
||||
|
||||
Attributes
|
||||
----------
|
||||
d1: Decimal
|
||||
Delivered (consumed) kWh on the low/dal tariff register (delivered_1).
|
||||
d2: Decimal
|
||||
Delivered (consumed) kWh on the normal/high tariff register (delivered_2).
|
||||
r1: Decimal
|
||||
Returned (fed-to-grid) kWh on the low/dal tariff register (returned_1).
|
||||
r2: Decimal
|
||||
Returned (fed-to-grid) kWh on the normal/high tariff register (returned_2).
|
||||
"""
|
||||
|
||||
d1: Decimal # delivered low-tariff
|
||||
d2: Decimal # delivered high-tariff
|
||||
r1: Decimal # returned low-tariff
|
||||
r2: Decimal # returned high-tariff
|
||||
|
||||
|
||||
# Strategy callable signature.
|
||||
StrategyFn = Callable[
|
||||
[PeriodDeltas, datetime, dict[str, Any], Session],
|
||||
dict[str, Any],
|
||||
]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Registry
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_REGISTRY: dict[str, StrategyFn] = {}
|
||||
|
||||
|
||||
def register_strategy(kind: str, fn: StrategyFn) -> None:
|
||||
"""Register *fn* as the price-calculation strategy for *kind*.
|
||||
|
||||
Overwrites any previously registered strategy for the same kind (allows
|
||||
monkey-patching in tests).
|
||||
"""
|
||||
_REGISTRY[kind] = fn
|
||||
|
||||
|
||||
def get_strategy(kind: str) -> StrategyFn:
|
||||
"""Return the registered strategy for *kind*.
|
||||
|
||||
Raises
|
||||
------
|
||||
KeyError
|
||||
If no strategy is registered for *kind*.
|
||||
"""
|
||||
if kind not in _REGISTRY:
|
||||
raise KeyError(
|
||||
f"No price strategy registered for kind={kind!r}. "
|
||||
f"Available: {sorted(_REGISTRY)}"
|
||||
)
|
||||
return _REGISTRY[kind]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _to_decimal(value: Any) -> Decimal:
|
||||
"""Convert *value* to Decimal via str() to avoid float binary rounding."""
|
||||
return Decimal(str(value))
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Manual strategy — fixed dual-tariff
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _manual_strategy(
|
||||
deltas: PeriodDeltas,
|
||||
t0: datetime,
|
||||
values: dict[str, Any],
|
||||
session: Session,
|
||||
) -> dict[str, Any]:
|
||||
"""Calculate billing costs for one period using fixed dual-tariff rates.
|
||||
|
||||
Formula (§3.4):
|
||||
- ``buy_dal = energy.buy.dal + energy.energy_tax + energy.ode``
|
||||
- ``buy_normal = energy.buy.normal + energy.energy_tax + energy.ode``
|
||||
- ``import_cost = Δd1 × buy_dal + Δd2 × buy_normal``
|
||||
- ``sell_dal = energy.sell.dal`` (no tax on return price)
|
||||
- ``sell_normal = energy.sell.normal``
|
||||
- ``export_revenue = Δr1 × sell_dal + Δr2 × sell_normal``
|
||||
- ``net_cost = import_cost − export_revenue``
|
||||
|
||||
The ``pricing`` snapshot contains all per-unit prices used so that the
|
||||
result is fully auditable without re-querying the contract version.
|
||||
"""
|
||||
energy = values.get("energy", {})
|
||||
buy = energy.get("buy", {})
|
||||
sell = energy.get("sell", {})
|
||||
|
||||
energy_tax = _to_decimal(energy.get("energy_tax", 0))
|
||||
ode = _to_decimal(energy.get("ode", 0))
|
||||
|
||||
buy_dal_base = _to_decimal(buy.get("dal", 0))
|
||||
buy_normal_base = _to_decimal(buy.get("normal", 0))
|
||||
sell_dal = _to_decimal(sell.get("dal", 0))
|
||||
sell_normal = _to_decimal(sell.get("normal", 0))
|
||||
|
||||
# Effective buy prices including all taxes.
|
||||
buy_dal = buy_dal_base + energy_tax + ode
|
||||
buy_normal = buy_normal_base + energy_tax + ode
|
||||
|
||||
import_cost = deltas.d1 * buy_dal + deltas.d2 * buy_normal
|
||||
export_revenue = deltas.r1 * sell_dal + deltas.r2 * sell_normal
|
||||
net_cost = import_cost - export_revenue
|
||||
|
||||
pricing_snapshot = {
|
||||
"kind": "manual",
|
||||
"buy_dal": str(buy_dal),
|
||||
"buy_normal": str(buy_normal),
|
||||
"sell_dal": str(sell_dal),
|
||||
"sell_normal": str(sell_normal),
|
||||
"energy_tax": str(energy_tax),
|
||||
"ode": str(ode),
|
||||
}
|
||||
|
||||
return {
|
||||
"import_cost": import_cost,
|
||||
"export_revenue": export_revenue,
|
||||
"net_cost": net_cost,
|
||||
"pricing": pricing_snapshot,
|
||||
}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Tibber strategy — dynamic 15-minute spot price
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TibberPriceNotFoundError(LookupError):
|
||||
"""Raised when no TibberPrice row covers the requested period start.
|
||||
|
||||
The billing engine (T07) catches this to mark the period as ``degraded``
|
||||
or skip it until a price becomes available.
|
||||
"""
|
||||
|
||||
|
||||
def _tibber_strategy(
|
||||
deltas: PeriodDeltas,
|
||||
t0: datetime,
|
||||
values: dict[str, Any],
|
||||
session: Session,
|
||||
) -> dict[str, Any]:
|
||||
"""Calculate billing costs for one period using Tibber dynamic pricing.
|
||||
|
||||
Price lookup:
|
||||
Takes the most recent ``TibberPrice`` row where ``starts_at ≤ t0``
|
||||
(i.e. the slot that was in effect at *t0*). This is a DESC LIMIT 1
|
||||
query on ``starts_at``.
|
||||
|
||||
Formula (§3.4):
|
||||
- ``buy = total`` (Tibber's all-inclusive price, already includes tax)
|
||||
- ``sell = total − energy_tax − sell_adjust``
|
||||
- ``import_cost = (Δd1 + Δd2) × buy``
|
||||
- ``export_revenue = (Δr1 + Δr2) × sell``
|
||||
- ``net_cost = import_cost − export_revenue``
|
||||
|
||||
Tibber does not differentiate tariff slots (dal vs normal) — the 15-minute
|
||||
API price applies to the full delivered/returned volume.
|
||||
|
||||
Negative ``total`` (extreme negative spot prices):
|
||||
When ``total`` is negative the ``sell`` price will also be negative,
|
||||
meaning ``export_revenue`` becomes negative (feeding to grid *costs*
|
||||
money). This is the mathematically correct outcome and is left as-is.
|
||||
|
||||
Raises
|
||||
------
|
||||
TibberPriceNotFoundError
|
||||
If no ``TibberPrice`` row exists with ``starts_at ≤ t0``. The caller
|
||||
(T07 billing engine) should catch this and mark the period as
|
||||
``degraded`` or skip it for later recomputation.
|
||||
"""
|
||||
from app.models.energy import TibberPrice # local import to avoid circular
|
||||
from sqlalchemy import desc
|
||||
|
||||
price_row: TibberPrice | None = (
|
||||
session.query(TibberPrice)
|
||||
.filter(TibberPrice.starts_at <= t0)
|
||||
.order_by(desc(TibberPrice.starts_at))
|
||||
.first()
|
||||
)
|
||||
|
||||
if price_row is None:
|
||||
raise TibberPriceNotFoundError(
|
||||
f"No TibberPrice found covering t0={t0.isoformat()!r}; "
|
||||
"period cannot be billed until price data is available."
|
||||
)
|
||||
|
||||
energy = values.get("energy", {})
|
||||
energy_tax = _to_decimal(energy.get("energy_tax", 0))
|
||||
sell_adjust = _to_decimal(energy.get("sell_adjust", 0))
|
||||
|
||||
total = _to_decimal(price_row.total)
|
||||
buy = total
|
||||
sell = total - energy_tax - sell_adjust
|
||||
|
||||
total_delivered = deltas.d1 + deltas.d2
|
||||
total_returned = deltas.r1 + deltas.r2
|
||||
|
||||
import_cost = total_delivered * buy
|
||||
export_revenue = total_returned * sell
|
||||
net_cost = import_cost - export_revenue
|
||||
|
||||
pricing_snapshot = {
|
||||
"kind": "tibber",
|
||||
"tibber_price_starts_at": price_row.starts_at.isoformat(),
|
||||
"tibber_price_id": price_row.id,
|
||||
"total": str(total),
|
||||
"buy": str(buy),
|
||||
"sell": str(sell),
|
||||
"energy_tax": str(energy_tax),
|
||||
"sell_adjust": str(sell_adjust),
|
||||
}
|
||||
|
||||
return {
|
||||
"import_cost": import_cost,
|
||||
"export_revenue": export_revenue,
|
||||
"net_cost": net_cost,
|
||||
"pricing": pricing_snapshot,
|
||||
}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Register built-in strategies at module import time
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
register_strategy("manual", _manual_strategy)
|
||||
register_strategy("tibber", _tibber_strategy)
|
||||
@@ -0,0 +1,10 @@
|
||||
"""Tibber dynamic electricity pricing integration.
|
||||
|
||||
This package provides:
|
||||
- ``client``: HTTP client for the Tibber GraphQL API (price fetching).
|
||||
- Custom exceptions: ``TibberError`` (base) and ``TibberAuthError`` (auth failure).
|
||||
|
||||
Usage::
|
||||
|
||||
from app.integrations.tibber.client import fetch_price_range, TibberAuthError
|
||||
"""
|
||||
@@ -0,0 +1,357 @@
|
||||
"""Tibber GraphQL API client for fetching electricity price data.
|
||||
|
||||
Design decisions
|
||||
----------------
|
||||
- Uses ``httpx`` (already a project dependency) for all HTTP requests.
|
||||
- Token is **never** written to log messages or exception strings to prevent
|
||||
credential leakage into log aggregators.
|
||||
- ``fetch_price_range`` returns a list of ``PricePoint`` objects with UTC-aware
|
||||
``starts_at`` datetimes; the number of nodes is not assumed — all returned
|
||||
nodes are parsed regardless of count.
|
||||
- ``fetch_current_price`` uses a separate, simpler query that asks for the
|
||||
*current* price point only; used by the connection-test endpoint (T09) to
|
||||
produce a fast three-state result (success / auth-error / network-error).
|
||||
- Authentication failures (HTTP 401/403) are raised as ``TibberAuthError`` so
|
||||
that callers (the test endpoint) can distinguish them from generic network or
|
||||
API errors.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from dataclasses import dataclass
|
||||
from datetime import UTC, datetime
|
||||
from typing import Any
|
||||
|
||||
import httpx
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_TIBBER_API_URL = "https://api.tibber.com/v1-beta/gql"
|
||||
_DEFAULT_TIMEOUT = 15.0
|
||||
|
||||
# GraphQL query to fetch the forward-looking today + tomorrow price curve at
|
||||
# 15-minute resolution.
|
||||
#
|
||||
# ``priceInfo(resolution: QUARTER_HOURLY)`` returns two node lists:
|
||||
# * ``today`` — always the full current local day (96 quarter-hourly slots,
|
||||
# 00:00 → 23:45 local), regardless of the current time.
|
||||
# * ``tomorrow`` — the full next local day (96 slots) once Tibber publishes the
|
||||
# day-ahead prices (around 13:00–15:00 local); empty before that.
|
||||
#
|
||||
# This is deliberately NOT ``priceInfoRange``: that field is a historical cursor
|
||||
# connection whose range ends at "now" (it never returns future slots), so it
|
||||
# cannot supply upcoming prices. ``priceInfo`` is forward-looking, so every
|
||||
# 15-minute slot's price is present in the DB *before* the slot closes — which is
|
||||
# what makes per-slot billing accurate (each period finds its own exact slot
|
||||
# instead of falling back to a stale earlier price) and keeps the live current-
|
||||
# price entity fresh. The hourly refresh job re-runs this query, so tomorrow's
|
||||
# prices are picked up within an hour of publication without a restart.
|
||||
_PRICE_RANGE_QUERY = """
|
||||
{
|
||||
viewer {
|
||||
homes {
|
||||
id
|
||||
currentSubscription {
|
||||
priceInfo(resolution: QUARTER_HOURLY) {
|
||||
today {
|
||||
startsAt
|
||||
total
|
||||
energy
|
||||
tax
|
||||
currency
|
||||
level
|
||||
}
|
||||
tomorrow {
|
||||
startsAt
|
||||
total
|
||||
energy
|
||||
tax
|
||||
currency
|
||||
level
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
"""
|
||||
|
||||
# GraphQL query to fetch the single *current* price point.
|
||||
_CURRENT_PRICE_QUERY = """
|
||||
{
|
||||
viewer {
|
||||
homes {
|
||||
id
|
||||
currentSubscription {
|
||||
priceInfo {
|
||||
current {
|
||||
startsAt
|
||||
total
|
||||
energy
|
||||
tax
|
||||
currency
|
||||
level
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
"""
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Custom exceptions
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TibberError(Exception):
|
||||
"""Base class for all Tibber client errors.
|
||||
|
||||
Raised for network failures, timeouts, unexpected HTTP status codes, and
|
||||
malformed API responses. The exception message will **never** contain the
|
||||
API token.
|
||||
"""
|
||||
|
||||
|
||||
class TibberAuthError(TibberError):
|
||||
"""Raised when the Tibber API rejects the provided token (HTTP 401/403).
|
||||
|
||||
Callers that implement a three-state connection test should catch this
|
||||
exception separately from ``TibberError`` to distinguish auth problems
|
||||
from network / API problems.
|
||||
"""
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Data model
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
class PricePoint:
|
||||
"""One 15-minute price slot returned by the Tibber API.
|
||||
|
||||
All monetary values are in ``currency`` and include VAT (user-facing).
|
||||
``starts_at`` is always a timezone-aware UTC datetime regardless of the
|
||||
timezone offset that Tibber returns in ``startsAt``.
|
||||
"""
|
||||
|
||||
starts_at: datetime # UTC-aware
|
||||
total: float # full all-in price (energy + tax)
|
||||
energy: float # spot energy component
|
||||
tax: float # tax component
|
||||
currency: str # ISO 4217 (typically "EUR")
|
||||
level: str | None # e.g. "CHEAP", "NORMAL", "EXPENSIVE"; None if absent
|
||||
resolution: str # label for the slot resolution (e.g. "QUARTER_HOURLY")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Internal helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _parse_starts_at(raw: str) -> datetime:
|
||||
"""Parse an ISO 8601 timestamp with timezone offset and convert to UTC.
|
||||
|
||||
Tibber returns timestamps like ``2026-06-23T00:00:00.000+02:00``.
|
||||
``datetime.fromisoformat`` handles this format in Python 3.11+; the result
|
||||
is then converted to UTC via ``.astimezone(UTC)``.
|
||||
"""
|
||||
return datetime.fromisoformat(raw).astimezone(UTC)
|
||||
|
||||
|
||||
def _parse_node(node: dict[str, Any], resolution: str) -> PricePoint:
|
||||
"""Parse a single price node dict into a ``PricePoint``."""
|
||||
return PricePoint(
|
||||
starts_at=_parse_starts_at(node["startsAt"]),
|
||||
total=float(node["total"]),
|
||||
energy=float(node["energy"]),
|
||||
tax=float(node["tax"]),
|
||||
currency=str(node["currency"]),
|
||||
level=node.get("level") or None,
|
||||
resolution=resolution,
|
||||
)
|
||||
|
||||
|
||||
def _pick_home(homes: list[dict[str, Any]], home_id: str | None) -> dict[str, Any]:
|
||||
"""Return the home dict matching *home_id*, or the first home if None."""
|
||||
if not homes:
|
||||
raise TibberError("Tibber API returned no homes")
|
||||
if home_id is not None:
|
||||
for h in homes:
|
||||
if h.get("id") == home_id:
|
||||
return h
|
||||
raise TibberError("Tibber home id not found in API response")
|
||||
return homes[0]
|
||||
|
||||
|
||||
def _post_graphql(token: str, query: str, timeout: float) -> dict[str, Any]:
|
||||
"""POST the GraphQL *query* and return the parsed ``data`` dict.
|
||||
|
||||
Raises
|
||||
------
|
||||
TibberAuthError
|
||||
On HTTP 401 or 403.
|
||||
TibberError
|
||||
On timeouts, network errors, non-2xx responses, or unexpected body shape.
|
||||
"""
|
||||
headers = {
|
||||
"Authorization": f"Bearer {token}",
|
||||
"Content-Type": "application/json",
|
||||
}
|
||||
try:
|
||||
response = httpx.post(
|
||||
_TIBBER_API_URL,
|
||||
json={"query": query},
|
||||
headers=headers,
|
||||
timeout=timeout,
|
||||
)
|
||||
except httpx.TimeoutException as exc:
|
||||
raise TibberError("Tibber API request timed out") from exc
|
||||
except httpx.HTTPError as exc:
|
||||
raise TibberError("Tibber API request failed") from exc
|
||||
|
||||
if response.status_code in (401, 403):
|
||||
raise TibberAuthError("Tibber API authentication failed")
|
||||
|
||||
try:
|
||||
response.raise_for_status()
|
||||
except httpx.HTTPStatusError as exc:
|
||||
raise TibberError(f"Tibber API returned unexpected status {response.status_code}") from exc
|
||||
|
||||
try:
|
||||
body = response.json()
|
||||
except Exception as exc:
|
||||
raise TibberError("Tibber API returned non-JSON response") from exc
|
||||
|
||||
if "errors" in body:
|
||||
# GraphQL errors are not HTTP errors; surface them as TibberError.
|
||||
# Do not include token in the message.
|
||||
raise TibberError("Tibber GraphQL returned errors")
|
||||
|
||||
data = body.get("data")
|
||||
if data is None:
|
||||
raise TibberError("Tibber API response missing 'data' field")
|
||||
|
||||
return data
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Public API
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def fetch_price_range(
|
||||
token: str,
|
||||
home_id: str | None = None,
|
||||
*,
|
||||
timeout: float = _DEFAULT_TIMEOUT,
|
||||
) -> list[PricePoint]:
|
||||
"""Fetch the forward-looking today + tomorrow 15-minute price curve from Tibber.
|
||||
|
||||
Sends the ``priceInfo(resolution: QUARTER_HOURLY) { today tomorrow }`` query
|
||||
and parses every node from both lists (today first, then tomorrow) into a
|
||||
``PricePoint``. ``priceInfo`` is forward-looking — ``today`` is always the
|
||||
full current local day and ``tomorrow`` is populated once Tibber publishes the
|
||||
day-ahead prices — so upcoming slots are returned, unlike ``priceInfoRange``
|
||||
which only reaches "now". ``tomorrow`` may be empty (before publication); the
|
||||
number of nodes is not assumed and all returned nodes are parsed.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
token:
|
||||
Tibber API token. **Never** logged or included in exception messages.
|
||||
home_id:
|
||||
If given, the home with this Tibber home ID is selected. If ``None``,
|
||||
the first home in the account is used.
|
||||
timeout:
|
||||
HTTP request timeout in seconds (default 15 s).
|
||||
|
||||
Returns
|
||||
-------
|
||||
list[PricePoint]
|
||||
List of price points with UTC-aware ``starts_at`` values.
|
||||
|
||||
Raises
|
||||
------
|
||||
TibberAuthError
|
||||
If the token is rejected (HTTP 401/403).
|
||||
TibberError
|
||||
For network failures, timeouts, or unexpected API responses.
|
||||
"""
|
||||
data = _post_graphql(token, _PRICE_RANGE_QUERY, timeout)
|
||||
|
||||
try:
|
||||
homes = data["viewer"]["homes"]
|
||||
except (KeyError, TypeError) as exc:
|
||||
raise TibberError("Tibber API response has unexpected shape") from exc
|
||||
|
||||
home = _pick_home(homes, home_id)
|
||||
|
||||
try:
|
||||
price_info = home["currentSubscription"]["priceInfo"]
|
||||
today = price_info["today"]
|
||||
tomorrow = price_info["tomorrow"]
|
||||
except (KeyError, TypeError) as exc:
|
||||
raise TibberError("Tibber API response missing priceInfo today/tomorrow") from exc
|
||||
|
||||
# tomorrow is null/empty until Tibber publishes the day-ahead prices; treat
|
||||
# a missing list as empty so we still return today's slots.
|
||||
nodes = list(today or []) + list(tomorrow or [])
|
||||
return [_parse_node(node, "QUARTER_HOURLY") for node in nodes]
|
||||
|
||||
|
||||
def fetch_current_price(
|
||||
token: str,
|
||||
home_id: str | None = None,
|
||||
*,
|
||||
timeout: float = _DEFAULT_TIMEOUT,
|
||||
) -> PricePoint:
|
||||
"""Fetch the *current* price point from the Tibber API.
|
||||
|
||||
Uses the lighter ``priceInfo { current { ... } }`` query rather than the
|
||||
full range query, making it suitable for a fast connection test.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
token:
|
||||
Tibber API token. **Never** logged or included in exception messages.
|
||||
home_id:
|
||||
If given, the home with this Tibber home ID is selected. If ``None``,
|
||||
the first home in the account is used.
|
||||
timeout:
|
||||
HTTP request timeout in seconds (default 15 s).
|
||||
|
||||
Returns
|
||||
-------
|
||||
PricePoint
|
||||
The current price point with a UTC-aware ``starts_at``.
|
||||
|
||||
Raises
|
||||
------
|
||||
TibberAuthError
|
||||
If the token is rejected (HTTP 401/403).
|
||||
TibberError
|
||||
For network failures, timeouts, missing current price, or unexpected
|
||||
API responses.
|
||||
"""
|
||||
data = _post_graphql(token, _CURRENT_PRICE_QUERY, timeout)
|
||||
|
||||
try:
|
||||
homes = data["viewer"]["homes"]
|
||||
except (KeyError, TypeError) as exc:
|
||||
raise TibberError("Tibber API response has unexpected shape") from exc
|
||||
|
||||
home = _pick_home(homes, home_id)
|
||||
|
||||
try:
|
||||
current = home["currentSubscription"]["priceInfo"]["current"]
|
||||
except (KeyError, TypeError) as exc:
|
||||
raise TibberError("Tibber API response missing current price") from exc
|
||||
|
||||
if current is None:
|
||||
raise TibberError("Tibber API returned null for current price")
|
||||
|
||||
return _parse_node(current, "QUARTER_HOURLY")
|
||||
+205
-1
@@ -7,12 +7,18 @@ from fastapi import FastAPI, HTTPException, Request
|
||||
from fastapi.responses import FileResponse
|
||||
from fastapi.staticfiles import StaticFiles
|
||||
from apscheduler.schedulers.background import BackgroundScheduler
|
||||
from apscheduler.triggers.cron import CronTrigger
|
||||
from apscheduler.triggers.interval import IntervalTrigger
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app import models # noqa: F401
|
||||
from app.api.routes.api.config import router as api_config_router
|
||||
from app.api.routes.api.data import router as api_data_router
|
||||
from app.api.routes.api.energy import router as api_energy_router
|
||||
from app.api.routes.api.energy_contracts import router as api_energy_contracts_router
|
||||
from app.api.routes.api.expose import router as api_expose_router
|
||||
from app.api.routes.api.meters import router as api_meters_router
|
||||
from app.api.routes.api.modbus import router as api_modbus_router
|
||||
from app.api.routes.api.session import router as api_session_router
|
||||
from app.api.routes import status
|
||||
from app.db import get_session_local
|
||||
@@ -22,9 +28,16 @@ from app.api.routes.poo import router as poo_router
|
||||
from app.api.routes.public_ip import router as public_ip_router
|
||||
from app.api.routes.ticktick import router as ticktick_router
|
||||
from app.config import get_settings
|
||||
from app.integrations.mqtt import mqtt_manager
|
||||
from app.services.auth import AuthBootstrapError, initialize_auth_schema
|
||||
from app.services.config_page import seed_missing_config_from_bootstrap, sync_app_hostname_from_bootstrap
|
||||
from app.services.config_page import build_runtime_settings, seed_missing_config_from_bootstrap, sync_app_hostname_from_bootstrap
|
||||
from app.services.dsmr_ingest import apply_dsmr_subscription
|
||||
from app.services.public_ip import check_public_ipv4_and_notify
|
||||
from app.services.modbus_poll import poll_all_enabled_devices, BASE_POLL_TICK_SECONDS
|
||||
from app.services.ha_discovery import publish_discovery, publish_states
|
||||
from app.services.tibber_prices import refresh_prices
|
||||
from app.services.energy_cost import compute_closed_periods
|
||||
from app.services.timezone import local_tz
|
||||
from scripts.app_db_adopt import AppDatabaseAdoptionError, validate_app_runtime_db
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -48,6 +61,120 @@ def _run_scheduled_public_ip_check() -> None:
|
||||
session.close()
|
||||
|
||||
|
||||
def _run_scheduled_modbus_poll() -> None:
|
||||
session_local = get_session_local()
|
||||
session: Session = session_local()
|
||||
try:
|
||||
poll_all_enabled_devices(session, bootstrap_settings=get_settings())
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
|
||||
def _run_scheduled_tibber_refresh() -> None:
|
||||
"""Scheduled job: fetch Tibber 15-minute prices and upsert into tibber_price.
|
||||
|
||||
Runs every hour so that:
|
||||
- Today's prices are available from startup.
|
||||
- Tomorrow's prices (published by Tibber around 13:00 CET / 11:00 UTC) are
|
||||
picked up within an hour of publication without requiring a server restart.
|
||||
|
||||
The job is a no-op when:
|
||||
- No active energy contract with kind="tibber" exists.
|
||||
- The Tibber API token is empty in the runtime settings.
|
||||
|
||||
Any client exceptions (auth failures, network errors) are caught and logged
|
||||
so that a single failed fetch does not crash the scheduler or affect the
|
||||
other background jobs.
|
||||
"""
|
||||
session_local = get_session_local()
|
||||
session: Session = session_local()
|
||||
try:
|
||||
runtime_settings = build_runtime_settings(session, get_settings())
|
||||
refresh_prices(session, runtime_settings)
|
||||
except Exception:
|
||||
logger.exception("_run_scheduled_tibber_refresh: unexpected error")
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
|
||||
def _run_scheduled_energy_cost() -> None:
|
||||
"""Scheduled job: compute billing records for all uncalculated closed 15-minute periods.
|
||||
|
||||
Runs every minute so that a new period is picked up within 1 minute of
|
||||
closing. The job is a no-op when:
|
||||
- No active energy contract with a version covering the period exists.
|
||||
- DSMR data has not yet arrived for the period boundaries.
|
||||
- The period's billing record already exists and is not degraded.
|
||||
|
||||
Any unexpected exceptions are caught and logged so that a single failure
|
||||
does not crash the scheduler or affect the other background jobs.
|
||||
"""
|
||||
session_local = get_session_local()
|
||||
session: Session = session_local()
|
||||
try:
|
||||
compute_closed_periods(session)
|
||||
# After billing periods are computed, push fresh energy-cost state values
|
||||
# to MQTT/HA. publish_states is internally guarded by _should_publish
|
||||
# (MQTT disabled / not connected → no-op), so this never raises due to
|
||||
# unconfigured MQTT and does not block the billing job.
|
||||
try:
|
||||
from app.services.ha_discovery import publish_states
|
||||
publish_states(session)
|
||||
except Exception:
|
||||
logger.exception("_run_scheduled_energy_cost: publish_states failed (non-fatal)")
|
||||
except Exception:
|
||||
logger.exception("_run_scheduled_energy_cost: unexpected error")
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
|
||||
def _run_scheduled_ha_state_publish() -> None:
|
||||
"""Periodic job: publish discovery configs + state + availability for all enabled exposed entities.
|
||||
|
||||
Runs every 60 seconds. When the MQTT broker is connected:
|
||||
- Publishes (or re-publishes) all HA Discovery configs (retained, idempotent).
|
||||
- Publishes the current state / availability for all enabled entities.
|
||||
|
||||
This also serves as the reliable "publish discovery after connect" mechanism:
|
||||
because paho connects asynchronously, a synchronous call immediately after
|
||||
``mqtt_manager.connect()`` would fire before the TCP handshake completes and
|
||||
be a no-op. Instead, this periodic job picks it up within 60 seconds of the
|
||||
broker becoming available — retained payloads make repeated publishes harmless.
|
||||
|
||||
Additionally, if MQTT is configured in DB but the manager is not yet connected
|
||||
(e.g. MQTT was enabled via UI after startup), this job attempts to connect so
|
||||
the user does not need to restart the server.
|
||||
"""
|
||||
session_local = get_session_local()
|
||||
session: Session = session_local()
|
||||
try:
|
||||
runtime_settings = build_runtime_settings(session, get_settings())
|
||||
# Reconnect if MQTT is configured (in DB) but not yet connected.
|
||||
if mqtt_manager.is_configured(runtime_settings) and not mqtt_manager.is_connected:
|
||||
logger.info("_run_scheduled_ha_state_publish: MQTT configured but not connected — attempting connect.")
|
||||
mqtt_manager.connect(runtime_settings)
|
||||
publish_discovery(session)
|
||||
publish_states(session)
|
||||
except Exception:
|
||||
logger.exception("_run_scheduled_ha_state_publish: unexpected error")
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
|
||||
def _run_midnight_state_publish() -> None:
|
||||
"""本地午夜后不久专门发布一次状态,让 *_today 的每日归零稳稳落在午夜之后
|
||||
(对 HA 钟慢几秒鲁棒)。best-effort:失败仅记日志,不影响调度器。"""
|
||||
session_local = get_session_local()
|
||||
session = session_local()
|
||||
try:
|
||||
from app.services.ha_discovery import publish_states
|
||||
publish_states(session)
|
||||
except Exception:
|
||||
logger.exception("_run_midnight_state_publish: failed (non-fatal)")
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
|
||||
def ensure_auth_db_ready() -> None:
|
||||
session_local = get_session_local()
|
||||
session: Session = session_local()
|
||||
@@ -83,8 +210,80 @@ async def lifespan(_: FastAPI):
|
||||
max_instances=1,
|
||||
coalesce=True,
|
||||
)
|
||||
scheduler.add_job(
|
||||
_run_scheduled_modbus_poll,
|
||||
trigger=IntervalTrigger(seconds=BASE_POLL_TICK_SECONDS),
|
||||
id="modbus-poll",
|
||||
replace_existing=True,
|
||||
max_instances=1,
|
||||
coalesce=True,
|
||||
)
|
||||
# Periodic HA state / availability publish (60-second fallback sweep).
|
||||
scheduler.add_job(
|
||||
_run_scheduled_ha_state_publish,
|
||||
trigger=IntervalTrigger(seconds=60),
|
||||
id="ha-state-publish",
|
||||
replace_existing=True,
|
||||
max_instances=1,
|
||||
coalesce=True,
|
||||
)
|
||||
# Tibber price refresh: fetch today + tomorrow every hour.
|
||||
# The job is a no-op when no active tibber contract or token is configured,
|
||||
# so it is safe to register unconditionally.
|
||||
scheduler.add_job(
|
||||
_run_scheduled_tibber_refresh,
|
||||
trigger=IntervalTrigger(hours=1),
|
||||
id="tibber-refresh",
|
||||
replace_existing=True,
|
||||
max_instances=1,
|
||||
coalesce=True,
|
||||
)
|
||||
# Energy cost billing: compute uncalculated closed 15-minute periods every minute.
|
||||
# The job is a no-op when no active contract or DSMR data is present, so it is
|
||||
# safe to register unconditionally.
|
||||
scheduler.add_job(
|
||||
_run_scheduled_energy_cost,
|
||||
trigger=IntervalTrigger(minutes=1),
|
||||
id="energy-cost",
|
||||
replace_existing=True,
|
||||
max_instances=1,
|
||||
coalesce=True,
|
||||
)
|
||||
# Dedicated midnight publish: fire at local 00:00:10 so *_today grace (5 s) has
|
||||
# already elapsed and the day-rolled value is pushed to HA immediately, rather
|
||||
# than waiting for the next 60-second ha-state-publish sweep.
|
||||
scheduler.add_job(
|
||||
_run_midnight_state_publish,
|
||||
trigger=CronTrigger(hour=0, minute=0, second=10, timezone=local_tz()),
|
||||
id="midnight-today-publish",
|
||||
replace_existing=True,
|
||||
max_instances=1,
|
||||
coalesce=True,
|
||||
)
|
||||
scheduler.start()
|
||||
|
||||
# MQTT: connect using DB-merged runtime settings so broker configured via UI
|
||||
# is picked up on restart (not just from env/bootstrap settings).
|
||||
# Discovery will be published by the first run of _run_scheduled_ha_state_publish
|
||||
# (within 60 s of startup), after the async paho handshake completes.
|
||||
_startup_session_local = get_session_local()
|
||||
_startup_session: Session = _startup_session_local()
|
||||
try:
|
||||
_startup_runtime_settings = build_runtime_settings(_startup_session, get_settings())
|
||||
finally:
|
||||
_startup_session.close()
|
||||
mqtt_manager.connect(_startup_runtime_settings)
|
||||
|
||||
# DSMR ingest: subscribe to the configured MQTT topic when enabled. The same
|
||||
# applier is called from PUT /api/config, so toggling DSMR via the UI takes
|
||||
# effect without an app restart.
|
||||
apply_dsmr_subscription(_startup_runtime_settings)
|
||||
|
||||
yield
|
||||
|
||||
# MQTT: clean shutdown before the process exits.
|
||||
mqtt_manager.disconnect()
|
||||
|
||||
scheduler.shutdown(wait=False)
|
||||
|
||||
|
||||
@@ -107,6 +306,11 @@ def create_app() -> FastAPI:
|
||||
app.include_router(status.router)
|
||||
app.include_router(api_config_router)
|
||||
app.include_router(api_data_router)
|
||||
app.include_router(api_energy_router)
|
||||
app.include_router(api_energy_contracts_router)
|
||||
app.include_router(api_meters_router)
|
||||
app.include_router(api_expose_router)
|
||||
app.include_router(api_modbus_router)
|
||||
app.include_router(api_session_router)
|
||||
app.include_router(homeassistant_router)
|
||||
app.include_router(location_router)
|
||||
|
||||
@@ -15,8 +15,12 @@ class AuthUser(Base):
|
||||
is_active: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True)
|
||||
force_password_change: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True)
|
||||
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||
# TOTP fields (Phase B, M4-T04) — nullable/false by default so existing users are unaffected
|
||||
totp_secret: Mapped[str | None] = mapped_column(String(64), nullable=True)
|
||||
totp_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
|
||||
|
||||
sessions: Mapped[list["AuthSession"]] = relationship(back_populates="user")
|
||||
recovery_codes: Mapped[list["RecoveryCode"]] = relationship(back_populates="user")
|
||||
|
||||
|
||||
class AuthSession(Base):
|
||||
@@ -31,3 +35,21 @@ class AuthSession(Base):
|
||||
revoked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
|
||||
|
||||
user: Mapped[AuthUser] = relationship(back_populates="sessions")
|
||||
|
||||
|
||||
class RecoveryCode(Base):
|
||||
"""One-time TOTP recovery codes for AuthUser.
|
||||
|
||||
``code_hash`` stores the Argon2 hash of the plaintext code (plaintext is
|
||||
returned only at setup time and never stored). ``used_at`` is NULL while
|
||||
the code is still valid; set to the consumption timestamp when consumed.
|
||||
"""
|
||||
|
||||
__tablename__ = "auth_recovery_code"
|
||||
|
||||
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
||||
user_id: Mapped[int] = mapped_column(ForeignKey("auth_users.id"), nullable=False, index=True)
|
||||
code_hash: Mapped[str] = mapped_column(String(255), nullable=False)
|
||||
used_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
|
||||
|
||||
user: Mapped[AuthUser] = relationship(back_populates="recovery_codes")
|
||||
|
||||
@@ -0,0 +1,31 @@
|
||||
from datetime import datetime
|
||||
|
||||
from sqlalchemy import DateTime, Index, Integer, String, UniqueConstraint
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
from app.db import Base
|
||||
|
||||
|
||||
class LoginThrottle(Base):
|
||||
"""Tracks per-key login failure state for exponential back-off throttling.
|
||||
|
||||
``scope`` is either ``'ip'`` or ``'user'``; ``key`` is the IP address or
|
||||
username string respectively. The pair ``(scope, key)`` is unique — one
|
||||
row per tracked entity. A row is deleted on successful login (clear).
|
||||
"""
|
||||
|
||||
__tablename__ = "auth_login_throttle"
|
||||
__table_args__ = (
|
||||
UniqueConstraint("scope", "key", name="uq_auth_login_throttle_scope_key"),
|
||||
Index("ix_auth_login_throttle_scope_key", "scope", "key"),
|
||||
)
|
||||
|
||||
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
||||
key: Mapped[str] = mapped_column(String(255), nullable=False)
|
||||
scope: Mapped[str] = mapped_column(String(16), nullable=False) # 'ip' | 'user'
|
||||
failures: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
||||
first_failed_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||
last_failed_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||
next_allowed_at: Mapped[datetime | None] = mapped_column(
|
||||
DateTime(timezone=True), nullable=True
|
||||
)
|
||||
@@ -0,0 +1,318 @@
|
||||
"""SQLAlchemy models for the energy pricing and DSMR metering subsystem.
|
||||
|
||||
Six tables:
|
||||
- meter: physical electricity meter lifecycle epoch.
|
||||
- dsmr_reading: raw DSMR telegram blobs (10-second down-sampled).
|
||||
- energy_contract: contract head (manual or tibber, one active at a time).
|
||||
- energy_contract_version: versioned pricing values; append-only for auditability.
|
||||
- tibber_price: cached Tibber 15-minute spot prices (immutable).
|
||||
- energy_cost_period: computed 15-minute billing periods (immutable snapshot).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import uuid as _uuid
|
||||
from datetime import datetime
|
||||
|
||||
from sqlalchemy import Boolean, DateTime, Float, ForeignKey, Integer, String
|
||||
from sqlalchemy.orm import Mapped, mapped_column, relationship
|
||||
from sqlalchemy.types import JSON
|
||||
|
||||
from app.db import Base
|
||||
|
||||
|
||||
def _uuid4_str() -> str:
|
||||
return str(_uuid.uuid4())
|
||||
|
||||
|
||||
class Meter(Base):
|
||||
"""One physical electricity meter's installation epoch.
|
||||
|
||||
A ``meter`` record represents the period ``[started_at, ended_at)`` during
|
||||
which a particular physical meter was installed and active. Replacing a meter
|
||||
(swap, home move, etc.) is modelled by closing the current record
|
||||
(``ended_at = swap_timestamp``) and opening a new one
|
||||
(``started_at = swap_timestamp``).
|
||||
|
||||
**Invariant**: for each ``commodity`` there is at most one active meter
|
||||
(``ended_at IS NULL``) at any point in time. The service layer enforces
|
||||
this — no DB-level constraint is added to keep the migration simple and to
|
||||
allow the application to return a meaningful error message.
|
||||
|
||||
``commodity`` defaults to ``"electricity"``; the field is a free-form string
|
||||
(no CHECK constraint) so future commodities (``gas``, ``heating``) can be
|
||||
added without a schema change.
|
||||
|
||||
``reason`` captures why this epoch started — one of ``initial``,
|
||||
``meter_swap``, ``home_move``, or ``other`` — stored as a plain string so
|
||||
the application layer controls the allowed set.
|
||||
"""
|
||||
|
||||
__tablename__ = "meter"
|
||||
|
||||
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
||||
|
||||
# Stable internal identity — used as HA Discovery unique_id anchor.
|
||||
uuid: Mapped[str] = mapped_column(
|
||||
String(36), unique=True, nullable=False, default=_uuid4_str
|
||||
)
|
||||
|
||||
# Human-readable label for this physical meter (e.g. address, serial, tariff zone).
|
||||
label: Mapped[str] = mapped_column(String(255), nullable=False)
|
||||
|
||||
# Energy commodity this meter measures. Defaults to "electricity".
|
||||
commodity: Mapped[str] = mapped_column(String(32), nullable=False, default="electricity")
|
||||
|
||||
# UTC timestamp when this meter epoch starts (inclusive). May be in the past
|
||||
# (retroactive declaration); effective billing start = max(started_at, data start).
|
||||
started_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||
|
||||
# UTC timestamp when this meter epoch ends (exclusive). NULL = currently active.
|
||||
ended_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
|
||||
|
||||
# Why this epoch was created. Application-layer validation enforces the
|
||||
# allowed set; no CHECK constraint to keep migrations simple.
|
||||
reason: Mapped[str] = mapped_column(String(64), nullable=False)
|
||||
|
||||
# Free-form note (e.g. location, physical meter id, reason details).
|
||||
note: Mapped[str | None] = mapped_column(String(1024), nullable=True)
|
||||
|
||||
# UTC timestamp of when this row was created.
|
||||
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||
|
||||
# Relationship to cost periods attributed to this meter epoch (not loaded eagerly).
|
||||
cost_periods: Mapped[list["EnergyCostPeriod"]] = relationship(
|
||||
back_populates="meter", cascade="save-update, merge"
|
||||
)
|
||||
|
||||
|
||||
class DsmrReading(Base):
|
||||
"""One down-sampled DSMR telegram stored as a full JSON blob.
|
||||
|
||||
Identity & idempotency are **independent of the DSMR Reader's telegram id**
|
||||
(that field overflows and must be manually reset to zero — a known DSMR
|
||||
quirk — so relying on it for uniqueness risks silently dropping new data).
|
||||
The table's own autoincrement ``id`` PK is the stable internal identity, and
|
||||
``recorded_at`` (the telegram timestamp) is the UNIQUE de-duplication key: a
|
||||
single P1 meter emits exactly one telegram per timestamp.
|
||||
|
||||
``recorded_at`` is a real column (not inside the payload) so time-range
|
||||
queries are efficient. The entire telegram frame is stored verbatim in
|
||||
``payload``; no field allow-list is applied so future commodities (gas,
|
||||
heating, three-phase) are accommodated without a schema change.
|
||||
"""
|
||||
|
||||
__tablename__ = "dsmr_reading"
|
||||
|
||||
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
||||
|
||||
# UTC timestamp of the sample — real column, UNIQUE (telegram-id-independent
|
||||
# idempotency key). The unique index also serves time-range queries.
|
||||
recorded_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True), nullable=False, unique=True
|
||||
)
|
||||
|
||||
# Telegram's own id (DSMR Reader assigns it). Stored only as a reference /
|
||||
# debugging aid — NOT used for uniqueness or idempotency (it overflows and
|
||||
# gets reset to zero). Nullable because some DSMR sources may not emit one.
|
||||
source_id: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
||||
|
||||
# Full telegram frame as a JSON object; values are typically JSON strings
|
||||
# (e.g. "20915.154") — callers must cast to Decimal before arithmetic.
|
||||
payload: Mapped[dict] = mapped_column(JSON, nullable=False)
|
||||
|
||||
|
||||
class EnergyContract(Base):
|
||||
"""Contract head: a named energy contract with a chosen pricing strategy.
|
||||
|
||||
``kind`` determines which price strategy is used (``manual`` for fixed
|
||||
dual-tariff rates entered by the user, ``tibber`` for dynamic API prices).
|
||||
Only one contract may be ``active`` at a time; the service layer enforces
|
||||
mutual exclusion. Specific pricing values live in ``EnergyContractVersion``
|
||||
so that price changes can be tracked without modifying historical records.
|
||||
"""
|
||||
|
||||
__tablename__ = "energy_contract"
|
||||
|
||||
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
||||
|
||||
# Human-readable label; freely editable by the user.
|
||||
name: Mapped[str] = mapped_column(String(255), nullable=False)
|
||||
|
||||
# Strategy selector: "manual" or "tibber". Application-layer validation
|
||||
# enforces the allowed set; no DB CHECK constraint is added to keep the
|
||||
# migration simple and the strategy registry extensible.
|
||||
kind: Mapped[str] = mapped_column(String(32), nullable=False)
|
||||
|
||||
# Whether this is the currently active contract (at most one should be True).
|
||||
active: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
|
||||
|
||||
# ISO 4217 currency code for all monetary values in this contract.
|
||||
currency: Mapped[str] = mapped_column(String(8), nullable=False, default="EUR")
|
||||
|
||||
# Audit timestamps.
|
||||
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||
|
||||
# Relationship to versions (back-reference; not loaded eagerly).
|
||||
versions: Mapped[list["EnergyContractVersion"]] = relationship(
|
||||
back_populates="contract", cascade="save-update, merge"
|
||||
)
|
||||
|
||||
|
||||
class EnergyContractVersion(Base):
|
||||
"""One time-bounded version of an energy contract's pricing values.
|
||||
|
||||
Pricing changes are modelled as new versions (append-only); existing versions
|
||||
are never modified so that historical ``EnergyCostPeriod`` records remain
|
||||
fully auditable. ``effective_to`` is ``NULL`` for the currently open version.
|
||||
"""
|
||||
|
||||
__tablename__ = "energy_contract_version"
|
||||
|
||||
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
||||
|
||||
# FK to the parent contract. RESTRICT prevents deletion of a contract that
|
||||
# still has versioned pricing rows attached to it.
|
||||
contract_id: Mapped[int] = mapped_column(
|
||||
ForeignKey("energy_contract.id", ondelete="RESTRICT"), nullable=False
|
||||
)
|
||||
|
||||
# Start of this version's validity window (inclusive, UTC).
|
||||
effective_from: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True), nullable=False
|
||||
)
|
||||
|
||||
# End of this version's validity window (exclusive, UTC). NULL means open-ended
|
||||
# (i.e. this is the most recent / current version).
|
||||
effective_to: Mapped[datetime | None] = mapped_column(
|
||||
DateTime(timezone=True), nullable=True
|
||||
)
|
||||
|
||||
# Pricing values as a JSON object conforming to the profile structure for
|
||||
# ``contract.kind`` (validated by the application layer against the YAML profile).
|
||||
values: Mapped[dict] = mapped_column(JSON, nullable=False)
|
||||
|
||||
# Creation timestamp.
|
||||
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||
|
||||
# Relationship back to the parent contract.
|
||||
contract: Mapped["EnergyContract"] = relationship(back_populates="versions")
|
||||
|
||||
# Relationship to cost periods that reference this version.
|
||||
cost_periods: Mapped[list["EnergyCostPeriod"]] = relationship(
|
||||
back_populates="contract_version", cascade="save-update, merge"
|
||||
)
|
||||
|
||||
|
||||
class TibberPrice(Base):
|
||||
"""Cached Tibber 15-minute spot price point (immutable once fetched).
|
||||
|
||||
``starts_at`` is unique so that upserts are idempotent. Past prices are
|
||||
never overwritten; the fetch job only adds rows for future time slots.
|
||||
"""
|
||||
|
||||
__tablename__ = "tibber_price"
|
||||
|
||||
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
||||
|
||||
# UTC start of the 15-minute slot; unique so upsert is idempotent.
|
||||
starts_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True), nullable=False, unique=True
|
||||
)
|
||||
|
||||
# Resolution label as returned by the Tibber API (e.g. "QUARTER_HOURLY").
|
||||
resolution: Mapped[str] = mapped_column(String(32), nullable=False)
|
||||
|
||||
# Price components in the contract currency (all include VAT, user-facing).
|
||||
energy: Mapped[float] = mapped_column(Float, nullable=False)
|
||||
tax: Mapped[float] = mapped_column(Float, nullable=False)
|
||||
total: Mapped[float] = mapped_column(Float, nullable=False)
|
||||
|
||||
# Tibber price level (e.g. "NORMAL", "CHEAP", "EXPENSIVE"); may be absent.
|
||||
level: Mapped[str | None] = mapped_column(String(32), nullable=True)
|
||||
|
||||
# ISO 4217 currency code as returned by the API.
|
||||
currency: Mapped[str] = mapped_column(String(8), nullable=False)
|
||||
|
||||
# UTC timestamp of when this row was fetched/inserted.
|
||||
fetched_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||
|
||||
|
||||
class EnergyCostPeriod(Base):
|
||||
"""Computed billing record for one 15-minute metering period (immutable snapshot).
|
||||
|
||||
Each row captures the per-register kWh deltas, the resulting import cost and
|
||||
export revenue, and a full snapshot of the pricing values used so that the
|
||||
calculation is fully auditable and reproducible without re-querying the
|
||||
contract version. Rows are written once and never modified; explicit
|
||||
recomputation via the API is the only way to overwrite a period.
|
||||
"""
|
||||
|
||||
__tablename__ = "energy_cost_period"
|
||||
|
||||
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
||||
|
||||
# UTC start of the 15-minute period; unique so upsert is idempotent.
|
||||
period_start: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True), nullable=False, unique=True
|
||||
)
|
||||
|
||||
# Per-register kWh deltas for the period (end minus start of cumulative registers).
|
||||
# _1 = dal/low-tariff, _2 = normal/high-tariff (NL convention).
|
||||
d1_kwh: Mapped[float] = mapped_column(Float, nullable=False) # delivered low
|
||||
d2_kwh: Mapped[float] = mapped_column(Float, nullable=False) # delivered high
|
||||
r1_kwh: Mapped[float] = mapped_column(Float, nullable=False) # returned low
|
||||
r2_kwh: Mapped[float] = mapped_column(Float, nullable=False) # returned high
|
||||
|
||||
# Computed monetary amounts for the period (in ``currency``).
|
||||
import_cost: Mapped[float] = mapped_column(Float, nullable=False)
|
||||
export_revenue: Mapped[float] = mapped_column(Float, nullable=False)
|
||||
net_cost: Mapped[float] = mapped_column(Float, nullable=False)
|
||||
|
||||
# ISO 4217 currency code matching the contract.
|
||||
currency: Mapped[str] = mapped_column(String(8), nullable=False)
|
||||
|
||||
# Full snapshot of the pricing inputs used during computation. This makes
|
||||
# each row self-contained and auditable even if the contract is later changed.
|
||||
pricing: Mapped[dict] = mapped_column(JSON, nullable=False)
|
||||
|
||||
# FK to the exact contract version whose values were used. RESTRICT prevents
|
||||
# deletion of a version that has cost records attached. Nullable to support
|
||||
# periods computed in ``degraded`` mode (missing price data).
|
||||
contract_version_id: Mapped[int | None] = mapped_column(
|
||||
ForeignKey("energy_contract_version.id", ondelete="RESTRICT"), nullable=True
|
||||
)
|
||||
|
||||
# FK to the meter epoch this period belongs to. RESTRICT prevents deletion of
|
||||
# a meter that still has attributed cost periods. Nullable for backwards
|
||||
# compatibility (pre-M7 rows) and degraded periods where the meter was not
|
||||
# determinable.
|
||||
meter_id: Mapped[int | None] = mapped_column(
|
||||
ForeignKey("meter.id", ondelete="RESTRICT"), nullable=True
|
||||
)
|
||||
|
||||
# True when the period was computed with incomplete data (missing readings or
|
||||
# missing price); serves as a flag for later recomputation.
|
||||
degraded: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
|
||||
|
||||
# UTC timestamp of when this row was computed/inserted.
|
||||
computed_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
|
||||
|
||||
# Relationship back to the contract version.
|
||||
contract_version: Mapped["EnergyContractVersion | None"] = relationship(
|
||||
back_populates="cost_periods"
|
||||
)
|
||||
|
||||
# Relationship back to the meter epoch.
|
||||
meter: Mapped["Meter | None"] = relationship(back_populates="cost_periods")
|
||||
|
||||
|
||||
# Index on recorded_at for efficient time-range queries on DSMR readings.
|
||||
# (The ORM-level index=True on recorded_at already creates ix_dsmr_reading_recorded_at;
|
||||
# no composite index is needed for single-meter deployments.)
|
||||
|
||||
# Index on period_start is covered by the unique constraint (SQLite creates an
|
||||
# implicit index for UNIQUE columns), so no additional index is required.
|
||||
|
||||
# Index on starts_at for TibberPrice is covered by the unique constraint similarly.
|
||||
@@ -0,0 +1,49 @@
|
||||
"""SQLAlchemy model for the exposed-entity toggle table.
|
||||
|
||||
``ExposedEntityToggle`` stores the per-entity enable/disable state for the
|
||||
MQTT / HA Discovery expose framework. The *catalog* of what *can* be
|
||||
exposed is computed dynamically by provider functions (see
|
||||
``app/integrations/expose.py``); this table only records which entries the
|
||||
user has explicitly enabled.
|
||||
|
||||
Key design decisions
|
||||
--------------------
|
||||
- ``key`` is a stable string identifier derived from device uuid + metric key
|
||||
(e.g. ``"modbus.<uuid>.voltage"``), so it does not drift if rows are
|
||||
deleted and re-inserted.
|
||||
- Default state for an entity not yet in this table is **disabled** (false).
|
||||
``build_catalog`` in expose.py treats a missing row as ``enabled=False``.
|
||||
- Only one row per entity key (``key`` is UNIQUE).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime
|
||||
|
||||
from sqlalchemy import Boolean, DateTime, Integer, String
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
from app.db import Base
|
||||
|
||||
|
||||
class ExposedEntityToggle(Base):
|
||||
"""Per-entity on/off switch for MQTT / HA Discovery publishing.
|
||||
|
||||
Rows are created on demand (first toggle); entities with no row are
|
||||
treated as disabled.
|
||||
"""
|
||||
|
||||
__tablename__ = "exposed_entity_toggle"
|
||||
|
||||
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
||||
|
||||
# Stable entity key, e.g. "modbus.<uuid>.voltage" or "modbus.<uuid>.online".
|
||||
key: Mapped[str] = mapped_column(String(255), unique=True, nullable=False)
|
||||
|
||||
# Whether this entity should be published via MQTT / HA Discovery.
|
||||
enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
|
||||
|
||||
# Last time this row was created or modified.
|
||||
updated_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True), nullable=False
|
||||
)
|
||||
@@ -0,0 +1,114 @@
|
||||
"""SQLAlchemy models for Modbus device management and telemetry.
|
||||
|
||||
Two tables:
|
||||
- modbus_device: deployment/configurable metadata for each polled device.
|
||||
- modbus_reading: generic telemetry rows (one per device per poll cycle).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import uuid as _uuid
|
||||
from datetime import datetime
|
||||
|
||||
from sqlalchemy import Boolean, DateTime, ForeignKey, Index, Integer, String
|
||||
from sqlalchemy.orm import Mapped, mapped_column, relationship
|
||||
from sqlalchemy.types import JSON
|
||||
|
||||
from app.db import Base
|
||||
|
||||
|
||||
def _uuid4_str() -> str:
|
||||
return str(_uuid.uuid4())
|
||||
|
||||
|
||||
class ModbusDevice(Base):
|
||||
"""Deployment-layer record for a Modbus slave device reachable via a TCP gateway.
|
||||
|
||||
Protocol knowledge (register map, decoding rules) lives in the YAML profile
|
||||
referenced by ``profile``. Per-device variables (network address, slave ID,
|
||||
display name, poll rate) live here.
|
||||
"""
|
||||
|
||||
__tablename__ = "modbus_device"
|
||||
|
||||
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
||||
|
||||
# Stable internal identity — used as the API path key and HA Discovery unique_id anchor.
|
||||
uuid: Mapped[str] = mapped_column(
|
||||
String(36), unique=True, nullable=False, default=_uuid4_str
|
||||
)
|
||||
|
||||
# Human-readable label (may be changed; changing it triggers re-publish of HA Discovery).
|
||||
friendly_name: Mapped[str] = mapped_column(String(255), nullable=False)
|
||||
|
||||
# Transport protocol — only "tcp" is supported for now.
|
||||
transport: Mapped[str] = mapped_column(String(16), nullable=False, default="tcp")
|
||||
|
||||
# TCP gateway address.
|
||||
host: Mapped[str] = mapped_column(String(255), nullable=False)
|
||||
port: Mapped[int] = mapped_column(Integer, nullable=False, default=502)
|
||||
|
||||
# Modbus slave address (set on the device panel; different meters on the same gateway
|
||||
# must have distinct unit_ids).
|
||||
unit_id: Mapped[int] = mapped_column(Integer, nullable=False, default=1)
|
||||
|
||||
# Which YAML profile to use for register mapping and decoding (e.g. "sdm120").
|
||||
profile: Mapped[str] = mapped_column(String(64), nullable=False)
|
||||
|
||||
# Polling interval in seconds.
|
||||
poll_interval_s: Mapped[int] = mapped_column(Integer, nullable=False, default=5)
|
||||
|
||||
# Whether this device is included in the periodic poll sweep.
|
||||
enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True)
|
||||
|
||||
# Poll-status fields (updated by the poll service after each attempt).
|
||||
last_poll_at: Mapped[datetime | None] = mapped_column(
|
||||
DateTime(timezone=True), nullable=True
|
||||
)
|
||||
last_poll_ok: Mapped[bool | None] = mapped_column(Boolean, nullable=True)
|
||||
|
||||
# Audit timestamps.
|
||||
created_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True), nullable=False
|
||||
)
|
||||
updated_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True), nullable=False
|
||||
)
|
||||
|
||||
# Relationship to readings (back-reference; not loaded eagerly).
|
||||
readings: Mapped[list["ModbusReading"]] = relationship(
|
||||
back_populates="device", cascade="save-update, merge"
|
||||
)
|
||||
|
||||
|
||||
class ModbusReading(Base):
|
||||
"""Generic telemetry row: one device, one poll instant, all decoded metrics as JSON.
|
||||
|
||||
``payload`` contains the full dict of engineering values produced by the YAML profile
|
||||
decoder, e.g. ``{"voltage": 230.2, "current": 1.3, ...}``. The profile is the key
|
||||
to interpreting which keys exist and what units they carry.
|
||||
"""
|
||||
|
||||
__tablename__ = "modbus_reading"
|
||||
|
||||
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
||||
|
||||
# FK to the device that produced this reading.
|
||||
# ON DELETE RESTRICT: prevents accidental deletion of a device that has historical data.
|
||||
device_id: Mapped[int] = mapped_column(
|
||||
ForeignKey("modbus_device.id", ondelete="RESTRICT"), nullable=False
|
||||
)
|
||||
|
||||
# The UTC timestamp of when the sample was taken — real indexed column, NOT embedded in payload.
|
||||
recorded_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True), nullable=False, index=True
|
||||
)
|
||||
|
||||
# Profile-decoded engineering values as a JSON object.
|
||||
payload: Mapped[dict] = mapped_column(JSON, nullable=False)
|
||||
|
||||
device: Mapped["ModbusDevice"] = relationship(back_populates="readings")
|
||||
|
||||
|
||||
# Composite index for efficient time-range queries scoped to a single device.
|
||||
Index("ix_modbus_reading_device_recorded", ModbusReading.device_id, ModbusReading.recorded_at)
|
||||
@@ -38,3 +38,10 @@ class SmtpTestResponse(BaseModel):
|
||||
|
||||
result: Literal["success", "config-error", "failed"]
|
||||
message: str
|
||||
|
||||
|
||||
class MqttTestResponse(BaseModel):
|
||||
"""Response from POST /api/config/mqtt/test."""
|
||||
|
||||
result: Literal["success", "config-error", "failed"]
|
||||
message: str
|
||||
|
||||
@@ -0,0 +1,216 @@
|
||||
"""Pydantic schemas for the Energy data API (M6-T09).
|
||||
|
||||
Covers six endpoint groups under /api/energy:
|
||||
- GET /prices — price curve (tibber 15min points or manual tariff)
|
||||
- GET /costs — energy_cost_period rows (time-range, paginated)
|
||||
- GET /costs/summary — aggregated metered + standing charges − credits
|
||||
- GET /dsmr/latest — most recent dsmr_reading blob
|
||||
- POST /costs/recompute — explicit idempotent recompute (returns count)
|
||||
- POST /tibber/test — three-state Tibber connection test
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime
|
||||
from typing import Any, Literal
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# GET /api/energy/prices
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class PricePointSchema(BaseModel):
|
||||
"""A single 15-minute price point (tibber) or placeholder entry."""
|
||||
|
||||
starts_at: datetime
|
||||
buy: float = Field(description="All-in buy price in EUR/kWh (including taxes).")
|
||||
sell: float = Field(description="Net sell price in EUR/kWh.")
|
||||
level: str | None = Field(
|
||||
default=None,
|
||||
description="Tibber price level (CHEAP / NORMAL / EXPENSIVE); null for manual.",
|
||||
)
|
||||
|
||||
|
||||
class ManualTariffSchema(BaseModel):
|
||||
"""Fixed-tariff breakdown for manual contracts.
|
||||
|
||||
Prices are the *effective* buy prices as used by the billing engine
|
||||
(energy_buy_x + energy_tax + ode) and the raw sell prices.
|
||||
"""
|
||||
|
||||
buy_dal: float = Field(description="Effective buy price, low-tariff / dal (EUR/kWh).")
|
||||
buy_normal: float = Field(description="Effective buy price, normal / high-tariff (EUR/kWh).")
|
||||
sell_dal: float = Field(description="Sell price, low-tariff / dal (EUR/kWh).")
|
||||
sell_normal: float = Field(description="Sell price, normal / high-tariff (EUR/kWh).")
|
||||
|
||||
|
||||
class PricesResponse(BaseModel):
|
||||
"""Response for GET /api/energy/prices.
|
||||
|
||||
``kind`` mirrors the active contract kind:
|
||||
- ``"tibber"`` → ``points`` has actual 15-min price entries; ``tariff`` is null.
|
||||
- ``"manual"`` → ``points`` is empty; ``tariff`` carries the fixed-rate table.
|
||||
- ``None`` → no active contract; both ``points`` and ``tariff`` are empty/null.
|
||||
|
||||
``currency`` comes from the active contract (or "EUR" fallback).
|
||||
``points`` is always ascending by ``starts_at``.
|
||||
"""
|
||||
|
||||
kind: str | None = Field(
|
||||
description="Active contract kind ('tibber' or 'manual'), or null if no active contract."
|
||||
)
|
||||
currency: str = Field(description="ISO 4217 currency code.")
|
||||
points: list[PricePointSchema] = Field(
|
||||
description=(
|
||||
"15-minute price points for tibber contracts (ascending by starts_at). "
|
||||
"Empty for manual contracts or when no active contract exists."
|
||||
)
|
||||
)
|
||||
tariff: ManualTariffSchema | None = Field(
|
||||
default=None,
|
||||
description=(
|
||||
"Fixed tariff table for manual contracts. "
|
||||
"Null for tibber contracts and when no active contract exists."
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# GET /api/energy/costs
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class CostPeriodSchema(BaseModel):
|
||||
"""One 15-minute billing record from the energy_cost_period table."""
|
||||
|
||||
period_start: datetime
|
||||
d1_kwh: float = Field(description="Delivered low-tariff kWh for this period.")
|
||||
d2_kwh: float = Field(description="Delivered normal-tariff kWh for this period.")
|
||||
r1_kwh: float = Field(description="Returned low-tariff kWh for this period.")
|
||||
r2_kwh: float = Field(description="Returned normal-tariff kWh for this period.")
|
||||
import_cost: float = Field(description="Cost of electricity drawn from grid (EUR).")
|
||||
export_revenue: float = Field(description="Revenue from electricity fed to grid (EUR).")
|
||||
net_cost: float = Field(description="import_cost − export_revenue (EUR).")
|
||||
currency: str = Field(description="ISO 4217 currency code.")
|
||||
degraded: bool = Field(
|
||||
description="True when the period was computed with incomplete data."
|
||||
)
|
||||
contract_version_id: int | None = Field(
|
||||
default=None,
|
||||
description="FK to the contract version used for this billing period (null when degraded).",
|
||||
)
|
||||
|
||||
model_config = {"from_attributes": True}
|
||||
|
||||
|
||||
class CostsResponse(BaseModel):
|
||||
"""Response for GET /api/energy/costs."""
|
||||
|
||||
items: list[CostPeriodSchema]
|
||||
total: int = Field(description="Number of items returned.")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# GET /api/energy/costs/summary
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class SummaryResponse(BaseModel):
|
||||
"""Response for GET /api/energy/costs/summary.
|
||||
|
||||
All monetary values are in ``currency``.
|
||||
|
||||
``total_payable = metered_net + fixed_costs − credits``
|
||||
"""
|
||||
|
||||
currency: str
|
||||
metered_import: float = Field(description="Σ import_cost for non-degraded periods.")
|
||||
metered_export: float = Field(description="Σ export_revenue for non-degraded periods.")
|
||||
metered_net: float = Field(description="Σ net_cost for non-degraded periods.")
|
||||
fixed_costs: float = Field(
|
||||
description="Standing charges (network_fee + management_fee) apportioned over the interval."
|
||||
)
|
||||
credits: float = Field(
|
||||
description="Energy-tax credit (heffingskorting) apportioned over the interval."
|
||||
)
|
||||
total_payable: float = Field(
|
||||
description="metered_net + fixed_costs − credits (actual amount owed)."
|
||||
)
|
||||
period_count: int = Field(description="Number of non-degraded billing periods in range.")
|
||||
degraded_count: int = Field(description="Number of degraded billing periods in range.")
|
||||
days: float = Field(description="Interval length in days.")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# GET /api/energy/dsmr/latest
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class DsmrLatestResponse(BaseModel):
|
||||
"""Response for GET /api/energy/dsmr/latest.
|
||||
|
||||
``found`` is False when no dsmr_reading rows exist yet. The front-end
|
||||
should check ``found`` before reading ``recorded_at`` or ``payload``.
|
||||
"""
|
||||
|
||||
found: bool
|
||||
recorded_at: datetime | None = None
|
||||
payload: dict[str, Any] | None = None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# POST /api/energy/costs/recompute
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class RecomputeResponse(BaseModel):
|
||||
"""Response for POST /api/energy/costs/recompute."""
|
||||
|
||||
recomputed: int = Field(
|
||||
description=(
|
||||
"Number of 15-minute periods for which a billing record was written "
|
||||
"(inserted or updated). Periods skipped due to missing contract or "
|
||||
"missing Tibber price are not counted."
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# POST /api/energy/tibber/test
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TibberTestPriceSchema(BaseModel):
|
||||
"""Current Tibber price point returned on a successful test.
|
||||
|
||||
Carries enough fields for the front-end to confirm the API is working and
|
||||
display the live price. The API token is **never** included.
|
||||
"""
|
||||
|
||||
starts_at: datetime
|
||||
total: float
|
||||
energy: float
|
||||
tax: float
|
||||
currency: str
|
||||
level: str | None = None
|
||||
|
||||
|
||||
class TibberTestResponse(BaseModel):
|
||||
"""Three-state response for POST /api/energy/tibber/test.
|
||||
|
||||
Possible ``result`` values:
|
||||
|
||||
- ``"success"`` — Tibber API responded with a valid price.
|
||||
- ``"config-error"`` — Token is missing or not configured.
|
||||
- ``"failed"`` — API call failed (auth rejected, network error, timeout, etc.).
|
||||
|
||||
``price`` is populated only on ``"success"``; it is null otherwise.
|
||||
``message`` always contains a human-readable explanation.
|
||||
"""
|
||||
|
||||
result: Literal["success", "config-error", "failed"]
|
||||
message: str
|
||||
price: TibberTestPriceSchema | None = None
|
||||
@@ -0,0 +1,148 @@
|
||||
"""Pydantic schemas for the EnergyContract CRUD + versioning API (M6-T04).
|
||||
|
||||
Schema hierarchy
|
||||
----------------
|
||||
ContractVersionResponse — single version row (id, dates, values, created_at)
|
||||
ContractResponse — contract head (id, name, kind, active, currency, timestamps)
|
||||
ContractDetailResponse — contract head + embedded versions list (for GET{id})
|
||||
ContractListResponse — paginated list of ContractResponse items
|
||||
ContractCreate — POST /api/energy/contracts body
|
||||
ContractPatch — PATCH /api/energy/contracts/{id} body (all fields optional)
|
||||
VersionCreate — POST /api/energy/contracts/{id}/versions body
|
||||
ProfilesResponse — GET /api/energy/profiles response
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime
|
||||
from typing import Any
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Version schemas
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class ContractVersionResponse(BaseModel):
|
||||
"""Response schema for a single EnergyContractVersion row."""
|
||||
|
||||
id: int
|
||||
effective_from: datetime
|
||||
effective_to: datetime | None
|
||||
values: dict[str, Any]
|
||||
created_at: datetime
|
||||
|
||||
model_config = {"from_attributes": True}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Contract schemas
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class ContractResponse(BaseModel):
|
||||
"""Response schema for a single EnergyContract (without embedded versions).
|
||||
|
||||
Used for list responses where embedding all versions would be expensive.
|
||||
"""
|
||||
|
||||
id: int
|
||||
name: str
|
||||
kind: str
|
||||
active: bool
|
||||
currency: str
|
||||
created_at: datetime
|
||||
updated_at: datetime
|
||||
|
||||
model_config = {"from_attributes": True}
|
||||
|
||||
|
||||
class ContractDetailResponse(BaseModel):
|
||||
"""Response schema for a single EnergyContract with full version history.
|
||||
|
||||
Returned by GET /api/energy/contracts/{id} and by successful POST / PATCH
|
||||
operations where the caller needs to see all version data.
|
||||
"""
|
||||
|
||||
id: int
|
||||
name: str
|
||||
kind: str
|
||||
active: bool
|
||||
currency: str
|
||||
created_at: datetime
|
||||
updated_at: datetime
|
||||
versions: list[ContractVersionResponse]
|
||||
|
||||
model_config = {"from_attributes": True}
|
||||
|
||||
|
||||
class ContractListResponse(BaseModel):
|
||||
"""Response schema for GET /api/energy/contracts."""
|
||||
|
||||
items: list[ContractResponse]
|
||||
total: int
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Request schemas
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class ContractCreate(BaseModel):
|
||||
"""Request body for POST /api/energy/contracts.
|
||||
|
||||
``effective_from`` defaults to the current UTC time if not provided,
|
||||
giving the first version an open-ended start from "now".
|
||||
``kind`` is validated at the application layer against the profile registry;
|
||||
clients should send ``"manual"`` or ``"tibber"``.
|
||||
"""
|
||||
|
||||
name: str = Field(..., min_length=1, max_length=255)
|
||||
kind: str = Field(..., min_length=1, max_length=32)
|
||||
currency: str = Field(default="EUR", min_length=1, max_length=8)
|
||||
values: dict[str, Any]
|
||||
effective_from: datetime | None = Field(
|
||||
default=None,
|
||||
description=(
|
||||
"UTC datetime from which the first pricing version is effective. "
|
||||
"Defaults to the current UTC time when omitted."
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
class ContractPatch(BaseModel):
|
||||
"""Request body for PATCH /api/energy/contracts/{id}.
|
||||
|
||||
All fields are optional. Sending ``active=true`` activates this contract
|
||||
(deactivating all others); ``active=false`` deactivates it without affecting
|
||||
other contracts.
|
||||
"""
|
||||
|
||||
name: str | None = Field(default=None, min_length=1, max_length=255)
|
||||
active: bool | None = None
|
||||
|
||||
|
||||
class VersionCreate(BaseModel):
|
||||
"""Request body for POST /api/energy/contracts/{id}/versions."""
|
||||
|
||||
effective_from: datetime
|
||||
values: dict[str, Any]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Profile response schemas
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class ProfilesResponse(BaseModel):
|
||||
"""Response schema for GET /api/energy/profiles.
|
||||
|
||||
``profiles`` is a list of raw profile dicts as produced by
|
||||
``list_profiles()`` (Pydantic model dumps). Each entry contains at minimum
|
||||
``kind`` and ``label``; the full nested structure allows the front-end to
|
||||
render a type-appropriate form for each pricing profile.
|
||||
"""
|
||||
|
||||
profiles: list[dict[str, Any]]
|
||||
@@ -0,0 +1,95 @@
|
||||
"""Pydantic schemas for the Expose API (M5-T12).
|
||||
|
||||
Three endpoints:
|
||||
GET /api/expose — catalog + toggle state + MQTT/Discovery status
|
||||
PUT /api/expose — set toggles (map key → bool)
|
||||
POST /api/expose/republish — trigger discovery re-publish
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pydantic import BaseModel
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Nested schemas
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class DeviceInfoSchema(BaseModel):
|
||||
"""HA device grouping info for an exposable entity."""
|
||||
|
||||
identifiers: list[str]
|
||||
name: str
|
||||
|
||||
|
||||
class ExposableEntitySchema(BaseModel):
|
||||
"""One exposable entity in the catalog.
|
||||
|
||||
``value_getter`` is intentionally excluded — it is a non-serialisable
|
||||
callable and is only used internally by the HA Discovery service.
|
||||
"""
|
||||
|
||||
key: str
|
||||
component: str
|
||||
device: DeviceInfoSchema
|
||||
device_class: str | None
|
||||
unit: str
|
||||
name: str
|
||||
state_class: str | None = None
|
||||
|
||||
|
||||
class CatalogEntrySchema(BaseModel):
|
||||
"""An entity from the catalog with its current toggle state."""
|
||||
|
||||
entity: ExposableEntitySchema
|
||||
enabled: bool
|
||||
|
||||
|
||||
class MqttStatusSchema(BaseModel):
|
||||
"""Connection status for MQTT and HA Discovery."""
|
||||
|
||||
mqtt_configured: bool
|
||||
mqtt_connected: bool
|
||||
discovery_enabled: bool
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Response schemas
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class ExposeResponse(BaseModel):
|
||||
"""Response for GET /api/expose."""
|
||||
|
||||
catalog: list[CatalogEntrySchema]
|
||||
mqtt_status: MqttStatusSchema
|
||||
|
||||
|
||||
class ExposeUpdateResponse(BaseModel):
|
||||
"""Response for PUT /api/expose (returns updated catalog + status)."""
|
||||
|
||||
catalog: list[CatalogEntrySchema]
|
||||
mqtt_status: MqttStatusSchema
|
||||
|
||||
|
||||
class RepublishResponse(BaseModel):
|
||||
"""Response for POST /api/expose/republish."""
|
||||
|
||||
ok: bool
|
||||
message: str
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Request schemas
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class ExposeUpdateRequest(BaseModel):
|
||||
"""Request body for PUT /api/expose.
|
||||
|
||||
``toggles`` is a map from entity key to desired enabled state (bool).
|
||||
Only keys present in the map are updated; absent keys are untouched.
|
||||
"""
|
||||
|
||||
toggles: dict[str, bool]
|
||||
@@ -0,0 +1,131 @@
|
||||
"""Pydantic schemas for the Meter CRUD + swap declaration API (M7-T05).
|
||||
|
||||
Schema hierarchy
|
||||
----------------
|
||||
MeterResponse — single meter row (id/label/commodity/started_at/ended_at/reason/note/created_at)
|
||||
MeterListResponse — ordered list of MeterResponse items
|
||||
MeterDeclareRequest — POST /api/energy/meters body (declare a swap or initial meter)
|
||||
MeterPatchRequest — PATCH /api/energy/meters/{id} body (all fields optional)
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime
|
||||
from enum import Enum
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Enums
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class MeterReason(str, Enum):
|
||||
"""Allowed values for the meter epoch creation reason."""
|
||||
|
||||
initial = "initial"
|
||||
meter_swap = "meter_swap"
|
||||
home_move = "home_move"
|
||||
other = "other"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Response schemas
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class MeterResponse(BaseModel):
|
||||
"""Response schema for a single Meter epoch row.
|
||||
|
||||
``ended_at`` is ``null`` for the currently active meter.
|
||||
"""
|
||||
|
||||
id: int
|
||||
label: str
|
||||
commodity: str
|
||||
started_at: datetime
|
||||
ended_at: datetime | None
|
||||
reason: str
|
||||
note: str | None
|
||||
created_at: datetime
|
||||
|
||||
model_config = {"from_attributes": True}
|
||||
|
||||
|
||||
class MeterListResponse(BaseModel):
|
||||
"""Response schema for GET /api/energy/meters.
|
||||
|
||||
Meters are returned in ascending ``started_at`` order so the caller sees
|
||||
the historical installation sequence.
|
||||
"""
|
||||
|
||||
items: list[MeterResponse]
|
||||
total: int
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Request schemas
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_VALID_REASONS = ", ".join(r.value for r in MeterReason)
|
||||
|
||||
|
||||
class MeterDeclareRequest(BaseModel):
|
||||
"""Request body for POST /api/energy/meters.
|
||||
|
||||
Declares a new meter epoch (swap, home move, or initial declaration). The
|
||||
service layer closes the current active meter for the given commodity at
|
||||
``started_at`` and opens a new one.
|
||||
|
||||
``started_at`` follows the Principle-A localisation convention: a
|
||||
timezone-naive value is interpreted as the **server's local wall-clock time**
|
||||
(e.g. CEST midnight → stored as UTC the night before); a timezone-aware
|
||||
value is converted to UTC as-is. Omitting ``started_at`` is not allowed —
|
||||
every meter declaration must carry an explicit start timestamp.
|
||||
|
||||
``commodity`` defaults to ``"electricity"``; the field is available for
|
||||
future use with ``gas`` or ``heating``.
|
||||
"""
|
||||
|
||||
label: str = Field(..., min_length=1, max_length=255)
|
||||
started_at: datetime = Field(
|
||||
...,
|
||||
description=(
|
||||
"UTC (or server-local naive) datetime from which this meter epoch starts. "
|
||||
"May be in the past (retroactive declaration)."
|
||||
),
|
||||
)
|
||||
reason: MeterReason = Field(
|
||||
...,
|
||||
description=f"Why this epoch was created. One of: {_VALID_REASONS}.",
|
||||
)
|
||||
note: str | None = Field(default=None, max_length=1024)
|
||||
commodity: str = Field(
|
||||
default="electricity",
|
||||
min_length=1,
|
||||
max_length=32,
|
||||
description="Energy commodity this meter measures. Defaults to 'electricity'.",
|
||||
)
|
||||
|
||||
|
||||
class MeterPatchRequest(BaseModel):
|
||||
"""Request body for PATCH /api/energy/meters/{id}.
|
||||
|
||||
All fields are optional. Only non-``None`` values are applied.
|
||||
|
||||
Updating ``started_at`` is a **retroactive correction**: the service layer
|
||||
maintains timeline continuity (adjusting the preceding meter's ``ended_at``)
|
||||
and the API layer triggers ``recompute_range`` over the affected window so
|
||||
that billing attribution is re-judged.
|
||||
"""
|
||||
|
||||
label: str | None = Field(default=None, min_length=1, max_length=255)
|
||||
note: str | None = Field(default=None, max_length=1024)
|
||||
started_at: datetime | None = Field(
|
||||
default=None,
|
||||
description=(
|
||||
"Retroactive correction of the meter epoch start timestamp. "
|
||||
"Triggers billing recompute over the affected window."
|
||||
),
|
||||
)
|
||||
@@ -0,0 +1,171 @@
|
||||
"""Pydantic schemas for the Modbus device CRUD + readings + metrics API (M5-T05)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime
|
||||
from typing import Any
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Device schemas
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class ModbusDeviceCreate(BaseModel):
|
||||
"""Request body for POST /api/modbus/devices."""
|
||||
|
||||
friendly_name: str = Field(..., min_length=1, max_length=255)
|
||||
transport: str = Field(default="tcp", max_length=16)
|
||||
host: str = Field(..., min_length=1, max_length=255)
|
||||
port: int = Field(default=502, ge=1, le=65535)
|
||||
unit_id: int = Field(default=1, ge=0, le=247)
|
||||
profile: str = Field(..., min_length=1, max_length=64)
|
||||
poll_interval_s: int = Field(default=5, ge=1, le=3600)
|
||||
enabled: bool = Field(default=True)
|
||||
|
||||
|
||||
class ModbusDeviceUpdate(BaseModel):
|
||||
"""Request body for PATCH /api/modbus/devices/{uuid} — all fields optional."""
|
||||
|
||||
friendly_name: str | None = Field(default=None, min_length=1, max_length=255)
|
||||
transport: str | None = Field(default=None, max_length=16)
|
||||
host: str | None = Field(default=None, min_length=1, max_length=255)
|
||||
port: int | None = Field(default=None, ge=1, le=65535)
|
||||
unit_id: int | None = Field(default=None, ge=0, le=247)
|
||||
profile: str | None = Field(default=None, min_length=1, max_length=64)
|
||||
poll_interval_s: int | None = Field(default=None, ge=1, le=3600)
|
||||
enabled: bool | None = None
|
||||
|
||||
|
||||
class ModbusDeviceResponse(BaseModel):
|
||||
"""Response schema for a single Modbus device."""
|
||||
|
||||
uuid: str
|
||||
friendly_name: str
|
||||
transport: str
|
||||
host: str
|
||||
port: int
|
||||
unit_id: int
|
||||
profile: str
|
||||
poll_interval_s: int
|
||||
enabled: bool
|
||||
last_poll_at: datetime | None
|
||||
last_poll_ok: bool | None
|
||||
created_at: datetime
|
||||
updated_at: datetime
|
||||
|
||||
model_config = {"from_attributes": True}
|
||||
|
||||
|
||||
class ModbusDeviceListResponse(BaseModel):
|
||||
"""Response schema for listing Modbus devices."""
|
||||
|
||||
items: list[ModbusDeviceResponse]
|
||||
total: int
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Reading schemas
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class ModbusReadingResponse(BaseModel):
|
||||
"""A single reading row: timestamp + decoded payload."""
|
||||
|
||||
recorded_at: datetime
|
||||
payload: dict[str, Any]
|
||||
|
||||
model_config = {"from_attributes": True}
|
||||
|
||||
|
||||
class ModbusReadingsResponse(BaseModel):
|
||||
"""Response schema for the readings time-range endpoint."""
|
||||
|
||||
items: list[ModbusReadingResponse]
|
||||
|
||||
|
||||
class ModbusLatestResponse(BaseModel):
|
||||
"""Response for the /latest endpoint.
|
||||
|
||||
``found`` is False and ``recorded_at``/``payload`` are None when the device
|
||||
has no readings yet. Callers should check ``found`` before using the values.
|
||||
"""
|
||||
|
||||
found: bool
|
||||
recorded_at: datetime | None
|
||||
payload: dict[str, Any] | None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Metrics / profile schemas
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class MetricInfo(BaseModel):
|
||||
"""Metadata for a single measurable quantity in a device's profile."""
|
||||
|
||||
key: str
|
||||
label: str
|
||||
unit: str
|
||||
device_class: str
|
||||
|
||||
|
||||
class ModbusMetricsResponse(BaseModel):
|
||||
"""Response schema for GET /api/modbus/devices/{uuid}/metrics."""
|
||||
|
||||
profile: str
|
||||
metrics: list[MetricInfo]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Profile list schema
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class ProfileSummary(BaseModel):
|
||||
"""One entry in the GET /api/modbus/profiles response."""
|
||||
|
||||
name: str
|
||||
description: str
|
||||
|
||||
|
||||
class ModbusProfilesResponse(BaseModel):
|
||||
"""Response schema for GET /api/modbus/profiles."""
|
||||
|
||||
profiles: list[ProfileSummary]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Test-read schema
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class ModbusTestReadResponse(BaseModel):
|
||||
"""Response for POST /api/modbus/devices/{uuid}/test.
|
||||
|
||||
On success ``ok=True`` and ``payload`` contains the decoded values.
|
||||
On failure ``ok=False`` and ``error`` describes the problem.
|
||||
"""
|
||||
|
||||
ok: bool
|
||||
payload: dict[str, Any] | None = None
|
||||
error: str | None = None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Cascade-delete schema
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class ModbusDeleteResponse(BaseModel):
|
||||
"""Response for DELETE /api/modbus/devices/{uuid}?cascade=true.
|
||||
|
||||
Returned only when cascade deletion succeeds (HTTP 200). Non-cascade
|
||||
successful deletes continue to return HTTP 204 (no body).
|
||||
"""
|
||||
|
||||
deleted: bool
|
||||
readings_deleted: int
|
||||
toggles_deleted: int
|
||||
@@ -16,6 +16,7 @@ class SessionResponse(BaseModel):
|
||||
class LoginRequest(BaseModel):
|
||||
username: str
|
||||
password: str
|
||||
totp_code: str | None = None
|
||||
|
||||
|
||||
class PasswordChangeRequest(BaseModel):
|
||||
|
||||
@@ -0,0 +1,59 @@
|
||||
"""Pydantic schemas for TOTP setup / enable / disable / status endpoints (M4-T05)."""
|
||||
from __future__ import annotations
|
||||
|
||||
from pydantic import BaseModel
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Setup (POST /api/auth/totp/setup) — response
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TotpSetupResponse(BaseModel):
|
||||
"""Returned once after a setup call.
|
||||
|
||||
``secret`` and ``recovery_codes`` are **one-time plaintext values**.
|
||||
They are never returned again by any subsequent API call.
|
||||
The frontend must display and instruct the user to save them before confirming.
|
||||
"""
|
||||
|
||||
secret: str
|
||||
otpauth_uri: str
|
||||
recovery_codes: list[str]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Enable (POST /api/auth/totp/enable) — request
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TotpEnableRequest(BaseModel):
|
||||
"""The user confirms setup by providing the 6-digit TOTP code."""
|
||||
|
||||
code: str
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Disable (POST /api/auth/totp/disable) — request
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TotpDisableRequest(BaseModel):
|
||||
"""Disable TOTP by proving identity.
|
||||
|
||||
Exactly one of ``password`` or ``code`` must be provided.
|
||||
"""
|
||||
|
||||
password: str | None = None
|
||||
code: str | None = None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Status (GET /api/auth/totp) — response
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TotpStatusResponse(BaseModel):
|
||||
"""Minimal status response — never exposes secret or recovery codes."""
|
||||
|
||||
enabled: bool
|
||||
@@ -25,9 +25,9 @@ class ConfigField:
|
||||
CONFIG_FIELDS: tuple[ConfigField, ...] = (
|
||||
ConfigField("System", "APP_NAME", "app_name", "App Name"),
|
||||
ConfigField("System", "APP_ENV", "app_env", "App Env"),
|
||||
ConfigField("System", "APP_DEBUG", "app_debug", "App Debug"),
|
||||
ConfigField("System", "APP_DEBUG", "app_debug", "App Debug", input_type="checkbox"),
|
||||
ConfigField("System", "APP_HOSTNAME", "app_hostname", "App Hostname"),
|
||||
ConfigField("SMTP", "SMTP_ENABLED", "smtp_enabled", "SMTP Enabled"),
|
||||
ConfigField("SMTP", "SMTP_ENABLED", "smtp_enabled", "SMTP Enabled", input_type="checkbox"),
|
||||
ConfigField("SMTP", "SMTP_HOST", "smtp_host", "SMTP Host"),
|
||||
ConfigField("SMTP", "SMTP_PORT", "smtp_port", "SMTP Port"),
|
||||
ConfigField("SMTP", "SMTP_USERNAME", "smtp_username", "SMTP Username"),
|
||||
@@ -35,7 +35,7 @@ CONFIG_FIELDS: tuple[ConfigField, ...] = (
|
||||
ConfigField("SMTP", "SMTP_FROM_NAME", "smtp_from_name", "SMTP From Name"),
|
||||
ConfigField("SMTP", "SMTP_FROM_ADDRESS", "smtp_from_address", "SMTP From Address"),
|
||||
ConfigField("SMTP", "SMTP_TO_ADDRESS", "smtp_to_address", "SMTP To Address"),
|
||||
ConfigField("SMTP", "SMTP_USE_STARTTLS", "smtp_use_starttls", "SMTP Use STARTTLS"),
|
||||
ConfigField("SMTP", "SMTP_USE_STARTTLS", "smtp_use_starttls", "SMTP Use STARTTLS", input_type="checkbox"),
|
||||
ConfigField(
|
||||
"Authentication",
|
||||
"AUTH_SESSION_COOKIE_NAME",
|
||||
@@ -49,6 +49,13 @@ CONFIG_FIELDS: tuple[ConfigField, ...] = (
|
||||
"auth_cookie_secure_override",
|
||||
"Cookie Secure Override",
|
||||
),
|
||||
ConfigField(
|
||||
"Authentication",
|
||||
"AUTH_LOGIN_THROTTLE_ENABLED",
|
||||
"auth_login_throttle_enabled",
|
||||
"Login Throttle Enabled",
|
||||
input_type="checkbox",
|
||||
),
|
||||
ConfigField("Poo", "POO_WEBHOOK_ID", "poo_webhook_id", "Poo Webhook ID", secret=True),
|
||||
ConfigField(
|
||||
"Poo",
|
||||
@@ -96,6 +103,61 @@ CONFIG_FIELDS: tuple[ConfigField, ...] = (
|
||||
"home_assistant_action_task_project_id",
|
||||
"Home Assistant Action Task Project ID",
|
||||
),
|
||||
ConfigField("MQTT", "MQTT_ENABLED", "mqtt_enabled", "MQTT Enabled", input_type="checkbox"),
|
||||
ConfigField("MQTT", "MQTT_BROKER_HOST", "mqtt_broker_host", "MQTT Broker Host"),
|
||||
ConfigField("MQTT", "MQTT_BROKER_PORT", "mqtt_broker_port", "MQTT Broker Port", input_type="number"),
|
||||
ConfigField("MQTT", "MQTT_USERNAME", "mqtt_username", "MQTT Username"),
|
||||
ConfigField("MQTT", "MQTT_PASSWORD", "mqtt_password", "MQTT Password", secret=True),
|
||||
ConfigField("MQTT", "MQTT_TLS_ENABLED", "mqtt_tls_enabled", "MQTT TLS Enabled", input_type="checkbox"),
|
||||
ConfigField(
|
||||
"Home Assistant Discovery",
|
||||
"HA_DISCOVERY_ENABLED",
|
||||
"ha_discovery_enabled",
|
||||
"HA Discovery Enabled",
|
||||
input_type="checkbox",
|
||||
),
|
||||
ConfigField(
|
||||
"Home Assistant Discovery",
|
||||
"HA_DISCOVERY_PREFIX",
|
||||
"ha_discovery_prefix",
|
||||
"HA Discovery Prefix",
|
||||
),
|
||||
ConfigField(
|
||||
"Home Assistant Discovery",
|
||||
"HA_STATE_TOPIC_PREFIX",
|
||||
"ha_state_topic_prefix",
|
||||
"HA State Topic Prefix",
|
||||
),
|
||||
ConfigField("Modbus", "MODBUS_POLLING_ENABLED", "modbus_polling_enabled", "Modbus Polling Enabled", input_type="checkbox"),
|
||||
ConfigField(
|
||||
"DSMR",
|
||||
"DSMR_INGEST_ENABLED",
|
||||
"dsmr_ingest_enabled",
|
||||
"DSMR Ingest Enabled",
|
||||
input_type="checkbox",
|
||||
),
|
||||
ConfigField("DSMR", "DSMR_MQTT_TOPIC", "dsmr_mqtt_topic", "DSMR MQTT Topic"),
|
||||
ConfigField(
|
||||
"DSMR",
|
||||
"DSMR_SAMPLE_INTERVAL_S",
|
||||
"dsmr_sample_interval_s",
|
||||
"DSMR Sample Interval (s)",
|
||||
input_type="number",
|
||||
),
|
||||
ConfigField(
|
||||
"DSMR",
|
||||
"DSMR_TARIFF_TOPIC",
|
||||
"dsmr_tariff_topic",
|
||||
"DSMR Tariff Topic",
|
||||
),
|
||||
ConfigField(
|
||||
"Tibber",
|
||||
"TIBBER_API_TOKEN",
|
||||
"tibber_api_token",
|
||||
"Tibber API Token",
|
||||
secret=True,
|
||||
),
|
||||
ConfigField("Tibber", "TIBBER_HOME_ID", "tibber_home_id", "Tibber Home ID"),
|
||||
)
|
||||
|
||||
|
||||
@@ -284,4 +346,22 @@ def _settings_payload(settings: Settings) -> dict[str, Any]:
|
||||
"auth_session_cookie_name": settings.auth_session_cookie_name,
|
||||
"auth_session_ttl_hours": settings.auth_session_ttl_hours,
|
||||
"auth_cookie_secure_override": settings.auth_cookie_secure_override,
|
||||
"auth_login_throttle_enabled": settings.auth_login_throttle_enabled,
|
||||
"auth_trust_forwarded_for": settings.auth_trust_forwarded_for,
|
||||
"modbus_polling_enabled": settings.modbus_polling_enabled,
|
||||
"mqtt_enabled": settings.mqtt_enabled,
|
||||
"mqtt_broker_host": settings.mqtt_broker_host,
|
||||
"mqtt_broker_port": settings.mqtt_broker_port,
|
||||
"mqtt_username": settings.mqtt_username,
|
||||
"mqtt_password": settings.mqtt_password,
|
||||
"mqtt_tls_enabled": settings.mqtt_tls_enabled,
|
||||
"ha_discovery_enabled": settings.ha_discovery_enabled,
|
||||
"ha_discovery_prefix": settings.ha_discovery_prefix,
|
||||
"ha_state_topic_prefix": settings.ha_state_topic_prefix,
|
||||
"dsmr_ingest_enabled": settings.dsmr_ingest_enabled,
|
||||
"dsmr_mqtt_topic": settings.dsmr_mqtt_topic,
|
||||
"dsmr_sample_interval_s": settings.dsmr_sample_interval_s,
|
||||
"dsmr_tariff_topic": settings.dsmr_tariff_topic,
|
||||
"tibber_api_token": settings.tibber_api_token,
|
||||
"tibber_home_id": settings.tibber_home_id,
|
||||
}
|
||||
|
||||
@@ -0,0 +1,364 @@
|
||||
"""Service layer for EnergyContract CRUD, versioning, and activation.
|
||||
|
||||
All functions accept an explicit SQLAlchemy Session; callers are responsible
|
||||
for committing or rolling back the transaction.
|
||||
|
||||
Design decisions
|
||||
----------------
|
||||
- ``create_contract``: validates values against the pricing profile *before*
|
||||
writing any rows; raises ``ProfileValidationError`` on non-compliance.
|
||||
- ``add_version``: append-only; closes the previous open version by
|
||||
setting its ``effective_to`` to the new version's ``effective_from``; raises
|
||||
``ContractVersionError`` if the new date is strictly earlier than the previous
|
||||
version's ``effective_from``.
|
||||
- ``activate_contract``: mutual-exclusion; sets all other contracts' ``active``
|
||||
to False, then sets the given contract's ``active`` to True.
|
||||
- ``active_contract_version_at``: returns the single version of the currently
|
||||
active contract that covers *ts* (``effective_from ≤ ts < effective_to``,
|
||||
or open-ended when ``effective_to`` is None).
|
||||
|
||||
SQLite timezone note
|
||||
--------------------
|
||||
SQLite stores ``DateTime(timezone=True)`` columns as naive UTC strings; on
|
||||
read-back they come out as **timezone-naive** datetimes. Wherever this code
|
||||
compares timestamps from the DB against timezone-aware values (e.g. from
|
||||
Pydantic or ``datetime.now(UTC)``), it calls ``_as_utc()`` to make both sides
|
||||
comparable without tripping on "offset-naive vs offset-aware" TypeErrors.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from datetime import UTC, datetime
|
||||
from typing import Any
|
||||
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.integrations.pricing.profiles import validate_values
|
||||
from app.models.energy import EnergyContract, EnergyContractVersion
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Internal helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _as_utc(dt: datetime) -> datetime:
|
||||
"""Return *dt* as a timezone-aware UTC datetime.
|
||||
|
||||
SQLite's DateTime(timezone=True) column type stores datetimes as naive UTC
|
||||
strings and gives them back as naive datetimes on read. This helper
|
||||
re-attaches the UTC timezone info when it is missing, making cross-origin
|
||||
comparisons safe.
|
||||
"""
|
||||
if dt.tzinfo is None:
|
||||
return dt.replace(tzinfo=UTC)
|
||||
return dt
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Custom exception
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class ContractVersionError(ValueError):
|
||||
"""Raised when a new contract version has an invalid effective date.
|
||||
|
||||
Specifically: the new version's ``effective_from`` must be greater than or
|
||||
equal to the previous open version's ``effective_from``. Allowing equal
|
||||
timestamps would cause ambiguous overlap; the service therefore also rejects
|
||||
strictly-equal values (same second) to avoid silent data loss.
|
||||
"""
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def get_contract_or_none(session: Session, contract_id: int) -> EnergyContract | None:
|
||||
"""Return the contract with the given id, or None if not found."""
|
||||
return session.execute(
|
||||
select(EnergyContract).where(EnergyContract.id == contract_id)
|
||||
).scalar_one_or_none()
|
||||
|
||||
|
||||
def list_contracts(session: Session) -> list[EnergyContract]:
|
||||
"""Return all contracts ordered by id (ascending)."""
|
||||
return list(
|
||||
session.execute(select(EnergyContract).order_by(EnergyContract.id)).scalars().all()
|
||||
)
|
||||
|
||||
|
||||
def _open_version(session: Session, contract: EnergyContract) -> EnergyContractVersion | None:
|
||||
"""Return the current open version (effective_to IS NULL) for *contract*, or None."""
|
||||
return session.execute(
|
||||
select(EnergyContractVersion)
|
||||
.where(
|
||||
EnergyContractVersion.contract_id == contract.id,
|
||||
EnergyContractVersion.effective_to.is_(None),
|
||||
)
|
||||
.order_by(EnergyContractVersion.effective_from.desc())
|
||||
.limit(1)
|
||||
).scalar_one_or_none()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Core service functions
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def create_contract(
|
||||
session: Session,
|
||||
*,
|
||||
name: str,
|
||||
kind: str,
|
||||
currency: str = "EUR",
|
||||
values: dict[str, Any],
|
||||
effective_from: datetime,
|
||||
) -> EnergyContract:
|
||||
"""Create a new energy contract with its first pricing version.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
session:
|
||||
Active SQLAlchemy session. Caller must commit after this returns.
|
||||
name:
|
||||
Human-readable label for the contract.
|
||||
kind:
|
||||
Pricing strategy identifier (``"manual"`` or ``"tibber"``).
|
||||
currency:
|
||||
ISO 4217 currency code (default ``"EUR"``).
|
||||
values:
|
||||
Pricing values dict conforming to the named profile's structure.
|
||||
Validated via ``validate_values(kind, values)`` before any writes.
|
||||
effective_from:
|
||||
UTC datetime at which the first pricing version takes effect.
|
||||
|
||||
Returns
|
||||
-------
|
||||
EnergyContract
|
||||
The newly created contract (not yet committed).
|
||||
|
||||
Raises
|
||||
------
|
||||
ProfileNotFoundError
|
||||
If no YAML profile exists for *kind*.
|
||||
ProfileValidationError
|
||||
If *values* does not conform to the profile structure.
|
||||
"""
|
||||
# Validate (and fill defaults) before any DB write.
|
||||
filled_values = validate_values(kind, values)
|
||||
|
||||
now = datetime.now(UTC)
|
||||
contract = EnergyContract(
|
||||
name=name,
|
||||
kind=kind,
|
||||
currency=currency,
|
||||
active=False, # New contracts are inactive; caller must explicitly activate.
|
||||
created_at=now,
|
||||
updated_at=now,
|
||||
)
|
||||
session.add(contract)
|
||||
session.flush() # Assign contract.id so we can reference it in the version FK.
|
||||
|
||||
version = EnergyContractVersion(
|
||||
contract_id=contract.id,
|
||||
effective_from=effective_from,
|
||||
effective_to=None,
|
||||
values=filled_values,
|
||||
created_at=now,
|
||||
)
|
||||
session.add(version)
|
||||
|
||||
logger.info("Created energy contract %r (kind=%s, id=%d)", name, kind, contract.id)
|
||||
return contract
|
||||
|
||||
|
||||
def add_version(
|
||||
session: Session,
|
||||
contract: EnergyContract,
|
||||
*,
|
||||
effective_from: datetime,
|
||||
values: dict[str, Any],
|
||||
) -> EnergyContractVersion:
|
||||
"""Add a new pricing version to an existing contract (append-only).
|
||||
|
||||
The previous open version's ``effective_to`` is automatically set to the
|
||||
new version's ``effective_from`` (version closure), ensuring there is never
|
||||
a gap or overlap between consecutive versions.
|
||||
|
||||
The new ``effective_from`` **must be strictly greater than** the previous
|
||||
open version's ``effective_from``. Equal timestamps are rejected because
|
||||
they would produce two versions starting at the same instant, making it
|
||||
impossible to determine which is current. If this constraint is not met,
|
||||
``ContractVersionError`` is raised and **no rows are written**.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
session:
|
||||
Active SQLAlchemy session. Caller must commit after this returns.
|
||||
contract:
|
||||
The parent ``EnergyContract`` to append a version to.
|
||||
effective_from:
|
||||
UTC datetime at which this version's pricing takes effect.
|
||||
values:
|
||||
New pricing values dict conforming to the contract's profile.
|
||||
|
||||
Returns
|
||||
-------
|
||||
EnergyContractVersion
|
||||
The newly created version (not yet committed).
|
||||
|
||||
Raises
|
||||
------
|
||||
ContractVersionError
|
||||
If *effective_from* is not strictly after the previous open version's
|
||||
``effective_from``.
|
||||
ProfileNotFoundError
|
||||
If no YAML profile exists for the contract's kind.
|
||||
ProfileValidationError
|
||||
If *values* does not conform to the profile structure.
|
||||
"""
|
||||
# Validate (and fill defaults) before any DB write.
|
||||
filled_values = validate_values(contract.kind, values)
|
||||
|
||||
prev = _open_version(session, contract)
|
||||
if prev is not None:
|
||||
# Enforce strictly-after constraint to prevent ambiguous overlap.
|
||||
# Use _as_utc() on both sides: the incoming value may be tz-aware
|
||||
# while the DB-read value is tz-naive (SQLite limitation).
|
||||
if _as_utc(effective_from) <= _as_utc(prev.effective_from):
|
||||
raise ContractVersionError(
|
||||
f"New version effective_from ({effective_from.isoformat()}) must be strictly "
|
||||
f"after the previous open version's effective_from "
|
||||
f"({prev.effective_from.isoformat()})."
|
||||
)
|
||||
# Close the previous open version.
|
||||
prev.effective_to = effective_from
|
||||
|
||||
now = datetime.now(UTC)
|
||||
new_version = EnergyContractVersion(
|
||||
contract_id=contract.id,
|
||||
effective_from=effective_from,
|
||||
effective_to=None,
|
||||
values=filled_values,
|
||||
created_at=now,
|
||||
)
|
||||
session.add(new_version)
|
||||
|
||||
logger.info(
|
||||
"Added version to contract id=%d (kind=%s, effective_from=%s)",
|
||||
contract.id,
|
||||
contract.kind,
|
||||
effective_from.isoformat(),
|
||||
)
|
||||
return new_version
|
||||
|
||||
|
||||
def activate_contract(session: Session, contract: EnergyContract) -> None:
|
||||
"""Activate a contract with mutual exclusion.
|
||||
|
||||
Sets every other contract's ``active`` flag to False, then sets the given
|
||||
contract's ``active`` to True. This guarantees at most one active contract
|
||||
at any time.
|
||||
|
||||
Caller must commit after this returns.
|
||||
"""
|
||||
# Deactivate all contracts (including the target; we re-activate below).
|
||||
for other in session.execute(select(EnergyContract)).scalars().all():
|
||||
other.active = False
|
||||
contract.active = True
|
||||
contract.updated_at = datetime.now(UTC)
|
||||
logger.info("Activated contract %r (id=%d)", contract.name, contract.id)
|
||||
|
||||
|
||||
def deactivate_contract(session: Session, contract: EnergyContract) -> None:
|
||||
"""Deactivate a contract without touching other contracts.
|
||||
|
||||
Caller must commit after this returns.
|
||||
"""
|
||||
contract.active = False
|
||||
contract.updated_at = datetime.now(UTC)
|
||||
logger.info("Deactivated contract %r (id=%d)", contract.name, contract.id)
|
||||
|
||||
|
||||
def active_contract_versions(session: Session) -> list[EnergyContractVersion]:
|
||||
"""Return all versions of the currently active contract, ordered by effective_from ascending.
|
||||
|
||||
Returns an empty list when there is no active contract. The list spans the
|
||||
full history of the active contract (all closed + the current open version)
|
||||
and is used to iterate over pricing-rate segments for cross-version fixed-
|
||||
cost / credit accumulation (Principle C).
|
||||
"""
|
||||
active = session.execute(
|
||||
select(EnergyContract).where(EnergyContract.active.is_(True)).limit(1)
|
||||
).scalar_one_or_none()
|
||||
|
||||
if active is None:
|
||||
return []
|
||||
|
||||
return list(
|
||||
session.execute(
|
||||
select(EnergyContractVersion)
|
||||
.where(EnergyContractVersion.contract_id == active.id)
|
||||
.order_by(EnergyContractVersion.effective_from)
|
||||
)
|
||||
.scalars()
|
||||
.all()
|
||||
)
|
||||
|
||||
|
||||
def active_contract_version_at(
|
||||
session: Session, ts: datetime
|
||||
) -> EnergyContractVersion | None:
|
||||
"""Return the active contract's version that covers *ts*.
|
||||
|
||||
A version covers *ts* when:
|
||||
``effective_from ≤ ts`` AND (``effective_to IS NULL`` OR ``ts < effective_to``)
|
||||
|
||||
Returns None when:
|
||||
- There is no active contract.
|
||||
- The active contract has no version covering *ts*.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
session:
|
||||
Active SQLAlchemy session (read-only usage).
|
||||
ts:
|
||||
UTC datetime to look up.
|
||||
|
||||
Returns
|
||||
-------
|
||||
EnergyContractVersion | None
|
||||
"""
|
||||
active = session.execute(
|
||||
select(EnergyContract).where(EnergyContract.active.is_(True)).limit(1)
|
||||
).scalar_one_or_none()
|
||||
|
||||
if active is None:
|
||||
return None
|
||||
|
||||
# Build a query for all versions of the active contract covering ts.
|
||||
stmt = (
|
||||
select(EnergyContractVersion)
|
||||
.where(
|
||||
EnergyContractVersion.contract_id == active.id,
|
||||
EnergyContractVersion.effective_from <= ts,
|
||||
)
|
||||
.order_by(EnergyContractVersion.effective_from.desc())
|
||||
.limit(1)
|
||||
)
|
||||
version = session.execute(stmt).scalar_one_or_none()
|
||||
|
||||
if version is None:
|
||||
return None
|
||||
|
||||
# Exclude versions whose effective window has already closed before ts.
|
||||
if version.effective_to is not None and _as_utc(ts) >= _as_utc(version.effective_to):
|
||||
return None
|
||||
|
||||
return version
|
||||
@@ -0,0 +1,297 @@
|
||||
"""DSMR telegram ingest service.
|
||||
|
||||
Subscribes to the DSMR Reader MQTT topic (``dsmr/json``) and persists
|
||||
down-sampled DSMR telegram frames to the ``dsmr_reading`` table.
|
||||
|
||||
Design decisions
|
||||
----------------
|
||||
- **Whole-frame storage**: the entire parsed telegram dict is stored as a JSON
|
||||
blob in ``DsmrReading.payload``; no field allow-list is applied. This lets
|
||||
future commodities (gas, heating, three-phase) be accommodated without a
|
||||
table-schema change.
|
||||
- **10-second down-sampling** (configurable via ``dsmr_sample_interval_s``):
|
||||
only telegrams whose ``timestamp`` second falls on an exact multiple of the
|
||||
interval are persisted. This reduces write volume from ~60 rows/min to ~6
|
||||
rows/min while guaranteeing that every 15-minute boundary (second=00) is
|
||||
captured.
|
||||
- **Idempotency**: the telegram's own ``id`` field is stored as ``source_id``
|
||||
with a UNIQUE constraint. A second delivery of the same telegram (e.g. after
|
||||
a broker reconnect) is silently skipped.
|
||||
- **Network-thread safety**: ``handle_message`` is called from paho's background
|
||||
loop thread. It opens and closes its own short-lived SQLAlchemy session and
|
||||
swallows all exceptions so that a buggy payload or transient DB error never
|
||||
crashes the paho loop or drops the MQTT connection.
|
||||
- **Numeric values kept as strings**: the DSMR Reader emits all numeric readings
|
||||
as JSON strings (e.g. ``"20915.154"``). They are stored verbatim; conversion
|
||||
to ``Decimal`` is deferred to the billing engine (T07) where precision matters.
|
||||
- **Null phases**: some telegrams omit certain phase readings (``null`` in JSON);
|
||||
these are stored as-is without special handling.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import threading
|
||||
from datetime import datetime, timezone
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
import sqlalchemy.exc
|
||||
from sqlalchemy import select
|
||||
|
||||
from app.db import get_session_local
|
||||
from app.models.energy import DsmrReading
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from app.config import Settings
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
# Tracks the DSMR topic currently subscribed via the MQTT manager, so a config
|
||||
# change can unsubscribe the old topic before subscribing the new one.
|
||||
_current_dsmr_topic: str | None = None
|
||||
|
||||
# Tracks the DSMR tariff topic currently subscribed via the MQTT manager.
|
||||
_current_tariff_topic: str | None = None
|
||||
|
||||
# Current electricity tariff: 1 = dal/off-peak, 2 = normal/peak.
|
||||
# Written by the paho network thread, read by the publish job — guarded by a lock.
|
||||
_current_tariff: int | None = None
|
||||
_tariff_lock = threading.Lock()
|
||||
|
||||
|
||||
def get_current_tariff() -> int | None:
|
||||
"""Return the most recently received electricity tariff (1 or 2), or None."""
|
||||
with _tariff_lock:
|
||||
return _current_tariff
|
||||
|
||||
|
||||
def set_current_tariff(value: int | None) -> None:
|
||||
"""Set the current electricity tariff (1 or 2), or clear it with None."""
|
||||
with _tariff_lock:
|
||||
global _current_tariff
|
||||
_current_tariff = value
|
||||
|
||||
|
||||
def handle_tariff_message(payload_bytes: bytes) -> None:
|
||||
"""Parse one DSMR tariff MQTT payload and update the in-memory tariff state.
|
||||
|
||||
Called from the paho network thread; *must* swallow all exceptions so that
|
||||
a bad payload never crashes the loop or drops the broker connection.
|
||||
|
||||
Accepts payload as bytes or str (paho can deliver either). Ignores
|
||||
whitespace. Only accepts integer values 1 or 2; anything else is discarded
|
||||
and the previous known tariff is preserved.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
payload_bytes:
|
||||
Raw bytes from the MQTT message (may also be a str in some paho versions).
|
||||
"""
|
||||
try:
|
||||
# Decode bytes → str if needed; strip surrounding whitespace.
|
||||
if isinstance(payload_bytes, (bytes, bytearray)):
|
||||
raw = payload_bytes.decode("utf-8", errors="replace").strip()
|
||||
else:
|
||||
raw = str(payload_bytes).strip()
|
||||
|
||||
value = int(raw)
|
||||
if value not in (1, 2):
|
||||
logger.debug(
|
||||
"dsmr_ingest.handle_tariff_message: unexpected tariff value %d (expected 1 or 2, ignored).",
|
||||
value,
|
||||
)
|
||||
return
|
||||
|
||||
set_current_tariff(value)
|
||||
logger.debug("dsmr_ingest.handle_tariff_message: tariff updated to %d.", value)
|
||||
|
||||
except Exception:
|
||||
# Malformed payload (e.g. non-numeric); swallow silently to protect network thread.
|
||||
logger.debug(
|
||||
"dsmr_ingest.handle_tariff_message: could not parse payload %r (ignored).",
|
||||
payload_bytes,
|
||||
)
|
||||
|
||||
|
||||
def apply_dsmr_subscription(settings: "Settings") -> None:
|
||||
"""(Re)apply the DSMR MQTT subscriptions to match *settings* — restart-free.
|
||||
|
||||
Call this at startup and after every config save. It makes the live MQTT
|
||||
subscriptions reflect the current ``dsmr_ingest_enabled`` / ``dsmr_mqtt_topic``
|
||||
/ ``dsmr_sample_interval_s`` / ``dsmr_tariff_topic`` settings without an app
|
||||
restart:
|
||||
|
||||
- **Disabled** → unsubscribe any active DSMR and tariff subscriptions.
|
||||
- **Enabled** → (re)subscribe to ``dsmr_mqtt_topic`` with a handler bound to
|
||||
a *fresh* settings snapshot, so a changed sample interval also takes effect.
|
||||
Also subscribe to ``dsmr_tariff_topic`` when non-empty.
|
||||
- **Topic changed** → unsubscribe the old topic before subscribing the new one.
|
||||
|
||||
Idempotent and safe to call when MQTT is not connected (the subscription is
|
||||
queued in the manager and established on the next connect).
|
||||
"""
|
||||
# Imported here (not at module top) to avoid a circular import at app start.
|
||||
from app.integrations.mqtt import mqtt_manager
|
||||
|
||||
global _current_dsmr_topic, _current_tariff_topic
|
||||
|
||||
if not settings.dsmr_ingest_enabled:
|
||||
if _current_dsmr_topic is not None:
|
||||
mqtt_manager.unsubscribe(_current_dsmr_topic)
|
||||
logger.info("DSMR ingest disabled — unsubscribed from topic=%s.", _current_dsmr_topic)
|
||||
_current_dsmr_topic = None
|
||||
if _current_tariff_topic is not None:
|
||||
mqtt_manager.unsubscribe(_current_tariff_topic)
|
||||
logger.info(
|
||||
"DSMR ingest disabled — unsubscribed from tariff topic=%s.",
|
||||
_current_tariff_topic,
|
||||
)
|
||||
_current_tariff_topic = None
|
||||
return
|
||||
|
||||
# --- Main DSMR telegram topic ---
|
||||
topic = settings.dsmr_mqtt_topic
|
||||
if _current_dsmr_topic is not None and _current_dsmr_topic != topic:
|
||||
mqtt_manager.unsubscribe(_current_dsmr_topic)
|
||||
|
||||
# Re-subscribe (overwrites any existing handler for this topic) with a fresh
|
||||
# settings snapshot so dsmr_sample_interval_s changes take effect too.
|
||||
snapshot = settings
|
||||
mqtt_manager.subscribe(topic, lambda payload: handle_message(payload, snapshot))
|
||||
_current_dsmr_topic = topic
|
||||
logger.info("DSMR ingest enabled — subscribed to topic=%s.", topic)
|
||||
|
||||
# --- DSMR tariff topic (dual-tariff slot indicator) ---
|
||||
tariff_topic = settings.dsmr_tariff_topic if settings.dsmr_tariff_topic else ""
|
||||
if tariff_topic:
|
||||
if _current_tariff_topic is not None and _current_tariff_topic != tariff_topic:
|
||||
mqtt_manager.unsubscribe(_current_tariff_topic)
|
||||
mqtt_manager.subscribe(tariff_topic, lambda payload: handle_tariff_message(payload))
|
||||
_current_tariff_topic = tariff_topic
|
||||
logger.info("DSMR tariff topic — subscribed to topic=%s.", tariff_topic)
|
||||
else:
|
||||
# tariff_topic is empty → unsubscribe any existing tariff subscription.
|
||||
if _current_tariff_topic is not None:
|
||||
mqtt_manager.unsubscribe(_current_tariff_topic)
|
||||
logger.info(
|
||||
"DSMR tariff topic cleared — unsubscribed from topic=%s.",
|
||||
_current_tariff_topic,
|
||||
)
|
||||
_current_tariff_topic = None
|
||||
|
||||
|
||||
def handle_message(payload_bytes: bytes, settings: "Settings") -> None:
|
||||
"""Parse one DSMR MQTT payload and persist it if it passes the sample filter.
|
||||
|
||||
Called from the paho network thread; *must* swallow all exceptions so that
|
||||
a bad payload or transient error does not crash the loop or drop the broker
|
||||
connection.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
payload_bytes:
|
||||
Raw bytes from the MQTT message.
|
||||
settings:
|
||||
Runtime settings snapshot (captured at subscription time). Used for
|
||||
``dsmr_sample_interval_s``.
|
||||
"""
|
||||
try:
|
||||
_handle_message_inner(payload_bytes, settings)
|
||||
except Exception:
|
||||
logger.exception("dsmr_ingest.handle_message: unexpected error (swallowed).")
|
||||
|
||||
|
||||
def _handle_message_inner(payload_bytes: bytes, settings: "Settings") -> None:
|
||||
"""Inner implementation — may raise; caller wraps in try/except."""
|
||||
# --- 1. Parse JSON ---
|
||||
try:
|
||||
data: dict = json.loads(payload_bytes)
|
||||
except (json.JSONDecodeError, ValueError):
|
||||
logger.debug("dsmr_ingest: invalid JSON payload (skipped).")
|
||||
return
|
||||
|
||||
if not isinstance(data, dict):
|
||||
logger.debug("dsmr_ingest: payload is not a JSON object (skipped).")
|
||||
return
|
||||
|
||||
# --- 2. Parse timestamp ---
|
||||
raw_ts = data.get("timestamp")
|
||||
if raw_ts is None:
|
||||
logger.debug("dsmr_ingest: missing 'timestamp' field (skipped).")
|
||||
return
|
||||
|
||||
try:
|
||||
# Python 3.11+ accepts the trailing 'Z' directly; for 3.10 compat we
|
||||
# replace 'Z' with '+00:00' before parsing.
|
||||
ts_str = raw_ts if not isinstance(raw_ts, str) else raw_ts.replace("Z", "+00:00")
|
||||
ts_utc: datetime = datetime.fromisoformat(ts_str)
|
||||
# Ensure it is timezone-aware UTC.
|
||||
if ts_utc.tzinfo is None:
|
||||
ts_utc = ts_utc.replace(tzinfo=timezone.utc)
|
||||
except (ValueError, TypeError, AttributeError):
|
||||
logger.debug(
|
||||
"dsmr_ingest: cannot parse 'timestamp' value %r (skipped).", raw_ts
|
||||
)
|
||||
return
|
||||
|
||||
# --- 3. Down-sample: only persist if second falls on interval boundary ---
|
||||
interval = settings.dsmr_sample_interval_s
|
||||
if interval > 0 and (ts_utc.second % interval) != 0:
|
||||
# This telegram is between sample points; discard silently.
|
||||
return
|
||||
|
||||
# --- 4. Extract source_id (telegram's own id) — stored only as a reference,
|
||||
# NOT used for uniqueness/idempotency (it overflows and gets reset). ---
|
||||
source_id: int | None = data.get("id")
|
||||
if source_id is not None and not isinstance(source_id, int):
|
||||
# Unexpected type — treat as missing rather than raising.
|
||||
logger.debug(
|
||||
"dsmr_ingest: 'id' field has unexpected type %s (ignoring).",
|
||||
type(source_id).__name__,
|
||||
)
|
||||
source_id = None
|
||||
|
||||
# --- 5. Persist to database ---
|
||||
# Idempotency is keyed on recorded_at (the telegram timestamp), which is
|
||||
# telegram-id-independent: a single P1 meter emits one telegram per second,
|
||||
# and down-sampling keeps at most one per interval-aligned second. The
|
||||
# UNIQUE(recorded_at) constraint is the backstop for the IntegrityError race.
|
||||
session_local = get_session_local()
|
||||
session = session_local()
|
||||
try:
|
||||
existing = session.scalar(
|
||||
select(DsmrReading).where(DsmrReading.recorded_at == ts_utc)
|
||||
)
|
||||
if existing is not None:
|
||||
logger.debug(
|
||||
"dsmr_ingest: recorded_at=%s already in DB, skipping.",
|
||||
ts_utc.isoformat(),
|
||||
)
|
||||
return
|
||||
|
||||
reading = DsmrReading(
|
||||
recorded_at=ts_utc,
|
||||
source_id=source_id,
|
||||
payload=data, # full frame, verbatim
|
||||
)
|
||||
session.add(reading)
|
||||
session.commit()
|
||||
logger.debug(
|
||||
"dsmr_ingest: persisted reading recorded_at=%s source_id=%s.",
|
||||
ts_utc.isoformat(),
|
||||
source_id,
|
||||
)
|
||||
except sqlalchemy.exc.IntegrityError:
|
||||
# Race / duplicate: another insert beat us to the same recorded_at.
|
||||
session.rollback()
|
||||
logger.debug(
|
||||
"dsmr_ingest: IntegrityError for recorded_at=%s (duplicate, skipped).",
|
||||
ts_utc.isoformat(),
|
||||
)
|
||||
except Exception:
|
||||
session.rollback()
|
||||
logger.exception("dsmr_ingest: DB error (swallowed).")
|
||||
finally:
|
||||
session.close()
|
||||
@@ -0,0 +1,894 @@
|
||||
"""Billing engine for DSMR 15-minute energy metering periods.
|
||||
|
||||
This module implements the two-layer billing model described in §3.4 of the
|
||||
M6 design document, extended in M7-T03 to be meter-aware:
|
||||
|
||||
**Layer 1 — per-period metering cost (immutable, price-snapshot)**
|
||||
``compute_period(session, t0)`` computes the import cost, export revenue, and
|
||||
net cost for the 15-minute period ``[t0, t0+15min)``. The result is written
|
||||
to ``energy_cost_period`` with a full pricing snapshot so each row is
|
||||
self-contained and auditable. Existing *successful* rows are never overwritten
|
||||
by the normal tick path; only an explicit ``recompute_range`` call passes
|
||||
``overwrite=True``.
|
||||
|
||||
**Layer 2 — summary (computed at read time, not stored)**
|
||||
``summarize(session, start, end)`` aggregates all non-degraded
|
||||
``energy_cost_period`` rows in ``[start, end)``, then adds the daily
|
||||
standing charges (network_fee + management_fee, apportioned at EUR/month
|
||||
÷ 30 per day) and subtracts the energy-tax credit (heffingskorting,
|
||||
apportioned at EUR/year ÷ 365 per day).
|
||||
|
||||
Design notes
|
||||
------------
|
||||
- **Decimal arithmetic throughout**: all monetary computations use
|
||||
``decimal.Decimal`` to avoid float binary rounding errors. Only when
|
||||
writing to ``EnergyCostPeriod`` columns (Float) are values converted to
|
||||
float. ``summarize`` converts back to Decimal for summation.
|
||||
- **UTC quarter-hour grid**: period boundaries are aligned to UTC 00/15/30/45
|
||||
minutes (``floor_to_quarter``). NL local time (CET/CEST) is always a whole
|
||||
number of hours from UTC, so the quarter-hour grid is the same in both
|
||||
timezone representations.
|
||||
- **Register keys**: DSMR payload uses JSON strings like ``"20915.154"``
|
||||
for cumulative kWh registers. ``register_at`` converts them to Decimal.
|
||||
- **Degraded vs skip semantics**:
|
||||
- *No meter coverage* (``meter_at`` returns None for t0): write a
|
||||
``degraded=True`` row with ``meter_id=None``.
|
||||
- *Cross-meter boundary* (m0.id != m1.id for t0/t1): write a ``degraded=True``
|
||||
row with ``meter_id=m0.id``; losing this one period at the swap boundary is
|
||||
acceptable (D5 decision).
|
||||
- *Missing readings* (``register_at`` returns None for start or end
|
||||
boundary within the meter window): write a ``degraded=True`` row with
|
||||
``meter_id=m0.id`` so the period is tracked and can be retried by
|
||||
``compute_closed_periods``.
|
||||
- *Negative or excessively large delta* (delta sanity guard D6): write a
|
||||
``degraded=True`` row with ``meter_id=m0.id``; prevents negative costs and
|
||||
grossly inflated costs from meter resets, DSMR rollover, or data spikes.
|
||||
- *Missing Tibber price* (``TibberPriceNotFoundError``): skip entirely (do
|
||||
not write a row); the period will be retried once prices arrive.
|
||||
- *Missing active contract version*: skip (no contract to compute against).
|
||||
- **Meter-aware register lookup**: ``register_at`` now accepts a ``meter``
|
||||
parameter and restricts the DSMR reading query to readings within
|
||||
``[meter.started_at, meter.ended_at)`` (half-open), preventing old-meter
|
||||
readings from leaking into a new-meter epoch.
|
||||
- **Lookback window in ``compute_closed_periods``**: to avoid scanning all
|
||||
historical DSMR data on every tick, the function looks back at most 7 days
|
||||
from the current time. This covers typical short outages (no data / no
|
||||
contract) while staying bounded. Periods older than 7 days must be
|
||||
recovered via an explicit ``recompute_range`` call.
|
||||
|
||||
Meter-aware compute_period ordering rationale (M7-T03)
|
||||
-------------------------------------------------------
|
||||
The order of checks inside ``compute_period`` is:
|
||||
|
||||
1. **Immutability guard** (existing non-degraded row, overwrite=False) → return False.
|
||||
2. **Meter determination** (m0 = meter_at(t0), m1 = meter_at(t1)):
|
||||
- No meter (m0 is None) → write degraded, meter_id=None.
|
||||
- Cross-meter boundary (m0.id != m1.id) → write degraded, meter_id=m0.id.
|
||||
3. **Active contract version check** → skip (no write) if absent.
|
||||
4. **Boundary register readings** within m0's window → write degraded if missing.
|
||||
5. **Delta sanity guard** → write degraded if any delta < 0 or > _MAX_DELTA_KWH.
|
||||
6. **Price strategy** → skip (no write) if Tibber price missing.
|
||||
7. **Upsert billing record** with meter_id=m0.id.
|
||||
|
||||
Why meter before contract? The meter is a *structural* prerequisite: without a
|
||||
known meter epoch we cannot trust the delta at all, so we commit a degraded row
|
||||
immediately. The contract skip, by contrast, is transient (the period can be
|
||||
re-computed once a contract is configured), so it produces no row.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from datetime import UTC, datetime, timedelta
|
||||
from decimal import Decimal
|
||||
from typing import Any
|
||||
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.integrations.pricing.strategies import (
|
||||
PeriodDeltas,
|
||||
TibberPriceNotFoundError,
|
||||
get_strategy,
|
||||
)
|
||||
from app.models.energy import DsmrReading, EnergyCostPeriod, Meter
|
||||
from app.services.contracts import active_contract_version_at, active_contract_versions
|
||||
from app.services.meters import meter_at
|
||||
from app.services.timezone import local_date, local_now
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Constants
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_PERIOD_MINUTES = 15
|
||||
_LOOKBACK_DAYS = 7 # maximum lookback window for compute_closed_periods
|
||||
|
||||
# Maximum age a DSMR reading may have relative to the boundary being queried.
|
||||
# Under normal operation DSMR readings arrive every ~10 seconds, so a reading
|
||||
# more than one period (15 minutes) old at the boundary indicates either a data
|
||||
# gap or — critically — a *future* boundary being resolved against the last
|
||||
# historical reading. In both cases the reading is considered stale and
|
||||
# ``register_at`` returns None, letting the period be marked degraded instead of
|
||||
# producing a spurious zero-delta "successful" row.
|
||||
_READING_MAX_STALENESS = timedelta(minutes=_PERIOD_MINUTES)
|
||||
|
||||
# Maximum plausible kWh delta for a single 15-minute period (D6 sanity guard).
|
||||
# A typical Dutch household uses well under 5 kWh per quarter hour even under
|
||||
# heavy load. 100 kWh per 15 minutes corresponds to ~400 kW — far beyond any
|
||||
# residential consumption — but is lenient enough to never fire on legitimate
|
||||
# data. Any delta at or above this threshold indicates a meter reset, DSMR
|
||||
# rollover, sign error, or other data anomaly, and the period is marked
|
||||
# degraded to prevent negative costs or grossly inflated charges.
|
||||
_MAX_DELTA_KWH = Decimal("100")
|
||||
|
||||
# 每日固定费/税补在"本地午夜后多久"才结算入账。延后到 01:05 是为了让累计成本的
|
||||
# 整天阶跃落在新一天、且避开 01:00 整点(HA 长期统计的小时桶边界)。
|
||||
_SETTLEMENT_OFFSET = timedelta(hours=1, minutes=5)
|
||||
|
||||
# DSMR payload register keys (cumulative kWh, JSON string values).
|
||||
_KEY_D1 = "electricity_delivered_1" # delivered low-tariff (dal / _1)
|
||||
_KEY_D2 = "electricity_delivered_2" # delivered high-tariff (normal / _2)
|
||||
_KEY_R1 = "electricity_returned_1" # returned low-tariff
|
||||
_KEY_R2 = "electricity_returned_2" # returned high-tariff
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Internal helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def floor_to_quarter(dt: datetime) -> datetime:
|
||||
"""Return *dt* floored to the nearest UTC quarter-hour boundary.
|
||||
|
||||
The result always has seconds=0 and microseconds=0, and minutes in
|
||||
{0, 15, 30, 45}. Timezone info is preserved if present.
|
||||
"""
|
||||
floored_minute = (dt.minute // _PERIOD_MINUTES) * _PERIOD_MINUTES
|
||||
return dt.replace(minute=floored_minute, second=0, microsecond=0)
|
||||
|
||||
|
||||
def _to_decimal(value: Any) -> Decimal:
|
||||
"""Convert *value* to Decimal via str() to avoid float binary rounding."""
|
||||
return Decimal(str(value))
|
||||
|
||||
|
||||
def _as_utc(dt: datetime) -> datetime:
|
||||
"""Attach UTC tzinfo to a naive datetime (SQLite read-back workaround)."""
|
||||
if dt.tzinfo is None:
|
||||
return dt.replace(tzinfo=UTC)
|
||||
return dt
|
||||
|
||||
|
||||
def _existing_period(session: Session, t0: datetime) -> EnergyCostPeriod | None:
|
||||
"""Return the EnergyCostPeriod row for period_start=t0, or None."""
|
||||
return session.execute(
|
||||
select(EnergyCostPeriod).where(EnergyCostPeriod.period_start == t0)
|
||||
).scalar_one_or_none()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# register_at — boundary reading lookup (meter-aware)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def register_at(
|
||||
session: Session,
|
||||
boundary: datetime,
|
||||
meter: Meter,
|
||||
) -> dict[str, Decimal] | None:
|
||||
"""Return the four cumulative kWh register values at *boundary*, within *meter*'s window.
|
||||
|
||||
Queries the most recent ``DsmrReading`` with:
|
||||
``recorded_at ≤ boundary``
|
||||
AND ``recorded_at ≥ meter.started_at``
|
||||
AND (``meter.ended_at IS NULL`` OR ``recorded_at < meter.ended_at``)
|
||||
|
||||
The meter window constraint (half-open ``[started_at, ended_at)``) ensures
|
||||
that readings from a previous meter epoch are never used to anchor a new
|
||||
meter's computation. Without this guard, the final reading of the old meter
|
||||
would be visible at the start of the new meter's epoch and produce a
|
||||
cross-meter delta, defeating the isolation guarantee.
|
||||
|
||||
Extracts the four energy registers from ``payload``:
|
||||
|
||||
d1 — electricity_delivered_1 (delivered low-tariff / dal)
|
||||
d2 — electricity_delivered_2 (delivered high-tariff / normal)
|
||||
r1 — electricity_returned_1 (returned low-tariff)
|
||||
r2 — electricity_returned_2 (returned high-tariff)
|
||||
|
||||
Returns
|
||||
-------
|
||||
dict[str, Decimal] with keys ``d1``, ``d2``, ``r1``, ``r2``, or ``None``
|
||||
when:
|
||||
- No ``DsmrReading`` row exists with ``recorded_at ≤ boundary`` within
|
||||
*meter*'s epoch window.
|
||||
- The most recent such reading is older than ``_READING_MAX_STALENESS``
|
||||
relative to *boundary* (freshness guard).
|
||||
- Any of the four register keys is absent from the payload.
|
||||
- Any of the four register values is ``None`` (null in JSON).
|
||||
|
||||
SQLite naive datetime note
|
||||
--------------------------
|
||||
``recorded_at`` is stored as a naive UTC datetime in SQLite. Comparisons
|
||||
against *boundary* (always tz-aware UTC) use ``_as_utc()`` for the
|
||||
freshness check. The SQL ``WHERE`` clause comparisons work correctly
|
||||
because SQLAlchemy's SQLite dialect strips tzinfo when binding parameters
|
||||
(leaving the wall-clock UTC value unchanged), consistent with the storage
|
||||
format.
|
||||
"""
|
||||
# Build the meter-window constraints: [started_at, ended_at).
|
||||
meter_lower = meter.started_at # DsmrReading.recorded_at >= meter.started_at
|
||||
meter_upper = meter.ended_at # DsmrReading.recorded_at < meter.ended_at (if set)
|
||||
|
||||
stmt = (
|
||||
select(DsmrReading)
|
||||
.where(
|
||||
DsmrReading.recorded_at <= boundary,
|
||||
DsmrReading.recorded_at >= meter_lower,
|
||||
)
|
||||
.order_by(DsmrReading.recorded_at.desc())
|
||||
.limit(1)
|
||||
)
|
||||
# Apply the upper bound only when the meter is closed (ended_at is not None).
|
||||
if meter_upper is not None:
|
||||
stmt = stmt.where(DsmrReading.recorded_at < meter_upper)
|
||||
|
||||
row: DsmrReading | None = session.execute(stmt).scalar_one_or_none()
|
||||
|
||||
if row is None:
|
||||
return None
|
||||
|
||||
# Freshness guard: reject readings that are too old relative to *boundary*.
|
||||
# ``recorded_at`` is stored as a naive UTC datetime in SQLite; attach UTC
|
||||
# tzinfo before comparing with *boundary* (which is always tz-aware UTC) to
|
||||
# avoid an "offset-naive vs offset-aware" TypeError.
|
||||
if _as_utc(row.recorded_at) < _as_utc(boundary) - _READING_MAX_STALENESS:
|
||||
return None
|
||||
|
||||
payload = row.payload or {}
|
||||
try:
|
||||
d1_raw = payload[_KEY_D1]
|
||||
d2_raw = payload[_KEY_D2]
|
||||
r1_raw = payload[_KEY_R1]
|
||||
r2_raw = payload[_KEY_R2]
|
||||
except KeyError:
|
||||
return None
|
||||
|
||||
if any(v is None for v in (d1_raw, d2_raw, r1_raw, r2_raw)):
|
||||
return None
|
||||
|
||||
return {
|
||||
"d1": _to_decimal(d1_raw),
|
||||
"d2": _to_decimal(d2_raw),
|
||||
"r1": _to_decimal(r1_raw),
|
||||
"r2": _to_decimal(r2_raw),
|
||||
}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# compute_period — single 15-minute period
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def compute_period(session: Session, t0: datetime, *, overwrite: bool = False) -> bool:
|
||||
"""Compute and upsert the billing record for the period ``[t0, t0+15min)``.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
session:
|
||||
Active SQLAlchemy session. Caller is responsible for committing.
|
||||
t0:
|
||||
UTC start of the 15-minute period. **Must** lie on a quarter-hour
|
||||
grid boundary (minutes ∈ {0, 15, 30, 45}, seconds=0, microseconds=0).
|
||||
overwrite:
|
||||
If ``True``, overwrite an existing *successful* row (i.e. re-compute
|
||||
even when a non-degraded record already exists). The normal tick path
|
||||
always passes ``False``; only ``recompute_range`` passes ``True``.
|
||||
|
||||
Returns
|
||||
-------
|
||||
bool
|
||||
``True`` if a record was written (inserted or updated), ``False`` if
|
||||
the period was skipped (missing contract or missing Tibber price).
|
||||
|
||||
Side-effects
|
||||
------------
|
||||
- Inserts or updates an ``EnergyCostPeriod`` row keyed on ``period_start=t0``.
|
||||
- If no meter covers t0 (``meter_at`` returns None for t0): inserts/updates
|
||||
a degraded row with ``meter_id=None``.
|
||||
- If the period spans a meter boundary (``meter_at(t0).id != meter_at(t1).id``):
|
||||
inserts/updates a degraded row with ``meter_id=m0.id`` (D5 decision).
|
||||
- If readings are missing at either boundary within the meter window:
|
||||
inserts/updates a degraded row with ``meter_id=m0.id``.
|
||||
- If any delta is negative or exceeds ``_MAX_DELTA_KWH`` (D6 sanity guard):
|
||||
inserts/updates a degraded row with ``meter_id=m0.id``.
|
||||
- If the active contract version is missing: **skips** (returns False, no write).
|
||||
- If the Tibber price is missing (TibberPriceNotFoundError): **skips**
|
||||
(returns False, no write).
|
||||
"""
|
||||
t1 = t0 + timedelta(minutes=_PERIOD_MINUTES)
|
||||
now = datetime.now(UTC)
|
||||
|
||||
# Immutability guard: skip if a successful record already exists and we
|
||||
# are not in overwrite mode.
|
||||
existing = _existing_period(session, t0)
|
||||
if existing is not None and not existing.degraded and not overwrite:
|
||||
return False
|
||||
|
||||
# --- Meter determination (structural prerequisite, checked before contract) ---
|
||||
#
|
||||
# A missing or cross-boundary meter is a structural problem: we cannot trust
|
||||
# the delta at all, so we write a degraded row immediately. This is different
|
||||
# from the contract skip (transient, no write): the degraded row ensures the
|
||||
# period appears in the history and can be revisited once the meter timeline
|
||||
# is corrected and a recompute_range is triggered.
|
||||
#
|
||||
# Ordering rationale:
|
||||
# 1. No meter (m0 is None) → degraded(meter_id=None): no epoch for t0.
|
||||
# 2. Cross-meter boundary (m0.id != m1.id) → degraded(meter_id=m0.id): D5.
|
||||
# 3. (Single meter, proceed) → contract check → readings → delta guard → price.
|
||||
#
|
||||
# We place meter before contract so that "cross-table period" is always
|
||||
# marked degraded regardless of contract state. If we checked contract
|
||||
# first, a missing-contract skip would silently discard the cross-table
|
||||
# evidence; once a contract is added and recompute runs, the engine would
|
||||
# incorrectly use cross-table reads.
|
||||
m0 = meter_at(session, t0)
|
||||
m1 = meter_at(session, t1)
|
||||
|
||||
if m0 is None:
|
||||
# No meter epoch covers t0 — degraded with no meter attribution.
|
||||
logger.debug(
|
||||
"compute_period(%s): no active meter at t0 — writing degraded (meter_id=None).",
|
||||
t0.isoformat(),
|
||||
)
|
||||
_upsert_degraded(session, t0, now, existing, meter_id=None)
|
||||
return True
|
||||
|
||||
if m1 is None or m0.id != m1.id:
|
||||
# Period spans a meter boundary or t1 has no meter. Degrade with m0's id
|
||||
# (t0's meter attribution): the period's start belongs to m0's epoch.
|
||||
logger.debug(
|
||||
"compute_period(%s): period crosses meter boundary "
|
||||
"(m0.id=%s, m1.id=%s) — writing degraded.",
|
||||
t0.isoformat(),
|
||||
m0.id,
|
||||
m1.id if m1 is not None else None,
|
||||
)
|
||||
_upsert_degraded(session, t0, now, existing, meter_id=m0.id)
|
||||
return True
|
||||
|
||||
# --- Active contract version at t0 ---
|
||||
# If there is no active contract covering t0, skip the period entirely.
|
||||
# We do not write a degraded row — there is no meaningful state to recover
|
||||
# without a contract (we would not know which strategy to apply once data
|
||||
# arrives). The period can be recovered via an explicit recompute_range once
|
||||
# a contract is configured and activated.
|
||||
version = active_contract_version_at(session, t0)
|
||||
if version is None:
|
||||
logger.debug("compute_period(%s): no active contract version — skipping.", t0.isoformat())
|
||||
return False
|
||||
|
||||
# --- Boundary readings within m0's meter window ---
|
||||
start_regs = register_at(session, t0, m0)
|
||||
end_regs = register_at(session, t1, m0)
|
||||
|
||||
if start_regs is None or end_regs is None:
|
||||
# Missing readings within the meter window → degraded with m0 attribution.
|
||||
_upsert_degraded(session, t0, now, existing, meter_id=m0.id)
|
||||
return True # a record was written (degraded)
|
||||
|
||||
# --- Compute deltas (end − start) ---
|
||||
deltas = PeriodDeltas(
|
||||
d1=end_regs["d1"] - start_regs["d1"],
|
||||
d2=end_regs["d2"] - start_regs["d2"],
|
||||
r1=end_regs["r1"] - start_regs["r1"],
|
||||
r2=end_regs["r2"] - start_regs["r2"],
|
||||
)
|
||||
|
||||
# --- Delta sanity guard (D6) ---
|
||||
# Any negative delta indicates a meter reset, DSMR rollover, or data error.
|
||||
# Any delta exceeding _MAX_DELTA_KWH (100 kWh per 15 min = 400 kW average)
|
||||
# is implausible for residential use and indicates an anomaly.
|
||||
# Both cases produce a degraded row so no negative or grossly inflated cost
|
||||
# is ever written to the billing record.
|
||||
all_deltas = (deltas.d1, deltas.d2, deltas.r1, deltas.r2)
|
||||
if any(d < Decimal("0") for d in all_deltas) or any(d > _MAX_DELTA_KWH for d in all_deltas):
|
||||
logger.debug(
|
||||
"compute_period(%s): delta sanity guard triggered "
|
||||
"(d1=%s, d2=%s, r1=%s, r2=%s) — writing degraded.",
|
||||
t0.isoformat(),
|
||||
deltas.d1,
|
||||
deltas.d2,
|
||||
deltas.r1,
|
||||
deltas.r2,
|
||||
)
|
||||
_upsert_degraded(session, t0, now, existing, meter_id=m0.id)
|
||||
return True
|
||||
|
||||
# --- Price strategy ---
|
||||
strategy = get_strategy(version.contract.kind)
|
||||
try:
|
||||
result = strategy(deltas, t0, version.values, session)
|
||||
except TibberPriceNotFoundError:
|
||||
# Missing Tibber price → skip the period; it will be retried once the
|
||||
# price arrives (e.g. after the next Tibber refresh job runs).
|
||||
logger.debug(
|
||||
"compute_period(%s): no Tibber price found — skipping.", t0.isoformat()
|
||||
)
|
||||
return False
|
||||
|
||||
# --- Upsert the billing record ---
|
||||
import_cost: Decimal = result["import_cost"]
|
||||
export_revenue: Decimal = result["export_revenue"]
|
||||
net_cost: Decimal = result["net_cost"]
|
||||
pricing: dict = result["pricing"]
|
||||
|
||||
if existing is not None:
|
||||
# Update in-place (overwrite=True or previous record was degraded).
|
||||
existing.d1_kwh = float(deltas.d1)
|
||||
existing.d2_kwh = float(deltas.d2)
|
||||
existing.r1_kwh = float(deltas.r1)
|
||||
existing.r2_kwh = float(deltas.r2)
|
||||
existing.import_cost = float(import_cost)
|
||||
existing.export_revenue = float(export_revenue)
|
||||
existing.net_cost = float(net_cost)
|
||||
existing.currency = version.contract.currency
|
||||
existing.pricing = pricing
|
||||
existing.contract_version_id = version.id
|
||||
existing.meter_id = m0.id
|
||||
existing.degraded = False
|
||||
existing.computed_at = now
|
||||
else:
|
||||
period = EnergyCostPeriod(
|
||||
period_start=t0,
|
||||
d1_kwh=float(deltas.d1),
|
||||
d2_kwh=float(deltas.d2),
|
||||
r1_kwh=float(deltas.r1),
|
||||
r2_kwh=float(deltas.r2),
|
||||
import_cost=float(import_cost),
|
||||
export_revenue=float(export_revenue),
|
||||
net_cost=float(net_cost),
|
||||
currency=version.contract.currency,
|
||||
pricing=pricing,
|
||||
contract_version_id=version.id,
|
||||
meter_id=m0.id,
|
||||
degraded=False,
|
||||
computed_at=now,
|
||||
)
|
||||
session.add(period)
|
||||
|
||||
return True
|
||||
|
||||
|
||||
def _upsert_degraded(
|
||||
session: Session,
|
||||
t0: datetime,
|
||||
now: datetime,
|
||||
existing: EnergyCostPeriod | None,
|
||||
*,
|
||||
meter_id: int | None,
|
||||
) -> None:
|
||||
"""Insert or update a degraded placeholder for period *t0*.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
session:
|
||||
Active SQLAlchemy session.
|
||||
t0:
|
||||
UTC start of the 15-minute period.
|
||||
now:
|
||||
Current UTC timestamp for the ``computed_at`` field.
|
||||
existing:
|
||||
The existing ``EnergyCostPeriod`` row for this period, or ``None``.
|
||||
meter_id:
|
||||
The meter ID to attribute this degraded period to, or ``None`` when
|
||||
no meter epoch covers the period (no-meter degraded case).
|
||||
|
||||
When *existing* is not None (row was previously written — either degraded
|
||||
or successful), the row is explicitly reset to the standard degraded state.
|
||||
This is required for the ``recompute_range`` (overwrite=True) path: if the
|
||||
row was previously a *successful* computation and the boundary readings have
|
||||
since disappeared, the stale non-zero costs must be cleared so the row
|
||||
accurately reflects the current "missing readings" state rather than
|
||||
masquerading as a valid result.
|
||||
|
||||
The ``meter_id`` is always updated to reflect the current meter attribution
|
||||
judgment (the result of ``meter_at`` at the time of recompute). This
|
||||
ensures that a retroactive ``started_at`` change + ``recompute_range`` will
|
||||
re-attribute historical degraded periods to the correct meter epoch.
|
||||
"""
|
||||
if existing is not None:
|
||||
# Explicitly reset to degraded state — identical field values to the
|
||||
# new-row path below. This covers the recompute-over-successful-row
|
||||
# case where old non-zero costs must not survive the downgrade.
|
||||
existing.d1_kwh = 0.0
|
||||
existing.d2_kwh = 0.0
|
||||
existing.r1_kwh = 0.0
|
||||
existing.r2_kwh = 0.0
|
||||
existing.import_cost = 0.0
|
||||
existing.export_revenue = 0.0
|
||||
existing.net_cost = 0.0
|
||||
existing.pricing = {}
|
||||
existing.contract_version_id = None
|
||||
existing.meter_id = meter_id
|
||||
existing.degraded = True
|
||||
existing.computed_at = now
|
||||
else:
|
||||
period = EnergyCostPeriod(
|
||||
period_start=t0,
|
||||
d1_kwh=0.0,
|
||||
d2_kwh=0.0,
|
||||
r1_kwh=0.0,
|
||||
r2_kwh=0.0,
|
||||
import_cost=0.0,
|
||||
export_revenue=0.0,
|
||||
net_cost=0.0,
|
||||
currency="EUR", # placeholder; real currency known after contract lookup
|
||||
pricing={},
|
||||
contract_version_id=None,
|
||||
meter_id=meter_id,
|
||||
degraded=True,
|
||||
computed_at=now,
|
||||
)
|
||||
session.add(period)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# compute_closed_periods — periodic tick
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def compute_closed_periods(session: Session) -> int:
|
||||
"""Find and compute all uncalculated closed 15-minute periods.
|
||||
|
||||
A period ``[t0, t1)`` is *closed* when ``t1 ≤ now``. This function:
|
||||
|
||||
1. Determines the lookback window: from ``now − LOOKBACK_DAYS`` to ``now``,
|
||||
floored to the nearest quarter-hour. This avoids an unbounded full
|
||||
historical scan on every tick while still covering the typical recovery
|
||||
window (short outages, missing contract, etc.). Periods older than
|
||||
``LOOKBACK_DAYS`` must be recovered via an explicit ``recompute_range``.
|
||||
2. Iterates over all quarter-hour boundaries in that window where
|
||||
``t1 ≤ now`` (i.e. the period has already closed).
|
||||
3. For each boundary, calls ``compute_period(overwrite=False)``, which:
|
||||
- Skips periods that already have a *successful* (non-degraded) record.
|
||||
- Retries periods that have a *degraded* record.
|
||||
- Writes degraded rows for periods with no meter or cross-meter boundaries.
|
||||
- Skips periods for which no active contract version exists or the
|
||||
Tibber price is unavailable (without writing a degraded row).
|
||||
|
||||
Returns
|
||||
-------
|
||||
int
|
||||
Number of periods for which a record was written (inserted or updated).
|
||||
Does not count skipped periods.
|
||||
"""
|
||||
now = datetime.now(UTC)
|
||||
# Current period boundary (the one whose t1 has not yet passed).
|
||||
current_t0 = floor_to_quarter(now)
|
||||
# Earliest boundary to consider.
|
||||
earliest_t0 = floor_to_quarter(now - timedelta(days=_LOOKBACK_DAYS))
|
||||
|
||||
written = 0
|
||||
t0 = earliest_t0
|
||||
while t0 < current_t0:
|
||||
t1 = t0 + timedelta(minutes=_PERIOD_MINUTES)
|
||||
if t1 <= now:
|
||||
try:
|
||||
did_write = compute_period(session, t0, overwrite=False)
|
||||
if did_write:
|
||||
written += 1
|
||||
except Exception:
|
||||
logger.exception(
|
||||
"compute_closed_periods: unexpected error for t0=%s — continuing.",
|
||||
t0.isoformat(),
|
||||
)
|
||||
t0 += timedelta(minutes=_PERIOD_MINUTES)
|
||||
|
||||
if written:
|
||||
session.commit()
|
||||
logger.info("compute_closed_periods: wrote %d period(s).", written)
|
||||
return written
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# recompute_range — explicit full recompute
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def recompute_range(session: Session, start: datetime, end: datetime) -> int:
|
||||
"""Recompute (overwrite) all 15-minute periods in ``[start, end)``.
|
||||
|
||||
This is the *explicit opt-in* path for recovering from:
|
||||
- Periods where readings or prices arrived late.
|
||||
- Price corrections (new contract version retroactively applied).
|
||||
- Retroactive meter changes (``update_meter`` with new ``started_at``) —
|
||||
re-running this function will re-judge meter attribution and re-compute
|
||||
costs using the corrected epoch boundaries.
|
||||
- Any other reason to override the immutability guard.
|
||||
|
||||
The function iterates over every UTC quarter-hour boundary in
|
||||
``[floor(start), end)`` and calls ``compute_period(overwrite=True)``.
|
||||
Existing rows (including successful ones) are overwritten; their
|
||||
``meter_id`` fields will reflect the *current* ``meter_at`` judgment for
|
||||
each period's start timestamp, naturally re-attributing periods when meter
|
||||
``started_at`` values have been retroactively corrected.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
session:
|
||||
Active SQLAlchemy session. The function commits after all periods
|
||||
have been processed.
|
||||
start:
|
||||
Inclusive start datetime (floored to the nearest quarter-hour internally).
|
||||
end:
|
||||
Exclusive end datetime.
|
||||
|
||||
Returns
|
||||
-------
|
||||
int
|
||||
Number of periods for which a record was written (inserted or updated).
|
||||
Periods skipped due to missing contract or missing Tibber price are
|
||||
*not* counted.
|
||||
"""
|
||||
t0 = floor_to_quarter(_as_utc(start))
|
||||
end_utc = _as_utc(end)
|
||||
now = datetime.now(UTC)
|
||||
|
||||
written = 0
|
||||
while t0 < end_utc:
|
||||
t1 = t0 + timedelta(minutes=_PERIOD_MINUTES)
|
||||
# Only recompute periods that have already closed (t1 ≤ now). Even
|
||||
# when the caller passes a future *end*, we must not write speculative
|
||||
# "future" rows: register_at would resolve to the latest historical
|
||||
# reading for both boundaries, producing a zero-delta fake-success row
|
||||
# that blocks the real computation once the period actually closes.
|
||||
if t1 <= now:
|
||||
try:
|
||||
did_write = compute_period(session, t0, overwrite=True)
|
||||
if did_write:
|
||||
written += 1
|
||||
except Exception:
|
||||
logger.exception(
|
||||
"recompute_range: unexpected error for t0=%s — continuing.",
|
||||
t0.isoformat(),
|
||||
)
|
||||
t0 += timedelta(minutes=_PERIOD_MINUTES)
|
||||
|
||||
session.commit()
|
||||
logger.info(
|
||||
"recompute_range(%s, %s): wrote %d period(s).",
|
||||
start.isoformat(),
|
||||
end.isoformat(),
|
||||
written,
|
||||
)
|
||||
return written
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# summarize — layer-2 aggregation (read-time, not stored)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def summarize(session: Session, start: datetime, end: datetime) -> dict[str, Any]:
|
||||
"""Aggregate billing for the interval ``[start, end)``.
|
||||
|
||||
Computes the total payable as:
|
||||
|
||||
total_payable = Σ(net_cost) -- metered electricity
|
||||
+ fixed_costs -- per-day standing charges, cross-version
|
||||
- credits -- per-day heffingskorting, cross-version
|
||||
|
||||
**Fixed-cost / credit counting — Principle C (symmetric begin/end)**:
|
||||
Both the local calendar day in which *start* falls and the local calendar
|
||||
day in which *end* falls are counted as full days. Fixed charges
|
||||
(network_fee, management_fee) and the energy-tax credit (heffingskorting)
|
||||
are assessed on a "service-is-active" basis — if the meter was online on a
|
||||
given calendar day, the full day's charge/credit applies, regardless of
|
||||
whether the window starts at midnight or mid-morning.
|
||||
|
||||
For each local calendar date D in the range
|
||||
``[local_date(start), min(local_date(end), today_local)]``:
|
||||
- D ≤ today_local (only elapsed / today days count as "whole days").
|
||||
- The contract version whose effective_from local-date ≤ D is used.
|
||||
- This is cross-version: if V1 is from June 1 and V2 from June 25,
|
||||
querying June 1–30 uses V1 for days 1-24 and V2 for day 25.
|
||||
|
||||
The end-day (last_counted) is included when the end's local midnight falls
|
||||
strictly before end_utc; combined with the always-counted start day this
|
||||
makes the begin/end handling symmetric. A short same-day window therefore
|
||||
counts its single local day. A window contributes 0 days only when the
|
||||
counted range is empty (first_counted > last_counted) — e.g. a window lying
|
||||
entirely in the future, since last_counted is capped at today_local.
|
||||
|
||||
Daily getters (``*_today``) use windows exactly aligned to local midnight,
|
||||
so their ``first_counted`` is always today — unaffected by this fix.
|
||||
|
||||
Days that have not yet started in local time (D > today_local) are
|
||||
never counted. Switching versions never resets the counter.
|
||||
|
||||
**Timezone note**: the ``days`` field in the returned dict still represents
|
||||
the window length in calendar days (total_seconds / 86400), for backward
|
||||
compatibility with existing API consumers. The fixed/credit calculation
|
||||
independently counts whole local days as described above.
|
||||
|
||||
All arithmetic uses Decimal; the returned dict contains Python floats for
|
||||
JSON-serialisation convenience.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
session:
|
||||
Active read-only SQLAlchemy session.
|
||||
start:
|
||||
Inclusive start of the summary interval (UTC or naive-UTC).
|
||||
end:
|
||||
Exclusive end of the summary interval (UTC or naive-UTC).
|
||||
|
||||
Returns
|
||||
-------
|
||||
dict with keys:
|
||||
|
||||
currency str ISO 4217 currency (from contract, or "EUR" fallback)
|
||||
metered_import float Σ import_cost from non-degraded periods
|
||||
metered_export float Σ export_revenue from non-degraded periods
|
||||
metered_net float Σ net_cost from non-degraded periods
|
||||
fixed_costs float standing charges for elapsed whole local days
|
||||
credits float energy-tax credit for elapsed whole local days
|
||||
total_payable float metered_net + fixed_costs − credits
|
||||
period_count int number of non-degraded periods in range
|
||||
degraded_count int number of degraded periods in range
|
||||
days float interval length in days (total_seconds / 86400)
|
||||
"""
|
||||
from datetime import timedelta as _td, date as _date
|
||||
|
||||
start_utc = _as_utc(start)
|
||||
end_utc = _as_utc(end)
|
||||
|
||||
# --- Fetch all EnergyCostPeriod rows in [start, end) ---
|
||||
rows = session.execute(
|
||||
select(EnergyCostPeriod).where(
|
||||
EnergyCostPeriod.period_start >= start_utc,
|
||||
EnergyCostPeriod.period_start < end_utc,
|
||||
)
|
||||
).scalars().all()
|
||||
|
||||
good_rows = [r for r in rows if not r.degraded]
|
||||
degraded_rows = [r for r in rows if r.degraded]
|
||||
|
||||
# Σ monetary amounts (Decimal arithmetic).
|
||||
sum_import = sum((_to_decimal(r.import_cost) for r in good_rows), Decimal("0"))
|
||||
sum_export = sum((_to_decimal(r.export_revenue) for r in good_rows), Decimal("0"))
|
||||
sum_net = sum((_to_decimal(r.net_cost) for r in good_rows), Decimal("0"))
|
||||
|
||||
# --- Interval length in days (window, not elapsed — kept for API compat) ---
|
||||
total_seconds = (end_utc - start_utc).total_seconds()
|
||||
days = _to_decimal(str(total_seconds)) / _to_decimal("86400")
|
||||
|
||||
# --- Fixed costs and credits: Principle C cross-version whole-day counting ---
|
||||
_now_local = local_now()
|
||||
today_local: _date = _now_local.date()
|
||||
|
||||
# --- Compute [first_counted, last_counted] local date range (inclusive) ---
|
||||
#
|
||||
# Both the window-start day and the window-end day are counted as complete
|
||||
# local calendar days, regardless of whether the window starts/ends at midnight.
|
||||
#
|
||||
# Principle C (symmetric begin/end):
|
||||
# • first_counted = local calendar date of start_utc (start day always counted)
|
||||
# • last_counted = local calendar date of end_utc (end day counted if its
|
||||
# local midnight is strictly before end_utc)
|
||||
# • Both are then capped at today_local (only elapsed / today days count).
|
||||
#
|
||||
# Why symmetric? Fixed charges (network_fee, management_fee) and energy-tax credits
|
||||
# (heffingskorting) are assessed on a "service-is-active" basis, not on how many
|
||||
# hours the service was actually running within that calendar day. If the meter
|
||||
# anchor (started_at) falls at 09:18 on June 24, the full June 24 standing charge
|
||||
# and credit still apply because the service was online for that day.
|
||||
#
|
||||
# Previous asymmetric behaviour: a start_utc that was *later* than the local
|
||||
# midnight of local_start_date caused first_counted to be bumped to the *next*
|
||||
# day, silently dropping the anchor day's charges/credits. This was incorrect
|
||||
# for cumulative entities (import_cost_total / export_revenue_total) whose anchor
|
||||
# is often an above-midnight started_at. The end-side had always been symmetric
|
||||
# (counted if local midnight < end_utc), creating an inconsistency.
|
||||
#
|
||||
# Daily getters (window = [local today 00:00, local tomorrow 00:00)) are
|
||||
# unaffected: local_date(local_midnight_utc(today)) == today, so first_counted
|
||||
# is still today regardless of the fix.
|
||||
#
|
||||
# A short same-day window now counts its single local day (the start day is
|
||||
# always counted, symmetric with the end side). A window contributes 0 days
|
||||
# only when the counted range is empty (first_counted > last_counted) — e.g. a
|
||||
# window lying entirely in the future, where last_counted is capped at
|
||||
# today_local while first_counted is later.
|
||||
from app.services.timezone import local_midnight_utc as _lmu
|
||||
|
||||
# Start side: always count the local calendar day in which start_utc falls.
|
||||
first_counted: _date = local_date(start_utc)
|
||||
|
||||
local_end_date: _date = local_date(end_utc)
|
||||
# Is the midnight of local_end_date < end_utc? If yes, that day's midnight is in range.
|
||||
if _lmu(local_end_date) < end_utc:
|
||||
last_counted: _date = local_end_date
|
||||
else:
|
||||
last_counted = local_end_date - _td(days=1)
|
||||
|
||||
# Settlement cap: "today" is only counted once the local clock has passed the
|
||||
# settlement offset since midnight (01:05). This defers the daily standing-charge
|
||||
# step from 00:00 to 01:05, ensuring the cumulative-cost adiabatic jump lands
|
||||
# inside the new calendar day and avoids the HA 01:00 hourly-bucket boundary.
|
||||
_today_midnight_utc = _lmu(today_local)
|
||||
if _now_local >= _today_midnight_utc + _SETTLEMENT_OFFSET:
|
||||
settled_cap = today_local
|
||||
else:
|
||||
settled_cap = today_local - _td(days=1)
|
||||
last_counted = min(last_counted, settled_cap)
|
||||
|
||||
# If the range is empty (first_counted > last_counted), no days are counted.
|
||||
|
||||
# Fetch all versions of the active contract once (single DB call).
|
||||
versions = active_contract_versions(session)
|
||||
|
||||
fixed_dec = Decimal("0")
|
||||
credits_dec = Decimal("0")
|
||||
currency = "EUR"
|
||||
|
||||
if versions:
|
||||
# Currency comes from the contract regardless of day count.
|
||||
currency = versions[0].contract.currency
|
||||
|
||||
if versions and first_counted <= last_counted:
|
||||
# For each version segment, find its overlap with [first_counted, last_counted]
|
||||
# and accumulate whole days.
|
||||
version_segments: list[tuple[_date, _date | None, dict]] = []
|
||||
for v in versions:
|
||||
v_start_local = local_date(_as_utc(v.effective_from))
|
||||
v_end_local = local_date(_as_utc(v.effective_to)) if v.effective_to is not None else None
|
||||
version_segments.append((v_start_local, v_end_local, v.values or {}))
|
||||
|
||||
for v_start, v_end_excl, v_values in version_segments:
|
||||
# The version covers [v_start, v_end_excl) in local dates,
|
||||
# where v_end_excl=None means open-ended (no upper bound).
|
||||
# Intersection with [first_counted, last_counted]:
|
||||
seg_start = max(first_counted, v_start)
|
||||
if v_end_excl is not None:
|
||||
seg_end_excl = min(last_counted + _td(days=1), v_end_excl)
|
||||
else:
|
||||
seg_end_excl = last_counted + _td(days=1)
|
||||
|
||||
if seg_start >= seg_end_excl:
|
||||
continue # no overlap
|
||||
|
||||
n_days = (seg_end_excl - seg_start).days
|
||||
if n_days <= 0:
|
||||
continue
|
||||
|
||||
standing: dict = v_values.get("standing", {})
|
||||
creds: dict = v_values.get("credits", {})
|
||||
|
||||
network_fee = _to_decimal(standing.get("network_fee", 0))
|
||||
management_fee = _to_decimal(standing.get("management_fee", 0))
|
||||
heffingskorting = _to_decimal(creds.get("heffingskorting", 0))
|
||||
|
||||
# Standing charges: EUR/month → EUR/day (÷ 30) × n_days.
|
||||
fixed_dec += (network_fee + management_fee) / Decimal("30") * Decimal(str(n_days))
|
||||
# Energy-tax credit: EUR/year → EUR/day (÷ 365) × n_days.
|
||||
credits_dec += heffingskorting / Decimal("365") * Decimal(str(n_days))
|
||||
|
||||
total_payable = sum_net + fixed_dec - credits_dec
|
||||
|
||||
return {
|
||||
"currency": currency,
|
||||
"metered_import": float(sum_import),
|
||||
"metered_export": float(sum_export),
|
||||
"metered_net": float(sum_net),
|
||||
"fixed_costs": float(fixed_dec),
|
||||
"credits": float(credits_dec),
|
||||
"total_payable": float(total_payable),
|
||||
"period_count": len(good_rows),
|
||||
"degraded_count": len(degraded_rows),
|
||||
"days": float(days),
|
||||
}
|
||||
@@ -0,0 +1,499 @@
|
||||
"""HA MQTT Discovery service — build and publish discovery configs + state.
|
||||
|
||||
This module handles:
|
||||
|
||||
1. Building HA MQTT Discovery payloads for each ``ExposableEntity``.
|
||||
2. Publishing discovery configs (retained) for enabled entities.
|
||||
3. Clearing (empty-payload) discovery configs for disabled entities.
|
||||
4. Publishing current entity state values.
|
||||
5. A per-device helper for use in the Modbus poll loop.
|
||||
|
||||
Design
|
||||
------
|
||||
- **No-op when MQTT / discovery is not configured**: every public function
|
||||
checks ``ha_discovery_enabled`` and ``mqtt_manager.is_connected`` before
|
||||
doing any work. Callers never need to guard these conditions.
|
||||
- **Stable unique_id**: derived from device ``uuid`` + metric ``key``, never
|
||||
from mutable fields like ``friendly_name`` or auto-increment DB ids.
|
||||
- **Discovery topic format**: ``<discovery_prefix>/<component>/<node_id>/<object_id>/config``
|
||||
where ``node_id`` is the device uuid (slugified to be safe) and
|
||||
``object_id`` is a uuid-prefixed stable string. Uses ``ha_discovery_prefix``
|
||||
(default ``"homeassistant"``), which must stay under the HA discovery namespace.
|
||||
- **State topic format**: ``<state_prefix>/<component>/<node_id>/<object_id>/state``
|
||||
Uses ``ha_state_topic_prefix`` (default ``"home_automation"``), separate from
|
||||
the discovery namespace so HA discovery and state/availability topics live under
|
||||
different prefixes.
|
||||
- **Availability topic**: ``<state_prefix>/modbus/<node_id>/availability`` (shared
|
||||
across all entities of the same device; publishes "online"/"offline"). Also uses
|
||||
the state prefix, not the discovery prefix.
|
||||
- **best-effort**: functions catch all exceptions internally so that callers
|
||||
(e.g. the Modbus poll loop) never crash due to MQTT publish errors.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
from typing import Any
|
||||
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.integrations.expose import ExposableEntity, build_catalog
|
||||
from app.integrations.mqtt import mqtt_manager
|
||||
from app.services.config_page import build_runtime_settings
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Internal helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _should_publish(settings: Any) -> bool:
|
||||
"""Return True only if MQTT and HA Discovery are both enabled and connected."""
|
||||
return bool(
|
||||
settings.mqtt_enabled
|
||||
and settings.ha_discovery_enabled
|
||||
and mqtt_manager.is_connected
|
||||
)
|
||||
|
||||
|
||||
def _node_id(device_uuid: str) -> str:
|
||||
"""Stable MQTT node_id derived from device uuid (replace hyphens for safety)."""
|
||||
return device_uuid.replace("-", "_")
|
||||
|
||||
|
||||
def _object_id(entity: ExposableEntity) -> str:
|
||||
"""Stable MQTT object_id derived from entity key (dots + hyphens → underscores)."""
|
||||
return entity.key.replace(".", "_").replace("-", "_")
|
||||
|
||||
|
||||
def _discovery_topic(entity: ExposableEntity, prefix: str) -> str:
|
||||
"""Build the HA Discovery config topic for *entity*.
|
||||
|
||||
Format: ``<prefix>/<component>/<node_id>/<object_id>/config``
|
||||
"""
|
||||
node = _node_id(entity.device.identifiers[1]) # device uuid
|
||||
obj = _object_id(entity)
|
||||
return f"{prefix}/{entity.component}/{node}/{obj}/config"
|
||||
|
||||
|
||||
def _state_topic(entity: ExposableEntity, prefix: str) -> str:
|
||||
"""Build the state publish topic for *entity*.
|
||||
|
||||
Format: ``<prefix>/<component>/<node_id>/<object_id>/state``
|
||||
"""
|
||||
node = _node_id(entity.device.identifiers[1])
|
||||
obj = _object_id(entity)
|
||||
return f"{prefix}/{entity.component}/{node}/{obj}/state"
|
||||
|
||||
|
||||
def _availability_topic(device_uuid: str, prefix: str) -> str:
|
||||
"""Shared availability topic for all entities of a device.
|
||||
|
||||
Format: ``<prefix>/modbus/<node_id>/availability``
|
||||
"""
|
||||
node = _node_id(device_uuid)
|
||||
return f"{prefix}/modbus/{node}/availability"
|
||||
|
||||
|
||||
def _unique_id(entity: ExposableEntity) -> str:
|
||||
"""Stable unique_id — device uuid + metric key (never from mutable fields)."""
|
||||
device_uuid = entity.device.identifiers[1]
|
||||
# entity.key is already "modbus.<uuid>.<metric_key>" — use it as the seed
|
||||
return f"{device_uuid}_{entity.key.replace('.', '_')}"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Public: build discovery payload
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def build_discovery_payload(
|
||||
entity: ExposableEntity,
|
||||
discovery_prefix: str,
|
||||
state_prefix: str | None = None,
|
||||
) -> tuple[str, dict[str, Any]]:
|
||||
"""Build the HA MQTT Discovery config topic and payload dict for *entity*.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
entity:
|
||||
An ``ExposableEntity`` (from ``build_catalog``).
|
||||
discovery_prefix:
|
||||
The HA Discovery config topic prefix (e.g. ``"homeassistant"``).
|
||||
This determines where HA looks for the discovery config topic.
|
||||
state_prefix:
|
||||
The prefix used for state and availability topics (e.g.
|
||||
``"home_automation"``). When omitted or ``None``, falls back to
|
||||
``discovery_prefix`` for backward compatibility (e.g. in tests that
|
||||
pass only one prefix).
|
||||
|
||||
Returns
|
||||
-------
|
||||
tuple[str, dict]
|
||||
``(topic, config_dict)`` — the config topic and the HA Discovery
|
||||
payload (not yet JSON-serialised).
|
||||
"""
|
||||
if state_prefix is None:
|
||||
state_prefix = discovery_prefix
|
||||
|
||||
device_uuid = entity.device.identifiers[1]
|
||||
avail_topic = _availability_topic(device_uuid, state_prefix)
|
||||
state_t = _state_topic(entity, state_prefix)
|
||||
topic = _discovery_topic(entity, discovery_prefix)
|
||||
|
||||
config: dict[str, Any] = {
|
||||
"unique_id": _unique_id(entity),
|
||||
"name": entity.name,
|
||||
"state_topic": state_t,
|
||||
"device": {
|
||||
"identifiers": list(entity.device.identifiers),
|
||||
"name": entity.device.name,
|
||||
},
|
||||
}
|
||||
|
||||
# Only declare an availability topic for devices that actually publish an
|
||||
# online/offline heartbeat (e.g. Modbus, via its "online" binary_sensor).
|
||||
# Devices without a heartbeat (e.g. energy-cost) omit availability so HA
|
||||
# treats their entities as always-available — otherwise HA would mark them
|
||||
# ``unavailable`` forever even though their state is being published.
|
||||
if entity.device.provides_availability:
|
||||
config["availability"] = [{"topic": avail_topic}]
|
||||
config["availability_mode"] = "all"
|
||||
|
||||
# Component-specific fields
|
||||
if entity.component == "binary_sensor":
|
||||
# "online" binary sensor: payload ON/OFF
|
||||
config["payload_on"] = "ON"
|
||||
config["payload_off"] = "OFF"
|
||||
if entity.device_class:
|
||||
config["device_class"] = entity.device_class
|
||||
else:
|
||||
# sensor (and others)
|
||||
if entity.device_class:
|
||||
config["device_class"] = entity.device_class
|
||||
if entity.unit:
|
||||
config["unit_of_measurement"] = entity.unit
|
||||
if entity.state_class:
|
||||
config["state_class"] = entity.state_class
|
||||
|
||||
return topic, config
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Public: publish discovery
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def publish_discovery(session: Session) -> None:
|
||||
"""Publish HA Discovery configs for all entities in the catalog.
|
||||
|
||||
- Enabled entities: publish retained JSON config payload.
|
||||
- Disabled entities: publish empty (b"") payload to clear any previously
|
||||
retained config from HA.
|
||||
|
||||
No-op if MQTT / discovery is not enabled or the client is not connected.
|
||||
All MQTT errors are caught internally — this function never raises.
|
||||
"""
|
||||
from app.config import get_settings
|
||||
settings = build_runtime_settings(session, get_settings())
|
||||
if not _should_publish(settings):
|
||||
logger.debug("publish_discovery: skipped (MQTT/Discovery not enabled or not connected)")
|
||||
return
|
||||
|
||||
discovery_prefix = settings.ha_discovery_prefix
|
||||
state_prefix = settings.ha_state_topic_prefix
|
||||
|
||||
try:
|
||||
catalog = build_catalog(session)
|
||||
except Exception:
|
||||
logger.exception("publish_discovery: failed to build catalog; aborting")
|
||||
return
|
||||
|
||||
for entry in catalog:
|
||||
entity = entry.entity
|
||||
try:
|
||||
topic, config = build_discovery_payload(entity, discovery_prefix, state_prefix)
|
||||
if entry.enabled:
|
||||
payload = json.dumps(config)
|
||||
mqtt_manager.publish(topic, payload, retain=True)
|
||||
logger.debug(
|
||||
"publish_discovery: published config for %r → %s", entity.key, topic
|
||||
)
|
||||
else:
|
||||
# Clear retained config for disabled entities.
|
||||
mqtt_manager.publish(topic, b"", retain=True)
|
||||
logger.debug(
|
||||
"publish_discovery: cleared config for disabled entity %r → %s",
|
||||
entity.key,
|
||||
topic,
|
||||
)
|
||||
except Exception:
|
||||
logger.exception(
|
||||
"publish_discovery: error processing entity %r; continuing", entity.key
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Public: publish states
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def publish_states(session: Session) -> None:
|
||||
"""Publish current state values for all enabled entities.
|
||||
|
||||
Calls each entity's ``value_getter`` to obtain the current value and
|
||||
publishes it to the entity's state topic. Also publishes availability
|
||||
(online/offline) for each device.
|
||||
|
||||
No-op if MQTT / discovery is not enabled or the client is not connected.
|
||||
All errors are caught internally — this function never raises.
|
||||
"""
|
||||
from app.config import get_settings
|
||||
settings = build_runtime_settings(session, get_settings())
|
||||
if not _should_publish(settings):
|
||||
logger.debug("publish_states: skipped (MQTT/Discovery not enabled or not connected)")
|
||||
return
|
||||
|
||||
state_prefix = settings.ha_state_topic_prefix
|
||||
|
||||
try:
|
||||
catalog = build_catalog(session)
|
||||
except Exception:
|
||||
logger.exception("publish_states: failed to build catalog; aborting")
|
||||
return
|
||||
|
||||
# Collect enabled entries and publish
|
||||
for entry in catalog:
|
||||
if not entry.enabled:
|
||||
continue
|
||||
entity = entry.entity
|
||||
try:
|
||||
_publish_entity_state(entity, state_prefix, session)
|
||||
except Exception:
|
||||
logger.exception(
|
||||
"publish_states: error publishing state for %r; continuing", entity.key
|
||||
)
|
||||
|
||||
|
||||
def _publish_entity_state(
|
||||
entity: ExposableEntity,
|
||||
prefix: str,
|
||||
session: Session,
|
||||
) -> None:
|
||||
"""Publish one entity's current state value to its state topic.
|
||||
|
||||
Also publishes the availability topic for ``binary_sensor`` "online" entities.
|
||||
"""
|
||||
state_t = _state_topic(entity, prefix)
|
||||
device_uuid = entity.device.identifiers[1]
|
||||
|
||||
if entity.component == "binary_sensor" and "online" in entity.key:
|
||||
# The online sensor represents device availability.
|
||||
# Ask the value_getter for the current ON/OFF string, then also publish
|
||||
# the shared availability topic (online/offline).
|
||||
raw_value = None
|
||||
if entity.value_getter is not None:
|
||||
try:
|
||||
raw_value = entity.value_getter(session)
|
||||
except Exception:
|
||||
logger.exception("value_getter raised for online entity %r", entity.key)
|
||||
# Default to offline when no reading is available.
|
||||
online = (raw_value == "ON")
|
||||
avail_payload = "online" if online else "offline"
|
||||
avail_topic = _availability_topic(device_uuid, prefix)
|
||||
mqtt_manager.publish(avail_topic, avail_payload, retain=False)
|
||||
# The state of the binary_sensor itself
|
||||
state_payload = "ON" if online else "OFF"
|
||||
mqtt_manager.publish(state_t, state_payload, retain=False)
|
||||
else:
|
||||
# Regular sensor: call value_getter(session) if available.
|
||||
value = None
|
||||
if entity.value_getter is not None:
|
||||
try:
|
||||
value = entity.value_getter(session)
|
||||
except Exception:
|
||||
logger.exception("value_getter raised for entity %r", entity.key)
|
||||
|
||||
if value is None:
|
||||
logger.debug("publish_states: no value for %r — skipping state publish", entity.key)
|
||||
return
|
||||
|
||||
mqtt_manager.publish(state_t, str(value), retain=False)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Public: per-device state push (called by modbus_poll after each poll)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def publish_device_state(session: Session, device: Any) -> None:
|
||||
"""Publish state + availability for all enabled entities of *device*.
|
||||
|
||||
Called by ``modbus_poll.poll_device`` after a successful (or failed) poll.
|
||||
Publishes:
|
||||
- The current availability topic ("online" / "offline") for the device.
|
||||
- The state topic for each **enabled** entity of the device.
|
||||
|
||||
No-op if MQTT / discovery is not enabled or the client is not connected.
|
||||
All errors are caught internally — never raises.
|
||||
"""
|
||||
from app.config import get_settings
|
||||
settings = build_runtime_settings(session, get_settings())
|
||||
if not _should_publish(settings):
|
||||
return
|
||||
|
||||
state_prefix = settings.ha_state_topic_prefix
|
||||
device_uuid = device.uuid
|
||||
online = bool(device.last_poll_ok)
|
||||
|
||||
# Publish availability topic first.
|
||||
try:
|
||||
avail_topic = _availability_topic(device_uuid, state_prefix)
|
||||
avail_payload = "online" if online else "offline"
|
||||
mqtt_manager.publish(avail_topic, avail_payload, retain=False)
|
||||
except Exception:
|
||||
logger.exception(
|
||||
"publish_device_state: failed to publish availability for device uuid=%s", device_uuid
|
||||
)
|
||||
|
||||
if not online:
|
||||
# Device is offline — no point pushing stale state values.
|
||||
return
|
||||
|
||||
# Publish state for each enabled entity of this device.
|
||||
try:
|
||||
catalog = build_catalog(session)
|
||||
except Exception:
|
||||
logger.exception(
|
||||
"publish_device_state: failed to build catalog for device uuid=%s", device_uuid
|
||||
)
|
||||
return
|
||||
|
||||
for entry in catalog:
|
||||
if not entry.enabled:
|
||||
continue
|
||||
entity = entry.entity
|
||||
# Only process entities belonging to this device.
|
||||
if entity.device.identifiers[1] != device_uuid:
|
||||
continue
|
||||
# Skip the online binary_sensor itself (availability handled above).
|
||||
if entity.component == "binary_sensor" and "online" in entity.key:
|
||||
# Publish the binary_sensor state too.
|
||||
try:
|
||||
state_t = _state_topic(entity, state_prefix)
|
||||
mqtt_manager.publish(state_t, "ON", retain=False)
|
||||
except Exception:
|
||||
logger.exception(
|
||||
"publish_device_state: error publishing online sensor state for %r",
|
||||
entity.key,
|
||||
)
|
||||
continue
|
||||
|
||||
# Regular sensor — call value_getter(session).
|
||||
try:
|
||||
value = None
|
||||
if entity.value_getter is not None:
|
||||
value = entity.value_getter(session)
|
||||
if value is None:
|
||||
continue
|
||||
state_t = _state_topic(entity, state_prefix)
|
||||
mqtt_manager.publish(state_t, str(value), retain=False)
|
||||
except Exception:
|
||||
logger.exception(
|
||||
"publish_device_state: error publishing state for entity %r", entity.key
|
||||
)
|
||||
|
||||
|
||||
def clear_device_discovery(session: Session, device_uuid: str) -> None:
|
||||
"""Best-effort: send empty retained payloads to all HA Discovery config topics
|
||||
for the given device, effectively removing the device's entities from HA.
|
||||
|
||||
This must be called **before** the device rows are deleted from the DB, so
|
||||
that ``build_catalog`` can still enumerate the device's entities.
|
||||
|
||||
Behaviour
|
||||
---------
|
||||
- No-op if MQTT / HA Discovery is not enabled or the MQTT client is not
|
||||
connected.
|
||||
- All exceptions are caught internally; this function never raises.
|
||||
The caller proceeds with the DB deletion regardless of MQTT outcome.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
session:
|
||||
Active SQLAlchemy session (device must still exist in DB at call time).
|
||||
device_uuid:
|
||||
UUID string of the device being deleted.
|
||||
"""
|
||||
from app.config import get_settings
|
||||
settings = build_runtime_settings(session, get_settings())
|
||||
if not _should_publish(settings):
|
||||
logger.debug(
|
||||
"clear_device_discovery: skipped (MQTT/Discovery not enabled or not connected)"
|
||||
)
|
||||
return
|
||||
|
||||
discovery_prefix = settings.ha_discovery_prefix
|
||||
state_prefix = settings.ha_state_topic_prefix
|
||||
|
||||
try:
|
||||
catalog = build_catalog(session)
|
||||
except Exception:
|
||||
logger.exception(
|
||||
"clear_device_discovery: failed to build catalog for device uuid=%s; skipping HA cleanup",
|
||||
device_uuid,
|
||||
)
|
||||
return
|
||||
|
||||
cleared = 0
|
||||
for entry in catalog:
|
||||
entity = entry.entity
|
||||
# Only clear entities belonging to this device.
|
||||
if entity.device.identifiers[1] != device_uuid:
|
||||
continue
|
||||
try:
|
||||
topic, _config = build_discovery_payload(entity, discovery_prefix, state_prefix)
|
||||
mqtt_manager.publish(topic, b"", retain=True)
|
||||
cleared += 1
|
||||
logger.debug(
|
||||
"clear_device_discovery: cleared config topic for entity %r → %s",
|
||||
entity.key,
|
||||
topic,
|
||||
)
|
||||
except Exception:
|
||||
logger.exception(
|
||||
"clear_device_discovery: error clearing entity %r; continuing", entity.key
|
||||
)
|
||||
|
||||
logger.info(
|
||||
"clear_device_discovery: cleared %d HA discovery topic(s) for device uuid=%s",
|
||||
cleared,
|
||||
device_uuid,
|
||||
)
|
||||
|
||||
|
||||
def publish_device_offline(session: Session, device_uuid: str) -> None:
|
||||
"""Publish an "offline" availability payload for *device_uuid*.
|
||||
|
||||
Called by ``modbus_poll.poll_device`` when a poll fails, to immediately
|
||||
reflect the device going offline in HA (before the next full state sweep).
|
||||
|
||||
No-op if MQTT / discovery is not enabled or the client is not connected.
|
||||
Never raises.
|
||||
"""
|
||||
from app.config import get_settings
|
||||
settings = build_runtime_settings(session, get_settings())
|
||||
if not _should_publish(settings):
|
||||
return
|
||||
|
||||
try:
|
||||
state_prefix = settings.ha_state_topic_prefix
|
||||
avail_topic = _availability_topic(device_uuid, state_prefix)
|
||||
mqtt_manager.publish(avail_topic, "offline", retain=False)
|
||||
except Exception:
|
||||
logger.exception(
|
||||
"publish_device_offline: failed for device uuid=%s", device_uuid
|
||||
)
|
||||
@@ -0,0 +1,179 @@
|
||||
"""Exponential back-off throttle service for the login endpoint.
|
||||
|
||||
Design
|
||||
------
|
||||
Failures are tracked per (scope, key) in the ``auth_login_throttle`` table:
|
||||
|
||||
* scope ``'ip'`` → key is the client IP address
|
||||
* scope ``'user'`` → key is the username
|
||||
|
||||
On each login attempt the endpoint checks **both** keys and takes the **maximum**
|
||||
remaining wait (in seconds). A wait > 0 means the caller is still inside a
|
||||
back-off window and should receive 429.
|
||||
|
||||
Exponential formula
|
||||
-------------------
|
||||
The first ``N_FREE`` failures are free (no delay). After that::
|
||||
|
||||
wait = min(BACKOFF_CAP_S, BACKOFF_BASE_S * 2 ** (failures - N_FREE))
|
||||
|
||||
Constants (module-level so they can be tuned without touching the algorithm):
|
||||
|
||||
* ``N_FREE`` = 3 — failures before any delay starts
|
||||
* ``BACKOFF_BASE_S`` = 1 — base wait in seconds (1st delayed attempt → 2 s)
|
||||
* ``BACKOFF_CAP_S`` = 900 — hard cap (15 min)
|
||||
|
||||
Example delay schedule:
|
||||
failures 1, 2, 3 → 0 s (free)
|
||||
failures 4 → 2 s
|
||||
failures 5 → 4 s
|
||||
failures 6 → 8 s
|
||||
failures 7 → 16 s
|
||||
…
|
||||
failures 13+ → 900 s (capped)
|
||||
|
||||
Timezone notes
|
||||
--------------
|
||||
The model uses ``DateTime(timezone=True)``. SQLite stores timestamps without
|
||||
timezone info, so when rows are read back the column value may be timezone-naive.
|
||||
We normalise every stored value to UTC-aware via ``_as_utc`` before comparing
|
||||
against ``datetime.now(UTC)``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import math
|
||||
from datetime import UTC, datetime, timedelta
|
||||
|
||||
from sqlalchemy import delete, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.models.auth_throttle import LoginThrottle
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Tunable constants
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
N_FREE: int = 3 # failures before any backoff starts
|
||||
BACKOFF_BASE_S: int = 1 # seconds — wait after the first delayed failure
|
||||
BACKOFF_CAP_S: int = 900 # seconds — maximum wait (15 min)
|
||||
|
||||
# Internal scope labels (keep in sync with model constraint)
|
||||
_SCOPE_IP = "ip"
|
||||
_SCOPE_USER = "user"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Public API
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def check_and_get_wait(session: Session, *, ip: str, username: str) -> int:
|
||||
"""Return the number of seconds the caller must still wait, or 0 if allowed.
|
||||
|
||||
Checks both the IP row and the username row; returns the *larger* remaining
|
||||
wait so that both keys must have their back-off window expire before the
|
||||
caller is permitted to try again.
|
||||
"""
|
||||
now = datetime.now(UTC)
|
||||
wait_ip = _remaining_wait(session, scope=_SCOPE_IP, key=ip, now=now)
|
||||
wait_user = _remaining_wait(session, scope=_SCOPE_USER, key=username, now=now)
|
||||
return max(wait_ip, wait_user)
|
||||
|
||||
|
||||
def register_failure(session: Session, *, ip: str, username: str) -> None:
|
||||
"""Record one login failure for both the IP and the username keys.
|
||||
|
||||
Updates (or creates) the throttle row for each key, increments the failure
|
||||
counter, and recomputes ``next_allowed_at`` using the exponential formula.
|
||||
"""
|
||||
now = datetime.now(UTC)
|
||||
_upsert_failure(session, scope=_SCOPE_IP, key=ip, now=now)
|
||||
_upsert_failure(session, scope=_SCOPE_USER, key=username, now=now)
|
||||
session.commit()
|
||||
|
||||
|
||||
def clear(session: Session, *, ip: str, username: str) -> None:
|
||||
"""Delete the throttle rows for the given IP and username (successful login).
|
||||
|
||||
It is safe to call this even if no rows exist for the given keys.
|
||||
"""
|
||||
session.execute(
|
||||
delete(LoginThrottle).where(
|
||||
(LoginThrottle.scope == _SCOPE_IP) & (LoginThrottle.key == ip)
|
||||
| (LoginThrottle.scope == _SCOPE_USER) & (LoginThrottle.key == username)
|
||||
)
|
||||
)
|
||||
session.commit()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _remaining_wait(session: Session, *, scope: str, key: str, now: datetime) -> int:
|
||||
"""Return the number of seconds that ``key`` in ``scope`` must still wait.
|
||||
|
||||
Returns 0 if the key has no throttle row or if the back-off window has
|
||||
already expired.
|
||||
"""
|
||||
row = session.scalar(
|
||||
select(LoginThrottle)
|
||||
.where(LoginThrottle.scope == scope, LoginThrottle.key == key)
|
||||
.limit(1)
|
||||
)
|
||||
if row is None or row.next_allowed_at is None:
|
||||
return 0
|
||||
|
||||
next_allowed = _as_utc(row.next_allowed_at)
|
||||
if next_allowed <= now:
|
||||
return 0
|
||||
|
||||
remaining = (next_allowed - now).total_seconds()
|
||||
# Round up so callers are never told "0 seconds" while still blocked.
|
||||
return math.ceil(remaining)
|
||||
|
||||
|
||||
def _compute_next_allowed_at(failures: int, now: datetime) -> datetime | None:
|
||||
"""Compute the earliest time the key should be allowed to try again.
|
||||
|
||||
Returns ``None`` for the first ``N_FREE`` failures (no delay).
|
||||
"""
|
||||
if failures <= N_FREE:
|
||||
return None
|
||||
wait_s = min(BACKOFF_CAP_S, BACKOFF_BASE_S * (2 ** (failures - N_FREE)))
|
||||
return now + timedelta(seconds=wait_s)
|
||||
|
||||
|
||||
def _upsert_failure(session: Session, *, scope: str, key: str, now: datetime) -> None:
|
||||
"""Create or update the throttle row for (scope, key) with one more failure."""
|
||||
row = session.scalar(
|
||||
select(LoginThrottle)
|
||||
.where(LoginThrottle.scope == scope, LoginThrottle.key == key)
|
||||
.limit(1)
|
||||
)
|
||||
if row is None:
|
||||
new_failures = 1
|
||||
row = LoginThrottle(
|
||||
scope=scope,
|
||||
key=key,
|
||||
failures=new_failures,
|
||||
first_failed_at=now,
|
||||
last_failed_at=now,
|
||||
next_allowed_at=_compute_next_allowed_at(new_failures, now),
|
||||
)
|
||||
session.add(row)
|
||||
else:
|
||||
row.failures += 1
|
||||
row.last_failed_at = now
|
||||
row.next_allowed_at = _compute_next_allowed_at(row.failures, now)
|
||||
|
||||
|
||||
def _as_utc(value: datetime | None) -> datetime | None:
|
||||
"""Normalise a possibly-naive datetime to UTC-aware."""
|
||||
if value is None:
|
||||
return None
|
||||
if value.tzinfo is None:
|
||||
return value.replace(tzinfo=UTC)
|
||||
return value.astimezone(UTC)
|
||||
@@ -0,0 +1,480 @@
|
||||
"""Service layer for Meter epoch CRUD, swap/close, and time-range lookup.
|
||||
|
||||
All functions accept an explicit SQLAlchemy Session; callers are responsible
|
||||
for committing or rolling back the transaction.
|
||||
|
||||
Design decisions
|
||||
----------------
|
||||
- ``meter_at``: half-open interval ``[started_at, ended_at)`` lookup;
|
||||
returns the meter whose epoch covers *ts* for the given commodity.
|
||||
SQLite naive datetime is normalised via ``_as_utc()`` before comparison.
|
||||
|
||||
- ``declare_meter``: validates that the new ``started_at`` is **not earlier
|
||||
than** the current active meter's ``started_at`` (rejects back-dating below
|
||||
the active meter's own start). Equal timestamps are allowed because the
|
||||
typical "swap now" use-case sets ``started_at`` to the current moment, which
|
||||
coincides with the active meter's ``started_at`` only in degenerate test
|
||||
scenarios — but blocking equal values would make that workflow impossible.
|
||||
The old active meter is closed (``ended_at = started_at``) and a new active
|
||||
meter is opened in the same operation, guaranteeing continuity: the old
|
||||
meter's ``ended_at`` equals the new meter's ``started_at`` (contiguous,
|
||||
no gap, no overlap).
|
||||
|
||||
- ``update_meter``: when ``started_at`` is modified, the service keeps the
|
||||
timeline contiguous by also updating the **previous** meter's ``ended_at``
|
||||
(the one whose ``ended_at`` matched the old ``started_at``) to the new
|
||||
``started_at``. Validation ensures the new ``started_at``:
|
||||
* is strictly after the previous meter's own ``started_at``
|
||||
(cannot push the boundary before the previous meter even started);
|
||||
* is strictly before the current meter's ``ended_at``, if set
|
||||
(cannot push the boundary past where the current meter was already
|
||||
closed).
|
||||
|
||||
- Mutual exclusion (at most one active meter per commodity) is enforced by the
|
||||
service layer; no DB-level unique partial index is added to keep migrations
|
||||
simple and to allow the application to return a meaningful error message.
|
||||
|
||||
SQLite timezone note
|
||||
--------------------
|
||||
SQLite stores ``DateTime(timezone=True)`` columns as naive UTC strings; on
|
||||
read-back they come out as **timezone-naive** datetimes. Wherever this code
|
||||
compares timestamps from the DB against timezone-aware values, it calls
|
||||
``_as_utc()`` to make both sides comparable without tripping on
|
||||
"offset-naive vs offset-aware" TypeErrors.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from datetime import UTC, datetime
|
||||
from typing import Optional
|
||||
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.models.energy import Meter
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Internal helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _as_utc(dt: datetime) -> datetime:
|
||||
"""Return *dt* as a timezone-aware UTC datetime.
|
||||
|
||||
SQLite's ``DateTime(timezone=True)`` column type stores datetimes as naive
|
||||
UTC strings and gives them back as naive datetimes on read. This helper
|
||||
re-attaches the UTC timezone info when it is missing, making cross-origin
|
||||
comparisons safe.
|
||||
"""
|
||||
if dt.tzinfo is None:
|
||||
return dt.replace(tzinfo=UTC)
|
||||
return dt
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Custom exceptions
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class MeterError(ValueError):
|
||||
"""Base class for meter service validation errors."""
|
||||
|
||||
|
||||
class MeterOverlapError(MeterError):
|
||||
"""Raised when a new meter's ``started_at`` would create an overlap or backdate.
|
||||
|
||||
Specifically: the new meter's ``started_at`` must be greater than or equal
|
||||
to the current active meter's ``started_at``. Allowing a value strictly
|
||||
earlier than the active meter's start would mean the new epoch begins before
|
||||
the current one, which is chronologically inconsistent.
|
||||
"""
|
||||
|
||||
|
||||
class MeterIntervalError(MeterError):
|
||||
"""Raised when an ``update_meter`` call would produce an inconsistent interval.
|
||||
|
||||
Examples of inconsistent intervals:
|
||||
- New ``started_at`` ≥ this meter's ``ended_at`` (epoch would be empty/inverted).
|
||||
- New ``started_at`` ≤ the previous meter's own ``started_at`` (previous meter
|
||||
would become empty/inverted after its ``ended_at`` is updated).
|
||||
"""
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Internal query helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _active_meter(session: Session, commodity: str) -> Optional[Meter]:
|
||||
"""Return the current active (``ended_at IS NULL``) meter for *commodity*, or None."""
|
||||
return session.execute(
|
||||
select(Meter)
|
||||
.where(
|
||||
Meter.commodity == commodity,
|
||||
Meter.ended_at.is_(None),
|
||||
)
|
||||
.limit(1)
|
||||
).scalar_one_or_none()
|
||||
|
||||
|
||||
def _meter_before(session: Session, meter: Meter) -> Optional[Meter]:
|
||||
"""Return the meter whose ``ended_at`` equals *meter*'s ``started_at``.
|
||||
|
||||
This is the meter that was closed when *meter* was opened; its ``ended_at``
|
||||
needs to stay equal to *meter*'s ``started_at`` to maintain timeline
|
||||
continuity. Returns None if *meter* is the first epoch for its commodity.
|
||||
|
||||
The lookup compares naive/aware datetimes via string to avoid SQLite timezone
|
||||
quirks: both are formatted as ISO 8601 UTC strings for the WHERE clause.
|
||||
We rely on the fact that the service layer always stores the same timestamp
|
||||
object as both ``prev.ended_at`` and ``new.started_at``, so their string
|
||||
representations are identical.
|
||||
"""
|
||||
# Normalise the target to a tz-aware UTC datetime for comparison.
|
||||
# We scan in Python (rather than SQL) to avoid SQLite naive-vs-aware
|
||||
# mismatch issues when comparing DateTime columns against tz-aware values.
|
||||
target = _as_utc(meter.started_at)
|
||||
|
||||
# Fetch all closed meters of the same commodity and find the one whose
|
||||
# ended_at equals this meter's started_at (the standard contiguous handoff).
|
||||
candidates = session.execute(
|
||||
select(Meter)
|
||||
.where(
|
||||
Meter.commodity == meter.commodity,
|
||||
Meter.ended_at.is_not(None),
|
||||
)
|
||||
).scalars().all()
|
||||
|
||||
for candidate in candidates:
|
||||
if candidate.id == meter.id:
|
||||
continue
|
||||
candidate_ended = _as_utc(candidate.ended_at)
|
||||
if candidate_ended == target:
|
||||
return candidate
|
||||
|
||||
return None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Core service functions
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def meter_at(
|
||||
session: Session,
|
||||
ts: datetime,
|
||||
commodity: str = "electricity",
|
||||
) -> Optional[Meter]:
|
||||
"""Return the meter epoch that covers *ts* for *commodity*.
|
||||
|
||||
A meter covers *ts* when:
|
||||
``started_at ≤ ts`` AND (``ended_at IS NULL`` OR ``ts < ended_at``)
|
||||
|
||||
This is the standard half-open interval ``[started_at, ended_at)`` lookup,
|
||||
consistent with ``active_contract_version_at`` in ``contracts.py``.
|
||||
|
||||
Returns ``None`` when no meter covers *ts* (e.g. before the first epoch
|
||||
was declared, or after a gap).
|
||||
|
||||
Parameters
|
||||
----------
|
||||
session:
|
||||
Active SQLAlchemy session (read-only usage).
|
||||
ts:
|
||||
UTC datetime to look up.
|
||||
commodity:
|
||||
Energy commodity (default ``"electricity"``).
|
||||
|
||||
Returns
|
||||
-------
|
||||
Meter | None
|
||||
|
||||
Implementation note — why the upper bound is pushed into SQL
|
||||
------------------------------------------------------------
|
||||
When ``declare_meter`` is called with ``started_at == active.started_at``
|
||||
(the "equal-timestamp swap" allowed by §3.5), the old meter is closed as a
|
||||
zero-width epoch ``[T0, T0)`` and the new active meter also has
|
||||
``started_at == T0``. Two rows share the same ``started_at``; SQLite's
|
||||
row-ordering for ``ORDER BY started_at DESC LIMIT 1`` is then determined by
|
||||
rowid (i.e. insertion order), which returns the older zero-width row first.
|
||||
|
||||
If the upper bound were checked in Python *after* fetching that one row, the
|
||||
condition ``ts < ended_at`` would be False for **any** ``ts >= T0`` (because
|
||||
``ended_at == T0``), causing the function to return ``None`` and making the
|
||||
new active meter permanently invisible.
|
||||
|
||||
Pushing both bounds into the SQL ``WHERE`` clause eliminates the ambiguity:
|
||||
the zero-width row is excluded by ``ts < ended_at`` before ``LIMIT 1`` is
|
||||
applied, so only the genuinely covering row survives.
|
||||
|
||||
SQLite naive-vs-aware datetime note: SQLAlchemy's SQLite dialect strips the
|
||||
``tzinfo`` from aware datetimes when binding parameters (it does *not*
|
||||
convert to UTC first). Since all datetimes in this codebase are UTC (either
|
||||
naive-UTC from the DB or aware-UTC from ``datetime.now(UTC)``), stripping
|
||||
the tzinfo leaves the wall-clock value unchanged and comparisons remain
|
||||
correct. This is consistent with how ``contracts.py`` handles the same
|
||||
situation (see OBS 2 in the M7-T02 review notes).
|
||||
"""
|
||||
# Push both the lower and upper bounds into the SQL WHERE clause so that
|
||||
# zero-width epochs (ended_at == started_at) are excluded *before* LIMIT 1
|
||||
# is applied. This prevents an equal-timestamp swap from making the new
|
||||
# active meter invisible (see implementation note above).
|
||||
stmt = (
|
||||
select(Meter)
|
||||
.where(
|
||||
Meter.commodity == commodity,
|
||||
Meter.started_at <= ts,
|
||||
(Meter.ended_at.is_(None)) | (Meter.ended_at > ts),
|
||||
)
|
||||
.order_by(Meter.started_at.desc())
|
||||
.limit(1)
|
||||
)
|
||||
return session.execute(stmt).scalar_one_or_none()
|
||||
|
||||
|
||||
def declare_meter(
|
||||
session: Session,
|
||||
*,
|
||||
label: str,
|
||||
started_at: datetime,
|
||||
reason: str,
|
||||
commodity: str = "electricity",
|
||||
note: Optional[str] = None,
|
||||
) -> Meter:
|
||||
"""Declare a meter swap or initial meter epoch.
|
||||
|
||||
Closes the current active meter for *commodity* (if one exists) by setting
|
||||
its ``ended_at`` to *started_at*, then opens a new active meter. The
|
||||
resulting timeline is **contiguous**: old meter's ``ended_at`` equals new
|
||||
meter's ``started_at``.
|
||||
|
||||
If there is no current active meter (first-ever declaration for this
|
||||
commodity), the new meter is simply opened without closing anything.
|
||||
|
||||
Validation
|
||||
----------
|
||||
- If a current active meter exists, *started_at* must be **≥** that meter's
|
||||
own ``started_at``. A value strictly earlier would place the new epoch
|
||||
entirely before the current active meter, which is a chronological
|
||||
contradiction ("back-dating before the active epoch's start"). Raises
|
||||
``MeterOverlapError`` when this constraint is violated.
|
||||
|
||||
Note: equal timestamps (``started_at == active.started_at``) are allowed
|
||||
because that scenario effectively replaces the current meter at the same
|
||||
logical moment (e.g. correcting a mis-entry), which is a valid use-case.
|
||||
The old meter is then closed with ``ended_at == started_at`` (a zero-width
|
||||
epoch), which is intentional and auditable.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
session:
|
||||
Active SQLAlchemy session. Caller must commit after this returns.
|
||||
label:
|
||||
Human-readable label for the new meter.
|
||||
started_at:
|
||||
UTC datetime at which this meter epoch starts. May be in the past.
|
||||
reason:
|
||||
Why this epoch was created (``"initial"``, ``"meter_swap"``,
|
||||
``"home_move"``, or ``"other"``).
|
||||
commodity:
|
||||
Energy commodity (default ``"electricity"``).
|
||||
note:
|
||||
Optional free-form note.
|
||||
|
||||
Returns
|
||||
-------
|
||||
Meter
|
||||
The newly created, not-yet-committed active meter.
|
||||
|
||||
Raises
|
||||
------
|
||||
MeterOverlapError
|
||||
If *started_at* is strictly earlier than the current active meter's
|
||||
``started_at`` (chronological backdate below the active epoch's start).
|
||||
"""
|
||||
active = _active_meter(session, commodity)
|
||||
|
||||
if active is not None:
|
||||
# Reject back-dating: new started_at must be ≥ current active's started_at.
|
||||
if _as_utc(started_at) < _as_utc(active.started_at):
|
||||
raise MeterOverlapError(
|
||||
f"New meter started_at ({started_at.isoformat()}) must be ≥ the current "
|
||||
f"active {commodity!r} meter's started_at "
|
||||
f"({active.started_at.isoformat()}). "
|
||||
"Declare a started_at on or after the active meter's start to avoid "
|
||||
"a chronologically inconsistent epoch ordering."
|
||||
)
|
||||
# Close the current active meter at the swap point (contiguous handoff).
|
||||
active.ended_at = started_at
|
||||
logger.info(
|
||||
"Closed active %r meter id=%d (ended_at=%s)",
|
||||
commodity,
|
||||
active.id,
|
||||
started_at.isoformat(),
|
||||
)
|
||||
|
||||
now = datetime.now(UTC)
|
||||
new_meter = Meter(
|
||||
label=label,
|
||||
commodity=commodity,
|
||||
started_at=started_at,
|
||||
ended_at=None,
|
||||
reason=reason,
|
||||
note=note,
|
||||
created_at=now,
|
||||
)
|
||||
session.add(new_meter)
|
||||
|
||||
logger.info(
|
||||
"Declared new %r meter %r (started_at=%s, reason=%s)",
|
||||
commodity,
|
||||
label,
|
||||
started_at.isoformat(),
|
||||
reason,
|
||||
)
|
||||
return new_meter
|
||||
|
||||
|
||||
def list_meters(
|
||||
session: Session,
|
||||
commodity: Optional[str] = None,
|
||||
) -> list[Meter]:
|
||||
"""List all meter epochs, ordered by started_at ascending.
|
||||
|
||||
Active meters (``ended_at IS NULL``) sort naturally to the end of the
|
||||
timeline since they have the latest ``started_at``. Within a single
|
||||
commodity, the ascending ``started_at`` order reflects the historical
|
||||
sequence of installed meters.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
session:
|
||||
Active SQLAlchemy session (read-only usage).
|
||||
commodity:
|
||||
If provided, filter to this commodity only. If ``None``, return all
|
||||
meters across all commodities.
|
||||
|
||||
Returns
|
||||
-------
|
||||
list[Meter]
|
||||
Meters ordered by (started_at ASC).
|
||||
"""
|
||||
stmt = select(Meter).order_by(Meter.started_at.asc())
|
||||
if commodity is not None:
|
||||
stmt = stmt.where(Meter.commodity == commodity)
|
||||
|
||||
return list(session.execute(stmt).scalars().all())
|
||||
|
||||
|
||||
def update_meter(
|
||||
session: Session,
|
||||
meter: Meter,
|
||||
*,
|
||||
label: Optional[str] = None,
|
||||
note: Optional[str] = None,
|
||||
started_at: Optional[datetime] = None,
|
||||
) -> Meter:
|
||||
"""Update a meter's mutable fields (label, note, started_at).
|
||||
|
||||
Passing ``None`` for a field leaves it unchanged. At least one keyword
|
||||
argument must be non-``None``; calling with all-``None`` is a no-op but
|
||||
is not an error.
|
||||
|
||||
Updating ``started_at`` (retroactive correction)
|
||||
------------------------------------------------
|
||||
When ``started_at`` is provided the service maintains **timeline
|
||||
continuity** across the adjacent meter boundaries:
|
||||
|
||||
1. **Previous meter's ``ended_at``** — if the meter immediately before
|
||||
this one has ``ended_at == meter.started_at`` (the standard contiguous
|
||||
handoff), its ``ended_at`` is updated to the new ``started_at`` so the
|
||||
boundary between the two epochs stays seamless.
|
||||
|
||||
2. **Validation** — the new ``started_at`` is checked for consistency:
|
||||
a. It must be **strictly after** the previous meter's own ``started_at``
|
||||
(otherwise the previous meter's epoch would collapse to zero or
|
||||
invert).
|
||||
b. It must be **strictly before** this meter's ``ended_at`` (if set),
|
||||
so this meter's epoch remains non-empty.
|
||||
|
||||
Note: triggering a billing recompute (``recompute_range``) after a
|
||||
retroactive ``started_at`` change is **out of scope** for this service
|
||||
layer; that is the API layer's responsibility (M7-T05).
|
||||
|
||||
Parameters
|
||||
----------
|
||||
session:
|
||||
Active SQLAlchemy session. Caller must commit after this returns.
|
||||
meter:
|
||||
The ``Meter`` ORM instance to update (already loaded from the session).
|
||||
label:
|
||||
New human-readable label; ``None`` = keep existing.
|
||||
note:
|
||||
New free-form note; ``None`` = keep existing.
|
||||
started_at:
|
||||
New start timestamp for this meter epoch; ``None`` = keep existing.
|
||||
|
||||
Returns
|
||||
-------
|
||||
Meter
|
||||
The updated ``Meter`` instance (not yet committed).
|
||||
|
||||
Raises
|
||||
------
|
||||
MeterIntervalError
|
||||
If the new ``started_at`` would produce an invalid (empty or inverted)
|
||||
epoch for this meter or the immediately preceding one.
|
||||
"""
|
||||
if label is not None:
|
||||
meter.label = label
|
||||
logger.info("Updated meter id=%d label=%r", meter.id, label)
|
||||
|
||||
if note is not None:
|
||||
meter.note = note
|
||||
logger.info("Updated meter id=%d note=%r", meter.id, note)
|
||||
|
||||
if started_at is not None:
|
||||
old_started_at = meter.started_at
|
||||
|
||||
# --- Validate upper bound: new started_at must be < this meter's ended_at (if set).
|
||||
if meter.ended_at is not None:
|
||||
if _as_utc(started_at) >= _as_utc(meter.ended_at):
|
||||
raise MeterIntervalError(
|
||||
f"New started_at ({started_at.isoformat()}) must be strictly before "
|
||||
f"this meter's ended_at ({meter.ended_at.isoformat()}). "
|
||||
"The meter epoch would become empty or inverted."
|
||||
)
|
||||
|
||||
# --- Find the immediately preceding meter (its ended_at == meter's old started_at).
|
||||
prev = _meter_before(session, meter)
|
||||
|
||||
# --- Validate lower bound: new started_at must be strictly after prev's started_at.
|
||||
if prev is not None:
|
||||
if _as_utc(started_at) <= _as_utc(prev.started_at):
|
||||
raise MeterIntervalError(
|
||||
f"New started_at ({started_at.isoformat()}) must be strictly after "
|
||||
f"the previous meter's started_at ({prev.started_at.isoformat()}). "
|
||||
"Moving the boundary that far back would collapse the previous meter's epoch."
|
||||
)
|
||||
# Maintain continuity: update the previous meter's ended_at to match the new start.
|
||||
prev.ended_at = started_at
|
||||
logger.info(
|
||||
"Updated previous meter id=%d ended_at=%s (boundary shift from %s)",
|
||||
prev.id,
|
||||
started_at.isoformat(),
|
||||
old_started_at.isoformat(),
|
||||
)
|
||||
|
||||
meter.started_at = started_at
|
||||
logger.info(
|
||||
"Updated meter id=%d started_at=%s (was %s)",
|
||||
meter.id,
|
||||
started_at.isoformat(),
|
||||
old_started_at.isoformat(),
|
||||
)
|
||||
|
||||
return meter
|
||||
@@ -0,0 +1,210 @@
|
||||
"""Modbus polling service — periodic read-and-store for enabled Modbus devices.
|
||||
|
||||
Design notes
|
||||
------------
|
||||
- ``poll_device`` loads the device's YAML profile, calls the driver to bulk-read
|
||||
all register blocks in a single TCP connection, decodes the raw registers into
|
||||
an engineering-value dict via the profile, and persists one ``ModbusReading``
|
||||
row. It also stamps ``last_poll_at`` / ``last_poll_ok`` on the device.
|
||||
- All exceptions are caught and logged inside ``poll_device``; the function
|
||||
never re-raises. This prevents a single broken device from aborting the
|
||||
sweep for all other devices.
|
||||
- ``poll_all_enabled_devices`` respects per-device ``poll_interval_s``: it skips
|
||||
a device whose ``last_poll_at`` is recent enough. This lets the APScheduler
|
||||
job run on a short base tick (e.g. 5 s) while each device self-regulates its
|
||||
own cadence.
|
||||
- The global ``modbus_polling_enabled`` flag from ``Settings`` (merged with any
|
||||
DB overrides via ``build_runtime_settings``) gates the entire sweep as a
|
||||
no-op. T08 will expose this flag in CONFIG_FIELDS/UI; for now it is
|
||||
controlled by the ``MODBUS_POLLING_ENABLED`` env var (or the default True).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from datetime import UTC, datetime
|
||||
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.config import Settings
|
||||
from app.integrations.modbus import driver, profiles
|
||||
from app.models.modbus import ModbusDevice, ModbusReading
|
||||
from app.services.config_page import build_runtime_settings
|
||||
from app.services.ha_discovery import publish_device_offline, publish_device_state
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Base polling tick in seconds (used by APScheduler; each device checks its own interval).
|
||||
BASE_POLL_TICK_SECONDS = 5
|
||||
|
||||
|
||||
def poll_device(session: Session, device: ModbusDevice) -> ModbusReading | None:
|
||||
"""Read one enabled Modbus device, decode, and persist a reading row.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
session:
|
||||
Open SQLAlchemy session. The caller is responsible for session
|
||||
lifecycle (open / commit / close).
|
||||
device:
|
||||
``ModbusDevice`` ORM instance to poll.
|
||||
|
||||
Returns
|
||||
-------
|
||||
ModbusReading | None
|
||||
The newly-persisted reading on success, or ``None`` on any failure.
|
||||
Failure is logged and the device's ``last_poll_ok`` is set to ``False``.
|
||||
"""
|
||||
now = datetime.now(UTC)
|
||||
try:
|
||||
profile = profiles.load_profile(device.profile)
|
||||
registers = driver.read_blocks(
|
||||
device.host,
|
||||
device.port,
|
||||
device.unit_id,
|
||||
[{"start": b.start, "count": b.count} for b in profile.blocks],
|
||||
function_code=profile.function_code,
|
||||
)
|
||||
payload = profiles.decode(profile, registers)
|
||||
|
||||
reading = ModbusReading(
|
||||
device_id=device.id,
|
||||
recorded_at=now,
|
||||
payload=payload,
|
||||
)
|
||||
session.add(reading)
|
||||
|
||||
device.last_poll_at = now
|
||||
device.last_poll_ok = True
|
||||
session.commit()
|
||||
|
||||
logger.debug(
|
||||
"Polled device %r (id=%d): %d metrics recorded at %s",
|
||||
device.friendly_name,
|
||||
device.id,
|
||||
len(payload),
|
||||
now.isoformat(),
|
||||
)
|
||||
|
||||
# Best-effort: push MQTT state after successful poll.
|
||||
# Must never let MQTT errors crash the poll loop.
|
||||
try:
|
||||
publish_device_state(session, device)
|
||||
except Exception: # noqa: BLE001
|
||||
logger.exception(
|
||||
"poll_device: MQTT state publish failed for device %r (id=%d); "
|
||||
"continuing (poll itself succeeded)",
|
||||
device.friendly_name,
|
||||
device.id,
|
||||
)
|
||||
|
||||
return reading
|
||||
|
||||
except Exception: # noqa: BLE001
|
||||
logger.exception(
|
||||
"Failed to poll device %r (id=%d, host=%s:%d, unit_id=%d)",
|
||||
device.friendly_name,
|
||||
device.id,
|
||||
device.host,
|
||||
device.port,
|
||||
device.unit_id,
|
||||
)
|
||||
# Rollback first — if session.add(reading) already ran in the success
|
||||
# branch before commit() failed, that pending ModbusReading must be
|
||||
# discarded. Without rollback it would remain staged and could be
|
||||
# flushed/committed by a subsequent operation (even for a different
|
||||
# device), producing a spurious reading row for a poll that failed.
|
||||
# Rollback also expires all ORM objects; device fields are re-set below.
|
||||
try:
|
||||
session.rollback()
|
||||
except Exception: # noqa: BLE001
|
||||
logger.exception(
|
||||
"session.rollback() failed for device id=%d; session may be unusable", device.id
|
||||
)
|
||||
try:
|
||||
device.last_poll_at = now
|
||||
device.last_poll_ok = False
|
||||
session.commit()
|
||||
except Exception: # noqa: BLE001
|
||||
logger.exception(
|
||||
"Also failed to persist last_poll_ok=False for device id=%d", device.id
|
||||
)
|
||||
|
||||
# Best-effort: publish "offline" availability after failed poll.
|
||||
try:
|
||||
publish_device_offline(session, device.uuid)
|
||||
except Exception: # noqa: BLE001
|
||||
logger.exception(
|
||||
"poll_device: MQTT offline publish failed for device %r (id=%d)",
|
||||
device.friendly_name,
|
||||
device.id,
|
||||
)
|
||||
|
||||
return None
|
||||
|
||||
|
||||
def poll_all_enabled_devices(
|
||||
session: Session,
|
||||
*,
|
||||
bootstrap_settings: Settings,
|
||||
) -> None:
|
||||
"""Sweep all enabled Modbus devices and poll each one whose interval has elapsed.
|
||||
|
||||
Global kill-switch: if ``build_runtime_settings(...).modbus_polling_enabled``
|
||||
is ``False``, this function returns immediately without touching any device.
|
||||
|
||||
Per-device interval: a device is skipped if its ``last_poll_at`` is not yet
|
||||
``poll_interval_s`` seconds in the past. This allows the APScheduler job to
|
||||
run on a short fixed tick while each device self-regulates its own cadence.
|
||||
|
||||
Exceptions from individual devices are swallowed inside ``poll_device``;
|
||||
this function itself will not raise.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
session:
|
||||
Open SQLAlchemy session.
|
||||
bootstrap_settings:
|
||||
The bootstrap ``Settings`` instance (from ``get_settings()``), used as
|
||||
the base for ``build_runtime_settings`` to pick up any DB overrides.
|
||||
"""
|
||||
try:
|
||||
runtime_settings = build_runtime_settings(session, bootstrap_settings)
|
||||
except Exception: # noqa: BLE001
|
||||
logger.exception("Failed to build runtime settings for Modbus poll; aborting sweep")
|
||||
return
|
||||
|
||||
if not runtime_settings.modbus_polling_enabled:
|
||||
logger.debug("Modbus polling is disabled (modbus_polling_enabled=False); skipping sweep")
|
||||
return
|
||||
|
||||
try:
|
||||
devices = session.execute(
|
||||
select(ModbusDevice).where(ModbusDevice.enabled == True) # noqa: E712
|
||||
).scalars().all()
|
||||
except Exception: # noqa: BLE001
|
||||
logger.exception("Failed to query enabled Modbus devices; aborting sweep")
|
||||
return
|
||||
|
||||
now = datetime.now(UTC)
|
||||
for device in devices:
|
||||
# Respect per-device poll interval: skip if last poll was too recent.
|
||||
if device.last_poll_at is not None:
|
||||
# SQLite may return a naive datetime despite DateTime(timezone=True) —
|
||||
# treat naive values as UTC so the comparison is safe.
|
||||
last_poll = device.last_poll_at
|
||||
if last_poll.tzinfo is None:
|
||||
last_poll = last_poll.replace(tzinfo=UTC)
|
||||
elapsed = (now - last_poll).total_seconds()
|
||||
if elapsed < device.poll_interval_s:
|
||||
logger.debug(
|
||||
"Skipping device %r (id=%d): %.1fs elapsed < %ds interval",
|
||||
device.friendly_name,
|
||||
device.id,
|
||||
elapsed,
|
||||
device.poll_interval_s,
|
||||
)
|
||||
continue
|
||||
|
||||
poll_device(session, device)
|
||||
@@ -0,0 +1,139 @@
|
||||
"""Service layer for fetching and persisting Tibber 15-minute electricity prices.
|
||||
|
||||
``refresh_prices`` is the main entry point. It is designed to be called from a
|
||||
scheduled background job (see ``app/main.py``) and from tests.
|
||||
|
||||
Design decisions
|
||||
----------------
|
||||
- **Guard clause**: if no active contract with ``kind="tibber"`` exists, or if
|
||||
the Tibber API token is empty in the runtime settings, the function is a
|
||||
complete no-op (returns 0) and does not raise. This means the scheduler can
|
||||
call ``refresh_prices`` unconditionally; the service itself decides whether to
|
||||
do anything based on current configuration.
|
||||
- **Upsert idempotency**: rows are matched by ``starts_at`` (the unique
|
||||
constraint on ``tibber_price``). If a row already exists, its price fields
|
||||
and ``fetched_at`` are updated in-place; if it does not exist, a new row is
|
||||
inserted. Running ``refresh_prices`` twice in a row must not double-insert.
|
||||
- **No destructive operations**: this service never deletes ``tibber_price``
|
||||
rows. Only INSERT or UPDATE.
|
||||
- **Exception propagation**: exceptions from the Tibber client are *not*
|
||||
swallowed here. The scheduled job wrapper in ``main.py`` is responsible for
|
||||
catching and logging errors so that a single fetch failure does not crash the
|
||||
scheduler.
|
||||
- **SQLite timezone note**: ``DateTime(timezone=True)`` columns come back as
|
||||
timezone-naive UTC datetimes on read. We store UTC-aware datetimes on write
|
||||
(``starts_at`` from ``PricePoint`` is already UTC-aware; ``fetched_at`` is
|
||||
``datetime.now(UTC)``). Reads elsewhere that compare against these values
|
||||
must normalise accordingly.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from datetime import UTC, datetime
|
||||
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.integrations.tibber.client import fetch_price_range
|
||||
from app.models.energy import EnergyContract, TibberPrice
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _active_tibber_contract_exists(session: Session) -> bool:
|
||||
"""Return True if there is an active contract with kind='tibber'."""
|
||||
row = session.execute(
|
||||
select(EnergyContract).where(
|
||||
EnergyContract.active.is_(True),
|
||||
EnergyContract.kind == "tibber",
|
||||
).limit(1)
|
||||
).scalar_one_or_none()
|
||||
return row is not None
|
||||
|
||||
|
||||
def refresh_prices(session: Session, settings: object) -> int:
|
||||
"""Fetch today-and-tomorrow Tibber prices and upsert them into ``tibber_price``.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
session:
|
||||
An active SQLAlchemy session. The caller is responsible for closing it;
|
||||
this function commits each upserted row individually to keep transactions
|
||||
short.
|
||||
settings:
|
||||
A runtime settings object that exposes ``tibber_api_token`` and
|
||||
``tibber_home_id`` attributes (typically a ``Settings`` or merged
|
||||
runtime-settings instance from ``build_runtime_settings``).
|
||||
|
||||
Returns
|
||||
-------
|
||||
int
|
||||
Number of rows upserted (inserted or updated). Returns 0 for a no-op.
|
||||
|
||||
Notes
|
||||
-----
|
||||
- No-op (returns 0) when:
|
||||
* No active contract exists with ``kind="tibber"``, OR
|
||||
* ``settings.tibber_api_token`` is empty or whitespace.
|
||||
- Exceptions from the Tibber client are *not* caught here; they propagate to
|
||||
the caller (the scheduled job wrapper swallows them and logs).
|
||||
"""
|
||||
token: str = getattr(settings, "tibber_api_token", "") or ""
|
||||
if not token.strip():
|
||||
logger.debug("refresh_prices: tibber_api_token is empty — no-op")
|
||||
return 0
|
||||
|
||||
if not _active_tibber_contract_exists(session):
|
||||
logger.debug("refresh_prices: no active tibber contract — no-op")
|
||||
return 0
|
||||
|
||||
home_id: str = getattr(settings, "tibber_home_id", "") or ""
|
||||
home_id_or_none: str | None = home_id.strip() or None
|
||||
|
||||
logger.info("refresh_prices: fetching Tibber price range (home_id=%r)", home_id_or_none)
|
||||
|
||||
# May raise TibberError or TibberAuthError — let them propagate.
|
||||
price_points = fetch_price_range(token, home_id_or_none)
|
||||
|
||||
if not price_points:
|
||||
logger.info("refresh_prices: API returned zero price points — nothing to upsert")
|
||||
return 0
|
||||
|
||||
fetched_at = datetime.now(UTC)
|
||||
upserted = 0
|
||||
|
||||
for point in price_points:
|
||||
existing = session.execute(
|
||||
select(TibberPrice).where(TibberPrice.starts_at == point.starts_at)
|
||||
).scalar_one_or_none()
|
||||
|
||||
if existing is not None:
|
||||
# Update in-place — price fields may change (e.g. Tibber corrects a
|
||||
# forecast), but starts_at and resolution stay the same.
|
||||
existing.total = point.total
|
||||
existing.energy = point.energy
|
||||
existing.tax = point.tax
|
||||
existing.level = point.level
|
||||
existing.currency = point.currency
|
||||
existing.fetched_at = fetched_at
|
||||
else:
|
||||
session.add(
|
||||
TibberPrice(
|
||||
starts_at=point.starts_at,
|
||||
resolution=point.resolution,
|
||||
total=point.total,
|
||||
energy=point.energy,
|
||||
tax=point.tax,
|
||||
level=point.level,
|
||||
currency=point.currency,
|
||||
fetched_at=fetched_at,
|
||||
)
|
||||
)
|
||||
|
||||
upserted += 1
|
||||
|
||||
session.commit()
|
||||
|
||||
logger.info("refresh_prices: upserted %d price points", upserted)
|
||||
return upserted
|
||||
@@ -0,0 +1,100 @@
|
||||
"""Server local-timezone helpers.
|
||||
|
||||
All time is stored as UTC (no change). This module provides a single,
|
||||
monkeypatch-friendly entry point for converting UTC moments to the server's
|
||||
local timezone — used for business-day boundary calculations (standing charges,
|
||||
effective-from date interpretation, etc.).
|
||||
|
||||
Design goals
|
||||
------------
|
||||
- Zero third-party dependencies (no tzlocal, no pytz).
|
||||
- Testable without depending on CI host timezone:
|
||||
monkeypatch ``local_tz`` to a fixed ``ZoneInfo`` in unit tests.
|
||||
- DST-correct: conversions use ``astimezone(local_tz())`` per instant, so each
|
||||
moment is independently correct even during DST transitions.
|
||||
- Consistent with the existing ``.astimezone()`` pattern already used in
|
||||
``homeassistant_inbound.py`` and ``poo.py``.
|
||||
|
||||
Priority for resolving the local timezone
|
||||
-----------------------------------------
|
||||
1. ``TZ`` environment variable — ``ZoneInfo(os.environ["TZ"])``.
|
||||
Set ``TZ=Europe/Amsterdam`` in the deployment env for correct NL handling.
|
||||
2. System local timezone fallback: ``datetime.now().astimezone().tzinfo``.
|
||||
This matches the behaviour callers already relied on implicitly.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from datetime import date, datetime, timezone
|
||||
from typing import TYPE_CHECKING
|
||||
from zoneinfo import ZoneInfo
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from datetime import tzinfo
|
||||
|
||||
|
||||
def local_tz() -> "tzinfo":
|
||||
"""Return the server's local timezone.
|
||||
|
||||
Resolution order:
|
||||
1. ``TZ`` environment variable (``ZoneInfo(TZ)``). Set
|
||||
``TZ=Europe/Amsterdam`` in production for correct NL/DST handling.
|
||||
2. System local timezone via ``datetime.now().astimezone().tzinfo``.
|
||||
|
||||
**Monkeypatch this function in tests** to get deterministic timezone
|
||||
behaviour regardless of CI host configuration::
|
||||
|
||||
monkeypatch.setattr("app.services.timezone.local_tz",
|
||||
lambda: ZoneInfo("Europe/Amsterdam"))
|
||||
"""
|
||||
tz_env = os.environ.get("TZ", "").strip()
|
||||
if tz_env:
|
||||
return ZoneInfo(tz_env)
|
||||
# System fallback — identical to the .astimezone() pattern already used
|
||||
# in homeassistant_inbound.py and poo.py.
|
||||
return datetime.now().astimezone().tzinfo # type: ignore[return-value]
|
||||
|
||||
|
||||
def to_local(dt: datetime) -> datetime:
|
||||
"""Convert *dt* to the server local timezone.
|
||||
|
||||
Handles three input cases:
|
||||
- timezone-aware (any zone): converted via ``astimezone``.
|
||||
- timezone-naive: assumed UTC (SQLite read-back convention), then converted.
|
||||
|
||||
Returns a timezone-aware datetime in ``local_tz()``.
|
||||
"""
|
||||
if dt.tzinfo is None:
|
||||
dt = dt.replace(tzinfo=timezone.utc)
|
||||
return dt.astimezone(local_tz())
|
||||
|
||||
|
||||
def local_now() -> datetime:
|
||||
"""Return the current moment as a timezone-aware local datetime."""
|
||||
return datetime.now(local_tz())
|
||||
|
||||
|
||||
def local_date(dt: datetime) -> date:
|
||||
"""Return the local calendar date for *dt*.
|
||||
|
||||
Equivalent to ``to_local(dt).date()``. A distinct helper to make
|
||||
call sites readable.
|
||||
"""
|
||||
return to_local(dt).date()
|
||||
|
||||
|
||||
def local_midnight_utc(d: date) -> datetime:
|
||||
"""Return the UTC instant that corresponds to local midnight on *d*.
|
||||
|
||||
Example (Europe/Amsterdam, CEST=UTC+2):
|
||||
local_midnight_utc(date(2026, 6, 25))
|
||||
→ datetime(2026, 6, 24, 22, 0, 0, tzinfo=UTC)
|
||||
|
||||
This is used to convert a local calendar date back to a UTC boundary, e.g.
|
||||
to compute the start/end of a local day for DB queries.
|
||||
"""
|
||||
tz = local_tz()
|
||||
# Build a naive local datetime at midnight, then localize and convert to UTC.
|
||||
local_midnight = datetime(d.year, d.month, d.day, 0, 0, 0, tzinfo=tz)
|
||||
return local_midnight.astimezone(timezone.utc)
|
||||
@@ -0,0 +1,234 @@
|
||||
"""TOTP service: secret management, recovery-code lifecycle, setup/enable/disable (M4-T05).
|
||||
|
||||
State-machine summary
|
||||
---------------------
|
||||
DISABLED (default)
|
||||
totp_secret=None, totp_enabled=False, no recovery codes
|
||||
|
||||
PENDING (after setup, before enable)
|
||||
totp_secret=<base32>, totp_enabled=False, recovery codes stored as hashes
|
||||
- Re-calling setup replaces the pending secret and regenerates recovery codes.
|
||||
|
||||
ENABLED (after enable)
|
||||
totp_secret=<base32>, totp_enabled=True, recovery codes stored as hashes
|
||||
|
||||
After disable:
|
||||
totp_secret=None, totp_enabled=False, recovery codes deleted → back to DISABLED
|
||||
|
||||
|
||||
Recovery-code timing
|
||||
--------------------
|
||||
1. setup → generate 10 plaintext codes, persist their Argon2 hashes immediately,
|
||||
return plaintext to caller (ONLY time).
|
||||
2. enable → verify 6-digit TOTP code; if ok, flip totp_enabled=True.
|
||||
Recovery codes are already in the DB from step 1.
|
||||
3. disable → clear secret, clear enabled flag, delete all recovery codes.
|
||||
|
||||
Security note: ``totp_secret`` is stored as plaintext (same posture as other
|
||||
secrets in this project: protected by file-system permissions on the SQLite
|
||||
database). Recovery codes are stored as Argon2 hashes only.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import secrets
|
||||
import string
|
||||
|
||||
import pyotp
|
||||
from sqlalchemy import delete, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from app.models.auth import AuthUser, RecoveryCode
|
||||
from app.services.auth import hash_password, verify_password
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Number of recovery codes to generate per setup
|
||||
_RECOVERY_CODE_COUNT = 10
|
||||
# Alphabet for each 4-character segment: lowercase letters + digits, no ambiguous chars
|
||||
_CODE_ALPHABET = string.ascii_lowercase + string.digits
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Internal helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _generate_recovery_code() -> str:
|
||||
"""Return a single recovery code in ``xxxx-xxxx`` format."""
|
||||
segment = lambda: "".join(secrets.choice(_CODE_ALPHABET) for _ in range(4)) # noqa: E731
|
||||
return f"{segment()}-{segment()}"
|
||||
|
||||
|
||||
def _verify_totp_code(secret: str, code: str) -> bool:
|
||||
"""Verify a 6-digit TOTP code against the given base-32 secret (±1 window)."""
|
||||
return pyotp.TOTP(secret).verify(code, valid_window=1)
|
||||
|
||||
|
||||
def _delete_recovery_codes(db: Session, *, user_id: int) -> None:
|
||||
"""Delete ALL recovery codes for a user (called on setup re-run and disable)."""
|
||||
db.execute(delete(RecoveryCode).where(RecoveryCode.user_id == user_id))
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Public API
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def setup(
|
||||
db: Session,
|
||||
*,
|
||||
user: AuthUser,
|
||||
issuer: str,
|
||||
) -> tuple[str, str, list[str]]:
|
||||
"""Generate a new pending TOTP secret and recovery codes.
|
||||
|
||||
The secret is stored in ``user.totp_secret`` (``totp_enabled`` stays
|
||||
``False``). Any pre-existing pending secret and recovery codes are
|
||||
replaced atomically.
|
||||
|
||||
Returns
|
||||
-------
|
||||
(secret, otpauth_uri, plaintext_recovery_codes)
|
||||
|
||||
``plaintext_recovery_codes`` are returned **once** here and MUST NOT be
|
||||
stored or returned elsewhere.
|
||||
"""
|
||||
# Generate new TOTP secret
|
||||
new_secret = pyotp.random_base32()
|
||||
otpauth_uri = pyotp.TOTP(new_secret).provisioning_uri(
|
||||
name=user.username,
|
||||
issuer_name=issuer,
|
||||
)
|
||||
|
||||
# Generate plaintext recovery codes
|
||||
plaintext_codes = [_generate_recovery_code() for _ in range(_RECOVERY_CODE_COUNT)]
|
||||
|
||||
# --- Persist atomically ---
|
||||
# 1. Delete any old pending recovery codes (idempotent on re-setup)
|
||||
assert user.id is not None # mypy guard; always set after DB insertion
|
||||
_delete_recovery_codes(db, user_id=user.id)
|
||||
|
||||
# 2. Update the secret (pending — totp_enabled stays False)
|
||||
user.totp_secret = new_secret
|
||||
|
||||
# 3. Persist new recovery codes as Argon2 hashes
|
||||
for plaintext in plaintext_codes:
|
||||
db.add(RecoveryCode(user_id=user.id, code_hash=hash_password(plaintext)))
|
||||
|
||||
db.commit()
|
||||
db.refresh(user)
|
||||
|
||||
logger.info("TOTP setup (pending) for user '%s'; %d recovery codes generated.", user.username, _RECOVERY_CODE_COUNT)
|
||||
return new_secret, otpauth_uri, plaintext_codes
|
||||
|
||||
|
||||
def enable(db: Session, *, user: AuthUser, code: str) -> bool:
|
||||
"""Enable TOTP after the user proves they can generate the correct code.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
code:
|
||||
The current 6-digit TOTP code from the user's authenticator app.
|
||||
|
||||
Returns
|
||||
-------
|
||||
True on success; False if the code is invalid or no secret is pending.
|
||||
"""
|
||||
if not user.totp_secret:
|
||||
logger.info("TOTP enable rejected for '%s': no pending secret.", user.username)
|
||||
return False
|
||||
|
||||
if not _verify_totp_code(user.totp_secret, code):
|
||||
logger.info("TOTP enable rejected for '%s': wrong code.", user.username)
|
||||
return False
|
||||
|
||||
user.totp_enabled = True
|
||||
db.commit()
|
||||
db.refresh(user)
|
||||
logger.info("TOTP enabled for user '%s'.", user.username)
|
||||
return True
|
||||
|
||||
|
||||
def disable(
|
||||
db: Session,
|
||||
*,
|
||||
user: AuthUser,
|
||||
password: str | None = None,
|
||||
code: str | None = None,
|
||||
) -> bool:
|
||||
"""Disable TOTP. Caller must supply exactly one of ``password`` or ``code``.
|
||||
|
||||
On success: ``totp_enabled=False``, ``totp_secret=None``, all recovery
|
||||
codes deleted.
|
||||
|
||||
Returns
|
||||
-------
|
||||
True on success; False if neither credential is valid.
|
||||
"""
|
||||
if not password and not code:
|
||||
logger.info("TOTP disable rejected for '%s': no credential provided.", user.username)
|
||||
return False
|
||||
|
||||
if password:
|
||||
if not verify_password(password, user.password_hash):
|
||||
logger.info("TOTP disable rejected for '%s': wrong password.", user.username)
|
||||
return False
|
||||
elif code:
|
||||
if not user.totp_secret or not _verify_totp_code(user.totp_secret, code):
|
||||
logger.info("TOTP disable rejected for '%s': wrong TOTP code.", user.username)
|
||||
return False
|
||||
|
||||
# Clear TOTP state
|
||||
assert user.id is not None
|
||||
_delete_recovery_codes(db, user_id=user.id)
|
||||
user.totp_enabled = False
|
||||
user.totp_secret = None
|
||||
db.commit()
|
||||
db.refresh(user)
|
||||
logger.info("TOTP disabled for user '%s'.", user.username)
|
||||
return True
|
||||
|
||||
|
||||
def verify_totp_code(user: AuthUser, code: str) -> bool:
|
||||
"""Verify a 6-digit TOTP code for a user.
|
||||
|
||||
Returns True if the code is valid (±1 time window), False otherwise.
|
||||
The user must have a ``totp_secret`` set; returns False if not.
|
||||
|
||||
This is the public entry-point for T06 login two-factor verification.
|
||||
"""
|
||||
if not user.totp_secret:
|
||||
return False
|
||||
return _verify_totp_code(user.totp_secret, code)
|
||||
|
||||
|
||||
def verify_recovery_code(db: Session, *, user: AuthUser, code: str) -> bool:
|
||||
"""Verify and consume a one-time recovery code.
|
||||
|
||||
Finds the first unused recovery code whose hash matches ``code``, marks it
|
||||
as consumed (sets ``used_at``), and returns ``True``. Returns ``False`` if
|
||||
no matching unused code exists.
|
||||
|
||||
This function is provided for T06 (login two-factor) but lives here so the
|
||||
TOTP service owns all recovery-code logic.
|
||||
"""
|
||||
from datetime import UTC, datetime
|
||||
|
||||
unused = db.execute(
|
||||
select(RecoveryCode).where(
|
||||
RecoveryCode.user_id == user.id,
|
||||
RecoveryCode.used_at.is_(None),
|
||||
)
|
||||
).scalars().all()
|
||||
|
||||
for rc in unused:
|
||||
if verify_password(code, rc.code_hash):
|
||||
rc.used_at = datetime.now(UTC)
|
||||
db.commit()
|
||||
logger.info(
|
||||
"Recovery code consumed for user '%s' (id=%d).", user.username, rc.id
|
||||
)
|
||||
return True
|
||||
|
||||
return False
|
||||
@@ -62,6 +62,8 @@ packaging==26.1
|
||||
# build
|
||||
# pytest
|
||||
# wheel
|
||||
paho-mqtt==2.1.0
|
||||
# via -r requirements.in
|
||||
pip-tools==7.5.3
|
||||
# via -r dev-requirements.in
|
||||
pluggy==1.6.0
|
||||
@@ -78,6 +80,10 @@ pydantic-settings==2.13.1
|
||||
# via -r requirements.in
|
||||
pygments==2.20.0
|
||||
# via pytest
|
||||
pymodbus==3.13.1
|
||||
# via -r requirements.in
|
||||
pyotp==2.10.0
|
||||
# via -r requirements.in
|
||||
pyproject-hooks==1.2.0
|
||||
# via
|
||||
# build
|
||||
|
||||
@@ -20,9 +20,9 @@ services:
|
||||
build: .
|
||||
image: home-automation:dev
|
||||
container_name: home-automation-app-dev
|
||||
# Publish on 8001 for dev. `!override` REPLACES the base ports list instead of
|
||||
# Publish on 8002 for dev. `!override` REPLACES the base ports list instead of
|
||||
# appending to it, so the dev stack does NOT also bind the production 8881.
|
||||
ports: !override
|
||||
- "127.0.0.1:8001:8000"
|
||||
- "127.0.0.1:8002:8000"
|
||||
environment:
|
||||
APP_DATABASE_URL: "sqlite:////app/data/app.db"
|
||||
|
||||
@@ -9,6 +9,7 @@ services:
|
||||
volumes:
|
||||
- ./data:/app/data
|
||||
- ./.env:/app/.env:ro
|
||||
- /etc/localtime:/etc/localtime:ro
|
||||
|
||||
app:
|
||||
container_name: home-automation-app
|
||||
@@ -24,4 +25,5 @@ services:
|
||||
volumes:
|
||||
- ./data:/app/data
|
||||
- ./.env:/app/.env:ro
|
||||
- /etc/localtime:/etc/localtime:ro
|
||||
|
||||
|
||||
@@ -19,31 +19,42 @@
|
||||
|
||||
- `main.py`
|
||||
- FastAPI app factory
|
||||
- lifespan
|
||||
- lifespan(APScheduler 启停、MQTT 客户端起停、连接后触发 HA Discovery 发布;M6 新增 `tibber-refresh` 抓价 job + `energy-cost` 1 分钟计费 tick job;M6 启用时注册 DSMR MQTT 订阅)
|
||||
- 基础路由注册
|
||||
- `config.py`
|
||||
- 环境变量驱动的 settings
|
||||
- 环境变量驱动的 settings(含 M5 新增的 MQTT/HA Discovery/Modbus 配置项;M6 新增 `dsmr_ingest_enabled`、`dsmr_mqtt_topic`、`dsmr_sample_interval_s`、`tibber_api_token`(secret)、`tibber_home_id`)
|
||||
- `db.py`
|
||||
- 统一数据层:一个 `Base`、一个绑定 `app_database_url` 的 cached engine(SQLite WAL)、`get_engine` / `get_session_local` / `reset_db_caches` / `get_db_session`
|
||||
- `dependencies.py`
|
||||
- 通用依赖注入
|
||||
- `api/`
|
||||
- HTTP routes
|
||||
- `api/routes/api/`:JSON API(`/api/*` 前缀),供 React SPA 调用:会话/鉴权、配置读写、数据查询、记录 CRUD
|
||||
- `api/routes/api/`:JSON API(`/api/*` 前缀),供 React SPA 调用:会话/鉴权、配置读写、数据查询、记录 CRUD、Modbus 设备 CRUD + readings + metrics + test(`/api/modbus/*`)、Expose 勾选 + 重发 discovery(`/api/expose`)、MQTT 测试连接(`/api/config/mqtt/test`)、M6 新增合同 CRUD + 版本(`/api/energy/contracts*`)、pricing profile 列表(`/api/energy/profiles`)、价格/费用/汇总/DSMR 最新/重算/Tibber 测试(`/api/energy/prices`、`/api/energy/costs`、`/api/energy/costs/summary`、`/api/energy/dsmr/latest`、`/api/energy/costs/recompute`、`/api/energy/tibber/test`)
|
||||
- 裸 ingestion 端点:`GET /public-ip/check`、`POST /homeassistant/publish`、`POST /poo/record`、`GET /poo/latest`、TickTick OAuth 等
|
||||
- `models/`
|
||||
- SQLAlchemy models
|
||||
- 所有模型(auth / config / public_ip / location / poo)共用同一个 `Base`,均落在单一 `app.db` 中
|
||||
- 所有模型(auth / config / public_ip / location / poo / modbus / expose / energy)共用同一个 `Base`,均落在单一 `app.db` 中
|
||||
- M5 新增:`ModbusDevice`(设备部署层)、`ModbusReading`(通用遥测,JSON payload)、`ExposedEntityToggle`(HA 实体暴露开关)
|
||||
- M6 新增:`DsmrReading`(整帧 DSMR telegram,10s 降采样)、`EnergyContract`(合同头,含 active 标记)、`EnergyContractVersion`(版本/时段,values JSON,只增不改)、`TibberPrice`(15 分钟价缓存,不可变)、`EnergyCostPeriod`(每 15 分钟计量电费,快照价,不可变)
|
||||
- `schemas/`
|
||||
- Pydantic schemas
|
||||
- Pydantic schemas(M5 新增 `modbus.py`、`expose.py`;M6 新增 `energy_contract.py`、`energy.py`)
|
||||
- `services/`
|
||||
- 业务服务层
|
||||
- 当前已迁入 config page 的 DB 持久化逻辑
|
||||
- 当前已迁入 public IPv4 检查、状态持久化与变化通知逻辑
|
||||
- 当前已迁入 SMTP 发信与测试发信逻辑
|
||||
- M5 新增:`modbus_poll.py`(采集 service,逐设备 poll + 落库 + 推 MQTT state)、`ha_discovery.py`(构建 HA Discovery payload、发布 retained config、发布 state)
|
||||
- M6 新增:`tibber_prices.py`(httpx GraphQL 抓 15 分钟价,upsert `tibber_price`,幂等;仅 active=tibber 且 token 存在时运行)、`dsmr_ingest.py`(MQTT handler,整帧 JSON blob + 10s 降采样落库,`source_id` 幂等)、`energy_cost.py`(计费引擎:每 15 分钟寄存器差 × strategy 出价 → `energy_cost_period` 不可变快照;汇总 Σnet + 固定费 − heffingskorting;重算显式 opt-in)
|
||||
- `integrations/`
|
||||
- 外部系统适配层
|
||||
- 当前已迁入 Home Assistant outbound adapter
|
||||
- Home Assistant outbound adapter(REST 通道,原有)
|
||||
- M5 新增:`modbus/`(pymodbus 薄封装:`driver.py` 块读 + float32 解码;`profiles.py` YAML profile 加载/校验/解码;`profiles/sdm120.yaml` SDM120 协议声明)
|
||||
- M5 新增:`mqtt.py`(paho-mqtt 长连接 `MqttManager`:lifespan 起/停、配置变更重连、`publish(topic, payload, retain)`)
|
||||
- M5 新增:`expose.py`(通用 expose 框架:`ExposableEntity`、provider 注册表、`build_catalog`;Modbus provider 从 YAML profile 派生 sensor/binary_sensor 实体目录)
|
||||
- M6 新增:`pricing/`(通用电价层:`profiles.py` pydantic 加载/校验 YAML profile + `validate_values`;`strategies.py` manual/tibber 出价策略注册表;`profiles/manual.yaml` 固定/双费率结构;`profiles/tibber.yaml` 动态电价结构)
|
||||
- M6 新增:`tibber/`(`client.py`:httpx GraphQL 客户端,`priceInfoRange(QUARTER_HOURLY)` 查询,按 `starts_at` 解析,不假设固定节点数)
|
||||
- M6 扩展:`mqtt.py` 新增订阅端(`subscribe(topic, handler)`,`on_connect` 里 subscribe,`on_message` 按 topic 分发,handler 异常吞掉不崩连接)
|
||||
- M6 扩展:`expose.py` 新增 `_energy_cost_provider`(`buy_price_now`、`sell_price_now`、`import_cost_total`/`export_revenue_total` 均 `total_increasing`,反哺 HA Energy)
|
||||
- `static/`
|
||||
- 极简静态资源
|
||||
|
||||
@@ -69,18 +80,94 @@ React SPA 前端(M2 引入)。Vite + React + TypeScript + Mantine,由 Fast
|
||||
|
||||
### `scripts/`
|
||||
|
||||
辅助脚本目录。当前包含 OpenAPI 导出脚本(`export_openapi.py`)与数据层辅助脚本。
|
||||
辅助脚本目录。当前包含:
|
||||
|
||||
- `export_openapi.py`:导出 OpenAPI schema 静态产物
|
||||
- `run_migrations.py`:运行 Alembic migration
|
||||
- `app_db_adopt.py`:App DB 接管 / 初始化
|
||||
- `migrate_legacy_data.py`:一次性历史数据搬迁脚本
|
||||
- `admin_cli.py`:Admin CLI 逃生通道(M4),见下方"登录加固"说明
|
||||
- `modbus_cli.py`(M5):Modbus 手工试读 CLI(`read`/`probe` 两个只读子命令),不依赖 DB,供受控手工验证链路连通性
|
||||
|
||||
### `openapi/`
|
||||
|
||||
OpenAPI schema 静态产物(`openapi.json` / `openapi.yaml`),由 `python scripts/export_openapi.py` 生成,纳入版本控制。前端 codegen 以此为契约源。
|
||||
|
||||
## 登录加固(M4)
|
||||
|
||||
M4 在基础 Argon2 + server-side session 鉴权之上叠加了三层防御:
|
||||
|
||||
**防爆破 / 指数退避**:`app/services/login_throttle.py` 按 client IP 与 username 双键记失败计数,失败超过 3 次后指数增长等待时间(最长 15 分钟),`POST /api/auth/login` 在退避窗口内直接返回 `429 + Retry-After`,不执行 Argon2 验证。成功登录后清零。退避是延迟而非永久封号;全局开关 `AUTH_LOGIN_THROTTLE_ENABLED`(CONFIG_FIELDS,默认开);反代后需设 `AUTH_TRUST_FORWARDED_FOR=true`(`.env` 部署级,默认 false)。
|
||||
|
||||
**CLI 逃生通道**:`scripts/admin_cli.py`(入口 `python -m scripts.admin_cli`)直连本地 DB,**无需 HTTP 服务运行、无需任何已存凭据**,支持:重置密码(`reset-password`)、解锁退避(`unlock`)、关停 TOTP(`disable-totp`,零凭据最终逃生)、重新发放 TOTP secret(`reissue-totp`)、查看用户列表(`list-admin`)。CLI 只动 auth 行,不触碰用户数据表。
|
||||
|
||||
**可选 TOTP 二次验证**:admin 可在设置页自选启用 RFC 6238 TOTP。启用后登录为两步(密码 → 6 位动态码或一次性恢复码);不启用维持纯密码。TOTP secret 明文存库(与其他 secret 一致,靠文件权限保护);恢复码以 Argon2 哈希存储,使用后消费(一次性)。后端使用 `pyotp`,二维码由前端 `qrcode.react` 渲染。issuer 标签由 `AUTH_TOTP_ISSUER`(`.env` 部署级)配置,默认回退 `app_name`。
|
||||
|
||||
详细说明:[`docs/auth.md`](./auth.md)
|
||||
|
||||
## M5 — 通用 Modbus 采集链路与 MQTT 通道
|
||||
|
||||
### Modbus 采集链路
|
||||
|
||||
```
|
||||
YAML profile(协议知识) modbus_device 行(部署信息,DB)
|
||||
- 寄存器块/地址/类型 - host / port(网关 IP)
|
||||
- 每量:key/unit/device_class - unit_id(Modbus slave 地址)
|
||||
- ha_component - friendly_name / profile / poll_interval_s / enabled
|
||||
│ │
|
||||
└────────────┬───────────────────────┘
|
||||
│ APScheduler job(轮询所有 enabled 设备)
|
||||
│ driver.py: ModbusTcpClient → FC04 块读 → 大端 float32 解码
|
||||
│ profiles.py: decode(profile, registers) → dict[key → value]
|
||||
▼
|
||||
modbus_reading 行
|
||||
device_id · recorded_at · payload(JSON)
|
||||
{"voltage": 230.2, "current": 1.3, "active_power": 295.0, ...}
|
||||
```
|
||||
|
||||
**关键设计决策**:
|
||||
- 命名分层:存储/采集/API 全部通用 `modbus_*`(`/api/modbus/devices`);面向用户的领域视图叫 **Energy**(第一个)。接入新设备型号只需新增 YAML profile,不改表/不改 API。
|
||||
- 读数为 JSON `payload`(无固定列),SQLite `json_extract` 在 DB 端做 AVG/GROUP BY(被 `(device_id, recorded_at)` 索引圈住)。
|
||||
- FK `ON DELETE RESTRICT`:有读数的设备拒删,引导改为 `enabled=false`。
|
||||
- 设备 `uuid`(uuid4 内部生成)是**稳定身份锚点**:API 路径键、HA Discovery `unique_id` 来源,不随 friendly_name 改变。
|
||||
|
||||
### 第二条 MQTT 通道(HA Discovery 发布)
|
||||
|
||||
与已有 REST 通道(`app/integrations/homeassistant.py`、`POST /homeassistant/publish`)**并行、不冲突**:
|
||||
|
||||
```
|
||||
MQTT 通道(M5 新增):
|
||||
paho-mqtt MqttManager(lifespan 长连接,配置变更可重连)
|
||||
│
|
||||
├─► Discovery config(retained)
|
||||
│ topic: <prefix>/<component>/<node>/<object>/config
|
||||
│ 内容:device 块(identifiers=uuid)、state_topic、unique_id(uuid+key)、
|
||||
│ name(friendly_name)、device_class、unit_of_measurement、availability
|
||||
│ 时机:连接成功时 / 目录或勾选变更时(全量重发);
|
||||
│ 取消勾选时发空 payload(清除 entity)
|
||||
│
|
||||
└─► State / Availability(非 retained)
|
||||
时机:每次轮询成功后推该设备所有 enabled entity 的最新值;
|
||||
周期兜底 job 重推所有 enabled entity + online topic
|
||||
|
||||
expose 框架:
|
||||
provider 动态产出 ExposableEntity 目录(元数据从 YAML profile 派生)
|
||||
ExposedEntityToggle 表:只存逐 key 开关(default=disabled)
|
||||
build_catalog(session) → 目录 + 勾选状态(合并所有 provider)
|
||||
```
|
||||
|
||||
**HA entity 身份模型(Z2M 语义)**:
|
||||
- `unique_id` = `f"{device.uuid}_{metric.key}"`(稳定,改名不变)
|
||||
- `name` = `friendly_name`(改名重发 discovery,HA 显示名跟着变、历史不丢)
|
||||
- 每设备除各 sensor entity 外,另有 `binary_sensor` `online`(取 `last_poll_ok`)——这是"不止 sensor"的体现
|
||||
|
||||
## 当前约束
|
||||
|
||||
- 当前数据库继续使用 SQLite
|
||||
- ~~当前不引入前后端分离~~ **已退役(M2)**:现为 React SPA + JSON `/api` 层,由 FastAPI 同源托管
|
||||
- 当前不设计 Notion 模块
|
||||
- 当前通知能力仍保持极小范围,不引入独立通知中心或多渠道抽象
|
||||
- Modbus 当前**仅 TCP**(Waveshare RTU↔TCP 网关),**只读**(FC03/04),不写设备寄存器
|
||||
|
||||
## 关于 Notion
|
||||
|
||||
|
||||
+191
-82
@@ -1,120 +1,229 @@
|
||||
# 基础鉴权说明
|
||||
# 鉴权说明
|
||||
|
||||
本文档说明当前 Python 重构项目里已经落地的第一版鉴权基座。
|
||||
|
||||
这一轮只解决:
|
||||
|
||||
- 登录页
|
||||
- 登录 / 登出流程
|
||||
- server-side session
|
||||
- 一个最小受保护页面
|
||||
|
||||
这一轮明确不解决:
|
||||
|
||||
- 完整 config persistence
|
||||
- 完整 config CRUD
|
||||
- 多用户权限系统
|
||||
- OAuth / SSO / RBAC
|
||||
本文档说明当前已落地的鉴权基座(基础 session 鉴权 + M4 登录加固)。
|
||||
|
||||
## 当前 auth 模型
|
||||
|
||||
- 认证方式:`username/password`
|
||||
- 认证方式:`username/password`(可选启用 TOTP 二次验证)
|
||||
- 会话方式:server-side session
|
||||
- 客户端凭据:session cookie
|
||||
- 页面形态:Jinja server-side template
|
||||
|
||||
## 当前持久化
|
||||
## 持久化
|
||||
|
||||
当前新增一个共享 App DB:
|
||||
所有 auth 相关数据存放在单一 App DB(`APP_DATABASE_URL`,默认 `sqlite:///./data/app.db`)中:
|
||||
|
||||
- `APP_DATABASE_URL`
|
||||
- 默认值:`sqlite:///./data/app.db`
|
||||
|
||||
当前 auth 相关数据存放在这个 DB 中:
|
||||
|
||||
- `auth_users`
|
||||
- `auth_sessions`
|
||||
- `app_config`
|
||||
|
||||
当前没有把 auth 数据和 `location` / `poo` DB 混放。
|
||||
|
||||
当前这部分现在也走 Alembic 管理:
|
||||
|
||||
- Alembic 环境:`alembic_app.ini` + `alembic_app/`
|
||||
- 初始化脚本:`python scripts/app_db_adopt.py`
|
||||
|
||||
当前没有 legacy app DB,所以这一版脚本只负责初始化新库,不负责 legacy adoption。
|
||||
|
||||
`app_config` 现在承接运行时配置持久化。
|
||||
|
||||
其中:
|
||||
|
||||
- `.env` 负责 bootstrap / fallback
|
||||
- `app_config` 表负责运行时配置覆盖
|
||||
- 登录密码仍然属于认证数据,使用 Argon2 哈希,不存进 `app_config`
|
||||
- `auth_users`:用户表(含 TOTP 字段 `totp_secret` / `totp_enabled`)
|
||||
- `auth_sessions`:session token 哈希与过期时间
|
||||
- `auth_login_throttle`:登录失败退避状态(按 IP / username 双键)
|
||||
- `auth_recovery_code`:TOTP 恢复码哈希(一次性)
|
||||
- `app_config`:runtime 配置持久化
|
||||
|
||||
## 首次启动与 bootstrap
|
||||
|
||||
如果 auth DB 中还没有任何用户,应用启动时会要求:
|
||||
如果 auth DB 中还没有任何用户,应用启动时会使用:
|
||||
|
||||
- `AUTH_BOOTSTRAP_USERNAME`
|
||||
- `AUTH_BOOTSTRAP_PASSWORD`
|
||||
|
||||
并创建首个 admin 用户。
|
||||
|
||||
当前默认 bootstrap 值就是:
|
||||
创建首个 admin 用户。当前默认 bootstrap 值为:
|
||||
|
||||
- username: `admin`
|
||||
- password: `admin`
|
||||
|
||||
首次登录后,系统会强制要求修改密码。
|
||||
|
||||
如果你希望在首次启动前就覆盖默认值,可以直接设置环境变量:
|
||||
## 基础安全设计
|
||||
|
||||
- `AUTH_BOOTSTRAP_USERNAME`
|
||||
- `AUTH_BOOTSTRAP_PASSWORD`
|
||||
|
||||
建议流程是:
|
||||
|
||||
1. 配好 `.env`
|
||||
2. 运行 `python scripts/app_db_adopt.py`
|
||||
3. 启动应用
|
||||
4. 用 `admin / admin` 首次登录
|
||||
5. 立即修改密码
|
||||
|
||||
## 安全设计
|
||||
|
||||
当前这版已经落实的基础安全点:
|
||||
当前这版已经落实的安全点:
|
||||
|
||||
- 密码不明文存储,使用 Argon2 哈希
|
||||
- session cookie 为 `HttpOnly`
|
||||
- cookie 使用 `SameSite=Lax`
|
||||
- `Secure` cookie 在非 `development` 环境默认开启
|
||||
- 登录表单与登出表单都有基础 CSRF 校验
|
||||
- 写请求(POST/PUT/PATCH/DELETE)需携带 `X-CSRF-Token` header(SameSite=Lax + 自定义 header 纵深防御,无需 per-session token 值比对)
|
||||
- session token 为随机生成,服务端只持久化 token hash
|
||||
- session 有过期时间与显式失效机制
|
||||
- session 有过期时间(默认 12 小时)与显式失效机制
|
||||
|
||||
## 当前受保护范围
|
||||
---
|
||||
|
||||
当前这轮只保护了页面入口:
|
||||
## M4 登录加固
|
||||
|
||||
- `GET /config`
|
||||
- `POST /config`
|
||||
- `POST /config/change-password`
|
||||
- `POST /logout`
|
||||
M4 在基础鉴权之上叠加了三层防御:防爆破/指数退避、CLI 逃生通道、可选 TOTP 二次验证。
|
||||
|
||||
相关流程:
|
||||
### 1. 防爆破 / 指数退避
|
||||
|
||||
- `GET /login`
|
||||
- `POST /login`
|
||||
#### 机制
|
||||
|
||||
未登录访问 `/config` 时会被重定向到 `/login`。
|
||||
登录失败按指数增长延迟,目标是拖垮暴力枚举,同时不因此永久锁定账号。
|
||||
|
||||
## 下一步不在本轮内
|
||||
**退避是延迟(429 + Retry-After),不是永久封号**——单 admin 场景下永久锁会被攻击者反向用来故意打锁,所以退避只增加等待时间,CLI 是最终逃生口。
|
||||
|
||||
后续可以在这个基座上继续做:
|
||||
#### 双键计算
|
||||
|
||||
- 配置页面接入
|
||||
- config persistence
|
||||
- 更细的受保护路由范围
|
||||
- 用户初始化 / 密码轮换的更正式 runbook
|
||||
每次登录请求同时按 **client IP** 和 **username** 各记一套失败计数,本次需等待时间 = 两者退避的**较大值**。
|
||||
|
||||
- 按 IP:堵单点暴力(攻击 IP 越敲越慢,合法用户换 IP 不受影响)
|
||||
- 按 username:全局兜底(即便攻击者分布式换 IP 也有一层保护)
|
||||
|
||||
#### 指数公式
|
||||
|
||||
```
|
||||
N_FREE = 3 — 前 3 次失败不延迟(免费次数)
|
||||
BASE = 1 秒
|
||||
CAP = 900 秒(15 分钟)
|
||||
|
||||
wait = min(CAP, BASE × 2^(failures - N_FREE)) (failures > N_FREE 时生效)
|
||||
```
|
||||
|
||||
延迟示意:
|
||||
|
||||
| 累计失败次数 | 等待时间 |
|
||||
| --- | --- |
|
||||
| 1–3 | 0 秒(免费) |
|
||||
| 4 | 2 秒 |
|
||||
| 5 | 4 秒 |
|
||||
| 6 | 8 秒 |
|
||||
| 7 | 16 秒 |
|
||||
| … | … |
|
||||
| 13+ | 900 秒(封顶) |
|
||||
|
||||
成功登录后,该 IP 和 username 的退避状态**立即清零**。
|
||||
|
||||
#### 登录端点行为(`POST /api/auth/login`)
|
||||
|
||||
1. 先查退避(在窗口内直接 `429`,**不验密码**,节省 Argon2 计算并防止枚举)
|
||||
2. 验密码失败 → 记一次失败(IP + username 各记),返回 `401`
|
||||
3. 密码正确但 TOTP 启用且缺少 `totp_code` → 返回 `401 {totp_required: true}`,**不记失败**(这是正常两步流程的第一步,不是攻击信号)
|
||||
4. 密码正确但 TOTP 验证失败 → 记一次失败,返回 `401`
|
||||
5. 全部通过 → 清零退避状态,发 session cookie
|
||||
|
||||
#### 配置项
|
||||
|
||||
| 配置项 | 类型 | 默认 | 说明 |
|
||||
| --- | --- | --- | --- |
|
||||
| `AUTH_LOGIN_THROTTLE_ENABLED` | bool(CONFIG_FIELDS) | `true` | 全局开关,关闭后整个退避机制 no-op |
|
||||
| `AUTH_TRUST_FORWARDED_FOR` | bool(`.env` 部署级) | `false` | `true` 时取 `X-Forwarded-For` 最左作为 client IP;**反代后必须显式开启**,否则反代 IP 全视为同一 IP,按 IP 退避失效 |
|
||||
|
||||
> **反代部署注意**:`AUTH_TRUST_FORWARDED_FOR` 默认 `false`(直接用 socket IP)。部署在 nginx 等反代后面时,应设 `AUTH_TRUST_FORWARDED_FOR=true`,并确保反代正确设置 `X-Forwarded-For`。
|
||||
|
||||
---
|
||||
|
||||
### 2. CLI 逃生通道
|
||||
|
||||
#### 设计原则
|
||||
|
||||
CLI 通过 **直连本地 DB**(`get_session_local()`)工作,**无需 HTTP 服务器运行、无需任何已存凭据**(密码、恢复码均不需要)。拿到服务器 CLI 权限本身就意味着对系统有完全控制,因此这是可接受且必须存在的最终逃生口。
|
||||
|
||||
CLI 只动 auth 相关行(`auth_users` 的密码/TOTP 字段、`auth_login_throttle`、`auth_recovery_code`),**绝不**触碰用户数据表(`location`、`poo_records`、`public_ip_state` 等)。
|
||||
|
||||
入口:`python -m scripts.admin_cli <command>`
|
||||
|
||||
#### 子命令
|
||||
|
||||
| 命令 | 用途 | 逃生场景 |
|
||||
| --- | --- | --- |
|
||||
| `reset-password <username> [--password <pwd>]` | 重置密码(不带 `--password` 则交互式输入,不回显) | 忘记密码 |
|
||||
| `unlock [--all \| --ip <ip> \| --username <u>]` | 清 `auth_login_throttle` 行 | 被退避锁住(429)时解锁 |
|
||||
| `disable-totp <username>` | 关停 TOTP(清 secret + 删全部恢复码),**零凭据可执行** | 恢复码全丢也能进 |
|
||||
| `reissue-totp <username>` | 生成新 TOTP secret 并打印 `otpauth://` URI | 设备丢失、需重新注册 Authenticator |
|
||||
| `list-admin` | 列出所有用户与状态列 | 排障用 |
|
||||
|
||||
#### 使用示例
|
||||
|
||||
```bash
|
||||
# 重置密码(不带 --password 时交互式 prompt,不回显)
|
||||
python -m scripts.admin_cli reset-password admin
|
||||
|
||||
# 重置密码(非交互,脚本里用)
|
||||
python -m scripts.admin_cli reset-password admin --password "newpassword"
|
||||
|
||||
# 解锁所有退避
|
||||
python -m scripts.admin_cli unlock --all
|
||||
|
||||
# 解锁特定 IP
|
||||
python -m scripts.admin_cli unlock --ip 1.2.3.4
|
||||
|
||||
# 解锁特定 username
|
||||
python -m scripts.admin_cli unlock --username admin
|
||||
|
||||
# 关停 TOTP(零凭据,最终逃生口)
|
||||
python -m scripts.admin_cli disable-totp admin
|
||||
|
||||
# 重新发放 TOTP secret(打印新 otpauth:// URI,扫码重新注册)
|
||||
python -m scripts.admin_cli reissue-totp admin
|
||||
|
||||
# 查看用户列表
|
||||
python -m scripts.admin_cli list-admin
|
||||
```
|
||||
|
||||
在 Docker 容器内执行:
|
||||
|
||||
```bash
|
||||
docker compose exec app python -m scripts.admin_cli <command>
|
||||
```
|
||||
|
||||
#### `reissue-totp` 语义说明
|
||||
|
||||
- 对**已启用** TOTP 的用户:新 secret **立即在登录时生效**,旧 Authenticator 生成的码立即失效;**无需**再走 web `enable` 步骤。
|
||||
- 现有恢复码**不被删除**——恢复码是独立的随机哈希值,与 TOTP secret 无密码学绑定,reissue 后恢复码依然有效(仍可用于登录)。
|
||||
- 如果需要完整清理(secret + 所有恢复码),使用 `disable-totp`。
|
||||
|
||||
---
|
||||
|
||||
### 3. 可选 TOTP 二次验证
|
||||
|
||||
#### 设计
|
||||
|
||||
- TOTP 遵循 RFC 6238,使用 `pyotp` 库。
|
||||
- 二维码在**前端**由 `qrcode.react` 渲染(后端只返回 `otpauth://` URI,不引图像依赖)。
|
||||
- `totp_secret` 明文存库(与项目其他 secret 处理一致,靠数据库文件权限保护)。
|
||||
- 恢复码以 Argon2 哈希存库,一次性(使用后标记 `used_at`)。
|
||||
|
||||
#### 启用流程
|
||||
|
||||
1. **setup**(`POST /api/auth/totp/setup`):生成 pending secret,返回 secret、`otpauth://` URI、10 个明文恢复码(**仅此一次**);此时 `totp_enabled` 仍为 `false`。
|
||||
2. **扫码**:用 Authenticator App 扫前端渲染的二维码(或手动输入 secret)。
|
||||
3. **enable**(`POST /api/auth/totp/enable`):输入当前 6 位码确认 → `totp_enabled` 变为 `true`。
|
||||
4. 妥善保存恢复码(不会再次展示)。
|
||||
|
||||
#### 停用流程
|
||||
|
||||
**web 停用**(`POST /api/auth/totp/disable`):需提供当前密码或当前 6 位 TOTP 码。成功后清 secret、删全部恢复码,恢复纯密码登录。
|
||||
|
||||
**CLI 停用**(逃生口,恢复码全丢时):`python -m scripts.admin_cli disable-totp admin`,零凭据,立即生效。
|
||||
|
||||
#### 登录二步流程
|
||||
|
||||
1. 提交 `POST /api/auth/login {username, password}` → 密码正确但 TOTP 已启用 → `401 {totp_required: true}`(不发 session)
|
||||
2. 前端切到第二屏,提交 `POST /api/auth/login {username, password, totp_code}` → 通过 → 发 session cookie
|
||||
|
||||
`totp_code` 可以是 6 位 TOTP 动态码,也可以是 `xxxx-xxxx` 格式恢复码(命中即消费,不可复用)。
|
||||
|
||||
#### API 端点
|
||||
|
||||
| 端点 | 用途 |
|
||||
| --- | --- |
|
||||
| `POST /api/auth/totp/setup` | 生成 pending secret + URI + 恢复码(一次性明文返回) |
|
||||
| `POST /api/auth/totp/enable` | 带当前 6 位码确认启用 |
|
||||
| `POST /api/auth/totp/disable` | 带密码或当前码停用 |
|
||||
| `GET /api/auth/totp` | 返回当前 TOTP 状态(`{enabled: bool}`),不返回 secret/恢复码 |
|
||||
|
||||
全部端点需要 session cookie(`GET /api/auth/totp` 不需 CSRF;其余写端点需 `X-CSRF-Token`)。
|
||||
|
||||
#### 配置项
|
||||
|
||||
| 配置项 | 类型 | 默认 | 说明 |
|
||||
| --- | --- | --- | --- |
|
||||
| `AUTH_TOTP_ISSUER` | str(`.env` 部署级) | 空(回退 `app_name`) | 显示在 Authenticator App 里的 issuer 标签 |
|
||||
|
||||
---
|
||||
|
||||
## 受保护范围
|
||||
|
||||
当前 JSON API 端点(`/api/*`)需要 session cookie;写端点需额外携带 `X-CSRF-Token` header。
|
||||
|
||||
裸 ingestion 端点(`/location/record`、`/poo/record` 等设备调用端点)暂未收口到 session 保护(M3 计划引入 token 鉴权)。
|
||||
|
||||
## 下一步(不在当前范围)
|
||||
|
||||
- M3:token 鉴权(供脚本 / 设备 / 移动端调用 API),ingestion 端点收口。
|
||||
|
||||
@@ -1,12 +1,16 @@
|
||||
# 设计文档与多模型协作约定
|
||||
|
||||
本目录把 `docs/roadmap.md` 里的三个里程碑展开成**可被 coding agent 流水线执行**的详细设计文档。
|
||||
本目录把 `docs/roadmap.md` 里的各个里程碑展开成**可被 coding agent 流水线执行**的详细设计文档。
|
||||
|
||||
- [`m1-db-consolidation.md`](./m1-db-consolidation.md) — 单库化地基
|
||||
- [`m2-frontend-v2.md`](./m2-frontend-v2.md) — React SPA 前端 v2
|
||||
- [`m3-token-mobile.md`](./m3-token-mobile.md) — token 鉴权与移动端(远期)
|
||||
- [`m4-login-hardening.md`](./m4-login-hardening.md) — 登录加固(防爆破/指数退避 + CLI 逃生 + 可选 TOTP)**先做**
|
||||
- [`m5-iot-energy.md`](./m5-iot-energy.md) — IoT 集成与能耗采集(Modbus/Energy + MQTT/HA Discovery + 前端侧边栏)
|
||||
- [`m6-tibber-dynamic-energy.md`](./m6-tibber-dynamic-energy.md) — 通用电价层 + DSMR 实时电表接入 + 实时买卖电费计算 + HA Energy 反哺
|
||||
- [`m7-meter-epochs-archival.md`](./m7-meter-epochs-archival.md) — 电表生命周期 / 换表归档(Meter epochs)
|
||||
|
||||
本文件定义**所有任务共用的格式与协作规则**,三个里程碑文档不再重复这些约定。
|
||||
本文件定义**所有任务共用的格式与协作规则**,各个里程碑文档不再重复这些约定。
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -0,0 +1,309 @@
|
||||
# M4 — 登录加固(Login Hardening)
|
||||
|
||||
> 阅读前提:先读 [`README.md`](./README.md)(协作模型、任务卡格式、校验闸门、数据安全红线)。
|
||||
> **排期:本里程碑排在 [M5](./m5-iot-energy.md) 之前。** 系统现为公网 + 仅密码 + 无限重试,是现成风险;M4 与 M5 零文件重叠,先堵这个洞再做 IoT。
|
||||
|
||||
## 1. 目标
|
||||
|
||||
给暴露在公网的单 admin 登录做纵深防御:
|
||||
|
||||
1. **防爆破 / 指数退避**:失败登录按指数增长延迟,拖垮暴力枚举;成功即清零。
|
||||
2. **CLI 逃生通道**:纯命令行重置密码 / 解锁 / 关停 TOTP,**不依赖任何已存恢复凭据**——拿到服务器 CLI 权限的人本来就无所不能,所以这是可接受、且必须存在的最终逃生口(消除"锁死就彻底进不去"的死角)。
|
||||
3. **TOTP 二次验证(可选配)**:admin 可**自选启用** RFC 6238 TOTP;启用后登录需密码 + 6 位动态码;提供一次性恢复码;CLI 能在恢复码全丢时关掉 TOTP。**非强制**,不启用则维持纯密码登录。
|
||||
|
||||
> 三段顺序:A(防爆破,紧急、可独立先发版)→ B(TOTP,可选配)→ 收尾。Phase A 自成可用增量,Phase B 失败不影响 A。
|
||||
|
||||
## 2. 现状(实现者可据此工作,不必通读全仓库)
|
||||
|
||||
**单 admin + Argon2**
|
||||
- `app/models/auth.py`:`AuthUser`(`id / username unique / password_hash / is_active / force_password_change / created_at`)、`AuthSession`(`token_hash / csrf_token / expires_at / revoked_at`)。单库 `Base`。
|
||||
- `app/services/auth.py`:
|
||||
- `authenticate_user(session, *, username, password) -> AuthUser | None`:查用户→`verify_password`(Argon2);失败只 `logger.info`,**无任何节流**。
|
||||
- `create_session(session, *, user, settings) -> (AuthSession, raw_token)`:`secrets.token_urlsafe`,SHA256 存 `token_hash`,TTL `auth_session_ttl_hours`(默认 12h)。
|
||||
- `change_password(...)`、`get_authenticated_session(...)`、`initialize_auth_schema(session, settings)`(**仅当无任何用户时**用 `AUTH_BOOTSTRAP_USERNAME/PASSWORD` 建初始 admin)。
|
||||
- `app/api/routes/api/session.py`:
|
||||
- `POST /api/auth/login`(body `{username, password}`):`authenticate_user` → None 则 401;否则 `create_session` + `set_cookie`(HttpOnly/SameSite=Lax/`auth_cookie_secure`) → 返回 `{user, csrf_token}`。
|
||||
- `GET /api/session`(401 或 user+csrf)、`POST /api/auth/logout`(CSRF)、`POST /api/auth/password`(CSRF + 校验当前密码)。
|
||||
|
||||
**配置**
|
||||
- `app/config.py`:`auth_bootstrap_username/password`、`auth_session_cookie_name`、`auth_session_ttl_hours`、`auth_cookie_secure_override`(computed `auth_cookie_secure`)。
|
||||
- 配置系统:扁平 KV(`CONFIG_FIELDS` 注册表 + `Settings` + `_settings_payload`),新增标量配置项前端自动渲染(见 M5 文档 §2)。
|
||||
|
||||
**无现成 CLI**
|
||||
- `scripts/`:`run_migrations.py` / `app_db_adopt.py` / `migrate_legacy_data.py`(argparse) / `export_openapi.py`。**没有**任何 admin/密码/auth 的 CLI。
|
||||
- `pyproject.toml`:**无 `console_scripts`**。CLI 入口沿用 `python -m scripts.<mod>` 风格(与 `python -m scripts.run_migrations` 一致)。
|
||||
|
||||
**前端登录(单步)**
|
||||
- `frontend/src/pages/LoginPage.tsx`:`POST /api/auth/login {username,password}` → 401 显示通用错误;成功存 csrf、跳转。
|
||||
- `frontend/src/auth/SessionProvider.tsx`:`GET /api/session` 引导。
|
||||
- 无"锁定/稍后再试"提示、无二步流程。
|
||||
|
||||
**测试**
|
||||
- `tests/test_api_session.py`(凭据/cookie/登出/改密/强制改密);`tests/conftest.py` 用 env 设 bootstrap 凭据。
|
||||
|
||||
## 3. 目标架构
|
||||
|
||||
### 3.1 防爆破 / 指数退避
|
||||
|
||||
- **状态表 `auth_login_throttle`**:按 key 记失败窗口(`key / scope / failures / first_failed_at / last_failed_at / next_allowed_at`),成功即删该 key 的行。
|
||||
- **双键**:同时按 **client IP** 与 **username** 计;本次请求需等待 = 两者退避的**较大值**。
|
||||
- 按 IP:堵单点暴力(attacker IP 越敲越慢,合法用户换 IP 不受影响)。
|
||||
- 按 username:单 admin 的全局兜底(即便分布式换 IP 也有一层)。
|
||||
- **DoS 权衡**:单 admin 下纯按 username 硬锁会被攻击者故意失败把你锁死,所以退避是**延迟(429 + Retry-After)而非永久锁定**,且 CLI 是最终逃生口。
|
||||
- **client IP 来源**:默认用 socket IP;`AUTH_TRUST_FORWARDED_FOR=true` 时取 `X-Forwarded-For` 最左(部署在反代后才开)。**这点必须显式配置**,否则反代后所有请求同一 IP,按 IP 退避失效。
|
||||
- **指数公式(默认,常量可后调)**:前 `N_free=3` 次不延迟;之后 `wait = min(cap, base * 2^(failures - N_free))`,`base=1s`、`cap=900s`。
|
||||
- **接入点**:`POST /api/auth/login` 开头先查退避 → 在窗口内直接 429(带 `Retry-After`),**不验密码**(省 Argon2、防枚举);验密码失败 → 记一次失败;成功 → 清该 IP + username 的退避行。
|
||||
- **全局开关** `AUTH_LOGIN_THROTTLE_ENABLED`(CONFIG_FIELDS,默认 true)。
|
||||
|
||||
### 3.2 CLI 逃生通道
|
||||
|
||||
`scripts/admin_cli.py`(`python -m scripts.admin_cli <cmd>`,argparse 子命令;直接连本地 DB,无网络,复用 `get_session_local()` + 模型 + Argon2 hasher):
|
||||
|
||||
| 命令 | 作用 |
|
||||
| --- | --- |
|
||||
| `reset-password <username> [--password ...]` | 重置密码(不给则交互式 prompt,不回显);逃生:忘密码 |
|
||||
| `unlock [--all | --ip <ip> | --username <u>]` | 清 `auth_login_throttle` 行;逃生:被退避锁住 |
|
||||
| `disable-totp <username>` | 关 TOTP(清 secret + 恢复码);逃生:**恢复码全丢也能进**(Phase B 才有意义)|
|
||||
| `reissue-totp <username>` | 重新发放 TOTP secret(可选)|
|
||||
| `list-admin` | 列用户/状态(排障)|
|
||||
|
||||
- **数据安全**:CLI 只动 auth 状态行(人工执行的 admin 动作),**绝不**触碰用户数据表(location/poo/energy 等),不 drop 有数据的表。
|
||||
|
||||
### 3.3 TOTP 二次验证(可选配)
|
||||
|
||||
- **库**:`pyotp`(Python 侧);二维码在**前端**用 JS 库(`qrcode.react`)从 `otpauth://` URI 渲染——后端不引图像依赖。
|
||||
- **存储**:
|
||||
- `AuthUser` 增列 `totp_secret`(nullable)、`totp_enabled`(bool, default false)。secret 落库明文(与现有 secret 处理一致,靠文件权限保护;文档注明)。
|
||||
- `auth_recovery_code` 表(`user_id / code_hash / used_at`):恢复码**哈希**存(复用 Argon2 hasher),一次性。
|
||||
- **启用流程(opt-in)**:
|
||||
1. `POST /api/auth/totp/setup`(已登录)→ 生成 pending secret + `otpauth://` URI + 一组恢复码(**仅此一次明文返回**),尚未启用。
|
||||
2. `POST /api/auth/totp/enable`(带一个当前 6 位码确认)→ `totp_enabled=true`,持久化恢复码哈希。
|
||||
3. `POST /api/auth/totp/disable`(带密码或当前码)→ 关闭、清 secret/恢复码。
|
||||
- **登录二因子(单端点、无独立 challenge token,保持无状态)**:
|
||||
- `POST /api/auth/login` body 增可选 `totp_code`。
|
||||
- 密码通过且 `totp_enabled`:无 `totp_code` → 返回 401 + `{totp_required: true}`(**不发 session**);有 `totp_code` → 校验 TOTP **或**恢复码(命中恢复码则消费它)→ 通过才发 session。
|
||||
- 未启用 TOTP:维持现状单步。
|
||||
- 退避(§3.1)对两步都生效。
|
||||
- **issuer 标签**:`AUTH_TOTP_ISSUER`(默认 `app_name`),显示在 Authenticator app 里。
|
||||
|
||||
### 3.4 前端
|
||||
|
||||
- **登录页两步**:`LoginPage` 在收到 `401 + totp_required` 时切到第二屏(6 位码输入,也接受恢复码),再次 `POST /api/auth/login` 带 `totp_code`。被退避(429)显示"稍后再试"(用 `Retry-After`)。
|
||||
- **设置页 TOTP 区**:未启用→「启用」走 setup(展示二维码 + 恢复码一次性、要求输入码确认);已启用→「停用」。恢复码只在生成时展示一次,提示妥善保存。
|
||||
|
||||
## 4. API 契约(M4 要落地的端点)
|
||||
|
||||
> 全部 `/api`、写端点 session + CSRF 保护、JSON 进出;schema 经 `export_openapi.py` 固化。
|
||||
|
||||
| 分组 | 端点 | 用途 |
|
||||
| --- | --- | --- |
|
||||
| 登录 | `POST /api/auth/login` | 加退避(429+Retry-After);body 增可选 `totp_code`;启用 TOTP 且缺码→401 `{totp_required:true}` |
|
||||
| TOTP | `POST /api/auth/totp/setup` | 生成 pending secret + otpauth URI + 恢复码(一次性)|
|
||||
| TOTP | `POST /api/auth/totp/enable` | 带当前码确认启用 |
|
||||
| TOTP | `POST /api/auth/totp/disable` | 关闭 TOTP |
|
||||
| TOTP | `GET /api/auth/totp` | 返回当前 TOTP 状态(enabled 与否)|
|
||||
|
||||
> CLI 不是 HTTP 端点。throttle 开关与 issuer 走现有 `GET/PUT /api/config`(仅加 CONFIG_FIELDS)。
|
||||
|
||||
## 5. 已锁定决策(讨论后拍板)
|
||||
|
||||
1. **M4 独立里程碑、排在 M5 之前**。
|
||||
2. **范围**:防爆破 + 指数退避 + CLI 重置(密码/解锁/关 TOTP)+ TOTP(**可选配,非强制**)。
|
||||
3. **退避双键(IP + username)、延迟非永久锁定、成功清零、CLI 兜底**;反代后需开 `AUTH_TRUST_FORWARDED_FOR`。
|
||||
4. **CLI 逃生通道不依赖任何已存恢复凭据**;只动 auth 行,不碰用户数据。
|
||||
5. **TOTP 单端点二因子**(login 加可选 `totp_code`,无独立 challenge token);恢复码哈希存、一次性;secret 明文存(与现有一致)。
|
||||
6. **二维码前端渲染**(`qrcode.react`),后端只给 `otpauth://` URI + secret;Python 仅加 `pyotp`。
|
||||
7. **CLI 入口 = `python -m scripts.admin_cli`**(无 console_scripts)。
|
||||
|
||||
> 项目定位:个人自用、单 admin——可按单用户简化,不为多租户/找回邮件等过度设计。
|
||||
|
||||
## 6. 任务依赖图
|
||||
|
||||
```
|
||||
Phase A(防爆破,紧急,自成可发版增量)
|
||||
M4-T01 [schema] auth_login_throttle 表 + 模型
|
||||
└─► M4-T02 退避 service + 接入 login(429/Retry-After/成功清零)
|
||||
└─► M4-T03 CLI 骨架 + reset-password + unlock
|
||||
|
||||
Phase B(TOTP,可选配)
|
||||
M4-T04 [schema] AuthUser TOTP 字段 + auth_recovery_code 表
|
||||
├─► M4-T05 TOTP service + setup/enable/disable/status API
|
||||
├─► M4-T06 登录二因子(login 加 totp_code;依赖 T02 已改过 login)
|
||||
└─► M4-T07 CLI disable-totp / reissue-totp
|
||||
M4-T08 前端:两步登录 + 设置页 TOTP(依赖 T05+T06;引入 qrcode.react)
|
||||
|
||||
收尾
|
||||
M4-T09 文档 + OpenAPI + roadmap(依赖全部)
|
||||
```
|
||||
|
||||
`T01` 可先开。Phase A(T01–T03)跑完即可发一版只含防爆破 + CLI 的安全增量。
|
||||
|
||||
---
|
||||
|
||||
## 7. 原子任务(任务卡)
|
||||
|
||||
> 后端沿用校验闸门(`pytest`/`ruff`/改路由或 schema 则 `export_openapi` 重导出入库)。前端闸门见 §8。新增依赖(`pyotp`、`qrcode.react`)须同步 `requirements.in/.txt` 与 `frontend/package.json` 重新锁定。
|
||||
|
||||
### M4-T01 — `auth_login_throttle` 表 + 模型 `[schema]`
|
||||
- **Status**: `done` · **Depends**: none
|
||||
- **Context**: 存按 IP / username 的失败退避状态。仅建 schema。
|
||||
- **Files**: `create app/models/auth_throttle.py`(或并入 `app/models/auth.py`);`create alembic_app/versions/<date>_NN_auth_login_throttle.py`;`modify alembic_app/env.py`、`scripts/app_db_adopt.py`(baseline 常量);`create tests/test_auth_throttle_model.py`
|
||||
- **Steps**: 模型 `LoginThrottle`(`id / key str / scope str('ip'|'user') / failures int / first_failed_at / last_failed_at / next_allowed_at`,`(scope,key)` unique 索引);新 revision 建表 + 索引;更新 baseline 常量。
|
||||
- **Out of scope / 不要碰**: 不写退避逻辑(T02);不动登录端点。
|
||||
- **Acceptance criteria**:
|
||||
- [ ] 临时库 upgrade 到 head 含该表与唯一索引;`downgrade -1` 干净。
|
||||
- [ ] `APP_BASELINE_REVISION` 更新;`env.py` 已 import。
|
||||
- [ ] 校验闸门全绿。
|
||||
- **Reviewer checklist**: `(scope,key)` 唯一;链上单 head;env.py 注册模型。
|
||||
|
||||
### M4-T02 — 退避 service + 接入登录
|
||||
- **Status**: `done` · **Depends**: M4-T01
|
||||
- **Context**: 指数退避核心 + 接 `POST /api/auth/login`。
|
||||
- **Files**: `create app/services/login_throttle.py`;`modify app/api/routes/api/session.py`、`app/services/config_page.py`(+`CONFIG_FIELDS` `AUTH_LOGIN_THROTTLE_ENABLED`)、`app/config.py`(+`auth_login_throttle_enabled`、`auth_trust_forwarded_for`);`modify tests/test_api_session.py`、`create tests/test_login_throttle.py`
|
||||
- **Steps**:
|
||||
1. `check_and_get_wait(session, *, ip, username) -> int`(秒;>0 表示仍在窗口);`register_failure(...)`(更新双键 failures/next_allowed_at,按 §3.1 公式);`clear(session, *, ip, username)`(成功清零)。
|
||||
2. 登录端点:取 client IP(按 `auth_trust_forwarded_for` 决定是否信 XFF 最左);**先**查退避,>0 → 429 + `Retry-After`,不验密码;验密码失败 → `register_failure` 后 401;成功 → `clear` 再发 session。
|
||||
3. 开关关闭时整段 no-op。
|
||||
- **Out of scope / 不要碰**: 不做 TOTP(Phase B);不改 logout/password 端点。
|
||||
- **Acceptance criteria**:
|
||||
- [ ] 单测:连续失败后 `wait` 按指数增长且封顶;窗口内请求 429 带 `Retry-After` 且不验密码。
|
||||
- [ ] 单测:成功登录清零;下次无延迟。
|
||||
- [ ] 401 仍为通用文案(不泄露用户是否存在);现有登录测试仍绿。
|
||||
- [ ] 开关 false 时无节流。
|
||||
- [ ] 校验闸门全绿(OpenAPI 若变重导出)。
|
||||
- **Reviewer checklist**: 退避在验密码**之前**生效;双键较大值;XFF 仅在配置开启时信任;无把密码写进日志。
|
||||
|
||||
### M4-T03 — CLI 骨架 + reset-password + unlock
|
||||
- **Status**: `done` · **Depends**: M4-T01
|
||||
- **Context**: 逃生通道第一批命令。
|
||||
- **Files**: `create scripts/admin_cli.py`;`create tests/test_admin_cli.py`
|
||||
- **Steps**: argparse 子命令;`reset-password <username> [--password|prompt(getpass)]` 用 Argon2 hasher 重置 `password_hash`;`unlock [--all|--ip|--username]` 删 `auth_login_throttle` 行;`list-admin`;复用 `get_session_local()` + 模型;无网络。
|
||||
- **Out of scope / 不要碰**: 不动 HTTP 端点;不碰用户数据表;TOTP 命令在 T07。
|
||||
- **Acceptance criteria**:
|
||||
- [ ] 单测:`reset-password` 后新密码可 `verify_password` 通过、旧密码失败。
|
||||
- [ ] 单测:`unlock` 清掉指定/全部退避行。
|
||||
- [ ] 不存在的 user 友好报错、非零退出。
|
||||
- [ ] `grep` 确认 CLI 不含对用户数据表的删除/drop。
|
||||
- [ ] 校验闸门全绿。
|
||||
- **Reviewer checklist**: 密码可交互输入且不回显;只动 auth 行;幂等可重跑。
|
||||
|
||||
### M4-T04 — AuthUser TOTP 字段 + `auth_recovery_code` 表 `[schema]`
|
||||
- **Status**: `done` · **Depends**: none(与 Phase A 并行的 schema,但 T06 依赖 T02)
|
||||
- **Files**: `modify app/models/auth.py`(`AuthUser.totp_secret`/`totp_enabled`);`create` 模型 `RecoveryCode`;`create alembic_app/versions/<date>_NN_totp.py`;`modify alembic_app/env.py`、`scripts/app_db_adopt.py`;`create tests/test_totp_models.py`
|
||||
- **Steps**: `totp_secret` nullable、`totp_enabled` bool default false;`auth_recovery_code(user_id FK, code_hash, used_at nullable)`;revision 建列 + 表;更新 baseline。
|
||||
- **Out of scope / 不要碰**: 不写 TOTP 逻辑(T05)。
|
||||
- **Acceptance criteria**:
|
||||
- [ ] upgrade/downgrade 干净;默认 `totp_enabled=false`、`totp_secret=null`。
|
||||
- [ ] baseline 常量更新;env.py import。
|
||||
- [ ] 校验闸门全绿。
|
||||
- **Reviewer checklist**: 现有用户迁移后默认未启用 TOTP(不破坏现有登录)。
|
||||
|
||||
### M4-T05 — TOTP service + setup/enable/disable/status API
|
||||
- **Status**: `done` · **Depends**: M4-T04
|
||||
- **Files**: `create app/services/totp.py`、`app/schemas/totp.py`;`modify app/api/routes/api/session.py`(或新 `auth_totp.py` 路由)、`app/config.py`(+`auth_totp_issuer`);`modify requirements.in/.txt`(+`pyotp`);`create tests/test_api_totp.py`
|
||||
- **Steps**: pyotp 生成 secret + `otpauth://` URI(issuer=`auth_totp_issuer`);恢复码生成(N=10、`xxxx-xxxx`,Argon2 哈希存、明文仅 setup 返回一次);`setup`(pending,不启用)/`enable`(带当前码确认)/`disable`(带密码或当前码)/`GET status`;全部 session+CSRF。
|
||||
- **Out of scope / 不要碰**: 不改 login 校验(T06)。
|
||||
- **Acceptance criteria**:
|
||||
- [ ] `setup` 返回 secret+URI+恢复码(一次性);未确认前 `totp_enabled` 仍 false。
|
||||
- [ ] `enable` 用错码失败、对码成功;恢复码以哈希持久化。
|
||||
- [ ] `disable` 关闭并清 secret/恢复码。
|
||||
- [ ] schema 经 OpenAPI 固化;secret/恢复码不回显于 `GET status`。
|
||||
- [ ] 校验闸门全绿。
|
||||
- **Reviewer checklist**: 恢复码只存哈希、明文仅一次;secret 不进 OpenAPI 示例/日志;`requirements.txt` 锁定 `pyotp`。
|
||||
|
||||
### M4-T06 — 登录二因子(login 加 `totp_code`)
|
||||
- **Status**: `done` · **Depends**: M4-T05, M4-T02
|
||||
- **Files**: `modify app/api/routes/api/session.py`、`app/schemas/session.py`(`LoginRequest.totp_code` 可选)、`app/services/totp.py`(校验 + 消费恢复码);`modify tests/test_api_session.py`、`tests/test_api_totp.py`
|
||||
- **Steps**: 密码通过后:若 `totp_enabled` 且无 `totp_code` → 401 `{totp_required:true}`(不发 session、退避照算);有 `totp_code` → 校验 TOTP 或恢复码(命中恢复码置 `used_at`)→ 通过发 session;未启用 → 现状。
|
||||
- **Out of scope / 不要碰**: 不改 setup/enable(T05);不动退避公式(T02)。
|
||||
- **Acceptance criteria**:
|
||||
- [ ] 未启用 TOTP 的用户登录与现状完全一致。
|
||||
- [ ] 启用后:缺码→401 totp_required;错码→401;对码→发 session;恢复码可用且一次性。
|
||||
- [ ] 退避对二步同样生效。
|
||||
- [ ] OpenAPI 重导出入库;校验闸门全绿。
|
||||
- **Reviewer checklist**: `totp_required` 时确未发 cookie;恢复码消费后不可复用;通用错误不泄露细节。
|
||||
|
||||
### M4-T07 — CLI disable-totp / reissue-totp
|
||||
- **Status**: `done` · **Depends**: M4-T04(逻辑上配合 T03 的 CLI 骨架)
|
||||
- **Files**: `modify scripts/admin_cli.py`;`modify tests/test_admin_cli.py`
|
||||
- **Steps**: `disable-totp <username>`:`totp_enabled=false` + 清 secret + 删恢复码(**不需任何恢复码即可执行**);`reissue-totp <username>`:生成新 secret(可选打印新 URI)。
|
||||
- **Out of scope / 不要碰**: 不碰用户数据表。
|
||||
- **Acceptance criteria**:
|
||||
- [ ] 单测:启用 TOTP 的用户经 `disable-totp` 后可纯密码登录(结合 T06 行为)。
|
||||
- [ ] 不依赖任何恢复码即可关停。
|
||||
- [ ] 校验闸门全绿。
|
||||
- **Reviewer checklist**: 逃生不依赖已存凭据;只动 auth 行。
|
||||
|
||||
### M4-T08 — 前端:两步登录 + 设置页 TOTP
|
||||
- **Status**: `done` · **Depends**: M4-T05, M4-T06
|
||||
- **Files**: `modify frontend/src/pages/LoginPage.tsx`;`create frontend/src/pages/.../TotpSettings.tsx`(或并入 ConfigPage)、`frontend/src/auth/totp.ts`;`modify frontend/package.json`(+`qrcode.react`,lock 同步);`create` 对应测试
|
||||
- **Steps**: 登录页:401+`totp_required`→第二屏 6 位码(也接受恢复码);429→"稍后再试"(读 `Retry-After`)。设置页:未启用→启用流程(二维码由 `otpauth` URI 渲染 + 恢复码一次性展示 + 输码确认);已启用→停用。
|
||||
- **Out of scope / 不要碰**: 不改后端;不碰防爆破逻辑。
|
||||
- **Acceptance criteria**:
|
||||
- [ ] 未启用 TOTP 的登录体验不变;启用后两步可走通;恢复码可登录。
|
||||
- [ ] 二维码可被 Authenticator 扫描;恢复码仅展示一次。
|
||||
- [ ] 429 有"稍后再试"提示。
|
||||
- [ ] 前端闸门全绿。
|
||||
- **Reviewer checklist**: 全走类型化 client;secret/恢复码不落 localStorage/日志;空/错/加载态有处理。
|
||||
|
||||
### M4-T09 — 文档 + OpenAPI + roadmap 收尾
|
||||
- **Status**: `done` · **Depends**: 全部
|
||||
- **Files**: `modify README.md`、`docs/auth.md`(防爆破 + CLI + TOTP 说明、CLI 用法)、`docs/architecture-overview.md`;`modify docs/roadmap.md`(把"TOTP 二次验证"与新"登录防爆破"从「下一阶段」毕业、加 M4 行、注明 M4 先于 M5);`modify docs/design/README.md`(已含 m4);`run python scripts/export_openapi.py` 提交 `openapi/`
|
||||
- **Acceptance criteria**:
|
||||
- [ ] 文档含 CLI 逃生用法与 TOTP 启停说明;`git diff --exit-code openapi/` 无差异。
|
||||
- [ ] 校验闸门全绿。
|
||||
- **Reviewer checklist**: roadmap 反映 M4 已落地且先于 M5;无残留旧描述。
|
||||
|
||||
---
|
||||
|
||||
## 8. 前端校验闸门(前端任务每次结束都要全绿)
|
||||
|
||||
在 `frontend/` 下:`npm ci && npm run lint && npm run typecheck && npm run test && npm run build`。新增依赖 `qrcode.react` 须提交 `package.json` + `package-lock.json`。后端同任务改路由/schema 仍需根目录 `export_openapi.py` 并提交 `openapi/`。
|
||||
|
||||
## 9. 构建上下文 / 依赖完整性
|
||||
|
||||
- 新增 Python 依赖 `pyotp`、前端 `qrcode.react`:同步重新锁定 `requirements.txt` / `package-lock.json`,否则镜像缺包。
|
||||
- 新文件均在既有 `app/`、`scripts/`、`frontend/` 的 COPY 范围内;`scripts/admin_cli.py` 须在镜像里可 `python -m scripts.admin_cli` 调用(确认 `scripts/` 已进镜像)。
|
||||
- `tests/test_deployment.py::test_dockerfile_copy_sources_exist` 仍应通过。
|
||||
|
||||
## 10. 人工验收 walkthrough(实现完成后)
|
||||
|
||||
> 重点:连续输错 → 系统自动退避锁住 → CLI 解锁后恢复登录。可用 `docker compose` 起服务,CLI 在容器内或容器外跑均可。
|
||||
|
||||
**准备**:起服务(任选其一)
|
||||
- Docker:`docker compose up -d`(应用容器名见 `docker-compose.yml`,下文记为 `<app>`)。
|
||||
- 或本地:`source .venv/bin/activate && uvicorn app.main:app --port 8000`。
|
||||
|
||||
**1) 触发退避锁定** —— 用**错误密码**连续登录同一账号:
|
||||
```bash
|
||||
for i in $(seq 1 8); do \
|
||||
code=$(curl -s -o /dev/null -w "%{http_code}" -X POST http://localhost:8000/api/auth/login \
|
||||
-H 'Content-Type: application/json' -d '{"username":"admin","password":"wrong"}'); \
|
||||
echo "尝试 $i -> HTTP $code"; done
|
||||
```
|
||||
- 预期:前几次 `401`,超过免费次数后变 `429`(被锁/退避)。单独跑一次带 `-i` 可看到 `Retry-After` 头逐次增大(指数退避)。
|
||||
- 在退避窗口内,**即使输正确密码**也应被 `429` 挡住(验证锁定确实生效)。
|
||||
|
||||
**2) CLI 解锁**
|
||||
- 容器内:`docker compose exec <app> python -m scripts.admin_cli unlock --all`
|
||||
- 或容器外:`python -m scripts.admin_cli unlock --all`
|
||||
- 预期:打印清除的退避条目数。
|
||||
|
||||
**3) 验证恢复** —— 立即用**正确密码**登录:
|
||||
```bash
|
||||
curl -i -X POST http://localhost:8000/api/auth/login \
|
||||
-H 'Content-Type: application/json' -d '{"username":"admin","password":"<正确密码>"}'
|
||||
```
|
||||
- 预期:`200` 且 `Set-Cookie` 下发 session → 解锁成功。
|
||||
|
||||
**4)(可选)其它逃生命令**
|
||||
- 重置密码:`python -m scripts.admin_cli reset-password admin`(交互输入新密码、不回显)。
|
||||
- 关停 TOTP(若已启用):`python -m scripts.admin_cli disable-totp admin`,随后纯密码即可登录。
|
||||
|
||||
## 11. 里程碑完成定义(DoD)
|
||||
|
||||
- 登录失败按指数退避(429 + Retry-After),成功清零;可经 `AUTH_LOGIN_THROTTLE_ENABLED` 开关。
|
||||
- `python -m scripts.admin_cli` 能在**无任何 Web 访问、无恢复码**的情况下重置密码 / 解锁 / 关停 TOTP。
|
||||
- TOTP 可由 admin 自选启用:启用后两步登录、恢复码可用且一次性;不启用维持纯密码。
|
||||
- 后端 `pytest`/`ruff`/`export_openapi` + 前端 `lint/typecheck/test/build` 全绿且 `openapi/` 入库。
|
||||
- README/auth/architecture/roadmap 反映 M4 现实与"先于 M5"的排期。
|
||||
@@ -0,0 +1,533 @@
|
||||
# M5 — IoT 集成与能耗采集(Modbus 设备采集 + MQTT/HA Discovery + 前端侧边栏)
|
||||
|
||||
> 阅读前提:先读 [`README.md`](./README.md)(协作模型、任务卡格式、校验闸门、数据安全红线)。本里程碑建立在 M1 单库 + M2 React SPA 之上。
|
||||
> 配套参考:电表协议见 [`../references/SDM120-Modbus-Protocol.md`](../references/SDM120-Modbus-Protocol.md)。
|
||||
|
||||
## 1. 目标
|
||||
|
||||
给后端接入家庭 IoT 生态,并新增一条通用的 Modbus 设备采集链路(首个落地的领域是能耗):
|
||||
|
||||
1. **Modbus 设备采集(通用管道)**:通过 Modbus-TCP 网关周期读取挂在网关后面的 Modbus slave 设备(首个 profile 为 SDM120 单相电表),按设备的 **YAML profile** 解码为工程量,存入单库的**通用读数表**。后台静默轮询,支持多设备、多 profile。
|
||||
2. **MQTT + Home Assistant Discovery**:后端作为 MQTT 发布方,按"**可勾选暴露**"的方式把数据以 HA Discovery 自动注册成 device/entity(不止 sensor)。
|
||||
3. **前端侧边栏 + Energy 视图**:把现有顶栏改成侧边导航,承载首个领域视图 **Energy**(设备管理 + 最新读数 + 走势图)。
|
||||
|
||||
> **命名分层(本里程碑的核心决策)**:**存储 / 采集 / API 是通用的 `modbus_*`**(不锁死在"电表"上,以后接别的 Modbus 设备无需改表);**面向用户的呈现是领域特定的**——本里程碑只落第一个领域视图 **Energy**,消费通用 device 数据、用电表视角展示。
|
||||
|
||||
> 三段有依赖关系,按 §6 的 `Depends` 顺序推进:A(侧边栏,独立)→ B(Modbus 采集后端 + Energy 前端)→ C(MQTT/Discovery,消费 B 的数据)。
|
||||
|
||||
## 2. 现状(实现者可据此工作,不必通读全仓库)
|
||||
|
||||
**单库数据层**(M1 完成态)
|
||||
- `app/db.py`:`class Base(DeclarativeBase)`,绑 `settings.app_database_url` 的 cached engine(WAL 已开),`get_engine` / `get_session_local` / `reset_db_caches` / `get_db_session`。
|
||||
- 模型都继承同一 `Base`:`app/models/{auth,config,public_ip,location,poo}.py`。
|
||||
- 单 Alembic 链 `alembic_app/`,head = `20260611_06_merge_location_poo_tables`(见 `alembic_app/versions/`);`alembic_app/env.py` 逐个 import 所有模型。
|
||||
- 迁移命名惯例:`YYYYMMDD_NN_<desc>.py`,`revision` / `down_revision` 串链。
|
||||
- `scripts/app_db_adopt.py` 常量 `APP_BASELINE_REVISION` 指向当前 head;`scripts/run_migrations.py` 负责把 app 库升到 head。
|
||||
|
||||
**配置系统(扁平 KV,自动渲染)**
|
||||
- `app/config.py`:`class Settings(BaseSettings)`,每个配置项一个带类型的字段 + 默认值。
|
||||
- `app/services/config_page.py`:`CONFIG_FIELDS: tuple[ConfigField, ...]` 是**注册表**(`section / env_name / setting_attr / label / secret / input_type`);`build_config_sections`(读,secret 回空串)、`save_config_updates`(写,空 secret 保留旧值)、`build_runtime_settings`(DB override 合并进 Settings)、`_settings_payload`(把 Settings 摊平成 dict,新字段要在此补一行)。
|
||||
- `app/api/routes/api/config.py`:`GET/PUT /api/config`,session + CSRF 保护;非法值 422 且不写库。
|
||||
- 前端 `frontend/src/pages/ConfigPage.tsx`:**通用渲染**——按 section 分组、按 `input_type`/`secret` 渲染输入框。**新增标量配置项零前端改动**(追加 `CONFIG_FIELDS` + `Settings` 字段 + `_settings_payload` 一行即可)。
|
||||
- ⚠️ 扁平 KV **装不下"设备列表"和"逐实体勾选"**——这两者走专用表 + 专用 API + 自定义 UI(见 §3.2 / §3.4)。
|
||||
|
||||
**后台调度(APScheduler)**
|
||||
- `app/main.py` lifespan:`BackgroundScheduler(timezone="UTC")`,`scheduler.add_job(_run_scheduled_public_ip_check, IntervalTrigger(hours=4), id=..., max_instances=1, coalesce=True)`,`scheduler.start()`;`yield` 后 `scheduler.shutdown(wait=False)`。
|
||||
- 周期任务惯例:一个**同步** wrapper 自己开/关 session:`session = get_session_local()(); try: service(session, ...) finally: session.close()`,service 内部不抛崩溃。
|
||||
|
||||
**Home Assistant 现状(REST,不碰 MQTT)**
|
||||
- `app/integrations/homeassistant.py`:`HomeAssistantClient.publish_sensor()`(`POST /api/states/{entity}`)、`trigger_webhook()`。
|
||||
- 入站 webhook `app/api/routes/homeassistant.py`:`POST /homeassistant/publish`(envelope `target/action/content`)。
|
||||
- 新 MQTT Discovery 与此**并行、不冲突**,是第二条独立通道。
|
||||
|
||||
**前端(M2)**
|
||||
- React + react-router v6 + Mantine + TanStack Query + `openapi-fetch` 生成的类型化 client(`frontend/src/api/client.ts` + `schema.d.ts`)。
|
||||
- `frontend/src/App.tsx`:`AppLayout`(当前是**顶栏**),包住所有受保护页;路由 `/`(HomePage 地图)、`/config`、`/records`;`/login`、`/change-password` 不带 layout。
|
||||
- 数据请求惯例:`useQuery`/`useMutation` + `apiClient.GET/POST/...`(见 `frontend/src/records/hooks.ts`)。
|
||||
- **无图表库**(只有 Leaflet 地图);走势图需新引入 **Recharts**。
|
||||
|
||||
## 3. 目标架构
|
||||
|
||||
### 3.0 两层数据模型(本里程碑的地基决策)
|
||||
|
||||
把"设备是什么、怎么读"与"读到了什么"彻底分成两层,**协议知识与部署信息分离**:
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ 协议层(固定,随代码走) 部署层(可配置,落 DB) │
|
||||
│ ── YAML profile ────────── ── modbus_device 行 ────── │
|
||||
│ • 读哪些寄存器(FC/地址/块) • friendly_name(可改) │
|
||||
│ • 每个量:key / 类型 / 解码 • host / port(网关) │
|
||||
│ • 每个量:unit / device_class • unit_id(=电表 Meter ID, │
|
||||
│ / ha_component 设备上可设,故落 DB) │
|
||||
│ • 唯一真相源,喂给"读/存/HA注册" • profile 名(选哪个 YAML) │
|
||||
│ • uuid / poll_interval / enabled│
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
▲ 多个 device 可共享同一 profile │
|
||||
└─────────────────────────────────────────┘
|
||||
│ 采集
|
||||
▼
|
||||
── modbus_reading 行(通用遥测)──
|
||||
device_id · recorded_at · payload(JSON blob)
|
||||
```
|
||||
|
||||
- **profile = 仓库内只读 YAML**(如 `app/integrations/modbus/profiles/sdm120.yaml`),随镜像打包、纳入版本控制,**只描述固定协议知识**:读哪些寄存器、怎么解码、每个量的 `key`/`unit`/`device_class`/`ha_component`。**绝不**放 friendly_name / unit_id 这类每设备各异的可配置项。
|
||||
- **可配置项落 `modbus_device` 行**,由前端 UI 设置:friendly_name、host、port、**unit_id(Modbus 从机地址,设备面板上可改)**、选用的 profile 名、poll_interval、enabled。
|
||||
- **多设备共享一个 profile**:例如两块 SDM120 都 `profile="sdm120"`,一块 friendly_name="SDM120 空调"、unit_id=1,另一块 friendly_name="SDM120 服务器"、unit_id=2——共用同一份解码规则,部署参数各自独立。
|
||||
- **读数存通用 JSON `payload`**,不再用固定列。profile 是解释器:知道 payload 里有哪些 key、各自单位与 device_class。
|
||||
- 聚合不必搬到后端硬算:**SQLite 用 `json_extract` 在 SQL 端做 `AVG`/`MAX`/`GROUP BY`**(走势图都是带时间窗的查询,被 `(device_id, recorded_at)` 索引圈住,只扫窗内行)。
|
||||
- 真有某个量需要高频聚合 → 后续给该量加 **generated column + 表达式索引**(非破坏性,要哪个补哪个),无需一开始把表锁成固定列。
|
||||
|
||||
### 3.1 Modbus 采集
|
||||
|
||||
- **传输:仅 Modbus-TCP**(用户的 Waveshare RTU↔TCP 网关,RJ45 以太网;服务器无串口)。用 **pymodbus** 的 `ModbusTcpClient`,由它处理封帧 / CRC / 超时重试 / float 解码。
|
||||
- 角色厘清:**我们的后端 = Modbus client/master**;**网关 = TCP server**(在 "Modbus TCP" 模式下终结 TCP 再转 RTU);**电表 = 网关后面的 RTU slave**,由 `unit_id` 寻址。`modbus_device` 一行同时装下网关地址(host:port)与 slave 地址(unit_id)。
|
||||
- 网关若开"Modbus TCP"协议转换 → pymodbus 默认 framer 直连。
|
||||
- 网关若是透传(RTU-over-TCP,裸 RTU 帧含 CRC)→ pymodbus 用 RTU framer over TCP。
|
||||
- **二者实现时对一次即可确定**(连上读 Voltage 寄存器验证),不影响表结构与上层。
|
||||
- **协议知识在 YAML profile,部署信息在 DB**(见 §3.0):
|
||||
- `app/integrations/modbus/driver.py`:薄封装 pymodbus 的连接 + 块读 + 大端 float32 解码(word/byte 都大端,高寄存器在前)。
|
||||
- `app/integrations/modbus/profiles.py`:YAML profile 的**加载 + 校验 + 解码 + 实体枚举**——纯"数据 + 函数",**不做 OOP 抽象基类/继承**。`load_profile(name) -> ModbusProfile`(pydantic 模型,启动期校验)、`decode(profile, registers) -> dict[key -> value]`、`enumerate_entities(device, profile) -> list[ExposableEntity]`(供 §3.3)。
|
||||
- `app/integrations/modbus/profiles/*.yaml`:每个设备型号一个 YAML。首个 = `sdm120.yaml`(寄存器地址见参考文档 §4)。
|
||||
- profile 输出统一的 `dict[key -> value]`,由 service 整个塞进 `modbus_reading.payload`(JSON);profile 没列出的量根本不读。
|
||||
- **只读**:不写电表配置寄存器(改 Meter ID/波特率有通信中断风险)。
|
||||
- **轮询**:每个设备一个 `poll_interval_s`(默认 5s),APScheduler 一个 job 扫所有 `enabled` 设备,逐设备块读→解码→落库。9600 总线上多设备串行,5s 余量充足。
|
||||
- 全局开关 `MODBUS_POLLING_ENABLED`(CONFIG_FIELDS)可一键停采。
|
||||
- 采集成功/失败写入设备的 `last_poll_at` / `last_poll_ok`(供 §3.3 的 online binary_sensor)。
|
||||
- **两级周期 / 降采样 / 保留是后续杠杆**(见 §10),本里程碑用单周期读全。
|
||||
|
||||
**`sdm120.yaml` 示例(只含固定协议知识)**
|
||||
|
||||
```yaml
|
||||
name: sdm120
|
||||
description: Eastron SDM120 single-phase energy meter
|
||||
function_code: 4 # input registers (FC04)
|
||||
word_order: big # 高寄存器在前
|
||||
byte_order: big
|
||||
blocks: # 块读,减少 Modbus 事务
|
||||
- { start: 0x0000, count: 0x0060 } # 30001..30095 连续段
|
||||
- { start: 0x0156, count: 0x0004 } # total active/reactive energy
|
||||
metrics:
|
||||
- { key: voltage, address: 0x0000, type: float32, unit: V, device_class: voltage, ha_component: sensor }
|
||||
- { key: current, address: 0x0006, type: float32, unit: A, device_class: current, ha_component: sensor }
|
||||
- { key: active_power, address: 0x000C, type: float32, unit: W, device_class: power, ha_component: sensor }
|
||||
- { key: power_factor, address: 0x001E, type: float32, unit: "", device_class: power_factor, ha_component: sensor }
|
||||
- { key: frequency, address: 0x0046, type: float32, unit: Hz, device_class: frequency, ha_component: sensor }
|
||||
- { key: import_energy, address: 0x0048, type: float32, unit: kWh, device_class: energy, state_class: total_increasing, ha_component: sensor }
|
||||
- { key: export_energy, address: 0x004A, type: float32, unit: kWh, device_class: energy, state_class: total_increasing, ha_component: sensor }
|
||||
- { key: total_energy, address: 0x0156, type: float32, unit: kWh, device_class: energy, state_class: total_increasing, ha_component: sensor }
|
||||
# 冷门量(demands / maxima / 无功电能…)可后续按需补进 metrics;未列出的就不读。
|
||||
```
|
||||
|
||||
> 注意 YAML 里**没有** unit_id / friendly_name——那些是 `modbus_device` 行各自带的。
|
||||
|
||||
### 3.2 数据模型(新增两张通用表,单库 app 链)
|
||||
|
||||
**`modbus_device`**(设备定义 = 部署/可配置项,CRUD 管理)
|
||||
|
||||
| 列 | 类型 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| id | int PK | 内部代理主键(FK join 用)|
|
||||
| uuid | str unique | uuid4 生成的**稳定内部身份**;也作 API 路径键与 HA `unique_id` 的锚 |
|
||||
| friendly_name | str | 显示名(可改;改名重发 discovery,HA 显示名跟着变)|
|
||||
| transport | str | 现仅 `'tcp'` |
|
||||
| host | str | 网关 IP |
|
||||
| port | int | 网关端口(默认 502)|
|
||||
| unit_id | int | Modbus 从机地址 = 电表 Meter ID(设备面板可改,默认 1)|
|
||||
| profile | str | 用哪个 YAML profile(首个 `'sdm120'`)|
|
||||
| poll_interval_s | int | 采样周期(默认 5)|
|
||||
| enabled | bool | 是否轮询 |
|
||||
| last_poll_at | datetime null | 最近一次轮询时刻(online 判定)|
|
||||
| last_poll_ok | bool null | 最近一次轮询成败(online 判定)|
|
||||
| created_at / updated_at | datetime | |
|
||||
|
||||
**`modbus_reading`**(通用遥测表,一行 = 一个设备的一次采样)
|
||||
|
||||
| 列 | 类型 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| id | int PK | |
|
||||
| device_id | int FK→modbus_device.id | **ON DELETE RESTRICT**(见 §5 删除语义)|
|
||||
| recorded_at | datetime (UTC) | 采样时刻,**真实列、带索引**(所有查询按它走时间窗)|
|
||||
| payload | JSON | profile 解码出的全部工程量 `{key: value}`;无固定列 |
|
||||
|
||||
- 索引:`(device_id, recorded_at)`。
|
||||
- **SDM120 单相 payload 示例**:`{"voltage": 230.2, "current": 1.3, "active_power": 295.0, "power_factor": 0.98, "frequency": 50.0, "import_energy": 123.4, "export_energy": 0.0, "total_energy": 123.4}`——key 由 `sdm120.yaml` 的 `metrics[].key` 决定。
|
||||
- **接入新设备型号无需改表**:换 profile,payload 里的 key 集合随之变;表结构不动。
|
||||
- **三相电表**以后用三相 profile,payload 里多几个相位 key(如 `voltage_l1/l2/l3`),仍是同一张表。
|
||||
|
||||
### 3.3 MQTT + HA Discovery(通用 expose 框架,元数据由 profile 派生)
|
||||
|
||||
HA MQTT Discovery 模型 = **device → entities**:往 `<prefix>/<component>/<node>/<object>/config` 发 retained 消息定义一个 entity;config 内 `device.identifiers` 相同的 entity 归到同一个 HA device 卡片下;之后往 `state_topic` 推值。
|
||||
|
||||
- **可暴露实体目录由 provider 动态产出**:
|
||||
- `app/integrations/expose.py`:定义 `ExposableEntity`(`key`(稳定)、`component`(sensor/binary_sensor/switch…)、`device`(归属,决定 HA device 分组)、`device_class`、`unit`、取值来源)+ 一个 provider 注册表。
|
||||
- **Modbus/energy provider**:每个 `enabled` 设备 = 一个 HA **device**(`identifiers` 用设备 `uuid`),其各工程量 = 一组 sensor entity——**`device_class`/`unit`/`component` 直接取自该设备 profile 的 `metrics[]`**(不再单独维护一份映射),外加一个 `binary_sensor`「online」(取 `last_poll_ok`)——这就是"**不止 sensor**"的体现。
|
||||
- 其它 provider(如 public-ip、poo)可后续挂入,本里程碑只接 Modbus/energy provider。
|
||||
- **HA 实体身份锚定(Z2M 模型)**:discovery config 的 **`unique_id` 用设备 `uuid` + 量的 `key` 派生**(稳定,不随改名变);可见的 **`name` 用 friendly_name**。改 friendly_name → 重发 discovery → HA 显示名跟着变、但 `unique_id` 不变故历史不丢。topic 的 object_id 也用 uuid 派生(稳定、丑无所谓)。
|
||||
- **`exposed_entities` 表**:只存"逐 key 的开关"(`key` unique + `enabled` + `updated_at`)。目录本身由 provider 计算,表只记被勾选的状态(默认未勾 = 不暴露)。
|
||||
- **MQTT 客户端**:`app/integrations/mqtt.py` 用 **paho-mqtt**,`loop_start()` 后台线程;在 lifespan 起/停;支持配置变更后**重连 + 重发 discovery**。
|
||||
- **发布时机**:
|
||||
- discovery config(retained):连接成功时、目录/勾选变更时全量发;取消勾选时发空 payload 清除该 entity。
|
||||
- state:采集在每次轮询后推最新值;另有一个周期 job 兜底重发所有 enabled 实体的 state + availability(在线)topic。
|
||||
- **配置**(走现有扁平 CONFIG_FIELDS):`MQTT_ENABLED`、`MQTT_BROKER_HOST/PORT/USERNAME/PASSWORD(secret)`、`MQTT_TLS_ENABLED`、`HA_DISCOVERY_ENABLED`、`HA_DISCOVERY_PREFIX`(默认 `homeassistant`)。
|
||||
|
||||
### 3.4 前端
|
||||
|
||||
- **侧边栏**:把 `AppLayout` 从顶栏重构为侧边导航(Mantine `AppShell` 或 flex sidebar),导航项:Home / Records / Energy / Config + 主题切换 + 注销;当前路由高亮;移动端可折叠。仅改 `App.tsx`(+ 可抽 `AppSidebar`/`NavItem` 组件),各页面主体不动。
|
||||
- **Energy 视图**(新页 `/energy`,首个领域视图,消费通用 `/api/modbus` 数据):
|
||||
- 设备管理:列表 + 新建/编辑/删除(删除有二次确认;后端对有读数的设备拒删,引导改用"禁用")。新建/编辑表单里设 friendly_name、host、port、unit_id、profile(下拉选 `sdm120` 等)、poll_interval、enabled。
|
||||
- 最新读数卡片(每设备当前各工程量;字段标签/单位取自 profile 的 metrics 元数据,见 `/metrics` 端点)。
|
||||
- 走势图:用 **Recharts** 画时间序列(电压/电流/功率/电能),时间范围选择,取数走 readings API(窗口 + 上限),从 `payload` 里按 key 取序列。
|
||||
- **Expose 设置**:设置页内一块「Home Assistant Expose」——列出可暴露实体目录、逐项勾选、显示 MQTT/Discovery 连接状态、一个"重新发布 discovery"按钮。
|
||||
|
||||
## 4. API 契约(M5 要落地的端点)
|
||||
|
||||
> 全部 `/api` 前缀、session + CSRF(写)保护、JSON 进出。schema 经 `export_openapi.py` 固化入库。路径键用设备 `uuid`(稳定、非自增)。
|
||||
|
||||
| 分组 | 端点 | 用途 |
|
||||
| --- | --- | --- |
|
||||
| Modbus | `GET /api/modbus/devices` | 列出设备 |
|
||||
| Modbus | `POST /api/modbus/devices` | 新建设备 |
|
||||
| Modbus | `GET /api/modbus/devices/{uuid}` | 单个设备 |
|
||||
| Modbus | `PATCH /api/modbus/devices/{uuid}` | 修改设备(含 enable/disable)|
|
||||
| Modbus | `DELETE /api/modbus/devices/{uuid}` | 删除设备;**有读数时 409**,引导改 disable |
|
||||
| Modbus | `GET /api/modbus/devices/{uuid}/metrics` | 该设备 profile 的量目录(key/label/unit/device_class),供前端渲染卡片与图表标签 |
|
||||
| Modbus | `GET /api/modbus/devices/{uuid}/latest` | 该设备最新一条读数(payload)|
|
||||
| Modbus | `GET /api/modbus/devices/{uuid}/readings` | 时间范围读数(`start/end/limit`,limit 有上限),返回 `recorded_at + payload`,供走势图 |
|
||||
| Modbus | `POST /api/modbus/devices/{uuid}/test` | 即时试读一次(验证网关连通/地址),返回解码 payload,不落库 |
|
||||
| Modbus | `GET /api/modbus/profiles` | 列出可用 profile 名 + 描述(前端建设备时的下拉选项)|
|
||||
| Expose | `GET /api/expose` | 返回可暴露实体目录 + 勾选状态 + MQTT/Discovery 状态 |
|
||||
| Expose | `PUT /api/expose` | 设置逐 key 勾选(map key→bool)|
|
||||
| Expose | `POST /api/expose/republish` | 手动重发 discovery |
|
||||
| 配置 | `POST /api/config/mqtt/test` | 试连 broker **并发布一条测试消息**(MQTT Explorer 可见),仿 SMTP 测试三态 |
|
||||
|
||||
> MQTT broker / discovery 的**标量配置**复用现有 `GET/PUT /api/config`(只新增 CONFIG_FIELDS,不新增端点)。
|
||||
|
||||
## 5. 已锁定决策(讨论后拍板)
|
||||
|
||||
1. **里程碑编排**:一个 M5 文档分三段,`Depends` 串顺序(A 侧边栏 → B Modbus/Energy → C MQTT)。
|
||||
2. **两层数据模型,协议与部署分离**:`modbus_device`(部署/可配置项:friendly_name、host、port、unit_id、profile 名、poll、enabled)+ `modbus_reading`(通用遥测:device_id、recorded_at、`payload` JSON)。取代原"宽表 + 固定列"。
|
||||
3. **读数 = JSON `payload`,无固定列**;聚合走 SQLite `json_extract`(DB 端做 AVG/MAX/GROUP BY),热点量后补 generated column + 表达式索引。两级周期/降采样为后续杠杆。
|
||||
4. **协议知识 = 仓库内只读 YAML profile**(声明式"数据 + 函数",无 OOP 继承,pydantic 启动期校验);YAML 携带寄存器 + 每个量的 key/unit/device_class/ha_component,是**唯一真相源**,同时驱动"读 / 存 / HA 注册"。多设备可共享一个 profile。首个 profile `sdm120`。
|
||||
5. **Modbus 仅 TCP**,pymodbus,framer 配网关模式,**只读**采集。设备是网关后面的 slave,`unit_id` 寻址(设备面板可改,故落 DB)。
|
||||
6. **命名分层(方案 C)**:存储/采集/API 一律通用 `modbus_*`、`/api/modbus/devices`;前端首个领域视图叫 **Energy**(消费通用 device 数据)。
|
||||
7. **UUID = 内部生成(uuid4)的稳定身份**:既做内部索引/ API 路径键,也做 HA discovery `unique_id` 的锚;friendly_name 可改,改名重发 discovery、HA 显示名跟着变(Z2M 模型,历史不丢)。
|
||||
8. **MQTT = 通用 expose 框架**:provider 动态产出可暴露实体目录,**实体元数据(device_class/unit/component)从 profile 派生**;`exposed_entities` 只存逐 key 开关;支持 sensor/binary_sensor/switch 等多 component;设备自动注册(每设备一 device、各量为 entity + 一个 online binary_sensor)。
|
||||
9. **MQTT 库 = paho-mqtt**,lifespan 长连接,配置变更后重连 + 重发 discovery。
|
||||
10. **MQTT broker/discovery 标量配置走现有扁平 CONFIG_FIELDS**(自动渲染);设备清单与 expose 勾选走专用表 + 自定义 UI。
|
||||
11. **图表库 = Recharts**(封在自包含组件后,仿 M2 对 Leaflet 的隔离)。
|
||||
12. **删除设备安全**:FK `ON DELETE RESTRICT`,有读数拒删(避免一键删表丢历史);"停用"用 `enabled=false`。
|
||||
13. **Config 页用 Accordion 分区**(进页见大类、逐类展开),**不在 config 页内再放第二个 side nav**——避免与主侧栏(T01)的"双抽屉"冲突;纯前端、独立任务 M5-T01B。
|
||||
14. **CLI 手工测试工具为"受控手工链路验证"而设**(设备接市电、非随时在线,不进自动化):`scripts/modbus_cli` 提供 `read`(profile 解码)与 `probe`(手工指定请求内容、看原始回复)两个子命令,**一律只读**(仅 FC03/04),不暴露写寄存器。
|
||||
15. **MQTT/HA 发布链路的手工验证走 UI + 外部工具,不另做 CLI**:Config 页「发送测试消息」(`mqtt/test` 发一条到测试 topic)→ 在 **MQTT Explorer** 查看;Expose 勾选实体 + 开 `HA_DISCOVERY_ENABLED` + 「重新发布 discovery」(`/api/expose/republish`)→ 到 **Home Assistant** 查看。不通则迭代配置再重发。
|
||||
|
||||
> 项目定位:个人自用、家庭特化、不开源——可按单用户场景简化,不过度抽象。
|
||||
|
||||
## 6. 任务依赖图
|
||||
|
||||
```
|
||||
Phase A(独立,可最先做,纯前端)
|
||||
M5-T01 [structural] 侧边栏布局重构
|
||||
M5-T01B Config 页分区折叠(Accordion) ← 与 T01 互不依赖,可并行/先后任意
|
||||
|
||||
Phase B(Modbus 采集 + Energy 前端)
|
||||
M5-T02 [schema] modbus_device + modbus_reading 表 + 模型
|
||||
├─► M5-T03 Modbus 驱动 + YAML profile 框架(pymodbus + sdm120.yaml,纯模块)
|
||||
│ └─► M5-T04 采集 service + APScheduler 轮询(接 lifespan,落 payload)
|
||||
└─► M5-T05 Modbus JSON API(device CRUD + readings + metrics + test)
|
||||
└─► M5-T06 前端:设备管理 UI(依赖 T01 侧栏 + T05 API)
|
||||
└─► M5-T07 前端:读数展示 + Recharts 走势图(依赖 T05;引入 recharts)
|
||||
|
||||
Phase C(MQTT / Discovery,依赖 B 的设备数据与 provider 接口)
|
||||
M5-T08 MQTT/Discovery 配置项(CONFIG_FIELDS)
|
||||
M5-T09 [schema] exposed_entities 表 + ExposableEntity/provider 框架(modbus provider 从 profile 派生)
|
||||
├─► M5-T10 MQTT 客户端(paho,lifespan 连接 + 重连 + config/mqtt/test)
|
||||
│ └─► M5-T11 Discovery 发布 + state 发布(连 T04 轮询推 state)
|
||||
└─► M5-T12 前端:Expose 勾选 UI + /api/expose API
|
||||
|
||||
收尾
|
||||
M5-T13 文档 + OpenAPI + roadmap 收尾(依赖全部)
|
||||
```
|
||||
|
||||
`T01`、`T01B`、`T02`、`T08` 无前置可先开。
|
||||
|
||||
---
|
||||
|
||||
## 7. 原子任务(任务卡)
|
||||
|
||||
> 后端任务沿用校验闸门(`pytest` / `ruff` / 改路由或 schema 则 `export_openapi` 重导出入库)。前端任务闸门见 §8。新增依赖(`pymodbus`、`PyYAML`、`paho-mqtt`、`recharts`)须在对应任务里同步 `requirements.in/.txt` 或 `frontend/package.json` 并重新锁定。
|
||||
|
||||
### M5-T01 — 侧边栏布局重构 `[structural]`
|
||||
- **Status**: `done` · **Depends**: none
|
||||
- **Context**: 把 `AppLayout` 从顶栏改为侧边导航,给后续 Energy 等视图腾入口。纯前端,不碰各页主体。
|
||||
- **Files**: `modify frontend/src/App.tsx`;`create frontend/src/components/AppSidebar.tsx`、`frontend/src/components/NavItem.tsx`(可选);`modify` 受影响的 `frontend/src/pages/*.test.tsx`(导航断言)
|
||||
- **Steps**:
|
||||
1. 用 Mantine `AppShell`(或 flex sidebar)重构 `AppLayout`:左侧竖直导航(Home/Records/Config + 主题切换 + 注销),`<Outlet/>` 在右。
|
||||
2. 当前路由高亮(`useLocation` 比对 `pathname`);移动端可折叠(burger)。
|
||||
3. 导航项图标沿用 `react-feather`;样式走 Mantine(暗色模式自动适配)。
|
||||
4. 不在本任务加 Energy 项(页面还不存在,T06 加),保持导航无死链。
|
||||
- **Out of scope / 不要碰**: 不改各页面主体;不动鉴权(SessionProvider/ProtectedRoute);不引入图表库。
|
||||
- **Acceptance criteria**:
|
||||
- [ ] 受保护页都在侧边栏布局内;`/login`、`/change-password` 不带布局(与现状一致)。
|
||||
- [ ] 当前路由在侧栏高亮;移动端宽度下可折叠/展开。
|
||||
- [ ] 前端闸门全绿(`lint`/`typecheck`/`test`/`build`)。
|
||||
- **Reviewer checklist**: 布局只在 `App.tsx`/新组件内变动,未误改页面或鉴权;无死链导航项。
|
||||
|
||||
### M5-T01B — Config 页分区折叠(Accordion)
|
||||
- **Status**: `done` · **Depends**: none
|
||||
- **Context**: config 内容会越来越多(M5 还要加 MQTT / HA Discovery / Modbus 配置 + Expose 面板)。把 ConfigPage 从一长串 section 改为 Mantine `Accordion`:进页只见各大类标题,逐类展开编辑。**纯前端、单列内容**——它不是"第二个 app 级侧栏",与主侧栏(T01)无冲突,故 `Depends: none`、可独立先做。
|
||||
- **Files**: `modify frontend/src/pages/ConfigPage.tsx`;`modify frontend/src/pages/ConfigPage.test.tsx`(折叠/展开断言)
|
||||
- **Steps**:
|
||||
1. 用 Mantine `Accordion` 包住现有"按 section 分组"的渲染:每个 config section = 一个 `Accordion.Item`(标题 = section 名,面板 = 该 section 的字段表单)。
|
||||
2. 默认折叠(或首个展开);**保留现有保存逻辑与 config API 不变**(M2 的整页/按 section 保存语义照旧,本任务不碰后端)。
|
||||
3. 预留:将来 T12 的「Home Assistant Expose」自定义面板也作为一个 `Accordion.Item` 接入,保持一致。
|
||||
4. 移动端单列堆叠即可,**不引入第二个侧栏/抽屉**。
|
||||
- **Out of scope / 不要碰**: 不改后端 config API/字段(`CONFIG_FIELDS` / `_settings_payload`);不动主侧栏(T01);不改保存语义;不在 config 页内放任何 app 级 side nav。
|
||||
- **Acceptance criteria**:
|
||||
- [ ] ConfigPage 以 accordion 呈现,每大类可独立展开/折叠;字段渲染与保存行为与现状一致。
|
||||
- [ ] 与主侧栏无视觉/交互冲突(单列内容;移动端不出现双抽屉)。
|
||||
- [ ] 前端闸门全绿(`lint`/`typecheck`/`test`/`build`)。
|
||||
- **Reviewer checklist**: 仅改 ConfigPage(+其测试),未碰 config 后端或主 layout;保存逻辑无回归;页面内**无第二个 app 级 sidebar**(accordion 是内容、非 chrome)。
|
||||
|
||||
### M5-T02 — `modbus_device` + `modbus_reading` 表与模型 `[schema]`
|
||||
- **Status**: `done` · **Depends**: none
|
||||
- **Context**: 单库 app 链新增两张**通用**表,建出 §3.2 结构(设备 = 部署层,读数 = JSON payload 通用遥测层)。本任务只建 schema + 模型,不写采集/接口。
|
||||
- **Files**: `create app/models/modbus.py`(`ModbusDevice`、`ModbusReading`,继承 `app.db.Base`);`create alembic_app/versions/<date>_07_modbus_tables.py`;`modify alembic_app/env.py`(import 新模型);`modify scripts/app_db_adopt.py`(`APP_BASELINE_REVISION` → 新 head);`create tests/test_modbus_models.py`
|
||||
- **Steps**:
|
||||
1. 模型按 §3.2 列定义(`Mapped[...]` 2.0 风格):`ModbusDevice` 含 `uuid`(unique)、`friendly_name`、`transport`、`host`、`port`、`unit_id`、`profile`、`poll_interval_s`、`enabled`、`last_poll_at`/`last_poll_ok`(nullable)、时间戳;`uuid` 用 `default` 生成 uuid4 字符串。`ModbusReading` 含 `device_id` FK→`modbus_device.id`(**`ondelete="RESTRICT"`**)、`recorded_at`、`payload`(JSON 列,非 null)。
|
||||
2. 新 revision:`down_revision` = 当前 head;`upgrade()` 用 `op.create_table` 建两表 + 索引 `(device_id, recorded_at)`;`downgrade()` 反向 drop。
|
||||
3. 更新 `APP_BASELINE_REVISION`。
|
||||
- **Out of scope / 不要碰**: 不写 pymodbus/采集(T03/T04);不加路由(T05);不动其它模型。
|
||||
- **Acceptance criteria**:
|
||||
- [ ] 全新临时 app 库 upgrade 到 head 后含 `modbus_device`、`modbus_reading` 及索引;`downgrade -1` 干净回滚。
|
||||
- [ ] `Base.metadata.tables` 含两新表;FK 为 RESTRICT;`uuid` 唯一且自动生成。
|
||||
- [ ] `APP_BASELINE_REVISION` == 新 head。
|
||||
- [ ] 校验闸门全绿。
|
||||
- **Reviewer checklist**: 列/约束与 §3.2 一致;`payload` 为 JSON 列;`recorded_at` 为真实索引列(非塞进 payload);链上单 head;`env.py` 已 import 新模型(否则 autogenerate/建表漏表)。
|
||||
|
||||
### M5-T03 — Modbus 驱动 + YAML profile 框架(`sdm120`)
|
||||
- **Status**: `done` · **Depends**: M5-T02
|
||||
- **Context**: 薄封装 pymodbus 的连接/块读/大端 float 解码,加 YAML profile 加载/校验/解码框架与 SDM120 profile。纯模块,mock client 单测。
|
||||
- **Files**: `create app/integrations/modbus/__init__.py`、`app/integrations/modbus/driver.py`、`app/integrations/modbus/profiles.py`、`app/integrations/modbus/profiles/sdm120.yaml`、`scripts/modbus_cli.py`;`modify requirements.in`/`requirements.txt`(加 `pymodbus`、`PyYAML`,重新锁定);`create tests/test_modbus_driver.py`、`tests/test_modbus_profiles.py`、`tests/test_modbus_cli.py`
|
||||
- **Steps**:
|
||||
1. `driver.py`:`read_blocks(host, port, unit_id, blocks) -> dict[int,int]`(按 profile 的 blocks 块读 input registers,FC04),用 `ModbusTcpClient`;超时/连接失败抛明确异常;大端 float32 解码 helper(`registers_to_float`,高寄存器在前)。framer 选择留可配置/可探测。
|
||||
2. `profiles.py`:定义 pydantic `ModbusProfile` / `MetricSpec`(`key`/`address`/`type`/`unit`/`device_class`/`state_class`?/`ha_component`);`load_profile(name) -> ModbusProfile`(读 `profiles/<name>.yaml`,校验失败抛错);`decode(profile, registers) -> dict[key -> value]`(按各 metric 的 address+type 从寄存器对解码);`list_profiles() -> list[(name, description)]`。**不做抽象基类/继承**,全是数据 + 模块函数。
|
||||
3. `profiles/sdm120.yaml`:按 §3.1 示例写全核心量(参考文档 §4 地址)。
|
||||
4. `scripts/modbus_cli.py`:纯命令行、**不依赖 DB / 不需先配设备**,供**受控手工测试**(设备接市电、非随时在线,不进自动化)。两个**只读**子命令:
|
||||
- `read --host H --port P --unit U --profile sdm120`:按 profile 读一次、解码后**把各工程量打印成可读结果**(表格/JSON)——验证整套解码链路。
|
||||
- `probe --host H --port P --unit U --fc 4 --address 0x0000 --count 2 [--decode float32]`:**手工指定要发送的请求内容**(功能码 FC03/04 + 起始地址 + 数量),打印**原始寄存器(hex)+ 可选大端 float 解码**——first-contact 验证网关连通、framer 模式与 unit 地址,可单读一个寄存器,不依赖 profile。
|
||||
连接失败给清晰报错 + 非零退出。**只读**:CLI 仅暴露读功能码(FC03/04),**不提供任何写寄存器子命令**(数据红线 + 防改坏电表通信参数)。
|
||||
- **Out of scope / 不要碰**: 不连真实硬件(单测用 mock/fake 返回已知寄存器字节);不写调度(T04);不写电表配置寄存器;不在 profile 里放 unit_id/friendly_name。
|
||||
- **Acceptance criteria**:
|
||||
- [ ] 单测:给定 `0x4366,0x3334` 解码为 `230.2`(参考文档实例);字序/字节序正确。
|
||||
- [ ] 单测:`load_profile("sdm120")` 校验通过;`decode(profile, ...)` 把已知寄存器映射到正确 key 与值。
|
||||
- [ ] 单测:profile YAML 缺字段/类型错时 `load_profile` 抛可识别校验错。
|
||||
- [ ] 连接失败/超时抛可识别异常,不静默返回错值。
|
||||
- [ ] `python -m scripts.modbus_cli read ...` 能(对 mock/真实网关)打印解码后的各工程量;连接失败非零退出。
|
||||
- [ ] `python -m scripts.modbus_cli probe --fc 4 --address 0x0000 --count 2 ...` 能打印原始寄存器与可选解码值;CLI **无任何写寄存器子命令**。
|
||||
- [ ] 校验闸门全绿。
|
||||
- **Reviewer checklist**: 解码确为大端、高寄存器在前;地址与参考文档一致;**CLI 与 driver 都无任何写寄存器路径(仅 FC03/04)**;profile 纯协议知识、无部署项;`requirements.txt` 已同步锁定 `pymodbus`、`PyYAML`。
|
||||
|
||||
### M5-T04 — 采集 service + APScheduler 轮询
|
||||
- **Status**: `done` · **Depends**: M5-T03
|
||||
- **Context**: 周期扫所有 enabled 设备,调用 driver 读+解码,落 `modbus_reading.payload`。仿 public-ip 的同步 job 模式。
|
||||
- **Files**: `create app/services/modbus_poll.py`;`modify app/main.py`(lifespan 注册 job);`create tests/test_modbus_poll.py`
|
||||
- **Steps**:
|
||||
1. `modbus_poll.py`:`poll_device(session, device) -> ModbusReading | None`(`load_profile` → driver 读 → `decode` → 把 dict 存进 `payload` 插一行;更新 `last_poll_at`/`last_poll_ok`);`poll_all_enabled_devices(session)` 遍历 enabled 设备;service 内吞异常并日志,不让 job 崩。
|
||||
2. `main.py`:加同步 wrapper `_run_scheduled_modbus_poll`(自管 session),`add_job(IntervalTrigger(seconds=...), id="modbus-poll", max_instances=1, coalesce=True)`。周期取**最小 per-device interval 或一个基础 tick**(实现可用单一基础 tick + 各设备按自身 interval 取模决定本 tick 是否读,保持 job 简单);受全局 `MODBUS_POLLING_ENABLED` 控制。
|
||||
3. 失败的设备记 `last_poll_ok=false`(供 T11 暴露),不影响其它设备。
|
||||
- **Out of scope / 不要碰**: 不发 MQTT(T11);不加 HTTP 路由(T05);不引入两级周期(后续杠杆)。
|
||||
- **Acceptance criteria**:
|
||||
- [ ] 单测:mock driver 返回已知 dict,`poll_all_enabled_devices` 后 `modbus_reading` 精确 +N 行、`payload` 内容正确。
|
||||
- [ ] 单测:某设备读失败时其它设备仍正常落库,job 不抛;失败设备 `last_poll_ok=false`。
|
||||
- [ ] `MODBUS_POLLING_ENABLED=false` 时不轮询。
|
||||
- [ ] 校验闸门全绿。
|
||||
- **Reviewer checklist**: session 在 wrapper 内开关、try/finally 关闭;job `max_instances=1` 防叠加;无 N+1/每行单独 connect 的明显低效;异常不外泄崩 job;payload 为解码后的 dict(非裸寄存器)。
|
||||
|
||||
### M5-T05 — Modbus JSON API(device CRUD + readings + metrics + test)
|
||||
- **Status**: `done` · **Depends**: M5-T02
|
||||
- **Context**: 给前端提供设备 CRUD、最新读数、时间范围读数、量目录、即时试读。
|
||||
- **Files**: `create app/api/routes/api/modbus.py`、`app/schemas/modbus.py`;`modify app/main.py`(注册路由);`create tests/test_api_modbus.py`
|
||||
- **Steps**:
|
||||
1. devices:`GET`(list)/`POST`/`GET{uuid}`/`PATCH{uuid}`/`DELETE{uuid}`;session+CSRF;`POST` 校验 profile 名存在;`DELETE` 有读数 → 409。
|
||||
2. readings:`GET {uuid}/latest`(最新一行 payload)、`GET {uuid}/readings`(`start/end/limit`,limit 有上限防全表导出,按 `recorded_at` 升序,返回 `recorded_at + payload`)。
|
||||
3. `GET {uuid}/metrics`:返回该设备 profile 的量目录(key/label/unit/device_class),供前端渲染。
|
||||
4. `GET /api/modbus/profiles`:`list_profiles()` 的名+描述。
|
||||
5. `POST {uuid}/test`:用 driver 即时读一次返回解码 payload(或错误),**不落库**。
|
||||
- **Out of scope / 不要碰**: 不在此处发 MQTT;不写采集逻辑(复用 T03/T04 的 driver/service)。
|
||||
- **Acceptance criteria**:
|
||||
- [ ] CRUD 行为正确:创建/改/删行数精确;删有读数的设备返回 409;未登录 401、缺 CSRF 403;建设备引用不存在 profile → 422。
|
||||
- [ ] readings 时间范围 + limit 上限生效;latest 返回最新一条 payload;metrics 返回 profile 量目录。
|
||||
- [ ] schema 经 OpenAPI 固化入库。
|
||||
- [ ] 校验闸门全绿(含 `openapi/` 重导出)。
|
||||
- **Reviewer checklist**: 删除受 RESTRICT 保护、无批量删/清表路径;查询走 `(device_id, recorded_at)` 索引;test 端点确不落库;路径键用 `uuid`。
|
||||
|
||||
### M5-T06 — 前端:设备管理 UI(Energy 视图)
|
||||
- **Status**: `done` · **Depends**: M5-T01, M5-T05
|
||||
- **Context**: 在侧栏加 Energy 入口与 `/energy` 路由;设备增删改 + 试读。
|
||||
- **Files**: `create frontend/src/pages/EnergyPage.tsx`、`frontend/src/energy/DeviceForm.tsx`、`frontend/src/energy/hooks.ts`;`modify frontend/src/App.tsx`(路由)、`frontend/src/components/AppSidebar.tsx`(Energy 项);`create` 对应 `*.test.tsx`
|
||||
- **Steps**: `useQuery`/`useMutation` 接 `/api/modbus/devices` API;列表 + 新建/编辑表单(friendly_name/host/port/unit_id/profile 下拉/poll/enabled)+ 删除二次确认(删失败 409 提示改用禁用);"试读"按钮调 `POST {uuid}/test` 显示结果。
|
||||
- **Out of scope / 不要碰**: 走势图在 T07;不碰其它页面。
|
||||
- **Acceptance criteria**:
|
||||
- [ ] 能增/改/删设备并即时刷新;删除有二次确认;409 有友好提示;profile 走 `/api/modbus/profiles` 下拉。
|
||||
- [ ] 侧栏出现 Energy 入口、`/energy` 可达。
|
||||
- [ ] 前端闸门全绿。
|
||||
- **Reviewer checklist**: 全部走生成的类型化 client;删除走确认;无与契约不符的手写请求。
|
||||
|
||||
### M5-T07 — 前端:读数展示 + Recharts 走势图
|
||||
- **Status**: `done` · **Depends**: M5-T05(数据), M5-T06(页面壳)
|
||||
- **Context**: 在 Energy 页展示每设备最新读数 + 时间序列走势。
|
||||
- **Files**: `modify frontend/src/pages/EnergyPage.tsx`;`create frontend/src/energy/EnergyCharts.tsx`(封装 Recharts);`modify frontend/package.json`(加 `recharts`,`package-lock.json` 同步);`create` 对应测试
|
||||
- **Steps**: 最新读数卡片(接 `latest`,字段标签/单位取自 `/metrics`);时间范围选择 + 折线图(电压/电流/功率/电能,从 `payload` 按 key 取序列),接 `readings`(窗口 + limit);图表封在 `EnergyCharts` 内(仿 Leaflet 隔离,便于将来换库)。
|
||||
- **Out of scope / 不要碰**: 不做服务端降采样(后续);不改后端。
|
||||
- **Acceptance criteria**:
|
||||
- [ ] 最新读数与走势图渲染正确;时间范围只取窗口数据(不拉全量);标签/单位来自 metrics。
|
||||
- [ ] Recharts 封装自包含、仅此处 import。
|
||||
- [ ] 前端闸门全绿(`build` 通过,注意 chunk 体积提示)。
|
||||
- **Reviewer checklist**: 图表组件隔离;查询有窗口/上限;空数据/加载/错误态有处理;从 payload 取 key 的逻辑容忍缺 key。
|
||||
|
||||
### M5-T08 — MQTT / Discovery 配置项(CONFIG_FIELDS)
|
||||
- **Status**: `done` · **Depends**: none
|
||||
- **Context**: 把 MQTT broker 与 discovery 的标量配置接入扁平配置系统(前端自动渲染)。
|
||||
- **Files**: `modify app/config.py`(新增 Settings 字段)、`app/services/config_page.py`(追加 CONFIG_FIELDS + `_settings_payload`);`modify .env.example`;`modify tests/test_api_config.py`
|
||||
- **Steps**: 加字段 `mqtt_enabled`、`mqtt_broker_host/port/username/password`(secret)、`mqtt_tls_enabled`、`ha_discovery_enabled`、`ha_discovery_prefix`(默认 `homeassistant`)、`modbus_polling_enabled`;CONFIG_FIELDS 归入「MQTT」「Home Assistant Discovery」「Modbus」section;`_settings_payload` 补齐对应行。
|
||||
- **Out of scope / 不要碰**: 不建 MQTT 客户端(T10);不动 expose 表(T09)。
|
||||
- **Acceptance criteria**:
|
||||
- [ ] 新配置项在 `GET /api/config` 出现且分 section;password 为 secret(回空、留空保留);port 为 number。
|
||||
- [ ] 非法值(端口非数字)422 不写库。
|
||||
- [ ] 校验闸门全绿(OpenAPI 若变化则重导出)。
|
||||
- **Reviewer checklist**: secret 不回显/不入 OpenAPI 示例;`_settings_payload` 未漏字段(否则运行期 override 丢失)。
|
||||
|
||||
### M5-T09 — `exposed_entities` 表 + ExposableEntity/provider 框架 `[schema]`
|
||||
- **Status**: `done` · **Depends**: M5-T02
|
||||
- **Context**: 建"可暴露实体目录"的抽象与开关存储;modbus provider 把设备映射成 device/entities,**实体元数据从 profile 派生**。
|
||||
- **Files**: `create app/integrations/expose.py`(`ExposableEntity`、provider 协议、注册表、modbus provider);`create app/models/expose.py`(`ExposedEntityToggle`:`key` unique + `enabled` + `updated_at`);`create alembic_app/versions/<date>_08_exposed_entities.py`;`modify alembic_app/env.py`、`scripts/app_db_adopt.py`;`create tests/test_expose_catalog.py`
|
||||
- **Steps**:
|
||||
1. `ExposableEntity`(`key/component/device/device_class/unit/value_getter`)+ provider 接口 `enumerate(session) -> list[ExposableEntity]`。
|
||||
2. modbus provider:每个 enabled 设备 → 一个 device(`identifiers` 用设备 `uuid`),其各量 → sensor entity(**device_class/unit/component 取自 profile 的 `metrics[]`**),加一个 `binary_sensor` online(取 `last_poll_ok`)。
|
||||
3. `build_catalog(session)` 合并所有 provider 的实体 + 各自 `enabled`(来自 toggle 表,缺省 false)。
|
||||
4. migration 建 toggle 表;更新 baseline 常量。
|
||||
- **Out of scope / 不要碰**: 不发 MQTT(T11);不加 HTTP(T12)。
|
||||
- **Acceptance criteria**:
|
||||
- [ ] 单测:建若干设备后 `build_catalog` 产出每设备对应 entity(含 online binary_sensor)+ 正确 device 分组、device_class、unit(与 profile 一致)。
|
||||
- [ ] toggle 表 migration 可升/降;缺省 enabled=false。
|
||||
- [ ] 至少含一个非 sensor component(online binary_sensor)。
|
||||
- [ ] 校验闸门全绿。
|
||||
- **Reviewer checklist**: `key` 稳定(用设备 `uuid` + 量 key,不用自增 id,避免重建漂移);component 支持多类型;目录由 provider 计算、元数据源自 profile 而非写死。
|
||||
|
||||
### M5-T10 — MQTT 客户端(paho,lifespan 连接 + 重连)
|
||||
- **Status**: `done` · **Depends**: M5-T08
|
||||
- **Context**: 长连接 MQTT 客户端,配置变更可重连;含连接测试端点。
|
||||
- **Files**: `create app/integrations/mqtt.py`;`modify app/main.py`(lifespan 起/停)、`app/api/routes/api/config.py`(`POST /api/config/mqtt/test`);`modify requirements.in`/`requirements.txt`(加 `paho-mqtt` 重新锁定);`create tests/test_mqtt_client.py`
|
||||
- **Steps**:
|
||||
1. `MqttManager`:`is_configured()`、`connect()/disconnect()/reconnect(settings)`、`publish(topic, payload, retain)`;paho `loop_start()` 后台线程;未配置/未启用则 no-op。
|
||||
2. lifespan:启用则 connect;shutdown disconnect。
|
||||
3. 配置保存后若 MQTT 设置变化 → 触发 manager 重连(在 config 保存路径加 hook 或保存后比对)。
|
||||
4. `POST /api/config/mqtt/test`:用提交/现存配置试连**并发布一条测试消息**到一个测试 topic(如 `<discovery_prefix>/home-automation/test`),返回三态(success/config-error/failed)。仿 SMTP 测试"发一封测试邮件"的语义——用户随后在 **MQTT Explorer** 里就能看到这条消息,确认 broker 发布链路通(这是 MQTT 端的手工验证手段,**不另做 CLI**)。
|
||||
- **Out of scope / 不要碰**: 不构建 discovery/state 消息(T11)。
|
||||
- **Acceptance criteria**:
|
||||
- [ ] 单测(fake broker/paho mock):configured 时 connect 调用正确;未配置 no-op;publish 透传 topic/payload/retain。
|
||||
- [ ] `POST /api/config/mqtt/test` 试连**并发布一条测试消息**(可在 MQTT Explorer 看到);三态有明确返回;session+CSRF 保护。
|
||||
- [ ] 校验闸门全绿。
|
||||
- **Reviewer checklist**: 断网/连接失败不崩主进程;线程在 shutdown 正确停止;`requirements.txt` 同步锁定 `paho-mqtt`;密码不进日志。
|
||||
|
||||
### M5-T11 — Discovery 发布 + state 发布
|
||||
- **Status**: `done` · **Depends**: M5-T09, M5-T10
|
||||
- **Context**: 把 enabled 实体发成 HA discovery config(retained)并周期推 state;采集轮询后推最新值。
|
||||
- **Files**: `create app/services/ha_discovery.py`;`modify app/services/modbus_poll.py`(轮询后推 state)、`app/main.py`(state 周期 job + 连接后/勾选变更后发 discovery)、`app/api/routes/api/...`(`/api/expose/republish` 在 T12 接,本任务提供 service);`create tests/test_ha_discovery.py`
|
||||
- **Steps**:
|
||||
1. `build_discovery_payload(entity)` → HA 规范 config(`<prefix>/<component>/<node>/<object>/config`,含 `device` 块、`state_topic`、`device_class`、`unit_of_measurement`、`availability`;**`unique_id` 用设备 uuid + 量 key 派生,`name` 用 friendly_name**)。
|
||||
2. `publish_discovery(session)`:对 enabled 实体发 retained config;对取消勾选的发空 payload 清除。
|
||||
3. `publish_states(session)`:取各实体当前值发 state;采集在 `poll_device` 成功后顺带推该设备实体 state + online。
|
||||
4. lifespan:连接成功 / 目录或勾选变更后 `publish_discovery`;周期 job 兜底 `publish_states` + availability。
|
||||
- **Out of scope / 不要碰**: 不做前端(T12);不改采集解码逻辑。
|
||||
- **Acceptance criteria**:
|
||||
- [ ] 单测:discovery payload 符合 HA 结构(device 分组正确、topic/device_class/unit 正确、`unique_id` 取自 uuid);取消勾选发空 payload。
|
||||
- [ ] 单测:采集轮询成功后推对应 state topic;失败推 online=false。
|
||||
- [ ] discovery 用 retained。
|
||||
- [ ] 校验闸门全绿。
|
||||
- **Reviewer checklist**: 仅发 enabled 实体;entity `unique_id` 稳定(源自 uuid,不随改名变);MQTT 未启用时整链 no-op;不阻塞轮询。
|
||||
|
||||
### M5-T12 — 前端:Expose 勾选 UI + `/api/expose`
|
||||
- **Status**: `done` · **Depends**: M5-T09, M5-T11
|
||||
- **Context**: 后端 expose 读写端点 + 设置页勾选界面。
|
||||
- **Files**: `create app/api/routes/api/expose.py`、`app/schemas/expose.py`;`modify app/main.py`;`create tests/test_api_expose.py`;前端 `create frontend/src/pages/.../ExposeSettings.tsx`(或并入 ConfigPage)、`modify` 路由/设置入口;`create` 前端测试
|
||||
- **Steps**: `GET /api/expose`(目录 + 勾选 + MQTT/Discovery 状态)、`PUT /api/expose`(key→bool)、`POST /api/expose/republish`(调 T11 service);前端列出目录、按 device 分组、逐项开关、显示连接状态、"重新发布"按钮。
|
||||
- **Out of scope / 不要碰**: 不改采集/发布逻辑(T11)。
|
||||
- **Acceptance criteria**:
|
||||
- [ ] `GET/PUT /api/expose` 正确读写勾选;session+CSRF;OpenAPI 固化。
|
||||
- [ ] 勾选变更后(或点重新发布)触发 discovery 重发。
|
||||
- [ ] 前端能勾选并显示状态;前后端闸门全绿。
|
||||
- **Reviewer checklist**: PUT 只改 toggle 不误碰其它配置;republish 真触发 T11;类型化 client。
|
||||
|
||||
### M5-T13 — 文档 + OpenAPI + roadmap 收尾
|
||||
- **Status**: `done` · **Depends**: 全部
|
||||
- **Files**: `modify README.md`(Modbus/Energy/MQTT 段、新依赖)、`docs/roadmap.md`(M5 行 + 把"MQTT/IoT"从"下一阶段"毕业、新增 Modbus 采集方向)、`docs/architecture-overview.md`(新增 MQTT 通道与 Modbus 采集);`modify docs/design/README.md`(列入 m5);`run python scripts/export_openapi.py` 并提交 `openapi/`
|
||||
- **Acceptance criteria**:
|
||||
- [ ] 文档反映新链路;`git diff --exit-code openapi/` 无未提交差异。
|
||||
- [ ] 校验闸门全绿。
|
||||
- **Reviewer checklist**: 无残留旧描述(含旧 `energy_meters`/`/api/energy` 字样);OpenAPI 已入库。
|
||||
|
||||
---
|
||||
|
||||
## 8. 前端校验闸门(前端任务每次结束都要全绿)
|
||||
|
||||
在 `frontend/` 下:
|
||||
```bash
|
||||
npm ci
|
||||
npm run lint
|
||||
npm run typecheck
|
||||
npm run test
|
||||
npm run build # 必须产出 dist;留意 chunk 体积告警
|
||||
```
|
||||
- 后端若同任务改了路由/schema,仍需根目录 `python scripts/export_openapi.py` 并提交 `openapi/`。
|
||||
- 新增前端依赖(recharts)须提交 `package.json` + `package-lock.json`。
|
||||
|
||||
## 9. 构建上下文完整性(M1 教训)
|
||||
|
||||
- 本里程碑**不删/移文件**,但新增了 Python 依赖(`pymodbus`、`PyYAML`、`paho-mqtt`)与前端依赖(`recharts`):必须同步 `requirements.in/.txt` 的重新锁定与 `package-lock.json`,否则镜像构建会缺包。
|
||||
- 新增源文件都在 `app/`、`scripts/` 与 `frontend/` 既有 COPY 范围内,无需改 `Dockerfile` 的 `COPY`;**但 `app/integrations/modbus/profiles/*.yaml` 是非 .py 资源**——确认 `COPY app ...` 把整个目录(含 YAML)带进镜像,且运行期能按相对路径定位 YAML(建议用 `importlib.resources` 或基于 `__file__` 的路径,别用 CWD 相对路径)。`scripts/modbus_cli.py` 须在镜像里可 `python -m scripts.modbus_cli` 调用;`tests/test_deployment.py::test_dockerfile_copy_sources_exist` 仍应通过。
|
||||
- 发版前置走查(见 CLAUDE.md):真起 app 跑一次轮询、真连一次 broker、前端 Energy 视图人工瞄一眼渲染,再打 tag。
|
||||
|
||||
## 10. 后续杠杆(本里程碑不做,文档留痕)
|
||||
|
||||
- **热点量的 generated column + 索引**:JSON `payload` 默认无法对单个量建索引;某个量若需高频聚合,给 `modbus_reading` 加一列 `GENERATED ALWAYS AS (json_extract(payload,'$.<key>'))` 并建索引——非破坏性、要哪个补哪个。
|
||||
- **两级采样周期**:瞬时量(V/I/P/PF/Hz)快、累计电能慢——9600 总线吃紧或要把功率压到 5s 以下时再开(device 表加 `slow_interval_s`,读数表是通用 payload、无需改结构)。
|
||||
- **保留 / 降采样**:5s 采样长期行数大(≈630 万行/年/设备),加定期降采样或保留窗口任务(独立于本里程碑),`GROUP BY` + `AVG(json_extract(...))` 在 SQL 端做。
|
||||
- **更多 device profile**:三相电表(如 SDM630)新增一个 YAML profile(payload 里多几个相位 key),不改表、不改采集主链。
|
||||
- **更多 expose provider**:public-ip / poo 等挂入 expose 框架。
|
||||
- **写 Modbus 寄存器**:当前只读;如需经 MQTT/UI 控制设备(switch 类),再单独评估安全边界。
|
||||
|
||||
## 11. 人工验收 walkthrough(实现完成后)
|
||||
|
||||
> 重点:用命令行**手工**读到设备数据并展示结果。这些是**受控手工测试**——设备接市电、并非随时在线,故**不纳入自动化测试**(自动化只用 mock);CLI 工具(`modbus_cli read/probe`)就是为这种"我想测的时候手动测一次"而设。可用 `docker compose` 起环境,命令在容器内或容器外跑均可。
|
||||
|
||||
**前提**:网关(Waveshare RTU↔TCP)已上电接入网络,电表 Meter ID 已知(默认 1)。
|
||||
|
||||
**1) 命令行直接试读(不依赖 DB,最快验证)**
|
||||
- 容器内:`docker compose exec <app> python -m scripts.modbus_cli read --host <网关IP> --port 502 --unit 1 --profile sdm120`
|
||||
- 或容器外:`source .venv/bin/activate && python -m scripts.modbus_cli read --host <网关IP> --port 502 --unit 1 --profile sdm120`
|
||||
- 预期:打印解码后的工程量(电压/电流/有功功率/功率因数/频率/导入导出电能等),数值合理;连不上则清晰报错。
|
||||
- **first-contact / 原始验证**:`python -m scripts.modbus_cli probe --host <网关IP> --port 502 --unit 1 --fc 4 --address 0x0000 --count 2 --decode float32` —— 手工指定请求内容、看原始回复(电压寄存器应解出约 230V),用于确认网关 framer 模式与 unit 地址;建 profile / 设备前就能跑。
|
||||
|
||||
**2)(可选)经 API 试读已配置的设备**
|
||||
- 先在前端 Energy 页或 `POST /api/modbus/devices` 建一个设备;
|
||||
- `POST /api/modbus/devices/{uuid}/test` 即时试读,返回解码 payload(不落库)。
|
||||
|
||||
**3)(可选)验证后台轮询落库**
|
||||
- 确认 `MODBUS_POLLING_ENABLED=true` 且设备 `enabled`;等一个采样周期;
|
||||
- 看前端 Energy 视图的最新读数/走势图,或查 `GET /api/modbus/devices/{uuid}/readings` 有新行。
|
||||
|
||||
**4)(可选)手工验证 MQTT / HA Discovery 发布链路**(受控手工步骤,不进自动化)
|
||||
- **broker 发布链路**:配好 MQTT broker 后,在 Config 页点「发送测试消息」(`POST /api/config/mqtt/test`)——它试连并发一条测试消息;打开 **MQTT Explorer** 确认能收到,即链路通。
|
||||
- **HA Discovery**:在 Expose 设置勾选若干实体、开 `HA_DISCOVERY_ENABLED`,点「重新发布 discovery」(`POST /api/expose/republish`);到 **Home Assistant** 确认对应 device/entity 出现、值正确。
|
||||
- **改名验证**:改某设备 friendly_name 后重发,确认 HA 显示名跟着变、历史不丢(`unique_id` 稳定)。
|
||||
- 不通则按需调整配置/勾选再重发即可。
|
||||
|
||||
## 12. 里程碑完成定义(DoD)
|
||||
|
||||
- 后端能按 per-device 周期静默轮询 Modbus-TCP 设备、按 YAML profile 解码落 `modbus_reading.payload`,支持多设备 CRUD。
|
||||
- MQTT 启用时,勾选的实体以 HA Discovery 注册成 device/entities(含非 sensor),state 周期发布;配置变更可重连重发;改 friendly_name 重发后 HA 显示名跟着变、`unique_id` 不变。
|
||||
- 前端侧边栏可切换功能;Energy 视图能管理设备、看最新读数与走势图;设置页可勾选 expose。
|
||||
- 后端 `pytest`/`ruff`/`export_openapi` + 前端 `lint/typecheck/test/build` 全绿且 `openapi/` 已入库。
|
||||
- README / architecture / roadmap / design 索引反映 M5 现实。
|
||||
@@ -0,0 +1,483 @@
|
||||
# M6 — 通用电价层 + DSMR 实时数据 → 实时买卖电费计算(含 Tibber 动态 / 固定合同两种 profile)
|
||||
|
||||
> 阅读前提:先读 [`README.md`](./README.md)(协作模型、任务卡格式、校验闸门、数据安全红线)。本里程碑建立在 **M5(MQTT + HA Discovery + expose 框架 + Energy 视图 + Recharts)** 之上,复用其大部分基建,并把 M5 的"两层模型"(YAML profile 定结构 + DB 行存部署值)从设备采集复制到**电价合同**上。
|
||||
> **状态**:架构与原子任务已拆到可开工。少数价格常数(卖价残差、能源税年值、双费率寄存器映射)等真实 Tibber token / 账单确认,已落成 §13 的可调值 + TODO,不阻塞实现。
|
||||
|
||||
## 1. 目标
|
||||
|
||||
把"电价合同"与"DSMR 实时电表数据"接起来,按 **15 分钟**周期算出**实际买电支出 / 卖电收入**,自己落库留底(不可变、可审计),并通过 MQTT + HA Discovery 反哺 Home Assistant 的 Energy 仪表盘:
|
||||
|
||||
1. **通用电价层(两层模型)**:仓库内**只读 YAML profile 定义合同结构**(`manual` 固定/可变费率、`tibber` 动态电价各一个结构);DB 的 **`EnergyContract`(+ 版本)存可改的数值**(UI 填);**price strategy** 按 `kind` 出价。一次激活一个合同。
|
||||
2. **DSMR 实时数据接入**:订阅 DSMR Reader 的 `dsmr/json`(每秒一条完整 telegram),**整帧存为 JSON blob、按 10 秒降采样**落库(电、气、各相全存,未来加供暖不改表)。
|
||||
3. **实时买卖电费计算(两层)**:每 15 分钟用**电表累计寄存器差值** × 当时合同的买/卖价算计量电费(不可变、快照价);日/月/年**汇总**再加固定费、减能源税抵扣。**不做净计量**(saldering 2027 取消)。
|
||||
4. **反哺 Home Assistant**:复用 M5 expose/Discovery,把当前买/卖价、累计买电支出、累计卖电收入发成 HA 实体(累计用 `total_increasing`)。
|
||||
5. **前端**:Energy 视图下新增合同管理(按 profile 结构生成表单)+ 价格曲线 + 费用走势/明细 + Tibber 测试。
|
||||
|
||||
> **命名分层(延续 M5)**:存储/采集/计算层中性命名(`dsmr_*` / `tibber_price` / `energy_cost_period` / `energy_contract`);面向用户并入既有 **Energy** 视图。
|
||||
|
||||
## 2. 现状(实现者可据此工作,不必通读全仓库)
|
||||
|
||||
**单库 + Alembic(M5 完成态)**
|
||||
- `app/db.py`:`class Base(DeclarativeBase)`,cached engine(已开 `journal_mode=WAL` + `foreign_keys=ON`)。
|
||||
- 单 Alembic 链 `alembic_app/`,**当前 head = `20260622_10_exposed_entities`**;`scripts/app_db_adopt.py` 的 `APP_BASELINE_REVISION` 同。迁移命名 `YYYYMMDD_NN_<desc>.py`,串 `down_revision`;`alembic_app/env.py` 逐个 import 模型。
|
||||
- `modbus_reading` 已确立**通用 JSON payload 遥测**范式(`recorded_at` 真实索引列 + `payload` JSON)——`dsmr_reading` 直接沿用。
|
||||
- M5 两层模型(`app/integrations/modbus/profiles/*.yaml` 定结构 + `modbus_device` 存部署值 + pydantic 启动期校验 + `decode`)是本里程碑电价层的范本。
|
||||
|
||||
**配置系统(扁平 KV,自动渲染)**
|
||||
- `app/config.py` `Settings(BaseSettings)`;`app/services/config_page.py` 的 `CONFIG_FIELDS` 注册表 + `build_runtime_settings(session, bootstrap)`(DB override 合并,**消费方都用它**)+ `_settings_payload`(新字段补一行)。前端 `ConfigPage.tsx` 按 section/`input_type`/`secret` 通用渲染,bool 渲染成开关。
|
||||
- ⚠️ 扁平 KV **装不下"合同清单 + 逐字段数值 + 版本"**——这些走专用表 + 专用 API + 自定义 UI(仿 M5 `modbus_device`)。
|
||||
|
||||
**MQTT(M5,关键:当前只发不收)**
|
||||
- `app/integrations/mqtt.py` `MqttManager`:`is_configured/is_connected/connect/disconnect/reconnect/publish`,paho `loop_start()` 后台线程,VERSION2 回调。**无 `subscribe`/`on_message`**——M6-T06 在此扩订阅端。
|
||||
- `app/integrations/expose.py`:`ExposableEntity`(dataclass,`value_getter: Callable[[Session],Any]`)、`EntityProvider = Protocol(session)->list[ExposableEntity]`、`register_provider` + `build_catalog`。已有 `_modbus_provider`。**M6 加 `_energy_cost_provider` 复用整条 Discovery 链**。
|
||||
- `app/services/ha_discovery.py`:`build_discovery_payload`/`publish_discovery`/`publish_states`/`publish_device_state`。`app/models/expose.py` `ExposedEntityToggle`(默认未勾不暴露)。
|
||||
|
||||
**后台调度(APScheduler,UTC)**
|
||||
- `app/main.py` lifespan:`BackgroundScheduler(timezone="UTC")`,`add_job(..., max_instances=1, coalesce=True)`;同步 wrapper 自管 session、吞异常不崩 job。已有 public-ip / modbus-poll / discovery-state 等 job——**新增 job/订阅不得破坏它们**。
|
||||
|
||||
**依赖**:`httpx`(Tibber GraphQL)、`paho-mqtt`、`pyyaml`、`apscheduler` 均已在 `requirements.in/.txt`。**M6 不新增 Python 依赖**。前端已有 Recharts / TanStack Query / `openapi-fetch` 类型化 client / Mantine。
|
||||
|
||||
## 3. 目标架构
|
||||
|
||||
### 3.0 数据流总览
|
||||
|
||||
```
|
||||
EnergyContract(active, 版本带生效日期) DSMR Reader
|
||||
│ kind=manual → 常数双费率 │ MQTT dsmr/json (1s)
|
||||
│ kind=tibber → API 15min ▼
|
||||
│ 订阅 + 10s 降采样 → dsmr_reading(整帧 JSON blob)
|
||||
│ (kind=tibber 时) │
|
||||
▼ │ 累计寄存器=真值
|
||||
Tibber GraphQL → tibber_price(15min, 不可变) │
|
||||
│ ▼
|
||||
└──────────────► price strategy ◄────── 计费引擎(每 15min 边界)
|
||||
(manual/tibber 出价) │ 寄存器差 × 价,快照
|
||||
▼
|
||||
energy_cost_period(每15min 计量电费, 不可变)
|
||||
│
|
||||
┌────────────────────────┼───────────────────────────┐
|
||||
▼ ▼ ▼
|
||||
日/月/年汇总(+固定费 −抵扣) expose provider + HA Discovery /api/energy/* + 前端
|
||||
```
|
||||
|
||||
三条核心原则:
|
||||
1. **算钱用寄存器差值,不用功率积分**:每 15 分钟能量 = 累计寄存器@末 − @初,电表真值、与采样频率无关。瞬时功率只用于显示(且 HA 里已有,后端不专门用它算钱)。
|
||||
2. **结构固定(YAML)、数值可改(UI/DB)、历史不可变(快照 + 版本)**:profile YAML 定合同长什么样;UI 填数值、改价存新版本;算过的周期快照当时的价,永不被回改。
|
||||
3. **两层费用**:每 15 分钟只算**计量电费**;按天/年的**固定费、能源税抵扣**在**汇总层**加,不污染每周期引擎。三相 / 双费率 / 燃气都靠"通用 blob + profile 结构"容纳,不改表。
|
||||
|
||||
### 3.1 两层电价模型(本里程碑地基,仿 M5)
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────────────────────┐
|
||||
│ 知识层(固定,随代码走) 部署层(可改,落 DB,UI 填) │
|
||||
│ ── pricing profile YAML ── ── EnergyContract 行 ── │
|
||||
│ • kind=manual / tibber • name(UI 自由填) │
|
||||
│ • 有哪些字段、单位、计费语义 • kind(UI 选) │
|
||||
│ • 哪些分档 / 是否动态源 • active(一次一个) │
|
||||
│ • UI 不可编辑 • currency │
|
||||
│ ── EnergyContractVersion ── │
|
||||
│ • effective_from / to │
|
||||
│ • values(JSON,符合 profile) │
|
||||
│ • 改价=加新版本,旧版本保留 │
|
||||
└──────────────────────────────────────────────────────────────┘
|
||||
│ kind 决定 strategy
|
||||
▼
|
||||
price strategy(代码): manual=常数双费率出价; tibber=读 tibber_price 出价
|
||||
```
|
||||
|
||||
- **profile YAML(仓库内只读,定结构)**:`app/integrations/pricing/profiles/manual.yaml`、`tibber.yaml`,pydantic 启动期校验。声明该 kind **有哪些字段、单位、计费语义、哪些分档**——**不放具体数值**。
|
||||
- **`EnergyContract` 行(部署,UI 填值)**:`name`(自由填,不 hardcode "fixed")、`kind`(UI 选)、`active`、`currency`。
|
||||
- **`EnergyContractVersion`(版本/时段,可审计)**:`effective_from`/`effective_to`(null=开口) + `values`(JSON,符合 profile 结构)。**改价 = 加一个新版本行,旧版本保留**;"2026 上半年一个价、下半年另一个价" = 同合同两个版本。引擎算某周期时挑**覆盖该时刻的版本**。
|
||||
- **price strategy(代码,按 kind)**:`manual` 用版本里的双费率常数出价;`tibber` 读 `tibber_price` + 版本里的 energy_tax/sell_adjust 出价。计费引擎对两者一视同仁。
|
||||
|
||||
**`manual.yaml` 结构(块状、可读;值在 UI 填)**
|
||||
|
||||
```yaml
|
||||
kind: manual
|
||||
label: 固定 / 可变费率(NL,双费率)
|
||||
energy:
|
||||
dual_tariff: true # 用 delivered_1/2、returned_1/2 分高低谷
|
||||
buy:
|
||||
normal: { unit: EUR/kWh } # 高价档(_2)
|
||||
dal: { unit: EUR/kWh } # 低价档(_1)
|
||||
sell:
|
||||
normal: { unit: EUR/kWh } # 回送高价
|
||||
dal: { unit: EUR/kWh } # 回送低价(现两档同价,仍分开填)
|
||||
energy_tax: { unit: EUR/kWh } # 能源税,加到买价上(含 VAT)
|
||||
ode: { unit: EUR/kWh, default: 0 } # 现并入能源税
|
||||
standing: # 固定费:UI 按月填,内部摊到每天
|
||||
network_fee: { unit: EUR/month }
|
||||
management_fee: { unit: EUR/month }
|
||||
credits:
|
||||
heffingskorting: { unit: EUR/year } # 能源税抵扣,年度,汇总时扣
|
||||
```
|
||||
|
||||
**`tibber.yaml` 结构**
|
||||
|
||||
```yaml
|
||||
kind: tibber
|
||||
label: Tibber 动态电价(15 分钟)
|
||||
energy:
|
||||
source: tibber_api # buy = total; sell = total − energy_tax − sell_adjust
|
||||
energy_tax: { unit: EUR/kWh }
|
||||
sell_adjust: { unit: EUR/kWh, default: 0 }
|
||||
standing:
|
||||
management_fee: { unit: EUR/month, default: 5.99 }
|
||||
network_fee: { unit: EUR/month }
|
||||
credits:
|
||||
heffingskorting: { unit: EUR/year }
|
||||
```
|
||||
|
||||
> 所有金额含 VAT(用户口径)。**Tibber token** 不在合同里、走 secret config(凭据归 config)。
|
||||
|
||||
### 3.2 DSMR 接入
|
||||
|
||||
- **数据源**:DSMR Reader 的 **Telegram JSON**(单 topic `dsmr/json`,一条 = 一帧完整 telegram,已是解析后的表寄存器值)。**每秒一条**。
|
||||
- **整帧存 JSON blob**(沿用 `modbus_reading` payload 哲学):**不做字段 allowlist**,整帧存下来(电、气 `extra_device_*`、各相全要)。理由:现有燃气、未来供暖、三相——blob 全装,**不改表**;读取时按 key 取。
|
||||
- **实测 JSON 的坑(已确认)**:数值是 **JSON 字符串**(`"20915.154"`)→ 计费读取时转 **Decimal**;缺测相位是 **`null`** → 视为缺测;`id`(telegram 自增)做幂等去重。
|
||||
- **降采样**:仅当 `timestamp` 秒数落在 `dsmr_sample_interval_s`(默认 10)整数倍时落盘,其余丢弃(量从 60 行/分降到 ~6 行/分;秒=00 在网格上,每个 15 分钟边界都抓得到)。
|
||||
- **订阅**:扩 `MqttManager` 订阅端(`on_connect` 里 subscribe、`topic→handler`、`on_message` 分发,handler 在网络线程内只做"解析+降采样+一次短事务"、吞异常不崩连接)。`dsmr_ingest_enabled`(默认 false,opt-in)。
|
||||
- **进口/出口总量** = `delivered_1+delivered_2` / `returned_1+returned_2`;**双费率分档** = `_1`(dal/低) vs `_2`(normal/高)(NL 惯例,§13 确认别接反)。
|
||||
- **三相 / 燃气 / 供暖**:blob 天然容纳,单相→三相只是多几个 key,不改表(详 §10)。
|
||||
|
||||
### 3.3 Tibber 价格接入(仅 `kind=tibber` 时)
|
||||
|
||||
- **客户端**:`app/integrations/tibber/client.py`,httpx POST `https://api.tibber.com/v1-beta/gql`,`Bearer <token>`。
|
||||
- **15 分钟查询**(introspection + demo 实证):
|
||||
```graphql
|
||||
{ viewer { homes { id currentSubscription {
|
||||
priceInfoRange(resolution: QUARTER_HOURLY, first: 96) {
|
||||
nodes { startsAt total energy tax currency level }
|
||||
} } } } }
|
||||
```
|
||||
- `Price` 字段:`total`(=energy+tax,**全包价**,demo 已证 `total==energy+tax`)、`energy`(纯 spot ex-VAT)、`tax`、`startsAt`(含时区偏移)、`currency`、`level`。
|
||||
- 枚举 **`PriceInfoRangeResolution`** 值 `DAILY/HOURLY/QUARTER_HOURLY`(不是 `PriceResolution`);老的 `priceInfo.range` 已废弃。
|
||||
- **已验证(demo key)**:返回 96 节点、间隔 15 分钟、`total==energy+tax`;老数据无真 15min 价时 API 把小时价**重复成 4 个刻钟**,当前 NL 数据则是真 15 分钟。**解析不假设固定间隔/节点数**,按 `startsAt` 逐点存。
|
||||
- **抓取调度**:启动 + 每日 job 拉「今天+明天」,按 `starts_at` **upsert** `tibber_price`(幂等)。次日价约 13:00 CET 发布 → 每日 14:00 CET(12:00 UTC)抓明天,缺失每小时重试。**仅当 active 合同 kind=tibber 且 token 存在时跑**。
|
||||
- **连接测试** `POST /api/energy/tibber/test`:拉一次 current price,三态(success 带当前价 / config-error / failed),仿 M5 `mqtt/test`。
|
||||
|
||||
### 3.4 计费引擎(两层)
|
||||
|
||||
**第一层 — 每 15 分钟计量电费(`energy_cost_period`,不可变、快照价)**
|
||||
- 触发:APScheduler 1 分钟 tick,找"已闭合未算"的 15 分钟周期算(也支持按区间手动重算)。
|
||||
- 单周期 `[t0,t1)`(`t1` 已过):
|
||||
1. 取各寄存器在 `t0`/`t1` 的值(`recorded_at ≤ 边界` 的最后一行,Decimal),算 **per-register 差**:`Δd1,Δd2,Δr1,Δr2`。
|
||||
2. 取 active 合同**在 t0 生效的版本** + 其 strategy:
|
||||
- `manual`:`import_cost = Δd1×(buy_dal) + Δd2×(buy_normal)`(`buy_x = energy_buy_x + energy_tax + ode`);`export_revenue = Δr1×sell_dal + Δr2×sell_normal`。
|
||||
- `tibber`:取覆盖 t0 的 `tibber_price`(`starts_at ≤ t0` 最近一条);`buy = total`、`sell = total − energy_tax − sell_adjust`;`import_cost = (Δd1+Δd2)×buy`、`export_revenue = (Δr1+Δr2)×sell`。
|
||||
3. `net_cost = import_cost − export_revenue`;**upsert** `energy_cost_period`,**快照**当时用的价 + `contract_version_id`。
|
||||
- 缺价/缺数据:跳过或标 `degraded`,留待重算。**不做净计量**(进出口分开累加)。
|
||||
|
||||
**第二层 — 汇总(日/月/年,读时计算,不存表)**
|
||||
- `Σ 周期 net_cost`(区间内)**+ 固定费**(`network_fee + management_fee` 按月→按天 × 天数)**− 能源税抵扣**(`heffingskorting` 按年→按天 × 天数)= **实际应付**。
|
||||
- 固定费/抵扣取 active 合同版本的值;UI 按月/按年填,引擎内部摊到每天。
|
||||
|
||||
### 3.5 数据模型(新增 5 张表,单库 app 链)
|
||||
|
||||
> 单一 P1 智能电表,`dsmr_reading` 不设 device 表;将来第二块表加 `source` 列即可。
|
||||
|
||||
| 表 | 关键列 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| **`dsmr_reading`** | `recorded_at`(idx)、`source_id`(unique,=telegram id)、`payload`(JSON) | 整帧 telegram blob,10s 降采样 |
|
||||
| **`energy_contract`** | `name`、`kind`(manual/tibber)、`active`、`currency`、时间戳 | 合同头;一次一个 active |
|
||||
| **`energy_contract_version`** | `contract_id`(FK)、`effective_from`、`effective_to`(null)、`values`(JSON)、`created_at` | 版本/时段;改价加新版本、旧版本保留(审计)|
|
||||
| **`tibber_price`** | `starts_at`(unique)、`resolution`、`energy`、`tax`、`total`、`level`、`currency`、`fetched_at` | 15min 价缓存,**不可变**;仅 tibber kind |
|
||||
| **`energy_cost_period`** | `period_start`(unique)、`d1/d2/r1/r2_kwh`(周期差)、`import_cost`、`export_revenue`、`net_cost`、`currency`、`pricing`(JSON 快照)、`contract_version_id`(FK)、`degraded`、`computed_at` | 每15min 计量电费,**不可变**(快照价+来源版本)|
|
||||
|
||||
- `energy_cost_period` 存 per-register 差 + 快照价 → 完全可审计、source-agnostic、可重算覆盖。
|
||||
- FK:`energy_contract_version.contract_id → energy_contract.id`(ON DELETE RESTRICT);`energy_cost_period.contract_version_id → energy_contract_version.id`(RESTRICT,保审计链)。
|
||||
|
||||
### 3.6 不可变 / 审计(贯穿)
|
||||
|
||||
1. **算账快照**:`energy_cost_period` 存死当时实际用的价 + `contract_version_id`;改合同 / 改价**不回改**已算周期——"以前的价就是以前的价"。重算是**显式 opt-in**(`POST /recompute`),不自动覆盖历史。
|
||||
2. **合同版本只增不改**:改价 = 新 `energy_contract_version`(带生效日期),旧版本保留,全程审计链。
|
||||
3. **Tibber 价不可变**:过去某 15 分钟的价不会变。
|
||||
4. 三者叠加 → manual / tibber 都满足审计与不可变。
|
||||
|
||||
### 3.7 MQTT / HA expose(复用 M5)
|
||||
|
||||
加 `app/integrations/expose.py::_energy_cost_provider`(`register_provider`),归一个 HA device(`identifiers="energy-cost"`):
|
||||
|
||||
| key | component | 取值 | 用途 |
|
||||
| --- | --- | --- | --- |
|
||||
| `buy_price_now` | sensor | 当前周期/当前档买价(€/kWh,monetary) | HA 当前单价 |
|
||||
| `sell_price_now` | sensor | 当前卖价(€/kWh) | |
|
||||
| `import_cost_total` | sensor | 累计 import_cost(`total_increasing`,monetary,currency) | HA Energy "总成本实体" |
|
||||
| `export_revenue_total` | sensor | 累计 export_revenue(`total_increasing`) | HA Energy 回送补偿 |
|
||||
|
||||
- `value_getter(session)` 从 `energy_cost_period`(累计/当前)取值。沿用 M5 勾选(默认未勾)、`publish_discovery`、`publish_states`(计费 job 后顺带推 + 周期兜底)。
|
||||
|
||||
### 3.8 前端(并入 Energy 视图)
|
||||
|
||||
- **合同管理**:列表 + 新建/编辑(**表单按选中 profile 结构渲染字段**:manual 双费率/能源税/固定费/抵扣;tibber token 提示走 Config)+ 激活(一次一个 active)+ 改价=新版本(带生效日期)+ 版本历史只读展示。
|
||||
- **价格曲线**:今天+明天 15min 买/卖价(tibber)或固定档位(manual)折线(复用 `EnergyCharts`),取 `/api/energy/prices`。
|
||||
- **费用走势 + 明细 + 汇总**:当日/月 import_cost/export_revenue/net 走势 + 每 15min 明细 + 汇总卡片(含固定费/抵扣),取 `/api/energy/costs` + `/costs/summary`。
|
||||
- **Tibber 测试**:Config 页一块「Tibber」,三态(仿 MQTT 测试)。
|
||||
- 全走类型化 client;空/加载/错误态齐全;缺 key 容错。
|
||||
|
||||
## 4. API 契约(M6 要落地的端点)
|
||||
|
||||
> 全部 `/api` 前缀、session + CSRF(写)保护、JSON 进出;schema 经 `scripts/export_openapi.py` 固化入库。
|
||||
|
||||
| 分组 | 端点 | 用途 |
|
||||
| --- | --- | --- |
|
||||
| 合同 | `GET /api/energy/contracts` | 列出合同 + active 标记 |
|
||||
| 合同 | `POST /api/energy/contracts` | 新建合同(kind + 首版本值,按 profile 校验)|
|
||||
| 合同 | `GET /api/energy/contracts/{id}` | 单个合同 + 版本历史 |
|
||||
| 合同 | `PATCH /api/energy/contracts/{id}` | 改名 / 激活(一次一个 active)|
|
||||
| 合同 | `POST /api/energy/contracts/{id}/versions` | 加新版本(改价,带生效日期)|
|
||||
| 合同 | `GET /api/energy/profiles` | 列出 pricing profile 结构(前端按它渲染表单)|
|
||||
| 价格 | `GET /api/energy/prices?start&end` | 区间价格点(曲线)|
|
||||
| 费用 | `GET /api/energy/costs?start&end` | 区间 `energy_cost_period`(走势/明细)|
|
||||
| 费用 | `GET /api/energy/costs/summary?start&end` | 区间汇总(计量电费 + 固定费 − 抵扣)|
|
||||
| 费用 | `POST /api/energy/costs/recompute?start&end` | 幂等重算(补数据/改价后,显式)|
|
||||
| DSMR | `GET /api/energy/dsmr/latest` | 最新 `dsmr_reading` |
|
||||
| 测试 | `POST /api/energy/tibber/test` | 试连 Tibber + 拉当前价,三态 |
|
||||
|
||||
> DSMR/Tibber 的**标量配置 + token** 复用现有 `GET/PUT /api/config`(只加 CONFIG_FIELDS)。**价格数值全在合同里**,不进 flat config。
|
||||
|
||||
## 5. 已锁定决策(讨论后拍板)
|
||||
|
||||
1. **两层电价模型**:profile YAML 定结构(仓库、固定、UI 不可编辑)+ `EnergyContract`(+版本) 存数值(UI 填、版本化)+ strategy 按 kind 出价。仿 M5。
|
||||
2. **kind 不叫 "fixed"**:`manual`(人工填、可双费率、可带时段)/ `tibber`(API 动态);合同 `name` UI 自由填。
|
||||
3. **买价**:tibber = API `total`(全包,已证 total=energy+tax);manual = `energy_buy_档 + energy_tax`。**卖价**:tibber = `total − energy_tax − sell_adjust`;manual = `sell_档`(回送价,无能源税)。均含 VAT。
|
||||
4. **双费率**:manual 用 `delivered_1/2`、`returned_1/2` 分 dal/normal 计价(`_1`=dal/低、`_2`=normal/高);tibber 求和、15min 价不分档。
|
||||
5. **两层费用**:每 15min `energy_cost_period` 只算计量电费(不可变、快照价);日/月/年汇总再加固定费(按月→天)− heffingskorting(按年→天)。
|
||||
6. **回送阶梯罚金(terugleverkosten)不做**:按自然年累计、用户住不到年底算不准——不算、不记、不加功能(留痕见 §10)。
|
||||
7. **DSMR 整帧存 JSON blob**(不做字段 allowlist),10s 降采样;数值转 Decimal、null 容错、`id` 幂等。扩 `MqttManager` 订阅端。
|
||||
8. **5 张表**:`dsmr_reading`、`energy_contract`、`energy_contract_version`、`tibber_price`、`energy_cost_period`。单电表不设 device 表。
|
||||
9. **不可变 / 审计**:算账快照价 + 合同版本只增不改 + tibber 价天然不可变;重算显式 opt-in。
|
||||
10. **Tibber 用 httpx + `priceInfoRange(QUARTER_HOURLY)`**,仅 active=tibber 时抓;token 走 secret config。
|
||||
11. **复用 M5 expose/Discovery**:加 `_energy_cost_provider`;累计成本/收入用 `total_increasing` 喂 HA Energy。
|
||||
12. **前端并入 Energy 视图**:合同管理表单按 profile 结构渲染;Recharts 画价/费。
|
||||
13. **opt-in 开关**:`dsmr_ingest_enabled` 默认 false;Tibber 抓取/计费由 active 合同存在性驱动。
|
||||
14. **周期边界用 UTC 刻钟**(NL 偏移整小时,本地/UTC 重合)。**不新增依赖**。
|
||||
|
||||
> 项目定位:个人自用、家庭特化、不开源——按单用户场景简化,不过度抽象。
|
||||
|
||||
## 6. 任务依赖图
|
||||
|
||||
```
|
||||
Phase A(地基)
|
||||
M6-T01 [schema] 5 张表 + 模型 + 迁移
|
||||
M6-T02 flat 配置(dsmr_* + tibber token/home_id) ← 与 T01 并行
|
||||
M6-T03 pricing profile 框架(manual.yaml/tibber.yaml + pydantic + strategy 注册表) ← 纯模块,可早做
|
||||
|
||||
Phase B(合同 + 数据接入)
|
||||
M6-T04 EnergyContract CRUD + 版本 + 按 profile 校验(API) (Depends T01,T03)
|
||||
M6-T05 Tibber 客户端 + 抓价 service + 调度 + tibber/test (Depends T01,T02,T03)
|
||||
M6-T06 MqttManager 扩订阅 + DSMR ingest(blob/10s) (Depends T01,T02)
|
||||
|
||||
Phase C(计算 + 反哺)
|
||||
M6-T07 计费引擎(每15min 计量 + 汇总)+ 周期 job + 重算 (Depends T01,T03,T04,T05,T06)
|
||||
└─► M6-T08 energy_cost expose provider + state 发布 (Depends T07)
|
||||
|
||||
Phase D(API + 前端)
|
||||
M6-T09 Energy 数据 API(prices/costs/summary/dsmr-latest/recompute) (Depends T05,T07)
|
||||
└─► M6-T10 前端:合同管理 + 价格/费用视图 + Tibber 测试 (Depends T04,T09)
|
||||
|
||||
收尾
|
||||
M6-T11 文档 + OpenAPI + roadmap(Depends 全部)
|
||||
```
|
||||
|
||||
`T01`、`T02`、`T03` 无前置可先开。
|
||||
|
||||
---
|
||||
|
||||
## 7. 原子任务(任务卡)
|
||||
|
||||
> 后端任务沿用校验闸门(`pytest` / `ruff` / 改路由或 schema 则 `export_openapi` 重导出入库);前端闸门见 §8。**不新增依赖**。纯新增、不删/移文件;改 `app/main.py` lifespan 时不得破坏既有 job/连接。
|
||||
|
||||
### M6-T01 — 5 张表与模型 `[schema]`
|
||||
- **Status**: `done` · **Depends**: none
|
||||
- **Context**: 单库 app 链新增 5 张表(§3.5)。只建 schema + 模型,不写采集/计算/接口。
|
||||
- **Files**: `create app/models/energy.py`(`DsmrReading`/`EnergyContract`/`EnergyContractVersion`/`TibberPrice`/`EnergyCostPeriod`);`create alembic_app/versions/20260623_11_energy_tables.py`;`modify alembic_app/env.py`、`scripts/app_db_adopt.py`(`APP_BASELINE_REVISION`→新 head);`create tests/test_energy_models.py`
|
||||
- **Steps**:
|
||||
1. 按 §3.5 定义模型(`Mapped[...]`):`DsmrReading`(`recorded_at` idx、`source_id` unique nullable、`payload` JSON);`EnergyContract`(`name`/`kind`/`active`/`currency`/时间戳);`EnergyContractVersion`(`contract_id` FK RESTRICT、`effective_from`/`effective_to` nullable、`values` JSON、`created_at`);`TibberPrice`(`starts_at` unique、`resolution`、`energy/tax/total` float、`level` nullable、`currency`、`fetched_at`);`EnergyCostPeriod`(`period_start` unique、`d1/d2/r1/r2_kwh` float、`import_cost/export_revenue/net_cost` float、`currency`、`pricing` JSON、`contract_version_id` FK RESTRICT nullable、`degraded` bool、`computed_at`)。
|
||||
2. 新 revision:`down_revision="20260622_10_exposed_entities"`;`upgrade()` 建 5 表 + 索引/唯一/FK;`downgrade()` 反向 drop。更新 `APP_BASELINE_REVISION`。
|
||||
- **Out of scope**: 不写客户端/采集/计费/接口;不动其它模型。
|
||||
- **Acceptance criteria**:
|
||||
- [ ] 全新临时库 upgrade 到 head 含 5 表及约束;`downgrade -1` 干净回滚;链上单 head。
|
||||
- [ ] FK 为 RESTRICT;唯一/索引正确;`payload`/`values`/`pricing` 为 JSON 列。
|
||||
- [ ] `APP_BASELINE_REVISION` == 新 head。
|
||||
- [ ] 校验闸门全绿。
|
||||
- **Reviewer checklist**: 列/约束/FK 与 §3.5 一致;`recorded_at`/`period_start`/`starts_at` 真实索引或唯一列;`env.py` 已 import 新模型;迁移 `down_revision` 指当前真实 head;无破坏性操作。
|
||||
|
||||
### M6-T02 — flat 配置(DSMR + Tibber 凭据)
|
||||
- **Status**: `done` · **Depends**: none
|
||||
- **Context**: DSMR 订阅设置 + Tibber 凭据接入扁平配置(前端自动渲染)。**价格数值不在这里**(在合同里)。
|
||||
- **Files**: `modify app/config.py`、`app/services/config_page.py`(CONFIG_FIELDS + `_settings_payload`);`modify .env.example`、`tests/test_api_config.py`
|
||||
- **Steps**: `Settings` 加 `dsmr_ingest_enabled: bool=False`、`dsmr_mqtt_topic: str="dsmr/json"`、`dsmr_sample_interval_s: int=10`、`tibber_api_token: str=""`、`tibber_home_id: str=""`;CONFIG_FIELDS 归「DSMR」「Tibber」section,`tibber_api_token` 为 secret,bool=checkbox、数值=number;`_settings_payload` 补齐;`.env.example` 加注释样例(默认 off)。
|
||||
- **Out of scope**: 不建客户端/采集;不动既有字段;不放任何价格常数。
|
||||
- **Acceptance criteria**:
|
||||
- [ ] 新项在 `GET /api/config` 分 section;token 为 secret(回空/留空保留);非法值 422 不写库。
|
||||
- [ ] 校验闸门全绿(OpenAPI 变化则重导出)。
|
||||
- **Reviewer checklist**: secret 不回显/不进 OpenAPI 示例;`_settings_payload` 无遗漏;默认 off。
|
||||
|
||||
### M6-T03 — pricing profile 框架 + strategy 注册表
|
||||
- **Status**: `done` · **Depends**: none
|
||||
- **Context**: 仓库内 `manual.yaml`/`tibber.yaml` 定结构(pydantic 校验),加 price strategy 注册表(manual/tibber 出价)。纯模块,单测。
|
||||
- **Files**: `create app/integrations/pricing/__init__.py`、`pricing/profiles.py`(pydantic 模型 + `load_profile`/`list_profiles`/`validate_values`)、`pricing/profiles/manual.yaml`、`pricing/profiles/tibber.yaml`、`pricing/strategies.py`(`register_strategy`/`get_strategy`、`manual`/`tibber` 实现);`create tests/test_pricing_profiles.py`、`tests/test_pricing_strategies.py`
|
||||
- **Steps**:
|
||||
1. `profiles.py`:pydantic 模型描述 §3.1 结构;`load_profile(kind)`(读 YAML,校验失败抛错);`validate_values(kind, values)`(合同数值是否符合结构)。
|
||||
2. `profiles/*.yaml`:按 §3.1 写 manual / tibber 结构。
|
||||
3. `strategies.py`:策略接口 `price_period(deltas, t0, values, session) -> {import_cost, export_revenue, net, pricing_snapshot}`(Decimal):
|
||||
- `manual`:双费率公式(§3.4),`buy_x = energy_buy_x + energy_tax + ode`、`sell_x` 直接用。
|
||||
- `tibber`:取覆盖 t0 的 `tibber_price`,`buy=total`、`sell=total−energy_tax−sell_adjust`,进出口求和。
|
||||
- **Out of scope**: 不连真实 API(tibber 策略只读 `tibber_price` 表,由 T05 填);不写采集/调度/接口。
|
||||
- **Acceptance criteria**:
|
||||
- [ ] 单测:`load_profile("manual"/"tibber")` 校验通过;缺字段/类型错抛可识别错;`validate_values` 正确判合规。
|
||||
- [ ] 单测:manual 策略双费率出价正确(`import = Δd1×buy_dal + Δd2×buy_normal` 等);tibber 策略 `buy=total`、`sell=total−energy_tax−sell_adjust`,负 total 出口得负值。
|
||||
- [ ] Decimal 计算无 float 误差。
|
||||
- [ ] 校验闸门全绿。
|
||||
- **Reviewer checklist**: profile 纯结构、无具体数值;策略用 Decimal;进出口分开(不相抵);tibber 价匹配用 `starts_at ≤ t0` 最近一条;无 OOP 过度抽象(数据+函数,仿 M5 profiles.py)。
|
||||
|
||||
### M6-T04 — EnergyContract CRUD + 版本 + 校验(API)
|
||||
- **Status**: `done` · **Depends**: M6-T01, M6-T03
|
||||
- **Context**: 合同增删改、版本(改价加新版本、生效日期)、激活(一次一个)、按 profile 校验数值。
|
||||
- **Files**: `create app/api/routes/api/energy_contracts.py`、`app/schemas/energy_contract.py`、`app/services/contracts.py`;`modify app/main.py`(注册路由);`create tests/test_api_energy_contracts.py`
|
||||
- **Steps**:
|
||||
1. `contracts.py` service:建合同(含首版本,`validate_values` 校验)、加版本(带 `effective_from`,自动闭合上一版本 `effective_to`)、激活(把其它 active 置 false)、`active_contract_version_at(session, ts)`(取 active 合同在 ts 生效的版本)。
|
||||
2. 路由:`GET/POST/GET{id}/PATCH{id}`、`POST {id}/versions`、`GET /profiles`(返回结构供前端渲染表单);session+CSRF;数值不合规 422。
|
||||
- **Out of scope**: 不算费用(T07);不发 MQTT;不写采集。
|
||||
- **Acceptance criteria**:
|
||||
- [ ] CRUD + 版本 + 激活行为正确:新建按 profile 校验(不合规 422);加版本自动闭合上一段;激活互斥(同时只一个 active);`GET /profiles` 返回结构。
|
||||
- [ ] 未登录 401、缺 CSRF 403;schema 经 OpenAPI 固化入库。
|
||||
- [ ] 校验闸门全绿。
|
||||
- **Reviewer checklist**: 版本只增不改(不覆盖旧版本);激活互斥;删除受 FK RESTRICT(有版本/有费用记录不可裸删);数值校验走 T03 `validate_values`;路径键用合同 id。
|
||||
|
||||
### M6-T05 — Tibber 客户端 + 抓价 service + 调度 + tibber/test
|
||||
- **Status**: `done` · **Depends**: M6-T01, M6-T02, M6-T03
|
||||
- **Context**: httpx 拉 15min 价 upsert `tibber_price`;启动+每日抓(仅 active=tibber);连接测试。
|
||||
- **Files**: `create app/integrations/tibber/__init__.py`、`tibber/client.py`、`app/services/tibber_prices.py`;`modify app/main.py`(抓价 job);`create tests/test_tibber_client.py`、`tests/test_tibber_prices.py`
|
||||
- **Steps**:
|
||||
1. `client.py`:`fetch_price_range(token, home_id|None) -> list[PricePoint]`(§3.3 query,httpx 超时/认证失败抛错,不假设固定节点数);`fetch_current_price(...)`(供 test)。
|
||||
2. `tibber_prices.py`:`refresh_prices(session, settings)`(拉今天+明天 upsert,幂等);仅当 active 合同 kind=tibber 且 token 存在时执行。
|
||||
3. `main.py`:同步 wrapper + `IntervalTrigger`,`max_instances=1, coalesce=True`,吞异常不崩。
|
||||
- **Out of scope**: 不算费用(T07);不发 MQTT;test 路由在 T09(本任务备 service)。
|
||||
- **Acceptance criteria**:
|
||||
- [ ] 单测(httpx mock):解析返回为 PricePoint;认证失败/超时抛可识别异常。
|
||||
- [ ] 单测:`refresh_prices` 幂等(按 `starts_at` upsert,不重复插);非 tibber active 时 no-op。
|
||||
- [ ] 校验闸门全绿。
|
||||
- **Reviewer checklist**: token 不进日志;httpx 超时;upsert 幂等;时间存 UTC;无对 `tibber_price` 的破坏性批删。
|
||||
|
||||
### M6-T06 — MqttManager 扩订阅 + DSMR ingest
|
||||
- **Status**: `done` · **Depends**: M6-T01, M6-T02
|
||||
- **Context**: 给只发不收的 `MqttManager` 扩订阅;DSMR ingest 整帧 blob + 10s 降采样落库。
|
||||
- **Files**: `modify app/integrations/mqtt.py`(订阅注册 + on_message 分发);`create app/services/dsmr_ingest.py`;`modify app/main.py`(启用时注册订阅);`create tests/test_mqtt_subscribe.py`、`tests/test_dsmr_ingest.py`
|
||||
- **Steps**:
|
||||
1. `mqtt.py`:`subscribe(topic, handler)`(`on_connect` 里对已注册 topic subscribe,`on_message` 按 topic 分发,handler 异常吞掉不崩连接);保持既有 publish 不回归。
|
||||
2. `dsmr_ingest.py`:`handle_message(payload_bytes, settings)`:parse JSON → 取 `timestamp`(UTC) → 降采样(`second % dsmr_sample_interval_s != 0` 丢弃)→ **整帧存 payload**(值原样/Decimal 友好)→ 短 session 写 `dsmr_reading`(`source_id`=`id`,存在则跳过)。
|
||||
3. `main.py`:`dsmr_ingest_enabled` 时把 handler 注册到 `dsmr_mqtt_topic`。
|
||||
- **Out of scope**: 不算费用(T07);不改既有 publish/discovery 语义;不做字段 allowlist(整帧存)。
|
||||
- **Acceptance criteria**:
|
||||
- [ ] 单测:`subscribe` 注册后 `on_message` 分发到 handler;既有 publish 无回归。
|
||||
- [ ] 单测:喂样本 JSON(字符串数值、null 相位),秒=00 整帧落盘、秒≠10 整数倍丢弃;同 `id` 不重插。
|
||||
- [ ] `dsmr_ingest_enabled=false` 不订阅/不落盘。
|
||||
- [ ] 校验闸门全绿。
|
||||
- **Reviewer checklist**: on_message 网络线程内只做短事务、吞异常不崩连接;整帧存(含 gas/各相);null 容错;`source_id` 幂等;10s 降采样正确。
|
||||
|
||||
### M6-T07 — 计费引擎 + 周期 job + 汇总 + 重算
|
||||
- **Status**: `done` · **Depends**: M6-T01, M6-T03, M6-T04, M6-T05, M6-T06
|
||||
- **Context**: 每 15min 用寄存器差 × active 合同 strategy 算计量电费(快照、不可变);汇总加固定费 − 抵扣;周期 job + 重算。
|
||||
- **Files**: `create app/services/energy_cost.py`;`modify app/main.py`(周期 job);`create tests/test_energy_cost.py`
|
||||
- **Steps**:
|
||||
1. `energy_cost.py`:`register_at(session, boundary)`(取 ≤boundary 最后一行的 d1/d2/r1/r2,Decimal);`compute_period(session, t0)`(取 active 版本 + strategy(T03/T04)→ 算 → upsert + 快照 `pricing` + `contract_version_id`,缺价/缺数据标 `degraded` 或跳过);`compute_closed_periods(session)`;`recompute_range(session, start, end)`(幂等覆盖);`summarize(session, start, end)`(Σnet + 固定费按天 − heffingskorting 按天,取 active 版本值)。
|
||||
2. `main.py`:1 分钟 tick job(`max_instances=1, coalesce=True`),有 active 合同 + DSMR 数据时算;吞异常不崩。
|
||||
- **Out of scope**: 不发 MQTT(T08);不加 HTTP(T09,备 service);不改采集/价格抓取。
|
||||
- **Acceptance criteria**:
|
||||
- [ ] 单测:构造 `dsmr_reading`+合同版本(manual 双费率 / tibber + `tibber_price`),`compute_period` 算出正确 cost/revenue/net,快照价 + `contract_version_id` 落库;upsert 幂等。
|
||||
- [ ] 单测:跨价格版本时按 t0 选对版本;缺价跳过、缺读数标 `degraded`。
|
||||
- [ ] 单测:`summarize` = Σnet + 固定费(月→天×天数)− heffingskorting(年→天×天数)。
|
||||
- [ ] 校验闸门全绿。
|
||||
- **Reviewer checklist**: Decimal 算钱;寄存器差为"末−初";进出口分开;周期边界 UTC 刻钟;快照不可变(重算才覆盖,且显式);版本选择按 `effective_from≤t0<effective_to`;job 不崩。
|
||||
|
||||
### M6-T08 — energy_cost expose provider + state 发布
|
||||
- **Status**: `done` · **Depends**: M6-T07
|
||||
- **Context**: 复用 M5 expose/Discovery,发当前买/卖价 + 累计成本/收入。
|
||||
- **Files**: `modify app/integrations/expose.py`(`_energy_cost_provider` + 注册);`modify app/services/energy_cost.py` 或 `app/main.py`(算完顺带 `publish_states`);`create tests/test_energy_expose.py`
|
||||
- **Steps**: provider 产出 §3.7 实体(`buy_price_now`/`sell_price_now`/`import_cost_total`/`export_revenue_total`),累计项 `total_increasing`+`monetary`+currency;`value_getter` 从 `energy_cost_period` 取;计费 job 后 `publish_states`。
|
||||
- **Out of scope**: 不改 Discovery 机制本身;不改采集/计费。
|
||||
- **Acceptance criteria**:
|
||||
- [ ] 单测:`build_catalog` 含 energy_cost 实体;累计项 `total_increasing`;勾选默认未勾。
|
||||
- [ ] 单测:`value_getter` 取到正确当前价/累计值;MQTT 未启用整链 no-op。
|
||||
- [ ] 校验闸门全绿。
|
||||
- **Reviewer checklist**: `key` 稳定(固定字符串);复用既有 provider 协议、未改其它 provider;累计 `total_increasing` 语义正确;不阻塞计费 job。
|
||||
|
||||
### M6-T09 — Energy 数据 API
|
||||
- **Status**: `done` · **Depends**: M6-T05, M6-T07
|
||||
- **Context**: 价格/费用/汇总/最新 DSMR/重算/Tibber 测试端点(合同 CRUD 在 T04)。
|
||||
- **Files**: `create app/api/routes/api/energy.py`、`app/schemas/energy.py`;`modify app/main.py`(注册);`create tests/test_api_energy.py`
|
||||
- **Steps**: `GET /prices`、`GET /costs`、`GET /costs/summary`、`GET /dsmr/latest`、`POST /costs/recompute`(调 T07)、`POST /tibber/test`(调 T05 client,三态);session+CSRF;查询走索引 + 窗口/上限。
|
||||
- **Out of scope**: 不写采集/计费/发布逻辑(复用 service)。
|
||||
- **Acceptance criteria**:
|
||||
- [ ] 各端点行为正确:窗口+limit 上限生效;recompute 幂等;tibber/test 三态;401/403 正确;schema 入库(`git diff --exit-code openapi/` 无差异)。
|
||||
- [ ] 校验闸门全绿。
|
||||
- **Reviewer checklist**: 查询有时间窗+上限;recompute 复用 T07、无破坏删;test 不泄 token。
|
||||
|
||||
### M6-T10 — 前端:合同管理 + 价格/费用视图 + Tibber 测试
|
||||
- **Status**: `done` · **Depends**: M6-T04, M6-T09
|
||||
- **Files**: `modify frontend/src/pages/EnergyPage.tsx`;`create frontend/src/energy/ContractManager.tsx`、`ContractForm.tsx`、`TibberPrices.tsx`、`CostView.tsx`、扩 `hooks.ts`;`modify` Config 页 Tibber 测试入口;`create` 对应 `*.test.tsx`
|
||||
- **Steps**: 合同列表/新建/编辑/激活/加版本(**表单字段按 `/profiles` 结构渲染**,manual 双费率/税/固定费/抵扣)+ 版本历史只读;价格曲线 + 费用走势/明细 + 汇总卡片;Tibber 测试三态;全走类型化 client;空/加载/错误态 + 缺 key 容错。
|
||||
- **Out of scope**: 不做服务端降采样;不改后端。
|
||||
- **Acceptance criteria**:
|
||||
- [ ] 合同 CRUD/激活/版本可用,表单按 profile 渲染;价格/费用/汇总渲染正确;区间只取窗口;币种/单位来自 API。
|
||||
- [ ] Tibber 测试三态可用。
|
||||
- [ ] 前端闸门全绿(`lint`/`typecheck`/`test`/`build`)。
|
||||
- **Reviewer checklist**: 全走类型化 client;图表组件隔离复用;表单按结构渲染、不 hardcode 字段;窗口/上限;空/错/加载/缺 key 容错。
|
||||
|
||||
### M6-T11 — 文档 + OpenAPI + roadmap 收尾
|
||||
- **Status**: `done` · **Depends**: 全部
|
||||
- **Files**: `modify README.md`、`docs/roadmap.md`(M6 行 + 毕业)、`docs/architecture-overview.md`、`docs/design/README.md`(列入 m6);`run python scripts/export_openapi.py` 并提交 `openapi/`
|
||||
- **Acceptance criteria**:
|
||||
- [ ] 文档反映新链路;`git diff --exit-code openapi/` 无未提交差异;校验闸门全绿。
|
||||
- **Reviewer checklist**: 无残留过时描述;OpenAPI 入库;§13 已敲定项同步回文档。
|
||||
|
||||
---
|
||||
|
||||
## 8. 前端校验闸门
|
||||
|
||||
在 `frontend/` 下:`npm ci && npm run lint && npm run typecheck && npm run test && npm run build`(留意 chunk 体积告警)。后端同任务改路由/schema 仍需根目录 `export_openapi.py` 并提交 `openapi/`。**不新增前端依赖**(Recharts 已在)。
|
||||
|
||||
## 9. 构建上下文完整性(M1 教训)
|
||||
|
||||
- **纯新增、不删/移文件**,**不新增依赖**(httpx/paho/pyyaml 已在)。新增源文件都在 `app/`、`scripts/`、`frontend/` 既有 `COPY` 范围内,无需改 `Dockerfile`;**`app/integrations/pricing/profiles/*.yaml` 是非 .py 资源**——确认随 `COPY app` 进镜像、运行期按 `__file__`/`importlib.resources` 定位(别用 CWD 相对路径)。`tests/test_deployment.py::test_dockerfile_copy_sources_exist` 仍应通过。
|
||||
- 改 `app/main.py` lifespan 注册新 job/订阅时**不得破坏**既有 public-ip / modbus-poll / discovery job 与 MQTT 连接。
|
||||
- 发版前置走查(CLAUDE.md):真起 app → 订到 `dsmr/json`、(合同生效后)拉一次 Tibber 价、看费用落库、前端合同/价格/费用视图人工瞄一眼,再打 tag。
|
||||
|
||||
## 10. 后续杠杆(本里程碑不做,文档留痕)
|
||||
|
||||
- **回送阶梯罚金(terugleverkosten)**:按自然年累计、阶梯式;本期不做。将来做就把阶梯表存进合同结构,在**年度汇总**层按 YTD 累计回送量套阶梯(不进每周期引擎)。
|
||||
- **燃气 / 区域供暖计费**:数据已在 blob 里(gas `extra_device_*`);将来在合同结构加"商品价"+ `energy_cost_period` 加 `commodity` 维度即可,不改主链。
|
||||
- **能源税分档**:年用电跨 10000 kWh 档时税率变;现按单档配置。
|
||||
- **降采样 / 保留**:`dsmr_reading` 10s 长期行数大,加定期降采样/保留窗口任务(SQL 端 `AVG(json_extract(...))`)。
|
||||
- **价格 level 配色 / 便宜时段提示**:用 Tibber `level` 驱动前端配色与用电建议。
|
||||
- **HA 让 HA 算 vs 后端算**:当前后端算累计;也可只发买/卖价 €/kWh 让 HA Energy 自乘(实体都已具备)。
|
||||
|
||||
## 11. 人工验收 walkthrough(实现完成后)
|
||||
|
||||
> 受控手工验证(依赖家里 DSMR Reader + broker;Tibber 部分待合同生效),不进自动化。
|
||||
|
||||
1. **合同(先用 manual,立即可用)**:Energy 页新建一个 `manual` 合同,按表单填双费率/能源税/固定费/抵扣,激活。
|
||||
2. **DSMR 落库**:开 `dsmr_ingest_enabled`、确认 topic;MQTT Explorer 看 `dsmr/json` 有消息;等若干 10 秒 → `GET /api/energy/dsmr/latest` 有最新整帧;`dsmr_reading` 约每 10 秒一行(整帧 blob)。
|
||||
3. **费用计算**:跨过一个整刻钟边界 → `GET /api/energy/costs` 出现该周期 import/export/cost/revenue/net(manual 双费率分档对得上手算);`GET /costs/summary` 含固定费/抵扣;改价加新版本后旧周期不变、`POST /recompute` 才覆盖。
|
||||
4. **Tibber(合同生效后)**:建 `tibber` 合同、Config 填 token、点「Tibber 测试」三态成功;激活后抓价 → `GET /prices` 有 15min 价、费用切到动态。
|
||||
5. **反哺 HA**:Expose 勾选 energy_cost 实体、开 Discovery、重发 → HA 出现当前买/卖价 + 累计成本/收入;HA Energy 挂累计实体。
|
||||
|
||||
## 12. 里程碑完成定义(DoD)
|
||||
|
||||
- 后端能:订 `dsmr/json` 按 10s 整帧落库;按 active 合同(manual 双费率 / tibber 动态)每 15min 用寄存器差 × 价算计量电费(不可变、快照);日/月/年汇总加固定费 − 抵扣;不做净计量;改价走版本、历史不回改。
|
||||
- 复用 M5 Discovery 把当前买/卖价 + 累计成本/收入(`total_increasing`)发成 HA 实体。
|
||||
- 前端 Energy 视图能管理合同(表单按 profile 渲染、激活、版本)、看价格曲线与费用走势/明细/汇总、测 Tibber。
|
||||
- 后端 `pytest`/`ruff`/`export_openapi` + 前端 `lint/typecheck/test/build` 全绿且 `openapi/` 入库。
|
||||
- README / architecture / roadmap / design 索引反映 M6。
|
||||
|
||||
## 13. 待确认 / TODO(拿到真实 token + 账单后钉死,均已落成配置/默认值,不阻塞实现)
|
||||
|
||||
1. **买价**:✅ tibber = API `total`(demo 已证 total=energy+tax);manual = energy_buy_档 + energy_tax。无待办。
|
||||
2. **卖价残差(tibber)**:`sell = total − energy_tax − sell_adjust`,`sell_adjust` 默认 0(买卖费相等抵消)。真实账单确认后若有残差再调。
|
||||
3. **双费率寄存器映射**:`_1`=dal/低、`_2`=normal/高(NL 惯例)——接价前用真实数据确认别接反(差价小但要对)。
|
||||
4. **能源税年值**:manual/tibber 的 `energy_tax` 默认 ~0.1108(2026 第一档含 VAT),按当年实际值核。
|
||||
5. **Tibber 15min + 币种**:✅ 查询/分辨率已 demo 证实;仍需合同生效后用**真实 token** 确认 NL 返回真 15 分钟价 + 币种 EUR。
|
||||
6. **manual 现行数值**(你的固定合同):normal 0.133 / dal 0.127 / 回送两档(现同价)/ 管理费 ≈€0.329/天(≈€9.87/月)/ 电网费 + heffingskorting 待填。
|
||||
</content>
|
||||
@@ -0,0 +1,322 @@
|
||||
# M7 — 电表生命周期 / 换表归档(Meter epochs)
|
||||
|
||||
> 协作格式、任务卡结构、校验闸门、数据安全红线见 [`README.md`](./README.md),本文不再重复。
|
||||
|
||||
## 1. 目标
|
||||
|
||||
让计费系统正确处理**电表更换**这一**必然事件**:
|
||||
|
||||
- 荷兰 2G 智能电表退网后,电网公司**一定**会把表换成 4G 表(同址换表);搬家继承新表也是同类。
|
||||
- 计费是**寄存器差值(delta)模型**:每个 15 分钟周期成本 =(周期末读数 − 周期初读数)× 单价。换表后新表寄存器基数与旧表**无关**(可能更高也可能更低),若跨表算 delta 会产出**负成本或巨额假成本**。
|
||||
- 目标:引入显式的 **Meter(电表)** 概念,标记"某时刻起属于哪一块物理表",使引擎**永不跨表算 delta**,并让累计量按表归零、历史可追溯可查。
|
||||
|
||||
非目标(本里程碑不做,§9 留痕):gas / 区域供暖的**计费**;"家庭(home)"分组实体;多合同时间线积分;自动识别换表(DSMR 帧无电表序列号,无法自动识别,靠用户显式声明)。
|
||||
|
||||
## 2. 现状(实现者可据此工作,不必通读全仓库)
|
||||
|
||||
- 计费数据模型(M6):
|
||||
- `dsmr_reading`:整帧 JSON,`recorded_at`(UNIQUE) 去重;含 4 个累计寄存器 `electricity_delivered_1/2`、`electricity_returned_1/2`。**帧内无电表序列号字段**。
|
||||
- `energy_cost_period`:每 15 分钟一行,存 `d1/d2/r1/r2_kwh`(delta)、`import_cost/export_revenue/net_cost`、`pricing` 快照、`contract_version_id`、`degraded`、`computed_at`、`period_start`(unique)。
|
||||
- 计费引擎 `app/services/energy_cost.py`:
|
||||
- `register_at(session, boundary)`:取 `recorded_at ≤ boundary` 的最近一条;有 15 分钟 staleness 护栏(超过则返回 None → 周期降级)。
|
||||
- `compute_period(session, t0)`:取 t0/t1 两端寄存器,算 delta,按 `contract` 的 strategy 计价;缺读数→降级行(成本 0)。
|
||||
- `compute_closed_periods` / `recompute_range`:遍历刻钟边界批量算。
|
||||
- `summarize(session, start, end)`:聚合非降级周期电费 + 固定费/抵扣(FU10 已按本地日跨版本积分)。
|
||||
- 计价策略 `app/integrations/pricing/strategies.py`:`import_cost = Δd1×buy_dal + Δd2×buy_normal` 等,**对 delta 无任何 clamp/sanity 检查**(负 delta → 负成本,超大 delta → 巨额成本)。
|
||||
- 暴露 `app/integrations/expose.py`:累计 `import_cost_total`/`export_revenue_total` 锚点(FU11)= `max(合同最早 effective_from, 最早非降级 period_start)`;`*_today` 日归零实体(FU11)。
|
||||
- 合同 `app/services/contracts.py`:`active` 互斥 + 版本时间线 + `active_contract_version_at(ts)`(半开 `[from,to)`)。合同**不引用电表**。
|
||||
- 配置/迁移:单库 app,Alembic 链 `alembic_app/`;`PRAGMA foreign_keys=ON` 已开(`app/db.py`)。
|
||||
- 时区:FU10 的 `app/services/timezone.py`(`local_*`,可 monkeypatch)。
|
||||
|
||||
## 3. 目标架构
|
||||
|
||||
### 3.1 核心概念:Meter(= epoch)
|
||||
|
||||
一条 `meter` 记录 = **一块物理电表的一段安装期** `[started_at, ended_at)`。换表 = 关闭当前 active 表(`ended_at=T`)+ 新建一条 `started_at=T` 的表。计费**永不跨 meter 算 delta**:跨表的那个周期按设计降级。
|
||||
|
||||
- **每种 `commodity` 至多一个 active 表**(`ended_at IS NULL`)。本里程碑只 `electricity` 参与计费,`commodity` 字段为未来 gas/heating 预留。
|
||||
- `meter_at(session, ts, commodity="electricity")`:返回 `started_at ≤ ts AND (ended_at IS NULL OR ts < ended_at)` 的那块表(半开区间,仿 `active_contract_version_at`)。
|
||||
- 计费基准:某表窗口内**最早的一条 `dsmr_reading`**(`recorded_at ≥ started_at`)即基准;用户**只需填 `started_at` 日期,无需知道当前读数**。追溯(`started_at` 在过去)时,有效起点自然 = `max(started_at, 数据起点)`。
|
||||
- **隔离的前提 = 显式声明**:系统无法自动识别换表(DSMR 帧无序列号),搬家后新旧读数同落 `dsmr_reading`,靠用户**显式新建那块表**才隔离。万一忘了声明:搬家必有的数据空档(staleness)+ delta 护栏(§3.3)保证**不会产出垃圾成本**,只是新数据暂混在旧表 epoch、累计未归零;**事后补声明 + 追溯 `recompute_range`** 即可纠正。
|
||||
|
||||
### 3.2 数据模型(新增 1 张表 + 1 列)
|
||||
|
||||
`meter` 表(`app/models/energy.py`,单库 app 链):
|
||||
|
||||
| 列 | 类型 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| `id` | int PK | 内置自增 ID |
|
||||
| `label` | str | 人读标签,编址/标识(如 "旧 2G 表 @ Dorpsstraat 1") |
|
||||
| `commodity` | str | 默认 `"electricity"`;预留 `gas`/`heating` |
|
||||
| `started_at` | datetime(UTC) | 安装/继承/搬家起点(可在过去) |
|
||||
| `ended_at` | datetime(UTC) \| null | null = 当前 active |
|
||||
| `reason` | str | `initial` / `meter_swap` / `home_move` / `other` |
|
||||
| `note` | str \| null | 备注 |
|
||||
| `created_at` | datetime(UTC) | |
|
||||
|
||||
`energy_cost_period` 新增 `meter_id`(nullable FK → `meter.id`):记录该周期归属哪块表,便于审计/按表查询/归档。
|
||||
|
||||
### 3.3 计费引擎改造(`energy_cost.py` / `strategies.py` 调用处)
|
||||
|
||||
1. `register_at(session, boundary, meter)`:**仅在该 meter 窗口内**取读数(`started_at ≤ recorded_at < ended_at` 且 `≤ boundary` 且过 staleness 护栏)。绝不把旧表读数拉进新表周期。
|
||||
2. `compute_period(session, t0)`:
|
||||
- 查 `m0 = meter_at(t0)`、`m1 = meter_at(t1)`。
|
||||
- `m0 is None`(无表覆盖)→ **降级**(不计费)。
|
||||
- `m0.id != m1.id`(周期跨表边界)→ **降级**(D5:丢这一个周期可接受)。
|
||||
- 否则在 `m0` 窗口内 `register_at` 两端、算 delta、计价;新增写 `period.meter_id = m0.id`。
|
||||
3. **delta sanity 护栏(D6)**:算出 delta 后,若 `d1/d2/r1/r2` 任一 `< 0`,或任一 `> _MAX_DELTA_KWH`(常量,给一个远超住宅的宽松上限,如每 15 分钟 100 kWh)→ **降级**(兜底表重置/DSMR 回绕/数据毛刺,与换表无关也防住)。
|
||||
4. `compute_closed_periods` / `recompute_range`:逐周期按 `meter_at` 判定,无需额外结构。
|
||||
|
||||
### 3.4 累计语义(per-meter 归零,D2)
|
||||
|
||||
- `expose.py` 累计 `import_cost_total`/`export_revenue_total` 锚点改为**当前 active electricity 表的起点**:`anchor = active electricity meter.started_at`(窗口内首条读数自然 ≥ 它)。换表后累计**从零起一段新序列**,不维护偏移、不跨表累加。
|
||||
- `*_today` 日归零实体不变(其本地今天窗口必落在当前表内)。
|
||||
- (可选,非核心)暴露"当前电表 label"作为一个文本 sensor,便于 HA 侧识别当前表。
|
||||
- **已知行为(可接受)**:累计 `_total` 是 `state_class: total`,换表归零时其值会下台阶,HA 长期统计可能在换表那一刻记一次性负 blip。HA 侧自己存旧值做 down-sample——已与用户确认**先这样、观察后按需处理**(`last_reset` 信号见 §9 杠杆,本里程碑不做)。
|
||||
|
||||
### 3.5 API(`app/api/routes/api/` + schemas)
|
||||
|
||||
- `GET /api/energy/meters` — 列出全部表(时间线,active 在前/或按 started_at)。
|
||||
- `POST /api/energy/meters` — 声明换表/继承:`{label, started_at, reason, note?}`;服务层关闭当前 active 同 commodity 表(`ended_at=started_at`)+ 建新表。`started_at` 必须 `≥` 当前 active 表的 `started_at`(拒绝倒挂,仿合同版本"strictly after")。
|
||||
- `PATCH /api/energy/meters/{id}` — 改 `label`/`note`,或修正 `started_at`(追溯)。
|
||||
- `POST /api/energy/meters/{id}/recompute`(或复用现有 recompute)— 追溯变更后对受影响窗口 `recompute_range`,重判跨表周期与归属。
|
||||
- 改了路由/schema → 重导出 OpenAPI 并提交。
|
||||
|
||||
### 3.6 前端(并入 Energy 视图)
|
||||
|
||||
- 一个"电表(Meters)"管理区:表的时间线(label / 区间 / active 标记 / reason),"我换了表 / 继承了新表"表单(只填 label + 日期 + reason),编辑 label/日期。
|
||||
- 追溯保存后给出"已重算受影响周期"的反馈(或提供 recompute 按钮)。
|
||||
- 前端闸门见 §8 引用。
|
||||
|
||||
### 3.7 迁移与回填(数据安全 runbook)
|
||||
|
||||
- Alembic(`alembic_app/`):建 `meter` 表 + 给 `energy_cost_period` 加 `meter_id` 列(nullable)。
|
||||
- **回填(只增不删、幂等、对账)**:
|
||||
1. 若库内已有任何 `dsmr_reading`/`energy_cost_period`,创建一条**初始表** `meter(label="Initial meter", commodity="electricity", started_at = 最早 dsmr_reading.recorded_at(无则最早 period_start,再无则迁移时刻), ended_at=NULL, reason="initial")`。
|
||||
2. 把现有 `energy_cost_period.meter_id` 全部回填为这条初始表的 id。
|
||||
3. **对账**:回填后 `meter_id IS NULL` 的非降级周期数必须为 0;对不上立即中止、非零退出。
|
||||
- **红线**:迁移**不删除/不覆盖**任何 `dsmr_reading`/`energy_cost_period`/旧 `.db`/备份;纯新增表+列+回填。先在备份副本演练再对真实库执行。
|
||||
|
||||
## 4. 已锁定决策(讨论后拍板)
|
||||
|
||||
- **D1 粒度**:Meter 为一等公民,**不绑定 home**;`label` 编址 + `commodity` 区分品类(每 commodity 一个 active)。
|
||||
- **D2 累计**:换表后累计量**归零**(锚当前表起点),直接吃 DSMR 累计 delta,不维护偏移。
|
||||
- **D3 合同耦合**:**不引入 home 实体**。合同与电表是两条独立时间线,合同不引用电表 → 同址换表时合同自动照常套用,无需上层抽象。搬家若换合同:用"给现有合同加 version"保持历史连续(多合同时间线积分留作后续杠杆)。
|
||||
- **D4 追溯**:`started_at` 可在过去;有效计费起点 = `max(started_at, 数据起点)`;追溯变更触发 `recompute_range`。
|
||||
- **D5 边界周期**:跨表周期判 **degraded**(可接受,换表常伴随长时间无数据)。
|
||||
- **D6 delta 护栏**:加(负/异常大 delta → degraded),通用兜底。
|
||||
- **D7 扩展性**:gas/heating **计费**不在 scope;`commodity` 字段 + 每品类 active 表使未来加表低成本。
|
||||
|
||||
## 5. 任务依赖图
|
||||
|
||||
```
|
||||
M7-T01 (meter 表+模型+列+迁移回填)
|
||||
├──► M7-T02 (meter 服务层: swap/close/edit, meter_at, 互斥/校验)
|
||||
│ ├──► M7-T03 (计费引擎 meter-aware + delta 护栏 + period.meter_id)
|
||||
│ │ └──► M7-T04 (累计 per-meter 归零, expose 锚点)
|
||||
│ └──► M7-T05 (meter API + 追溯 recompute + OpenAPI)
|
||||
│ └──► M7-T06 (前端电表管理 UI)
|
||||
└──────────────────────────────► M7-T07 (文档/roadmap/收尾)
|
||||
```
|
||||
|
||||
## 6. 原子任务(任务卡)
|
||||
|
||||
### M7-T01 — `meter` 表 + 模型 + `energy_cost_period.meter_id` + 迁移回填 `[schema]`
|
||||
|
||||
- **Status**: `done`
|
||||
- **Depends**: none
|
||||
- **Context**: 引入 Meter 实体的数据地基;为现有数据回填初始表。
|
||||
|
||||
**Files**
|
||||
- `modify app/models/energy.py`(新增 `Meter`;`EnergyCostPeriod` 加 `meter_id`)
|
||||
- `create alembic_app/versions/<rev>_meter_table.py`
|
||||
- `modify tests/test_energy_models.py`
|
||||
|
||||
**Steps**
|
||||
1. 加 `Meter` 模型(字段见 §3.2);`EnergyCostPeriod` 加 nullable `meter_id` FK。
|
||||
2. 写迁移:建表 + 加列;按 §3.7 回填初始表并回填 `meter_id`;带对账(不一致则 raise)。
|
||||
3. 测试:模型字段、active-per-commodity 约束的服务层留给 T02;这里测建表/列/回填幂等。
|
||||
|
||||
**Out of scope / 不要碰**:计费引擎、expose、API(后续任务)。
|
||||
|
||||
**Acceptance criteria**
|
||||
- [ ] `meter` 表与 `energy_cost_period.meter_id` 建出;迁移可升降级。
|
||||
- [ ] 有历史数据时回填出恰好一条 `reason="initial"` 的 active 表,且非降级周期 `meter_id` 全部非空。
|
||||
- [ ] 迁移**不删除/覆盖**任何既有数据;回填幂等。
|
||||
- [ ] 校验闸门全绿。
|
||||
|
||||
**Reviewer checklist**:回填对账逻辑;空库/有数据两种路径;数据安全红线(无删/覆盖)。
|
||||
|
||||
### M7-T02 — Meter 服务层(swap/close/edit + `meter_at` + 互斥/校验)
|
||||
|
||||
- **Status**: `done`
|
||||
- **Depends**: M7-T01
|
||||
- **Context**: 换表/继承的业务逻辑与按时刻查表。
|
||||
|
||||
**Files**
|
||||
- `create app/services/meters.py`
|
||||
- `create tests/test_meters.py`
|
||||
|
||||
**Steps**
|
||||
1. `meter_at(session, ts, commodity="electricity")`(半开区间)。
|
||||
2. `declare_meter(session, *, label, started_at, reason, commodity="electricity", note=None)`:校验 `started_at ≥ 当前 active 同 commodity 表的 started_at`(拒绝倒挂);关闭旧 active(`ended_at=started_at`)+ 建新 active。
|
||||
3. `list_meters` / `update_meter`(label/note/started_at);改 started_at 时维持区间自洽。
|
||||
4. 测试:互斥(每 commodity 单 active)、倒挂拒绝、追溯、`meter_at` 边界。
|
||||
|
||||
**Out of scope**:API、引擎、前端。
|
||||
|
||||
**Acceptance criteria**
|
||||
- [ ] 每 commodity 至多一个 active;swap 正确续接区间。
|
||||
- [ ] `meter_at` 半开区间正确(含边界)。
|
||||
- [ ] 校验闸门全绿。
|
||||
|
||||
**Reviewer checklist**:互斥与续接的事务正确性;追溯改 started_at 的区间一致性。
|
||||
|
||||
### M7-T03 — 计费引擎 meter-aware + delta 护栏 + `period.meter_id`
|
||||
|
||||
- **Status**: `done`
|
||||
- **Depends**: M7-T02
|
||||
- **Context**: 让计费永不跨表算 delta,并兜底异常 delta。
|
||||
|
||||
**Files**
|
||||
- `modify app/services/energy_cost.py`
|
||||
- `modify tests/test_energy_cost.py`
|
||||
|
||||
**Steps**
|
||||
1. `register_at` 限定在传入 meter 的窗口内取读数。
|
||||
2. `compute_period`:按 §3.3 判定 `m0/m1`;无表/跨表 → 降级;否则算 delta、写 `meter_id`。
|
||||
3. 加 `_MAX_DELTA_KWH` 常量与护栏:任一寄存器 delta `<0` 或 `>上限` → 降级。
|
||||
4. `compute_closed_periods`/`recompute_range` 逐周期 meter 判定。
|
||||
5. 测试:同表内正确;跨表周期降级;负/超大 delta 降级;无 active 表降级。
|
||||
|
||||
**Out of scope**:expose 锚点(T04)、API(T05)。
|
||||
|
||||
**Acceptance criteria**
|
||||
- [ ] 跨 meter 边界周期 = degraded;无表覆盖 = degraded。
|
||||
- [ ] 负 delta / 超 `_MAX_DELTA_KWH` = degraded,绝不产负/巨额成本。
|
||||
- [ ] 同表内 delta 计费与归属 `meter_id` 正确。
|
||||
- [ ] 校验闸门全绿。
|
||||
|
||||
**Reviewer checklist**:staleness × meter 窗口的交互;护栏阈值是否远超住宅、不会误伤真实用量;recompute 重判归属。
|
||||
|
||||
### M7-T04 — 累计 per-meter 归零(expose 锚点)
|
||||
|
||||
- **Status**: `done`
|
||||
- **Depends**: M7-T03
|
||||
- **Context**: 换表后 MQTT 累计量从零起算。
|
||||
|
||||
**Files**
|
||||
- `modify app/integrations/expose.py`
|
||||
- `modify tests/test_energy_expose.py`
|
||||
|
||||
**Steps**
|
||||
1. 累计 `import_cost_total`/`export_revenue_total` 锚点改为**当前 active electricity 表的 `started_at`**(替换 FU11 的 `max(合同, 首周期)`)。
|
||||
2. `*_today` 不变;确认无非降级周期/无 active 表时仍 `None` 保护。
|
||||
3.(可选)暴露当前电表 label 文本实体。
|
||||
4. 测试:换表后累计仅含当前表;anchor 取当前表起点。
|
||||
|
||||
**Out of scope**:API/前端。
|
||||
|
||||
**Acceptance criteria**
|
||||
- [ ] 累计量锚 = 当前 active electricity meter 起点;换表归零。
|
||||
- [ ] None 保护分支保留;daily 实体不受影响。
|
||||
- [ ] 校验闸门全绿。
|
||||
|
||||
**Reviewer checklist**:与 FU11 行为差异是否符合 D2;无表/空数据分支。
|
||||
|
||||
### M7-T05 — Meter API + 追溯 recompute + OpenAPI
|
||||
|
||||
- **Status**: `done`
|
||||
- **Depends**: M7-T02
|
||||
- **Context**: 暴露电表 CRUD 与换表声明;追溯后重算。
|
||||
|
||||
**Files**
|
||||
- `create app/api/routes/api/meters.py`
|
||||
- `create app/schemas/meter.py`
|
||||
- `modify app/api/routes/...`(注册路由)
|
||||
- `modify openapi/openapi.json`, `openapi/openapi.yaml`
|
||||
- `create tests/test_api_meters.py`
|
||||
|
||||
**Steps**
|
||||
1. `GET/POST/PATCH /api/energy/meters`(+ 追溯触发 `recompute_range`)。
|
||||
2. 鉴权沿用 `require_session`。
|
||||
3. `python scripts/export_openapi.py` 并提交 `openapi/`。
|
||||
4. 测试:列出/声明换表/编辑/倒挂拒绝/追溯重算。
|
||||
|
||||
**Out of scope**:前端。
|
||||
|
||||
**Acceptance criteria**
|
||||
- [ ] 端点行为符合 §3.5;追溯变更触发受影响窗口重算。
|
||||
- [ ] `git diff --exit-code openapi/` 干净(已重导出提交)。
|
||||
- [ ] 校验闸门全绿。
|
||||
|
||||
**Reviewer checklist**:错误码(倒挂/不存在);recompute 范围是否覆盖受影响周期。
|
||||
|
||||
### M7-T06 — 前端电表管理 UI
|
||||
|
||||
- **Status**: `done`
|
||||
- **Depends**: M7-T05
|
||||
- **Context**: 让用户在 Energy 页声明换表、查看电表时间线。
|
||||
|
||||
**Files**
|
||||
- `create frontend/src/energy/MeterManager.tsx`
|
||||
- `modify frontend/src/energy/EnergyPage.tsx`, `frontend/src/energy/hooks.ts`
|
||||
- `modify frontend/src/energy/*.test.tsx`
|
||||
- `modify frontend/src/api/schema.d.ts`(`npm run codegen`)
|
||||
|
||||
**Steps**
|
||||
1. `npm run codegen` 拉新端点类型。
|
||||
2. 电表时间线 + "换表/继承"表单(label + 日期 + reason)+ 编辑。
|
||||
3. 追溯保存后反馈"已重算"。
|
||||
4. 前端闸门(§8)。
|
||||
|
||||
**Out of scope**:后端。
|
||||
|
||||
**Acceptance criteria**
|
||||
- [ ] 能列出/声明换表/编辑;UI 合理。
|
||||
- [ ] 前端 lint/typecheck/test/build 全绿;`schema.d.ts` 已 codegen。
|
||||
|
||||
**Reviewer checklist**:日期→后端时区语义(沿用 FU10 本地午夜 naive 约定);空状态/加载/错误。
|
||||
|
||||
### M7-T07 — 文档 / OpenAPI / roadmap 收尾
|
||||
|
||||
- **Status**: `done`
|
||||
- **Depends**: M7-T01..T06
|
||||
- **Context**: 把 M7 落档、更新 roadmap 与索引。
|
||||
|
||||
**Files**
|
||||
- `modify docs/roadmap.md`(新增 M7 条目,标完成)
|
||||
- `modify docs/design/README.md`(索引加 m7)
|
||||
- `modify docs/design/m7-meter-epochs-archival.md`(Status 收尾)
|
||||
- `modify docs/*`(如新增 meter 模块说明,按需)
|
||||
|
||||
**Acceptance criteria**
|
||||
- [ ] roadmap/README/本文 Status 一致;OpenAPI 已是最新并提交。
|
||||
- [ ] 校验闸门全绿。
|
||||
|
||||
## 7. 构建上下文完整性(M1 教训)
|
||||
|
||||
- 本里程碑只**新增** `app/services/meters.py`、`app/api/routes/api/meters.py`、`app/schemas/meter.py`、`alembic_app/versions/<rev>_*.py` 与前端文件;均落在现有 `Dockerfile` 的 `COPY app ./app` / `COPY alembic_app ./alembic_app` / 前端构建阶段覆盖范围内,**无新增顶层目录**。
|
||||
- `tests/test_deployment.py::test_dockerfile_copy_sources_exist` 应仍绿;删/移文件时按 README 规则 grep 构建清单(本里程碑预期无删除)。
|
||||
|
||||
## 8. 前端校验闸门
|
||||
|
||||
`frontend/` 下:`npm run lint && npm run typecheck && npm run test && npm run build`;改了端点/schema 还需 `npm run codegen` 并提交 `schema.d.ts`。
|
||||
|
||||
## 9. 后续杠杆(本里程碑不做,留痕)
|
||||
|
||||
- **Home/Site 分组**:若将来要"一个地址下多品类表合并出账 + 整屋归档",再加薄分组层(可仅 `site_label` 字段,不必新实体)。
|
||||
- **多合同时间线积分**:让 `summarize` 跨多个非重叠合同积分固定费/抵扣,免去搬家"加 version"的折中。
|
||||
- **换表时给累计实体发 `last_reset` 信号**:消除 HA 长期统计在归零时刻的一次性负 blip(需给 expose/discovery 加 `last_reset` 管线,FU11 当时为省事未做)。已与用户确认本里程碑**先不做**,先观察、需要时再加。
|
||||
- **Gas / 区域供暖计费**:`commodity!="electricity"` 的 strategy 与寄存器映射。
|
||||
- **当前电表 label 暴露给 HA**、按表/按地址的成本报表分段。
|
||||
|
||||
## 10. 里程碑完成定义(DoD)
|
||||
|
||||
✅ **已完成**(M7-T01..T07 全部 done)
|
||||
|
||||
- `meter` 数据地基 + 回填上线;计费引擎永不跨表算 delta,跨表/无表/异常 delta 一律降级。
|
||||
- 累计量按当前表归零;追溯换表可重算。
|
||||
- Meter CRUD API + 前端管理 UI 可用;OpenAPI/schema 已同步。
|
||||
- 全程数据安全红线无违反;pytest/ruff/前端闸门全绿。
|
||||
- 人工 walkthrough:声明一次(含追溯)换表 → 旧表周期保留、跨表周期降级、新表从零累计、HA 累计实体不出现负/巨额跳变。
|
||||
@@ -0,0 +1,114 @@
|
||||
# Meter Epochs(电表生命周期 / 换表归档)
|
||||
|
||||
本文档说明 **Meter epoch** 概念、换表声明流程、计费隔离行为、累计量归零、追溯重算,以及与 HA 累计 blip 的已知行为。
|
||||
|
||||
## 为什么需要 Meter epoch
|
||||
|
||||
计费引擎采用**寄存器差值(delta)模型**:每 15 分钟成本 =(周期末读数 − 周期初读数)× 单价。荷兰 2G 智能电表退网后,电网公司会把表换成 4G 表(同址换表);搬家继承新表也是同类场景。
|
||||
|
||||
问题在于:**新表的寄存器基数与旧表无关**,若跨表算 delta,会产出负成本或巨额假成本。Meter epoch 让引擎知道"某时刻起属于哪一块物理表",从而永不跨表算 delta。
|
||||
|
||||
## 核心概念
|
||||
|
||||
一条 `meter` 记录 = **一块物理电表的一段安装期** `[started_at, ended_at)`:
|
||||
|
||||
- `started_at`:表安装/继承/搬家的起点(可在过去,追溯声明)。
|
||||
- `ended_at`:null = 当前 active;非 null = 已退役表的结束时刻。
|
||||
- `label`:人读标签,便于识别(如 "旧 2G 表 @ Dorpsstraat 1")。
|
||||
- `reason`:`initial`(初始建档)/ `meter_swap`(换表)/ `home_move`(搬家)/ `other`。
|
||||
- `commodity`:默认 `electricity`;为未来 gas/heating 预留,每 commodity 至多一个 active 表。
|
||||
|
||||
**换表** = 关闭旧 active 表(`ended_at = T`)+ 新建 `started_at = T` 的表。系统**无法自动识别换表**(DSMR 帧无电表序列号),靠用户**显式声明**才能隔离。
|
||||
|
||||
## 计费隔离行为
|
||||
|
||||
每个 15 分钟计费周期,引擎先查该周期两端(t0/t1)分别属于哪块表:
|
||||
|
||||
| 情况 | 结果 |
|
||||
| --- | --- |
|
||||
| 同一块表(正常)| 在表窗口内取读数、算 delta、正常计费 |
|
||||
| 无表覆盖(未声明)| **降级**(成本置 0,`degraded=true`) |
|
||||
| 跨表边界(t0 ≠ t1 的表)| **降级**(丢这一个周期,可接受;换表常伴随长时间无数据) |
|
||||
| delta < 0 或 > 100 kWh/15min | **降级**(兜底异常,如 DSMR 回绕/毛刺,与换表无关也防住) |
|
||||
|
||||
**忘了声明换表怎么办**:搬家必有的数据空档(staleness)+ delta 护栏保证不会产出垃圾成本,只是新数据暂混在旧表 epoch、累计未归零。事后补声明 + 追溯重算即可纠正(见下方"追溯换表")。
|
||||
|
||||
## 换表声明流程
|
||||
|
||||
### 通过前端(Energy 页 → Meters tab)
|
||||
|
||||
1. 在 Energy 页切到 **Meters** tab,查看当前电表时间线。
|
||||
2. 点击"声明换表 / 继承新表",填写:
|
||||
- **Label**:新表的人读标签(如 "4G 新表 2026")
|
||||
- **安装日期**:新表的 `started_at`(可选择历史日期追溯)
|
||||
- **Reason**:`meter_swap` / `home_move` / `other`
|
||||
3. 保存后,旧表自动关闭(`ended_at = 填写日期`),新表成为 active。
|
||||
4. 如填写的是过去日期,系统自动触发受影响窗口的重算(`recompute_range`),跨表周期变为降级,新表内周期重新归属。
|
||||
|
||||
### 通过 API
|
||||
|
||||
```http
|
||||
POST /api/energy/meters
|
||||
Content-Type: application/json
|
||||
|
||||
{
|
||||
"label": "4G 新表 2026",
|
||||
"started_at": "2026-07-01T00:00:00",
|
||||
"reason": "meter_swap",
|
||||
"note": "荷兰电网公司换装"
|
||||
}
|
||||
```
|
||||
|
||||
`started_at` 必须 `≥` 当前 active 表的 `started_at`(拒绝倒挂)。
|
||||
|
||||
查询所有表(时间线):
|
||||
|
||||
```http
|
||||
GET /api/energy/meters
|
||||
```
|
||||
|
||||
编辑 label / 备注 / 修正日期:
|
||||
|
||||
```http
|
||||
PATCH /api/energy/meters/{id}
|
||||
```
|
||||
|
||||
## 追溯换表(started_at 在过去)
|
||||
|
||||
若换表日期在过去但当时忘了声明:
|
||||
|
||||
1. 声明新表,填写过去的安装日期。
|
||||
2. 系统自动对 `[started_at, now]` 范围内的周期触发 `recompute_range`:
|
||||
- 跨表边界周期 → 降级。
|
||||
- 归属新表窗口内的周期 → `meter_id` 更新为新表。
|
||||
3. MQTT 累计量锚点切换至新表起点,下一次 expose 推送时累计量从新表起点重算。
|
||||
|
||||
## 累计量归零(per-meter)
|
||||
|
||||
MQTT 上报的累计成本实体(`import_cost_total` / `export_revenue_total`)锚点 = **当前 active electricity 表的 `started_at`**。换表后:
|
||||
|
||||
- 累计量从新表安装时刻起重新累加,不跨表维护历史偏移。
|
||||
- 日归零实体(`*_today`)不受影响(其窗口必落在当前表内)。
|
||||
|
||||
### 已知行为:HA 长期统计的一次性负 blip
|
||||
|
||||
累计实体 `state_class: total`,换表归零时其值会**下台阶**。Home Assistant 长期统计(energy dashboard / statistics)可能在换表那一刻记录一次性负 blip。
|
||||
|
||||
这是**已知、可接受的行为**:HA 自身存旧值做 down-sample,短期 blip 不影响日常电费读数。已与用户确认先这样观察。如需消除 blip,可在未来通过 `last_reset` 信号通知 HA(见设计文档 §9 后续杠杆,本里程碑不做)。
|
||||
|
||||
## 回填(迁移时的初始表)
|
||||
|
||||
首次升级到含 M7 迁移的版本时:
|
||||
|
||||
- 若库内已有任何 `dsmr_reading` / `energy_cost_period`,迁移自动创建一条 `reason="initial"` 的初始表(`started_at = 最早 dsmr_reading.recorded_at`),并回填现有 `energy_cost_period.meter_id`。
|
||||
- 若是全新空库,初始表不创建(无历史数据)。
|
||||
- 回填幂等:重复跑迁移不会创建多条初始表;回填后对账(非降级周期 `meter_id IS NULL` 数必须为 0)。
|
||||
|
||||
## 非目标(本里程碑不做)
|
||||
|
||||
- Gas / 区域供暖的计费(`commodity != "electricity"` 的 strategy)。
|
||||
- "家庭(home)"分组实体;多合同时间线积分。
|
||||
- 自动识别换表(DSMR 帧无电表序列号,无法自动识别,靠用户显式声明)。
|
||||
- `last_reset` 信号(消除 HA 长期统计 blip)。
|
||||
|
||||
> 详细设计与任务卡:[`docs/design/m7-meter-epochs-archival.md`](./design/m7-meter-epochs-archival.md)
|
||||
Binary file not shown.
@@ -0,0 +1,198 @@
|
||||
# DDSU666 Modbus 协议(从官方 PDF 提取)
|
||||
|
||||
> 来源:`docs/references/DDSU666 Single phase Smart Meter.pdf`
|
||||
> (CHINT / 正泰仪表 **DDSU666 Single phase Smart Meter — Operation Manual**,文档号 `ZTY0.464.1224`,版本 **V2**,2020 年 8 月;厂商 Zhejiang Chint Instrument & Meter Co., Ltd.)
|
||||
> 本文件是 PDF 的可读化提取,供本项目的 Modbus 采集驱动设计参考。**以官方 PDF 为准**,本文件如有出入以 PDF 为准。
|
||||
> 同类文档见 SDM120 的 `SDM120-Modbus-Protocol.md`;两表差异较大,见下方 §6「与 SDM120 的关键差异」。
|
||||
|
||||
## 设备速览(来自手册 Table 1 / Table 5)
|
||||
|
||||
| 项 | DDSU666(直接接入) | DDSU666-CT(经互感器) |
|
||||
| --- | --- | --- |
|
||||
| 精度等级 | Active Class B | Active Class C |
|
||||
| 参考电压 | 230 V | 230 V |
|
||||
| 电流规格 | 0.25–5(80) A | 0.015–1.5(6) A |
|
||||
| 表常数 | 800 imp/kWh | 6400 imp/kWh |
|
||||
| 接入方式 | 直接接入 | 经电流互感器 |
|
||||
|
||||
- 单相电子式电能表,DIN35mm 导轨安装;测量电压、电流、有功/无功功率、频率、功率因数、正/反向有功电能。
|
||||
- 电能测量范围 `0~999999.99 kWh`(LCD 只显示 6 位,自动移动小数点)。
|
||||
- 通信:RS485,**Modbus-RTU**(也支持 DL/T 645-2007,可切换,见 §5 `0005H ChangeProtocol`)。
|
||||
- 手册 Table 1 标注 Frequency Reference = 60Hz,但 LCD 示例又写 `F=50.00Hz`(手册自身不一致);**实际频率以寄存器 `200EH` 读数为准**,不要把 50/60 写死。
|
||||
|
||||
## 0. 本项目的接入方式(重要)
|
||||
|
||||
DDSU666 物理层是 **Modbus RTU(RS-485 串口)**,半双工。和 SDM120 一样,本项目通过一个 **Modbus-TCP 网关**接入:
|
||||
|
||||
- 后端用 **Modbus TCP**(`IP:port`)连到网关,网关在串口侧转成 RTU 与电表通信。
|
||||
- TCP 帧用 MBAP header、**无 CRC**(CRC 由网关在 RTU 侧处理)。本文档里 RTU 帧的 `CRC (Lo/Hi)` 字段在 TCP 模式下不需要我们关心。
|
||||
- **Slave Address / Unit ID = 电表的通信地址 Addr**(范围 1–247;面板按键只能设 1–99;见 §5 `0006H`),在 TCP 请求里作为 unit id 传入。
|
||||
- 若以后直连串口(RTU),才需要管波特率 / 数据格式 / CRC:**默认串口格式是 8 数据位、无校验、2 停止位(8N2)**,与 SDM120 的 8N1 不同——直连时务必对齐。
|
||||
- 电表必须处于 **Modbus 协议模式**(而非 DL/T 645)才能用本协议;可经面板长按切换,或写 `0005H = 2`(见 §5)。
|
||||
|
||||
## 1. 协议帧格式(Appendix A)
|
||||
|
||||
异步传输,按字节为单位。一帧 10 位字符 = **1 起始位(0) + 8 数据位(无校验) + 2 停止位(1)**(其它格式可定制)。
|
||||
|
||||
### 信息帧结构(Table A.1)
|
||||
|
||||
| 字段 | 长度 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| Start(起始) | >3.5 字符静默 | 帧间至少 3.5 字符空闲时间作为分隔 |
|
||||
| Address code(地址码) | 1 字节 | 目标从机地址 1–247;每个从机在总线上地址唯一 |
|
||||
| Function code(功能码) | 1 字节 | 仅支持 **03H / 10H**(见 §2) |
|
||||
| Data(数据域) | n 字节 | 随功能码不同而不同(起始地址、寄存器数、寄存器数据等) |
|
||||
| CRC check code(CRC 校验) | 2 字节 | 16-bit CRC(**低字节在前、高字节在后**;多项式 `A001`) |
|
||||
| End(结束) | >3.5 字符静默 | 帧间静默 |
|
||||
|
||||
> TCP 网关模式下 Start/End 静默与 CRC 由网关处理,本项目不关心。
|
||||
|
||||
### 功能码 03H 示例(读寄存器,Table A.3/A.4)
|
||||
|
||||
读从机 `01H`、起始地址 `0CH`、2 个寄存器:
|
||||
|
||||
- 主机发送:`01 03 00 0C 00 02 04 08`(最后 `04 08` 是 CRC,低字节 `04` 在前)。
|
||||
- 从机返回(设 `0CH/0DH` 内容为 `0000H`、`1388H`):`01 03 04 00 00 13 88 F7 65`
|
||||
- `04` = 字节数;`00 00` = `0CH` 数据;`13 88` = `0DH` 数据;`F7 65` = CRC(低字节 `F7` 在前)。
|
||||
|
||||
> **注意**:单个 16-bit 寄存器内是「高字节在前、低字节在后」(Table A.4 里 `0DH` 数据返回 `13 88` = `0x1388`)。这一点对解码浮点的字节序很关键,见 §3。
|
||||
|
||||
### 功能码 10H 示例(写多个寄存器,Table A.5/A.6)
|
||||
|
||||
向从机 `01H`、起始地址 `00H` 连续写 3 个寄存器 `0002H,1388H,000AH`:
|
||||
|
||||
- 主机发送:`01 10 00 00 00 03 06 00 02 13 88 00 0A 9B E9`
|
||||
- `06` = 写入字节数(3 寄存器 × 2 字节);随后是 3 个寄存器的数据;末尾 `9B E9` CRC。
|
||||
- 从机返回:`01 10 00 00 00 03 80 08`(回显起始地址 + 寄存器数 + CRC `80 08`)。
|
||||
|
||||
### 异常响应(Table A.7/A.8)
|
||||
|
||||
- 异常时返回的 Function Code = **原功能码 + 128**(即最高位置 1,`03H→83H`、`10H→90H`)。
|
||||
- 数据为单字节 Error Code:
|
||||
|
||||
| Error Code | 含义 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| `01H` | Illegal function code | 收到的功能码本表不支持 |
|
||||
| `02H` | Illegal register address | 寄存器地址超出有效范围 |
|
||||
| `03H` | Illegal data value | 数据值超出对应地址的取值范围 |
|
||||
|
||||
## 2. 功能码
|
||||
|
||||
DDSU666 **只支持两个功能码**(Table A.2):
|
||||
|
||||
| 功能码 | 作用 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| **03H** | Read register(读寄存器) | 读一个或多个寄存器——**测量值、电量、配置全部走它** |
|
||||
| **10H** | Write multiple registers(写多个寄存器) | 向 n 个连续寄存器写 n 个 16-bit 数据(改配置 / 清电量) |
|
||||
|
||||
> ⚠️ **DDSU666 没有「输入寄存器 / FC04」概念**——所有数据(包括电压电流功率)都用 **FC 03H** 读保持寄存器。这是它和 SDM120(测量值走 FC04)最大的踩坑差异,见 §6。
|
||||
|
||||
## 3. 数据编码
|
||||
|
||||
DDSU666 有两类数据:
|
||||
|
||||
1. **配置 / 参数寄存器(§5,`0000H`–`0010H`)**:每个 1 个寄存器、`16-bit with symbols`(**16 位有符号整数**)。
|
||||
2. **测量 / 电量寄存器(§4,`2000H`+ / `4000H`+)**:每个参数 = **32-bit IEEE-754 单精度浮点**(手册写 “single precision floating decimal”),占 **2 个相邻寄存器**(Length = 2 Word)。
|
||||
|
||||
**浮点字节序 / 字序**
|
||||
|
||||
- **字节序(byte order)= 大端:寄存器内高字节在前** —— 由 Table A.4 的 `0x1388` 返回为 `13 88` 确认。
|
||||
- **字序(word order,两个寄存器谁是高 16 位)**:手册**没有给出浮点解码的实例**,未明确标注。按标准 Modbus 浮点惯例应为**大端字序(高寄存器在前,`ABCD`)**,与本项目 SDM120 驱动一致(`registers_to_float` 用 `>f`,高寄存器在前)。
|
||||
- ⚠️ **需上机实测确认**:读 `2000H`(电压)应解出 ~230V 这样的合理值;若解出乱数,多半是字序相反,改成「低寄存器在前」再试。CHINT 同系列(DTSU/DDSU666)现场固件偶有字序差异,**首次接入务必用一个已知量(电压)校验**,不要凭手册想当然。
|
||||
|
||||
> Python 解码(大端、高寄存器在前):`struct.unpack('>f', struct.pack('>HH', hi_reg, lo_reg))[0]`。
|
||||
> pymodbus:`BinaryPayloadDecoder.fromRegisters(regs, byteorder=Endian.BIG, wordorder=Endian.BIG)`。
|
||||
|
||||
## 4. 测量 / 电量寄存器表(FC 03H 读)
|
||||
|
||||
全部为只读、`Float`(32-bit),每项占 **2 个寄存器**。地址为 Modbus 协议原始地址(即帧里的 Start Register Address Hi/Lo),手册用十六进制。
|
||||
|
||||
### 4.1 瞬时量(“Electric quantity of the secondary side”,`2000H` 段)
|
||||
|
||||
| 地址(hex) | 参数 | 代号 | 单位 | 备注 |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| `2000H` | 电压 Voltage | U | V | |
|
||||
| `2002H` | 电流 Current | I | A | |
|
||||
| `2004H` | 有功功率 Active power | P | **kW** | 手册标注 “the unit is KW”——**不是 W** |
|
||||
| `2006H` | 无功功率 Reactive power | Q | **kvar** | |
|
||||
| `2008H` | (保留 RESERVED) | — | — | 占 2 寄存器,跳过 |
|
||||
| `200AH` | 功率因数 Power factor | PF | — | 无量纲 |
|
||||
| `200CH` | (保留 RESERVED) | — | — | 占 2 寄存器,跳过 |
|
||||
| `200EH` | 频率 Frequency | Freq | Hz | |
|
||||
|
||||
### 4.2 电量(“Electrical data of the secondary side”,`4000H` 段)
|
||||
|
||||
| 地址(hex) | 参数 | 代号 | 单位 | 备注 |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| `4000H` | 正向(导入)有功电能 Active in electricity | Ep | kWh | 正向 / forward active energy |
|
||||
| `400AH` | 反向(导出)有功电能 Reverse in electricity | -Ep | kWh | 反向 / reverse active energy |
|
||||
|
||||
> 手册里 `4000H` 与 `400AH` 之间(`4002H`–`4009H`)未列出,视为保留/未文档化。
|
||||
|
||||
**读取分块建议**
|
||||
|
||||
- 瞬时量:`2000H`–`200FH` 共 **16 个寄存器连续**,一次块读即可覆盖 U…Freq(含两段 RESERVED,解码时跳过)。
|
||||
- 电量:`4000H`(2 寄存器)与 `400AH`(2 寄存器)相距较远,分两小块读,或读 `4000H`–`400BH`(12 寄存器)一次取出后挑用——以网关 / 电表是否允许跨保留地址块读为准,谨慎起见分开读更稳。
|
||||
|
||||
### 常用核心子集(日常监控够用)
|
||||
|
||||
电压 `2000H`、电流 `2002H`、有功功率 `2004H`(kW)、功率因数 `200AH`、频率 `200EH`、正向有功电能 `4000H`、反向有功电能 `400AH`。
|
||||
|
||||
## 5. 配置 / 参数寄存器表(FC 03H 读 / FC 10H 写)(Table 9)
|
||||
|
||||
每个 1 个寄存器、**16-bit 有符号整数**。`R/W` 列来自手册。
|
||||
|
||||
| 地址(hex) | 代号 | 含义 | R/W | 取值 / 说明 |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| `0000H` | UCode | 编程密码 Programming password code | R/W | 写配置前的密码字 |
|
||||
| `0001H` | REV. | 保留;**实际读出的是版本号** | R | |
|
||||
| `0002H` | ClrE | 电能清零 CLr.E | R/W | **写 `1` 清除总电量**(不可逆,慎用) |
|
||||
| `0003H`–`0004H` | RESERVED | 保留 | — | |
|
||||
| `0005H` | ChangeProtocol | 协议切换 | R/W | **`2` = Modbus-RTU**,`1` = DL/T 645-2007 |
|
||||
| `0006H` | Addr | 通信地址 | R/W | 1–247(面板按键仅 1–99) |
|
||||
| `0007H`–`000AH` | RESERVED | 保留 | — | |
|
||||
| `000BH` | Meter type | 表型 Meter type | R | 只读设备类型标识 |
|
||||
| `000CH` | BAud | 通信波特率 | R/W | **`1`=2400bps,`2`=4800bps,`3`=9600bps**(手册寄存器仅列这三档;通信章另提到也支持 1200bps) |
|
||||
| `000DH`–`0010H` | RESERVED | 保留 | — | |
|
||||
|
||||
> ⚠️ 写 `0002H`(清电量)、`0006H`(改地址)、`000CH`(改波特率)、`0005H`(切协议)都会改变电表状态或通信参数,配错可能**清空累计电量**或**导致通信中断**。本项目默认**只读采集**,不在自动化链路里写电表配置寄存器。
|
||||
> 写配置通常需先经 `0000H UCode` 密码字校验;具体密码值手册正文未给出,需向厂商确认或经面板操作。
|
||||
|
||||
## 6. 与 SDM120 的关键差异(迁移 / 复用驱动时必看)
|
||||
|
||||
本项目已有 SDM120 profile(`app/integrations/modbus/profiles/sdm120.yaml`)。DDSU666 **不能照搬**,主要差异:
|
||||
|
||||
| 维度 | SDM120 (Eastron) | DDSU666 (CHINT) |
|
||||
| --- | --- | --- |
|
||||
| 读测量值功能码 | **FC 04**(输入寄存器 3X) | **FC 03**(保持寄存器,无 FC04) |
|
||||
| 测量值起始地址 | `0x0000` 起(30001) | 瞬时量 `0x2000` 起;电量 `0x4000`/`0x400A` |
|
||||
| 有功功率单位 | **W**(瓦) | **kW(千瓦)** —— 入库前注意换算 / 单位标注 |
|
||||
| 无功功率单位 | VAr | kvar |
|
||||
| 配置寄存器格式 | Float(FC03/16) | **16-bit 有符号整数**(FC03/10) |
|
||||
| 写功能码 | 16 / 0x10 | 10H(同 0x10) |
|
||||
| 串口默认格式 | 8N1(1 停止位) | **8N2(2 停止位)** |
|
||||
| 多协议 | 仅 Modbus | Modbus **与 DL/T 645-2007 可切换**(需确保在 Modbus 模式) |
|
||||
| 浮点字序 | 大端、高寄存器在前(手册有实例佐证) | 字节序大端已确认;**字序手册无实例,需上机实测** |
|
||||
|
||||
## 7. 给本项目采集驱动的要点小结
|
||||
|
||||
1. 走 **Modbus TCP 网关**:`ModbusTcpClient(host, port)`,`slave=<Addr>`;电表须在 **Modbus 协议模式**。
|
||||
2. 所有读取(测量 + 电量 + 配置)都用 **FC 03H**——**没有 FC04**。
|
||||
3. 测量值在 `0x2000` 段、电量在 `0x4000`/`0x400A`,均为 **float32**;解码大端字节序,**字序默认高寄存器在前但务必用电压实测校验**。
|
||||
4. **有功功率单位是 kW、无功是 kvar**——与 SDM120 的 W/VAr 不同,新建 profile / 入库映射时单位别抄错。
|
||||
5. 配置寄存器(`0x0000`–`0x0010`)是 **16-bit 有符号整数**,不是 float。
|
||||
6. 默认**只读**;`0002H` 写 1 会**清空累计电量**、`0006H/000CH/0005H` 会改通信参数,自动化链路里一律不写。
|
||||
7. 新建 profile 时这是 `ddsu666` 这一个 register profile 的定义;建议 `function_code: 3`、`word_order: big`(先按大端字序,接入后用电压读数验证)、瞬时量与电量分块读。
|
||||
|
||||
## 8. 实测记录(真机验证,2026-06-30)
|
||||
|
||||
首次接入一台 **DDSU666 直接接入版(5(80)A)** 实测,确认以下几点:
|
||||
|
||||
- **字序大端,确认无误**:电压 / 电流 / 频率 / 电能用「大端、高寄存器在前」解码全部得到合理值(如 233.9 V / 0.055 A / 49.99 Hz),与 §3 的假设一致。`ddsu666.yaml` 的 `word_order: big` / `byte_order: big` **无需修改**,§3 里「字序需上机实测」一项可视为已关闭。
|
||||
- **FC03 读通**:所有量走 FC03,profile `ddsu666` 在采集链路(CLI `read` / 设备 `/test` / 后台轮询)中工作正常。
|
||||
- **低电流下「瞬时功率读 0、但电能照常累加」**:测试负载仅为一台 PoE 交换机(≈230 V / 0.05 A,真实有功仅几瓦),**远低于本表测量量程下限 Imin≈0.25 A**。此工况下:
|
||||
- 瞬时 `active_power`(`0x2004`)与 `power_factor`(`0x200A`)寄存器返回**全零**(原始 hex `0x0000 0x0000`);电表 LCD 上功率在 0~3.3 W、PF 在 0~0.6 之间抖动。
|
||||
- 抖动成因 = **低电流测量噪声 + 开关电源(SMPS,无 PFC)畸变电流**(电流为电压峰值附近的窄脉冲、谐波重 → 畸变功率因数天然偏低)。**不是**「采样率与开关频率拍频」:计量芯片 SH79F7019 采样在 kHz 量级,远低于开关电源 50–200 kHz 的开关频率,两者不在一个频段。
|
||||
- 但累计电能 `import_energy`(`0x4000`)**正常累加**(实测 0 → 0.01 kWh)——电表内部积分器在防潜动起始电流(`0.004·Ib`≈0.02 A)之上照常计量。
|
||||
- **结论**:低于量程下限时**瞬时功率 / PF 不可信,但电能计量不丢**;电流进入量程(正常负载)后瞬时量即稳定可信。这是 5(80)A 大量程表对极小负载的固有特性,**非缺陷、非解码问题**。
|
||||
- **排错提示**:若日后看到 DDSU666「功率一直 0」,先确认负载电流是否在 Imin 以上——多半是负载太轻而非链路故障;可用 `scripts/modbus_cli probe --fc 3 --address 0x2000 --count 16` 看 `0x2004/0x2005` 原始寄存器是否真为全零佐证。
|
||||
Binary file not shown.
@@ -0,0 +1,134 @@
|
||||
# SDM120 Modbus 协议(从官方 PDF 提取)
|
||||
|
||||
> 来源:`docs/references/SDM120-MODBUS_Protocol.pdf`
|
||||
> (Eastron SDM120 Modbus Smart Meter Modbus Protocol Implementation **V2.4**)
|
||||
> 本文件是 PDF 的可读化提取,供本项目的 Modbus 采集驱动设计参考。**以官方 PDF 为准**,本文件如有出入以 PDF 为准。
|
||||
|
||||
## 0. 本项目的接入方式(重要)
|
||||
|
||||
SDM120 物理层是 **Modbus RTU(RS-485 串口)**。本项目通过一个 **Modbus-TCP 网关**接入:
|
||||
|
||||
- 后端用 **Modbus TCP**(`IP:port`)连到网关,网关在串口侧转成 RTU 与电表通信。
|
||||
- TCP 帧用 MBAP header、**无 CRC**(CRC/Error Check 由网关在 RTU 侧处理)。本文档里 RTU 帧的 `Error Check (Lo/Hi)` 字段在 TCP 模式下不需要我们关心。
|
||||
- **Slave Address / Unit ID = 电表的 Meter ID**(默认 `1`,范围 1–247),在 TCP 请求里作为 unit id 传入。
|
||||
- 若以后直连串口(RTU),才需要管波特率 / 校验位 / CRC(见 §5 holding 寄存器)。
|
||||
|
||||
## 1. 协议帧格式
|
||||
|
||||
MODBUS 定义 master 查询 / slave 响应的格式。Eastron 电表用 16-bit 寄存器在主从间传值,**但实际数据是 32-bit IEEE-754 浮点**,因此每个测量参数占**两个相邻的 16-bit 寄存器**。
|
||||
|
||||
### Query(master → slave,RTU 帧)
|
||||
|
||||
| 字段 | 说明 |
|
||||
| --- | --- |
|
||||
| Slave Address | 8-bit,目标从机地址 1–247(0=广播,Eastron 不支持广播) |
|
||||
| Function Code | 8-bit,功能码(Eastron 支持 **03 / 04 / 08 / 16(=0x10)**) |
|
||||
| Start Address (Hi/Lo) | 16-bit 起始寄存器地址;**寄存器成对使用、从 0 开始,所以起始地址必须是偶数** |
|
||||
| Number of Points (Hi/Lo) | 16-bit 请求的寄存器数量;**也必须是偶数**(成对读浮点) |
|
||||
| Error Check (Lo/Hi) | 16-bit CRC(RTU 模式;TCP 网关模式无此字段) |
|
||||
|
||||
### Response(slave → master)
|
||||
|
||||
| 字段 | 说明 |
|
||||
| --- | --- |
|
||||
| Slave Address | 响应从机地址 |
|
||||
| Function Code | 与查询相同的功能码(表示识别并已响应) |
|
||||
| Byte Count | 8-bit,本次返回的数据字节数 |
|
||||
| Data (寄存器对) | 每个寄存器 Hi byte / Lo byte,按"高寄存器在前"排列 |
|
||||
| Error Check (Lo/Hi) | 16-bit CRC(RTU 模式) |
|
||||
|
||||
### Exception Response(异常响应)
|
||||
|
||||
- 异常响应的 Function Code = 查询功能码 **OR 0x80**(最高位置 1)。
|
||||
- 数据是单字节 Error Code(异常码)。
|
||||
- PDF 正文写了"见后文 Table Of Exception Codes",但提取的 8 页里**没有附上该表**。标准 Modbus 异常码(供参考,非本 PDF 内容):`01` 非法功能、`02` 非法数据地址、`03` 非法数据值、`04` 从机设备故障。
|
||||
|
||||
## 2. 功能码
|
||||
|
||||
| 功能码 | 作用 | 寄存器区 |
|
||||
| --- | --- | --- |
|
||||
| **04** | Read Input Registers(读输入寄存器,3X)—— **所有测量值都在这里** | 30001+ |
|
||||
| **03** | Read Holding Registers(读保持寄存器,4X)—— 配置项 | 40001+ |
|
||||
| **16 / 0x10** | Write Holding Registers(写保持寄存器,4X)—— 改配置 | 40001+ |
|
||||
| 08 | Diagnostics(诊断) | — |
|
||||
|
||||
> ⚠️ **测量值用 FC 04(输入寄存器),不是 FC 03。** 这是最常见的踩坑点。
|
||||
|
||||
## 3. 浮点数据编码
|
||||
|
||||
- 每个参数 = **32-bit IEEE-754 float**,占两个相邻 16-bit 寄存器。
|
||||
- **字序(word order)= 大端:高寄存器在前。**
|
||||
- **字节序(byte order)= 大端:寄存器内高字节在前。**
|
||||
- 即整体就是标准大端 float(`>f`),4 字节顺序 = `[Reg1 Hi][Reg1 Lo][Reg2 Hi][Reg2 Lo]`。
|
||||
|
||||
**实例(来自 PDF)**
|
||||
|
||||
| 含义 | 原始 4 字节 (hex) | 解码值 |
|
||||
| --- | --- | --- |
|
||||
| Volts 1 | `43 66 33 34` | `230.2` V |
|
||||
| Demand Time | `3F 80 00 00` | `1.0` |
|
||||
| Network Node | `42 70 00 00` | `60.0` |
|
||||
|
||||
> Python 解码:`struct.unpack('>f', bytes([0x43,0x66,0x33,0x34]))[0]` → `230.2`。
|
||||
> pymodbus 用 `BinaryPayloadDecoder.fromRegisters(regs, byteorder=Endian.BIG, wordorder=Endian.BIG)`。
|
||||
|
||||
## 4. 输入寄存器表(FC 04 读测量值)
|
||||
|
||||
全部为 `Float`,长度 4 字节,每项占 2 个寄存器。"Hex 起始"是 Modbus 协议起始地址(即 Start Address Hi/Lo)。
|
||||
|
||||
| 寄存器 | 参数 | 单位 | Hex 起始 |
|
||||
| --- | --- | --- | --- |
|
||||
| 30001 | Voltage(电压) | Volts | `0000` |
|
||||
| 30007 | Current(电流) | Amps | `0006` |
|
||||
| 30013 | Active power(有功功率) | Watts | `000C` |
|
||||
| 30019 | Apparent power(视在功率) | VA | `0012` |
|
||||
| 30025 | Reactive power(无功功率) | VAr | `0018` |
|
||||
| 30031 | Power factor(功率因数) | — | `001E` |
|
||||
| 30071 | Frequency(频率) | Hz | `0046` |
|
||||
| 30073 | Import active energy(导入有功电能) | kWh | `0048` |
|
||||
| 30075 | Export active energy(导出有功电能) | kWh | `004A` |
|
||||
| 30077 | Import reactive energy(导入无功电能) | kvarh | `004C` |
|
||||
| 30079 | Export reactive energy(导出无功电能) | kvarh | `004E` |
|
||||
| 30085 | Total system power demand | W | `0054` |
|
||||
| 30087 | Maximum total system power demand | W | `0056` |
|
||||
| 30089 | Import system power demand | W | `0058` |
|
||||
| 30091 | Maximum import system power demand | W | `005A` |
|
||||
| 30093 | Export system power demand | W | `005C` |
|
||||
| 30095 | Maximum export system power demand | W | `005E` |
|
||||
| 30259 | Current demand | Amps | `0102` |
|
||||
| 30265 | Maximum current demand | Amps | `0108` |
|
||||
| 30343 | Total active energy(总有功电能) | kWh | `0156` |
|
||||
| 30345 | Total reactive energy(总无功电能) | kvarh | `0158` |
|
||||
|
||||
**读取分块建议**:`0x0000–0x005E`(30001–30095)地址连续,可一次块读;`0x0102/0x0108`、`0x0156/0x0158` 各为独立小块。整表用 2–3 次块读即可覆盖,减少 Modbus 事务数。
|
||||
|
||||
### 常用核心子集(日常监控够用)
|
||||
|
||||
电压 `0000`、电流 `0006`、有功功率 `000C`、功率因数 `001E`、频率 `0046`、导入有功电能 `0048`、导出有功电能 `004A`、总有功电能 `0156`。
|
||||
|
||||
## 5. 保持寄存器表(FC 03 读 / FC 16 写配置)
|
||||
|
||||
| 寄存器 | 参数 | Hex 起始 | 格式 | 说明 |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| 40013 | Relay Pulse Width | `000C` | Float | 继电器脉宽 60/100/200 ms,默认 100ms |
|
||||
| 40019 | Network Parity Stop | `0012` | Float | 0=1停止位无校验(默认),1=1停止位偶校验,2=1停止位奇校验,3=2停止位无校验;**改后需重启生效** |
|
||||
| 40021 | Meter ID | `0014` | Float | 从机地址 1–247,默认 1 |
|
||||
| 40029 | Baud rate | `001C` | Float | 0=2400(默认),1=4800,2=9600,5=1200 |
|
||||
| 40087 | Pulse 1 output mode | `0056` | Float | 0001 导入有功,0002 导入+导出有功,0004 导出有功(默认),0005 导入无功,0006 导入+导出无功,0008 导出无功 |
|
||||
| 463745 | Time of scroll display | `F900` | HEX 2字节 | 滚动显示时间 0–30s,默认 0(不滚动) |
|
||||
| 463761 | Pulse 1 output | `F910` | HEX 2字节 | 0000:0.001kWh/imp(默认),0001:0.01,0002:0.1,0003:1 kWh/imp |
|
||||
| 463777 | Measurement mode | `F920` | HEX 2字节 | 1:total=import,2:total=import+export(默认),3:total=import-export |
|
||||
| 464513 | Serial number | `FC00` | uint32 4字节 | 序列号,**只读** |
|
||||
| 464515 | Meter code | `FC02` | Hex 2字节 | 设备码=`0020`,**只读** |
|
||||
| 464516 | Software version | `FC03` | Hex 2字节 | 软件版本,**只读** |
|
||||
|
||||
> ⚠️ 写保持寄存器(改 Meter ID / 波特率 / 校验位等)会改变电表通信参数,配错可能导致**通信中断**。本项目默认**只读采集**,不建议在自动化链路里写电表配置。
|
||||
|
||||
## 6. 给本项目采集驱动的要点小结
|
||||
|
||||
1. 走 **Modbus TCP 网关**:`ModbusTcpClient(host, port)`,`slave=<Meter ID>`。
|
||||
2. 测量值用 **FC 04 / 输入寄存器**,按 §4 地址读。
|
||||
3. 解码 **大端 float32**(word & byte 都大端,高寄存器在前)。
|
||||
4. 起始地址与数量**都用偶数**(成对读)。
|
||||
5. 默认**只读**,不写电表配置寄存器。
|
||||
6. 不同型号电表 → 不同"寄存器 profile"。本表是 `sdm120` 这一个 profile 的定义。
|
||||
@@ -0,0 +1,249 @@
|
||||
# Tibber API + 荷兰电价 + DSMR Reader 参考
|
||||
|
||||
> 本文件汇总 M6(动态/固定电价 → 实时买卖电费计算)所需的全部外部事实,供 `docs/design/m6-tibber-dynamic-energy.md` 与实现者直接 refer。
|
||||
> 来源:Tibber GraphQL introspection(实证)、Tibber Explorer demo 数据、Tibber NL support 文档、用户实际合同与 DSMR Reader 配置(2026-06 整理)。
|
||||
> 凡标「✅ 实证」的为代码/数据验证过的事实;金额类以用户真实账单 / 合同生效后真实 token 为最终准(见末尾「待真实数据核对」)。
|
||||
|
||||
---
|
||||
|
||||
## 1. Tibber GraphQL API
|
||||
|
||||
- **Endpoint**:`POST https://api.tibber.com/v1-beta/gql`
|
||||
- **认证**:HTTP header `Authorization: Bearer <token>`;token 在 [developer.tibber.com](https://developer.tibber.com) 登录后生成(每个 Tibber 账户一个)。
|
||||
- **在线 Explorer**:[developer.tibber.com/explorer](https://developer.tibber.com/explorer),自带一个公共 **demo token**。
|
||||
- ⚠️ **demo token 现在只能做 schema introspection 与历史 demo 数据查询**;对 `viewer` 的真实数据查询返回 `UNAUTHENTICATED`。要拿自己家的实时价,必须用**自己账户的 token**。
|
||||
|
||||
### 1.1 `Price` 类型(✅ introspection 实证)
|
||||
|
||||
`Price` 对象恰好 6 个字段:
|
||||
|
||||
| 字段 | 类型 | 含义 |
|
||||
| --- | --- | --- |
|
||||
| `total` | Float | **全包价 = energy + tax**(即 App 里的 All-in Price,含 spot+能源税+VAT 等)|
|
||||
| `energy` | Float | 纯能源/现货分量(ex-VAT 市场价)|
|
||||
| `tax` | Float | 税费分量(能源税 + VAT 合并)|
|
||||
| `startsAt` | String | 该价格段起点(ISO8601,**含时区偏移**,如 `+02:00`)|
|
||||
| `currency` | String! | 币种(唯一 non-null 字段;NL 为 EUR,demo 为 SEK)|
|
||||
| `level` | PriceLevel | 价格档枚举(`VERY_CHEAP/CHEAP/NORMAL/EXPENSIVE/VERY_EXPENSIVE`)|
|
||||
|
||||
**✅ 实证**:`total == energy + tax` 精确成立(见 §1.4 样本)。
|
||||
|
||||
### 1.2 15 分钟价格:`Subscription.priceInfoRange`(✅ introspection 实证)
|
||||
|
||||
- **字段**:`currentSubscription.priceInfoRange`
|
||||
- **参数**:`resolution: PriceInfoRangeResolution!`(必填)、`first: Int`、`last: Int`、`before: String`、`after: String`(游标分页)
|
||||
- **resolution 枚举 `PriceInfoRangeResolution`**:值 `DAILY` / `HOURLY` / `QUARTER_HOURLY`
|
||||
- ⚠️ 注意别用错枚举:另有 `PriceResolution`(只有 HOURLY/DAILY,**无 QUARTER_HOURLY**)和 `PriceInfoResolution`——`priceInfoRange` 用的是 **`PriceInfoRangeResolution`**。
|
||||
- **返回**:`SubscriptionPriceConnection { nodes: [Price]!, edges: [...], pageInfo }`
|
||||
- **废弃说明**:老的 `priceInfo.range`(`deprecationReason: "use Subscription.priceInfoRange instead"`)**不支持 QUARTER_HOURLY**,必须改用 `Subscription.priceInfoRange`。
|
||||
- **✅ 实证(demo key,2018 历史日)**:`QUARTER_HOURLY` 返回 **96 个节点、间隔精确 15 分钟**;无真 15 分钟价的历史日,API 把**小时价重复成 4 个相同刻钟**(00/15/30/45 同值,到下一整点才变);NL 当前数据则是真 15 分钟。
|
||||
- **解析注意**:**不要假设固定 96 个节点 / 固定 15 分钟间隔**,按 `startsAt` 逐点存最稳。
|
||||
|
||||
**15 分钟查询(拉今天 96 个刻钟)**:
|
||||
```graphql
|
||||
{ viewer { homes { id currentSubscription {
|
||||
priceInfoRange(resolution: QUARTER_HOURLY, first: 96) {
|
||||
nodes { startsAt total energy tax currency level }
|
||||
} } } } }
|
||||
```
|
||||
|
||||
### 1.3 小时价 / 当前价 / 探测:`priceInfo`
|
||||
|
||||
`currentSubscription.priceInfo` 含 `current`(当前价)、`today[]`、`tomorrow[]`(次日价约 13:00 CET 发布)。探测账户能力 + 当前价:
|
||||
```graphql
|
||||
{ viewer { homes {
|
||||
id appNickname
|
||||
features { realTimeConsumptionEnabled }
|
||||
currentSubscription { priceInfo {
|
||||
current { total energy tax startsAt level currency }
|
||||
today { total energy tax startsAt level }
|
||||
tomorrow { total energy tax startsAt level }
|
||||
} } } } }
|
||||
```
|
||||
|
||||
### 1.4 现成 curl + 样本响应
|
||||
|
||||
```bash
|
||||
# 15 分钟价(换成自己账户 token;demo token 拿不到 viewer 真实数据)
|
||||
curl -s -X POST https://api.tibber.com/v1-beta/gql \
|
||||
-H "Authorization: Bearer <YOUR_TOKEN>" -H "Content-Type: application/json" \
|
||||
-d '{"query":"{ viewer { homes { id currentSubscription { priceInfoRange(resolution: QUARTER_HOURLY, first: 96) { nodes { startsAt total energy tax currency } } } } } }"}'
|
||||
```
|
||||
|
||||
**样本:`priceInfo`(demo,瑞典家庭,SEK)**
|
||||
```json
|
||||
{"current": {"total": 0.8239, "energy": 0.5663, "tax": 0.2576,
|
||||
"startsAt": "2026-06-23T14:00:00.000+02:00", "level": "NORMAL", "currency": "SEK"},
|
||||
"today": [{"total": 1.276, "energy": 0.928, "tax": 0.348,
|
||||
"startsAt": "2026-06-23T00:00:00.000+02:00", "level": "VERY_EXPENSIVE"}, … 24 项],
|
||||
"tomorrow":[… 24 项]}
|
||||
```
|
||||
|
||||
**样本:`priceInfoRange` QUARTER_HOURLY(demo,2018-11-02,96 节点)**
|
||||
```json
|
||||
{"nodes": [
|
||||
{"startsAt":"2018-11-02T00:00:00.000+01:00","total":0.63, "energy":0.435,"tax":0.195,"currency":"SEK"},
|
||||
{"startsAt":"2018-11-02T00:15:00.000+01:00","total":0.63, "energy":0.435,"tax":0.195,"currency":"SEK"},
|
||||
{"startsAt":"2018-11-02T00:30:00.000+01:00","total":0.63, "energy":0.435,"tax":0.195,"currency":"SEK"},
|
||||
{"startsAt":"2018-11-02T00:45:00.000+01:00","total":0.63, "energy":0.435,"tax":0.195,"currency":"SEK"},
|
||||
{"startsAt":"2018-11-02T01:00:00.000+01:00","total":0.6141,"energy":0.4223,"tax":0.1918,"currency":"SEK"},
|
||||
… 共 96 个,末节点 23:45 total 0.6342]}
|
||||
```
|
||||
(注意 00:00–00:45 四个刻钟同值 = 小时价被重复;`total==energy+tax`。)
|
||||
|
||||
---
|
||||
|
||||
## 2. 荷兰电价构成(Tibber NL + 用户合同)
|
||||
|
||||
> 所有金额**含 VAT(incl. btw)**,除非另注。VAT/BTW 标准税率 **21%**。
|
||||
|
||||
| 分量 | 荷兰语 | 单位 | 值 / 说明 |
|
||||
| --- | --- | --- | --- |
|
||||
| 现货/市场价 | marktprijs / dynamische kwartierprijs | €/kWh | 每 15 分钟跟随交易所;= API `energy` |
|
||||
| 买侧采购费 | **inkoopvergoeding** | €/kWh | **€0.0248**(覆盖 onbalans + 绿证;见下「相等」事实)|
|
||||
| 卖侧上网费 | **verkoopvergoeding** | €/kWh | **€0.0248**(2026-01-01 起)|
|
||||
| 能源税 | energiebelasting | €/kWh(含 VAT)| 2026 第一档(0–10000 kWh)≈ **€0.1108**;2025 ≈ €0.1228;高档更低 |
|
||||
| 增值税 | BTW | % | **21%** |
|
||||
| 固定月费 | vast bedrag / leveringskosten | €/月/合同 | Tibber **€5.99/月**(电、气各一次)|
|
||||
| 电网维护费 | netbeheerkosten / systeembeheerkosten | €/天或/月 | 电网公司定、供应商代收;按地区,2026 约 +3.38% |
|
||||
| 能源税抵扣 | heffingskorting / vermindering energiebelasting | €/年/连接 | 政府年度减免,从总费用扣 |
|
||||
| ODE | Opslag Duurzame Energie | €/kWh | **现为 0**(已并入 energiebelasting)|
|
||||
|
||||
**✅ 关键事实(Tibber NL 文档原文)**:
|
||||
> "De verkoopvergoeding van 2,48 cent is gelijk aan de inkoopvergoeding die je bij je afgenomen stroom betaalt."
|
||||
> (卖侧 verkoopvergoeding 2.48 分 = 买侧 inkoopvergoeding。)
|
||||
|
||||
→ **买卖服务费相等(均 €0.0248/kWh)**,在买卖里一进一出**相互抵消**。
|
||||
|
||||
---
|
||||
|
||||
## 3. 净计量(saldering)、回送(teruglevering)、负电价、2027
|
||||
|
||||
### 3.1 回送价(净计量期内,文档原文)
|
||||
> "Op het moment dat je teruglevert geven we je per kWh de beursprijs die op dat moment geldt …, inclusief energiebelasting en inkoopvergoeding plus de btw minus de verkoopvergoeding."
|
||||
|
||||
即净计量期内回送价 = `beursprijs + energiebelasting + inkoopvergoeding + btw − verkoopvergoeding`。因 inkoopvergoeding = verkoopvergoeding 抵消 → **= 全额零售价**(spot+能源税+VAT),正是 saldering "回送 1 度 = 用 1 度"的本质。
|
||||
|
||||
### 3.2 年末盈余 / 取消净计量后(文档原文,Scenario 2)
|
||||
> "Voor de overproductie van 500 kWh heb je recht op de beursprijs en de inkoopvergoeding, maar heb je geen recht op de energiebelasting. … ontvang je nog een factuur van ons voor de te veel uitgekeerde belastingen …"
|
||||
|
||||
即**超额回送(或 2027 取消净计量后)**:盈余按 `beursprijs + inkoopvergoeding − verkoopvergoeding`(**无能源税**)计价 → 因两费抵消 → **= 纯 spot**。
|
||||
|
||||
### 3.3 净计量机制
|
||||
- 法定**按自然年**结算;可抵扣到「用电量」为止(回送抵到用电为止)。
|
||||
- Tibber 不用预付,**净计量周期在第 12 张账单后结束**。
|
||||
- **2027 起荷兰取消净计量(saldering stopt in 2027)**——用户据此设计:**不再考虑净计量,直接按实时买卖算**。
|
||||
|
||||
### 3.4 负电价
|
||||
- 负价时**用电**:理论上你拿钱(但仍计能源税,净计量期内税会退回)。
|
||||
- 负价时**回送**:你要为回送的电付那个负价(= 倒贴)。
|
||||
- 因 Tibber 给的是 all-in `total`(已含能源税),**只有 spot 负到比能源税还多,total 才转负**;中等负价时 total 仍正(照付)。
|
||||
- **结论:负电价不需要特殊处理**——`buy=total` 与 `sell=total−能源税` 符号自洽。
|
||||
|
||||
---
|
||||
|
||||
## 4. 本项目采用的买/卖价公式(M6)
|
||||
|
||||
> spot 取 API `energy`;`total = energy + tax`(全包)。**买价直接用 `total`**,卖价从 `total` 扣掉卖电不交的能源税。
|
||||
|
||||
- **Tibber 动态合同**(post-2027 口径):
|
||||
- 买价 `buy = price.total`
|
||||
- 卖价 `sell = price.total − energy_tax_per_kwh − sell_adjust`(`sell_adjust` 默认 0;含 VAT 归己;买卖费抵消已隐含在 total 里)
|
||||
- **固定合同(manual,双费率)**:
|
||||
- 买价 `buy_档 = energy_buy_档 + energy_tax`(档 ∈ {normal, dal})
|
||||
- 卖价 `sell_档 = sell_档`(回送价,**无能源税**)
|
||||
- **固定费/抵扣不进每度**:`network_fee`、`management_fee`(月→天)、`heffingskorting`(年→天)在**日/月/年汇总**层加减。
|
||||
- 详见 `docs/design/m6-tibber-dynamic-energy.md` §3.4。
|
||||
|
||||
---
|
||||
|
||||
## 5. 用户当前固定合同(manual 实例,2026-06)
|
||||
|
||||
> 双费率(NL:`_1`=dal/低谷、`_2`=normal/高峰)。以下为当前值,会随合同阶段/年度变(系统按"加新版本+生效日期"留底)。
|
||||
|
||||
| 项 | 当前值 | 备注 |
|
||||
| --- | --- | --- |
|
||||
| 能源价 normal(高) | €0.133/kWh | 买侧,含 VAT,不含能源税 |
|
||||
| 能源价 dal(低) | €0.127/kWh | 买侧 |
|
||||
| 回送价 normal | (现与 dal 同)| 卖侧,分档但现同价;系统仍分开填 |
|
||||
| 回送价 dal | — | 卖侧 |
|
||||
| 能源税 energiebelasting | 待填(≈€0.1108)| 含 VAT,加到买价 |
|
||||
| ODE | 0 | 已并入能源税 |
|
||||
| 电网维护费 | 待填 | 按天(合同按天收)|
|
||||
| 供应商管理费 | ≈€0.329/天(≈€9.87/月)| 对应 Tibber 的 €5.99/月 |
|
||||
| heffingskorting | 待填 | 年度减免,汇总时扣 |
|
||||
| 回送阶梯罚金 terugleverkosten | **不做** | 按自然年累计、阶梯式;用户住不到年底算不准,M6 不实现 |
|
||||
|
||||
---
|
||||
|
||||
## 6. DSMR Reader → MQTT
|
||||
|
||||
### 6.1 取数方式
|
||||
- 用 **Telegram JSON**(单 topic **`dsmr/json`**,一条 = 一帧完整 telegram 的 JSON),**不用** split-topic(每字段一 topic、要按 id 拼)、**不用** raw(未解析 OBIS)。
|
||||
- DSMR Reader **每秒一条**。本项目按 **10 秒降采样**(仅秒数整 10 落盘)、**整帧存 JSON blob**(不做字段 allowlist)。
|
||||
|
||||
### 6.2 telegram JSON 字段(DSMR Reader 配置的 JSON mapping)
|
||||
```
|
||||
id, timestamp,
|
||||
electricity_delivered_1, electricity_delivered_2, # 进口累计 kWh(_1=dal 低谷, _2=normal 高峰)
|
||||
electricity_returned_1, electricity_returned_2, # 出口/回送累计 kWh
|
||||
electricity_currently_delivered, electricity_currently_returned, # 瞬时进/出口功率 kW
|
||||
phase_currently_delivered_l1/l2/l3, phase_currently_returned_l1/l2/l3, # 各相瞬时功率 kW
|
||||
phase_voltage_l1/l2/l3, # 各相电压 V
|
||||
phase_power_current_l1/l2/l3, # 各相电流 A
|
||||
extra_device_timestamp, extra_device_delivered # 燃气表(m³,每 5 分钟更新)
|
||||
```
|
||||
|
||||
### 6.3 实测样本(单相,`dsmr/json`)
|
||||
```json
|
||||
{
|
||||
"id": 200086230,
|
||||
"timestamp": "2026-06-23T12:16:48Z",
|
||||
"electricity_delivered_1": "20915.154",
|
||||
"electricity_returned_1": "2979.905",
|
||||
"electricity_delivered_2": "15212.090",
|
||||
"electricity_returned_2": "6786.406",
|
||||
"electricity_currently_delivered": "0.000",
|
||||
"electricity_currently_returned": "2.704",
|
||||
"phase_currently_delivered_l1": "0.000",
|
||||
"phase_currently_delivered_l2": null,
|
||||
"phase_currently_delivered_l3": null,
|
||||
"extra_device_timestamp": "2026-06-23T12:15:00Z",
|
||||
"extra_device_delivered": "6208.234",
|
||||
"phase_currently_returned_l1": "2.704",
|
||||
"phase_currently_returned_l2": null,
|
||||
"phase_currently_returned_l3": null,
|
||||
"phase_voltage_l1": "237.0",
|
||||
"phase_voltage_l2": null,
|
||||
"phase_voltage_l3": null
|
||||
}
|
||||
```
|
||||
|
||||
### 6.4 解析要点(实测确认)
|
||||
- **数值都是 JSON 字符串**(`"20915.154"`、`"0.000"`)→ 计费读取转 **Decimal**(累计寄存器算钱要精度)。
|
||||
- **缺测相位 = `null`**(不是缺 key)→ 视为缺测;单相只有 `*_l1`,三相后 `_l2/_l3` 由 null 变数字。
|
||||
- **进口总量** = `electricity_delivered_1 + _2`;**出口总量** = `electricity_returned_1 + _2`(双费率档对动态合同无意义,求和;对固定合同分档计价)。
|
||||
- **`timestamp` 为 UTC(Z)**;`id` 自增,做幂等去重。
|
||||
- 历史样本里电压 key 曾被错配成带前缀 `dsmr/reading/phase_voltage_l1`(DSMR Reader 的 JSON mapping 配置笔误,用户已改对)——解析仍按"键名容错"。
|
||||
- **整表寄存器与相数无关**:`delivered/returned_1/2` 是整表总量,三相只多了各相瞬时通道 → **计费逻辑相数无关**。
|
||||
- 燃气 `extra_device_delivered`(m³)一并存入 blob;燃气计费 M6 不做(数据先留,未来按 commodity 扩展)。
|
||||
|
||||
---
|
||||
|
||||
## 7. 来源 URL
|
||||
|
||||
- Tibber GraphQL reference / explorer:`https://developer.tibber.com/docs/reference#rootsubscription`、`https://developer.tibber.com/explorer`
|
||||
- Tibber NL 费用构成:`https://support.tibber.com/nl/articles/5605892-de-kosten-bij-tibber`
|
||||
- Tibber NL 净计量与回送:`https://support.tibber.com/nl/articles/4669873-salderen-en-terugleveren-bij-tibber`
|
||||
- DSMR Reader(HA 集成):`https://www.home-assistant.io/integrations/dsmr_reader/`
|
||||
|
||||
---
|
||||
|
||||
## 8. 待真实数据核对(合同生效后用真实 token / 账单)
|
||||
|
||||
1. **真实 token 复核**:跑 §1.4 的 15 分钟 curl,确认 NL 返回**真** 15 分钟价(非重复小时价)+ 币种 EUR。
|
||||
2. **卖价残差**:确认 `total` 里 purchase fee 是否被卖侧 sales fee 完全抵掉、回送 VAT 口径 → 调 `sell_adjust`(默认 0)。
|
||||
3. **双费率寄存器映射**:确认 `_1`=dal/`_2`=normal 没接反(差价小但要对)。
|
||||
4. **能源税年值**:按当年实际值与年用电档位核 `energy_tax`。
|
||||
5. **固定合同数值**:回送两档价、电网费、heffingskorting 待用户从账单填。
|
||||
</content>
|
||||
+168
-12
@@ -2,7 +2,7 @@
|
||||
|
||||
本文档记录 `home-automation` 在 `v1.0.3` 之后的下一阶段规划。这一阶段不是小修补,而是几次较大的结构性改动:单库化、前端重写、以及远期的移动端试水。
|
||||
|
||||
> 每个里程碑的**可执行原子任务**展开在 [`docs/design/`](./design/README.md):M1 [`m1-db-consolidation.md`](./design/m1-db-consolidation.md)、M2 [`m2-frontend-v2.md`](./design/m2-frontend-v2.md)、M3 [`m3-token-mobile.md`](./design/m3-token-mobile.md)。这些文档为 Orchestrator→Implementer→Reviewer 的多模型流水线设计。
|
||||
> 每个里程碑的**可执行原子任务**展开在 [`docs/design/`](./design/README.md):M1 [`m1-db-consolidation.md`](./design/m1-db-consolidation.md)、M2 [`m2-frontend-v2.md`](./design/m2-frontend-v2.md)、M3 [`m3-token-mobile.md`](./design/m3-token-mobile.md)、M4 [`m4-login-hardening.md`](./design/m4-login-hardening.md)、M5 [`m5-iot-energy.md`](./design/m5-iot-energy.md)、M6 [`m6-tibber-dynamic-energy.md`](./design/m6-tibber-dynamic-energy.md)、M7 [`m7-meter-epochs-archival.md`](./design/m7-meter-epochs-archival.md)。这些文档为 Orchestrator→Implementer→Reviewer 的多模型流水线设计。
|
||||
|
||||
## 当前基线(v1.0.3)
|
||||
|
||||
@@ -36,9 +36,13 @@
|
||||
| --- | --- | --- |
|
||||
| **M1** ✅ | 单库化地基 | 把三库合并成单一 `app.db`,清理散落数据层,删掉 Grafana |
|
||||
| **M2** ✅ | 前端 v2 | React SPA 取代 Jinja,承载 config + 可视化 + 记录增删改 |
|
||||
| **M4** ✅ | 登录加固 | 防爆破/指数退避 + CLI 逃生通道 + 可选 TOTP 二次验证(**先于 M5**) |
|
||||
| **M5** ✅ | IoT / 能耗采集 | 通用 Modbus 采集(YAML profile + JSON readings)+ MQTT/HA Discovery + 前端侧边栏 + Energy 视图 |
|
||||
| **M6** ✅ | 通用电价层 + DSMR 接入 + 实时电费计算 | 通用电价层(manual/tibber profile + 合同版本)+ DSMR 实时电表接入 + 每 15min 寄存器差×价计量电费(不可变快照)+ 日/月/年汇总 + 反哺 HA Energy + 前端合同/价格/费用视图 |
|
||||
| **M7** ✅ | 电表生命周期 / 换表归档 | 引入 Meter epoch,计费永不跨表算 delta,跨表/无表/异常 delta 一律降级,累计按当前表归零,追溯换表可重算,Meter CRUD API + 前端管理 UI |
|
||||
| **M3** | 开放与移动端(远期试水) | token 鉴权 + React Native 移动端 |
|
||||
|
||||
排序原则:**先清地基,再在干净结构上盖楼。** M2 的新 API 和 React 必须建立在合并后的单库之上,否则就是在准备推倒的旧数据层上盖新楼、之后回头返工。
|
||||
排序原则:**先清地基,再在干净结构上盖楼。** M2 的新 API 和 React 必须建立在合并后的单库之上;M4 是公网安全加固,在 M5 IoT 集成之前先堵住裸密码这个洞;M5 在安全基座就绪后再做 IoT 接入。
|
||||
|
||||
---
|
||||
|
||||
@@ -132,6 +136,127 @@
|
||||
|
||||
---
|
||||
|
||||
## M4 — 登录加固(✅ 已完成,排在 M5 之前)
|
||||
|
||||
### 目标
|
||||
|
||||
给暴露在公网的单 admin 登录做纵深防御,先于 M5 IoT 集成关闭暴力枚举和单因子风险。
|
||||
|
||||
### 范围
|
||||
|
||||
- **防爆破 / 指数退避**:失败登录按双键(IP + username)指数增长延迟,成功即清零。退避是延迟(429 + Retry-After),不是永久封号。全局开关 `AUTH_LOGIN_THROTTLE_ENABLED`(CONFIG_FIELDS);反代后需开 `AUTH_TRUST_FORWARDED_FOR`(`.env` 部署级)。
|
||||
- **CLI 逃生通道**:`python -m scripts.admin_cli` 直连本地 DB,无需 HTTP 服务、无需任何已存凭据;支持重置密码、解锁退避、关停/重发 TOTP、查看用户。
|
||||
- **可选 TOTP 二次验证**:admin 可自选启用;启用后两步登录(密码 + 6 位动态码或一次性恢复码);不启用维持纯密码。CLI `disable-totp` 是连恢复码都丢了时的最终逃生口。
|
||||
|
||||
> 详细设计与任务卡:[`docs/design/m4-login-hardening.md`](./design/m4-login-hardening.md)
|
||||
|
||||
---
|
||||
|
||||
## M5 — IoT 集成与能耗采集(✅ 已完成)
|
||||
|
||||
### 目标
|
||||
|
||||
给后端接入家庭 IoT 生态,建立通用 Modbus 设备采集链路(首个领域:能耗),接入 MQTT + Home Assistant Discovery,并重构前端为侧边导航并新增 Energy 视图。
|
||||
|
||||
### 范围
|
||||
|
||||
- **两层数据模型(协议与部署分离)**:YAML profile(随代码走,声明协议知识:寄存器/解码/key/unit/ha_component)+ `modbus_device`(部署层 DB 行:friendly_name/host/port/unit_id/profile/poll_interval/enabled)+ `modbus_reading`(通用遥测:device_id + recorded_at + JSON payload)。多设备可共享同一 profile。
|
||||
- **通用 Modbus-TCP 采集**:pymodbus,APScheduler 后台 job 轮询所有 enabled 设备;per-device `last_poll_at`/`last_poll_ok`;全局开关 `MODBUS_POLLING_ENABLED`(CONFIG_FIELDS)。首个 profile:SDM120 单相电表。
|
||||
- **手工 CLI 试读**:`scripts/modbus_cli`(`read`/`probe` 两个只读子命令),供受控手工验证,不进自动化。
|
||||
- **MQTT + HA Discovery**:paho-mqtt 长连接;通用 expose 框架(provider 动态产出可暴露实体目录,元数据从 YAML profile 派生);`exposed_entity_toggle` 表存逐 key 开关(默认不暴露);每设备 = 一个 HA device,各量 = sensor entity + online binary_sensor;`unique_id` 锚定于设备 `uuid`(稳定,不随改名变);配置变更可重连重发 discovery。
|
||||
- **前端侧边栏**:`AppShell` 侧边导航(Home / Records / Energy / Config + 主题切换 + 注销),移动端可折叠,当前路由高亮。
|
||||
- **Energy 视图**(`/energy`):设备 CRUD;最新读数卡片(标签/单位取自 profile metrics 端点);Recharts 走势图(时间范围 + limit)。
|
||||
- **Config 页 Accordion**:各大 section 折叠/展开;「Home Assistant Expose」面板勾选可暴露实体 + 连接状态 + 重新发布按钮。
|
||||
- **API**(`/api/modbus/*` + `/api/expose` + `/api/config/mqtt/test`):完整 CRUD + readings + metrics + test + expose 勾选 + 重发 discovery。
|
||||
- `pytest`/`ruff`/`export_openapi` + 前端 `lint/typecheck/test/build` 全绿,`openapi/` 已入库。
|
||||
|
||||
### 命名决策(已锁定)
|
||||
|
||||
存储/采集/API 全部使用通用 `modbus_*`(`/api/modbus/devices`),**不锁死"电表"**;面向用户的领域呈现叫 **Energy**。接入新 Modbus 设备型号只需新增 YAML profile,无需改表/改 API。
|
||||
|
||||
> 详细设计与任务卡:[`docs/design/m5-iot-energy.md`](./design/m5-iot-energy.md)
|
||||
|
||||
---
|
||||
|
||||
## M6 — 通用电价层 + DSMR 实时数据 + 实时买卖电费计算(✅ 已完成)
|
||||
|
||||
### 目标
|
||||
|
||||
在 M5 IoT 基建之上,把"电价合同"与"DSMR 实时电表数据"接起来,按 **15 分钟**周期算出**实际买电支出 / 卖电收入**,自己落库留底(不可变、可审计),并通过 MQTT + HA Discovery 反哺 Home Assistant 的 Energy 仪表盘。
|
||||
|
||||
### 关键能力
|
||||
|
||||
- **通用电价层(两层模型)**:仓库内 YAML profile 定结构(`manual` 固定/双费率、`tibber` 动态电价);`EnergyContract`(+ 版本)存 UI 可改的数值;price strategy 按 `kind` 出价。一次激活一个合同。改价 = 加新版本,旧版本保留(审计链)。
|
||||
- **DSMR 实时数据接入**:扩 `MqttManager` 订阅 `dsmr/json`(每秒一帧),整帧 JSON blob 按 10 秒降采样落库(电、气、各相全存;字段 allowlist 不做,blob 天然容纳未来字段)。
|
||||
- **实时买卖电费计算(两层)**:每 15 分钟用电表累计寄存器差值(`delivered_1/2`、`returned_1/2`)× 当时合同的买/卖价算计量电费(不可变、快照价)。买价:tibber = API `total`(含税全包),manual = `energy_buy_档 + energy_tax`;卖价:tibber = `total − energy_tax − sell_adjust`,manual = `sell_档`(无能源税)。双费率:`_1`=dal/低、`_2`=normal/高(NL 惯例)。日/月/年汇总再加固定费(按月→天)减 heffingskorting(按年→天)。不做净计量。卖价残差(sell_adjust)及能源税等当前数值在真实账单确认后钉死。
|
||||
- **反哺 Home Assistant**:`_energy_cost_provider` 发 `buy_price_now`、`sell_price_now`、`import_cost_total`(`total_increasing`)、`export_revenue_total`(`total_increasing`)—— 直接挂 HA Energy 仪表盘。
|
||||
- **Tibber 动态电价**:httpx GraphQL 客户端抓 `QUARTER_HOURLY` 15 分钟价(已 demo 验证 `total=energy+tax`);启动 + 每小时抓取今明两天价格、幂等 upsert `tibber_price`(hourly trigger,确保每日刷新且可补重试);仅当 active 合同 kind=tibber 且 token 存在时运行。
|
||||
- **前端**:Energy 视图新增三个 Tab——Contracts(合同管理,表单按 profile 结构渲染)、Prices(15 分钟价格曲线)、Costs(费用走势/明细/汇总卡片);Config 页 Tibber 测试三态入口。
|
||||
|
||||
### 命名分层(延续 M5)
|
||||
|
||||
存储/采集/计算层中性命名(`dsmr_*` / `tibber_price` / `energy_cost_period` / `energy_contract`);面向用户并入既有 **Energy** 视图。
|
||||
|
||||
### 新增 5 张表(单库 app 链,migration `20260623_11_energy_tables`)
|
||||
|
||||
| 表 | 关键列 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| `dsmr_reading` | `recorded_at`(idx)、`source_id`(unique)、`payload`(JSON) | 整帧 DSMR telegram blob,10 秒降采样 |
|
||||
| `energy_contract` | `name`、`kind`、`active`、`currency`、时间戳 | 合同头;一次一个 active |
|
||||
| `energy_contract_version` | `contract_id`(FK)、`effective_from`、`effective_to`(null)、`values`(JSON)、`created_at` | 版本/时段;改价加新版本、旧版本保留 |
|
||||
| `tibber_price` | `starts_at`(unique)、`resolution`、`energy/tax/total`、`level`、`currency`、`fetched_at` | 15 分钟价缓存,不可变 |
|
||||
| `energy_cost_period` | `period_start`(unique)、`d1/d2/r1/r2_kwh`、`import_cost/export_revenue/net_cost`、`currency`、`pricing`(JSON 快照)、`contract_version_id`(FK)、`degraded`、`computed_at` | 每 15 分钟计量电费,不可变(快照价+版本) |
|
||||
|
||||
### 不新增依赖
|
||||
|
||||
httpx / paho-mqtt / pyyaml / apscheduler 均为 M5 已有依赖,M6 复用,不新增任何 Python 包。
|
||||
|
||||
> 详细设计与任务卡:[`docs/design/m6-tibber-dynamic-energy.md`](./design/m6-tibber-dynamic-energy.md)
|
||||
|
||||
---
|
||||
|
||||
## M7 — 电表生命周期 / 换表归档(✅ 已完成)
|
||||
|
||||
### 目标
|
||||
|
||||
让计费系统正确处理**电表更换**这一必然事件:荷兰 2G 智能电表退网后,电网公司会把表换成 4G 表(同址换表);搬家继承新表也是同类。引入显式的 **Meter(电表)epoch** 概念,标记"某时刻起属于哪一块物理表",使引擎**永不跨表算 delta**,并让累计量按表归零、历史可追溯可查。
|
||||
|
||||
### 关键能力
|
||||
|
||||
- **Meter epoch 数据模型**:一条 `meter` 记录 = 一块物理电表的一段安装期 `[started_at, ended_at)`。换表 = 关闭旧 active 表(`ended_at=T`)+ 新建 `started_at=T` 的表。每 `commodity` 至多一个 active 表;`commodity` 字段预留 `gas`/`heating`。
|
||||
- **计费永不跨表**:`compute_period` 先查 t0/t1 两端的 `meter_at`;无表覆盖或跨表边界 → **降级**(成本置 0),绝不产负成本或巨额假成本。
|
||||
- **delta 护栏**:任一寄存器 delta `< 0` 或 `> _MAX_DELTA_KWH`(100 kWh/15min,远超住宅用量)→ 降级。兜底表重置 / DSMR 回绕 / 数据毛刺,与换表无关的异常也一并防住。
|
||||
- **累计按当前表归零(D2)**:`expose.py` 累计 `import_cost_total`/`export_revenue_total` 锚点 = 当前 active electricity meter 的 `started_at`;换表后累计从零起新序列,不维护跨表偏移。
|
||||
- **追溯换表可重算**:`started_at` 可在过去;PATCH 修正日期后,触发受影响窗口 `recompute_range` 重判跨表周期归属。
|
||||
- **Meter CRUD API + 前端管理 UI**:`GET/POST/PATCH /api/energy/meters`(+ 追溯 recompute);Energy 页新增 Meters tab,展示电表时间线,支持"换表/继承"表单与编辑。
|
||||
|
||||
### 新增表与列(单库 app 链,migration `20260625_13_meter_table`)
|
||||
|
||||
| 表/列 | 关键设计 |
|
||||
| --- | --- |
|
||||
| `meter` | `id`、`label`、`commodity`(默认 `electricity`)、`started_at`、`ended_at`(null=active)、`reason`(`initial`/`meter_swap`/`home_move`/`other`)、`note`、`created_at` |
|
||||
| `energy_cost_period.meter_id` | nullable FK → `meter.id`;记录每周期归属,便于审计/按表查询/归档 |
|
||||
|
||||
迁移含回填:有历史数据时自动创建一条 `reason="initial"` 的初始表,并回填现有 `energy_cost_period.meter_id`;幂等 + 对账(非降级周期回填后 `meter_id IS NULL` 数必须为 0)。
|
||||
|
||||
### 已锁定决策摘要
|
||||
|
||||
- **D1**:Meter 不绑定 home,`label` 编址 + `commodity` 区分品类。
|
||||
- **D2**:换表后累计量归零(锚当前表起点),不维护跨表偏移。
|
||||
- **D3**:合同与电表是独立时间线,合同不引用电表;同址换表时合同自动沿用。
|
||||
- **D4**:`started_at` 可在过去;有效计费起点 = `max(started_at, 数据起点)`。
|
||||
- **D5**:跨表边界周期判 degraded(换表常伴随长时间无数据,可接受)。
|
||||
- **D6**:负/异常大 delta → degraded,通用兜底。
|
||||
- **D7**:gas/heating 计费不在本里程碑;`commodity` 字段为未来扩展预留。
|
||||
|
||||
**已知行为(可接受)**:累计 `_total` 是 `state_class: total`,换表归零时 HA 长期统计可能在换表那一刻记一次性负 blip。已与用户确认先这样、观察后按需处理(`last_reset` 信号见设计文档 §9 留痕,本里程碑不做)。
|
||||
|
||||
> 详细设计与任务卡:[`docs/design/m7-meter-epochs-archival.md`](./design/m7-meter-epochs-archival.md)
|
||||
>
|
||||
> 模块概念说明:[`docs/meter-epochs.md`](./meter-epochs.md)
|
||||
|
||||
---
|
||||
|
||||
## M3 — 开放与移动端(远期试水)
|
||||
|
||||
### 目标
|
||||
@@ -149,22 +274,53 @@
|
||||
- 移动端是这一阶段最远期、最不确定的部分。
|
||||
- token 主要是移动端的前置条件;Web 端 React 用现有 session cookie 即可,不需要为它提前引入 token。
|
||||
|
||||
## Future Ideas(暂不排期,想到先记下)
|
||||
## 下一阶段:已确定要做(尚未拆解为任务卡)
|
||||
|
||||
> 这里收集**还没排进里程碑**的想法。不是承诺、也没有先后顺序;想做时再从这里捞出来细化成 `docs/design/` 的任务卡。**明确不开 M2.5**——下列条目一律先躺在 Future Ideas,之后再说。
|
||||
> 这些是 M5 之后**已经定下来要做**的方向——区别于下面的 Future Ideas(仅备忘、未必做)。这里只记到 roadmap 粒度:确定**做什么、为什么**;具体排期、依赖与原子任务,等动手时再展开成 `docs/design/` 的任务卡。**先后顺序未定**,具体排期等动手时再定。
|
||||
|
||||
### TOTP 二次验证(Dashboard 加固)
|
||||
### 1. 前端优化
|
||||
|
||||
**动机**:M2 之后多了一个 Web Dashboard。它虽有单 admin 密码保护,但**大概率会暴露在公网**上,只靠密码这一层不够。给登录再叠一层 **TOTP(基于时间的一次性密码,RFC 6238)** 作为第二因子,做纵深防御。
|
||||
**动机**:M2 的 React SPA 先把功能跑通,性能 / 体验层面的打磨还没做。这一项**确定要做,但具体优化什么还没定**。
|
||||
|
||||
**范围(待定)**:方向先留空,想清楚再细化。可能的候选(仅占位、非承诺):打包体积与代码分割(M2/M5 构建已提示存在 > 500 kB 的单 chunk)、首屏加载、热力图 / 地图的渲染性能、移动端适配、可访问性等。等确定具体目标后再拆任务卡。
|
||||
|
||||
### 2. 设置页生成 Long-lived Token(供 API 调用)
|
||||
|
||||
**动机**:浏览器端走 session cookie 即可,但**脚本 / 设备 / 外部程序调用 API** 需要一种长期有效、可随身携带的凭据。在设置页加一组功能,由 admin **手动签发 long-lived token**,之后用它来调 API。
|
||||
|
||||
**本次明确的首要目标 = 给现在裸奔的 ingestion 端点上鉴权**(2026-06-27 与用户确认):
|
||||
|
||||
- `POST /location/record`(`app/api/routes/location.py:18`)——位置记录上报。**目前无任何鉴权**。当前数据经 Home Assistant 转发进来,上 token 后 **HA 侧需携带该 token**;也可由其他客户端直接上报。
|
||||
- `POST /poo/record`(`app/api/routes/poo.py:21`)+ `GET /poo/latest`(`poo.py:57`)——小狗排便记录上报 / 最新查询。**目前无任何鉴权**。
|
||||
- 这些是设备 / 脚本(非浏览器)端点,session cookie 不适用,正是 long-lived token 的用武之地。(浏览器 CRUD `/api/data/*` 已由 session 保护,不在此列。)
|
||||
|
||||
**范围(粗略,待细化)**:
|
||||
|
||||
- 在现有单 admin(Argon2 + server-side session)登录之上,叠加 TOTP 第二步:密码校验通过后再验 6 位动态码,通过才发 session cookie。
|
||||
- 首次启用时生成 TOTP secret,给出可导入 Authenticator 的二维码 / 可手输密钥;同时生成一组一次性**恢复码(recovery codes)**。
|
||||
- 设置页新增「API Token」区:生成 / 命名 / 吊销 long-lived token;明文只在**生成时展示一次**,此后只存哈希。
|
||||
- 后端支持用该 token 鉴权访问 API(与现有 session cookie 并存,互不影响);给上述 ingestion 端点加 token 鉴权依赖。
|
||||
- 与 [M3](#m3--开放与移动端远期试水) 的 token 主题相关,但**这条是 Web 设置页手动签发的 PAT 风格**,不依赖移动端 OAuth 流程;两者实现时可复用同一套 token 存储 / 校验。
|
||||
- 与下面第 3 条「Session 滑动续期」同属 Authentication 主题(一个是设备/脚本的长期凭据,一个是浏览器短会话体验),实现时鉴权层可一并梳理。
|
||||
|
||||
**运维 / 命令行要求(关键,实现时必须满足)**:
|
||||
### 3. Session 滑动自动续期(Authentication)
|
||||
|
||||
1. **忘记密码**:不需要任何 Web 端“找回密码”流程——直接在命令行里重置 admin 密码即可(沿用现有 CLI 思路)。
|
||||
2. **TOTP 重置 / 恢复**:必须提供**命令行重置入口**。要覆盖最坏情况——**连恢复码(restore key)都丢了**,也能纯靠 CLI 把 TOTP 关掉 / 重新发放新的 secret,从而恢复登录。即:**CLI 是不依赖任何已存恢复凭据的最终逃生通道**,不能出现“密钥丢了就彻底锁死”的死角。
|
||||
**动机**(2026-06-27 与用户确认):当前 session 是**绝对过期**——登录即定死、活动不续期,满 TTL 必须重新登录,体验割裂。希望改成**滑动续期(sliding / rolling)**:只要用户还在活动就自动延长,提供"在用就不掉线"的体验。
|
||||
|
||||
**先不做**:本条仅记入 Future Ideas,不进 M2.5、不排期;之后再细化为 design 任务卡。
|
||||
**现状(实现起点,便于快速拾起)**:
|
||||
|
||||
- TTL 默认 **12 小时**:`auth_session_ttl_hours`(`app/config.py:38`;配置页 `app/services/config_page.py:45` 可运行时改)。
|
||||
- 登录时**一次性写死**:`create_session` 设 `expires_at = now + ttl`(`app/services/auth.py:94`)+ cookie `max_age = ttl`(`app/api/routes/api/session.py:153`)。
|
||||
- 每请求**只读校验、从不延长**:`get_authenticated_session`(`app/services/auth.py:103`)只判断 `expires_at <= now`,过期时仅顺手标 `revoked`。`set_cookie` 只在登录路由调用一次,**无 per-request 中间件**。→ 所以是绝对过期,不是滑动。
|
||||
|
||||
**设计要点(待写设计文档时展开)**:
|
||||
|
||||
- 校验通过时 bump `expires_at = now + ttl` 并**重发 cookie**(滑动窗口)。
|
||||
- **写节流**:不要每个请求都写 DB——仅当剩余寿命已过半(或距上次续期 > N 分钟)才续期,避免高频写放大。
|
||||
- **绝对寿命硬顶**:除滑动 TTL 外再设 `created_at + max_lifetime` 上限,防止"永不过期"的会话(安全考量)。
|
||||
- 新增配置项:滑动 TTL、绝对寿命上限、续期节流阈值。
|
||||
- 注意:改动只对**新逻辑生效**,已存在 session 的 `expires_at` 行为按新校验路径走即可;上线前过校验闸门。
|
||||
|
||||
## Future Ideas(暂不排期,想到先记下)
|
||||
|
||||
> 这里收集**还没排进里程碑、也还没决定要不要做**的想法。不是承诺、也没有先后顺序;想做时再从这里捞出来——先升进上面的「下一阶段」,再细化成 `docs/design/` 的任务卡。
|
||||
|
||||
_(暂无条目。)_
|
||||
|
||||
Generated
+396
-3
@@ -17,11 +17,13 @@
|
||||
"leaflet.heat": "^0.2.0",
|
||||
"leaflet.markercluster": "^1.5.3",
|
||||
"openapi-fetch": "^0.17.0",
|
||||
"qrcode.react": "^4.2.0",
|
||||
"react": "^18.3.1",
|
||||
"react-dom": "^18.3.1",
|
||||
"react-feather": "^2.0.10",
|
||||
"react-leaflet": "^4.2.1",
|
||||
"react-router-dom": "^6.30.4"
|
||||
"react-router-dom": "^6.30.4",
|
||||
"recharts": "^3.8.1"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@eslint/js": "^9.39.4",
|
||||
@@ -1445,6 +1447,42 @@
|
||||
"node": ">=10"
|
||||
}
|
||||
},
|
||||
"node_modules/@reduxjs/toolkit": {
|
||||
"version": "2.12.0",
|
||||
"resolved": "https://registry.npmjs.org/@reduxjs/toolkit/-/toolkit-2.12.0.tgz",
|
||||
"integrity": "sha512-KiT+RzZbp6mQET+Mg+h2c97+9j1sNflUxQkIHI7Yuzf6Peu+OYpmkn6nbHWmLLWj+1ZODUJFwGZ7gx3L9R9EOw==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@standard-schema/spec": "^1.0.0",
|
||||
"@standard-schema/utils": "^0.3.0",
|
||||
"immer": "^11.0.0",
|
||||
"redux": "^5.0.1",
|
||||
"redux-thunk": "^3.1.0",
|
||||
"reselect": "^5.1.0"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"react": "^16.9.0 || ^17.0.0 || ^18 || ^19",
|
||||
"react-redux": "^7.2.1 || ^8.1.3 || ^9.0.0"
|
||||
},
|
||||
"peerDependenciesMeta": {
|
||||
"react": {
|
||||
"optional": true
|
||||
},
|
||||
"react-redux": {
|
||||
"optional": true
|
||||
}
|
||||
}
|
||||
},
|
||||
"node_modules/@reduxjs/toolkit/node_modules/immer": {
|
||||
"version": "11.1.8",
|
||||
"resolved": "https://registry.npmjs.org/immer/-/immer-11.1.8.tgz",
|
||||
"integrity": "sha512-/tbkHMW7y10Lx6i1crLjD4/OhNkRG+Fo7byZHtah0547nIeXYcpIXaUh0IAQY6gO5459qpGGYapcEOHtFXkIuA==",
|
||||
"license": "MIT",
|
||||
"funding": {
|
||||
"type": "opencollective",
|
||||
"url": "https://opencollective.com/immer"
|
||||
}
|
||||
},
|
||||
"node_modules/@remix-run/router": {
|
||||
"version": "1.23.3",
|
||||
"resolved": "https://registry.npmjs.org/@remix-run/router/-/router-1.23.3.tgz",
|
||||
@@ -1854,7 +1892,12 @@
|
||||
"version": "1.1.0",
|
||||
"resolved": "https://registry.npmjs.org/@standard-schema/spec/-/spec-1.1.0.tgz",
|
||||
"integrity": "sha512-l2aFy5jALhniG5HgqrD6jXLi/rUWrKvqN/qJx6yoJsgKhblVd+iqqU4RCXavm/jPityDo5TCvKMnpjKnOriy0w==",
|
||||
"dev": true,
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/@standard-schema/utils": {
|
||||
"version": "0.3.0",
|
||||
"resolved": "https://registry.npmjs.org/@standard-schema/utils/-/utils-0.3.0.tgz",
|
||||
"integrity": "sha512-e7Mew686owMaPJVNNLs55PUvgz371nKgwsc4vxE49zsODpJEnxgxRo2y/OKrqueavXgZNMDVj3DdHFlaSAeU8g==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/@tanstack/query-core": {
|
||||
@@ -2057,6 +2100,69 @@
|
||||
"assertion-error": "^2.0.1"
|
||||
}
|
||||
},
|
||||
"node_modules/@types/d3-array": {
|
||||
"version": "3.2.2",
|
||||
"resolved": "https://registry.npmjs.org/@types/d3-array/-/d3-array-3.2.2.tgz",
|
||||
"integrity": "sha512-hOLWVbm7uRza0BYXpIIW5pxfrKe0W+D5lrFiAEYR+pb6w3N2SwSMaJbXdUfSEv+dT4MfHBLtn5js0LAWaO6otw==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/@types/d3-color": {
|
||||
"version": "3.1.3",
|
||||
"resolved": "https://registry.npmjs.org/@types/d3-color/-/d3-color-3.1.3.tgz",
|
||||
"integrity": "sha512-iO90scth9WAbmgv7ogoq57O9YpKmFBbmoEoCHDB2xMBY0+/KVrqAaCDyCE16dUspeOvIxFFRI+0sEtqDqy2b4A==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/@types/d3-ease": {
|
||||
"version": "3.0.2",
|
||||
"resolved": "https://registry.npmjs.org/@types/d3-ease/-/d3-ease-3.0.2.tgz",
|
||||
"integrity": "sha512-NcV1JjO5oDzoK26oMzbILE6HW7uVXOHLQvHshBUW4UMdZGfiY6v5BeQwh9a9tCzv+CeefZQHJt5SRgK154RtiA==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/@types/d3-interpolate": {
|
||||
"version": "3.0.4",
|
||||
"resolved": "https://registry.npmjs.org/@types/d3-interpolate/-/d3-interpolate-3.0.4.tgz",
|
||||
"integrity": "sha512-mgLPETlrpVV1YRJIglr4Ez47g7Yxjl1lj7YKsiMCb27VJH9W8NVM6Bb9d8kkpG/uAQS5AmbA48q2IAolKKo1MA==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@types/d3-color": "*"
|
||||
}
|
||||
},
|
||||
"node_modules/@types/d3-path": {
|
||||
"version": "3.1.1",
|
||||
"resolved": "https://registry.npmjs.org/@types/d3-path/-/d3-path-3.1.1.tgz",
|
||||
"integrity": "sha512-VMZBYyQvbGmWyWVea0EHs/BwLgxc+MKi1zLDCONksozI4YJMcTt8ZEuIR4Sb1MMTE8MMW49v0IwI5+b7RmfWlg==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/@types/d3-scale": {
|
||||
"version": "4.0.9",
|
||||
"resolved": "https://registry.npmjs.org/@types/d3-scale/-/d3-scale-4.0.9.tgz",
|
||||
"integrity": "sha512-dLmtwB8zkAeO/juAMfnV+sItKjlsw2lKdZVVy6LRr0cBmegxSABiLEpGVmSJJ8O08i4+sGR6qQtb6WtuwJdvVw==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@types/d3-time": "*"
|
||||
}
|
||||
},
|
||||
"node_modules/@types/d3-shape": {
|
||||
"version": "3.1.8",
|
||||
"resolved": "https://registry.npmjs.org/@types/d3-shape/-/d3-shape-3.1.8.tgz",
|
||||
"integrity": "sha512-lae0iWfcDeR7qt7rA88BNiqdvPS5pFVPpo5OfjElwNaT2yyekbM0C9vK+yqBqEmHr6lDkRnYNoTBYlAgJa7a4w==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@types/d3-path": "*"
|
||||
}
|
||||
},
|
||||
"node_modules/@types/d3-time": {
|
||||
"version": "3.0.4",
|
||||
"resolved": "https://registry.npmjs.org/@types/d3-time/-/d3-time-3.0.4.tgz",
|
||||
"integrity": "sha512-yuzZug1nkAAaBlBBikKZTgzCeA+k1uy4ZFwWANOfKw5z5LRhV0gNA7gNkKm7HoK+HRN0wX3EkxGk0fpbWhmB7g==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/@types/d3-timer": {
|
||||
"version": "3.0.2",
|
||||
"resolved": "https://registry.npmjs.org/@types/d3-timer/-/d3-timer-3.0.2.tgz",
|
||||
"integrity": "sha512-Ps3T8E8dZDam6fUyNiMkekK3XUsaUEik+idO9/YjPtfj2qruF8tFBXS7XhtE4iIXBLxhmLjP3SXpLhVf21I9Lw==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/@types/deep-eql": {
|
||||
"version": "4.0.2",
|
||||
"resolved": "https://registry.npmjs.org/@types/deep-eql/-/deep-eql-4.0.2.tgz",
|
||||
@@ -2130,6 +2236,12 @@
|
||||
"@types/react": "^18.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/@types/use-sync-external-store": {
|
||||
"version": "0.0.6",
|
||||
"resolved": "https://registry.npmjs.org/@types/use-sync-external-store/-/use-sync-external-store-0.0.6.tgz",
|
||||
"integrity": "sha512-zFDAD+tlpf2r4asuHEj0XH6pY6i0g5NeAHPn+15wk3BV6JA69eERFXC1gyGThDkVa1zCyKr5jox1+2LbV/AMLg==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/@typescript-eslint/eslint-plugin": {
|
||||
"version": "8.61.0",
|
||||
"resolved": "https://registry.npmjs.org/@typescript-eslint/eslint-plugin/-/eslint-plugin-8.61.0.tgz",
|
||||
@@ -3119,6 +3231,127 @@
|
||||
"devOptional": true,
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/d3-array": {
|
||||
"version": "3.2.4",
|
||||
"resolved": "https://registry.npmjs.org/d3-array/-/d3-array-3.2.4.tgz",
|
||||
"integrity": "sha512-tdQAmyA18i4J7wprpYq8ClcxZy3SC31QMeByyCFyRt7BVHdREQZ5lpzoe5mFEYZUWe+oq8HBvk9JjpibyEV4Jg==",
|
||||
"license": "ISC",
|
||||
"dependencies": {
|
||||
"internmap": "1 - 2"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=12"
|
||||
}
|
||||
},
|
||||
"node_modules/d3-color": {
|
||||
"version": "3.1.0",
|
||||
"resolved": "https://registry.npmjs.org/d3-color/-/d3-color-3.1.0.tgz",
|
||||
"integrity": "sha512-zg/chbXyeBtMQ1LbD/WSoW2DpC3I0mpmPdW+ynRTj/x2DAWYrIY7qeZIHidozwV24m4iavr15lNwIwLxRmOxhA==",
|
||||
"license": "ISC",
|
||||
"engines": {
|
||||
"node": ">=12"
|
||||
}
|
||||
},
|
||||
"node_modules/d3-ease": {
|
||||
"version": "3.0.1",
|
||||
"resolved": "https://registry.npmjs.org/d3-ease/-/d3-ease-3.0.1.tgz",
|
||||
"integrity": "sha512-wR/XK3D3XcLIZwpbvQwQ5fK+8Ykds1ip7A2Txe0yxncXSdq1L9skcG7blcedkOX+ZcgxGAmLX1FrRGbADwzi0w==",
|
||||
"license": "BSD-3-Clause",
|
||||
"engines": {
|
||||
"node": ">=12"
|
||||
}
|
||||
},
|
||||
"node_modules/d3-format": {
|
||||
"version": "3.1.2",
|
||||
"resolved": "https://registry.npmjs.org/d3-format/-/d3-format-3.1.2.tgz",
|
||||
"integrity": "sha512-AJDdYOdnyRDV5b6ArilzCPPwc1ejkHcoyFarqlPqT7zRYjhavcT3uSrqcMvsgh2CgoPbK3RCwyHaVyxYcP2Arg==",
|
||||
"license": "ISC",
|
||||
"engines": {
|
||||
"node": ">=12"
|
||||
}
|
||||
},
|
||||
"node_modules/d3-interpolate": {
|
||||
"version": "3.0.1",
|
||||
"resolved": "https://registry.npmjs.org/d3-interpolate/-/d3-interpolate-3.0.1.tgz",
|
||||
"integrity": "sha512-3bYs1rOD33uo8aqJfKP3JWPAibgw8Zm2+L9vBKEHJ2Rg+viTR7o5Mmv5mZcieN+FRYaAOWX5SJATX6k1PWz72g==",
|
||||
"license": "ISC",
|
||||
"dependencies": {
|
||||
"d3-color": "1 - 3"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=12"
|
||||
}
|
||||
},
|
||||
"node_modules/d3-path": {
|
||||
"version": "3.1.0",
|
||||
"resolved": "https://registry.npmjs.org/d3-path/-/d3-path-3.1.0.tgz",
|
||||
"integrity": "sha512-p3KP5HCf/bvjBSSKuXid6Zqijx7wIfNW+J/maPs+iwR35at5JCbLUT0LzF1cnjbCHWhqzQTIN2Jpe8pRebIEFQ==",
|
||||
"license": "ISC",
|
||||
"engines": {
|
||||
"node": ">=12"
|
||||
}
|
||||
},
|
||||
"node_modules/d3-scale": {
|
||||
"version": "4.0.2",
|
||||
"resolved": "https://registry.npmjs.org/d3-scale/-/d3-scale-4.0.2.tgz",
|
||||
"integrity": "sha512-GZW464g1SH7ag3Y7hXjf8RoUuAFIqklOAq3MRl4OaWabTFJY9PN/E1YklhXLh+OQ3fM9yS2nOkCoS+WLZ6kvxQ==",
|
||||
"license": "ISC",
|
||||
"dependencies": {
|
||||
"d3-array": "2.10.0 - 3",
|
||||
"d3-format": "1 - 3",
|
||||
"d3-interpolate": "1.2.0 - 3",
|
||||
"d3-time": "2.1.1 - 3",
|
||||
"d3-time-format": "2 - 4"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=12"
|
||||
}
|
||||
},
|
||||
"node_modules/d3-shape": {
|
||||
"version": "3.2.0",
|
||||
"resolved": "https://registry.npmjs.org/d3-shape/-/d3-shape-3.2.0.tgz",
|
||||
"integrity": "sha512-SaLBuwGm3MOViRq2ABk3eLoxwZELpH6zhl3FbAoJ7Vm1gofKx6El1Ib5z23NUEhF9AsGl7y+dzLe5Cw2AArGTA==",
|
||||
"license": "ISC",
|
||||
"dependencies": {
|
||||
"d3-path": "^3.1.0"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=12"
|
||||
}
|
||||
},
|
||||
"node_modules/d3-time": {
|
||||
"version": "3.1.0",
|
||||
"resolved": "https://registry.npmjs.org/d3-time/-/d3-time-3.1.0.tgz",
|
||||
"integrity": "sha512-VqKjzBLejbSMT4IgbmVgDjpkYrNWUYJnbCGo874u7MMKIWsILRX+OpX/gTk8MqjpT1A/c6HY2dCA77ZN0lkQ2Q==",
|
||||
"license": "ISC",
|
||||
"dependencies": {
|
||||
"d3-array": "2 - 3"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=12"
|
||||
}
|
||||
},
|
||||
"node_modules/d3-time-format": {
|
||||
"version": "4.1.0",
|
||||
"resolved": "https://registry.npmjs.org/d3-time-format/-/d3-time-format-4.1.0.tgz",
|
||||
"integrity": "sha512-dJxPBlzC7NugB2PDLwo9Q8JiTR3M3e4/XANkreKSUxF8vvXKqm1Yfq4Q5dl8budlunRVlUUaDUgFt7eA8D6NLg==",
|
||||
"license": "ISC",
|
||||
"dependencies": {
|
||||
"d3-time": "1 - 3"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=12"
|
||||
}
|
||||
},
|
||||
"node_modules/d3-timer": {
|
||||
"version": "3.0.1",
|
||||
"resolved": "https://registry.npmjs.org/d3-timer/-/d3-timer-3.0.1.tgz",
|
||||
"integrity": "sha512-ndfJ/JxxMd3nw31uyKoY2naivF+r29V+Lc0svZxe1JvvIRmi8hUsrMvdOwgS1o6uBHmiz91geQ0ylPP0aj1VUA==",
|
||||
"license": "ISC",
|
||||
"engines": {
|
||||
"node": ">=12"
|
||||
}
|
||||
},
|
||||
"node_modules/data-urls": {
|
||||
"version": "7.0.0",
|
||||
"resolved": "https://registry.npmjs.org/data-urls/-/data-urls-7.0.0.tgz",
|
||||
@@ -3212,6 +3445,12 @@
|
||||
"dev": true,
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/decimal.js-light": {
|
||||
"version": "2.5.1",
|
||||
"resolved": "https://registry.npmjs.org/decimal.js-light/-/decimal.js-light-2.5.1.tgz",
|
||||
"integrity": "sha512-qIMFpTMZmny+MMIitAB6D7iVPEorVw6YQRWkvarTkT4tBeSLLiHzcwj6q0MmYSFCiVpiqPJTJEYIrpcPzVEIvg==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/deep-equal": {
|
||||
"version": "2.2.3",
|
||||
"resolved": "https://registry.npmjs.org/deep-equal/-/deep-equal-2.2.3.tgz",
|
||||
@@ -3564,6 +3803,16 @@
|
||||
"url": "https://github.com/sponsors/ljharb"
|
||||
}
|
||||
},
|
||||
"node_modules/es-toolkit": {
|
||||
"version": "1.48.1",
|
||||
"resolved": "https://registry.npmjs.org/es-toolkit/-/es-toolkit-1.48.1.tgz",
|
||||
"integrity": "sha512-wfnXlwd5I75eXRtdD2vuEs50xHHESECDsGD7yiQnfFVNoa5522NwXEbmgo98LfiukSQHs+mBM7/YG3qKJB9/mQ==",
|
||||
"license": "MIT",
|
||||
"workspaces": [
|
||||
"docs",
|
||||
"benchmarks"
|
||||
]
|
||||
},
|
||||
"node_modules/esbuild": {
|
||||
"version": "0.25.12",
|
||||
"resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.25.12.tgz",
|
||||
@@ -3846,6 +4095,12 @@
|
||||
"node": ">=0.10.0"
|
||||
}
|
||||
},
|
||||
"node_modules/eventemitter3": {
|
||||
"version": "5.0.4",
|
||||
"resolved": "https://registry.npmjs.org/eventemitter3/-/eventemitter3-5.0.4.tgz",
|
||||
"integrity": "sha512-mlsTRyGaPBjPedk6Bvw+aqbsXDtoAyAzm5MO7JgU+yVRyMQ5O8bD4Kcci7BS85f93veegeCPkL8R4GLClnjLFw==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/expect-type": {
|
||||
"version": "1.3.0",
|
||||
"resolved": "https://registry.npmjs.org/expect-type/-/expect-type-1.3.0.tgz",
|
||||
@@ -4311,6 +4566,16 @@
|
||||
"node": ">= 4"
|
||||
}
|
||||
},
|
||||
"node_modules/immer": {
|
||||
"version": "10.2.0",
|
||||
"resolved": "https://registry.npmjs.org/immer/-/immer-10.2.0.tgz",
|
||||
"integrity": "sha512-d/+XTN3zfODyjr89gM3mPq1WNX2B8pYsu7eORitdwyA2sBubnTl3laYlBk4sXY5FUa5qTZGBDPJICVbvqzjlbw==",
|
||||
"license": "MIT",
|
||||
"funding": {
|
||||
"type": "opencollective",
|
||||
"url": "https://opencollective.com/immer"
|
||||
}
|
||||
},
|
||||
"node_modules/import-fresh": {
|
||||
"version": "3.3.1",
|
||||
"resolved": "https://registry.npmjs.org/import-fresh/-/import-fresh-3.3.1.tgz",
|
||||
@@ -4376,6 +4641,15 @@
|
||||
"node": ">= 0.4"
|
||||
}
|
||||
},
|
||||
"node_modules/internmap": {
|
||||
"version": "2.0.3",
|
||||
"resolved": "https://registry.npmjs.org/internmap/-/internmap-2.0.3.tgz",
|
||||
"integrity": "sha512-5Hh7Y1wQbvY5ooGgPbDaL5iYLAPzMTUrjMulskHLH6wnv/A+1q5rgEaiuqEjB+oxGXIVZs1FF+R/KPN3ZSQYYg==",
|
||||
"license": "ISC",
|
||||
"engines": {
|
||||
"node": ">=12"
|
||||
}
|
||||
},
|
||||
"node_modules/is-arguments": {
|
||||
"version": "1.2.0",
|
||||
"resolved": "https://registry.npmjs.org/is-arguments/-/is-arguments-1.2.0.tgz",
|
||||
@@ -5661,6 +5935,15 @@
|
||||
"node": ">=6"
|
||||
}
|
||||
},
|
||||
"node_modules/qrcode.react": {
|
||||
"version": "4.2.0",
|
||||
"resolved": "https://registry.npmjs.org/qrcode.react/-/qrcode.react-4.2.0.tgz",
|
||||
"integrity": "sha512-QpgqWi8rD9DsS9EP3z7BT+5lY5SFhsqGjpgW5DY/i3mK4M9DTBNz3ErMi8BWYEfI3L0d8GIbGmcdFAS1uIRGjA==",
|
||||
"license": "ISC",
|
||||
"peerDependencies": {
|
||||
"react": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/react": {
|
||||
"version": "18.3.1",
|
||||
"resolved": "https://registry.npmjs.org/react/-/react-18.3.1.tgz",
|
||||
@@ -5702,7 +5985,6 @@
|
||||
"version": "17.0.2",
|
||||
"resolved": "https://registry.npmjs.org/react-is/-/react-is-17.0.2.tgz",
|
||||
"integrity": "sha512-w2GsyukL62IJnlaff/nRegPQR94C/XXamvMWmSHRJ4y7Ts/4ocGRmTHvOs8PSE6pB3dWOrD/nueuU5sduBsQ4w==",
|
||||
"dev": true,
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/react-leaflet": {
|
||||
@@ -5729,6 +6011,29 @@
|
||||
"react-dom": "^0.14 || ^15.0.0 || ^16.0.0 || ^17.0.0 || ^18.0.0 || ^19.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/react-redux": {
|
||||
"version": "9.3.0",
|
||||
"resolved": "https://registry.npmjs.org/react-redux/-/react-redux-9.3.0.tgz",
|
||||
"integrity": "sha512-KQopgqFo/p/fgmAs5qz6p5RWaNAzq40WAu7fJIXnQpYxFPbJYtsJPWvGeF2rOBaY/kEuV77AVsX8TsQzKm+A/g==",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@types/use-sync-external-store": "^0.0.6",
|
||||
"use-sync-external-store": "^1.4.0"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"@types/react": "^18.2.25 || ^19",
|
||||
"react": "^18.0 || ^19",
|
||||
"redux": "^5.0.0"
|
||||
},
|
||||
"peerDependenciesMeta": {
|
||||
"@types/react": {
|
||||
"optional": true
|
||||
},
|
||||
"redux": {
|
||||
"optional": true
|
||||
}
|
||||
}
|
||||
},
|
||||
"node_modules/react-refresh": {
|
||||
"version": "0.17.0",
|
||||
"resolved": "https://registry.npmjs.org/react-refresh/-/react-refresh-0.17.0.tgz",
|
||||
@@ -5857,6 +6162,36 @@
|
||||
"react": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/recharts": {
|
||||
"version": "3.8.1",
|
||||
"resolved": "https://registry.npmjs.org/recharts/-/recharts-3.8.1.tgz",
|
||||
"integrity": "sha512-mwzmO1s9sFL0TduUpwndxCUNoXsBw3u3E/0+A+cLcrSfQitSG62L32N69GhqUrrT5qKcAE3pCGVINC6pqkBBQg==",
|
||||
"license": "MIT",
|
||||
"workspaces": [
|
||||
"www"
|
||||
],
|
||||
"dependencies": {
|
||||
"@reduxjs/toolkit": "^1.9.0 || 2.x.x",
|
||||
"clsx": "^2.1.1",
|
||||
"decimal.js-light": "^2.5.1",
|
||||
"es-toolkit": "^1.39.3",
|
||||
"eventemitter3": "^5.0.1",
|
||||
"immer": "^10.1.1",
|
||||
"react-redux": "8.x.x || 9.x.x",
|
||||
"reselect": "5.1.1",
|
||||
"tiny-invariant": "^1.3.3",
|
||||
"use-sync-external-store": "^1.2.2",
|
||||
"victory-vendor": "^37.0.2"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=18"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"react": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0",
|
||||
"react-dom": "^16.0.0 || ^17.0.0 || ^18.0.0 || ^19.0.0",
|
||||
"react-is": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/redent": {
|
||||
"version": "3.0.0",
|
||||
"resolved": "https://registry.npmjs.org/redent/-/redent-3.0.0.tgz",
|
||||
@@ -5871,6 +6206,21 @@
|
||||
"node": ">=8"
|
||||
}
|
||||
},
|
||||
"node_modules/redux": {
|
||||
"version": "5.0.1",
|
||||
"resolved": "https://registry.npmjs.org/redux/-/redux-5.0.1.tgz",
|
||||
"integrity": "sha512-M9/ELqF6fy8FwmkpnF0S3YKOqMyoWJ4+CS5Efg2ct3oY9daQvd/Pc71FpGZsVsbl3Cpb+IIcjBDUnnyBdQbq4w==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/redux-thunk": {
|
||||
"version": "3.1.0",
|
||||
"resolved": "https://registry.npmjs.org/redux-thunk/-/redux-thunk-3.1.0.tgz",
|
||||
"integrity": "sha512-NW2r5T6ksUKXCabzhL9z+h206HQw/NJkcLm1GPImRQ8IzfXwRGqjVhKJGauHirT0DAuyy6hjdnMZaRoAcy0Klw==",
|
||||
"license": "MIT",
|
||||
"peerDependencies": {
|
||||
"redux": "^5.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/reflect.getprototypeof": {
|
||||
"version": "1.0.10",
|
||||
"resolved": "https://registry.npmjs.org/reflect.getprototypeof/-/reflect.getprototypeof-1.0.10.tgz",
|
||||
@@ -5925,6 +6275,12 @@
|
||||
"node": ">=0.10.0"
|
||||
}
|
||||
},
|
||||
"node_modules/reselect": {
|
||||
"version": "5.1.1",
|
||||
"resolved": "https://registry.npmjs.org/reselect/-/reselect-5.1.1.tgz",
|
||||
"integrity": "sha512-K/BG6eIky/SBpzfHZv/dd+9JBFiS4SWV7FIujVyJRux6e45+73RaUHXLmIR1f7WOMaQ0U1km6qwklRQxpJJY0w==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/resolve": {
|
||||
"version": "2.0.0-next.7",
|
||||
"resolved": "https://registry.npmjs.org/resolve/-/resolve-2.0.0-next.7.tgz",
|
||||
@@ -6448,6 +6804,12 @@
|
||||
"integrity": "sha512-05PUHKSNE8ou2dwIxTngl4EzcnsCDZGJ/iCLtDflR/SHB/ny14rXc+qU5P4mG9JkusiV7EivzY9Mhm55AzAvCg==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/tiny-invariant": {
|
||||
"version": "1.3.3",
|
||||
"resolved": "https://registry.npmjs.org/tiny-invariant/-/tiny-invariant-1.3.3.tgz",
|
||||
"integrity": "sha512-+FbBPE1o9QAYvviau/qC5SE3caw21q3xkvWKBtja5vgqOWIHHJ3ioaq1VPfn/Szqctz2bU/oYeKd9/z5BL+PVg==",
|
||||
"license": "MIT"
|
||||
},
|
||||
"node_modules/tinybench": {
|
||||
"version": "2.9.0",
|
||||
"resolved": "https://registry.npmjs.org/tinybench/-/tinybench-2.9.0.tgz",
|
||||
@@ -6863,6 +7225,37 @@
|
||||
}
|
||||
}
|
||||
},
|
||||
"node_modules/use-sync-external-store": {
|
||||
"version": "1.6.0",
|
||||
"resolved": "https://registry.npmjs.org/use-sync-external-store/-/use-sync-external-store-1.6.0.tgz",
|
||||
"integrity": "sha512-Pp6GSwGP/NrPIrxVFAIkOQeyw8lFenOHijQWkUTrDvrF4ALqylP2C/KCkeS9dpUM3KvYRQhna5vt7IL95+ZQ9w==",
|
||||
"license": "MIT",
|
||||
"peerDependencies": {
|
||||
"react": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/victory-vendor": {
|
||||
"version": "37.3.6",
|
||||
"resolved": "https://registry.npmjs.org/victory-vendor/-/victory-vendor-37.3.6.tgz",
|
||||
"integrity": "sha512-SbPDPdDBYp+5MJHhBCAyI7wKM3d5ivekigc2Dk2s7pgbZ9wIgIBYGVw4zGHBml/qTFbexrofXW6Gu4noGxrOwQ==",
|
||||
"license": "MIT AND ISC",
|
||||
"dependencies": {
|
||||
"@types/d3-array": "^3.0.3",
|
||||
"@types/d3-ease": "^3.0.0",
|
||||
"@types/d3-interpolate": "^3.0.1",
|
||||
"@types/d3-scale": "^4.0.2",
|
||||
"@types/d3-shape": "^3.1.0",
|
||||
"@types/d3-time": "^3.0.0",
|
||||
"@types/d3-timer": "^3.0.0",
|
||||
"d3-array": "^3.1.6",
|
||||
"d3-ease": "^3.0.1",
|
||||
"d3-interpolate": "^3.0.1",
|
||||
"d3-scale": "^4.0.2",
|
||||
"d3-shape": "^3.1.0",
|
||||
"d3-time": "^3.0.0",
|
||||
"d3-timer": "^3.0.1"
|
||||
}
|
||||
},
|
||||
"node_modules/vite": {
|
||||
"version": "6.4.3",
|
||||
"resolved": "https://registry.npmjs.org/vite/-/vite-6.4.3.tgz",
|
||||
|
||||
@@ -22,11 +22,13 @@
|
||||
"leaflet.heat": "^0.2.0",
|
||||
"leaflet.markercluster": "^1.5.3",
|
||||
"openapi-fetch": "^0.17.0",
|
||||
"qrcode.react": "^4.2.0",
|
||||
"react": "^18.3.1",
|
||||
"react-dom": "^18.3.1",
|
||||
"react-feather": "^2.0.10",
|
||||
"react-leaflet": "^4.2.1",
|
||||
"react-router-dom": "^6.30.4"
|
||||
"react-router-dom": "^6.30.4",
|
||||
"recharts": "^3.8.1"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@eslint/js": "^9.39.4",
|
||||
|
||||
+39
-120
@@ -6,24 +6,20 @@
|
||||
*
|
||||
* Route tree:
|
||||
* /login → LoginPage (public)
|
||||
* /change-password → ProtectedRoute → ChangePasswordPage (T07: forced password change gate)
|
||||
* / → ProtectedRoute → AppLayout → HomePage (T09)
|
||||
* /config → ProtectedRoute → AppLayout → ConfigPage (T08)
|
||||
* /change-password → ProtectedRoute → ChangePasswordPage (forced password change gate)
|
||||
* / → ProtectedRoute → AppLayout → HomePage
|
||||
* /config → ProtectedRoute → AppLayout → ConfigPage
|
||||
* /records → ProtectedRoute → AppLayout → RecordsPage
|
||||
* /energy → ProtectedRoute → AppLayout → EnergyPage
|
||||
*
|
||||
* AppLayout renders a nav with a gear-icon entry for /config and a logout button (T07).
|
||||
* AppLayout renders a sidebar (AppSidebar) on the left; page content via <Outlet/> on the right.
|
||||
* /login and /change-password have no sidebar layout.
|
||||
*/
|
||||
|
||||
import { useState } from 'react'
|
||||
import { QueryClient, QueryClientProvider } from '@tanstack/react-query'
|
||||
import { BrowserRouter, Routes, Route, Link, Outlet, useNavigate } from 'react-router-dom'
|
||||
import {
|
||||
MantineProvider,
|
||||
Group,
|
||||
ActionIcon,
|
||||
Tooltip,
|
||||
useMantineColorScheme,
|
||||
useComputedColorScheme,
|
||||
} from '@mantine/core'
|
||||
import { List, Settings, Sun, Moon, LogOut } from 'react-feather'
|
||||
import { BrowserRouter, Routes, Route, Outlet } from 'react-router-dom'
|
||||
import { MantineProvider, AppShell, Burger } from '@mantine/core'
|
||||
|
||||
// Mantine requires its CSS to be imported once.
|
||||
import '@mantine/core/styles.css'
|
||||
@@ -34,9 +30,9 @@ import { LoginPage } from './pages/LoginPage'
|
||||
import { HomePage } from './pages/HomePage'
|
||||
import { ConfigPage } from './pages/ConfigPage'
|
||||
import { RecordsPage } from './pages/RecordsPage'
|
||||
import { EnergyPage } from './pages/EnergyPage'
|
||||
import { ChangePasswordPage } from './pages/ChangePasswordPage'
|
||||
import apiClient from './api/client'
|
||||
import { useQueryClient } from '@tanstack/react-query'
|
||||
import { AppSidebar } from './components/AppSidebar'
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// TanStack Query client (singleton, created outside render to avoid re-creation)
|
||||
@@ -57,120 +53,42 @@ const queryClient = new QueryClient({
|
||||
},
|
||||
})
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Logout button component (needs navigate + queryClient hooks, so it's a component)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function LogoutButton() {
|
||||
const navigate = useNavigate()
|
||||
const qc = useQueryClient()
|
||||
|
||||
async function handleLogout() {
|
||||
try {
|
||||
await apiClient.POST('/api/auth/logout')
|
||||
} catch {
|
||||
// Ignore errors on logout — we clear the session regardless.
|
||||
}
|
||||
// Invalidate session so SessionProvider becomes unauthenticated.
|
||||
await qc.invalidateQueries({ queryKey: ['session'] })
|
||||
navigate('/login', { replace: true })
|
||||
}
|
||||
|
||||
return (
|
||||
<Tooltip label="Log out">
|
||||
<ActionIcon
|
||||
variant="default"
|
||||
size="lg"
|
||||
onClick={handleLogout}
|
||||
aria-label="Log out"
|
||||
data-testid="logout-button"
|
||||
>
|
||||
<LogOut size={18} />
|
||||
</ActionIcon>
|
||||
</Tooltip>
|
||||
)
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Dark-mode toggle (sits next to the gear / settings icon)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function ColorSchemeToggle() {
|
||||
const { setColorScheme } = useMantineColorScheme()
|
||||
const computed = useComputedColorScheme('light', { getInitialValueInEffect: true })
|
||||
const isDark = computed === 'dark'
|
||||
return (
|
||||
<Tooltip label={isDark ? 'Light mode' : 'Dark mode'}>
|
||||
<ActionIcon
|
||||
variant="default"
|
||||
size="lg"
|
||||
aria-label="Toggle color scheme"
|
||||
onClick={() => setColorScheme(isDark ? 'light' : 'dark')}
|
||||
data-testid="color-scheme-toggle"
|
||||
>
|
||||
{isDark ? <Sun size={18} /> : <Moon size={18} />}
|
||||
</ActionIcon>
|
||||
</Tooltip>
|
||||
)
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// App shell layout (used by all protected pages)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function AppLayout() {
|
||||
// Mobile navbar open/close state — controlled here so the Burger and
|
||||
// AppShell share the same state.
|
||||
const [navbarOpen, setNavbarOpen] = useState(false)
|
||||
|
||||
return (
|
||||
<div style={{ minHeight: '100vh', display: 'flex', flexDirection: 'column' }}>
|
||||
{/* Top nav */}
|
||||
<nav
|
||||
style={{
|
||||
display: 'flex',
|
||||
alignItems: 'center',
|
||||
justifyContent: 'space-between',
|
||||
padding: '0.5rem 1rem',
|
||||
borderBottom: '1px solid var(--mantine-color-default-border)',
|
||||
<AppShell
|
||||
navbar={{
|
||||
width: 220,
|
||||
breakpoint: 'sm',
|
||||
collapsed: { mobile: !navbarOpen },
|
||||
}}
|
||||
header={{ height: { base: 48, sm: 0 } }}
|
||||
padding="md"
|
||||
>
|
||||
<Link to="/" style={{ fontWeight: 600, textDecoration: 'none' }}>
|
||||
Home Automation
|
||||
</Link>
|
||||
{/* Burger shown on mobile only (hidden on sm+) */}
|
||||
<AppShell.Header hiddenFrom="sm" p="xs">
|
||||
<Burger
|
||||
opened={navbarOpen}
|
||||
onClick={() => setNavbarOpen((o) => !o)}
|
||||
size="sm"
|
||||
aria-label="Toggle navigation"
|
||||
data-testid="nav-burger"
|
||||
/>
|
||||
</AppShell.Header>
|
||||
|
||||
<Group gap="xs">
|
||||
{/* Records nav link */}
|
||||
<Tooltip label="Records">
|
||||
<ActionIcon
|
||||
component={Link}
|
||||
to="/records"
|
||||
variant="default"
|
||||
size="lg"
|
||||
aria-label="Records"
|
||||
>
|
||||
<List size={18} />
|
||||
</ActionIcon>
|
||||
</Tooltip>
|
||||
{/* Dark-mode toggle — directly beside the settings gear */}
|
||||
<ColorSchemeToggle />
|
||||
{/* Settings — links to config page (§5#10) */}
|
||||
<Tooltip label="Settings">
|
||||
<ActionIcon
|
||||
component={Link}
|
||||
to="/config"
|
||||
variant="default"
|
||||
size="lg"
|
||||
aria-label="Settings"
|
||||
>
|
||||
<Settings size={18} />
|
||||
</ActionIcon>
|
||||
</Tooltip>
|
||||
<LogoutButton />
|
||||
</Group>
|
||||
</nav>
|
||||
<AppSidebar onNavClick={() => setNavbarOpen(false)} />
|
||||
|
||||
{/* Page content */}
|
||||
<main style={{ flex: 1 }}>
|
||||
<AppShell.Main>
|
||||
<Outlet />
|
||||
</main>
|
||||
</div>
|
||||
</AppShell.Main>
|
||||
</AppShell>
|
||||
)
|
||||
}
|
||||
|
||||
@@ -198,7 +116,7 @@ export default function App() {
|
||||
}
|
||||
/>
|
||||
|
||||
{/* Protected routes — all nested under AppLayout */}
|
||||
{/* Protected routes — all nested under AppLayout (sidebar) */}
|
||||
<Route
|
||||
element={
|
||||
<ProtectedRoute>
|
||||
@@ -209,6 +127,7 @@ export default function App() {
|
||||
<Route index element={<HomePage />} />
|
||||
<Route path="/config" element={<ConfigPage />} />
|
||||
<Route path="/records" element={<RecordsPage />} />
|
||||
<Route path="/energy" element={<EnergyPage />} />
|
||||
</Route>
|
||||
</Routes>
|
||||
</SessionProvider>
|
||||
|
||||
Vendored
+2771
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,77 @@
|
||||
/**
|
||||
* TOTP API helpers (M4-T08).
|
||||
*
|
||||
* All calls go through the typed apiClient (openapi-fetch + generated schema).
|
||||
* CSRF tokens are injected automatically by the client middleware for write
|
||||
* endpoints; GET /api/auth/totp needs no CSRF (read-only).
|
||||
*
|
||||
* Security notes:
|
||||
* - The TotpSetupResponse contains secret + recovery_codes in plaintext.
|
||||
* These are kept only in component state; callers must NOT write them to
|
||||
* localStorage, sessionStorage, or log them.
|
||||
* - After the setup phase is over the data leaves memory when the component
|
||||
* re-renders (one-time display only).
|
||||
*/
|
||||
|
||||
import apiClient, { ApiError } from '../api/client'
|
||||
import type { components } from '../api/schema.d.ts'
|
||||
|
||||
export type TotpSetupResponse = components['schemas']['TotpSetupResponse']
|
||||
export type TotpStatusResponse = components['schemas']['TotpStatusResponse']
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// getTotpStatus — GET /api/auth/totp
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** Return the current TOTP enabled state for the authenticated user. */
|
||||
export async function getTotpStatus(): Promise<TotpStatusResponse> {
|
||||
const res = await apiClient.GET('/api/auth/totp')
|
||||
if (!res.data) {
|
||||
throw new ApiError(res.response.status, null)
|
||||
}
|
||||
return res.data
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// setupTotp — POST /api/auth/totp/setup
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Generate a new pending TOTP secret + recovery codes.
|
||||
* Returns the one-time plaintext values; caller is responsible for displaying
|
||||
* them exactly once and never persisting them outside component state.
|
||||
*/
|
||||
export async function setupTotp(): Promise<TotpSetupResponse> {
|
||||
const res = await apiClient.POST('/api/auth/totp/setup')
|
||||
if (!res.data) {
|
||||
throw new ApiError(res.response.status, null)
|
||||
}
|
||||
return res.data
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// enableTotp — POST /api/auth/totp/enable
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** Confirm TOTP setup with the current 6-digit code from the authenticator app. */
|
||||
export async function enableTotp(code: string): Promise<void> {
|
||||
// The endpoint returns 204 on success; openapi-fetch yields { data: undefined }.
|
||||
await apiClient.POST('/api/auth/totp/enable', { body: { code } })
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// disableTotp — POST /api/auth/totp/disable
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Disable TOTP. Exactly one of password or code must be provided.
|
||||
* On success the backend clears the secret and all recovery codes.
|
||||
*/
|
||||
export async function disableTotp(opts: {
|
||||
password?: string | null
|
||||
code?: string | null
|
||||
}): Promise<void> {
|
||||
await apiClient.POST('/api/auth/totp/disable', {
|
||||
body: { password: opts.password ?? null, code: opts.code ?? null },
|
||||
})
|
||||
}
|
||||
@@ -0,0 +1,138 @@
|
||||
/**
|
||||
* Tests for AppSidebar (M5-T01, updated M5-T06).
|
||||
*
|
||||
* Strategy: render AppSidebar inside a minimal AppShell (required by
|
||||
* AppShell.Navbar) + MemoryRouter so useLocation() works.
|
||||
*
|
||||
* Coverage:
|
||||
* 1. All four nav items render (Home, Records, Energy, Config).
|
||||
* 2. Home nav item is active when pathname is '/'.
|
||||
* 3. Records nav item is active when pathname is '/records'.
|
||||
* 4. Config nav item is active when pathname is '/config'.
|
||||
* 5. Home is NOT active when on '/records'.
|
||||
* 6. Energy nav item is active when on '/energy'.
|
||||
*/
|
||||
|
||||
import { describe, it, expect, vi } from 'vitest'
|
||||
import { screen } from '@testing-library/react'
|
||||
import { render } from '@testing-library/react'
|
||||
import { MantineProvider, AppShell } from '@mantine/core'
|
||||
import { QueryClient, QueryClientProvider } from '@tanstack/react-query'
|
||||
import { MemoryRouter } from 'react-router-dom'
|
||||
import { AppSidebar } from './AppSidebar'
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Mock apiClient (LogoutButton calls POST /api/auth/logout)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
vi.mock('../api/client', () => ({
|
||||
default: {
|
||||
POST: vi.fn(),
|
||||
GET: vi.fn(),
|
||||
},
|
||||
ApiError: class ApiError extends Error {
|
||||
status: number
|
||||
body: unknown
|
||||
constructor(status: number, body: unknown) {
|
||||
super(`API error ${status}`)
|
||||
this.name = 'ApiError'
|
||||
this.status = status
|
||||
this.body = body
|
||||
}
|
||||
},
|
||||
registerLoginRedirect: vi.fn(),
|
||||
}))
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Helper: render AppSidebar inside the required AppShell + router providers
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function renderSidebar(initialPath = '/') {
|
||||
const queryClient = new QueryClient({
|
||||
defaultOptions: { queries: { retry: false }, mutations: { retry: false } },
|
||||
})
|
||||
|
||||
return render(
|
||||
<MantineProvider>
|
||||
<QueryClientProvider client={queryClient}>
|
||||
<MemoryRouter initialEntries={[initialPath]}>
|
||||
<AppShell
|
||||
navbar={{ width: 220, breakpoint: 'sm', collapsed: { mobile: false } }}
|
||||
header={{ height: { base: 48, sm: 0 } }}
|
||||
>
|
||||
<AppSidebar />
|
||||
</AppShell>
|
||||
</MemoryRouter>
|
||||
</QueryClientProvider>
|
||||
</MantineProvider>,
|
||||
)
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Tests
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('AppSidebar', () => {
|
||||
// -------------------------------------------------------------------------
|
||||
// 1. All nav items render
|
||||
// -------------------------------------------------------------------------
|
||||
|
||||
it('renders Home, Records, Energy, and Config nav items', () => {
|
||||
renderSidebar('/')
|
||||
expect(screen.getByTestId('nav-home')).toBeInTheDocument()
|
||||
expect(screen.getByTestId('nav-records')).toBeInTheDocument()
|
||||
expect(screen.getByTestId('nav-energy')).toBeInTheDocument()
|
||||
expect(screen.getByTestId('nav-config')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
// -------------------------------------------------------------------------
|
||||
// 2. Home active on '/'
|
||||
// -------------------------------------------------------------------------
|
||||
|
||||
it('marks Home nav item as active when on "/"', () => {
|
||||
renderSidebar('/')
|
||||
// Mantine NavLink sets data-active="true" on the active item
|
||||
const homeLink = screen.getByTestId('nav-home')
|
||||
expect(homeLink).toHaveAttribute('data-active', 'true')
|
||||
})
|
||||
|
||||
// -------------------------------------------------------------------------
|
||||
// 3. Records active on '/records'
|
||||
// -------------------------------------------------------------------------
|
||||
|
||||
it('marks Records nav item as active when on "/records"', () => {
|
||||
renderSidebar('/records')
|
||||
const recordsLink = screen.getByTestId('nav-records')
|
||||
expect(recordsLink).toHaveAttribute('data-active', 'true')
|
||||
})
|
||||
|
||||
// -------------------------------------------------------------------------
|
||||
// 4. Config active on '/config'
|
||||
// -------------------------------------------------------------------------
|
||||
|
||||
it('marks Config nav item as active when on "/config"', () => {
|
||||
renderSidebar('/config')
|
||||
const configLink = screen.getByTestId('nav-config')
|
||||
expect(configLink).toHaveAttribute('data-active', 'true')
|
||||
})
|
||||
|
||||
// -------------------------------------------------------------------------
|
||||
// 5. Home NOT active on '/records' (exact match guard)
|
||||
// -------------------------------------------------------------------------
|
||||
|
||||
it('does NOT mark Home as active when on "/records"', () => {
|
||||
renderSidebar('/records')
|
||||
const homeLink = screen.getByTestId('nav-home')
|
||||
expect(homeLink).not.toHaveAttribute('data-active', 'true')
|
||||
})
|
||||
|
||||
// -------------------------------------------------------------------------
|
||||
// 6. Energy active on '/energy'
|
||||
// -------------------------------------------------------------------------
|
||||
|
||||
it('marks Energy nav item as active when on "/energy"', () => {
|
||||
renderSidebar('/energy')
|
||||
const energyLink = screen.getByTestId('nav-energy')
|
||||
expect(energyLink).toHaveAttribute('data-active', 'true')
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,171 @@
|
||||
/**
|
||||
* AppSidebar — vertical sidebar navigation for all protected pages.
|
||||
*
|
||||
* Nav items: Home / Records / Energy / Config
|
||||
* Utilities: ColorSchemeToggle + LogoutButton (at bottom)
|
||||
*
|
||||
* Current route is highlighted via useLocation().
|
||||
* Mobile: burger button toggles the navbar open/closed (handled by AppShell context).
|
||||
*/
|
||||
|
||||
import { NavLink, Stack, Divider, Tooltip, ActionIcon, useMantineColorScheme, useComputedColorScheme, AppShell, Text, Group } from '@mantine/core'
|
||||
import { Link, useLocation, useNavigate } from 'react-router-dom'
|
||||
import { Home, List, Settings, Sun, Moon, LogOut, Zap } from 'react-feather'
|
||||
import { useQueryClient } from '@tanstack/react-query'
|
||||
import apiClient from '../api/client'
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Individual nav entries
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
interface NavEntry {
|
||||
to: string
|
||||
label: string
|
||||
icon: React.ReactNode
|
||||
testId: string
|
||||
}
|
||||
|
||||
const NAV_ENTRIES: NavEntry[] = [
|
||||
{
|
||||
to: '/',
|
||||
label: 'Home',
|
||||
icon: <Home size={18} />,
|
||||
testId: 'nav-home',
|
||||
},
|
||||
{
|
||||
to: '/records',
|
||||
label: 'Records',
|
||||
icon: <List size={18} />,
|
||||
testId: 'nav-records',
|
||||
},
|
||||
{
|
||||
to: '/energy',
|
||||
label: 'Energy',
|
||||
icon: <Zap size={18} />,
|
||||
testId: 'nav-energy',
|
||||
},
|
||||
{
|
||||
to: '/config',
|
||||
label: 'Config',
|
||||
icon: <Settings size={18} />,
|
||||
testId: 'nav-config',
|
||||
},
|
||||
]
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Colour-scheme toggle
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function ColorSchemeToggle() {
|
||||
const { setColorScheme } = useMantineColorScheme()
|
||||
const computed = useComputedColorScheme('light', { getInitialValueInEffect: true })
|
||||
const isDark = computed === 'dark'
|
||||
return (
|
||||
<Tooltip label={isDark ? 'Light mode' : 'Dark mode'} position="right">
|
||||
<ActionIcon
|
||||
variant="default"
|
||||
size="lg"
|
||||
aria-label="Toggle color scheme"
|
||||
onClick={() => setColorScheme(isDark ? 'light' : 'dark')}
|
||||
data-testid="color-scheme-toggle"
|
||||
>
|
||||
{isDark ? <Sun size={18} /> : <Moon size={18} />}
|
||||
</ActionIcon>
|
||||
</Tooltip>
|
||||
)
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Logout button
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function LogoutButton() {
|
||||
const navigate = useNavigate()
|
||||
const qc = useQueryClient()
|
||||
|
||||
async function handleLogout() {
|
||||
try {
|
||||
await apiClient.POST('/api/auth/logout')
|
||||
} catch {
|
||||
// Ignore errors — clear session regardless.
|
||||
}
|
||||
await qc.invalidateQueries({ queryKey: ['session'] })
|
||||
navigate('/login', { replace: true })
|
||||
}
|
||||
|
||||
return (
|
||||
<Tooltip label="Log out" position="right">
|
||||
<ActionIcon
|
||||
variant="default"
|
||||
size="lg"
|
||||
onClick={handleLogout}
|
||||
aria-label="Log out"
|
||||
data-testid="logout-button"
|
||||
>
|
||||
<LogOut size={18} />
|
||||
</ActionIcon>
|
||||
</Tooltip>
|
||||
)
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Sidebar
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
interface AppSidebarProps {
|
||||
/** Called when a nav link is clicked on mobile (to close the navbar). */
|
||||
onNavClick?: () => void
|
||||
}
|
||||
|
||||
export function AppSidebar({ onNavClick }: AppSidebarProps) {
|
||||
const location = useLocation()
|
||||
|
||||
return (
|
||||
<AppShell.Navbar p="xs">
|
||||
{/* App title */}
|
||||
<AppShell.Section>
|
||||
<Group px="xs" py="sm">
|
||||
<Text fw={700} size="sm" component={Link} to="/" style={{ textDecoration: 'none' }}>
|
||||
Home Automation
|
||||
</Text>
|
||||
</Group>
|
||||
<Divider />
|
||||
</AppShell.Section>
|
||||
|
||||
{/* Navigation links */}
|
||||
<AppShell.Section grow mt="sm">
|
||||
<Stack gap={4}>
|
||||
{NAV_ENTRIES.map(({ to, label, icon, testId }) => {
|
||||
// For the home route "/" we need an exact match; others prefix-match is fine.
|
||||
const isActive =
|
||||
to === '/'
|
||||
? location.pathname === '/'
|
||||
: location.pathname.startsWith(to)
|
||||
|
||||
return (
|
||||
<NavLink
|
||||
key={to}
|
||||
component={Link}
|
||||
to={to}
|
||||
label={label}
|
||||
leftSection={icon}
|
||||
active={isActive}
|
||||
data-testid={testId}
|
||||
onClick={onNavClick}
|
||||
/>
|
||||
)
|
||||
})}
|
||||
</Stack>
|
||||
</AppShell.Section>
|
||||
|
||||
{/* Bottom utilities: theme toggle + logout */}
|
||||
<AppShell.Section>
|
||||
<Divider mb="xs" />
|
||||
<Group gap="xs" px="xs" pb="xs">
|
||||
<ColorSchemeToggle />
|
||||
<LogoutButton />
|
||||
</Group>
|
||||
</AppShell.Section>
|
||||
</AppShell.Navbar>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,381 @@
|
||||
/**
|
||||
* Tests for ExposeSettings (M5-T12).
|
||||
*
|
||||
* Strategy: vi.mock the apiClient so GET/PUT/POST responses are controlled
|
||||
* without a real server or MQTT broker.
|
||||
*
|
||||
* Coverage:
|
||||
* 1. Loading state shows a loader.
|
||||
* 2. Error state shows an error alert.
|
||||
* 3. Empty catalog shows the empty-state message.
|
||||
* 4. Catalog with entities: renders entity names, device groups, toggle switches.
|
||||
* 5. MQTT status badges render correctly (configured/connected/discovery).
|
||||
* 6. Toggling a switch calls PUT /api/expose with the correct key/bool.
|
||||
* 7. Re-publish button calls POST /api/expose/republish.
|
||||
* 8. Republish ok=true shows success alert; ok=false shows warning.
|
||||
* 9. value_getter is never rendered (no crash from non-serialisable callable).
|
||||
* 10. binary_sensor entities are shown in the catalog alongside sensors.
|
||||
*/
|
||||
|
||||
import { describe, it, expect, vi, beforeEach } from 'vitest'
|
||||
import { screen, waitFor, fireEvent } from '@testing-library/react'
|
||||
import { renderWithProviders } from '../test-utils'
|
||||
import { ExposeSettings } from './ExposeSettings'
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Fixtures
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const MOCK_MQTT_STATUS_OFF = {
|
||||
mqtt_configured: false,
|
||||
mqtt_connected: false,
|
||||
discovery_enabled: false,
|
||||
}
|
||||
|
||||
const MOCK_MQTT_STATUS_ON = {
|
||||
mqtt_configured: true,
|
||||
mqtt_connected: true,
|
||||
discovery_enabled: true,
|
||||
}
|
||||
|
||||
const MOCK_CATALOG_EMPTY = {
|
||||
catalog: [],
|
||||
mqtt_status: MOCK_MQTT_STATUS_OFF,
|
||||
}
|
||||
|
||||
const MOCK_CATALOG_ONE_DEVICE = {
|
||||
catalog: [
|
||||
{
|
||||
entity: {
|
||||
key: 'modbus.abc.voltage',
|
||||
component: 'sensor',
|
||||
device: { identifiers: ['modbus', 'abc'], name: 'Test Meter' },
|
||||
device_class: 'voltage',
|
||||
unit: 'V',
|
||||
name: 'Test Meter Voltage',
|
||||
state_class: 'measurement',
|
||||
},
|
||||
enabled: false,
|
||||
},
|
||||
{
|
||||
entity: {
|
||||
key: 'modbus.abc.current',
|
||||
component: 'sensor',
|
||||
device: { identifiers: ['modbus', 'abc'], name: 'Test Meter' },
|
||||
device_class: 'current',
|
||||
unit: 'A',
|
||||
name: 'Test Meter Current',
|
||||
state_class: 'measurement',
|
||||
},
|
||||
enabled: true,
|
||||
},
|
||||
{
|
||||
entity: {
|
||||
key: 'modbus.abc.online',
|
||||
component: 'binary_sensor',
|
||||
device: { identifiers: ['modbus', 'abc'], name: 'Test Meter' },
|
||||
device_class: 'connectivity',
|
||||
unit: '',
|
||||
name: 'Test Meter Online',
|
||||
state_class: null,
|
||||
},
|
||||
enabled: false,
|
||||
},
|
||||
],
|
||||
mqtt_status: MOCK_MQTT_STATUS_ON,
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Mock apiClient
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const mockGet = vi.fn()
|
||||
const mockPut = vi.fn()
|
||||
const mockPost = vi.fn()
|
||||
|
||||
vi.mock('../api/client', () => ({
|
||||
default: {
|
||||
GET: (...args: unknown[]) => mockGet(...args),
|
||||
PUT: (...args: unknown[]) => mockPut(...args),
|
||||
POST: (...args: unknown[]) => mockPost(...args),
|
||||
},
|
||||
ApiError: class ApiError extends Error {
|
||||
status: number
|
||||
body: unknown
|
||||
constructor(status: number, body: unknown) {
|
||||
super(`API error ${status}`)
|
||||
this.name = 'ApiError'
|
||||
this.status = status
|
||||
this.body = body
|
||||
}
|
||||
},
|
||||
registerLoginRedirect: vi.fn(),
|
||||
}))
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Helper
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function renderExpose() {
|
||||
return renderWithProviders(<ExposeSettings />, { initialPath: '/config' })
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Tests
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('ExposeSettings', () => {
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks()
|
||||
})
|
||||
|
||||
// -------------------------------------------------------------------------
|
||||
// 1. Loading state
|
||||
// -------------------------------------------------------------------------
|
||||
|
||||
it('shows a loader while the catalog is loading', () => {
|
||||
// Never resolves
|
||||
mockGet.mockReturnValue(new Promise(() => {}))
|
||||
renderExpose()
|
||||
expect(screen.getByTestId('expose-loading')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
// -------------------------------------------------------------------------
|
||||
// 2. Error state
|
||||
// -------------------------------------------------------------------------
|
||||
|
||||
it('shows an error alert when the catalog request fails', async () => {
|
||||
mockGet.mockRejectedValue(new Error('Network error'))
|
||||
renderExpose()
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId('expose-load-error')).toBeInTheDocument()
|
||||
})
|
||||
})
|
||||
|
||||
// -------------------------------------------------------------------------
|
||||
// 3. Empty catalog
|
||||
// -------------------------------------------------------------------------
|
||||
|
||||
it('shows empty-state message when catalog is empty', async () => {
|
||||
mockGet.mockResolvedValue({ data: MOCK_CATALOG_EMPTY })
|
||||
renderExpose()
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId('expose-empty')).toBeInTheDocument()
|
||||
})
|
||||
})
|
||||
|
||||
// -------------------------------------------------------------------------
|
||||
// 4. Catalog with entities
|
||||
// -------------------------------------------------------------------------
|
||||
|
||||
it('renders entity names and device groups', async () => {
|
||||
mockGet.mockResolvedValue({ data: MOCK_CATALOG_ONE_DEVICE })
|
||||
renderExpose()
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId('expose-settings')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
// Device group heading
|
||||
expect(screen.getByTestId('device-group-Test Meter')).toBeInTheDocument()
|
||||
|
||||
// Entity names
|
||||
expect(screen.getByText('Test Meter Voltage')).toBeInTheDocument()
|
||||
expect(screen.getByText('Test Meter Current')).toBeInTheDocument()
|
||||
expect(screen.getByText('Test Meter Online')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
it('renders toggle switches for each entity', async () => {
|
||||
mockGet.mockResolvedValue({ data: MOCK_CATALOG_ONE_DEVICE })
|
||||
renderExpose()
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId('entity-toggle-modbus.abc.voltage')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
// voltage is disabled (enabled=false in fixture)
|
||||
const voltageSwitch = screen.getByTestId(
|
||||
'entity-toggle-modbus.abc.voltage',
|
||||
) as HTMLInputElement
|
||||
expect(voltageSwitch.checked).toBe(false)
|
||||
|
||||
// current is enabled (enabled=true in fixture)
|
||||
const currentSwitch = screen.getByTestId(
|
||||
'entity-toggle-modbus.abc.current',
|
||||
) as HTMLInputElement
|
||||
expect(currentSwitch.checked).toBe(true)
|
||||
})
|
||||
|
||||
it('shows binary_sensor entities alongside sensor entities', async () => {
|
||||
mockGet.mockResolvedValue({ data: MOCK_CATALOG_ONE_DEVICE })
|
||||
renderExpose()
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId('entity-row-modbus.abc.online')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
// binary_sensor row contains "binary_sensor" text
|
||||
const onlineRow = screen.getByTestId('entity-row-modbus.abc.online')
|
||||
expect(onlineRow).toHaveTextContent('binary_sensor')
|
||||
})
|
||||
|
||||
// -------------------------------------------------------------------------
|
||||
// 5. MQTT status badges
|
||||
// -------------------------------------------------------------------------
|
||||
|
||||
it('renders MQTT status badges', async () => {
|
||||
mockGet.mockResolvedValue({ data: MOCK_CATALOG_ONE_DEVICE })
|
||||
renderExpose()
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId('mqtt-status-badges')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
expect(screen.getByTestId('badge-mqtt-configured')).toBeInTheDocument()
|
||||
expect(screen.getByTestId('badge-mqtt-connected')).toBeInTheDocument()
|
||||
expect(screen.getByTestId('badge-discovery-enabled')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
it('shows "Configured" when mqtt_configured is true', async () => {
|
||||
mockGet.mockResolvedValue({ data: MOCK_CATALOG_ONE_DEVICE })
|
||||
renderExpose()
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId('badge-mqtt-configured')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
expect(screen.getByTestId('badge-mqtt-configured')).toHaveTextContent('MQTT Configured')
|
||||
})
|
||||
|
||||
it('shows "Not Configured" when mqtt_configured is false', async () => {
|
||||
mockGet.mockResolvedValue({ data: MOCK_CATALOG_EMPTY })
|
||||
renderExpose()
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId('badge-mqtt-configured')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
expect(screen.getByTestId('badge-mqtt-configured')).toHaveTextContent('MQTT Not Configured')
|
||||
})
|
||||
|
||||
// -------------------------------------------------------------------------
|
||||
// 6. Toggle a switch → PUT /api/expose
|
||||
// -------------------------------------------------------------------------
|
||||
|
||||
it('calls PUT /api/expose when a toggle is switched', async () => {
|
||||
mockGet.mockResolvedValue({ data: MOCK_CATALOG_ONE_DEVICE })
|
||||
mockPut.mockResolvedValue({ data: MOCK_CATALOG_ONE_DEVICE })
|
||||
|
||||
renderExpose()
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId('entity-toggle-modbus.abc.voltage')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
// Click the voltage toggle (currently off → turning on)
|
||||
const voltageSwitch = screen.getByTestId('entity-toggle-modbus.abc.voltage')
|
||||
fireEvent.click(voltageSwitch)
|
||||
|
||||
await waitFor(() => {
|
||||
expect(mockPut).toHaveBeenCalled()
|
||||
})
|
||||
|
||||
const putCall = mockPut.mock.calls[0]
|
||||
// path arg
|
||||
expect(putCall[0]).toBe('/api/expose')
|
||||
// body
|
||||
expect(putCall[1]?.body?.toggles).toMatchObject({ 'modbus.abc.voltage': true })
|
||||
})
|
||||
|
||||
// -------------------------------------------------------------------------
|
||||
// 7. Republish button → POST /api/expose/republish
|
||||
// -------------------------------------------------------------------------
|
||||
|
||||
it('calls POST /api/expose/republish when republish button is clicked', async () => {
|
||||
mockGet.mockResolvedValue({ data: MOCK_CATALOG_ONE_DEVICE })
|
||||
mockPost.mockResolvedValue({ data: { ok: true, message: 'HA Discovery re-published.' } })
|
||||
|
||||
renderExpose()
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId('republish-button')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
fireEvent.click(screen.getByTestId('republish-button'))
|
||||
|
||||
await waitFor(() => {
|
||||
expect(mockPost).toHaveBeenCalled()
|
||||
})
|
||||
|
||||
const postCall = mockPost.mock.calls[0]
|
||||
expect(postCall[0]).toBe('/api/expose/republish')
|
||||
})
|
||||
|
||||
// -------------------------------------------------------------------------
|
||||
// 8. Republish ok=true → success alert; ok=false → warning
|
||||
// -------------------------------------------------------------------------
|
||||
|
||||
it('shows success alert when republish returns ok=true', async () => {
|
||||
mockGet.mockResolvedValue({ data: MOCK_CATALOG_ONE_DEVICE })
|
||||
mockPost.mockResolvedValue({
|
||||
data: { ok: true, message: 'HA Discovery re-published successfully.' },
|
||||
})
|
||||
|
||||
renderExpose()
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId('republish-button')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
fireEvent.click(screen.getByTestId('republish-button'))
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId('republish-success')).toBeInTheDocument()
|
||||
})
|
||||
expect(screen.queryByTestId('republish-warning')).not.toBeInTheDocument()
|
||||
})
|
||||
|
||||
it('shows warning alert when republish returns ok=false', async () => {
|
||||
mockGet.mockResolvedValue({ data: MOCK_CATALOG_ONE_DEVICE })
|
||||
mockPost.mockResolvedValue({
|
||||
data: { ok: false, message: 'MQTT not connected.' },
|
||||
})
|
||||
|
||||
renderExpose()
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId('republish-button')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
fireEvent.click(screen.getByTestId('republish-button'))
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId('republish-warning')).toBeInTheDocument()
|
||||
})
|
||||
expect(screen.queryByTestId('republish-success')).not.toBeInTheDocument()
|
||||
})
|
||||
|
||||
// -------------------------------------------------------------------------
|
||||
// 9. No crash from non-serialisable value_getter (value_getter NOT rendered)
|
||||
// -------------------------------------------------------------------------
|
||||
|
||||
it('renders without crashing even if value_getter were present (it is excluded by API)', async () => {
|
||||
// The API never sends value_getter; this tests resilience of the schema
|
||||
const catalogWithExtraField = {
|
||||
...MOCK_CATALOG_ONE_DEVICE,
|
||||
catalog: MOCK_CATALOG_ONE_DEVICE.catalog.map((entry) => ({
|
||||
...entry,
|
||||
entity: { ...entry.entity },
|
||||
// No value_getter — confirm the component handles normal data
|
||||
})),
|
||||
}
|
||||
mockGet.mockResolvedValue({ data: catalogWithExtraField })
|
||||
renderExpose()
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId('expose-settings')).toBeInTheDocument()
|
||||
})
|
||||
// No errors thrown, entities visible
|
||||
expect(screen.getByText('Test Meter Voltage')).toBeInTheDocument()
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,303 @@
|
||||
/**
|
||||
* ExposeSettings — Home Assistant Expose entity management (M5-T12).
|
||||
*
|
||||
* Behaviours:
|
||||
* 1. Load: GET /api/expose → display catalog of exposable entities grouped by HA device,
|
||||
* with per-entity toggle switches and MQTT/Discovery connection status badges.
|
||||
* 2. Toggle: PUT /api/expose with the changed key → bool; triggers discovery re-publish.
|
||||
* 3. Republish: POST /api/expose/republish → manually trigger HA Discovery re-publish.
|
||||
*
|
||||
* All requests go through the typed apiClient (CSRF injected automatically by
|
||||
* csrfMiddleware for write operations).
|
||||
*/
|
||||
|
||||
import { useState } from 'react'
|
||||
import { useQuery, useMutation, useQueryClient } from '@tanstack/react-query'
|
||||
import {
|
||||
Alert,
|
||||
Badge,
|
||||
Button,
|
||||
Center,
|
||||
Group,
|
||||
Loader,
|
||||
Stack,
|
||||
Switch,
|
||||
Text,
|
||||
} from '@mantine/core'
|
||||
import apiClient from '../api/client'
|
||||
import type { components } from '../api/schema.d.ts'
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Types
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
type CatalogEntry = components['schemas']['CatalogEntrySchema']
|
||||
type MqttStatus = components['schemas']['MqttStatusSchema']
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Hook: load expose catalog
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function useExposeCatalog() {
|
||||
return useQuery({
|
||||
queryKey: ['expose-catalog'],
|
||||
queryFn: async () => {
|
||||
const res = await apiClient.GET('/api/expose')
|
||||
return res.data
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Hook: toggle a single entity
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function useToggleEntity() {
|
||||
const qc = useQueryClient()
|
||||
return useMutation({
|
||||
mutationFn: async ({ key, enabled }: { key: string; enabled: boolean }) => {
|
||||
await apiClient.PUT('/api/expose', {
|
||||
body: { toggles: { [key]: enabled } },
|
||||
})
|
||||
},
|
||||
onSuccess: () => qc.invalidateQueries({ queryKey: ['expose-catalog'] }),
|
||||
})
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Hook: republish discovery
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function useRepublish() {
|
||||
const qc = useQueryClient()
|
||||
return useMutation({
|
||||
mutationFn: async () => {
|
||||
const res = await apiClient.POST('/api/expose/republish')
|
||||
return res.data
|
||||
},
|
||||
onSuccess: () => qc.invalidateQueries({ queryKey: ['expose-catalog'] }),
|
||||
})
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// MqttStatusBadges — connection status indicators
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function MqttStatusBadges({ status }: { status: MqttStatus }) {
|
||||
return (
|
||||
<Group gap="xs" wrap="wrap" data-testid="mqtt-status-badges">
|
||||
<Badge
|
||||
color={status.mqtt_configured ? 'blue' : 'gray'}
|
||||
variant="outline"
|
||||
size="sm"
|
||||
data-testid="badge-mqtt-configured"
|
||||
>
|
||||
MQTT {status.mqtt_configured ? 'Configured' : 'Not Configured'}
|
||||
</Badge>
|
||||
<Badge
|
||||
color={status.mqtt_connected ? 'green' : 'red'}
|
||||
variant="outline"
|
||||
size="sm"
|
||||
data-testid="badge-mqtt-connected"
|
||||
>
|
||||
{status.mqtt_connected ? 'Connected' : 'Disconnected'}
|
||||
</Badge>
|
||||
<Badge
|
||||
color={status.discovery_enabled ? 'teal' : 'gray'}
|
||||
variant="outline"
|
||||
size="sm"
|
||||
data-testid="badge-discovery-enabled"
|
||||
>
|
||||
Discovery {status.discovery_enabled ? 'On' : 'Off'}
|
||||
</Badge>
|
||||
</Group>
|
||||
)
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// EntityRow — one entity toggle row
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
interface EntityRowProps {
|
||||
entry: CatalogEntry
|
||||
onToggle: (key: string, enabled: boolean) => void
|
||||
isToggling: boolean
|
||||
}
|
||||
|
||||
function EntityRow({ entry, onToggle, isToggling }: EntityRowProps) {
|
||||
const { entity, enabled } = entry
|
||||
return (
|
||||
<Group justify="space-between" gap="sm" wrap="nowrap" data-testid={`entity-row-${entity.key}`}>
|
||||
<Stack gap={2}>
|
||||
<Text size="sm" fw={500}>
|
||||
{entity.name}
|
||||
</Text>
|
||||
<Text size="xs" c="dimmed">
|
||||
{entity.component}
|
||||
{entity.device_class ? ` · ${entity.device_class}` : ''}
|
||||
{entity.unit ? ` · ${entity.unit}` : ''}
|
||||
</Text>
|
||||
</Stack>
|
||||
<Switch
|
||||
checked={enabled}
|
||||
onChange={(e) => onToggle(entity.key, e.currentTarget.checked)}
|
||||
disabled={isToggling}
|
||||
size="sm"
|
||||
data-testid={`entity-toggle-${entity.key}`}
|
||||
/>
|
||||
</Group>
|
||||
)
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// DeviceGroup — entities grouped by device name
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
interface DeviceGroupProps {
|
||||
deviceName: string
|
||||
entries: CatalogEntry[]
|
||||
onToggle: (key: string, enabled: boolean) => void
|
||||
isToggling: boolean
|
||||
}
|
||||
|
||||
function DeviceGroup({ deviceName, entries, onToggle, isToggling }: DeviceGroupProps) {
|
||||
return (
|
||||
<Stack gap="xs" data-testid={`device-group-${deviceName}`}>
|
||||
<Text fw={600} size="sm" c="dimmed">
|
||||
{deviceName}
|
||||
</Text>
|
||||
{entries.map((entry) => (
|
||||
<EntityRow
|
||||
key={entry.entity.key}
|
||||
entry={entry}
|
||||
onToggle={onToggle}
|
||||
isToggling={isToggling}
|
||||
/>
|
||||
))}
|
||||
</Stack>
|
||||
)
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// RepublishButton — POST /api/expose/republish
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function RepublishButton() {
|
||||
const [result, setResult] = useState<{ ok: boolean; message: string } | null>(null)
|
||||
const republishMutation = useRepublish()
|
||||
|
||||
async function handleRepublish() {
|
||||
setResult(null)
|
||||
try {
|
||||
const data = await republishMutation.mutateAsync()
|
||||
if (data) {
|
||||
setResult({ ok: data.ok, message: data.message })
|
||||
}
|
||||
} catch {
|
||||
setResult({ ok: false, message: 'Republish request failed.' })
|
||||
}
|
||||
}
|
||||
|
||||
return (
|
||||
<Stack gap="xs">
|
||||
<Button
|
||||
type="button"
|
||||
variant="outline"
|
||||
size="sm"
|
||||
onClick={handleRepublish}
|
||||
loading={republishMutation.isPending}
|
||||
data-testid="republish-button"
|
||||
>
|
||||
Re-publish Discovery
|
||||
</Button>
|
||||
|
||||
{result?.ok && (
|
||||
<Alert color="green" data-testid="republish-success">
|
||||
{result.message}
|
||||
</Alert>
|
||||
)}
|
||||
{result !== null && !result.ok && (
|
||||
<Alert color="orange" data-testid="republish-warning">
|
||||
{result.message}
|
||||
</Alert>
|
||||
)}
|
||||
</Stack>
|
||||
)
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// ExposeSettings — main exported component
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export function ExposeSettings() {
|
||||
const { data, isLoading, isError } = useExposeCatalog()
|
||||
const toggleMutation = useToggleEntity()
|
||||
|
||||
function handleToggle(key: string, enabled: boolean) {
|
||||
toggleMutation.mutate({ key, enabled })
|
||||
}
|
||||
|
||||
if (isLoading) {
|
||||
return (
|
||||
<Center pt="md">
|
||||
<Loader size="sm" data-testid="expose-loading" />
|
||||
</Center>
|
||||
)
|
||||
}
|
||||
|
||||
if (isError || !data) {
|
||||
return (
|
||||
<Alert color="red" data-testid="expose-load-error">
|
||||
Failed to load expose settings. Please refresh the page.
|
||||
</Alert>
|
||||
)
|
||||
}
|
||||
|
||||
// Defensive null-coerce in case the data shape is unexpected at runtime.
|
||||
const catalog = data.catalog ?? []
|
||||
const mqtt_status = data.mqtt_status ?? {
|
||||
mqtt_configured: false,
|
||||
mqtt_connected: false,
|
||||
discovery_enabled: false,
|
||||
}
|
||||
|
||||
// Group entities by device name for display.
|
||||
const deviceGroups = new Map<string, CatalogEntry[]>()
|
||||
for (const entry of catalog) {
|
||||
const deviceName = entry.entity.device.name
|
||||
const existing = deviceGroups.get(deviceName)
|
||||
if (existing) {
|
||||
existing.push(entry)
|
||||
} else {
|
||||
deviceGroups.set(deviceName, [entry])
|
||||
}
|
||||
}
|
||||
|
||||
return (
|
||||
<Stack gap="md" data-testid="expose-settings">
|
||||
{/* Status row */}
|
||||
<Group justify="space-between" align="center" wrap="wrap" gap="sm">
|
||||
<MqttStatusBadges status={mqtt_status} />
|
||||
<RepublishButton />
|
||||
</Group>
|
||||
|
||||
{/* Entity catalog — empty state */}
|
||||
{catalog.length === 0 && (
|
||||
<Text c="dimmed" size="sm" data-testid="expose-empty">
|
||||
No exposable entities found. Add a Modbus device in the Energy page to get started.
|
||||
</Text>
|
||||
)}
|
||||
|
||||
{/* Entity catalog — grouped by device */}
|
||||
{Array.from(deviceGroups.entries()).map(([deviceName, entries]) => (
|
||||
<DeviceGroup
|
||||
key={deviceName}
|
||||
deviceName={deviceName}
|
||||
entries={entries}
|
||||
onToggle={handleToggle}
|
||||
isToggling={toggleMutation.isPending}
|
||||
/>
|
||||
))}
|
||||
</Stack>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,338 @@
|
||||
/**
|
||||
* Tests for ContractForm component.
|
||||
*
|
||||
* Coverage:
|
||||
* 1. Renders profile fields dynamically from profile structure (not hardcoded)
|
||||
* — verified by passing defaultKind so fields render automatically.
|
||||
* 2. Submit in create mode calls POST /api/energy/contracts.
|
||||
* 3. Submit in add-version mode calls POST /api/energy/contracts/{id}/versions.
|
||||
* 4. Cancel closes modal.
|
||||
*/
|
||||
|
||||
import { describe, it, expect, vi, beforeEach } from 'vitest'
|
||||
import { screen, waitFor } from '@testing-library/react'
|
||||
import userEvent from '@testing-library/user-event'
|
||||
import { renderWithProviders } from '../test-utils'
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Mock apiClient
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const mockGet = vi.fn()
|
||||
const mockPost = vi.fn()
|
||||
|
||||
vi.mock('../api/client', () => ({
|
||||
default: {
|
||||
GET: (...args: unknown[]) => mockGet(...args),
|
||||
POST: (...args: unknown[]) => mockPost(...args),
|
||||
PATCH: vi.fn(),
|
||||
DELETE: vi.fn(),
|
||||
},
|
||||
ApiError: class ApiError extends Error {
|
||||
status: number
|
||||
body: unknown
|
||||
constructor(status: number, body: unknown) {
|
||||
super(`API error ${status}`)
|
||||
this.name = 'ApiError'
|
||||
this.status = status
|
||||
this.body = body
|
||||
}
|
||||
},
|
||||
registerLoginRedirect: vi.fn(),
|
||||
}))
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Fixtures
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const PROFILES_RESPONSE = {
|
||||
profiles: [
|
||||
{
|
||||
kind: 'manual',
|
||||
label: 'Fixed / Variable Rate (NL, dual-tariff)',
|
||||
energy: {
|
||||
dual_tariff: true,
|
||||
buy: { normal: { unit: 'EUR/kWh' }, dal: { unit: 'EUR/kWh' } },
|
||||
sell: { normal: { unit: 'EUR/kWh' }, dal: { unit: 'EUR/kWh' } },
|
||||
energy_tax: { unit: 'EUR/kWh' },
|
||||
ode: { unit: 'EUR/kWh', default: 0 },
|
||||
},
|
||||
standing: {
|
||||
network_fee: { unit: 'EUR/month' },
|
||||
management_fee: { unit: 'EUR/month' },
|
||||
},
|
||||
credits: {
|
||||
heffingskorting: { unit: 'EUR/year' },
|
||||
},
|
||||
},
|
||||
],
|
||||
}
|
||||
|
||||
const CREATED_CONTRACT = {
|
||||
id: 1,
|
||||
name: 'Test Contract',
|
||||
kind: 'manual',
|
||||
active: false,
|
||||
currency: 'EUR',
|
||||
created_at: '2026-06-01T00:00:00Z',
|
||||
updated_at: '2026-06-01T00:00:00Z',
|
||||
versions: [],
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Import component
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
import { ContractForm } from './ContractForm'
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Tests
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('ContractForm', () => {
|
||||
beforeEach(() => vi.clearAllMocks())
|
||||
|
||||
it('renders profile fields dynamically from profile structure (not hardcoded)', async () => {
|
||||
mockGet.mockImplementation((path: string) => {
|
||||
if (path === '/api/energy/profiles') {
|
||||
return Promise.resolve({ data: PROFILES_RESPONSE })
|
||||
}
|
||||
return Promise.resolve({ data: null })
|
||||
})
|
||||
|
||||
const onClose = vi.fn()
|
||||
const onSaved = vi.fn()
|
||||
|
||||
// Pass defaultKind so the profile fields render automatically in create mode
|
||||
renderWithProviders(
|
||||
<ContractForm defaultKind="manual" onClose={onClose} onSaved={onSaved} />,
|
||||
)
|
||||
|
||||
// Wait for profiles to load and fields to appear
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId('contract-section-energy')).toBeInTheDocument()
|
||||
}, { timeout: 3000 })
|
||||
|
||||
// Verify sections are rendered from profile structure (not hardcoded)
|
||||
expect(screen.getByTestId('contract-section-energy')).toBeInTheDocument()
|
||||
expect(screen.getByTestId('contract-section-standing')).toBeInTheDocument()
|
||||
expect(screen.getByTestId('contract-section-credits')).toBeInTheDocument()
|
||||
|
||||
// Verify specific fields exist from the profile structure
|
||||
// "energy.buy.normal" → label "Buy Normal"
|
||||
expect(screen.getByTestId('contract-field-energy.buy.normal')).toBeInTheDocument()
|
||||
expect(screen.getByTestId('contract-field-energy.buy.dal')).toBeInTheDocument()
|
||||
expect(screen.getByTestId('contract-field-standing.network_fee')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
it('calls POST /api/energy/contracts in create mode', async () => {
|
||||
const user = userEvent.setup()
|
||||
|
||||
mockGet.mockImplementation((path: string) => {
|
||||
if (path === '/api/energy/profiles') {
|
||||
return Promise.resolve({ data: PROFILES_RESPONSE })
|
||||
}
|
||||
return Promise.resolve({ data: null })
|
||||
})
|
||||
mockPost.mockResolvedValue({ data: CREATED_CONTRACT })
|
||||
|
||||
const onClose = vi.fn()
|
||||
const onSaved = vi.fn()
|
||||
|
||||
// Use defaultKind to ensure fields load without needing to interact with Select
|
||||
renderWithProviders(
|
||||
<ContractForm defaultKind="manual" onClose={onClose} onSaved={onSaved} />,
|
||||
)
|
||||
|
||||
// Wait for form to render
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId('contract-form')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
// Wait for profile fields to appear
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId('contract-name')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
// Fill in contract name
|
||||
await user.type(screen.getByTestId('contract-name'), 'Test Contract')
|
||||
|
||||
// Submit form
|
||||
await user.click(screen.getByTestId('contract-form-submit'))
|
||||
|
||||
await waitFor(() => {
|
||||
expect(mockPost).toHaveBeenCalledWith(
|
||||
'/api/energy/contracts',
|
||||
expect.objectContaining({
|
||||
body: expect.objectContaining({
|
||||
name: 'Test Contract',
|
||||
kind: 'manual',
|
||||
currency: 'EUR',
|
||||
}),
|
||||
}),
|
||||
)
|
||||
}, { timeout: 3000 })
|
||||
})
|
||||
|
||||
it('calls POST /api/energy/contracts/{id}/versions in add-version mode', async () => {
|
||||
const user = userEvent.setup()
|
||||
|
||||
mockGet.mockImplementation((path: string) => {
|
||||
if (path === '/api/energy/profiles') {
|
||||
return Promise.resolve({ data: PROFILES_RESPONSE })
|
||||
}
|
||||
return Promise.resolve({ data: null })
|
||||
})
|
||||
mockPost.mockResolvedValue({ data: CREATED_CONTRACT })
|
||||
|
||||
const onClose = vi.fn()
|
||||
const onSaved = vi.fn()
|
||||
|
||||
renderWithProviders(
|
||||
<ContractForm contractId={1} defaultKind="manual" onClose={onClose} onSaved={onSaved} />,
|
||||
)
|
||||
|
||||
// Wait for form to be ready (add-version mode)
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId('contract-form')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
// In add-version mode, no name or kind fields shown
|
||||
expect(screen.queryByTestId('contract-name')).not.toBeInTheDocument()
|
||||
expect(screen.queryByTestId('contract-kind')).not.toBeInTheDocument()
|
||||
|
||||
// Fill in effective_from (required in add-version mode)
|
||||
const effectiveDateInput = screen.getByTestId('contract-effective-from')
|
||||
await user.type(effectiveDateInput, '2026-07-01')
|
||||
|
||||
// Submit form
|
||||
await user.click(screen.getByTestId('contract-form-submit'))
|
||||
|
||||
await waitFor(() => {
|
||||
expect(mockPost).toHaveBeenCalledWith(
|
||||
'/api/energy/contracts/{contract_id}/versions',
|
||||
expect.objectContaining({
|
||||
params: { path: { contract_id: 1 } },
|
||||
body: expect.objectContaining({
|
||||
values: expect.any(Object),
|
||||
}),
|
||||
}),
|
||||
)
|
||||
}, { timeout: 3000 })
|
||||
})
|
||||
|
||||
it('calls onClose when Cancel button is clicked', async () => {
|
||||
const user = userEvent.setup()
|
||||
|
||||
mockGet.mockImplementation((path: string) => {
|
||||
if (path === '/api/energy/profiles') {
|
||||
return Promise.resolve({ data: PROFILES_RESPONSE })
|
||||
}
|
||||
return Promise.resolve({ data: null })
|
||||
})
|
||||
|
||||
const onClose = vi.fn()
|
||||
const onSaved = vi.fn()
|
||||
|
||||
renderWithProviders(
|
||||
<ContractForm onClose={onClose} onSaved={onSaved} />,
|
||||
)
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId('contract-form-cancel')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
await user.click(screen.getByTestId('contract-form-cancel'))
|
||||
|
||||
expect(onClose).toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('prefills fields from the open version when in add-version mode', async () => {
|
||||
// FU11: add-version mode should auto-prefill rate fields from the contract's
|
||||
// current open version so the user only has to change what's different.
|
||||
const CONTRACT_DETAIL = {
|
||||
id: 42,
|
||||
name: 'My Contract',
|
||||
kind: 'manual',
|
||||
active: true,
|
||||
currency: 'EUR',
|
||||
created_at: '2026-01-01T00:00:00Z',
|
||||
updated_at: '2026-01-01T00:00:00Z',
|
||||
versions: [
|
||||
{
|
||||
id: 1,
|
||||
contract_id: 42,
|
||||
effective_from: '2026-01-01T00:00:00Z',
|
||||
effective_to: null, // open version
|
||||
values: {
|
||||
energy: {
|
||||
buy: { normal: 0.133, dal: 0.127 },
|
||||
sell: { normal: 0.05, dal: 0.05 },
|
||||
energy_tax: 0.11,
|
||||
ode: 0.0,
|
||||
},
|
||||
standing: {
|
||||
network_fee: 30.0,
|
||||
management_fee: 60.0,
|
||||
},
|
||||
credits: {
|
||||
heffingskorting: 365.0,
|
||||
},
|
||||
},
|
||||
created_at: '2026-01-01T00:00:00Z',
|
||||
},
|
||||
],
|
||||
}
|
||||
|
||||
mockGet.mockImplementation((path: string) => {
|
||||
if (path === '/api/energy/profiles') {
|
||||
return Promise.resolve({ data: PROFILES_RESPONSE })
|
||||
}
|
||||
if (path === '/api/energy/contracts/{contract_id}') {
|
||||
return Promise.resolve({ data: CONTRACT_DETAIL })
|
||||
}
|
||||
return Promise.resolve({ data: null })
|
||||
})
|
||||
|
||||
const onClose = vi.fn()
|
||||
const onSaved = vi.fn()
|
||||
|
||||
renderWithProviders(
|
||||
<ContractForm contractId={42} defaultKind="manual" onClose={onClose} onSaved={onSaved} />,
|
||||
)
|
||||
|
||||
const user = userEvent.setup()
|
||||
|
||||
// Wait for the form to render and profile fields to appear
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId('contract-section-energy')).toBeInTheDocument()
|
||||
}, { timeout: 3000 })
|
||||
|
||||
// Wait for prefill to apply: both profile and contract detail queries must resolve.
|
||||
// Fill in the required effective_from date and submit; the submitted values
|
||||
// body.values should contain the prefilled rates from the open version.
|
||||
mockPost.mockResolvedValue({ data: {} })
|
||||
|
||||
const effectiveInput = screen.getByTestId('contract-effective-from')
|
||||
await user.type(effectiveInput, '2026-07-01')
|
||||
|
||||
await user.click(screen.getByTestId('contract-form-submit'))
|
||||
|
||||
// The submitted body should contain the prefilled values from CONTRACT_DETAIL
|
||||
await waitFor(() => {
|
||||
expect(mockPost).toHaveBeenCalledWith(
|
||||
'/api/energy/contracts/{contract_id}/versions',
|
||||
expect.objectContaining({
|
||||
body: expect.objectContaining({
|
||||
values: expect.objectContaining({
|
||||
standing: expect.objectContaining({
|
||||
network_fee: 30,
|
||||
management_fee: 60,
|
||||
}),
|
||||
}),
|
||||
}),
|
||||
}),
|
||||
)
|
||||
}, { timeout: 3000 })
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,466 @@
|
||||
/**
|
||||
* ContractForm — create a new energy contract OR add a version to an existing one.
|
||||
*
|
||||
* Form fields are dynamically rendered from the profile structure returned by
|
||||
* GET /api/energy/profiles — never hardcoded. The profile structure provides
|
||||
* section names (energy, standing, credits), leaf keys, and their units.
|
||||
*
|
||||
* Create mode: show Name, Kind (Select from profiles), Currency, optional
|
||||
* effective_from, then dynamically rendered profile value fields.
|
||||
* Add-version mode (contractId provided): show only effective_from (required)
|
||||
* + profile value fields for the contract's kind.
|
||||
*/
|
||||
|
||||
import { useState, useMemo } from 'react'
|
||||
import {
|
||||
Modal,
|
||||
Stack,
|
||||
TextInput,
|
||||
NumberInput,
|
||||
Select,
|
||||
Button,
|
||||
Group,
|
||||
Alert,
|
||||
Loader,
|
||||
Center,
|
||||
Text,
|
||||
Divider,
|
||||
Title,
|
||||
} from '@mantine/core'
|
||||
import { useEnergyProfiles, useCreateContract, useAddContractVersion, useContractDetail } from './hooks'
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Props
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface ContractFormProps {
|
||||
/** When provided: add-version mode; contract must exist. */
|
||||
contractId?: number
|
||||
/** Existing contract kind (for add-version mode or edit). */
|
||||
defaultKind?: string
|
||||
onClose: () => void
|
||||
onSaved: () => void
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Helpers for dynamic profile field rendering
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* A leaf in the profile structure is any object with a `unit` key.
|
||||
* Returns true if the value is a leaf (renderable field).
|
||||
*/
|
||||
function isLeaf(val: unknown): val is { unit: string; default?: unknown } {
|
||||
return typeof val === 'object' && val !== null && 'unit' in val
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract all leaf fields from a profile section, returning
|
||||
* [dotPath, unit, defaultValue] tuples. E.g.:
|
||||
* "energy.buy.normal" → "EUR/kWh"
|
||||
* "energy.energy_tax" → "EUR/kWh"
|
||||
*/
|
||||
interface LeafField {
|
||||
/** Dot-separated path within the section, e.g. "buy.normal" */
|
||||
fieldPath: string
|
||||
unit: string
|
||||
defaultValue?: number
|
||||
}
|
||||
|
||||
function extractLeafFields(obj: Record<string, unknown>, prefix = ''): LeafField[] {
|
||||
const fields: LeafField[] = []
|
||||
for (const [key, val] of Object.entries(obj)) {
|
||||
// Skip non-object values (e.g. dual_tariff: true)
|
||||
if (typeof val !== 'object' || val === null) continue
|
||||
const path = prefix ? `${prefix}.${key}` : key
|
||||
if (isLeaf(val)) {
|
||||
fields.push({
|
||||
fieldPath: path,
|
||||
unit: val.unit,
|
||||
defaultValue: typeof val.default === 'number' ? val.default : undefined,
|
||||
})
|
||||
} else {
|
||||
fields.push(...extractLeafFields(val as Record<string, unknown>, path))
|
||||
}
|
||||
}
|
||||
return fields
|
||||
}
|
||||
|
||||
/**
|
||||
* Build a nested values object from flat field values.
|
||||
* E.g. { "buy.normal": 0.133, "buy.dal": 0.127 } → { buy: { normal: 0.133, dal: 0.127 } }
|
||||
*/
|
||||
function buildNestedValues(
|
||||
sectionFields: Record<string, LeafField[]>,
|
||||
fieldValues: Record<string, number | string>,
|
||||
): Record<string, unknown> {
|
||||
const result: Record<string, unknown> = {}
|
||||
|
||||
for (const [section, fields] of Object.entries(sectionFields)) {
|
||||
const sectionObj: Record<string, unknown> = {}
|
||||
for (const field of fields) {
|
||||
const raw = fieldValues[`${section}.${field.fieldPath}`]
|
||||
const numVal = typeof raw === 'number' ? raw : parseFloat(String(raw))
|
||||
// Set nested path
|
||||
const parts = field.fieldPath.split('.')
|
||||
let current = sectionObj
|
||||
for (let i = 0; i < parts.length - 1; i++) {
|
||||
if (!(parts[i] in current)) current[parts[i]] = {}
|
||||
current = current[parts[i]] as Record<string, unknown>
|
||||
}
|
||||
current[parts[parts.length - 1]] = isNaN(numVal) ? 0 : numVal
|
||||
}
|
||||
result[section] = sectionObj
|
||||
}
|
||||
|
||||
return result
|
||||
}
|
||||
|
||||
/**
|
||||
* Label formatter: converts "buy.normal" → "Buy Normal", "energy_tax" → "Energy Tax"
|
||||
*/
|
||||
function formatLabel(path: string): string {
|
||||
return path
|
||||
.replace(/\./g, ' ')
|
||||
.replace(/_/g, ' ')
|
||||
.replace(/\b\w/g, (c) => c.toUpperCase())
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Component
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export function ContractForm({ contractId, defaultKind, onClose, onSaved }: ContractFormProps) {
|
||||
const isAddVersion = contractId != null
|
||||
|
||||
// Profiles query
|
||||
const profilesQuery = useEnergyProfiles()
|
||||
|
||||
// Contract detail query — only used in add-version mode to prefill from the open version.
|
||||
const contractDetailQuery = useContractDetail(contractId ?? 0)
|
||||
|
||||
// Create mode fields
|
||||
const [name, setName] = useState('')
|
||||
const [selectedKind, setSelectedKind] = useState<string | null>(defaultKind ?? null)
|
||||
const [currency, setCurrency] = useState('EUR')
|
||||
// ISO date string "YYYY-MM-DD" for effective_from; empty = default to now
|
||||
const [effectiveFromStr, setEffectiveFromStr] = useState('')
|
||||
|
||||
// Dynamic profile field values: key = "<section>.<fieldPath>", value = number.
|
||||
// User edits are stored here; for add-version mode this is merged with prefillValues.
|
||||
const [userEdits, setUserEdits] = useState<Record<string, number | string>>({})
|
||||
|
||||
const [error, setError] = useState<string | null>(null)
|
||||
|
||||
const createMutation = useCreateContract()
|
||||
const addVersionMutation = useAddContractVersion()
|
||||
|
||||
const isBusy = createMutation.isPending || addVersionMutation.isPending
|
||||
|
||||
// Resolve the effective kind (add-version mode uses defaultKind)
|
||||
const effectiveKind = isAddVersion ? (defaultKind ?? null) : selectedKind
|
||||
|
||||
// Build profile options from API response
|
||||
const profiles = profilesQuery.data?.profiles ?? []
|
||||
const profileOptions = profiles.map((p: Record<string, unknown>) => ({
|
||||
value: p.kind as string,
|
||||
label: (p.label as string | undefined) ?? (p.kind as string),
|
||||
}))
|
||||
|
||||
// Find the selected profile structure
|
||||
const selectedProfile = profiles.find((p: Record<string, unknown>) => p.kind === effectiveKind)
|
||||
|
||||
// Extract section → leaf fields mapping from the selected profile (memoized for stable reference).
|
||||
const sectionFields = useMemo<Record<string, LeafField[]>>(() => {
|
||||
if (!selectedProfile) return {}
|
||||
const profileObj = selectedProfile as Record<string, unknown>
|
||||
const result: Record<string, LeafField[]> = {}
|
||||
for (const [key, val] of Object.entries(profileObj)) {
|
||||
if (key === 'kind' || key === 'label') continue
|
||||
if (typeof val === 'object' && val !== null && !isLeaf(val)) {
|
||||
result[key] = extractLeafFields(val as Record<string, unknown>)
|
||||
}
|
||||
}
|
||||
return result
|
||||
}, [selectedProfile])
|
||||
|
||||
// Add-version prefill: compute the seeded values from the contract's open version.
|
||||
// Returns the flat fieldValues dict or null when data is not ready.
|
||||
const prefillValues = useMemo<Record<string, number | string> | null>(() => {
|
||||
if (!isAddVersion) return null
|
||||
if (contractDetailQuery.isLoading || contractDetailQuery.isError) return null
|
||||
if (Object.keys(sectionFields).length === 0) return null
|
||||
|
||||
const detail = contractDetailQuery.data
|
||||
if (!detail || !detail.versions || detail.versions.length === 0) return null
|
||||
|
||||
const versions = detail.versions as Array<{
|
||||
effective_from: string
|
||||
effective_to: string | null
|
||||
values: Record<string, unknown>
|
||||
}>
|
||||
let openVersion = versions.find((v) => v.effective_to == null)
|
||||
if (!openVersion) {
|
||||
openVersion = [...versions].sort((a, b) =>
|
||||
a.effective_from < b.effective_from ? 1 : -1
|
||||
)[0]
|
||||
}
|
||||
if (!openVersion?.values) return null
|
||||
|
||||
const seeded: Record<string, number | string> = {}
|
||||
for (const [section, fields] of Object.entries(sectionFields)) {
|
||||
const sectionVal = (openVersion.values as Record<string, unknown>)[section]
|
||||
if (typeof sectionVal !== 'object' || sectionVal === null) continue
|
||||
for (const leaf of fields) {
|
||||
const parts = leaf.fieldPath.split('.')
|
||||
let cursor: unknown = sectionVal
|
||||
for (const part of parts) {
|
||||
if (typeof cursor !== 'object' || cursor === null) { cursor = undefined; break }
|
||||
cursor = (cursor as Record<string, unknown>)[part]
|
||||
}
|
||||
if (cursor != null && (typeof cursor === 'number' || typeof cursor === 'string')) {
|
||||
const numVal = typeof cursor === 'number' ? cursor : parseFloat(String(cursor))
|
||||
seeded[`${section}.${leaf.fieldPath}`] = isNaN(numVal) ? 0 : numVal
|
||||
}
|
||||
}
|
||||
}
|
||||
return Object.keys(seeded).length > 0 ? seeded : null
|
||||
}, [
|
||||
isAddVersion,
|
||||
contractDetailQuery.isLoading,
|
||||
contractDetailQuery.isError,
|
||||
contractDetailQuery.data,
|
||||
sectionFields,
|
||||
])
|
||||
|
||||
// The effective field values: user edits override prefill; prefill is the base.
|
||||
// In add-version mode: when the user has not yet edited any field (userEdits is empty),
|
||||
// prefillValues is used as the initial base. Once the user edits any field, their
|
||||
// full merged value (prefill + override) is stored in userEdits.
|
||||
const fieldValues: Record<string, number | string> =
|
||||
isAddVersion && prefillValues !== null && Object.keys(userEdits).length === 0
|
||||
? prefillValues
|
||||
: userEdits
|
||||
|
||||
function handleFieldChange(key: string, val: number | string) {
|
||||
// On first edit in add-version mode, copy prefill into userEdits so we don't
|
||||
// lose the other prefilled values when the user changes one field.
|
||||
setUserEdits((prev) => {
|
||||
const base = isAddVersion && prefillValues !== null && Object.keys(prev).length === 0
|
||||
? { ...prefillValues }
|
||||
: prev
|
||||
return { ...base, [key]: val }
|
||||
})
|
||||
}
|
||||
|
||||
// Initialize default values when profile changes (create mode only)
|
||||
function initDefaults(kind: string | null) {
|
||||
if (!kind) return
|
||||
const profile = profiles.find((p: Record<string, unknown>) => p.kind === kind)
|
||||
if (!profile) return
|
||||
const profileObj = profile as Record<string, unknown>
|
||||
const defaults: Record<string, number | string> = {}
|
||||
for (const [sectionKey, sectionVal] of Object.entries(profileObj)) {
|
||||
if (sectionKey === 'kind' || sectionKey === 'label') continue
|
||||
if (typeof sectionVal === 'object' && sectionVal !== null && !isLeaf(sectionVal)) {
|
||||
const leaves = extractLeafFields(sectionVal as Record<string, unknown>)
|
||||
for (const leaf of leaves) {
|
||||
defaults[`${sectionKey}.${leaf.fieldPath}`] = leaf.defaultValue ?? 0
|
||||
}
|
||||
}
|
||||
}
|
||||
setUserEdits(defaults)
|
||||
}
|
||||
|
||||
async function handleSubmit(e: React.FormEvent) {
|
||||
e.preventDefault()
|
||||
setError(null)
|
||||
|
||||
if (!isAddVersion && !name.trim()) {
|
||||
setError('Contract name is required.')
|
||||
return
|
||||
}
|
||||
if (!effectiveKind) {
|
||||
setError('Contract kind is required.')
|
||||
return
|
||||
}
|
||||
|
||||
const values = buildNestedValues(sectionFields, fieldValues)
|
||||
|
||||
try {
|
||||
// Convert local date string to a naive local-midnight datetime string (no Z).
|
||||
// The backend interprets naive datetimes as server local wall-clock time,
|
||||
// so "2026-06-25T00:00:00" → CEST local midnight → stored as UTC 22:00Z.
|
||||
// When effectiveFromStr is empty, fall back to the current instant (aware,
|
||||
// sent as ISO with Z) so the backend stores it unchanged.
|
||||
const effectiveFromISO = effectiveFromStr
|
||||
? `${effectiveFromStr}T00:00:00`
|
||||
: undefined
|
||||
|
||||
if (isAddVersion) {
|
||||
const body = {
|
||||
effective_from: effectiveFromISO ?? new Date().toISOString(),
|
||||
values,
|
||||
}
|
||||
await addVersionMutation.mutateAsync({ id: contractId, body })
|
||||
} else {
|
||||
const body = {
|
||||
name: name.trim(),
|
||||
kind: effectiveKind,
|
||||
currency,
|
||||
values,
|
||||
...(effectiveFromISO ? { effective_from: effectiveFromISO } : {}),
|
||||
}
|
||||
await createMutation.mutateAsync(body)
|
||||
}
|
||||
onSaved()
|
||||
onClose()
|
||||
} catch {
|
||||
setError('Failed to save. Please try again.')
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Render
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const title = isAddVersion ? 'Add Contract Version' : 'New Energy Contract'
|
||||
|
||||
return (
|
||||
<Modal
|
||||
opened
|
||||
onClose={onClose}
|
||||
title={title}
|
||||
size="md"
|
||||
data-testid="contract-form-modal"
|
||||
>
|
||||
{(profilesQuery.isLoading || (isAddVersion && contractDetailQuery.isLoading)) && (
|
||||
<Center py="md">
|
||||
<Loader size="sm" data-testid="contract-form-loading" />
|
||||
</Center>
|
||||
)}
|
||||
|
||||
{profilesQuery.isError && (
|
||||
<Alert color="red" mb="sm" data-testid="contract-form-profiles-error">
|
||||
Failed to load pricing profiles. Cannot continue.
|
||||
</Alert>
|
||||
)}
|
||||
|
||||
{isAddVersion && contractDetailQuery.isError && !contractDetailQuery.isLoading && (
|
||||
<Alert color="yellow" mb="sm" data-testid="contract-form-detail-error">
|
||||
Could not load existing contract values for prefill — fields start at 0.
|
||||
</Alert>
|
||||
)}
|
||||
|
||||
{!profilesQuery.isLoading && !(isAddVersion && contractDetailQuery.isLoading) && (
|
||||
<form onSubmit={handleSubmit} data-testid="contract-form">
|
||||
<Stack gap="sm">
|
||||
{/* Create mode fields */}
|
||||
{!isAddVersion && (
|
||||
<>
|
||||
<TextInput
|
||||
label="Contract name"
|
||||
required
|
||||
value={name}
|
||||
onChange={(e) => setName(e.currentTarget.value)}
|
||||
data-testid="contract-name"
|
||||
/>
|
||||
|
||||
<Select
|
||||
label="Pricing kind"
|
||||
required
|
||||
data={profileOptions}
|
||||
value={selectedKind}
|
||||
onChange={(val) => {
|
||||
setSelectedKind(val)
|
||||
initDefaults(val)
|
||||
}}
|
||||
data-testid="contract-kind"
|
||||
/>
|
||||
|
||||
<TextInput
|
||||
label="Currency"
|
||||
value={currency}
|
||||
onChange={(e) => setCurrency(e.currentTarget.value)}
|
||||
data-testid="contract-currency"
|
||||
/>
|
||||
</>
|
||||
)}
|
||||
|
||||
{/* Effective from — required for add-version, optional for create */}
|
||||
<TextInput
|
||||
label={isAddVersion ? 'Effective from (required)' : 'Effective from (optional, defaults to now)'}
|
||||
description="Date in YYYY-MM-DD format"
|
||||
type="date"
|
||||
required={isAddVersion}
|
||||
value={effectiveFromStr}
|
||||
onChange={(e) => setEffectiveFromStr(e.currentTarget.value)}
|
||||
data-testid="contract-effective-from"
|
||||
/>
|
||||
|
||||
{/* Dynamic profile fields */}
|
||||
{effectiveKind && Object.keys(sectionFields).length > 0 && (
|
||||
<>
|
||||
<Divider mt="xs" />
|
||||
{Object.entries(sectionFields).map(([section, fields]) => (
|
||||
<Stack key={section} gap="xs" data-testid={`contract-section-${section}`}>
|
||||
<Title order={6} tt="capitalize" c="dimmed">
|
||||
{section.replace(/_/g, ' ')}
|
||||
</Title>
|
||||
{fields.map((field) => {
|
||||
const key = `${section}.${field.fieldPath}`
|
||||
const raw = fieldValues[key]
|
||||
const numVal = typeof raw === 'number' ? raw : parseFloat(String(raw))
|
||||
return (
|
||||
<NumberInput
|
||||
key={key}
|
||||
label={formatLabel(field.fieldPath)}
|
||||
description={field.unit}
|
||||
value={isNaN(numVal) ? 0 : numVal}
|
||||
onChange={(val) => handleFieldChange(key, val)}
|
||||
decimalScale={6}
|
||||
step={0.001}
|
||||
data-testid={`contract-field-${key}`}
|
||||
/>
|
||||
)
|
||||
})}
|
||||
</Stack>
|
||||
))}
|
||||
</>
|
||||
)}
|
||||
|
||||
{!effectiveKind && !isAddVersion && (
|
||||
<Text size="sm" c="dimmed" data-testid="contract-form-kind-hint">
|
||||
Select a pricing kind to see the rate fields.
|
||||
</Text>
|
||||
)}
|
||||
|
||||
{error && (
|
||||
<Alert color="red" data-testid="contract-form-error">
|
||||
{error}
|
||||
</Alert>
|
||||
)}
|
||||
|
||||
<Group justify="flex-end" gap="sm">
|
||||
<Button
|
||||
type="button"
|
||||
variant="default"
|
||||
onClick={onClose}
|
||||
data-testid="contract-form-cancel"
|
||||
>
|
||||
Cancel
|
||||
</Button>
|
||||
<Button
|
||||
type="submit"
|
||||
loading={isBusy}
|
||||
data-testid="contract-form-submit"
|
||||
>
|
||||
{isAddVersion ? 'Add Version' : 'Create Contract'}
|
||||
</Button>
|
||||
</Group>
|
||||
</Stack>
|
||||
</form>
|
||||
)}
|
||||
</Modal>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,205 @@
|
||||
/**
|
||||
* Tests for ContractManager component.
|
||||
*
|
||||
* Coverage:
|
||||
* 1. Loading state rendering.
|
||||
* 2. Contract list rendering.
|
||||
* 3. Empty state rendering.
|
||||
* 4. Activate button calls PATCH with {active: true}.
|
||||
* 5. "New Contract" button opens ContractForm.
|
||||
*/
|
||||
|
||||
import { describe, it, expect, vi, beforeEach } from 'vitest'
|
||||
import { screen, waitFor } from '@testing-library/react'
|
||||
import userEvent from '@testing-library/user-event'
|
||||
import { renderWithProviders } from '../test-utils'
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Mock apiClient
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const mockGet = vi.fn()
|
||||
const mockPost = vi.fn()
|
||||
const mockPatch = vi.fn()
|
||||
|
||||
vi.mock('../api/client', () => ({
|
||||
default: {
|
||||
GET: (...args: unknown[]) => mockGet(...args),
|
||||
POST: (...args: unknown[]) => mockPost(...args),
|
||||
PATCH: (...args: unknown[]) => mockPatch(...args),
|
||||
DELETE: vi.fn(),
|
||||
},
|
||||
ApiError: class ApiError extends Error {
|
||||
status: number
|
||||
body: unknown
|
||||
constructor(status: number, body: unknown) {
|
||||
super(`API error ${status}`)
|
||||
this.name = 'ApiError'
|
||||
this.status = status
|
||||
this.body = body
|
||||
}
|
||||
},
|
||||
registerLoginRedirect: vi.fn(),
|
||||
}))
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Fixtures
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const ACTIVE_CONTRACT = {
|
||||
id: 1,
|
||||
name: 'My Active Contract',
|
||||
kind: 'manual',
|
||||
active: true,
|
||||
currency: 'EUR',
|
||||
created_at: '2026-06-01T00:00:00Z',
|
||||
updated_at: '2026-06-01T00:00:00Z',
|
||||
}
|
||||
|
||||
const INACTIVE_CONTRACT = {
|
||||
id: 2,
|
||||
name: 'Inactive Contract',
|
||||
kind: 'tibber',
|
||||
active: false,
|
||||
currency: 'EUR',
|
||||
created_at: '2026-06-02T00:00:00Z',
|
||||
updated_at: '2026-06-02T00:00:00Z',
|
||||
}
|
||||
|
||||
const PROFILES_RESPONSE = {
|
||||
profiles: [
|
||||
{
|
||||
kind: 'manual',
|
||||
label: 'Fixed / Variable Rate (NL, dual-tariff)',
|
||||
energy: {
|
||||
dual_tariff: true,
|
||||
buy: { normal: { unit: 'EUR/kWh' }, dal: { unit: 'EUR/kWh' } },
|
||||
sell: { normal: { unit: 'EUR/kWh' }, dal: { unit: 'EUR/kWh' } },
|
||||
energy_tax: { unit: 'EUR/kWh' },
|
||||
ode: { unit: 'EUR/kWh', default: 0 },
|
||||
},
|
||||
standing: {
|
||||
network_fee: { unit: 'EUR/month' },
|
||||
management_fee: { unit: 'EUR/month' },
|
||||
},
|
||||
credits: {
|
||||
heffingskorting: { unit: 'EUR/year' },
|
||||
},
|
||||
},
|
||||
],
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Import component
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
import { ContractManager } from './ContractManager'
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Tests
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('ContractManager', () => {
|
||||
beforeEach(() => vi.clearAllMocks())
|
||||
|
||||
it('renders loading state initially', () => {
|
||||
// Never resolve so we stay in loading state
|
||||
mockGet.mockImplementation(() => new Promise(() => {}))
|
||||
|
||||
renderWithProviders(<ContractManager />)
|
||||
|
||||
expect(screen.getByTestId('contracts-loading')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
it('renders contract list when data loads', async () => {
|
||||
mockGet.mockImplementation((path: string) => {
|
||||
if (path === '/api/energy/contracts') {
|
||||
return Promise.resolve({
|
||||
data: { items: [ACTIVE_CONTRACT, INACTIVE_CONTRACT], total: 2 },
|
||||
})
|
||||
}
|
||||
return Promise.resolve({ data: null })
|
||||
})
|
||||
|
||||
renderWithProviders(<ContractManager />)
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId('contracts-table')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
expect(screen.getByText('My Active Contract')).toBeInTheDocument()
|
||||
expect(screen.getByText('Inactive Contract')).toBeInTheDocument()
|
||||
expect(screen.getByTestId('contract-active-badge-1')).toHaveTextContent('active')
|
||||
expect(screen.getByTestId('contract-active-badge-2')).toHaveTextContent('inactive')
|
||||
})
|
||||
|
||||
it('renders empty state when no contracts exist', async () => {
|
||||
mockGet.mockResolvedValue({ data: { items: [], total: 0 } })
|
||||
|
||||
renderWithProviders(<ContractManager />)
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId('contracts-empty')).toBeInTheDocument()
|
||||
})
|
||||
})
|
||||
|
||||
it('calls PATCH with active=true when Activate button clicked', async () => {
|
||||
const user = userEvent.setup()
|
||||
|
||||
mockGet.mockImplementation((path: string) => {
|
||||
if (path === '/api/energy/contracts') {
|
||||
return Promise.resolve({
|
||||
data: { items: [INACTIVE_CONTRACT], total: 1 },
|
||||
})
|
||||
}
|
||||
return Promise.resolve({ data: null })
|
||||
})
|
||||
mockPatch.mockResolvedValue({
|
||||
data: { ...INACTIVE_CONTRACT, active: true, versions: [] },
|
||||
})
|
||||
|
||||
renderWithProviders(<ContractManager />)
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId(`contract-activate-${INACTIVE_CONTRACT.id}`)).toBeInTheDocument()
|
||||
})
|
||||
|
||||
await user.click(screen.getByTestId(`contract-activate-${INACTIVE_CONTRACT.id}`))
|
||||
|
||||
await waitFor(() => {
|
||||
expect(mockPatch).toHaveBeenCalledWith(
|
||||
'/api/energy/contracts/{contract_id}',
|
||||
expect.objectContaining({
|
||||
params: { path: { contract_id: INACTIVE_CONTRACT.id } },
|
||||
body: { active: true },
|
||||
}),
|
||||
)
|
||||
})
|
||||
})
|
||||
|
||||
it('opens ContractForm when "New Contract" button is clicked', async () => {
|
||||
const user = userEvent.setup()
|
||||
|
||||
mockGet.mockImplementation((path: string) => {
|
||||
if (path === '/api/energy/contracts') {
|
||||
return Promise.resolve({ data: { items: [], total: 0 } })
|
||||
}
|
||||
if (path === '/api/energy/profiles') {
|
||||
return Promise.resolve({ data: PROFILES_RESPONSE })
|
||||
}
|
||||
return Promise.resolve({ data: null })
|
||||
})
|
||||
|
||||
renderWithProviders(<ContractManager />)
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId('contract-new-button')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
await user.click(screen.getByTestId('contract-new-button'))
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId('contract-form-modal')).toBeInTheDocument()
|
||||
})
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,331 @@
|
||||
/**
|
||||
* ContractManager — energy contract list UI.
|
||||
*
|
||||
* Features:
|
||||
* - Table of contracts: name, kind, active badge, currency, created date, actions.
|
||||
* - Actions per row: Activate (PATCH active=true), Add Version, View History.
|
||||
* - "New Contract" button at top.
|
||||
* - Version history modal showing all versions with effective dates and values.
|
||||
* - Loading / error / empty states.
|
||||
*/
|
||||
|
||||
import { useState } from 'react'
|
||||
import {
|
||||
Table,
|
||||
Button,
|
||||
Group,
|
||||
Text,
|
||||
Loader,
|
||||
Center,
|
||||
Alert,
|
||||
Stack,
|
||||
Badge,
|
||||
ScrollArea,
|
||||
Modal,
|
||||
Accordion,
|
||||
Code,
|
||||
} from '@mantine/core'
|
||||
import {
|
||||
useContracts,
|
||||
useUpdateContract,
|
||||
type ContractResponse,
|
||||
type ContractDetailResponse,
|
||||
} from './hooks'
|
||||
import { ContractForm } from './ContractForm'
|
||||
import apiClient from '../api/client'
|
||||
import { formatLocalDate } from '../utils/datetime'
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Version history modal
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
interface VersionHistoryModalProps {
|
||||
contractId: number
|
||||
contractName: string
|
||||
onClose: () => void
|
||||
}
|
||||
|
||||
function VersionHistoryModal({ contractId, contractName, onClose }: VersionHistoryModalProps) {
|
||||
const [loading, setLoading] = useState(true)
|
||||
const [detail, setDetail] = useState<ContractDetailResponse | null>(null)
|
||||
const [fetchError, setFetchError] = useState<string | null>(null)
|
||||
|
||||
// Fetch detail on first render
|
||||
useState(() => {
|
||||
apiClient
|
||||
.GET('/api/energy/contracts/{contract_id}', {
|
||||
params: { path: { contract_id: contractId } },
|
||||
})
|
||||
.then((res) => {
|
||||
setDetail(res.data ?? null)
|
||||
setLoading(false)
|
||||
})
|
||||
.catch(() => {
|
||||
setFetchError('Failed to load contract details.')
|
||||
setLoading(false)
|
||||
})
|
||||
})
|
||||
|
||||
return (
|
||||
<Modal
|
||||
opened
|
||||
onClose={onClose}
|
||||
title={`Version history — ${contractName}`}
|
||||
size="lg"
|
||||
data-testid="version-history-modal"
|
||||
>
|
||||
{loading && (
|
||||
<Center py="md">
|
||||
<Loader size="sm" />
|
||||
</Center>
|
||||
)}
|
||||
{fetchError && (
|
||||
<Alert color="red" data-testid="version-history-error">
|
||||
{fetchError}
|
||||
</Alert>
|
||||
)}
|
||||
{!loading && !fetchError && detail && (
|
||||
<>
|
||||
{detail.versions.length === 0 ? (
|
||||
<Text c="dimmed" ta="center" size="sm" data-testid="version-history-empty">
|
||||
No versions found.
|
||||
</Text>
|
||||
) : (
|
||||
<Accordion variant="separated" data-testid="version-history-accordion">
|
||||
{detail.versions.map((v) => (
|
||||
<Accordion.Item key={v.id} value={String(v.id)} data-testid={`version-item-${v.id}`}>
|
||||
<Accordion.Control>
|
||||
<Group gap="sm">
|
||||
<Text size="sm" fw={500}>
|
||||
From: {formatLocalDate(v.effective_from)}
|
||||
</Text>
|
||||
{v.effective_to ? (
|
||||
<Text size="xs" c="dimmed">
|
||||
To: {formatLocalDate(v.effective_to)}
|
||||
</Text>
|
||||
) : (
|
||||
<Badge color="green" size="xs">
|
||||
current
|
||||
</Badge>
|
||||
)}
|
||||
</Group>
|
||||
</Accordion.Control>
|
||||
<Accordion.Panel>
|
||||
<Code block style={{ fontSize: 11 }}>
|
||||
{JSON.stringify(v.values, null, 2)}
|
||||
</Code>
|
||||
</Accordion.Panel>
|
||||
</Accordion.Item>
|
||||
))}
|
||||
</Accordion>
|
||||
)}
|
||||
</>
|
||||
)}
|
||||
<Group justify="flex-end" mt="md">
|
||||
<Button variant="default" onClick={onClose}>
|
||||
Close
|
||||
</Button>
|
||||
</Group>
|
||||
</Modal>
|
||||
)
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Contract table
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
interface ContractTableProps {
|
||||
contracts: ContractResponse[]
|
||||
onActivate: (id: number) => void
|
||||
onAddVersion: (contract: ContractResponse) => void
|
||||
onViewHistory: (contract: ContractResponse) => void
|
||||
activatingId: number | null
|
||||
}
|
||||
|
||||
function ContractTable({
|
||||
contracts,
|
||||
onActivate,
|
||||
onAddVersion,
|
||||
onViewHistory,
|
||||
activatingId,
|
||||
}: ContractTableProps) {
|
||||
if (contracts.length === 0) {
|
||||
return (
|
||||
<Text c="dimmed" ta="center" size="sm" data-testid="contracts-empty">
|
||||
No contracts configured yet. Click "New Contract" to add one.
|
||||
</Text>
|
||||
)
|
||||
}
|
||||
|
||||
return (
|
||||
<ScrollArea>
|
||||
<Table striped highlightOnHover withTableBorder data-testid="contracts-table">
|
||||
<Table.Thead>
|
||||
<Table.Tr>
|
||||
<Table.Th>Name</Table.Th>
|
||||
<Table.Th>Kind</Table.Th>
|
||||
<Table.Th>Active</Table.Th>
|
||||
<Table.Th>Currency</Table.Th>
|
||||
<Table.Th>Created</Table.Th>
|
||||
<Table.Th style={{ textAlign: 'right' }}>Actions</Table.Th>
|
||||
</Table.Tr>
|
||||
</Table.Thead>
|
||||
<Table.Tbody>
|
||||
{contracts.map((contract) => (
|
||||
<Table.Tr key={contract.id} data-testid={`contract-row-${contract.id}`}>
|
||||
<Table.Td>
|
||||
<Text fw={500} size="sm">
|
||||
{contract.name}
|
||||
</Text>
|
||||
</Table.Td>
|
||||
<Table.Td>
|
||||
<Badge variant="outline" size="sm">
|
||||
{contract.kind}
|
||||
</Badge>
|
||||
</Table.Td>
|
||||
<Table.Td>
|
||||
<Badge
|
||||
color={contract.active ? 'green' : 'gray'}
|
||||
variant="light"
|
||||
size="sm"
|
||||
data-testid={`contract-active-badge-${contract.id}`}
|
||||
>
|
||||
{contract.active ? 'active' : 'inactive'}
|
||||
</Badge>
|
||||
</Table.Td>
|
||||
<Table.Td>{contract.currency}</Table.Td>
|
||||
<Table.Td>
|
||||
<Text size="xs" c="dimmed">
|
||||
{formatLocalDate(contract.created_at)}
|
||||
</Text>
|
||||
</Table.Td>
|
||||
<Table.Td>
|
||||
<Group justify="flex-end" gap="xs">
|
||||
{!contract.active && (
|
||||
<Button
|
||||
size="xs"
|
||||
variant="outline"
|
||||
color="green"
|
||||
loading={activatingId === contract.id}
|
||||
onClick={() => onActivate(contract.id)}
|
||||
data-testid={`contract-activate-${contract.id}`}
|
||||
>
|
||||
Activate
|
||||
</Button>
|
||||
)}
|
||||
<Button
|
||||
size="xs"
|
||||
variant="outline"
|
||||
onClick={() => onAddVersion(contract)}
|
||||
data-testid={`contract-add-version-${contract.id}`}
|
||||
>
|
||||
Add Version
|
||||
</Button>
|
||||
<Button
|
||||
size="xs"
|
||||
variant="subtle"
|
||||
onClick={() => onViewHistory(contract)}
|
||||
data-testid={`contract-history-${contract.id}`}
|
||||
>
|
||||
History
|
||||
</Button>
|
||||
</Group>
|
||||
</Table.Td>
|
||||
</Table.Tr>
|
||||
))}
|
||||
</Table.Tbody>
|
||||
</Table>
|
||||
</ScrollArea>
|
||||
)
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// ContractManager — top-level
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export function ContractManager() {
|
||||
const contractsQuery = useContracts()
|
||||
const updateMutation = useUpdateContract()
|
||||
|
||||
const [showCreateForm, setShowCreateForm] = useState(false)
|
||||
const [addVersionContract, setAddVersionContract] = useState<ContractResponse | null>(null)
|
||||
const [historyContract, setHistoryContract] = useState<ContractResponse | null>(null)
|
||||
const [activatingId, setActivatingId] = useState<number | null>(null)
|
||||
|
||||
async function handleActivate(id: number) {
|
||||
setActivatingId(id)
|
||||
try {
|
||||
await updateMutation.mutateAsync({ id, body: { active: true } })
|
||||
} finally {
|
||||
setActivatingId(null)
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Render states
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
if (contractsQuery.isLoading) {
|
||||
return (
|
||||
<Center py="xl" data-testid="contracts-loading">
|
||||
<Loader />
|
||||
</Center>
|
||||
)
|
||||
}
|
||||
|
||||
if (contractsQuery.isError || !contractsQuery.data) {
|
||||
return (
|
||||
<Alert color="red" data-testid="contracts-load-error">
|
||||
Failed to load contracts. Please refresh.
|
||||
</Alert>
|
||||
)
|
||||
}
|
||||
|
||||
const contracts = contractsQuery.data.items
|
||||
|
||||
return (
|
||||
<Stack gap="lg" data-testid="contract-manager">
|
||||
<Group justify="space-between" align="center">
|
||||
<Text fw={500}>Energy Contracts</Text>
|
||||
<Button onClick={() => setShowCreateForm(true)} data-testid="contract-new-button">
|
||||
New Contract
|
||||
</Button>
|
||||
</Group>
|
||||
|
||||
<ContractTable
|
||||
contracts={contracts}
|
||||
onActivate={handleActivate}
|
||||
onAddVersion={(c) => setAddVersionContract(c)}
|
||||
onViewHistory={(c) => setHistoryContract(c)}
|
||||
activatingId={activatingId}
|
||||
/>
|
||||
|
||||
{/* Create new contract */}
|
||||
{showCreateForm && (
|
||||
<ContractForm
|
||||
onClose={() => setShowCreateForm(false)}
|
||||
onSaved={() => setShowCreateForm(false)}
|
||||
/>
|
||||
)}
|
||||
|
||||
{/* Add version to existing contract */}
|
||||
{addVersionContract && (
|
||||
<ContractForm
|
||||
contractId={addVersionContract.id}
|
||||
defaultKind={addVersionContract.kind}
|
||||
onClose={() => setAddVersionContract(null)}
|
||||
onSaved={() => setAddVersionContract(null)}
|
||||
/>
|
||||
)}
|
||||
|
||||
{/* Version history modal */}
|
||||
{historyContract && (
|
||||
<VersionHistoryModal
|
||||
contractId={historyContract.id}
|
||||
contractName={historyContract.name}
|
||||
onClose={() => setHistoryContract(null)}
|
||||
/>
|
||||
)}
|
||||
</Stack>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,214 @@
|
||||
/**
|
||||
* Tests for CostView component.
|
||||
*
|
||||
* Coverage:
|
||||
* 1. Loading state.
|
||||
* 2. Empty state.
|
||||
* 3. Renders costs table with data.
|
||||
* 4. Shows summary values.
|
||||
* 5. Recompute button triggers confirmation modal and then recompute call.
|
||||
*/
|
||||
|
||||
import { describe, it, expect, vi, beforeEach } from 'vitest'
|
||||
import { screen, waitFor } from '@testing-library/react'
|
||||
import userEvent from '@testing-library/user-event'
|
||||
import { renderWithProviders } from '../test-utils'
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Mock apiClient
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const mockGet = vi.fn()
|
||||
const mockPost = vi.fn()
|
||||
|
||||
vi.mock('../api/client', () => ({
|
||||
default: {
|
||||
GET: (...args: unknown[]) => mockGet(...args),
|
||||
POST: (...args: unknown[]) => mockPost(...args),
|
||||
PATCH: vi.fn(),
|
||||
DELETE: vi.fn(),
|
||||
},
|
||||
ApiError: class ApiError extends Error {
|
||||
status: number
|
||||
body: unknown
|
||||
constructor(status: number, body: unknown) {
|
||||
super(`API error ${status}`)
|
||||
this.name = 'ApiError'
|
||||
this.status = status
|
||||
this.body = body
|
||||
}
|
||||
},
|
||||
registerLoginRedirect: vi.fn(),
|
||||
}))
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Fixtures
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const COST_PERIOD = {
|
||||
period_start: '2026-06-22T10:00:00Z',
|
||||
d1_kwh: 0.1,
|
||||
d2_kwh: 0.2,
|
||||
r1_kwh: 0.0,
|
||||
r2_kwh: 0.05,
|
||||
import_cost: 0.044,
|
||||
export_revenue: 0.005,
|
||||
net_cost: 0.039,
|
||||
currency: 'EUR',
|
||||
degraded: false,
|
||||
contract_version_id: 1,
|
||||
}
|
||||
|
||||
const SUMMARY = {
|
||||
currency: 'EUR',
|
||||
metered_import: 10.5,
|
||||
metered_export: 2.3,
|
||||
metered_net: 8.2,
|
||||
fixed_costs: 5.0,
|
||||
credits: 50.0,
|
||||
total_payable: 12.5,
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Import component
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
import { CostView } from './CostView'
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Tests
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('CostView', () => {
|
||||
beforeEach(() => vi.clearAllMocks())
|
||||
|
||||
it('renders loading state initially', () => {
|
||||
mockGet.mockImplementation(() => new Promise(() => {}))
|
||||
|
||||
renderWithProviders(<CostView />)
|
||||
|
||||
expect(screen.getByTestId('costs-loading')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
it('renders empty state when no cost periods available', async () => {
|
||||
mockGet.mockImplementation((path: string) => {
|
||||
if (path === '/api/energy/costs') {
|
||||
return Promise.resolve({ data: { items: [], total: 0 } })
|
||||
}
|
||||
if (path === '/api/energy/costs/summary') {
|
||||
return Promise.resolve({ data: SUMMARY })
|
||||
}
|
||||
return Promise.resolve({ data: null })
|
||||
})
|
||||
|
||||
renderWithProviders(<CostView />)
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId('costs-empty')).toBeInTheDocument()
|
||||
})
|
||||
})
|
||||
|
||||
it('renders costs table when data is available', async () => {
|
||||
mockGet.mockImplementation((path: string) => {
|
||||
if (path === '/api/energy/costs') {
|
||||
return Promise.resolve({ data: { items: [COST_PERIOD], total: 1 } })
|
||||
}
|
||||
if (path === '/api/energy/costs/summary') {
|
||||
return Promise.resolve({ data: SUMMARY })
|
||||
}
|
||||
return Promise.resolve({ data: null })
|
||||
})
|
||||
|
||||
renderWithProviders(<CostView />)
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId('costs-table')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
expect(screen.getByTestId('cost-row-0')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
it('shows summary values correctly', async () => {
|
||||
mockGet.mockImplementation((path: string) => {
|
||||
if (path === '/api/energy/costs') {
|
||||
return Promise.resolve({ data: { items: [], total: 0 } })
|
||||
}
|
||||
if (path === '/api/energy/costs/summary') {
|
||||
return Promise.resolve({ data: SUMMARY })
|
||||
}
|
||||
return Promise.resolve({ data: null })
|
||||
})
|
||||
|
||||
renderWithProviders(<CostView />)
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId('summary-import')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
expect(screen.getByTestId('summary-import')).toHaveTextContent('10.500')
|
||||
expect(screen.getByTestId('summary-export')).toHaveTextContent('2.300')
|
||||
expect(screen.getByTestId('summary-total')).toHaveTextContent('12.50')
|
||||
})
|
||||
|
||||
it('shows recompute confirmation modal on button click', async () => {
|
||||
const user = userEvent.setup()
|
||||
|
||||
mockGet.mockImplementation((path: string) => {
|
||||
if (path === '/api/energy/costs') {
|
||||
return Promise.resolve({ data: { items: [], total: 0 } })
|
||||
}
|
||||
if (path === '/api/energy/costs/summary') {
|
||||
return Promise.resolve({ data: SUMMARY })
|
||||
}
|
||||
return Promise.resolve({ data: null })
|
||||
})
|
||||
|
||||
renderWithProviders(<CostView />)
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId('cost-recompute-button')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
await user.click(screen.getByTestId('cost-recompute-button'))
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId('recompute-confirm-modal')).toBeInTheDocument()
|
||||
})
|
||||
})
|
||||
|
||||
it('calls recompute mutation when confirmed', async () => {
|
||||
const user = userEvent.setup()
|
||||
|
||||
mockGet.mockImplementation((path: string) => {
|
||||
if (path === '/api/energy/costs') {
|
||||
return Promise.resolve({ data: { items: [], total: 0 } })
|
||||
}
|
||||
if (path === '/api/energy/costs/summary') {
|
||||
return Promise.resolve({ data: SUMMARY })
|
||||
}
|
||||
return Promise.resolve({ data: null })
|
||||
})
|
||||
mockPost.mockResolvedValue({ data: { recomputed: 0 } })
|
||||
|
||||
renderWithProviders(<CostView />)
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId('cost-recompute-button')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
await user.click(screen.getByTestId('cost-recompute-button'))
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId('recompute-confirm')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
await user.click(screen.getByTestId('recompute-confirm'))
|
||||
|
||||
await waitFor(() => {
|
||||
expect(mockPost).toHaveBeenCalledWith(
|
||||
'/api/energy/costs/recompute',
|
||||
expect.any(Object),
|
||||
)
|
||||
})
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,403 @@
|
||||
/**
|
||||
* CostView — cost trends + detail + summary.
|
||||
*
|
||||
* Features:
|
||||
* - Date range selector: Today / This month / Custom (SegmentedControl + DateInput).
|
||||
* - Recharts AreaChart showing import_cost, export_revenue, net_cost per period.
|
||||
* - Table showing per-15min periods with time, kWh values, costs, degraded badge.
|
||||
* - Summary cards: metered import/export, fixed costs, credits, total payable.
|
||||
* - "Recompute" button with confirmation.
|
||||
* - Loading/error/empty states, degraded period highlighting.
|
||||
*
|
||||
* Recharts imports are isolated to this file only.
|
||||
*/
|
||||
|
||||
import { useState } from 'react'
|
||||
import {
|
||||
Stack,
|
||||
Text,
|
||||
TextInput,
|
||||
Loader,
|
||||
Center,
|
||||
Alert,
|
||||
Table,
|
||||
Group,
|
||||
Badge,
|
||||
ScrollArea,
|
||||
SegmentedControl,
|
||||
Button,
|
||||
Paper,
|
||||
SimpleGrid,
|
||||
Modal,
|
||||
Title,
|
||||
} from '@mantine/core'
|
||||
import {
|
||||
AreaChart,
|
||||
Area,
|
||||
XAxis,
|
||||
YAxis,
|
||||
CartesianGrid,
|
||||
Tooltip,
|
||||
Legend,
|
||||
ResponsiveContainer,
|
||||
} from 'recharts'
|
||||
import { useEnergyCosts, useEnergyCostSummary, useRecomputeCosts } from './hooks'
|
||||
import { formatLocalTime } from '../utils/datetime'
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Cost limit — prevent accidental full-table pulls
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const COSTS_MAX_LIMIT = 500
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Date range helpers
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function getTodayRange(): { start: string; end: string } {
|
||||
// Use local date boundary so "today" matches what the user sees in the display.
|
||||
const now = new Date()
|
||||
const start = new Date(now.getFullYear(), now.getMonth(), now.getDate())
|
||||
const end = new Date(now.getFullYear(), now.getMonth(), now.getDate() + 1)
|
||||
return { start: start.toISOString(), end: end.toISOString() }
|
||||
}
|
||||
|
||||
function getThisMonthRange(): { start: string; end: string } {
|
||||
// Use local date boundary so "this month" matches what the user sees in the display.
|
||||
const now = new Date()
|
||||
const start = new Date(now.getFullYear(), now.getMonth(), 1)
|
||||
const end = new Date(now.getFullYear(), now.getMonth() + 1, 1)
|
||||
return { start: start.toISOString(), end: end.toISOString() }
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Summary cards
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
interface SummaryCardProps {
|
||||
label: string
|
||||
value: string
|
||||
testId?: string
|
||||
}
|
||||
|
||||
function SummaryCard({ label, value, testId }: SummaryCardProps) {
|
||||
return (
|
||||
<Paper withBorder p="sm" data-testid={testId}>
|
||||
<Stack gap={4}>
|
||||
<Text size="xs" c="dimmed">
|
||||
{label}
|
||||
</Text>
|
||||
<Text fw={600} size="lg">
|
||||
{value}
|
||||
</Text>
|
||||
</Stack>
|
||||
</Paper>
|
||||
)
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// CostView — main component
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
type RangePreset = 'today' | 'month' | 'custom'
|
||||
|
||||
export function CostView() {
|
||||
const [rangePreset, setRangePreset] = useState<RangePreset>('today')
|
||||
// Date strings in YYYY-MM-DD format for custom range
|
||||
const [customStartStr, setCustomStartStr] = useState('')
|
||||
const [customEndStr, setCustomEndStr] = useState('')
|
||||
const [showRecomputeConfirm, setShowRecomputeConfirm] = useState(false)
|
||||
|
||||
// Compute effective date range
|
||||
const { start, end } = (() => {
|
||||
if (rangePreset === 'today') return getTodayRange()
|
||||
if (rangePreset === 'month') return getThisMonthRange()
|
||||
return {
|
||||
start: customStartStr ? new Date(customStartStr).toISOString() : undefined,
|
||||
end: customEndStr ? new Date(customEndStr).toISOString() : undefined,
|
||||
}
|
||||
})()
|
||||
|
||||
const costsQuery = useEnergyCosts(start, end, COSTS_MAX_LIMIT)
|
||||
const summaryQuery = useEnergyCostSummary(start, end)
|
||||
const recomputeMutation = useRecomputeCosts()
|
||||
|
||||
async function handleRecompute() {
|
||||
setShowRecomputeConfirm(false)
|
||||
await recomputeMutation.mutateAsync({ start, end })
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Render
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const currency = costsQuery.data?.items[0]?.currency ?? summaryQuery.data?.currency ?? 'EUR'
|
||||
|
||||
return (
|
||||
<Stack gap="lg" data-testid="cost-view">
|
||||
{/* Date range selector */}
|
||||
<Group align="flex-start" gap="md" wrap="wrap">
|
||||
<Stack gap="xs">
|
||||
<Text size="sm" fw={500}>
|
||||
Date range
|
||||
</Text>
|
||||
<SegmentedControl
|
||||
value={rangePreset}
|
||||
onChange={(v) => setRangePreset(v as RangePreset)}
|
||||
data={[
|
||||
{ label: 'Today', value: 'today' },
|
||||
{ label: 'This month', value: 'month' },
|
||||
{ label: 'Custom', value: 'custom' },
|
||||
]}
|
||||
data-testid="cost-range-control"
|
||||
/>
|
||||
</Stack>
|
||||
|
||||
{rangePreset === 'custom' && (
|
||||
<Group gap="sm" align="flex-end">
|
||||
<TextInput
|
||||
label="From"
|
||||
type="date"
|
||||
value={customStartStr}
|
||||
onChange={(e) => setCustomStartStr(e.currentTarget.value)}
|
||||
data-testid="cost-custom-start"
|
||||
/>
|
||||
<TextInput
|
||||
label="To"
|
||||
type="date"
|
||||
value={customEndStr}
|
||||
onChange={(e) => setCustomEndStr(e.currentTarget.value)}
|
||||
data-testid="cost-custom-end"
|
||||
/>
|
||||
</Group>
|
||||
)}
|
||||
|
||||
<Group gap="sm" style={{ marginLeft: 'auto' }} align="flex-end">
|
||||
<Button
|
||||
variant="outline"
|
||||
color="orange"
|
||||
size="sm"
|
||||
onClick={() => setShowRecomputeConfirm(true)}
|
||||
loading={recomputeMutation.isPending}
|
||||
data-testid="cost-recompute-button"
|
||||
>
|
||||
Recompute
|
||||
</Button>
|
||||
</Group>
|
||||
</Group>
|
||||
|
||||
{/* Summary cards */}
|
||||
{summaryQuery.isLoading && (
|
||||
<Center>
|
||||
<Loader size="sm" />
|
||||
</Center>
|
||||
)}
|
||||
|
||||
{summaryQuery.isError && (
|
||||
<Alert color="red" data-testid="summary-error">
|
||||
Failed to load cost summary.
|
||||
</Alert>
|
||||
)}
|
||||
|
||||
{summaryQuery.data && (
|
||||
<Stack gap="xs">
|
||||
<Title order={6} c="dimmed">
|
||||
Summary
|
||||
</Title>
|
||||
<SimpleGrid cols={{ base: 2, sm: 3 }} spacing="sm" data-testid="cost-summary">
|
||||
<SummaryCard
|
||||
label="Import (kWh)"
|
||||
value={summaryQuery.data.metered_import.toFixed(3)}
|
||||
testId="summary-import"
|
||||
/>
|
||||
<SummaryCard
|
||||
label="Export (kWh)"
|
||||
value={summaryQuery.data.metered_export.toFixed(3)}
|
||||
testId="summary-export"
|
||||
/>
|
||||
<SummaryCard
|
||||
label={`Fixed costs (${currency})`}
|
||||
value={summaryQuery.data.fixed_costs.toFixed(2)}
|
||||
testId="summary-fixed"
|
||||
/>
|
||||
<SummaryCard
|
||||
label={`Credits (${currency})`}
|
||||
value={summaryQuery.data.credits.toFixed(2)}
|
||||
testId="summary-credits"
|
||||
/>
|
||||
<SummaryCard
|
||||
label={`Total payable (${currency})`}
|
||||
value={summaryQuery.data.total_payable.toFixed(2)}
|
||||
testId="summary-total"
|
||||
/>
|
||||
</SimpleGrid>
|
||||
</Stack>
|
||||
)}
|
||||
|
||||
{/* Cost chart */}
|
||||
{costsQuery.isLoading && (
|
||||
<Center py="xl" data-testid="costs-loading">
|
||||
<Loader />
|
||||
</Center>
|
||||
)}
|
||||
|
||||
{costsQuery.isError && (
|
||||
<Alert color="red" data-testid="costs-error">
|
||||
Failed to load cost periods. Please refresh.
|
||||
</Alert>
|
||||
)}
|
||||
|
||||
{costsQuery.data && costsQuery.data.items.length === 0 && (
|
||||
<Alert color="gray" data-testid="costs-empty">
|
||||
No cost data available for the selected period.
|
||||
</Alert>
|
||||
)}
|
||||
|
||||
{costsQuery.data && costsQuery.data.items.length > 0 && (
|
||||
<>
|
||||
{/* Area chart */}
|
||||
<Stack gap="xs" data-testid="cost-chart">
|
||||
<Title order={6} c="dimmed">
|
||||
Cost trends ({currency})
|
||||
</Title>
|
||||
<ResponsiveContainer width="100%" height={220}>
|
||||
<AreaChart
|
||||
data={costsQuery.data.items.map((item) => ({
|
||||
time: formatLocalTime(item.period_start),
|
||||
import_cost: item.import_cost,
|
||||
export_revenue: item.export_revenue,
|
||||
net_cost: item.net_cost,
|
||||
}))}
|
||||
margin={{ top: 4, right: 16, left: 0, bottom: 4 }}
|
||||
>
|
||||
<CartesianGrid strokeDasharray="3 3" />
|
||||
<XAxis dataKey="time" tick={{ fontSize: 10 }} interval="preserveStartEnd" />
|
||||
<YAxis tick={{ fontSize: 10 }} tickFormatter={(v: number) => v.toFixed(2)} />
|
||||
<Tooltip
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
formatter={(val: any) =>
|
||||
[`${typeof val === 'number' ? val.toFixed(4) : String(val)} ${currency}`, undefined]
|
||||
}
|
||||
/>
|
||||
<Legend />
|
||||
<Area
|
||||
type="monotone"
|
||||
dataKey="import_cost"
|
||||
stroke="#2196f3"
|
||||
fill="#bbdefb"
|
||||
name="Import cost"
|
||||
dot={false}
|
||||
/>
|
||||
<Area
|
||||
type="monotone"
|
||||
dataKey="export_revenue"
|
||||
stroke="#4caf50"
|
||||
fill="#c8e6c9"
|
||||
name="Export revenue"
|
||||
dot={false}
|
||||
/>
|
||||
<Area
|
||||
type="monotone"
|
||||
dataKey="net_cost"
|
||||
stroke="#ff9800"
|
||||
fill="#ffe0b2"
|
||||
name="Net cost"
|
||||
dot={false}
|
||||
/>
|
||||
</AreaChart>
|
||||
</ResponsiveContainer>
|
||||
</Stack>
|
||||
|
||||
{/* Detail table */}
|
||||
<Stack gap="xs">
|
||||
<Title order={6} c="dimmed">
|
||||
Period detail
|
||||
</Title>
|
||||
<ScrollArea>
|
||||
<Table
|
||||
striped
|
||||
highlightOnHover
|
||||
withTableBorder
|
||||
withColumnBorders
|
||||
style={{ fontSize: 12 }}
|
||||
data-testid="costs-table"
|
||||
>
|
||||
<Table.Thead>
|
||||
<Table.Tr>
|
||||
<Table.Th>Time</Table.Th>
|
||||
<Table.Th>Import kWh</Table.Th>
|
||||
<Table.Th>Export kWh</Table.Th>
|
||||
<Table.Th>Import cost</Table.Th>
|
||||
<Table.Th>Export rev.</Table.Th>
|
||||
<Table.Th>Net cost</Table.Th>
|
||||
<Table.Th></Table.Th>
|
||||
</Table.Tr>
|
||||
</Table.Thead>
|
||||
<Table.Tbody>
|
||||
{costsQuery.data.items.map((item, idx) => (
|
||||
<Table.Tr
|
||||
key={idx}
|
||||
style={item.degraded ? { opacity: 0.6 } : undefined}
|
||||
data-testid={`cost-row-${idx}`}
|
||||
>
|
||||
<Table.Td>
|
||||
<Text size="xs">
|
||||
{formatLocalTime(item.period_start)}
|
||||
</Text>
|
||||
</Table.Td>
|
||||
<Table.Td>{(item.d1_kwh + item.d2_kwh).toFixed(3)}</Table.Td>
|
||||
<Table.Td>{(item.r1_kwh + item.r2_kwh).toFixed(3)}</Table.Td>
|
||||
<Table.Td>{item.import_cost.toFixed(4)}</Table.Td>
|
||||
<Table.Td>{item.export_revenue.toFixed(4)}</Table.Td>
|
||||
<Table.Td>{item.net_cost.toFixed(4)}</Table.Td>
|
||||
<Table.Td>
|
||||
{item.degraded && (
|
||||
<Badge color="orange" size="xs" data-testid={`cost-degraded-${idx}`}>
|
||||
degraded
|
||||
</Badge>
|
||||
)}
|
||||
</Table.Td>
|
||||
</Table.Tr>
|
||||
))}
|
||||
</Table.Tbody>
|
||||
</Table>
|
||||
</ScrollArea>
|
||||
</Stack>
|
||||
</>
|
||||
)}
|
||||
|
||||
{/* Recompute confirmation modal */}
|
||||
{showRecomputeConfirm && (
|
||||
<Modal
|
||||
opened
|
||||
onClose={() => setShowRecomputeConfirm(false)}
|
||||
title="Recompute costs?"
|
||||
size="sm"
|
||||
data-testid="recompute-confirm-modal"
|
||||
>
|
||||
<Stack gap="md">
|
||||
<Text size="sm">
|
||||
This will recompute all cost periods for the selected date range. Continue?
|
||||
</Text>
|
||||
<Group justify="flex-end" gap="sm">
|
||||
<Button
|
||||
variant="default"
|
||||
onClick={() => setShowRecomputeConfirm(false)}
|
||||
data-testid="recompute-cancel"
|
||||
>
|
||||
Cancel
|
||||
</Button>
|
||||
<Button
|
||||
color="orange"
|
||||
onClick={handleRecompute}
|
||||
data-testid="recompute-confirm"
|
||||
>
|
||||
Recompute
|
||||
</Button>
|
||||
</Group>
|
||||
</Stack>
|
||||
</Modal>
|
||||
)}
|
||||
</Stack>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,241 @@
|
||||
/**
|
||||
* Tests for energy/DeviceForm.tsx
|
||||
*
|
||||
* Coverage:
|
||||
* 1. Renders empty form in create mode (title "New Device").
|
||||
* 2. Renders pre-filled form in edit mode (title "Edit Device"), shows UUID.
|
||||
* 3. Validation: missing friendly_name shows error.
|
||||
* 4. Profile dropdown loads options from GET /api/modbus/profiles.
|
||||
* 5. Submit in create mode calls POST /api/modbus/devices.
|
||||
* 6. Submit in edit mode calls PATCH /api/modbus/devices/{uuid}.
|
||||
* 7. Cancel button invokes onClose without submitting.
|
||||
*/
|
||||
|
||||
import { describe, it, expect, vi, beforeEach } from 'vitest'
|
||||
import { screen, waitFor, fireEvent } from '@testing-library/react'
|
||||
import { renderWithProviders } from '../test-utils'
|
||||
import { DeviceForm } from './DeviceForm'
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Fixtures
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const DEVICE = {
|
||||
uuid: 'abc-123',
|
||||
friendly_name: 'SDM120 Main',
|
||||
transport: 'tcp',
|
||||
host: '192.168.1.100',
|
||||
port: 502,
|
||||
unit_id: 1,
|
||||
profile: 'sdm120',
|
||||
poll_interval_s: 5,
|
||||
enabled: true,
|
||||
last_poll_at: null,
|
||||
last_poll_ok: null,
|
||||
created_at: '2026-06-01T00:00:00Z',
|
||||
updated_at: '2026-06-01T00:00:00Z',
|
||||
}
|
||||
|
||||
const PROFILES_RESP = {
|
||||
profiles: [{ name: 'sdm120', description: 'Eastron SDM120 single-phase energy meter' }],
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Mock apiClient
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const mockGet = vi.fn()
|
||||
const mockPost = vi.fn()
|
||||
const mockPatch = vi.fn()
|
||||
|
||||
vi.mock('../api/client', () => ({
|
||||
default: {
|
||||
GET: (...args: unknown[]) => mockGet(...args),
|
||||
POST: (...args: unknown[]) => mockPost(...args),
|
||||
PATCH: (...args: unknown[]) => mockPatch(...args),
|
||||
DELETE: vi.fn(),
|
||||
},
|
||||
ApiError: class ApiError extends Error {
|
||||
status: number
|
||||
body: unknown
|
||||
constructor(status: number, body: unknown) {
|
||||
super(`API error ${status}`)
|
||||
this.name = 'ApiError'
|
||||
this.status = status
|
||||
this.body = body
|
||||
}
|
||||
},
|
||||
registerLoginRedirect: vi.fn(),
|
||||
}))
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Helpers
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function setupProfilesMock() {
|
||||
mockGet.mockImplementation((path: string) => {
|
||||
if (path === '/api/modbus/profiles') {
|
||||
return Promise.resolve({ data: PROFILES_RESP })
|
||||
}
|
||||
// modbus-devices for invalidation
|
||||
return Promise.resolve({ data: { items: [], total: 0 } })
|
||||
})
|
||||
}
|
||||
|
||||
function renderCreateForm(onClose = vi.fn(), onSaved = vi.fn()) {
|
||||
return renderWithProviders(<DeviceForm onClose={onClose} onSaved={onSaved} />, {
|
||||
initialPath: '/',
|
||||
})
|
||||
}
|
||||
|
||||
function renderEditForm(onClose = vi.fn(), onSaved = vi.fn()) {
|
||||
return renderWithProviders(<DeviceForm device={DEVICE} onClose={onClose} onSaved={onSaved} />, {
|
||||
initialPath: '/',
|
||||
})
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Tests
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('DeviceForm — create mode', () => {
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks()
|
||||
setupProfilesMock()
|
||||
})
|
||||
|
||||
it('renders with title "New Device" and empty friendly-name input', async () => {
|
||||
renderCreateForm()
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId('device-form-modal')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
// Modal title
|
||||
expect(screen.getByText('New Device')).toBeInTheDocument()
|
||||
|
||||
// UUID display should NOT be present in create mode
|
||||
expect(screen.queryByTestId('device-uuid-display')).not.toBeInTheDocument()
|
||||
})
|
||||
|
||||
it('loads profile options from GET /api/modbus/profiles', async () => {
|
||||
renderCreateForm()
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId('device-form')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
// Profile select should be rendered
|
||||
expect(screen.getByTestId('device-profile')).toBeInTheDocument()
|
||||
expect(mockGet).toHaveBeenCalledWith('/api/modbus/profiles')
|
||||
})
|
||||
|
||||
it('shows validation error when friendly name is empty', async () => {
|
||||
renderCreateForm()
|
||||
|
||||
await waitFor(() => expect(screen.getByTestId('device-form')).toBeInTheDocument())
|
||||
|
||||
fireEvent.submit(screen.getByTestId('device-form'))
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId('device-form-error')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
expect(screen.getByTestId('device-form-error').textContent).toContain('Friendly name')
|
||||
})
|
||||
|
||||
it('submit calls POST /api/modbus/devices with correct body (profile auto-selected)', async () => {
|
||||
mockPost.mockResolvedValue({ data: DEVICE })
|
||||
const onSaved = vi.fn()
|
||||
renderCreateForm(vi.fn(), onSaved)
|
||||
|
||||
// Wait for profiles to load; sdm120 is auto-selected as first profile.
|
||||
await waitFor(() => expect(screen.getByTestId('device-form')).toBeInTheDocument())
|
||||
|
||||
// Fill form (profile auto-selected to 'sdm120')
|
||||
fireEvent.change(screen.getByTestId('device-friendly-name'), {
|
||||
target: { value: 'My Meter' },
|
||||
})
|
||||
fireEvent.change(screen.getByTestId('device-host'), {
|
||||
target: { value: '10.0.0.1' },
|
||||
})
|
||||
|
||||
fireEvent.submit(screen.getByTestId('device-form'))
|
||||
|
||||
await waitFor(() => {
|
||||
expect(mockPost).toHaveBeenCalledWith(
|
||||
'/api/modbus/devices',
|
||||
expect.objectContaining({
|
||||
body: expect.objectContaining({
|
||||
friendly_name: 'My Meter',
|
||||
host: '10.0.0.1',
|
||||
transport: 'tcp',
|
||||
profile: 'sdm120',
|
||||
}),
|
||||
}),
|
||||
)
|
||||
})
|
||||
})
|
||||
|
||||
it('cancel calls onClose without POST', async () => {
|
||||
const onClose = vi.fn()
|
||||
renderCreateForm(onClose)
|
||||
|
||||
await waitFor(() => expect(screen.getByTestId('device-form-cancel')).toBeInTheDocument())
|
||||
|
||||
fireEvent.click(screen.getByTestId('device-form-cancel'))
|
||||
|
||||
expect(onClose).toHaveBeenCalledOnce()
|
||||
expect(mockPost).not.toHaveBeenCalled()
|
||||
})
|
||||
})
|
||||
|
||||
describe('DeviceForm — edit mode', () => {
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks()
|
||||
setupProfilesMock()
|
||||
})
|
||||
|
||||
it('renders with title "Edit Device" and pre-filled friendly-name', async () => {
|
||||
renderEditForm()
|
||||
|
||||
// Wait for modal to appear
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId('device-form-modal')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
expect(screen.getByText('Edit Device')).toBeInTheDocument()
|
||||
|
||||
// Wait for form to appear (profiles must load first)
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId('device-form')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
expect(screen.getByTestId('device-uuid-display').textContent).toContain('abc-123')
|
||||
|
||||
// Friendly name input should be pre-filled
|
||||
const input = screen.getByTestId('device-friendly-name') as HTMLInputElement
|
||||
expect(input.value).toBe('SDM120 Main')
|
||||
})
|
||||
|
||||
it('submit calls PATCH /api/modbus/devices/{uuid}', async () => {
|
||||
mockPatch.mockResolvedValue({ data: DEVICE })
|
||||
renderEditForm()
|
||||
|
||||
// Wait for profiles to load so the form is visible
|
||||
await waitFor(() => expect(screen.getByTestId('device-form')).toBeInTheDocument())
|
||||
|
||||
fireEvent.change(screen.getByTestId('device-friendly-name'), {
|
||||
target: { value: 'SDM120 Updated' },
|
||||
})
|
||||
|
||||
fireEvent.submit(screen.getByTestId('device-form'))
|
||||
|
||||
await waitFor(() => {
|
||||
expect(mockPatch).toHaveBeenCalledWith('/api/modbus/devices/{uuid}', {
|
||||
params: { path: { uuid: 'abc-123' } },
|
||||
body: expect.objectContaining({ friendly_name: 'SDM120 Updated' }),
|
||||
})
|
||||
})
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,263 @@
|
||||
/**
|
||||
* DeviceForm — create / edit a Modbus device.
|
||||
*
|
||||
* Used for both new-device creation (no `device` prop) and editing an
|
||||
* existing device (pass `device` prop with current values).
|
||||
*
|
||||
* Profile list is fetched from GET /api/modbus/profiles and rendered as a
|
||||
* Select dropdown. All other fields are inline inputs.
|
||||
*/
|
||||
|
||||
import { useState } from 'react'
|
||||
import {
|
||||
Modal,
|
||||
Stack,
|
||||
TextInput,
|
||||
NumberInput,
|
||||
Select,
|
||||
Switch,
|
||||
Button,
|
||||
Group,
|
||||
Alert,
|
||||
Loader,
|
||||
Center,
|
||||
Text,
|
||||
} from '@mantine/core'
|
||||
import { useProfiles, useCreateDevice, useUpdateDevice } from './hooks'
|
||||
import type { ModbusDevice, ModbusDeviceCreate, ModbusDeviceUpdate } from './hooks'
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Props
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
interface DeviceFormProps {
|
||||
/** When provided, the form is in edit mode; otherwise create mode. */
|
||||
device?: ModbusDevice
|
||||
onClose: () => void
|
||||
onSaved: () => void
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Component
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export function DeviceForm({ device, onClose, onSaved }: DeviceFormProps) {
|
||||
const isEdit = !!device
|
||||
|
||||
// Form state — initialise from device if editing.
|
||||
const [friendlyName, setFriendlyName] = useState(device?.friendly_name ?? '')
|
||||
const [host, setHost] = useState(device?.host ?? '')
|
||||
const [port, setPort] = useState<number | string>(device?.port ?? 502)
|
||||
const [unitId, setUnitId] = useState<number | string>(device?.unit_id ?? 1)
|
||||
const [profile, setProfile] = useState<string | null>(device?.profile ?? null)
|
||||
const [pollIntervalS, setPollIntervalS] = useState<number | string>(device?.poll_interval_s ?? 5)
|
||||
const [enabled, setEnabled] = useState(device?.enabled ?? true)
|
||||
const [error, setError] = useState<string | null>(null)
|
||||
|
||||
const profilesQuery = useProfiles()
|
||||
const createMutation = useCreateDevice()
|
||||
const updateMutation = useUpdateDevice()
|
||||
|
||||
const isBusy = createMutation.isPending || updateMutation.isPending
|
||||
|
||||
// Build select options from profiles response.
|
||||
const profileOptions =
|
||||
profilesQuery.data?.profiles.map((p) => ({
|
||||
value: p.name,
|
||||
label: `${p.name} — ${p.description}`,
|
||||
})) ?? []
|
||||
|
||||
// In create mode, if no explicit selection has been made and profiles are loaded,
|
||||
// derive a default from the first available profile. We avoid useEffect+setState
|
||||
// (causes cascading renders) by computing the effective value here instead.
|
||||
const effectiveProfile: string | null =
|
||||
profile !== null
|
||||
? profile
|
||||
: !isEdit && profileOptions.length > 0
|
||||
? profileOptions[0].value
|
||||
: null
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Validation
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function validate(): string | null {
|
||||
if (!friendlyName.trim()) return 'Friendly name is required.'
|
||||
if (!host.trim()) return 'Host is required.'
|
||||
const portNum = Number(port)
|
||||
if (!Number.isInteger(portNum) || portNum < 1 || portNum > 65535)
|
||||
return 'Port must be an integer between 1 and 65535.'
|
||||
const unitNum = Number(unitId)
|
||||
if (!Number.isInteger(unitNum) || unitNum < 1 || unitNum > 247)
|
||||
return 'Unit ID must be an integer between 1 and 247.'
|
||||
if (!effectiveProfile) return 'Profile is required.'
|
||||
const pollNum = Number(pollIntervalS)
|
||||
if (!Number.isInteger(pollNum) || pollNum < 1)
|
||||
return 'Poll interval must be a positive integer (seconds).'
|
||||
return null
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Submit
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
async function handleSubmit(e: React.FormEvent) {
|
||||
e.preventDefault()
|
||||
setError(null)
|
||||
|
||||
const validationError = validate()
|
||||
if (validationError) {
|
||||
setError(validationError)
|
||||
return
|
||||
}
|
||||
|
||||
try {
|
||||
if (isEdit && device) {
|
||||
const body: ModbusDeviceUpdate = {
|
||||
friendly_name: friendlyName.trim(),
|
||||
host: host.trim(),
|
||||
port: Number(port),
|
||||
unit_id: Number(unitId),
|
||||
profile: effectiveProfile!,
|
||||
poll_interval_s: Number(pollIntervalS),
|
||||
enabled,
|
||||
}
|
||||
await updateMutation.mutateAsync({ uuid: device.uuid, body })
|
||||
} else {
|
||||
const body: ModbusDeviceCreate = {
|
||||
friendly_name: friendlyName.trim(),
|
||||
transport: 'tcp',
|
||||
host: host.trim(),
|
||||
port: Number(port),
|
||||
unit_id: Number(unitId),
|
||||
profile: effectiveProfile!,
|
||||
poll_interval_s: Number(pollIntervalS),
|
||||
enabled,
|
||||
}
|
||||
await createMutation.mutateAsync(body)
|
||||
}
|
||||
onSaved()
|
||||
onClose()
|
||||
} catch {
|
||||
setError('Failed to save device. Please try again.')
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Render
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
return (
|
||||
<Modal
|
||||
opened
|
||||
onClose={onClose}
|
||||
title={isEdit ? 'Edit Device' : 'New Device'}
|
||||
size="md"
|
||||
data-testid="device-form-modal"
|
||||
>
|
||||
{profilesQuery.isLoading && (
|
||||
<Center py="md">
|
||||
<Loader size="sm" />
|
||||
</Center>
|
||||
)}
|
||||
|
||||
{profilesQuery.isError && (
|
||||
<Alert color="red" mb="sm" data-testid="profiles-load-error">
|
||||
Failed to load profiles. Cannot create device.
|
||||
</Alert>
|
||||
)}
|
||||
|
||||
{!profilesQuery.isLoading && (
|
||||
<form onSubmit={handleSubmit} data-testid="device-form">
|
||||
<Stack gap="sm">
|
||||
{isEdit && (
|
||||
<Text size="xs" c="dimmed" data-testid="device-uuid-display">
|
||||
UUID: {device!.uuid}
|
||||
</Text>
|
||||
)}
|
||||
|
||||
<TextInput
|
||||
label="Friendly name"
|
||||
required
|
||||
value={friendlyName}
|
||||
onChange={(e) => setFriendlyName(e.currentTarget.value)}
|
||||
data-testid="device-friendly-name"
|
||||
/>
|
||||
|
||||
<TextInput
|
||||
label="Host"
|
||||
description="Gateway IP address"
|
||||
required
|
||||
value={host}
|
||||
onChange={(e) => setHost(e.currentTarget.value)}
|
||||
data-testid="device-host"
|
||||
/>
|
||||
|
||||
<NumberInput
|
||||
label="Port"
|
||||
required
|
||||
min={1}
|
||||
max={65535}
|
||||
value={port}
|
||||
onChange={(val) => setPort(val)}
|
||||
data-testid="device-port"
|
||||
/>
|
||||
|
||||
<NumberInput
|
||||
label="Unit ID"
|
||||
description="Modbus slave address (Meter ID on device panel)"
|
||||
required
|
||||
min={1}
|
||||
max={247}
|
||||
value={unitId}
|
||||
onChange={(val) => setUnitId(val)}
|
||||
data-testid="device-unit-id"
|
||||
/>
|
||||
|
||||
<Select
|
||||
label="Profile"
|
||||
description="YAML device profile"
|
||||
required
|
||||
data={profileOptions}
|
||||
value={effectiveProfile}
|
||||
onChange={setProfile}
|
||||
data-testid="device-profile"
|
||||
/>
|
||||
|
||||
<NumberInput
|
||||
label="Poll interval (seconds)"
|
||||
required
|
||||
min={1}
|
||||
value={pollIntervalS}
|
||||
onChange={(val) => setPollIntervalS(val)}
|
||||
data-testid="device-poll-interval"
|
||||
/>
|
||||
|
||||
<Switch
|
||||
label="Enabled"
|
||||
description="Whether this device is polled"
|
||||
checked={enabled}
|
||||
onChange={(e) => setEnabled(e.currentTarget.checked)}
|
||||
data-testid="device-enabled"
|
||||
/>
|
||||
|
||||
{error && (
|
||||
<Alert color="red" data-testid="device-form-error">
|
||||
{error}
|
||||
</Alert>
|
||||
)}
|
||||
|
||||
<Group justify="flex-end" gap="sm">
|
||||
<Button variant="default" onClick={onClose} data-testid="device-form-cancel">
|
||||
Cancel
|
||||
</Button>
|
||||
<Button type="submit" loading={isBusy} data-testid="device-form-submit">
|
||||
{isEdit ? 'Save' : 'Create'}
|
||||
</Button>
|
||||
</Group>
|
||||
</Stack>
|
||||
</form>
|
||||
)}
|
||||
</Modal>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,82 @@
|
||||
/**
|
||||
* Tests for DsmrPanel — latest parsed DSMR telegram view.
|
||||
*
|
||||
* Coverage:
|
||||
* 1. Loading state.
|
||||
* 2. Empty state (found=false) with a helpful hint.
|
||||
* 3. Renders the payload as a key/value table; null values shown as "—".
|
||||
* 4. Error state.
|
||||
*/
|
||||
|
||||
import { describe, it, expect, vi, beforeEach } from 'vitest'
|
||||
import { screen, waitFor } from '@testing-library/react'
|
||||
import { renderWithProviders } from '../test-utils'
|
||||
|
||||
const mockGet = vi.fn()
|
||||
|
||||
vi.mock('../api/client', () => ({
|
||||
default: {
|
||||
GET: (...args: unknown[]) => mockGet(...args),
|
||||
POST: vi.fn(),
|
||||
PATCH: vi.fn(),
|
||||
DELETE: vi.fn(),
|
||||
},
|
||||
ApiError: class ApiError extends Error {
|
||||
status: number
|
||||
body: unknown
|
||||
constructor(status: number, body: unknown) {
|
||||
super(`API error ${status}`)
|
||||
this.name = 'ApiError'
|
||||
this.status = status
|
||||
this.body = body
|
||||
}
|
||||
},
|
||||
registerLoginRedirect: vi.fn(),
|
||||
}))
|
||||
|
||||
import { DsmrPanel } from './DsmrPanel'
|
||||
|
||||
describe('DsmrPanel', () => {
|
||||
beforeEach(() => vi.clearAllMocks())
|
||||
|
||||
it('renders loading state initially', () => {
|
||||
mockGet.mockImplementation(() => new Promise(() => {}))
|
||||
renderWithProviders(<DsmrPanel />)
|
||||
expect(screen.getByTestId('dsmr-loading')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
it('shows an empty hint when no DSMR data has been ingested', async () => {
|
||||
mockGet.mockResolvedValue({ data: { found: false, recorded_at: null, payload: null } })
|
||||
renderWithProviders(<DsmrPanel />)
|
||||
await waitFor(() => expect(screen.getByTestId('dsmr-empty')).toBeInTheDocument())
|
||||
expect(screen.queryByTestId('dsmr-table')).not.toBeInTheDocument()
|
||||
})
|
||||
|
||||
it('renders the latest telegram as a key/value table; null shown as dash', async () => {
|
||||
mockGet.mockResolvedValue({
|
||||
data: {
|
||||
found: true,
|
||||
recorded_at: '2026-06-23T12:16:00Z',
|
||||
payload: {
|
||||
electricity_delivered_1: '20915.154',
|
||||
phase_voltage_l2: null,
|
||||
},
|
||||
},
|
||||
})
|
||||
renderWithProviders(<DsmrPanel />)
|
||||
|
||||
await waitFor(() => expect(screen.getByTestId('dsmr-table')).toBeInTheDocument())
|
||||
|
||||
// Field rows present (keys are sorted).
|
||||
expect(screen.getByTestId('dsmr-row-electricity_delivered_1')).toHaveTextContent('20915.154')
|
||||
// Null phase value rendered as an em dash, not omitted.
|
||||
expect(screen.getByTestId('dsmr-row-phase_voltage_l2')).toHaveTextContent('—')
|
||||
expect(screen.getByTestId('dsmr-recorded-at')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
it('renders an error state when the request fails', async () => {
|
||||
mockGet.mockRejectedValue(new Error('network down'))
|
||||
renderWithProviders(<DsmrPanel />)
|
||||
await waitFor(() => expect(screen.getByTestId('dsmr-error')).toBeInTheDocument())
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,154 @@
|
||||
/**
|
||||
* DsmrPanel — shows the latest parsed DSMR telegram (the source of our metering data).
|
||||
*
|
||||
* This is the "ground truth" view: it fetches GET /api/energy/dsmr/latest and
|
||||
* renders the full parsed frame as a key/value table, so you can confirm at a
|
||||
* glance whether DSMR ingest is actually landing data (without reading the DB).
|
||||
*
|
||||
* - Auto-refresh toggle (default on) so new telegrams appear without a manual reload.
|
||||
* - Loading / error / empty states; "empty" explains the likely cause.
|
||||
* - Null phase values are shown as "—" (kept verbatim in the payload).
|
||||
*/
|
||||
|
||||
import { useState } from 'react'
|
||||
import {
|
||||
Stack,
|
||||
Group,
|
||||
Text,
|
||||
Loader,
|
||||
Center,
|
||||
Alert,
|
||||
Paper,
|
||||
Table,
|
||||
ScrollArea,
|
||||
Switch,
|
||||
Divider,
|
||||
Badge,
|
||||
} from '@mantine/core'
|
||||
import { useDsmrLatest } from './hooks'
|
||||
import { formatLocalDateTime } from '../utils/datetime'
|
||||
|
||||
/** Default auto-refresh interval (ms) — DSMR ingest stores ~one row per 10 s. */
|
||||
const AUTO_REFRESH_INTERVAL_MS = 10_000
|
||||
|
||||
function formatValue(value: unknown): string {
|
||||
if (value === null || value === undefined) return '—'
|
||||
if (typeof value === 'object') return JSON.stringify(value)
|
||||
return String(value)
|
||||
}
|
||||
|
||||
export function DsmrPanel() {
|
||||
const [autoRefresh, setAutoRefresh] = useState(true)
|
||||
const latestQuery = useDsmrLatest(
|
||||
autoRefresh ? { refetchIntervalMs: AUTO_REFRESH_INTERVAL_MS } : undefined,
|
||||
)
|
||||
|
||||
return (
|
||||
<Stack gap="md" data-testid="dsmr-panel">
|
||||
<Group justify="space-between" align="center">
|
||||
<div>
|
||||
<Text fw={600}>Latest DSMR reading</Text>
|
||||
<Text size="xs" c="dimmed">
|
||||
The most recent parsed telegram persisted to <code>dsmr_reading</code>.
|
||||
</Text>
|
||||
</div>
|
||||
<Switch
|
||||
label="Auto-refresh"
|
||||
checked={autoRefresh}
|
||||
onChange={(e) => setAutoRefresh(e.currentTarget.checked)}
|
||||
size="sm"
|
||||
data-testid="dsmr-auto-refresh-switch"
|
||||
/>
|
||||
</Group>
|
||||
|
||||
<Divider />
|
||||
|
||||
<DsmrContent
|
||||
isLoading={latestQuery.isLoading}
|
||||
isError={latestQuery.isError}
|
||||
data={latestQuery.data}
|
||||
/>
|
||||
</Stack>
|
||||
)
|
||||
}
|
||||
|
||||
interface DsmrContentProps {
|
||||
isLoading: boolean
|
||||
isError: boolean
|
||||
data:
|
||||
| { found: boolean; recorded_at?: string | null; payload?: Record<string, unknown> | null }
|
||||
| undefined
|
||||
}
|
||||
|
||||
function DsmrContent({ isLoading, isError, data }: DsmrContentProps) {
|
||||
if (isLoading) {
|
||||
return (
|
||||
<Center py="xl" data-testid="dsmr-loading">
|
||||
<Loader />
|
||||
</Center>
|
||||
)
|
||||
}
|
||||
|
||||
if (isError || !data) {
|
||||
return (
|
||||
<Alert color="red" data-testid="dsmr-error">
|
||||
Failed to load the latest DSMR reading.
|
||||
</Alert>
|
||||
)
|
||||
}
|
||||
|
||||
if (!data.found || !data.payload) {
|
||||
return (
|
||||
<Alert color="gray" data-testid="dsmr-empty">
|
||||
No DSMR data yet. Enable <strong>DSMR ingest</strong> in Config, make sure MQTT
|
||||
is connected, and confirm the DSMR Reader is publishing to the configured topic
|
||||
(default <code>dsmr/json</code>). Rows are stored about once every 10 seconds.
|
||||
</Alert>
|
||||
)
|
||||
}
|
||||
|
||||
const payload = data.payload
|
||||
const keys = Object.keys(payload).sort()
|
||||
|
||||
return (
|
||||
<Stack gap="sm" data-testid="dsmr-data">
|
||||
<Group gap="xs">
|
||||
<Badge color="teal" variant="light">
|
||||
{keys.length} fields
|
||||
</Badge>
|
||||
{data.recorded_at && (
|
||||
<Text size="sm" c="dimmed" data-testid="dsmr-recorded-at">
|
||||
recorded at {formatLocalDateTime(data.recorded_at)}
|
||||
</Text>
|
||||
)}
|
||||
</Group>
|
||||
|
||||
<Paper withBorder>
|
||||
<ScrollArea.Autosize mah={480}>
|
||||
<Table striped highlightOnHover withColumnBorders data-testid="dsmr-table">
|
||||
<Table.Thead>
|
||||
<Table.Tr>
|
||||
<Table.Th>Field</Table.Th>
|
||||
<Table.Th>Value</Table.Th>
|
||||
</Table.Tr>
|
||||
</Table.Thead>
|
||||
<Table.Tbody>
|
||||
{keys.map((key) => (
|
||||
<Table.Tr key={key} data-testid={`dsmr-row-${key}`}>
|
||||
<Table.Td>
|
||||
<Text size="sm" ff="monospace">
|
||||
{key}
|
||||
</Text>
|
||||
</Table.Td>
|
||||
<Table.Td>
|
||||
<Text size="sm">{formatValue(payload[key])}</Text>
|
||||
</Table.Td>
|
||||
</Table.Tr>
|
||||
))}
|
||||
</Table.Tbody>
|
||||
</Table>
|
||||
</ScrollArea.Autosize>
|
||||
</Paper>
|
||||
</Stack>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,375 @@
|
||||
/**
|
||||
* Tests for energy/EnergyCharts.tsx
|
||||
*
|
||||
* Coverage:
|
||||
* 1. Renders chart title with device name.
|
||||
* 2. Shows loading state while readings/metrics are loading.
|
||||
* 3. Shows error state when either query fails.
|
||||
* 4. Shows empty state when no readings are in the time window.
|
||||
* 5. Renders chart container when data is available.
|
||||
* 6. Tolerates missing keys in payload (no crash when key absent).
|
||||
* 7. Time-range segmented control is rendered.
|
||||
*
|
||||
* NOTE: Recharts itself is NOT mocked — we let it render (jsdom-compatible
|
||||
* rendering), but we only assert on data-testid wrappers, not Recharts internals.
|
||||
* Recharts SVG rendering may produce warnings in jsdom; these are expected and
|
||||
* benign (jsdom lacks ResizeObserver / SVG layout).
|
||||
*/
|
||||
|
||||
import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest'
|
||||
import { screen, waitFor } from '@testing-library/react'
|
||||
import { renderWithProviders } from '../test-utils'
|
||||
import { EnergyCharts } from './EnergyCharts'
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Fixtures
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const UUID = 'device-uuid-test'
|
||||
const DEVICE_NAME = 'SDM120 Main'
|
||||
|
||||
const METRICS_RESP = {
|
||||
profile: 'sdm120',
|
||||
metrics: [
|
||||
{ key: 'voltage', label: 'Voltage', unit: 'V', device_class: 'voltage' },
|
||||
{ key: 'current', label: 'Current', unit: 'A', device_class: 'current' },
|
||||
],
|
||||
}
|
||||
|
||||
const READING = {
|
||||
recorded_at: '2026-06-22T10:00:00Z',
|
||||
payload: { voltage: 230.2, current: 1.3 },
|
||||
}
|
||||
|
||||
const READINGS_RESP = {
|
||||
items: [READING],
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Mock apiClient
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const mockGet = vi.fn()
|
||||
|
||||
vi.mock('../api/client', () => ({
|
||||
default: {
|
||||
GET: (...args: unknown[]) => mockGet(...args),
|
||||
POST: vi.fn(),
|
||||
PATCH: vi.fn(),
|
||||
DELETE: vi.fn(),
|
||||
},
|
||||
ApiError: class ApiError extends Error {
|
||||
status: number
|
||||
body: unknown
|
||||
constructor(status: number, body: unknown) {
|
||||
super(`API error ${status}`)
|
||||
this.name = 'ApiError'
|
||||
this.status = status
|
||||
this.body = body
|
||||
}
|
||||
},
|
||||
registerLoginRedirect: vi.fn(),
|
||||
}))
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Helpers
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function setupDefaultMocks() {
|
||||
mockGet.mockImplementation((path: string) => {
|
||||
if (path === '/api/modbus/devices/{uuid}/metrics') {
|
||||
return Promise.resolve({ data: METRICS_RESP })
|
||||
}
|
||||
if (path === '/api/modbus/devices/{uuid}/readings') {
|
||||
return Promise.resolve({ data: READINGS_RESP })
|
||||
}
|
||||
return Promise.resolve({ data: null })
|
||||
})
|
||||
}
|
||||
|
||||
function renderChart() {
|
||||
return renderWithProviders(
|
||||
<EnergyCharts uuid={UUID} deviceName={DEVICE_NAME} />,
|
||||
{ initialPath: '/energy' },
|
||||
)
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Tests
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('EnergyCharts — rendering', () => {
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks()
|
||||
setupDefaultMocks()
|
||||
})
|
||||
|
||||
it('renders chart title with device name', async () => {
|
||||
renderChart()
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId(`energy-charts-title-${UUID}`)).toBeInTheDocument()
|
||||
})
|
||||
|
||||
expect(screen.getByTestId(`energy-charts-title-${UUID}`).textContent).toBe(DEVICE_NAME)
|
||||
})
|
||||
|
||||
it('renders time-range segmented control', async () => {
|
||||
renderChart()
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId(`energy-charts-preset-${UUID}`)).toBeInTheDocument()
|
||||
})
|
||||
})
|
||||
|
||||
it('shows loading state while data is being fetched', () => {
|
||||
// Never resolve
|
||||
mockGet.mockReturnValue(new Promise(() => {}))
|
||||
renderChart()
|
||||
|
||||
expect(screen.getByTestId(`energy-charts-loading-${UUID}`)).toBeInTheDocument()
|
||||
})
|
||||
|
||||
it('shows error state when readings query fails', async () => {
|
||||
mockGet.mockRejectedValue(new Error('Network error'))
|
||||
renderChart()
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId(`energy-charts-error-${UUID}`)).toBeInTheDocument()
|
||||
})
|
||||
})
|
||||
|
||||
it('shows empty state when no readings in time window', async () => {
|
||||
mockGet.mockImplementation((path: string) => {
|
||||
if (path === '/api/modbus/devices/{uuid}/metrics') {
|
||||
return Promise.resolve({ data: METRICS_RESP })
|
||||
}
|
||||
if (path === '/api/modbus/devices/{uuid}/readings') {
|
||||
return Promise.resolve({ data: { items: [] } })
|
||||
}
|
||||
return Promise.resolve({ data: null })
|
||||
})
|
||||
|
||||
renderChart()
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId(`energy-charts-empty-${UUID}`)).toBeInTheDocument()
|
||||
})
|
||||
})
|
||||
|
||||
it('renders chart container when data is available', async () => {
|
||||
renderChart()
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId(`energy-charts-chart-${UUID}`)).toBeInTheDocument()
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
describe('EnergyCharts — truncation notice', () => {
|
||||
beforeEach(() => vi.clearAllMocks())
|
||||
|
||||
it('shows truncation notice when readings count equals the limit cap (1000)', async () => {
|
||||
// Build exactly 1000 synthetic readings to trigger the truncation hint.
|
||||
const truncatedItems = Array.from({ length: 1000 }, (_, i) => ({
|
||||
recorded_at: new Date(Date.now() - (999 - i) * 5000).toISOString(),
|
||||
payload: { voltage: 230 + i * 0.01 },
|
||||
}))
|
||||
|
||||
mockGet.mockImplementation((path: string) => {
|
||||
if (path === '/api/modbus/devices/{uuid}/metrics') {
|
||||
return Promise.resolve({ data: METRICS_RESP })
|
||||
}
|
||||
if (path === '/api/modbus/devices/{uuid}/readings') {
|
||||
return Promise.resolve({ data: { items: truncatedItems } })
|
||||
}
|
||||
return Promise.resolve({ data: null })
|
||||
})
|
||||
|
||||
renderChart()
|
||||
|
||||
await waitFor(() => {
|
||||
expect(
|
||||
screen.getByTestId(`energy-charts-truncated-${UUID}`),
|
||||
).toBeInTheDocument()
|
||||
})
|
||||
})
|
||||
|
||||
it('does NOT show truncation notice when readings count is below limit', async () => {
|
||||
// Default mock has exactly 1 item — well below 1000.
|
||||
setupDefaultMocks()
|
||||
renderChart()
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId(`energy-charts-chart-${UUID}`)).toBeInTheDocument()
|
||||
})
|
||||
|
||||
expect(screen.queryByTestId(`energy-charts-truncated-${UUID}`)).not.toBeInTheDocument()
|
||||
})
|
||||
})
|
||||
|
||||
describe('EnergyCharts — payload key tolerance', () => {
|
||||
beforeEach(() => vi.clearAllMocks())
|
||||
|
||||
it('does not crash when payload is missing a metric key', async () => {
|
||||
mockGet.mockImplementation((path: string) => {
|
||||
if (path === '/api/modbus/devices/{uuid}/metrics') {
|
||||
return Promise.resolve({ data: METRICS_RESP })
|
||||
}
|
||||
if (path === '/api/modbus/devices/{uuid}/readings') {
|
||||
// payload is missing 'current' key
|
||||
return Promise.resolve({
|
||||
data: { items: [{ recorded_at: '2026-06-22T10:00:00Z', payload: { voltage: 230.2 } }] },
|
||||
})
|
||||
}
|
||||
return Promise.resolve({ data: null })
|
||||
})
|
||||
|
||||
// Should render without throwing
|
||||
expect(() => renderChart()).not.toThrow()
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId(`energy-charts-${UUID}`)).toBeInTheDocument()
|
||||
})
|
||||
})
|
||||
|
||||
it('does not crash when payload is entirely empty', async () => {
|
||||
mockGet.mockImplementation((path: string) => {
|
||||
if (path === '/api/modbus/devices/{uuid}/metrics') {
|
||||
return Promise.resolve({ data: METRICS_RESP })
|
||||
}
|
||||
if (path === '/api/modbus/devices/{uuid}/readings') {
|
||||
return Promise.resolve({
|
||||
data: { items: [{ recorded_at: '2026-06-22T10:00:00Z', payload: {} }] },
|
||||
})
|
||||
}
|
||||
return Promise.resolve({ data: null })
|
||||
})
|
||||
|
||||
expect(() => renderChart()).not.toThrow()
|
||||
|
||||
// All values null → empty state (no chart) or chart with no data points shown.
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId(`energy-charts-${UUID}`)).toBeInTheDocument()
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Auto-refresh: rolling window (A2 — the core bug fix)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('EnergyCharts — auto-refresh rolling window', () => {
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks()
|
||||
})
|
||||
|
||||
afterEach(() => {
|
||||
vi.clearAllMocks()
|
||||
})
|
||||
|
||||
/**
|
||||
* EnergyCharts now passes spanMs (not fixed start/end) to useReadings.
|
||||
* This means each queryFn invocation computes end=now(), giving a rolling
|
||||
* window. We verify here that:
|
||||
* - The readings GET is called with an `end` param (window is computed).
|
||||
* - Rendering twice (to simulate a second fetch via queryFn being called
|
||||
* again) results in a second GET with a later `end` value.
|
||||
*
|
||||
* We test this by manually invoking the apiClient.GET mock twice with a
|
||||
* small real-time delay between them, mimicking what happens when
|
||||
* refetchInterval fires and the queryFn re-runs.
|
||||
*/
|
||||
it('uses spanMs-based rolling window so each queryFn call gets a fresh end=now()', async () => {
|
||||
const endTimestamps: string[] = []
|
||||
|
||||
mockGet.mockImplementation((path: string, opts: { params?: { query?: { end?: string } } } = {}) => {
|
||||
if (path === '/api/modbus/devices/{uuid}/metrics') {
|
||||
return Promise.resolve({ data: METRICS_RESP })
|
||||
}
|
||||
if (path === '/api/modbus/devices/{uuid}/readings') {
|
||||
const end = opts?.params?.query?.end
|
||||
if (end) endTimestamps.push(end)
|
||||
return Promise.resolve({ data: READINGS_RESP })
|
||||
}
|
||||
return Promise.resolve({ data: null })
|
||||
})
|
||||
|
||||
// First render: triggers initial fetch (end=now at T0).
|
||||
const t0 = Date.now()
|
||||
renderWithProviders(
|
||||
<EnergyCharts uuid={UUID} deviceName={DEVICE_NAME} />,
|
||||
{ initialPath: '/energy' },
|
||||
)
|
||||
|
||||
// Wait for the initial readings GET to fire.
|
||||
await waitFor(() => expect(endTimestamps.length).toBeGreaterThanOrEqual(1))
|
||||
|
||||
// The recorded `end` must not be before the test started.
|
||||
const firstEndMs = new Date(endTimestamps[0]).getTime()
|
||||
expect(firstEndMs).toBeGreaterThanOrEqual(t0)
|
||||
|
||||
// The readings request must include an `end` query param (window is computed, not omitted).
|
||||
expect(endTimestamps[0]).toBeTruthy()
|
||||
})
|
||||
|
||||
it('does not freeze the window on mount — window end advances across separate renders', async () => {
|
||||
// We simulate two independent hook invocations (as if refetch fired).
|
||||
// Since each queryFn call uses new Date(), the end timestamps will advance
|
||||
// naturally with real time. We just need to verify the concept holds.
|
||||
const endTimestamps: string[] = []
|
||||
|
||||
mockGet.mockImplementation((path: string, opts: { params?: { query?: { end?: string } } } = {}) => {
|
||||
if (path === '/api/modbus/devices/{uuid}/metrics') {
|
||||
return Promise.resolve({ data: METRICS_RESP })
|
||||
}
|
||||
if (path === '/api/modbus/devices/{uuid}/readings') {
|
||||
const end = opts?.params?.query?.end
|
||||
if (end) endTimestamps.push(end)
|
||||
return Promise.resolve({ data: READINGS_RESP })
|
||||
}
|
||||
return Promise.resolve({ data: null })
|
||||
})
|
||||
|
||||
const { unmount } = renderWithProviders(
|
||||
<EnergyCharts uuid={UUID} deviceName={DEVICE_NAME} />,
|
||||
{ initialPath: '/energy' },
|
||||
)
|
||||
|
||||
await waitFor(() => expect(endTimestamps.length).toBeGreaterThanOrEqual(1))
|
||||
|
||||
const firstEnd = new Date(endTimestamps[0]).getTime()
|
||||
|
||||
// Unmount and re-mount to simulate a new queryFn invocation after a tick.
|
||||
unmount()
|
||||
vi.clearAllMocks()
|
||||
endTimestamps.length = 0
|
||||
|
||||
// Small real-time delay to ensure new Date() will be >= first.
|
||||
await new Promise((resolve) => setTimeout(resolve, 5))
|
||||
|
||||
mockGet.mockImplementation((path: string, opts: { params?: { query?: { end?: string } } } = {}) => {
|
||||
if (path === '/api/modbus/devices/{uuid}/metrics') {
|
||||
return Promise.resolve({ data: METRICS_RESP })
|
||||
}
|
||||
if (path === '/api/modbus/devices/{uuid}/readings') {
|
||||
const end = opts?.params?.query?.end
|
||||
if (end) endTimestamps.push(end)
|
||||
return Promise.resolve({ data: READINGS_RESP })
|
||||
}
|
||||
return Promise.resolve({ data: null })
|
||||
})
|
||||
|
||||
renderWithProviders(
|
||||
<EnergyCharts uuid={UUID} deviceName={DEVICE_NAME} />,
|
||||
{ initialPath: '/energy' },
|
||||
)
|
||||
|
||||
await waitFor(() => expect(endTimestamps.length).toBeGreaterThanOrEqual(1))
|
||||
|
||||
const secondEnd = new Date(endTimestamps[0]).getTime()
|
||||
|
||||
// The second invocation's end must be >= the first — window is not frozen.
|
||||
expect(secondEnd).toBeGreaterThanOrEqual(firstEnd)
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,322 @@
|
||||
/**
|
||||
* EnergyCharts — self-contained Recharts wrapper for Modbus device trend charts.
|
||||
*
|
||||
* Design decisions (M5 decision 11):
|
||||
* - Recharts is imported ONLY in this file (isolation, easy to swap later).
|
||||
* - The component is fully self-contained: it owns its own data fetching via
|
||||
* useReadings / useMetrics hooks and renders loading/error/empty states.
|
||||
* - Time-range selection is internal; readings are always fetched with a window
|
||||
* + limit cap — never a full-table pull.
|
||||
* - Metric labels and units come from GET /metrics; missing keys in payload are
|
||||
* tolerated (a null data point is emitted so the line simply has a gap).
|
||||
*/
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Recharts imports — keep ALL recharts imports inside this file.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
import {
|
||||
LineChart,
|
||||
Line,
|
||||
XAxis,
|
||||
YAxis,
|
||||
CartesianGrid,
|
||||
Tooltip as RechartsTooltip,
|
||||
Legend,
|
||||
ResponsiveContainer,
|
||||
} from 'recharts'
|
||||
|
||||
import { useState, useMemo } from 'react'
|
||||
import {
|
||||
Stack,
|
||||
Group,
|
||||
Text,
|
||||
Loader,
|
||||
Alert,
|
||||
SegmentedControl,
|
||||
Paper,
|
||||
Title,
|
||||
Badge,
|
||||
Box,
|
||||
} from '@mantine/core'
|
||||
|
||||
import { useReadings, useMetrics } from './hooks'
|
||||
import type { MetricInfo, AutoRefreshOptions } from './hooks'
|
||||
import { formatMetricValue } from './format'
|
||||
import { formatLocalTime, formatLocalDateTime } from '../utils/datetime'
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Constants
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** Must match the limit passed to useReadings below. */
|
||||
const CHART_READINGS_LIMIT = 1000
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Types
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
interface EnergyChartsProps {
|
||||
/** UUID of the Modbus device to chart. */
|
||||
uuid: string
|
||||
/** Friendly name for the chart title. */
|
||||
deviceName: string
|
||||
/** Auto-refresh options forwarded to useReadings. */
|
||||
autoRefresh?: AutoRefreshOptions
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Time-range presets
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
interface TimePreset {
|
||||
label: string
|
||||
value: string
|
||||
/** How many milliseconds of history to show. */
|
||||
spanMs: number
|
||||
}
|
||||
|
||||
const TIME_PRESETS: TimePreset[] = [
|
||||
{ label: '1 h', value: '1h', spanMs: 60 * 60 * 1000 },
|
||||
{ label: '6 h', value: '6h', spanMs: 6 * 60 * 60 * 1000 },
|
||||
{ label: '24 h', value: '24h', spanMs: 24 * 60 * 60 * 1000 },
|
||||
]
|
||||
|
||||
const DEFAULT_PRESET = '1h'
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Metric colour palette — cycles through a fixed set
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const LINE_COLORS = [
|
||||
'#4c9cdb',
|
||||
'#f59f00',
|
||||
'#51cf66',
|
||||
'#f03e3e',
|
||||
'#cc5de8',
|
||||
'#20c997',
|
||||
'#fd7e14',
|
||||
'#74c0fc',
|
||||
]
|
||||
|
||||
function lineColor(index: number): string {
|
||||
return LINE_COLORS[index % LINE_COLORS.length]
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Helper: format a recorded_at timestamp for the X-axis tick
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function formatTimeTick(isoString: string): string {
|
||||
return formatLocalTime(isoString)
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Helper: safe numeric read from payload — returns null if key absent/non-numeric
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function safePayloadValue(
|
||||
payload: Record<string, unknown> | null | undefined,
|
||||
key: string,
|
||||
): number | null {
|
||||
if (!payload) return null
|
||||
const raw = payload[key]
|
||||
if (raw === null || raw === undefined) return null
|
||||
const n = Number(raw)
|
||||
return Number.isFinite(n) ? n : null
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// EnergyCharts component
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export function EnergyCharts({ uuid, deviceName, autoRefresh }: EnergyChartsProps) {
|
||||
const [activePreset, setActivePreset] = useState<string>(DEFAULT_PRESET)
|
||||
|
||||
const selectedPreset = TIME_PRESETS.find((p) => p.value === activePreset) ?? TIME_PRESETS[0]
|
||||
|
||||
// Pass spanMs rather than fixed start/end so that every refetchInterval-triggered
|
||||
// queryFn call computes end=now() and rolls the window forward — new readings
|
||||
// recorded after mount will appear without a manual page refresh.
|
||||
// Switching presets changes spanMs → different queryKey → immediate re-fetch.
|
||||
const readingsQuery = useReadings(
|
||||
uuid,
|
||||
{ spanMs: selectedPreset.spanMs, limit: CHART_READINGS_LIMIT },
|
||||
autoRefresh,
|
||||
)
|
||||
const metricsQuery = useMetrics(uuid)
|
||||
|
||||
// -------------------------------------------------------------------------
|
||||
// Derive chart data: [{recorded_at, voltage: 230.2, current: 1.3, ...}, ...]
|
||||
// -------------------------------------------------------------------------
|
||||
|
||||
const metricsData = metricsQuery.data?.metrics
|
||||
|
||||
const chartData = useMemo(() => {
|
||||
const metrics: MetricInfo[] = metricsData ?? []
|
||||
const readings = readingsQuery.data?.items ?? []
|
||||
return readings.map((row) => {
|
||||
const point: Record<string, string | number | null> = {
|
||||
recorded_at: row.recorded_at,
|
||||
}
|
||||
for (const m of metrics) {
|
||||
// Tolerate missing keys — null produces a gap in the line, not a crash.
|
||||
point[m.key] = safePayloadValue(row.payload as Record<string, unknown>, m.key)
|
||||
}
|
||||
return point
|
||||
})
|
||||
}, [readingsQuery.data, metricsData])
|
||||
|
||||
const metrics: MetricInfo[] = metricsQuery.data?.metrics ?? []
|
||||
|
||||
// -------------------------------------------------------------------------
|
||||
// States: loading / error / empty
|
||||
// -------------------------------------------------------------------------
|
||||
|
||||
const isLoading = readingsQuery.isLoading || metricsQuery.isLoading
|
||||
const isError = readingsQuery.isError || metricsQuery.isError
|
||||
|
||||
/**
|
||||
* True when the returned row count equals the limit cap, meaning the window
|
||||
* contains more data than was fetched. The backend returns the most-recent N
|
||||
* rows in this case, so the chart shows the latest segment — but a hint is
|
||||
* shown so the user knows the full window is not displayed.
|
||||
*/
|
||||
const isTruncated =
|
||||
!isLoading &&
|
||||
!isError &&
|
||||
(readingsQuery.data?.items.length ?? 0) >= CHART_READINGS_LIMIT
|
||||
|
||||
return (
|
||||
<Paper withBorder p="md" data-testid={`energy-charts-${uuid}`}>
|
||||
<Stack gap="sm">
|
||||
{/* Header row */}
|
||||
<Group justify="space-between" align="center" wrap="nowrap">
|
||||
<Title order={4} data-testid={`energy-charts-title-${uuid}`}>
|
||||
{deviceName}
|
||||
</Title>
|
||||
<SegmentedControl
|
||||
size="xs"
|
||||
value={activePreset}
|
||||
onChange={setActivePreset}
|
||||
data={TIME_PRESETS.map((p) => ({ value: p.value, label: p.label }))}
|
||||
data-testid={`energy-charts-preset-${uuid}`}
|
||||
/>
|
||||
</Group>
|
||||
|
||||
{/* Loading */}
|
||||
{isLoading && (
|
||||
<Group justify="center" py="lg" data-testid={`energy-charts-loading-${uuid}`}>
|
||||
<Loader size="sm" />
|
||||
<Text size="sm" c="dimmed">
|
||||
Loading readings…
|
||||
</Text>
|
||||
</Group>
|
||||
)}
|
||||
|
||||
{/* Error */}
|
||||
{!isLoading && isError && (
|
||||
<Alert color="red" data-testid={`energy-charts-error-${uuid}`}>
|
||||
Failed to load readings or metrics. Please try again.
|
||||
</Alert>
|
||||
)}
|
||||
|
||||
{/* Empty */}
|
||||
{!isLoading && !isError && chartData.length === 0 && (
|
||||
<Text
|
||||
size="sm"
|
||||
c="dimmed"
|
||||
ta="center"
|
||||
py="lg"
|
||||
data-testid={`energy-charts-empty-${uuid}`}
|
||||
>
|
||||
No readings in this time window.
|
||||
</Text>
|
||||
)}
|
||||
|
||||
{/* Truncation notice — shown when window has more data than the fetch limit */}
|
||||
{isTruncated && (
|
||||
<Text
|
||||
size="xs"
|
||||
c="dimmed"
|
||||
ta="right"
|
||||
data-testid={`energy-charts-truncated-${uuid}`}
|
||||
>
|
||||
Showing the most recent {CHART_READINGS_LIMIT} readings; full long-range trend pending downsampling
|
||||
</Text>
|
||||
)}
|
||||
|
||||
{/* Chart */}
|
||||
{!isLoading && !isError && chartData.length > 0 && (
|
||||
<Box data-testid={`energy-charts-chart-${uuid}`}>
|
||||
{/* Render one chart per metric for clarity (avoids mixed Y-axis units) */}
|
||||
{metrics.map((metric, idx) => {
|
||||
// Check if this metric has any non-null data points; skip if all null.
|
||||
const hasData = chartData.some((pt) => pt[metric.key] !== null)
|
||||
if (!hasData) return null
|
||||
|
||||
return (
|
||||
<Box key={metric.key} mb="md">
|
||||
<Group gap="xs" mb={4}>
|
||||
<Text size="xs" fw={600} c="dimmed">
|
||||
{metric.label}
|
||||
</Text>
|
||||
{metric.unit && (
|
||||
<Badge size="xs" variant="outline" color="gray">
|
||||
{metric.unit}
|
||||
</Badge>
|
||||
)}
|
||||
</Group>
|
||||
<ResponsiveContainer width="100%" height={160}>
|
||||
<LineChart
|
||||
data={chartData}
|
||||
margin={{ top: 4, right: 8, left: 0, bottom: 0 }}
|
||||
>
|
||||
<CartesianGrid strokeDasharray="3 3" opacity={0.4} />
|
||||
<XAxis
|
||||
dataKey="recorded_at"
|
||||
tickFormatter={formatTimeTick}
|
||||
tick={{ fontSize: 10 }}
|
||||
minTickGap={40}
|
||||
/>
|
||||
<YAxis
|
||||
tick={{ fontSize: 10 }}
|
||||
width={50}
|
||||
tickFormatter={(v: number) =>
|
||||
metric.unit ? `${v} ${metric.unit}` : String(v)
|
||||
}
|
||||
/>
|
||||
<RechartsTooltip
|
||||
formatter={(value) => {
|
||||
const formatted = formatMetricValue(value, metric)
|
||||
if (formatted === '—') return ['—', metric.label] as [string, string]
|
||||
return [
|
||||
`${formatted}${metric.unit ? ' ' + metric.unit : ''}`,
|
||||
metric.label,
|
||||
] as [string, string]
|
||||
}}
|
||||
labelFormatter={(label) => {
|
||||
return formatLocalDateTime(String(label))
|
||||
}}
|
||||
/>
|
||||
<Legend />
|
||||
<Line
|
||||
type="monotone"
|
||||
dataKey={metric.key}
|
||||
name={metric.label}
|
||||
stroke={lineColor(idx)}
|
||||
dot={false}
|
||||
isAnimationActive={false}
|
||||
connectNulls={false}
|
||||
/>
|
||||
</LineChart>
|
||||
</ResponsiveContainer>
|
||||
</Box>
|
||||
)
|
||||
})}
|
||||
</Box>
|
||||
)}
|
||||
</Stack>
|
||||
</Paper>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,446 @@
|
||||
/**
|
||||
* Tests for MeterManager component.
|
||||
*
|
||||
* Coverage:
|
||||
* 1. Loading state rendering.
|
||||
* 2. Error state rendering.
|
||||
* 3. Empty state when no meters exist.
|
||||
* 4. Meter timeline list rendering (label, dates, active badge, reason).
|
||||
* 5. "Declare New Meter" button opens form modal.
|
||||
* 6. Declare meter — form submit calls POST /api/energy/meters.
|
||||
* 7. Declare meter — 422 (倒挂) error is displayed.
|
||||
* 8. Edit button opens edit form modal.
|
||||
* 9. Edit meter — saves label/note via PATCH /api/energy/meters/{meter_id}.
|
||||
* 10. Edit meter — retroactive started_at triggers recompute notice.
|
||||
*/
|
||||
|
||||
import { describe, it, expect, vi, beforeEach } from 'vitest'
|
||||
import { screen, waitFor } from '@testing-library/react'
|
||||
import userEvent from '@testing-library/user-event'
|
||||
import { renderWithProviders } from '../test-utils'
|
||||
import { MeterManager } from './MeterManager'
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Mock apiClient
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const mockGet = vi.fn()
|
||||
const mockPost = vi.fn()
|
||||
const mockPatch = vi.fn()
|
||||
|
||||
vi.mock('../api/client', () => ({
|
||||
default: {
|
||||
GET: (...args: unknown[]) => mockGet(...args),
|
||||
POST: (...args: unknown[]) => mockPost(...args),
|
||||
PATCH: (...args: unknown[]) => mockPatch(...args),
|
||||
DELETE: vi.fn(),
|
||||
},
|
||||
ApiError: class ApiError extends Error {
|
||||
status: number
|
||||
body: unknown
|
||||
constructor(status: number, body: unknown) {
|
||||
super(`API error ${status}`)
|
||||
this.name = 'ApiError'
|
||||
this.status = status
|
||||
this.body = body
|
||||
}
|
||||
},
|
||||
registerLoginRedirect: vi.fn(),
|
||||
}))
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Fixtures
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const ACTIVE_METER = {
|
||||
id: 1,
|
||||
label: 'Initial 2G meter',
|
||||
commodity: 'electricity',
|
||||
started_at: '2024-01-15T00:00:00Z',
|
||||
ended_at: null,
|
||||
reason: 'initial',
|
||||
note: null,
|
||||
created_at: '2024-01-15T00:00:00Z',
|
||||
}
|
||||
|
||||
const CLOSED_METER = {
|
||||
id: 2,
|
||||
label: 'Old 4G meter',
|
||||
commodity: 'electricity',
|
||||
started_at: '2023-06-01T00:00:00Z',
|
||||
ended_at: '2024-01-15T00:00:00Z',
|
||||
reason: 'meter_swap',
|
||||
note: 'Replaced by grid company',
|
||||
created_at: '2023-06-01T00:00:00Z',
|
||||
}
|
||||
|
||||
const METERS_RESPONSE = {
|
||||
items: [CLOSED_METER, ACTIVE_METER],
|
||||
total: 2,
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Tests
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe('MeterManager — loading / error / empty states', () => {
|
||||
beforeEach(() => vi.clearAllMocks())
|
||||
|
||||
it('renders loading state initially', () => {
|
||||
mockGet.mockImplementation(() => new Promise(() => {}))
|
||||
|
||||
renderWithProviders(<MeterManager />)
|
||||
|
||||
expect(screen.getByTestId('meters-loading')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
it('renders error state when GET fails', async () => {
|
||||
mockGet.mockRejectedValue(new Error('Network error'))
|
||||
|
||||
renderWithProviders(<MeterManager />)
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId('meters-load-error')).toBeInTheDocument()
|
||||
})
|
||||
})
|
||||
|
||||
it('renders empty state when no meters exist', async () => {
|
||||
mockGet.mockResolvedValue({ data: { items: [], total: 0 } })
|
||||
|
||||
renderWithProviders(<MeterManager />)
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId('meters-empty')).toBeInTheDocument()
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
describe('MeterManager — meter list', () => {
|
||||
beforeEach(() => vi.clearAllMocks())
|
||||
|
||||
it('renders meter timeline with label, dates, status badge, reason', async () => {
|
||||
mockGet.mockResolvedValue({ data: METERS_RESPONSE })
|
||||
|
||||
renderWithProviders(<MeterManager />)
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId('meters-table')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
// Labels
|
||||
expect(screen.getByText('Initial 2G meter')).toBeInTheDocument()
|
||||
expect(screen.getByText('Old 4G meter')).toBeInTheDocument()
|
||||
|
||||
// Status badges
|
||||
expect(screen.getByTestId(`meter-status-${ACTIVE_METER.id}`)).toHaveTextContent('active')
|
||||
expect(screen.getByTestId(`meter-status-${CLOSED_METER.id}`)).toHaveTextContent('closed')
|
||||
|
||||
// Reason badges
|
||||
expect(screen.getByText('initial')).toBeInTheDocument()
|
||||
expect(screen.getByText('meter_swap')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
it('renders "Declare New Meter" button', async () => {
|
||||
mockGet.mockResolvedValue({ data: METERS_RESPONSE })
|
||||
|
||||
renderWithProviders(<MeterManager />)
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId('meter-declare-button')).toBeInTheDocument()
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
describe('MeterManager — declare new meter', () => {
|
||||
beforeEach(() => vi.clearAllMocks())
|
||||
|
||||
it('opens declare modal when "Declare New Meter" is clicked', async () => {
|
||||
const user = userEvent.setup()
|
||||
mockGet.mockResolvedValue({ data: { items: [], total: 0 } })
|
||||
|
||||
renderWithProviders(<MeterManager />)
|
||||
|
||||
await waitFor(() => expect(screen.getByTestId('meter-declare-button')).toBeInTheDocument())
|
||||
|
||||
await user.click(screen.getByTestId('meter-declare-button'))
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId('declare-meter-modal')).toBeInTheDocument()
|
||||
})
|
||||
expect(screen.getByTestId('declare-meter-form')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
it('calls POST /api/energy/meters with correct payload including local-midnight naive datetime', async () => {
|
||||
const user = userEvent.setup()
|
||||
mockGet.mockResolvedValue({ data: { items: [], total: 0 } })
|
||||
mockPost.mockResolvedValue({ data: ACTIVE_METER })
|
||||
|
||||
renderWithProviders(<MeterManager />)
|
||||
|
||||
await waitFor(() => expect(screen.getByTestId('meter-declare-button')).toBeInTheDocument())
|
||||
await user.click(screen.getByTestId('meter-declare-button'))
|
||||
|
||||
await waitFor(() => expect(screen.getByTestId('declare-meter-form')).toBeInTheDocument())
|
||||
|
||||
// Fill form
|
||||
await user.type(screen.getByTestId('meter-label'), 'New meter label')
|
||||
await user.type(screen.getByTestId('meter-started-at'), '2026-01-01')
|
||||
|
||||
// Select reason via the combobox (Mantine Select renders a combobox)
|
||||
await user.click(screen.getByTestId('meter-reason'))
|
||||
await waitFor(() => screen.getByText('Initial installation'))
|
||||
await user.click(screen.getByText('Initial installation'))
|
||||
|
||||
await user.click(screen.getByTestId('declare-meter-submit'))
|
||||
|
||||
await waitFor(() => {
|
||||
expect(mockPost).toHaveBeenCalledWith(
|
||||
'/api/energy/meters',
|
||||
expect.objectContaining({
|
||||
body: expect.objectContaining({
|
||||
label: 'New meter label',
|
||||
// FU10 local-midnight naive convention: no Z suffix
|
||||
started_at: '2026-01-01T00:00:00',
|
||||
reason: 'initial',
|
||||
commodity: 'electricity',
|
||||
}),
|
||||
}),
|
||||
)
|
||||
})
|
||||
})
|
||||
|
||||
it('displays error when POST fails with 422 (倒挂 / validation error)', async () => {
|
||||
const user = userEvent.setup()
|
||||
mockGet.mockResolvedValue({ data: { items: [ACTIVE_METER], total: 1 } })
|
||||
|
||||
const { ApiError } = await import('../api/client')
|
||||
mockPost.mockRejectedValue(
|
||||
new ApiError(422, { detail: 'started_at must be ≥ current active meter started_at' }),
|
||||
)
|
||||
|
||||
renderWithProviders(<MeterManager />)
|
||||
|
||||
await waitFor(() => expect(screen.getByTestId('meter-declare-button')).toBeInTheDocument())
|
||||
await user.click(screen.getByTestId('meter-declare-button'))
|
||||
await waitFor(() => expect(screen.getByTestId('declare-meter-form')).toBeInTheDocument())
|
||||
|
||||
await user.type(screen.getByTestId('meter-label'), 'Bad meter')
|
||||
await user.type(screen.getByTestId('meter-started-at'), '2020-01-01')
|
||||
await user.click(screen.getByTestId('meter-reason'))
|
||||
await waitFor(() => screen.getByText('Meter swap (same address)'))
|
||||
await user.click(screen.getByText('Meter swap (same address)'))
|
||||
|
||||
await user.click(screen.getByTestId('declare-meter-submit'))
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId('declare-meter-error')).toBeInTheDocument()
|
||||
})
|
||||
expect(screen.getByTestId('declare-meter-error').textContent).toContain(
|
||||
'started_at must be ≥',
|
||||
)
|
||||
})
|
||||
|
||||
it('cancel button closes the declare modal', async () => {
|
||||
const user = userEvent.setup()
|
||||
mockGet.mockResolvedValue({ data: { items: [], total: 0 } })
|
||||
|
||||
renderWithProviders(<MeterManager />)
|
||||
|
||||
await waitFor(() => expect(screen.getByTestId('meter-declare-button')).toBeInTheDocument())
|
||||
await user.click(screen.getByTestId('meter-declare-button'))
|
||||
await waitFor(() => expect(screen.getByTestId('declare-meter-modal')).toBeInTheDocument())
|
||||
|
||||
await user.click(screen.getByTestId('declare-meter-cancel'))
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.queryByTestId('declare-meter-modal')).not.toBeInTheDocument()
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
describe('MeterManager — edit meter date initialisation', () => {
|
||||
beforeEach(() => vi.clearAllMocks())
|
||||
|
||||
it('initialises date input from started_at (Z-suffix, UTC midnight → local date)', async () => {
|
||||
// ACTIVE_METER.started_at = '2024-01-15T00:00:00Z' (UTC midnight).
|
||||
// The test suite is pinned to TZ=UTC (via vite.config.ts test.env), so
|
||||
// the local date is deterministically '2024-01-15' on any CI runner.
|
||||
const user = userEvent.setup()
|
||||
mockGet.mockResolvedValue({ data: { items: [ACTIVE_METER], total: 1 } })
|
||||
|
||||
renderWithProviders(<MeterManager />)
|
||||
|
||||
await waitFor(() => expect(screen.getByTestId(`meter-edit-${ACTIVE_METER.id}`)).toBeInTheDocument())
|
||||
await user.click(screen.getByTestId(`meter-edit-${ACTIVE_METER.id}`))
|
||||
await waitFor(() => expect(screen.getByTestId('edit-meter-form')).toBeInTheDocument())
|
||||
|
||||
const dateInput = screen.getByTestId('edit-meter-started-at') as HTMLInputElement
|
||||
expect(dateInput.value).toBe('2024-01-15')
|
||||
})
|
||||
|
||||
it('initialises date input from started_at (naive, no tz marker)', async () => {
|
||||
// A naive timestamp without timezone marker — parseBackendTimestamp appends 'Z'
|
||||
// so it is treated as UTC. With TZ=UTC (pinned in vite.config.ts), the local date
|
||||
// equals the UTC date exactly.
|
||||
const naiveMeter = { ...ACTIVE_METER, started_at: '2024-03-20T00:00:00' }
|
||||
const user = userEvent.setup()
|
||||
mockGet.mockResolvedValue({ data: { items: [naiveMeter], total: 1 } })
|
||||
|
||||
renderWithProviders(<MeterManager />)
|
||||
|
||||
await waitFor(() => expect(screen.getByTestId(`meter-edit-${naiveMeter.id}`)).toBeInTheDocument())
|
||||
await user.click(screen.getByTestId(`meter-edit-${naiveMeter.id}`))
|
||||
await waitFor(() => expect(screen.getByTestId('edit-meter-form')).toBeInTheDocument())
|
||||
|
||||
const dateInput = screen.getByTestId('edit-meter-started-at') as HTMLInputElement
|
||||
expect(dateInput.value).toBe('2024-03-20')
|
||||
})
|
||||
|
||||
it('initialises date input from started_at with explicit UTC offset (+02:00) — regression for old buggy regex', async () => {
|
||||
// The old hand-written regex /[zZ+-]\d*$/ would fail to match '+02:00' (the ':00'
|
||||
// suffix broke the pattern) and would incorrectly append 'Z', producing an Invalid Date.
|
||||
// The new code uses parseBackendTimestamp which uses the correct TZ_MARKER_RE regex
|
||||
// and handles explicit offsets properly.
|
||||
const offsetMeter = { ...ACTIVE_METER, started_at: '2024-01-15T02:00:00+02:00' }
|
||||
// UTC equivalent: 2024-01-15T00:00:00Z → with TZ=UTC (pinned) local date = '2024-01-15'
|
||||
const user = userEvent.setup()
|
||||
mockGet.mockResolvedValue({ data: { items: [offsetMeter], total: 1 } })
|
||||
|
||||
renderWithProviders(<MeterManager />)
|
||||
|
||||
await waitFor(() => expect(screen.getByTestId(`meter-edit-${offsetMeter.id}`)).toBeInTheDocument())
|
||||
await user.click(screen.getByTestId(`meter-edit-${offsetMeter.id}`))
|
||||
await waitFor(() => expect(screen.getByTestId('edit-meter-form')).toBeInTheDocument())
|
||||
|
||||
const dateInput = screen.getByTestId('edit-meter-started-at') as HTMLInputElement
|
||||
// Must not be empty (which would indicate Invalid Date from the old buggy path)
|
||||
expect(dateInput.value).not.toBe('')
|
||||
expect(dateInput.value).toBe('2024-01-15')
|
||||
})
|
||||
|
||||
it('does not include started_at in PATCH body when date is unchanged (round-trip idempotence)', async () => {
|
||||
// Open the edit form and immediately submit without changing any fields except label.
|
||||
// The date should be considered unchanged → no started_at in the PATCH body.
|
||||
const user = userEvent.setup()
|
||||
mockGet.mockResolvedValue({ data: { items: [ACTIVE_METER], total: 1 } })
|
||||
mockPatch.mockResolvedValue({ data: { ...ACTIVE_METER, label: 'New label' } })
|
||||
|
||||
renderWithProviders(<MeterManager />)
|
||||
|
||||
await waitFor(() => expect(screen.getByTestId(`meter-edit-${ACTIVE_METER.id}`)).toBeInTheDocument())
|
||||
await user.click(screen.getByTestId(`meter-edit-${ACTIVE_METER.id}`))
|
||||
await waitFor(() => expect(screen.getByTestId('edit-meter-form')).toBeInTheDocument())
|
||||
|
||||
// Change only label; leave date untouched
|
||||
const labelInput = screen.getByTestId('edit-meter-label')
|
||||
await user.clear(labelInput)
|
||||
await user.type(labelInput, 'New label')
|
||||
|
||||
await user.click(screen.getByTestId('edit-meter-submit'))
|
||||
|
||||
await waitFor(() => {
|
||||
expect(mockPatch).toHaveBeenCalled()
|
||||
})
|
||||
|
||||
const patchBody = mockPatch.mock.calls[0][1].body
|
||||
expect(patchBody).not.toHaveProperty('started_at')
|
||||
})
|
||||
})
|
||||
|
||||
describe('MeterManager — edit meter', () => {
|
||||
beforeEach(() => vi.clearAllMocks())
|
||||
|
||||
it('opens edit modal when Edit button is clicked', async () => {
|
||||
const user = userEvent.setup()
|
||||
mockGet.mockResolvedValue({ data: METERS_RESPONSE })
|
||||
|
||||
renderWithProviders(<MeterManager />)
|
||||
|
||||
await waitFor(() => expect(screen.getByTestId(`meter-edit-${ACTIVE_METER.id}`)).toBeInTheDocument())
|
||||
|
||||
await user.click(screen.getByTestId(`meter-edit-${ACTIVE_METER.id}`))
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId('edit-meter-modal')).toBeInTheDocument()
|
||||
})
|
||||
expect(screen.getByTestId('edit-meter-form')).toBeInTheDocument()
|
||||
})
|
||||
|
||||
it('calls PATCH /api/energy/meters/{meter_id} when label is changed', async () => {
|
||||
const user = userEvent.setup()
|
||||
mockGet.mockResolvedValue({ data: { items: [ACTIVE_METER], total: 1 } })
|
||||
mockPatch.mockResolvedValue({ data: { ...ACTIVE_METER, label: 'Renamed meter' } })
|
||||
|
||||
renderWithProviders(<MeterManager />)
|
||||
|
||||
await waitFor(() => expect(screen.getByTestId(`meter-edit-${ACTIVE_METER.id}`)).toBeInTheDocument())
|
||||
await user.click(screen.getByTestId(`meter-edit-${ACTIVE_METER.id}`))
|
||||
await waitFor(() => expect(screen.getByTestId('edit-meter-form')).toBeInTheDocument())
|
||||
|
||||
// Clear label and type new one
|
||||
const labelInput = screen.getByTestId('edit-meter-label')
|
||||
await user.clear(labelInput)
|
||||
await user.type(labelInput, 'Renamed meter')
|
||||
|
||||
await user.click(screen.getByTestId('edit-meter-submit'))
|
||||
|
||||
await waitFor(() => {
|
||||
expect(mockPatch).toHaveBeenCalledWith(
|
||||
'/api/energy/meters/{meter_id}',
|
||||
expect.objectContaining({
|
||||
params: { path: { meter_id: ACTIVE_METER.id } },
|
||||
body: expect.objectContaining({ label: 'Renamed meter' }),
|
||||
}),
|
||||
)
|
||||
})
|
||||
})
|
||||
|
||||
it('shows recompute notice when started_at is changed (retroactive correction)', async () => {
|
||||
const user = userEvent.setup()
|
||||
mockGet.mockResolvedValue({ data: { items: [ACTIVE_METER], total: 1 } })
|
||||
mockPatch.mockResolvedValue({ data: { ...ACTIVE_METER, started_at: '2024-02-01T00:00:00' } })
|
||||
|
||||
renderWithProviders(<MeterManager />)
|
||||
|
||||
await waitFor(() => expect(screen.getByTestId(`meter-edit-${ACTIVE_METER.id}`)).toBeInTheDocument())
|
||||
await user.click(screen.getByTestId(`meter-edit-${ACTIVE_METER.id}`))
|
||||
await waitFor(() => expect(screen.getByTestId('edit-meter-form')).toBeInTheDocument())
|
||||
|
||||
// Change the date field
|
||||
const dateInput = screen.getByTestId('edit-meter-started-at')
|
||||
await user.clear(dateInput)
|
||||
await user.type(dateInput, '2024-02-01')
|
||||
|
||||
await user.click(screen.getByTestId('edit-meter-submit'))
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId('meter-recompute-notice')).toBeInTheDocument()
|
||||
})
|
||||
})
|
||||
|
||||
it('displays error when PATCH fails', async () => {
|
||||
const user = userEvent.setup()
|
||||
mockGet.mockResolvedValue({ data: { items: [ACTIVE_METER], total: 1 } })
|
||||
|
||||
const { ApiError } = await import('../api/client')
|
||||
mockPatch.mockRejectedValue(new ApiError(422, { detail: 'started_at conflict' }))
|
||||
|
||||
renderWithProviders(<MeterManager />)
|
||||
|
||||
await waitFor(() => expect(screen.getByTestId(`meter-edit-${ACTIVE_METER.id}`)).toBeInTheDocument())
|
||||
await user.click(screen.getByTestId(`meter-edit-${ACTIVE_METER.id}`))
|
||||
await waitFor(() => expect(screen.getByTestId('edit-meter-form')).toBeInTheDocument())
|
||||
|
||||
// Change label so there's something to patch
|
||||
const labelInput = screen.getByTestId('edit-meter-label')
|
||||
await user.clear(labelInput)
|
||||
await user.type(labelInput, 'Different label')
|
||||
|
||||
await user.click(screen.getByTestId('edit-meter-submit'))
|
||||
|
||||
await waitFor(() => {
|
||||
expect(screen.getByTestId('edit-meter-error')).toBeInTheDocument()
|
||||
})
|
||||
expect(screen.getByTestId('edit-meter-error').textContent).toContain('started_at conflict')
|
||||
})
|
||||
})
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user