Compare commits

..
18 Commits
Author SHA1 Message Date
tliu93 d7f04aee8c M8: finalize WarmteLink implementation plan
pytest / test (push) Successful in 2m22s
frontend / frontend (push) Successful in 29s
2026-08-22 21:23:19 +02:00
tliu93 8dbb59a3b7 PRE-M8-T03: finalize WarmteLink hardware validation
frontend / frontend (push) Successful in 30s
pytest / test (push) Successful in 2m23s
2026-08-22 19:29:54 +02:00
tliu93 2992bbb0ef PRE-M8-T02: add read-only WarmteLink serial probe 2026-08-22 18:06:15 +02:00
tliu93 22faeb45bb PRE-M8-T01: add WarmteLink P1 telegram parser 2026-08-22 17:47:30 +02:00
tliu93 c37dfacfc7 PRE-M8: record WarmteLink findings and implementation plan 2026-08-22 17:23:41 +02:00
tliu93 16b050d821 PRE-M8: add staged WarmteLink bring-up reference
frontend / frontend (push) Successful in 29s
pytest / test (push) Successful in 2m23s
2026-08-20 12:11:05 +02:00
tliu93 d3d914b117 PRE-M8: add WarmteLink P1 planning placeholders
frontend / frontend (push) Successful in 27s
pytest / test (push) Successful in 2m20s
2026-08-18 17:20:31 +02:00
tliu93 fc4af857e3 AGENTS: make repository guidance harness-neutral
frontend / frontend (push) Successful in 46s
pytest / test (push) Successful in 2m33s
2026-08-18 13:28:23 +02:00
tliu93 0958d9a2e9 fix(energy): report real kWh in the cost Summary instead of mislabelled money
docker-image / build-and-push (push) Successful in 1m37s
frontend / frontend (push) Successful in 10m24s
pytest / test (push) Successful in 11m57s
The Summary cards labelled `metered_import` / `metered_export` as "(kWh)", but
both fields are monetary totals (Σ import_cost / Σ export_revenue).  Today's
page therefore showed "Import 1.339 kWh" when the meter had actually imported
4.188 kWh — the 1.339 was EUR.  Cross-checked against the DSMR cumulative
registers and Home Assistant: our energy figures were correct all along, only
the label was wrong.

summarize() now also aggregates the metered energy, reusing the already-fetched
non-degraded rows so no extra query is issued:

  metered_import_kwh = Σ (d1_kwh + d2_kwh)
  metered_export_kwh = Σ (r1_kwh + r2_kwh)

The Import/Export cards show kWh as the headline figure and keep the monetary
equivalent as a sub-line, so the split between energy cost and standing
charges/credits behind total_payable stays visible.

The `_kwh` suffix is now the only thing separating energy from money in this
payload, so the docstrings on both summarize() and SummaryResponse call that
out explicitly.

app/integrations/expose.py reads only the money keys, so the HA outbound
sensors are unaffected by the additive fields.
2026-08-06 22:20:56 +02:00
tliu93 b405aea88b docs: require frontend codegen alongside the OpenAPI export in the gates
frontend / frontend (push) Successful in 9m59s
pytest / test (push) Successful in 11m51s
docker-image / build-and-push (push) Successful in 12m49s
The local gate list only covered `scripts/export_openapi.py` + a clean
`openapi/` diff, but `frontend/src/api/schema.d.ts` is generated from that JSON
and CI re-runs `npm run codegen` with `git diff --exit-code`. d07a083 changed a
route docstring, refreshed openapi.json, and skipped codegen — local green,
remote red on a one-line comment diff. Spell out both steps and note that route
docstrings feed the OpenAPI description too.

Also add AGENTS.md as a symlink to CLAUDE.md so other agent tooling picks up
the same contract.
2026-07-27 19:03:03 +02:00
tliu93 bfc7aa3031 chore(frontend): regenerate API schema after sell_fee docstring change
frontend / frontend (push) Successful in 9m59s
pytest / test (push) Successful in 12m27s
d07a083 changed the /api/energy/prices docstring (sell now deducts sell_fee)
and refreshed openapi/openapi.json, but frontend/src/api/schema.d.ts was not
regenerated, so CI's "check codegen is in sync" step failed. Comment-only diff.
2026-07-27 18:29:54 +02:00
tliu93 2f63b9630c fix(prices): keep hover marker on the hovered slot past midnight
The price chart keyed its X axis on formatLocalTime() "HH:mm" labels, which
repeat across a today+tomorrow range. Recharts resolves axis tooltips by value
(combineTooltipPayload -> findEntryInArray), so hovering a slot after midnight
matched today's identically labelled point: the tooltip showed today's prices
and the active dot jumped back to today's position instead of following the
cursor.

Key the axis on the ISO instant instead (buildChartRows, sorted by slot start)
and format down to HH:mm in the tick formatter; the tooltip label now carries
the date so today and tomorrow are distinguishable.

Also mark the price slot currently in effect by default: findActiveSlotIndex()
locates the slot containing now, rendered as a ReferenceDot on the buy and sell
lines plus a caption, re-evaluated every 30s.

Regression test drives a real mousemove over a sized chart in jsdom and asserts
the resolved slot and active-dot position.
2026-07-27 18:29:08 +02:00
tliu93 d07a083e03 fix(tibber): deduct verkoopvergoeding (sell_fee) from feed-in sell price
frontend / frontend (push) Failing after 5m49s
pytest / test (push) Successful in 20m30s
docker-image / build-and-push (push) Successful in 13m26s
Tibber's API `total` already includes the buy-side inkoopvergoeding
(verified from production data: total = spot×1.21 + energy_tax 0.11085 +
inkoopvergoeding 0.0248). Under net metering Tibber pays back
`total − verkoopvergoeding` per returned kWh (NL: EUR 0.28 -> 0.2552), so the
two EUR 0.0248 fees do NOT cancel — the feed-in price sits 0.0248 below buy.

Model the verkoopvergoeding as a first-class, always-subtracted contract
field `energy.sell_fee` (default 0.0248) instead of folding it into
`sell_adjust`. New sell formula:

    sell = total − energy_tax − sell_fee − sell_adjust

`sell_adjust` now carries only the net-metering energy-tax refund
(= −energy_tax). Applied in both the billing strategy and the /prices
endpoint; recorded in the pricing snapshot. Frontend renders the field
automatically (dynamic profile form). Docs (references, m6) corrected to
drop the wrong "fees cancel" premise.
2026-07-20 13:50:13 +02:00
tliu93 b65f700d56 fix(tibber): fetch forward-looking today+tomorrow via priceInfo(QUARTER_HOURLY)
pytest / test (push) Successful in 20m2s
frontend / frontend (push) Failing after 6m5s
docker-image / build-and-push (push) Successful in 13m27s
priceInfoRange is a historical cursor connection whose range ends at "now": it
never returns upcoming slots. With it, the DB only ever held prices up to the
last hourly refresh, which caused two problems:

  1. The price chart could only show history up to now, never a forward curve.
  2. Worse, per-slot billing was subtly wrong. _tibber_strategy looks up the
     price via `starts_at <= t0` (nearest slot at or before the period). Because
     a period's exact 15-min slot was not fetched until the next hourly refresh
     (~1h later), intra-hour periods were billed with the PREVIOUS quarter's
     price and then locked in by the immutability guard — never corrected.

Switch to priceInfo(resolution: QUARTER_HOURLY) { today tomorrow }, which is
forward-looking AND quarter-hourly: today is always the full local day (96
slots) and tomorrow fills in once Tibber publishes day-ahead prices (picked up
by the next hourly refresh). Every 15-min slot's exact price is now in the DB
before the slot closes, so each period finds its own slot (accurate billing)
and the live current-price entity stays fresh.

Verified against the live Tibber API: fetch_price_range returns 192 points
(96 today + 96 tomorrow), spanning local today 00:00 → tomorrow 23:45, with
future slots present (previously 0).
2026-07-18 20:20:33 +02:00
tliu93 f4cea3874b test(energy-cost): pin local_now in future-window summarize test
frontend / frontend (push) Failing after 6m38s
pytest / test (push) Successful in 20m16s
docker-image / build-and-push (push) Successful in 13m32s
test_future_window_counts_0_days relied on the real wall-clock date and assumed
7/1→8/1 2026 was entirely in the future; once that range started elapsing the
window counted fixed-fee days and the assertion failed. Pin local_now to
June 25 2026 (as the sibling window tests already do) so the case stays
deterministic.
2026-07-17 18:49:26 +02:00
tliu93 134f0abb5f fix(tibber): fetch newest price slots via priceInfoRange last:192
priceInfoRange is a Relay-style cursor connection over the subscription's
entire price history. With no cursor, first:96 returned the OLDEST 96 slots,
anchored at the subscription start date — a fixed window that never advanced.
For a contract added mid-period this meant refresh_prices kept re-upserting the
same day-one slots forever, so GET /api/energy/prices found nothing in the
today+tomorrow window and the UI showed "No Tibber price points available".

Use last:192 to return the newest 192 quarter-hourly slots (2 days), which ends
at the latest published slot and advances daily, fully covering the prices
endpoint's today+tomorrow window.
2026-07-17 18:49:26 +02:00
tliu93 3e04b15656 feat(modbus): add DDSU666 profile and select read function code per profile
frontend / frontend (push) Successful in 2m21s
pytest / test (push) Successful in 9m50s
docker-image / build-and-push (push) Successful in 4m24s
- Add CHINT DDSU666 profile (ddsu666.yaml): FC03 holding registers, voltage/
  current/active power (kW)/reactive power (kvar)/PF/frequency, import+export
  active energy. Word/byte order left at big-endian as a documented best guess
  (manual has no float example) — to be confirmed on-device.
- Add DDSU666-Modbus-Protocol.md reference extracted from the official manual,
  plus the source PDF (parity with the SDM120 reference).
- Generalize driver.read_blocks to dispatch FC03 (holding) or FC04 (input)
  based on a function_code argument (default 4, SDM120 behaviour unchanged);
  the code is validated before any connection is attempted.
- Wire profile.function_code through the CLI read command, the background
  poller, and the device /test endpoint — previously the profile field was
  declared but never honoured (read path was hardcoded to FC04).
- Tests: default -> FC04, function_code=3 -> FC03 holding, invalid FC rejected
  before connecting.
2026-06-30 17:16:25 +02:00
tliu93 f2e8f6a8e7 docs(roadmap): queue Authentication next-steps — sliding session renewal + long-lived token targets (location/poo ingestion)
frontend / frontend (push) Successful in 2m16s
pytest / test (push) Successful in 10m40s
2026-06-27 22:16:02 +02:00
43 changed files with 3646 additions and 318 deletions
+208
View File
@@ -0,0 +1,208 @@
# AGENTS.md — Home Automation Backend
本文件是本仓库 coding agent 指引的 **single source of truth**`CLAUDE.md` 通过符号链接指向本文件。它定义本项目的**工作流程、文档位置、commit 规范**。支持对应项目指引的 agent 在动手前应完整读取本文件。
## 项目速览
- 个人用 home-automation 应用:**FastAPI + React SPA + SQLite + SQLAlchemy + Alembic**,前后端同源托管。
- 单 admin 鉴权(Argon2 + server-side session cookie),runtime config 落 `app_config` 表。
- 模块:public IPv4 monitor、SMTP 通知、location / poo recorder、Home Assistant in/out、TickTick OAuth、Modbus / DSMR 能耗采集、MQTT / HA Discovery、动态电价与电费计算。
- 已发布 `v1.5.1`。M1、M2、M4-M7 已完成;M3 token / 移动端仍为远期方向。
- **当前现实**:已收敛为单一 `app.db`、一套 DeclarativeBase 和一条 Alembic 链;只有历史数据迁移 runbook 会读取旧 location / poo 数据库。
- 明确不做:Notion 模块。
## 文档地图与「开工前必读」
文档都在 `docs/`
| 路径 | 作用 |
| --- | --- |
| `docs/roadmap.md` | 全局规划与里程碑总览 |
| `docs/design/README.md` | **协作契约**:任务卡格式、原子任务定义、校验闸门、数据安全红线 |
| `docs/design/m1-db-consolidation.md` | M1 原子任务(含真实代码现状盘点 + 人工 runbook) |
| `docs/design/m2-frontend-v2.md` | M2 原子任务 + API 契约 + 前端校验闸门 |
| `docs/design/m3-token-mobile.md` | M3(远期,暂缓) |
| `docs/design/m4-login-hardening.md` | M4 登录加固(已完成) |
| `docs/design/m5-iot-energy.md` | M5 IoT / 能耗采集(已完成) |
| `docs/design/m6-tibber-dynamic-energy.md` | M6 动态电价、DSMR 与电费计算(已完成) |
| `docs/design/m7-meter-epochs-archival.md` | M7 电表生命周期 / 换表归档(已完成) |
| `docs/*.md`auth / public-ip-monitor / location-recorder …) | 各模块说明,按需读 |
**开工时读取顺序**
1. `docs/design/README.md`(每轮都读,它是流程与验收的共同契约)。
2. 本轮对应的 milestone 文档(如 `docs/design/m1-db-consolidation.md`),定位要做的任务卡。
3. 任务卡 `Files` 列出的源文件 + 该模块的 `docs/*.md`(按需)。
4. `docs/roadmap.md` 仅在需要全局视角时读。
## 工作流程
### 实现模式(由用户的提示词决定)
- **默认逐步**:给一个 milestone 文档,按其中原子任务**一步一步**实现。
- **(a) 只实现一步**:用户说"只实现一步 / 这一个任务"时,**只做那一个任务卡**,跑完校验闸门后停下,等用户确认,不要顺手往下做。
- **(b) 完成整个 milestone**:仅当用户在提示词里**显式要求启用 sub-agent**时,才起 implementer / reviewer / fixer sub-agent(按下方**『默认能力档位』**选择模型,用户人工指定则覆盖),按任务依赖顺序跑完整条链。
- **Sub-agent 纪律**:只在用户显式要求时才 spawn sub-agent;单步/小改动在主线内联完成。当前 harness 支持独立 sub-agent 时,使用其原生机制按下方**『默认能力档位』**派发;不支持时不得假装已创建 sub-agent,应明确说明限制,并仅在用户允许 fallback 时由主 agent 继续。
### 默认能力档位(实现模式 sub-agent;可被人工指定覆盖)
起 implementer / reviewer / fixer sub-agent 时,**默认**按下列能力档位选择当前 harness 支持的模型,无需用户每次人工指定:
| 角色 | 通用模型要求 | 推理档位 | Harness 示例(非强制) |
| --- | --- | --- | --- |
| **Implementer** | 平衡型代码实现模型 | `medium` 或等效档位 | Claude CodeSonnetCodex/OpenAIGPT-5.6 Terra (`gpt-5.6-terra`) |
| **Fixer**(返工) | 平衡型代码实现模型 | `medium` 或等效档位 | Claude CodeSonnetCodex/OpenAIGPT-5.6 Terra (`gpt-5.6-terra`) |
| **Reviewer** | 当前 harness 支持的最强通用推理 / 代码模型 | `extra-high` / `xhigh` 或等效档位 | Claude CodeOpusCodex/OpenAIGPT-5.6 Sol (`gpt-5.6-sol`) |
- **示例非强制**:示例模型只表示当前推荐映射,不构成跨 harness 的硬性模型 ID;当前 harness 不支持时,选择最符合「通用模型要求」的可用模型。
- **选择优先级**:用户显式指定 > 当前 harness 的原生角色配置 > 上表的 harness 示例 > 按通用模型要求自动选择。
- **推理档位说明**:若 harness 提供独立的 reasoning-effort 设置,按上表设置;若不提供,在 spawn prompt 中明确 implementer/fixer 按平衡深度思考,reviewer 按对抗性外部审计强度复核。
### 角色(Orchestrator → Implementer → Reviewer → Fixer
- 我(主线)= **Orchestrator**:挑依赖已满足的下一个任务、派发、转述结果、维护任务 `Status`
- **Implementer**(平衡型代码实现模型,medium 或等效档位):一次一个任务,严格按任务卡,不扩范围。
- **Reviewer**(最强通用推理 / 代码模型,extra-high / xhigh 或等效档位):实现完成后起 Reviewer sub-agent,按任务卡 `Acceptance criteria` + `Reviewer checklist` 复核、**独立重跑校验闸门**,驱动返工直到本轮 PASS。
- **Fixer**(平衡型代码实现模型,medium 或等效档位):按 reviewer 的编号返工清单返工;**每轮返工起一个干净的 Fixer**(与首次实现的 Implementer 分开冷启动),先读对应 `review-notes/<task>-review-<n>.md` 再改。
#### Reviewer 盲审纪律(M1 教训)
M1 里 review **从未触发过一次 rework**,根因是 orchestrator 把自己的结论 / 辩护喂给了 reviewer,造成 context bleed、review 沦为橡皮图章。所以:
- reviewer 必须**使用全新、独立的 sub-agent / thread 冷启动,并最小化喂料**——spawn prompt 只给:① 任务卡(`Acceptance criteria` + `Reviewer checklist`)、② 对应的 `review-notes/<task>-impl|rework-<n>.md` 路径、③ 要审的 diff / commit 范围。
- **不要**在 prompt 里塞 orchestrator 自己的判断、"我觉得没问题"、对实现选择的辩护,或上一轮 reviewer 的倾向性结论。让它**独立得出结论、独立重跑校验闸门**。
- 事后另起的整库**独立盲审**(如对抗复审)同理:使用全新独立的 agent / thread、最小上下文,把它当"**外部审计**"而非"确认自己没错"。
### 校验闸门(每个任务结束都要全绿)
根目录、激活 `.venv` 后:
```bash
pytest # 权威闸门(CI 跑的就是它)
ruff check . # line-length=100
python scripts/export_openapi.py && git diff --exit-code openapi/ # 改了路由/schema 才需要,且产物须入库
```
前端任务(M2)在 `frontend/` 下另跑 `npm run lint && npm run typecheck && npm run test && npm run build`(详见 m2 文档 §8)。
**不过闸门就不算完成**,不得跳过、不得留红给下一轮。
**Repo-meta 例外**:纯文档、agent 指引、符号链接等不影响可执行代码、构建与 API 契约的变更,可在用户明确同意时跳过代码闸门。仍须完成针对性校验(如链接目标、文件类型、diff 与 Git 状态),并在结果中明确记录未运行哪些闸门。
#### API 契约同步:`openapi/` 与 `schema.d.ts` 是**两步**v1.4.0 后教训)
**只跑 `export_openapi.py` 不够。** 前端的 `frontend/src/api/schema.d.ts` 是由 `openapi/openapi.json` 二次生成的,CI`.github/workflows/frontend.yml`*Check codegen is in sync*)会重跑 codegen 并 `git diff --exit-code src/api/schema.d.ts`。漏了第二步 → 本地闸门全绿、远端 CI 红。真出过:`d07a083` 改了 `/api/energy/prices` 的 docstring 并同步了 `openapi.json`,但没重跑 codegen,只差一行注释就把 CI 挂了。
所以**只要动了路由 / Pydantic schema / 路由 docstring**docstring 也会进 OpenAPI description!),两步都要跑,两个产物都要入库:
```bash
# 1) 后端契约
python scripts/export_openapi.py && git diff --exit-code openapi/
# 2) 前端类型(在 frontend/ 下)
npm run codegen && git diff --exit-code src/api/schema.d.ts
```
- 判据:`git diff --exit-code openapi/` 有输出 → **必然**还要跑一次 `npm run codegen`
- 反过来也成立:`schema.d.ts` 不要手改,它是生成物。
- Reviewer 审"动了路由 / schema / 路由 docstring"类任务时,把**这两个产物是否都已重新生成并入库**当作 acceptance 的一部分。
### 构建上下文完整性(M1 Dockerfile 教训)
`docker build` **不在 pytest/ruff 闸门里**——M1 删了 `alembic_location/poo` 后忘了同步 `Dockerfile``COPY`,单元闸门全绿却把坏掉的镜像构建一路漏到 release tag。所以:
- 任务**删除 / 移动 / 重命名文件或目录**时,必须 grep 构建清单是否还在引用它们:`Dockerfile`(尤其 `COPY` 源)、`docker/``*.ini`、CI workflow、`requirements*.txt` 等。
- 已有回归测试 `tests/test_deployment.py::test_dockerfile_copy_sources_exist` 守"Dockerfile `COPY` 源必须存在于构建上下文";新增 / 改动 `COPY` 时确保它仍覆盖得到。
- Reviewer 审"删 / 移文件"类任务时,**必须顺带核对构建清单引用**,把它当 acceptance 的一部分。
## 每轮简报(`review-notes/`
由 milestone 任务卡驱动的每轮实现、返工或 review,都要在 `review-notes/` 下产出**中文简报**。该目录**已在 `.gitignore` 忽略**,纯本地、不入库——它是 agent 之间和与人之间的交接载体,不是仓库产物。纯讨论、只读分析与不进入正式任务链的 repo-meta 变更无需产出简报,除非用户明确要求。
- **实现 / 返工简报**:每轮实现完成后(无论首次实现还是返工),写一份。文件名建议 `<task-id>-impl-<n>.md` / `<task-id>-rework-<n>.md`(如 `M1-T03-impl-1.md``M1-T03-rework-1.md`)。至少包含:
1. **本轮修改的具体内容**(改了哪些文件、做了什么、为什么)。
2. **自动化测试结果**`pytest` / `ruff` / 前端闸门的实际输出或结论,通过/失败逐项写清)。
3. **若需人工 walkthrough**:写明具体步骤(怎么启动、点哪里、预期看到什么);若无需人工验证,明确写"无需人工 walkthrough"。
- **review 简报**:每轮 review 后写一份,文件名建议 `<task-id>-review-<n>.md`(如 `M1-T03-review-1.md`)。至少包含:评审结论(`PASS` 或带编号的返工清单)、对照任务卡 `Acceptance criteria` + `Reviewer checklist` 的逐条核对、reviewer 独立重跑校验闸门的结果。
**用途**:① reviewer 审核时参考对应的实现简报;② implementer 返工时参考对应的 review 简报;③ 人类(用户)通读这些简报确认有无问题。简报之间用文件名里的 `<task-id>` 与轮次 `<n>` 对应起来。
### Orchestrator 派发契约(让简报真正被读到)
**关键**:sub-agent 冷启动、不继承主线上下文,**不会因为本文件提到简报就自动去读**对应文件。简报能流转,靠的是 orchestrator(主线)在**每次 spawn 时把路径显式写进 prompt**,而不是被动约定。所以派发时必须做到:
- **显式告诉它「先读哪个简报」**:
- 派 implementer 做**首次实现** → 传任务卡位置(milestone 文档路径 + task id);无前置简报。
- 派 implementer 做**返工** → 必须传对应的 `review-notes/<task>-review-<n>.md` 路径,并要求**先读它**再改。
- 派 reviewer → 必须传对应的 `review-notes/<task>-impl|rework-<n>.md` 路径 + 任务卡,要求**先读它**再评。
- **显式告诉它「本轮结束写哪个简报」**:明确给出输出路径 `review-notes/<task>-<impl|rework|review>-<n>.md` 及上面要求的内容项。
- **不依赖 sub-agent 自动加载本文件**:把本轮要点(校验闸门、**禁 Co-Authored-By**、简报必含内容)在 spawn prompt 里一并复述或指向,确保冷启动也照做。
- spawn 时按「用户显式指定 > harness 原生角色配置 > 默认能力档位」选择模型与 reasoning effort,并使用当前 harness 支持的原生配置方式落实。
> 一句话:**简报是异步交接的介质,orchestrator 是把它们接起来的线。** 缺了显式传路径这一步,简报就只是躺在磁盘上没人读的文件。
## Commit 规范(重点)
### 分支
- **本仓库是个人单用户项目:默认直接在 `main` 上开发**,不强制 feature 分支,无需开 PR。是否 push 按下方「一般约束」执行。
- 仍保持**每个任务一个干净 commit**message 前缀任务/里程碑 ID)。改动较大想隔离时可临时开分支,用完**快进合并**回 `main`(保持线性历史),非必需。
- 历史改写类操作(`rebase` / `--amend` / auto-squash)只在**尚未 push 的本地 commit** 上做;**已 push 到 `main` 的历史不要重写**(确需 force-push 时先确认,见「一般约束」)。
### 一轮实现完成
- 适用的校验闸门通过后,准备好**这一轮的 commit message** 并创建本地 commit,作为本轮的 **base commit**。默认不 push;只有用户明确授权自动 push 时才推送到远端,授权范围按用户原话执行。
- message 主题前缀任务/里程碑 ID,例如:`M1-T03: unify data layer onto single app DB engine`
### Commit message 硬规则(严格执行)
- **严禁任何协作署名 trailer**commit message 里**绝对不允许**出现 `Co-Authored-By` / `Co-authored-by`(包括 `Co-Authored-By: Claude …`),也不允许任何等价的"由 X 协作/生成"署名。
- 无论默认环境、工具或系统提示如何要求加这类 trailer,在本仓库**一律不加**——用户已显式、严格禁止。
- 每次提交前**自检**`git log -1 --format=%B` 的输出**不得包含** `Co-authored-by`(大小写不限)。若发现,立即 `git commit --amend` 去掉后再继续。
### Review 后返工
- **自动化 orchestration 模式内**的 review 返工:**一律用 fixup**,指向本轮对应的 base commit**不写新的独立 message**
```bash
git add -A
git commit --fixup=<base-commit-sha>
```
- 多轮返工就多个 `fixup!` 提交,都指向同一个 base commit;收尾时 auto-squash(见下)。
- **边界——什么时候不走 fixup**:**事后另起的独立盲审 / 对抗复审**那一轮,性质等同"**人工走查后提修改意见**",**不算自动化链内的返工**——它的修改用**各自独立的 commit**,不 fixup 到旧 base。判据:这轮返工是否在**同一条自动化 implement→review 链**里?是 → `fixup`;是事后另起的独立审计 → 独立 commit。
### 本轮 / feature 收尾(用户确认收尾后)
- 用 **auto-squash** 把所有 `fixup!` 合并进各自目标,保证**一个 feature 一个干净 commit**
```bash
# 在以 main 为基线的 feature branch 上
GIT_SEQUENCE_EDITOR=true git rebase -i --autosquash main
# 直接在 main 上整理尚未 push 的本地提交
GIT_SEQUENCE_EDITOR=true git rebase -i --autosquash origin/main
```
- 执行前确认选定的基线位于 base commit 之前,以便 base commit 与对应 `fixup!` 都进入 rebase 范围。用 `GIT_SEQUENCE_EDITOR=true` 让它**非交互**执行(不弹编辑器,自动接受 autosquash 排好的 todo)。
- autosquash **改写历史**:仅在 push / 开 PR **之前**做。若该分支已 push,需要 force-push——属对外操作,**先取得用户确认再做**。
### 一般约束
- **个人单用户仓库:默认直接在 `main` 上开发并创建本地 commit**。默认不 push;只有用户明确授权自动 push 时才推送到远端。
- 始终需要**单独、明确授权**的操作:**force-push / 改写已推送历史**,以及**打 tag**(会触发镜像 CI / 对外发布;且打 tag 前须按下方「发版前置走查」真跑一次 `docker build`)。
## 发版前置走查(打 tag 前必做)
单元闸门绿 ≠ 真的能跑、能构建、能用。M1 出过"绿了但 docker 构建坏了"的事故,所以**打版本 tag(触发镜像 CI)之前**,除了 `pytest` / `ruff` 全绿,还要:
- **真起 app**:迁移(`python -m scripts.run_migrations`)→ `uvicorn app.main:app ...`,确认能正常启动、关键路由不 500。
- **真跑镜像构建**:本地 `docker build`(多阶段就跑完整条),确认构建通过、`COPY` 源都在。
- **关键功能人工瞄一眼**:尤其前端 / 可视化类(M2 的热力图、首页地图)——自动闸门判断不了"渲染对不对、UX 顺不顺",这部分**靠看跑起来的 app,不靠读代码**。
- 上述任一不过 → **不打 tag**。tag 一旦 push 会触发 docker 镜像 CI / 对外发布,属对外操作,**先确认**。
## 数据安全红线(不可违反)
- 任何脚本 / migration **都不得删除或覆盖用户数据文件**(旧 `.db`、备份、volume)。删除只能是人工、事后、保留归档的独立步骤(见 `docs/design/m1-db-consolidation.md` §6 runbook)。
- 涉及历史数据的迁移**先在备份副本上演练**;迁移脚本必须幂等且搬完对账行数。
- Review 时只要发现"删文件 / drop 有数据的表 / truncate"出现在自动化任务里,直接判返工。
## 常用命令
```bash
# 环境
python -m venv .venv && source .venv/bin/activate && pip install -r dev-requirements.txt
# 迁移(初始化/适配 DB
python -m scripts.run_migrations
# 起服务
uvicorn app.main:app --reload --host 0.0.0.0 --port 8000
# 测试 / lint / OpenAPI 导出
pytest
ruff check .
python scripts/export_openapi.py
```
-181
View File
@@ -1,181 +0,0 @@
# CLAUDE.md — Home Automation Backend
本文件每次会话自动加载。它定义本项目的**工作流程、文档位置、commit 规范**。请在动手前先读完。
## 项目速览
- 个人用 home-automation 后端:**FastAPI + SQLite + SQLAlchemy + Alembic**,服务端模板(JinjaM2 将换成 React SPA)。
- 单 admin 鉴权(Argon2 + server-side session cookie),runtime config 落 `app_config` 表。
- 模块:public IPv4 monitor、SMTP 通知、location recorder、poo recorder、Home Assistant in/out、TickTick OAuth。
- 已发布 `v1.0.3`。下一阶段方向:**M1 单库化 → M2 React 前端 → M3 token/移动端(远期,M2 后再说)**。
- **当前现实**:在 M1 完成前仍是**三个独立 SQLite 库**app / location / poo),三套 DeclarativeBase、三条 Alembic 链。不要假设已经单库——以代码现状为准。
- 明确不做:Notion 模块。
## 文档地图与「开工前必读」
文档都在 `docs/`
| 路径 | 作用 |
| --- | --- |
| `docs/roadmap.md` | 全局规划与里程碑总览 |
| `docs/design/README.md` | **协作契约**:任务卡格式、原子任务定义、校验闸门、数据安全红线 |
| `docs/design/m1-db-consolidation.md` | M1 原子任务(含真实代码现状盘点 + 人工 runbook) |
| `docs/design/m2-frontend-v2.md` | M2 原子任务 + API 契约 + 前端校验闸门 |
| `docs/design/m3-token-mobile.md` | M3(远期,暂缓) |
| `docs/*.md`auth / public-ip-monitor / location-recorder …) | 各模块说明,按需读 |
**开工时读取顺序**
1. `docs/design/README.md`(每轮都读,它是流程与验收的共同契约)。
2. 本轮对应的 milestone 文档(如 `docs/design/m1-db-consolidation.md`),定位要做的任务卡。
3. 任务卡 `Files` 列出的源文件 + 该模块的 `docs/*.md`(按需)。
4. `docs/roadmap.md` 仅在需要全局视角时读。
## 工作流程
### 实现模式(由用户的提示词决定)
- **默认逐步**:给一个 milestone 文档,按其中原子任务**一步一步**实现。
- **(a) 只实现一步**:用户说"只实现一步 / 这一个任务"时,**只做那一个任务卡**,跑完校验闸门后停下,等用户确认,不要顺手往下做。
- **(b) 完成整个 milestone**:仅当用户在提示词里**显式要求启用 sub-agent**时,才起 implementer / reviewer / fixer sub-agent(模型用下方**『默认模型档位』**,用户人工指定则覆盖),按任务依赖顺序跑完整条链。
- **Sub-agent 纪律**:只在用户显式要求时才 spawn sub-agent;单步/小改动在主线内联完成。起 sub-agent 时按下方**『默认模型档位』**选择模型(用户人工指定则以人工指定为准),用 Agent 工具的 `model` 字段落实。
### 默认模型档位(实现模式 sub-agent;可被人工指定覆盖)
起 implementer / reviewer / fixer sub-agent 时,**默认**用下列模型档位,无需用户每次人工指定:
| 角色 | 默认模型 | 推理档位 |
| --- | --- | --- |
| **Implementer** | **Sonnet** | high reasoning effort |
| **Fixer**(返工) | **Sonnet** | high reasoning effort |
| **Reviewer** | **Opus** | extra-high reasoning effort |
- **人工指定覆盖**:若用户在提示词里**显式指定了其他模型**(针对任一角色),则**以用户人工指定为准**,覆盖上述默认。
- 用 Agent 工具的 `model` 字段落实模型选择;该字段当前仅支持 `sonnet` / `opus` / `haiku` / `fable`
- **推理档位说明**:Agent 工具未暴露独立的 reasoning-effort 旋钮,"high / extra-high reasoning" 通过 spawn prompt 里的显式指令传达(要求 implementer/fixer 动手前充分推理边界条件;要求 reviewer 以对抗性外部审计心态最高强度复核)。若所在 harness 提供真正的 effort 设置,则一并按此档位设置。
### 角色(Orchestrator → Implementer → Reviewer → Fixer
- 我(主线)= **Orchestrator**:挑依赖已满足的下一个任务、派发、转述结果、维护任务 `Status`
- **Implementer**(默认 **Sonnet**high reasoning;见上方『默认模型档位』):一次一个任务,严格按任务卡,不扩范围。
- **Reviewer**(默认 **Opus**extra-high reasoning):实现完成后起 Reviewer sub-agent,按任务卡 `Acceptance criteria` + `Reviewer checklist` 复核、**独立重跑校验闸门**,驱动返工直到本轮 PASS。
- **Fixer**(默认 **Sonnet**high reasoning):按 reviewer 的编号返工清单返工;**每轮返工起一个干净的 Fixer**(与首次实现的 Implementer 分开冷启动),先读对应 `review-notes/<task>-review-<n>.md` 再改。
#### Reviewer 盲审纪律(M1 教训)
M1 里 review **从未触发过一次 rework**,根因是 orchestrator 把自己的结论 / 辩护喂给了 reviewer,造成 context bleed、review 沦为橡皮图章。所以:
- reviewer 必须**冷启动(Clear-Agent)、最小化喂料**——spawn prompt 只给:① 任务卡(`Acceptance criteria` + `Reviewer checklist`)、② 对应的 `review-notes/<task>-impl|rework-<n>.md` 路径、③ 要审的 diff / commit 范围。
- **不要**在 prompt 里塞 orchestrator 自己的判断、"我觉得没问题"、对实现选择的辩护,或上一轮 reviewer 的倾向性结论。让它**独立得出结论、独立重跑校验闸门**。
- 事后另起的整库**独立盲审**(如对抗复审)同理:Clear-Agent、最小上下文,把它当"**外部审计**"而非"确认自己没错"。
### 校验闸门(每个任务结束都要全绿)
根目录、激活 `.venv` 后:
```bash
pytest # 权威闸门(CI 跑的就是它)
ruff check . # line-length=100
python scripts/export_openapi.py && git diff --exit-code openapi/ # 改了路由/schema 才需要,且产物须入库
```
前端任务(M2)在 `frontend/` 下另跑 `npm run lint && npm run typecheck && npm run test && npm run build`(详见 m2 文档 §8)。
**不过闸门就不算完成**,不得跳过、不得留红给下一轮。
### 构建上下文完整性(M1 Dockerfile 教训)
`docker build` **不在 pytest/ruff 闸门里**——M1 删了 `alembic_location/poo` 后忘了同步 `Dockerfile``COPY`,单元闸门全绿却把坏掉的镜像构建一路漏到 release tag。所以:
- 任务**删除 / 移动 / 重命名文件或目录**时,必须 grep 构建清单是否还在引用它们:`Dockerfile`(尤其 `COPY` 源)、`docker/``*.ini`、CI workflow、`requirements*.txt` 等。
- 已有回归测试 `tests/test_deployment.py::test_dockerfile_copy_sources_exist` 守"Dockerfile `COPY` 源必须存在于构建上下文";新增 / 改动 `COPY` 时确保它仍覆盖得到。
- Reviewer 审"删 / 移文件"类任务时,**必须顺带核对构建清单引用**,把它当 acceptance 的一部分。
## 每轮简报(`review-notes/`
每轮工作都要在 `review-notes/` 下产出**中文简报**。该目录**已在 `.gitignore` 忽略**,纯本地、不入库——它是 agent 之间和与人之间的交接载体,不是仓库产物。
- **实现 / 返工简报**:每轮实现完成后(无论首次实现还是返工),写一份。文件名建议 `<task-id>-impl-<n>.md` / `<task-id>-rework-<n>.md`(如 `M1-T03-impl-1.md``M1-T03-rework-1.md`)。至少包含:
1. **本轮修改的具体内容**(改了哪些文件、做了什么、为什么)。
2. **自动化测试结果**`pytest` / `ruff` / 前端闸门的实际输出或结论,通过/失败逐项写清)。
3. **若需人工 walkthrough**:写明具体步骤(怎么启动、点哪里、预期看到什么);若无需人工验证,明确写"无需人工 walkthrough"。
- **review 简报**:每轮 review 后写一份,文件名建议 `<task-id>-review-<n>.md`(如 `M1-T03-review-1.md`)。至少包含:评审结论(`PASS` 或带编号的返工清单)、对照任务卡 `Acceptance criteria` + `Reviewer checklist` 的逐条核对、reviewer 独立重跑校验闸门的结果。
**用途**:① reviewer 审核时参考对应的实现简报;② implementer 返工时参考对应的 review 简报;③ 人类(用户)通读这些简报确认有无问题。简报之间用文件名里的 `<task-id>` 与轮次 `<n>` 对应起来。
### Orchestrator 派发契约(让简报真正被读到)
**关键**:sub-agent 冷启动、不继承主线上下文,**不会因为本文件提到简报就自动去读**对应文件。简报能流转,靠的是 orchestrator(主线)在**每次 spawn 时把路径显式写进 prompt**,而不是被动约定。所以派发时必须做到:
- **显式告诉它「先读哪个简报」**:
- 派 implementer 做**首次实现** → 传任务卡位置(milestone 文档路径 + task id);无前置简报。
- 派 implementer 做**返工** → 必须传对应的 `review-notes/<task>-review-<n>.md` 路径,并要求**先读它**再改。
- 派 reviewer → 必须传对应的 `review-notes/<task>-impl|rework-<n>.md` 路径 + 任务卡,要求**先读它**再评。
- **显式告诉它「本轮结束写哪个简报」**:明确给出输出路径 `review-notes/<task>-<impl|rework|review>-<n>.md` 及上面要求的内容项。
- **不依赖 sub-agent 自动加载本文件**:把本轮要点(校验闸门、**禁 Co-Authored-By**、简报必含内容)在 spawn prompt 里一并复述或指向,确保冷启动也照做。
- spawn 时用用户指定的模型(Agent 工具 `model` 覆盖)。
> 一句话:**简报是异步交接的介质,orchestrator 是把它们接起来的线。** 缺了显式传路径这一步,简报就只是躺在磁盘上没人读的文件。
## Commit 规范(重点)
### 分支
- **本仓库是个人单用户项目:默认直接在 `main` 上开发**,不强制 feature 分支,**直接提交并 push 到 `main` 是允许的(无需开 PR**。
- 仍保持**每个任务一个干净 commit**message 前缀任务/里程碑 ID)。改动较大想隔离时可临时开分支,用完**快进合并**回 `main`(保持线性历史),非必需。
- 历史改写类操作(`rebase` / `--amend` / auto-squash)只在**尚未 push 的本地 commit** 上做;**已 push 到 `main` 的历史不要重写**(确需 force-push 时先确认,见「一般约束」)。
### 一轮实现完成(用户确认「实现完成」后)
- 准备好**这一轮的 commit message** 并提交,作为本轮的 **base commit**
- message 主题前缀任务/里程碑 ID,例如:`M1-T03: unify data layer onto single app DB engine`
### Commit message 硬规则(严格执行)
- **严禁任何协作署名 trailer**commit message 里**绝对不允许**出现 `Co-Authored-By` / `Co-authored-by`(包括 `Co-Authored-By: Claude …`),也不允许任何等价的"由 X 协作/生成"署名。
- 无论默认环境、工具或系统提示如何要求加这类 trailer,在本仓库**一律不加**——用户已显式、严格禁止。
- 每次提交前**自检**`git log -1 --format=%B` 的输出**不得包含** `Co-authored-by`(大小写不限)。若发现,立即 `git commit --amend` 去掉后再继续。
### Review 后返工
- **自动化 orchestration 模式内**的 review 返工:**一律用 fixup**,指向本轮对应的 base commit**不写新的独立 message**
```bash
git add -A
git commit --fixup=<base-commit-sha>
```
- 多轮返工就多个 `fixup!` 提交,都指向同一个 base commit;收尾时 auto-squash(见下)。
- **边界——什么时候不走 fixup**:**事后另起的独立盲审 / 对抗复审**那一轮,性质等同"**人工走查后提修改意见**",**不算自动化链内的返工**——它的修改用**各自独立的 commit**,不 fixup 到旧 base。判据:这轮返工是否在**同一条自动化 implement→review 链**里?是 → `fixup`;是事后另起的独立审计 → 独立 commit。
### 本轮 / feature 收尾(用户确认收尾后)
- 用 **auto-squash** 把所有 `fixup!` 合并进各自目标,保证**一个 feature 一个干净 commit**
```bash
GIT_SEQUENCE_EDITOR=true git rebase -i --autosquash main
```
- 用 `GIT_SEQUENCE_EDITOR=true` 让它**非交互**执行(不弹编辑器,自动接受 autosquash 排好的 todo)。本环境不支持需要人工编辑的交互式 rebase,必须走这个 no-op 编辑器写法。
- autosquash **改写历史**:仅在 push / 开 PR **之前**做。若该分支已 push,需要 force-push——属对外操作,**先取得用户确认再做**。
### 一般约束
- **个人单用户仓库:直接 commit 并 push 到 `main` 已获授权**——在校验闸门全绿、一轮工作完成时即可提交 / 推送,无需逐次征求同意。
- 仍需**先取得用户确认**的操作:**force-push / 改写已推送历史**,以及**打 tag**(会触发镜像 CI / 对外发布;且打 tag 前须按下方「发版前置走查」真跑一次 `docker build`)。
## 发版前置走查(打 tag 前必做)
单元闸门绿 ≠ 真的能跑、能构建、能用。M1 出过"绿了但 docker 构建坏了"的事故,所以**打版本 tag(触发镜像 CI)之前**,除了 `pytest` / `ruff` 全绿,还要:
- **真起 app**:迁移(`python -m scripts.run_migrations`)→ `uvicorn app.main:app ...`,确认能正常启动、关键路由不 500。
- **真跑镜像构建**:本地 `docker build`(多阶段就跑完整条),确认构建通过、`COPY` 源都在。
- **关键功能人工瞄一眼**:尤其前端 / 可视化类(M2 的热力图、首页地图)——自动闸门判断不了"渲染对不对、UX 顺不顺",这部分**靠看跑起来的 app,不靠读代码**。
- 上述任一不过 → **不打 tag**。tag 一旦 push 会触发 docker 镜像 CI / 对外发布,属对外操作,**先确认**。
## 数据安全红线(不可违反)
- 任何脚本 / migration **都不得删除或覆盖用户数据文件**(旧 `.db`、备份、volume)。删除只能是人工、事后、保留归档的独立步骤(见 `docs/design/m1-db-consolidation.md` §6 runbook)。
- 涉及历史数据的迁移**先在备份副本上演练**;迁移脚本必须幂等且搬完对账行数。
- Review 时只要发现"删文件 / drop 有数据的表 / truncate"出现在自动化任务里,直接判返工。
## 常用命令
```bash
# 环境
python -m venv .venv && source .venv/bin/activate && pip install -r dev-requirements.txt
# 迁移(初始化/适配 DB
python -m scripts.run_migrations
# 起服务
uvicorn app.main:app --reload --host 0.0.0.0 --port 8000
# 测试 / lint / OpenAPI 导出
pytest
ruff check .
python scripts/export_openapi.py
```
Symlink
+1
View File
@@ -0,0 +1 @@
AGENTS.md
+5 -3
View File
@@ -169,7 +169,7 @@ def get_prices(
Response ``points`` carries per-slot:
- ``buy = total`` (Tibber all-inclusive price)
- ``sell = total energy_tax sell_adjust`` (from active version values)
- ``sell = total energy_tax sell_fee sell_adjust`` (from active version values)
- ``level`` (Tibber price level, may be null)
``tariff`` is null.
@@ -222,7 +222,8 @@ def get_prices(
)
rows = list(reversed(db.execute(stmt).scalars().all()))
# Derive sell price per-point using version values (energy_tax + sell_adjust).
# Derive sell price per-point using version values
# (energy_tax + sell_fee + sell_adjust).
from decimal import Decimal
def _d(v: Any) -> Decimal:
@@ -230,12 +231,13 @@ def get_prices(
energy = version.values.get("energy", {}) if version.values else {}
energy_tax = _d(energy.get("energy_tax", 0))
sell_fee = _d(energy.get("sell_fee", 0))
sell_adjust = _d(energy.get("sell_adjust", 0))
points = []
for row in rows:
total = _d(row.total)
sell = float(total - energy_tax - sell_adjust)
sell = float(total - energy_tax - sell_fee - sell_adjust)
points.append(
PricePointSchema(
starts_at=_as_utc(row.starts_at),
+1
View File
@@ -488,6 +488,7 @@ def test_read(
device.port,
device.unit_id,
[{"start": b.start, "count": b.count} for b in profile.blocks],
function_code=profile.function_code,
)
payload: dict[str, Any] = decode_profile(profile, registers)
return ModbusTestReadResponse(ok=True, payload=payload)
+32 -5
View File
@@ -1,8 +1,11 @@
"""Modbus TCP driver — thin wrapper around pymodbus.
This module provides a single public function ``read_blocks`` that performs
one or more FC04 (Read Input Registers) block reads against a Modbus TCP
gateway and returns a flat ``dict[register_address -> 16-bit_value]`` map.
one or more block reads against a Modbus TCP gateway — using either FC04
(Read Input Registers) or FC03 (Read Holding Registers), selected per call
via the ``function_code`` argument — and returns a flat
``dict[register_address -> 16-bit_value]`` map. The function code comes from
the device profile (e.g. SDM120 uses FC04, DDSU666 uses FC03).
Design decisions
----------------
@@ -80,9 +83,10 @@ def read_blocks(
unit_id: int,
blocks: Sequence[Block],
*,
function_code: int = 4,
timeout: float = 3.0,
) -> dict[int, int]:
"""Read one or more contiguous register blocks via FC04 (input registers).
"""Read one or more contiguous register blocks via FC03 or FC04.
Parameters
----------
@@ -96,6 +100,11 @@ def read_blocks(
Sequence of ``{"start": int, "count": int}`` dicts describing the
contiguous register ranges to read. ``count`` is the number of
16-bit registers (not bytes).
function_code:
Modbus read function code: ``4`` for FC04 (Read Input Registers,
default — SDM120) or ``3`` for FC03 (Read Holding Registers —
DDSU666 and other devices that expose measurements as holding
registers). Comes from the device profile's ``function_code`` field.
timeout:
TCP connect/read timeout in seconds (default 3 s).
@@ -107,12 +116,21 @@ def read_blocks(
Raises
------
ModbusDriverError
If ``function_code`` is neither 3 nor 4 (validated before any
connection is attempted).
ModbusConnectionError
If the TCP connection to the gateway fails.
ModbusResponseError
If the gateway returns a Modbus exception frame or an unexpected
number of registers.
"""
if function_code not in (3, 4):
raise ModbusDriverError(
f"Unsupported read function code FC{function_code:02d} "
f"(only FC03 holding-register and FC04 input-register reads are supported)"
)
client = ModbusTcpClient(host, port=port, timeout=timeout)
try:
connected = client.connect()
@@ -125,7 +143,7 @@ def read_blocks(
for block in blocks:
start: int = block["start"]
count: int = block["count"]
_read_block(client, unit_id, start, count, registers)
_read_block(client, unit_id, start, count, registers, function_code=function_code)
return registers
@@ -145,9 +163,18 @@ def _read_block(
start: int,
count: int,
result: dict[int, int],
*,
function_code: int,
) -> None:
"""Read one block and merge into *result*. Raises on any error."""
"""Read one block and merge into *result*. Raises on any error.
``function_code`` is assumed already validated to be 3 or 4 by the caller
(``read_blocks``); 3 dispatches FC03 (holding) and 4 dispatches FC04 (input).
"""
try:
if function_code == 3:
response = client.read_holding_registers(start, count=count, device_id=unit_id)
else: # function_code == 4 (input registers)
response = client.read_input_registers(start, count=count, device_id=unit_id)
except ConnectionException as exc:
raise ModbusConnectionError(
@@ -0,0 +1,82 @@
name: ddsu666
description: CHINT DDSU666 single-phase smart meter
function_code: 3 # holding registers (FC03) — DDSU666 has NO input registers (no FC04)
word_order: big # high register first — ASSUMED; verify with a known voltage reading
byte_order: big # high byte first within each register (confirmed by manual CRC example)
# NOTE: the manual gives no worked float-decode example, so word_order is a best-guess
# (standard big-endian, high register first, matching sdm120). After wiring the meter,
# read 0x2000 (voltage) — it should decode to ~230 V. If it decodes to garbage, the
# device uses the opposite word order and this profile (and the decoder) need adjusting.
# Byte order is confirmed big-endian from the manual (Appendix A, Table A.4: 0x1388 -> 13 88).
blocks:
# Instantaneous quantities 0x20000x200F: voltage, current, P, Q, (rsv), PF, (rsv), Freq.
# 16 contiguous registers — single bulk read. (DDSU666 manual Table 9.)
- { start: 0x2000, count: 0x0010 }
# Active energy — import (0x4000) and export (0x400A) read as two small blocks rather
# than one span, to avoid touching the undocumented/reserved 0x40020x4009 gap.
- { start: 0x4000, count: 0x0002 }
- { start: 0x400A, count: 0x0002 }
metrics:
# Addresses are the raw Modbus protocol addresses (hex) from DDSU666 manual Table 9,
# read via FC03. Each float32 occupies two consecutive 16-bit registers.
- key: voltage
address: 0x2000 # U — Voltage (V)
type: float32
unit: "V"
device_class: voltage
ha_component: sensor
- key: current
address: 0x2002 # I — Current (A)
type: float32
unit: "A"
device_class: current
ha_component: sensor
- key: active_power
address: 0x2004 # P — Active power. Manual unit is kW (NOT W like sdm120).
type: float32
unit: "kW"
device_class: power
ha_component: sensor
- key: reactive_power
address: 0x2006 # Q — Reactive power (kvar)
type: float32
unit: "kvar"
device_class: reactive_power
ha_component: sensor
- key: power_factor
address: 0x200A # PF — Power factor (dimensionless)
type: float32
unit: ""
device_class: power_factor
ha_component: sensor
- key: frequency
address: 0x200E # Freq — Frequency (Hz)
type: float32
unit: "Hz"
device_class: frequency
ha_component: sensor
- key: import_energy
address: 0x4000 # Ep — positive/forward active energy (kWh)
type: float32
unit: "kWh"
device_class: energy
state_class: total_increasing
ha_component: sensor
- key: export_energy
address: 0x400A # -Ep — reverse active energy (kWh)
type: float32
unit: "kWh"
device_class: energy
state_class: total_increasing
ha_component: sensor
+10 -4
View File
@@ -123,6 +123,7 @@ class ManualProfile(BaseModel):
class TibberEnergySpec(BaseModel):
source: str # must be "tibber_api"
energy_tax: FieldSpec # subtracted from total to derive sell price
sell_fee: FieldSpec # verkoopvergoeding (feed-in fee); always subtracted from sell; default 0.0248
sell_adjust: FieldSpec # additional sell-price adjustment; default 0
@@ -268,10 +269,13 @@ def _fill_defaults_manual(values: dict[str, Any], profile: ManualProfile) -> dic
def _fill_defaults_tibber(values: dict[str, Any], profile: TibberProfile) -> dict[str, Any]:
"""Return a copy of *values* with sell_adjust and management_fee defaults applied."""
"""Return a copy of *values* with sell_fee, sell_adjust and management_fee defaults applied."""
filled = dict(values)
energy = dict(filled.get("energy", {}))
# Apply default for sell_fee (default=0.0248) if absent.
if "sell_fee" not in energy and profile.energy.sell_fee.default is not None:
energy["sell_fee"] = profile.energy.sell_fee.default
# Apply default for sell_adjust (default=0) if absent.
if "sell_adjust" not in energy and profile.energy.sell_adjust.default is not None:
energy["sell_adjust"] = profile.energy.sell_adjust.default
@@ -347,6 +351,7 @@ def _validate_tibber_values(values: dict[str, Any], profile: TibberProfile) -> d
# Required energy fields.
_require_numeric("energy", "energy_tax", energy)
_require_numeric("energy", "sell_fee", energy)
_require_numeric("energy", "sell_adjust", energy)
# Required standing fields.
_require_numeric("standing", "management_fee", standing)
@@ -361,9 +366,10 @@ def validate_values(kind: str, values: dict[str, Any]) -> dict[str, Any]:
"""Validate a contract-values dict against the named profile structure.
Fields that carry a ``default`` in the profile (e.g. ``ode``,
``sell_adjust``, tibber ``management_fee``) are silently filled in when
absent from *values*. Fields with no default that are absent, or fields
whose value is not a number, cause a ``ProfileValidationError``.
``sell_fee``, ``sell_adjust``, tibber ``management_fee``) are silently
filled in when absent from *values*. Fields with no default that are
absent, or fields whose value is not a number, cause a
``ProfileValidationError``.
Parameters
----------
@@ -2,9 +2,10 @@ kind: tibber
label: Tibber 动态电价(15 分钟)
energy:
source: tibber_api # buy = total (from API); sell = total energy_tax sell_adjust
source: tibber_api # buy = total (from API); sell = total energy_tax sell_fee sell_adjust
energy_tax: { unit: EUR/kWh } # subtracted from total to derive sell price (incl. VAT)
sell_adjust: { unit: EUR/kWh, default: 0 } # additional sell-price adjustment (residual spread)
sell_fee: { unit: EUR/kWh, default: 0.0248 } # verkoopvergoeding (feed-in fee, incl. VAT); always subtracted from sell
sell_adjust: { unit: EUR/kWh, default: 0 } # manual sell-price adjustment; net-metering: set = energy_tax to refund the tax
standing: # fixed charges; UI fills per month, engine prorates to days
management_fee: { unit: EUR/month, default: 5.99 }
+16 -3
View File
@@ -209,12 +209,23 @@ def _tibber_strategy(
query on ``starts_at``.
Formula (§3.4):
- ``buy = total`` (Tibber's all-inclusive price, already includes tax)
- ``sell = total energy_tax sell_adjust``
- ``buy = total`` (Tibber's all-inclusive price; already includes energy
tax, VAT and the buy-side ``inkoopvergoeding``)
- ``sell = total energy_tax sell_fee sell_adjust``
- ``import_cost = (Δd1 + Δd2) × buy``
- ``export_revenue = (Δr1 + Δr2) × sell``
- ``net_cost = import_cost export_revenue``
``sell_fee`` models Tibber's per-kWh **verkoopvergoeding** (feed-in fee,
€0.0248/kWh incl. VAT since 2026-01-01). It is always deducted from the
feed-in payout: even under the net-metering (saldering) scheme, Tibber pays
``total verkoopvergoeding`` per returned kWh (Tibber NL: "€0,28 €0,0248
= €0,2552"). ``total`` already contains the equal buy-side
``inkoopvergoeding``, so the two fees do **not** cancel — the feed-in price
sits ``sell_fee`` below the buy price. ``sell_adjust`` is a separate manual
correction: under net metering it carries back the refunded energy tax
(``sell_adjust = energy_tax``), leaving ``sell = total sell_fee``.
Tibber does not differentiate tariff slots (dal vs normal) — the 15-minute
API price applies to the full delivered/returned volume.
@@ -248,11 +259,12 @@ def _tibber_strategy(
energy = values.get("energy", {})
energy_tax = _to_decimal(energy.get("energy_tax", 0))
sell_fee = _to_decimal(energy.get("sell_fee", 0))
sell_adjust = _to_decimal(energy.get("sell_adjust", 0))
total = _to_decimal(price_row.total)
buy = total
sell = total - energy_tax - sell_adjust
sell = total - energy_tax - sell_fee - sell_adjust
total_delivered = deltas.d1 + deltas.d2
total_returned = deltas.r1 + deltas.r2
@@ -269,6 +281,7 @@ def _tibber_strategy(
"buy": str(buy),
"sell": str(sell),
"energy_tax": str(energy_tax),
"sell_fee": str(sell_fee),
"sell_adjust": str(sell_adjust),
}
+42 -14
View File
@@ -30,20 +30,39 @@ logger = logging.getLogger(__name__)
_TIBBER_API_URL = "https://api.tibber.com/v1-beta/gql"
_DEFAULT_TIMEOUT = 15.0
# GraphQL query to fetch a range of 15-minute price nodes.
# ``priceInfoRange(resolution: QUARTER_HOURLY, first: 96)`` fetches up to
# 96 quarter-hourly slots which covers today + tomorrow (2 × 24 × 4 = 192 max,
# but the Tibber API typically starts from the current slot and returns at
# most the remaining hours of today plus tomorrow, so 96 is a good cap for
# "today + tomorrow").
# GraphQL query to fetch the forward-looking today + tomorrow price curve at
# 15-minute resolution.
#
# ``priceInfo(resolution: QUARTER_HOURLY)`` returns two node lists:
# * ``today`` — always the full current local day (96 quarter-hourly slots,
# 00:00 → 23:45 local), regardless of the current time.
# * ``tomorrow`` — the full next local day (96 slots) once Tibber publishes the
# day-ahead prices (around 13:0015:00 local); empty before that.
#
# This is deliberately NOT ``priceInfoRange``: that field is a historical cursor
# connection whose range ends at "now" (it never returns future slots), so it
# cannot supply upcoming prices. ``priceInfo`` is forward-looking, so every
# 15-minute slot's price is present in the DB *before* the slot closes — which is
# what makes per-slot billing accurate (each period finds its own exact slot
# instead of falling back to a stale earlier price) and keeps the live current-
# price entity fresh. The hourly refresh job re-runs this query, so tomorrow's
# prices are picked up within an hour of publication without a restart.
_PRICE_RANGE_QUERY = """
{
viewer {
homes {
id
currentSubscription {
priceInfoRange(resolution: QUARTER_HOURLY, first: 96) {
nodes {
priceInfo(resolution: QUARTER_HOURLY) {
today {
startsAt
total
energy
tax
currency
level
}
tomorrow {
startsAt
total
energy
@@ -230,11 +249,15 @@ def fetch_price_range(
*,
timeout: float = _DEFAULT_TIMEOUT,
) -> list[PricePoint]:
"""Fetch a range of 15-minute price nodes from the Tibber API.
"""Fetch the forward-looking today + tomorrow 15-minute price curve from Tibber.
Sends the ``priceInfoRange(resolution: QUARTER_HOURLY, first: 96)`` query
and parses every returned node into a ``PricePoint``. The number of nodes
is not assumed — all returned nodes are parsed regardless of count.
Sends the ``priceInfo(resolution: QUARTER_HOURLY) { today tomorrow }`` query
and parses every node from both lists (today first, then tomorrow) into a
``PricePoint``. ``priceInfo`` is forward-looking — ``today`` is always the
full current local day and ``tomorrow`` is populated once Tibber publishes the
day-ahead prices — so upcoming slots are returned, unlike ``priceInfoRange``
which only reaches "now". ``tomorrow`` may be empty (before publication); the
number of nodes is not assumed and all returned nodes are parsed.
Parameters
----------
@@ -268,10 +291,15 @@ def fetch_price_range(
home = _pick_home(homes, home_id)
try:
nodes = home["currentSubscription"]["priceInfoRange"]["nodes"]
price_info = home["currentSubscription"]["priceInfo"]
today = price_info["today"]
tomorrow = price_info["tomorrow"]
except (KeyError, TypeError) as exc:
raise TibberError("Tibber API response missing priceInfoRange nodes") from exc
raise TibberError("Tibber API response missing priceInfo today/tomorrow") from exc
# tomorrow is null/empty until Tibber publishes the day-ahead prices; treat
# a missing list as empty so we still return today's slots.
nodes = list(today or []) + list(tomorrow or [])
return [_parse_node(node, "QUARTER_HOURLY") for node in nodes]
+18 -4
View File
@@ -121,15 +121,29 @@ class CostsResponse(BaseModel):
class SummaryResponse(BaseModel):
"""Response for GET /api/energy/costs/summary.
All monetary values are in ``currency``.
Monetary values are in ``currency``; the ``*_kwh`` fields are energy totals
in kWh. ``metered_import``/``metered_export`` are **money**, not energy —
only the ``_kwh``-suffixed fields carry kWh.
``total_payable = metered_net + fixed_costs credits``
"""
currency: str
metered_import: float = Field(description="Σ import_cost for non-degraded periods.")
metered_export: float = Field(description="Σ export_revenue for non-degraded periods.")
metered_net: float = Field(description="Σ net_cost for non-degraded periods.")
metered_import: float = Field(
description="Σ import_cost for non-degraded periods (money, in `currency`)."
)
metered_export: float = Field(
description="Σ export_revenue for non-degraded periods (money, in `currency`)."
)
metered_net: float = Field(
description="Σ net_cost for non-degraded periods (money, in `currency`)."
)
metered_import_kwh: float = Field(
description="Σ (d1_kwh + d2_kwh) for non-degraded periods (energy imported, kWh)."
)
metered_export_kwh: float = Field(
description="Σ (r1_kwh + r2_kwh) for non-degraded periods (energy exported, kWh)."
)
fixed_costs: float = Field(
description="Standing charges (network_fee + management_fee) apportioned over the interval."
)
+22 -3
View File
@@ -18,6 +18,12 @@ M6 design document, extended in M7-T03 to be meter-aware:
÷ 30 per day) and subtracts the energy-tax credit (heffingskorting,
apportioned at EUR/year ÷ 365 per day).
The summary reports **both** money and energy: ``metered_import`` /
``metered_export`` are monetary totals (Σ import_cost / Σ export_revenue),
while ``metered_import_kwh`` / ``metered_export_kwh`` are the corresponding
metered energy totals in kWh. The ``_kwh`` suffix is the only thing that
distinguishes them — always check it before labelling a value in a UI.
Design notes
------------
- **Decimal arithmetic throughout**: all monetary computations use
@@ -732,9 +738,11 @@ def summarize(session: Session, start: datetime, end: datetime) -> dict[str, Any
dict with keys:
currency str ISO 4217 currency (from contract, or "EUR" fallback)
metered_import float Σ import_cost from non-degraded periods
metered_export float Σ export_revenue from non-degraded periods
metered_net float Σ net_cost from non-degraded periods
metered_import float Σ import_cost from non-degraded periods (money)
metered_export float Σ export_revenue from non-degraded periods (money)
metered_net float Σ net_cost from non-degraded periods (money)
metered_import_kwh float Σ (d1_kwh + d2_kwh) from non-degraded periods (energy)
metered_export_kwh float Σ (r1_kwh + r2_kwh) from non-degraded periods (energy)
fixed_costs float standing charges for elapsed whole local days
credits float energy-tax credit for elapsed whole local days
total_payable float metered_net + fixed_costs credits
@@ -763,6 +771,15 @@ def summarize(session: Session, start: datetime, end: datetime) -> dict[str, Any
sum_export = sum((_to_decimal(r.export_revenue) for r in good_rows), Decimal("0"))
sum_net = sum((_to_decimal(r.net_cost) for r in good_rows), Decimal("0"))
# Σ metered energy (kWh), summed across both tariff registers. Reuses the
# already-fetched ``good_rows`` so no extra query is issued.
sum_import_kwh = sum(
(_to_decimal(r.d1_kwh) + _to_decimal(r.d2_kwh) for r in good_rows), Decimal("0")
)
sum_export_kwh = sum(
(_to_decimal(r.r1_kwh) + _to_decimal(r.r2_kwh) for r in good_rows), Decimal("0")
)
# --- Interval length in days (window, not elapsed — kept for API compat) ---
total_seconds = (end_utc - start_utc).total_seconds()
days = _to_decimal(str(total_seconds)) / _to_decimal("86400")
@@ -885,6 +902,8 @@ def summarize(session: Session, start: datetime, end: datetime) -> dict[str, Any
"metered_import": float(sum_import),
"metered_export": float(sum_export),
"metered_net": float(sum_net),
"metered_import_kwh": float(sum_import_kwh),
"metered_export_kwh": float(sum_export_kwh),
"fixed_costs": float(fixed_dec),
"credits": float(credits_dec),
"total_payable": float(total_payable),
+1
View File
@@ -64,6 +64,7 @@ def poll_device(session: Session, device: ModbusDevice) -> ModbusReading | None:
device.port,
device.unit_id,
[{"start": b.start, "count": b.count} for b in profile.blocks],
function_code=profile.function_code,
)
payload = profiles.decode(profile, registers)
+2
View File
@@ -88,6 +88,8 @@ pyproject-hooks==1.2.0
# via
# build
# pip-tools
pyserial==3.5
# via -r requirements.in
pytest==8.4.2
# via -r dev-requirements.in
python-dotenv==1.2.2
+2
View File
@@ -9,6 +9,8 @@
- [`m5-iot-energy.md`](./m5-iot-energy.md) — IoT 集成与能耗采集(Modbus/Energy + MQTT/HA Discovery + 前端侧边栏)
- [`m6-tibber-dynamic-energy.md`](./m6-tibber-dynamic-energy.md) — 通用电价层 + DSMR 实时电表接入 + 实时买卖电费计算 + HA Energy 反哺
- [`m7-meter-epochs-archival.md`](./m7-meter-epochs-archival.md) — 电表生命周期 / 换表归档(Meter epochs
- [`pre-m8-warmtelink-p1-poc.md`](./pre-m8-warmtelink-p1-poc.md) — WarmteLink P1 真机概念验证(已完成;正式 CLI 长测与供暖变化均经物理表复核)
- [`m8-warmtelink-energy.md`](./m8-warmtelink-energy.md) — WarmteLink P1、多数据源 Meter 与热力计费(Planning 已完成;M8-T01M8-T20 待实现)
本文件定义**所有任务共用的格式与协作规则**,各个里程碑文档不再重复这些约定。
+5 -4
View File
@@ -118,8 +118,9 @@ credits:
kind: tibber
label: Tibber 动态电价(15 分钟)
energy:
source: tibber_api # buy = total; sell = total energy_tax sell_adjust
source: tibber_api # buy = total; sell = total energy_tax sell_fee sell_adjust
energy_tax: { unit: EUR/kWh }
sell_fee: { unit: EUR/kWh, default: 0.0248 } # verkoopvergoeding, always subtracted
sell_adjust: { unit: EUR/kWh, default: 0 }
standing:
management_fee: { unit: EUR/month, default: 5.99 }
@@ -164,7 +165,7 @@ credits:
1. 取各寄存器在 `t0`/`t1` 的值(`recorded_at ≤ 边界` 的最后一行,Decimal),算 **per-register 差**`Δd1,Δd2,Δr1,Δr2`。
2. 取 active 合同**在 t0 生效的版本** + 其 strategy
- `manual``import_cost = Δd1×(buy_dal) + Δd2×(buy_normal)``buy_x = energy_buy_x + energy_tax + ode`);`export_revenue = Δr1×sell_dal + Δr2×sell_normal`。
- `tibber`:取覆盖 t0 的 `tibber_price``starts_at ≤ t0` 最近一条);`buy = total`、`sell = total energy_tax sell_adjust``import_cost = (Δd1+Δd2)×buy`、`export_revenue = (Δr1+Δr2)×sell`。
- `tibber`:取覆盖 t0 的 `tibber_price``starts_at ≤ t0` 最近一条);`buy = total`、`sell = total energy_tax sell_fee sell_adjust``sell_fee`=verkoopvergoeding,默认 0.0248,见下修正说明)`import_cost = (Δd1+Δd2)×buy`、`export_revenue = (Δr1+Δr2)×sell`。
3. `net_cost = import_cost export_revenue`**upsert** `energy_cost_period`**快照**当时用的价 + `contract_version_id`。
- 缺价/缺数据:跳过或标 `degraded`,留待重算。**不做净计量**(进出口分开累加)。
@@ -240,7 +241,7 @@ credits:
1. **两层电价模型**profile YAML 定结构(仓库、固定、UI 不可编辑)+ `EnergyContract`(+版本) 存数值(UI 填、版本化)+ strategy 按 kind 出价。仿 M5。
2. **kind 不叫 "fixed"**`manual`(人工填、可双费率、可带时段)/ `tibber`API 动态);合同 `name` UI 自由填。
3. **买价**tibber = API `total`(全包,已证 total=energy+tax);manual = `energy_buy_档 + energy_tax`。**卖价**tibber = `total energy_tax sell_adjust`manual = `sell_档`(回送价,无能源税)。均含 VAT。
3. **买价**tibber = API `total`(全包,已证 total=energy+tax;含 inkoopvergoeding);manual = `energy_buy_档 + energy_tax`。**卖价**tibber = `total energy_tax sell_fee sell_adjust``sell_fee`=verkoopvergoeding 默认 0.0248manual = `sell_档`(回送价,无能源税)。均含 VAT。
4. **双费率**manual 用 `delivered_1/2`、`returned_1/2` 分 dal/normal 计价(`_1`=dal/低、`_2`=normal/高);tibber 求和、15min 价不分档。
5. **两层费用**:每 15min `energy_cost_period` 只算计量电费(不可变、快照价);日/月/年汇总再加固定费(按月→天)− heffingskorting(按年→天)。
6. **回送阶梯罚金(terugleverkosten)不做**:按自然年累计、用户住不到年底算不准——不算、不记、不加功能(留痕见 §10)。
@@ -475,7 +476,7 @@ Phase DAPI + 前端)
## 13. 待确认 / TODO(拿到真实 token + 账单后钉死,均已落成配置/默认值,不阻塞实现)
1. **买价**:✅ tibber = API `total`demo 已证 total=energy+tax);manual = energy_buy_档 + energy_tax。无待办。
2. **卖价残差(tibber**`sell = total energy_tax sell_adjust``sell_adjust` 默认 0(买卖费相等抵消)。真实账单确认后若有残差再调。
2. **卖价残差(tibber**~~`sell = total energy_tax sell_adjust``sell_adjust` 默认 0(买卖费相等抵消)~~ → **已修正(2026-07,见 references §3.1**`total` 含 inkoopvergoeding,净计量回送 = `total verkoopvergoeding`,两费**不抵消**。公式改为 `sell = total energy_tax sell_fee sell_adjust`,新增 `sell_fee`(默认 0.0248,始终扣除);`sell_adjust` 净计量期设 `energy_tax`。真实账单确认后若有残差再调 `sell_fee`
3. **双费率寄存器映射**`_1`=dal/低、`_2`=normal/高(NL 惯例)——接价前用真实数据确认别接反(差价小但要对)。
4. **能源税年值**manual/tibber 的 `energy_tax` 默认 ~0.11082026 第一档含 VAT),按当年实际值核。
5. **Tibber 15min + 币种**:✅ 查询/分辨率已 demo 证实;仍需合同生效后用**真实 token** 确认 NL 返回真 15 分钟价 + 币种 EUR。
File diff suppressed because it is too large Load Diff
+333
View File
@@ -0,0 +1,333 @@
# Pre-M8 — WarmteLink P1 真机概念验证
> **状态:已完成。** 2026-08-22 已完成临时脚本真机 bring-up;随后仓库内正式 probe 以
> `115200 7N1` 连续运行 10 分钟,形成脱敏的可重复验收证据;最终人工走查开启供暖后又观察到
> 区域供暖累计量从 `0.017 GJ` 增至 `0.018 GJ`,并与物理表一致。Pre-M8 现已解除
> **M8 Planning** 的入口限制;M8 的架构和实现范围仍未锁定。
## 1. 目的
在进入 M8 正式设计和实现前,用新家的 Vattenfall WarmteLink 建立一条只读 P1 验证链,回答:
1. USB→P1 线、主机串口权限和 WarmteLink P1 端口能否稳定输出可解析数据。
2. 真机使用什么串口参数,telegram 的 framing、CRC、时间戳和 OBIS/M-Bus channel 有何特征。
3. 实际能够读取哪些累计量,以及单位、精度、更新频率和累计语义。
4. P1 值是否与热量表/生活热水表面板一致。
Pre-M8 是 M8 的证据门:它只交付真机事实和可重复 probe,不决定数据库结构、Meter 数据源
绑定、后台采集服务或前端布局。
## 2. 执行边界
本阶段只建立下面这条最短链路:
```text
WarmteLink P1 → USB serial → 原始 telegram → 完整性状态 → 字段解析 → 终端输出
```
明确不做:
- 不写入 `app.db`,不新增 Alembic migration。
- 不新增 FastAPI API、后台常驻 worker、配置页面或 Energy 前端。
- 不发布 MQTT / Home Assistant Discovery。
- 不修改现有 DSMR Reader MQTT、电费计算或 Meter 逻辑。
- 不在本阶段决定 WarmteLink 应落在哪个正式 Device/Source 模型中。
- 不写串口、不修改 FTDI EEPROM;probe 必须严格只读。
## 3. 2026-08-22 真机事实
### 3.1 USB、权限与线材
- 新 USB→P1 线是 FTDI FT232R,稳定路径形态为
`/dev/serial/by-id/usb-FTDI_FT232R_USB_UART_<redacted>-if00-port0`;正式配置不得依赖
`/dev/ttyUSB1`
- 运行用户加入 `dialout` 后可以直接读取 tty,无需 root,也不应把容器作为硬件 bring-up
的中间层。
- 已用另一根已知正常的 DSMR 线在电表上读到连续、CRC 正确的 DSMR telegram,证明宿主机
串口读取方法本身可用。
- 曾临时清除新 FTDI 线 EEPROM 的 `INVERT_RXD` 做对照;输出发生变化但仍不可解析,随后已将
EEPROM 逐字节恢复为原厂镜像并回读校验。正式方案不得依赖 EEPROM 修改。
### 3.2 串口参数矩阵
真机不是按最初假设的 `115200 8N1` 得到可读正文;当前线材与 WarmteLink 的实测最佳组合为:
```text
115200 baud, 7 data bits, no parity, 1 stop bit7N1
```
| 参数 | 实测结果 |
| --- | --- |
| `115200 7N1` | 正文稳定可读,可枚举 9 个 OBIS 字段 |
| `115200 7N2` | 同样可读;没有理由增加停止位,正式默认仍用 `7N1` |
| `115200 8N1/8E1/8O1` | 乱码,无有效 OBIS/CRC |
| `115200 7E1/7O1` | 乱码,无有效 OBIS/CRC |
| `1200003000000`,分别用 `8N1``7N1` | 无完整报文;`120000` 仅残留少量可辨识文本 |
| XON/XOFF 开/关 | 不改变 `7N1` 的字段解析结果 |
[DSMR 5.0.2 P1 Companion Standard](https://www.netbeheernederland.nl/sites/default/files/2024-02/dsmr_5.0.2_p1_companion_standard.pdf)
规定 `115200 8N1`。因此 `7N1` 是当前设备/线材组合的实测事实,不应被文档或代码包装成
标准 DSMR framing;CLI 必须允许显式覆盖数据位、校验位和停止位。
### 3.3 Framing 与 CRC 异常
- 临时 bring-up 的一次 8 帧盘点中每帧均为 256 字节;正式 10 分钟采样显示长度并不固定,
详见 §3.6。
- 正文结构稳定,版本、时间戳、两个 M-Bus channel、单位和值均可重复解析。
- 实测头部为 `)TU)2NWA-MYRSKY`,偶见 `)TU{2NWA-MYRSKY`;没有标准要求的 `/` 起始符。
- 帧尾存在 `!`,但其后的字符并非每帧都稳定为四位十六进制;即使恰好是四位,也无法从
缺失的 `/` 起点完成标准 DSMR CRC16 验证。
因此正式 probe 必须把完整性明确表示为 `valid``invalid``unverifiable`,保留原始字节并
输出原因。它可以在 `unverifiable` 状态下枚举字段用于 PoC,但绝不能把该帧报告为 CRC 已通过。
M8 若要持久化这些读数,必须在 Planning 中单独锁定异常帧的接纳、重复确认和告警策略。
### 3.4 实际字段清单
连续 8 帧只出现以下 9 个字段;除 capture timestamp 外,字段集合和值均稳定:
| OBIS | 实测结构/值 | 结论 |
| --- | --- | --- |
| `1-3:0.2.8` | `(50)` | DSMR P1 输出版本 5.0 |
| `0-0:1.0.0` | `(YYMMDDhhmmssX)` | telegram/capture timestamp,每 10 秒变化 |
| `0-0:96.1.1` | `<redacted>` | WarmteLink/gateway equipment identifier |
| `0-1:24.1.0` | `(006)` | channel 1M-Bus device type `0x06`,生活热水 |
| `0-1:96.1.0` | `<redacted>` | channel 1 equipment identifier;样本制造商可解码为 `KAM` |
| `0-1:24.2.1` | `(<timestamp>)(5.900*m3)` | 生活热水累计体积,输出到 `0.001 m³` 小数位 |
| `0-2:24.1.0` | `(012)` | channel 2M-Bus device type `0x0C`,热量表 |
| `0-2:96.1.0` | `<redacted>` | channel 2 equipment identifier;样本制造商可解码为 `KAM` |
| `0-2:24.2.1` | `(<timestamp>)(0.017*GJ)` | 区域供暖累计热量,输出到 `0.001 GJ` 小数位 |
人工面板在同一时间显示 `5.900 m³``0.017 GJ`,与 P1 值完全一致;用户已确认这两个累计量
足以作为后续 Home Assistant Energy 展示的数据基础。
没有发现瞬时流量、当前热功率、供水温度或回水温度字段。完整 OMS/M-Bus 模型允许这些可选
量,但 WarmteLink 当前 P1 telegram 没有导出它们;Pre-M8 和 M8 不得假设它们可用。参考
[OMS Specification Vol. 2 Annex A](https://oms-group.org/wp-content/uploads/2024/05/OMS-Spec_Vol2_AnnexA_F121.pdf)。
### 3.5 对 M8 已经成立的事实
- 一个 WarmteLink serial source 同时暴露两个独立累计 measurement channel。
- channel 1 是生活热水 `m³`channel 2 是区域供暖 `GJ`,不能按论坛样例固定 channel。
- 两个累计量都与物理表一致,可进入 M8 的 Meter/source 映射讨论。
- 当前没有瞬时流量、温度或热功率;M8 只承诺累计量采集与展示。
- 串口 framing 和 CRC 异常尚未消失,必须作为正式 ingestion 的显式质量状态处理。
### 3.6 正式 CLI 10 分钟复验(脱敏)
2026-08-22,仓库内 `python -m scripts.p1_probe` 在真实
`/dev/serial/by-id/<redacted>` 上以 `115200 7N1` 运行 `--duration 600 --show-changes`,原始
bytes 仅写入 `/tmp`。CLI 正常以 exit code 0 结束;期间没有 I/O error、未处理异常或断连。
- 读取到 60 个完整 frame、15,362 raw bytes;最后 2 bytes 是下一帧的不完整残片,60 个完整帧
合计 15,360 bytes。
- 设备 timestamp 从 `18:16:10``18:26:00`,严格每 10 秒递进。CLI 处理的 59 个相邻间隔中,
53 个为 10.0 s、1 个为 9.0 s、3 个为 0.0 s、2 个为 20.0 s,均值 9.81 s0/20 s 配对来自
serial chunk 的批量交付,不代表设备 cadence 改变。
- 完整 frame 长度分布为 237 bytes × 2、239 × 1、254 × 12、256 × 31、258 × 11、275 × 3
平均 256 bytes、范围 237–275。不能再把临时样本的「固定 256 bytes」视为帧格式契约;变长与
非标准 footer/分块边界一致。
- 60/60 的完整性均为 `unverifiable`,因为 60/60 缺少标准 `/` 起始符。footer 长度为 4 × 41、
6 × 16、23 × 3;仅 16/60 的 footer 恰为四位十六进制,但仍不能在缺失 `/` 时完成标准 CRC16
验证。**没有任何 frame 被报告为 CRC valid。**
- 60/60 帧均解析到同一组 9 个 OBIS code`1-3:0.2.8``0-0:1.0.0``0-0:96.1.1`
`0-1:24.1.0``0-1:96.1.0``0-1:24.2.1``0-2:24.1.0``0-2:96.1.0``0-2:24.2.1`
equipment identifier 已脱敏,不进入 Git。
- channel 1 的 device type 是 `006`,累计值稳定为 `5.900 m³`channel 2 的 device type 是
`012`,累计值稳定为 `0.017 GJ`。同日人工表盘复核也显示 `5.900 m³` / `0.017 GJ`,正式 CLI
因而复现了该基线。
- 60 帧中未见瞬时流量、热功率、供水温度或回水温度字段。10 分钟内累计值未变化只能证明这段
时间的累计值稳定;它不能证明发生消费时的更新频率,也不能证明 reset 或 wrap 行为。
### 3.7 最终人工走查:供暖累计量变化
正式长测交付后,用户又在终端直接运行只读 probe 十几分钟并开启供暖。channel 2 的区域供暖
累计量在本次运行中从 `0.017 GJ` 增至 `0.018 GJ`,同一时刻物理热量表也显示 `0.018 GJ`
因此可以确认当前 P1 输出会在实际供暖消费下更新累计量,且 `0.001 GJ` 的变化与物理表一致。
这次人工走查没有改变完整性结论:telegram 仍缺少标准 `/`,数值与物理表一致不能替代 CRC
验证。走查也没有覆盖 reset、wrap 或精确更新延迟;这些仍须由 M8 的接纳、重复确认和告警策略
处理,而不能从一次累计量递增外推。
## 4. 正式 probe 的预期操作方式
实现后,在 workspace virtual environment 中运行只读 probe
```bash
source .venv/bin/activate
python -m scripts.p1_probe \
--device /dev/serial/by-id/<usb-p1-device> \
--baudrate 115200 \
--bytesize 7 \
--parity N \
--stopbits 1 \
--duration 600 \
--show-changes \
--raw-output /tmp/warmtelink-p1-telegram.bin
```
参数名可在实现时小幅调整,但必须保留这些能力:
- 设备路径由用户显式传入,文档推荐 `/dev/serial/by-id/...`
- 串口默认采用本机实测 `115200 7N1`,同时允许显式覆盖 framing。
- 连续读取多帧,输出 telegram cadence、帧长度和读取/重连错误。
- 同时显示完整性/CRC 状态、原始 OBIS 字段和解析后的 channel、设备类型、值与单位。
- parser 不依赖字段固定顺序,也不把 GJ/m³ 固定到 channel 1 或 2。
- `--show-changes` 可只显示发生变化的字段,原始捕获可写入用户指定的 `/tmp` 路径。
- 捕获文件默认按原始 bytes 保存;未经脱敏不得提交到 Git。
- permission denied 时给出 `dialout` 指引;不得建议以 root 常驻运行。
Home Assistant Community 的
[最小 WarmteLink Bash 读取方法](https://community.home-assistant.io/t/solved-dsmr-add-warmtelink-as-data-source/485255/2)
仍可作为快速可见性检查,但论坛样例的 header、channel 和 device type 与本机均不同,不能作为
parser 契约。是否引入 [`dsmr_parser`](https://github.com/ndokter/dsmr_parser) 也由 T01 的实测
兼容性决定;若它要求标准 `/...!CRC` framing,则应保留小型专用 parser,而不是绕过其校验。
## 5. 实现任务
### PRE-M8-T01 — 纯函数 telegram framing、CRC 与 OBIS parser
- **Status**: `done`
- **Depends**: `none`
- **Context**: 先把串口 I/O 与解析分开,用脱敏 fixture 固定标准 DSMR 帧和本机异常帧行为。
**Files**
- `create scripts/p1_probe.py`
- `create tests/fixtures/dsmr_p1_valid.txt`
- `create tests/fixtures/warmtelink_p1_7n1.txt`
- `create tests/test_p1_probe.py`
**Steps**
1. 实现不依赖串口的增量 framing、DSMR CRC16 和通用 OBIS 行解析函数。
2. 用数据结构表达原始 header/footer、完整性状态、timestamp、channel、device type、equipment id、
value 和 unit;数值使用 `Decimal`,不使用二进制浮点保存累计量。
3. 加入一份完全脱敏的本机结构 fixture,并另造一份 CRC 正确的标准 DSMR fixture。
4. 对字段重排、分块输入、缺失 `/`、非十六进制 footer、CRC mismatch、未知字段和两个 channel
写单元测试。
**Out of scope / 不要碰**
- 不打开真实 serial device,不增加依赖,不写数据库/API/MQTT。
- 不因本机正文可读而伪造 `/` header 或把 CRC 状态升级为 valid。
**Acceptance criteria**
- [x] 标准 fixture 的 frame boundary 与 CRC 可验证为 `valid`
- [x] 脱敏本机 fixture 被标为 `unverifiable`,但能按字段而非位置解析 `m³``GJ` channel。
- [x] 任意 chunk boundary 和字段顺序不影响结果,未知字段原样保留。
- [x] 累计量以 `Decimal` + 原单位返回。
- [x] `pytest tests/test_p1_probe.py``pytest``ruff check .` 全绿。
**Reviewer checklist**
- CRC 覆盖范围必须严格从 `/``!`(包含二者),不得对缺失字节做猜测性修补。
- fixture 必须脱敏且保留足以复现 framing 异常的字节结构。
- parser 不得硬编码 channel 1=GJ 或 channel 2=m³。
### PRE-M8-T02 — 只读 serial probe CLI
- **Status**: `done`
- **Depends**: `PRE-M8-T01`
- **Context**: 在纯 parser 通过后增加最小 serial I/O,使真机验证可以从仓库稳定复现。
**Files**
- `modify requirements.in`
- `modify requirements.txt`
- `modify dev-requirements.txt`
- `modify scripts/p1_probe.py`
- `modify tests/test_p1_probe.py`
**Steps**
1. 增加受约束的 `pyserial` runtime 依赖并用仓库既有 pip-compile 流程同步生成 requirements。
2. 实现 `--device`、framing 参数、`--duration``--show-changes``--raw-output`
3. 默认使用 `115200 7N1`;串口只读,禁止 write、EEPROM 或自动修改设备配置。
4. 输出每帧完整性状态、全部字段、值变化、cadence 和错误;SIGINT/超时后关闭串口并正常退出。
5. 用 fake serial/chunk stream 测试 CLI,不要求 CI 存在 `/dev/ttyUSB*`
**Out of scope / 不要碰**
- 不做 daemon、自动重连 worker、Docker device mapping、数据库、API、MQTT 或 HA Discovery。
- 不内置本机 FTDI 序列号,不自动扫描或改写任意 USB 设备。
**Acceptance criteria**
- [x] CLI 可用 `/dev/serial/by-id/...` 读取,且所有 framing 参数都可显式覆盖。
- [x] 默认参数准确反映本机 `115200 7N1`,帮助文本说明它是实测值而非 DSMR 标准默认。
- [x] raw output 保留原始 bytes;终端清楚区分 `valid``invalid``unverifiable`
- [x] permission/busy/disconnect 错误非零退出并给出可执行诊断,绝不建议常驻 root。
- [x] 依赖输入与两个生成 requirements 文件同步。
- [x] `pytest tests/test_p1_probe.py``pytest``ruff check .` 全绿。
**Reviewer checklist**
- 确认所有 serial write path 均不存在。
- 确认测试完全 mock 硬件、没有 CI timing flake,退出路径总会关闭文件描述符。
- 确认 requirements 是生成结果而非仅手改 lock file。
### PRE-M8-T03 — 正式真机验收与 M8 交接
- **Status**: `done`
- **Depends**: `PRE-M8-T02`
- **Context**: 用仓库内 probe 替代本轮临时脚本,形成可重复、脱敏且能支撑 M8 Planning 的证据。
**Files**
- `modify docs/design/pre-m8-warmtelink-p1-poc.md`
- `modify docs/design/m8-warmtelink-energy.md`
- `modify docs/design/README.md`
- `modify docs/roadmap.md`
**Steps**
1. 在真实 `/dev/serial/by-id/...` 上运行 probe 至少 10 分钟,并保留原始捕获在 `/tmp`
2. 汇总帧数、cadence、长度、完整性状态、全部字段和读数变化;不得提交原始设备标识。
3. 再次与物理表对照 GJ 和 m³,并记录累计/重置行为中本次能证实和不能证实的部分。
4. 更新本节事实、通过条件和 M8 入口;只有证据齐全后才把 Pre-M8 标记为完成。
**Out of scope / 不要碰**
- 不为完成 checklist 而修补原始字节或放宽 CRC 结果。
- 不进入 M8 schema/API/worker/frontend 实现。
**Acceptance criteria**
- [x] 仓库内 probe 在真机连续运行至少 10 分钟,无未处理异常退出。
- [x] 脱敏摘要列出两个累计 channel、单位、精度、cadence 和完整性异常。
- [x] `0.017 GJ``5.900 m³` 的基线或运行时新值与物理表再次对照。
- [x] 明确没有从当前 P1 输出读取到瞬时流量、功率或温度。
- [x] Pre-M8 状态与 roadmap/M8 入口同步;代码闸门保持全绿。
**Reviewer checklist**
- 证据必须来自正式 CLI,不得只复述本轮临时脚本结果。
- 任何 equipment id、FTDI serial 和未脱敏 raw capture 都不得进入 Git。
- CRC/framing 风险必须原样交给 M8,不能用“数值看起来正确”替代完整性判断。
## 6. 当前通过条件
- [x] USB、tty 权限和稳定 `/dev/serial/by-id/...` 路径已验证。
- [x] 串口参数矩阵已完成,实测正文可读参数为 `115200 7N1`
- [x] 已枚举全部实际 channel/OBIS 字段,并确认没有轮换出现的额外测量量。
- [x] GJ 和生活热水 m³ 均与物理表面板完全一致。
- [x] 已记录精度、约 10 秒 telegram cadence 和当前字段集合。
- [x] framing/CRC 失败已保留为显式异常,没有误报为校验通过。
- [x] 仓库内 parser、fixtures、probe CLI 和自动化测试完成。
- [x] 正式 probe 完成至少 10 分钟真机复验并产出脱敏摘要。
- [x] 最终人工走查在供暖开启后观察到 `0.017 → 0.018 GJ`,并再次与物理表核对一致。
Pre-M8 已完成并向 M8 解锁 Planning;它只交付下列真机事实,不锁定正式架构或实现任务。
## 7. 向 M8 的交付物
Pre-M8 完成后只向 M8 交付事实,不交付正式架构:
- 脱敏 telegram 结构、fixture 和全部字段清单。
- GJ 与 m³ 的实际 channel、device type、单位、精度和更新时间。
- 串口参数、稳定设备路径形态与 `dialout` 权限要求。
- parser 适配结论和 `unverifiable` framing/CRC 异常样本。
- “一个 serial source 包含两个独立累计计量 channel”的实测结论。
- 当前 P1 不提供瞬时流量、热功率或温度的明确边界。
Binary file not shown.
+198
View File
@@ -0,0 +1,198 @@
# DDSU666 Modbus 协议(从官方 PDF 提取)
> 来源:`docs/references/DDSU666 Single phase Smart Meter.pdf`
> CHINT / 正泰仪表 **DDSU666 Single phase Smart Meter — Operation Manual**,文档号 `ZTY0.464.1224`,版本 **V2**2020 年 8 月;厂商 Zhejiang Chint Instrument & Meter Co., Ltd.
> 本文件是 PDF 的可读化提取,供本项目的 Modbus 采集驱动设计参考。**以官方 PDF 为准**,本文件如有出入以 PDF 为准。
> 同类文档见 SDM120 的 `SDM120-Modbus-Protocol.md`;两表差异较大,见下方 §6「与 SDM120 的关键差异」。
## 设备速览(来自手册 Table 1 / Table 5
| 项 | DDSU666(直接接入) | DDSU666-CT(经互感器) |
| --- | --- | --- |
| 精度等级 | Active Class B | Active Class C |
| 参考电压 | 230 V | 230 V |
| 电流规格 | 0.255(80) A | 0.0151.5(6) A |
| 表常数 | 800 imp/kWh | 6400 imp/kWh |
| 接入方式 | 直接接入 | 经电流互感器 |
- 单相电子式电能表,DIN35mm 导轨安装;测量电压、电流、有功/无功功率、频率、功率因数、正/反向有功电能。
- 电能测量范围 `0999999.99 kWh`(LCD 只显示 6 位,自动移动小数点)。
- 通信:RS485**Modbus-RTU**(也支持 DL/T 645-2007,可切换,见 §5 `0005H ChangeProtocol`)。
- 手册 Table 1 标注 Frequency Reference = 60Hz,但 LCD 示例又写 `F=50.00Hz`(手册自身不一致);**实际频率以寄存器 `200EH` 读数为准**,不要把 50/60 写死。
## 0. 本项目的接入方式(重要)
DDSU666 物理层是 **Modbus RTURS-485 串口)**,半双工。和 SDM120 一样,本项目通过一个 **Modbus-TCP 网关**接入:
- 后端用 **Modbus TCP**`IP:port`)连到网关,网关在串口侧转成 RTU 与电表通信。
- TCP 帧用 MBAP header、**无 CRC**CRC 由网关在 RTU 侧处理)。本文档里 RTU 帧的 `CRC (Lo/Hi)` 字段在 TCP 模式下不需要我们关心。
- **Slave Address / Unit ID = 电表的通信地址 Addr**(范围 1–247;面板按键只能设 1–99;见 §5 `0006H`),在 TCP 请求里作为 unit id 传入。
- 若以后直连串口(RTU),才需要管波特率 / 数据格式 / CRC:**默认串口格式是 8 数据位、无校验、2 停止位(8N2)**,与 SDM120 的 8N1 不同——直连时务必对齐。
- 电表必须处于 **Modbus 协议模式**(而非 DL/T 645)才能用本协议;可经面板长按切换,或写 `0005H = 2`(见 §5)。
## 1. 协议帧格式(Appendix A
异步传输,按字节为单位。一帧 10 位字符 = **1 起始位(0) + 8 数据位(无校验) + 2 停止位(1)**(其它格式可定制)。
### 信息帧结构(Table A.1
| 字段 | 长度 | 说明 |
| --- | --- | --- |
| Start(起始) | >3.5 字符静默 | 帧间至少 3.5 字符空闲时间作为分隔 |
| Address code(地址码) | 1 字节 | 目标从机地址 1–247;每个从机在总线上地址唯一 |
| Function code(功能码) | 1 字节 | 仅支持 **03H / 10H**(见 §2 |
| Data(数据域) | n 字节 | 随功能码不同而不同(起始地址、寄存器数、寄存器数据等) |
| CRC check codeCRC 校验) | 2 字节 | 16-bit CRC(**低字节在前、高字节在后**;多项式 `A001` |
| End(结束) | >3.5 字符静默 | 帧间静默 |
> TCP 网关模式下 Start/End 静默与 CRC 由网关处理,本项目不关心。
### 功能码 03H 示例(读寄存器,Table A.3/A.4
读从机 `01H`、起始地址 `0CH`、2 个寄存器:
- 主机发送:`01 03 00 0C 00 02 04 08`(最后 `04 08` 是 CRC,低字节 `04` 在前)。
- 从机返回(设 `0CH/0DH` 内容为 `0000H``1388H`):`01 03 04 00 00 13 88 F7 65`
- `04` = 字节数;`00 00` = `0CH` 数据;`13 88` = `0DH` 数据;`F7 65` = CRC(低字节 `F7` 在前)。
> **注意**:单个 16-bit 寄存器内是「高字节在前、低字节在后」(Table A.4 里 `0DH` 数据返回 `13 88` = `0x1388`)。这一点对解码浮点的字节序很关键,见 §3。
### 功能码 10H 示例(写多个寄存器,Table A.5/A.6
向从机 `01H`、起始地址 `00H` 连续写 3 个寄存器 `0002H,1388H,000AH`
- 主机发送:`01 10 00 00 00 03 06 00 02 13 88 00 0A 9B E9`
- `06` = 写入字节数(3 寄存器 × 2 字节);随后是 3 个寄存器的数据;末尾 `9B E9` CRC。
- 从机返回:`01 10 00 00 00 03 80 08`(回显起始地址 + 寄存器数 + CRC `80 08`)。
### 异常响应(Table A.7/A.8
- 异常时返回的 Function Code = **原功能码 + 128**(即最高位置 1`03H→83H``10H→90H`)。
- 数据为单字节 Error Code
| Error Code | 含义 | 说明 |
| --- | --- | --- |
| `01H` | Illegal function code | 收到的功能码本表不支持 |
| `02H` | Illegal register address | 寄存器地址超出有效范围 |
| `03H` | Illegal data value | 数据值超出对应地址的取值范围 |
## 2. 功能码
DDSU666 **只支持两个功能码**Table A.2):
| 功能码 | 作用 | 说明 |
| --- | --- | --- |
| **03H** | Read register(读寄存器) | 读一个或多个寄存器——**测量值、电量、配置全部走它** |
| **10H** | Write multiple registers(写多个寄存器) | 向 n 个连续寄存器写 n 个 16-bit 数据(改配置 / 清电量) |
> ⚠️ **DDSU666 没有「输入寄存器 / FC04」概念**——所有数据(包括电压电流功率)都用 **FC 03H** 读保持寄存器。这是它和 SDM120(测量值走 FC04)最大的踩坑差异,见 §6。
## 3. 数据编码
DDSU666 有两类数据:
1. **配置 / 参数寄存器(§5`0000H``0010H`**:每个 1 个寄存器、`16-bit with symbols`**16 位有符号整数**)。
2. **测量 / 电量寄存器(§4`2000H`+ / `4000H`+**:每个参数 = **32-bit IEEE-754 单精度浮点**(手册写 “single precision floating decimal”),占 **2 个相邻寄存器**Length = 2 Word)。
**浮点字节序 / 字序**
- **字节序(byte order)= 大端:寄存器内高字节在前** —— 由 Table A.4 的 `0x1388` 返回为 `13 88` 确认。
- **字序(word order,两个寄存器谁是高 16 位)**:手册**没有给出浮点解码的实例**,未明确标注。按标准 Modbus 浮点惯例应为**大端字序(高寄存器在前,`ABCD`)**,与本项目 SDM120 驱动一致(`registers_to_float``>f`,高寄存器在前)。
- ⚠️ **需上机实测确认**:读 `2000H`(电压)应解出 ~230V 这样的合理值;若解出乱数,多半是字序相反,改成「低寄存器在前」再试。CHINT 同系列(DTSU/DDSU666)现场固件偶有字序差异,**首次接入务必用一个已知量(电压)校验**,不要凭手册想当然。
> Python 解码(大端、高寄存器在前):`struct.unpack('>f', struct.pack('>HH', hi_reg, lo_reg))[0]`。
> pymodbus`BinaryPayloadDecoder.fromRegisters(regs, byteorder=Endian.BIG, wordorder=Endian.BIG)`。
## 4. 测量 / 电量寄存器表(FC 03H 读)
全部为只读、`Float`32-bit),每项占 **2 个寄存器**。地址为 Modbus 协议原始地址(即帧里的 Start Register Address Hi/Lo),手册用十六进制。
### 4.1 瞬时量(“Electric quantity of the secondary side”,`2000H` 段)
| 地址(hex) | 参数 | 代号 | 单位 | 备注 |
| --- | --- | --- | --- | --- |
| `2000H` | 电压 Voltage | U | V | |
| `2002H` | 电流 Current | I | A | |
| `2004H` | 有功功率 Active power | P | **kW** | 手册标注 “the unit is KW”——**不是 W** |
| `2006H` | 无功功率 Reactive power | Q | **kvar** | |
| `2008H` | (保留 RESERVED | — | — | 占 2 寄存器,跳过 |
| `200AH` | 功率因数 Power factor | PF | — | 无量纲 |
| `200CH` | (保留 RESERVED | — | — | 占 2 寄存器,跳过 |
| `200EH` | 频率 Frequency | Freq | Hz | |
### 4.2 电量(“Electrical data of the secondary side”,`4000H` 段)
| 地址(hex) | 参数 | 代号 | 单位 | 备注 |
| --- | --- | --- | --- | --- |
| `4000H` | 正向(导入)有功电能 Active in electricity | Ep | kWh | 正向 / forward active energy |
| `400AH` | 反向(导出)有功电能 Reverse in electricity | -Ep | kWh | 反向 / reverse active energy |
> 手册里 `4000H` 与 `400AH` 之间(`4002H``4009H`)未列出,视为保留/未文档化。
**读取分块建议**
- 瞬时量:`2000H``200FH`**16 个寄存器连续**,一次块读即可覆盖 U…Freq(含两段 RESERVED,解码时跳过)。
- 电量:`4000H`2 寄存器)与 `400AH`(2 寄存器)相距较远,分两小块读,或读 `4000H``400BH`(12 寄存器)一次取出后挑用——以网关 / 电表是否允许跨保留地址块读为准,谨慎起见分开读更稳。
### 常用核心子集(日常监控够用)
电压 `2000H`、电流 `2002H`、有功功率 `2004H`kW)、功率因数 `200AH`、频率 `200EH`、正向有功电能 `4000H`、反向有功电能 `400AH`
## 5. 配置 / 参数寄存器表(FC 03H 读 / FC 10H 写)(Table 9
每个 1 个寄存器、**16-bit 有符号整数**。`R/W` 列来自手册。
| 地址(hex) | 代号 | 含义 | R/W | 取值 / 说明 |
| --- | --- | --- | --- | --- |
| `0000H` | UCode | 编程密码 Programming password code | R/W | 写配置前的密码字 |
| `0001H` | REV. | 保留;**实际读出的是版本号** | R | |
| `0002H` | ClrE | 电能清零 CLr.E | R/W | **写 `1` 清除总电量**(不可逆,慎用) |
| `0003H``0004H` | RESERVED | 保留 | — | |
| `0005H` | ChangeProtocol | 协议切换 | R/W | **`2` = Modbus-RTU**`1` = DL/T 645-2007 |
| `0006H` | Addr | 通信地址 | R/W | 1247(面板按键仅 199 |
| `0007H``000AH` | RESERVED | 保留 | — | |
| `000BH` | Meter type | 表型 Meter type | R | 只读设备类型标识 |
| `000CH` | BAud | 通信波特率 | R/W | **`1`=2400bps`2`=4800bps`3`=9600bps**(手册寄存器仅列这三档;通信章另提到也支持 1200bps) |
| `000DH``0010H` | RESERVED | 保留 | — | |
> ⚠️ 写 `0002H`(清电量)、`0006H`(改地址)、`000CH`(改波特率)、`0005H`(切协议)都会改变电表状态或通信参数,配错可能**清空累计电量**或**导致通信中断**。本项目默认**只读采集**,不在自动化链路里写电表配置寄存器。
> 写配置通常需先经 `0000H UCode` 密码字校验;具体密码值手册正文未给出,需向厂商确认或经面板操作。
## 6. 与 SDM120 的关键差异(迁移 / 复用驱动时必看)
本项目已有 SDM120 profile`app/integrations/modbus/profiles/sdm120.yaml`)。DDSU666 **不能照搬**,主要差异:
| 维度 | SDM120 (Eastron) | DDSU666 (CHINT) |
| --- | --- | --- |
| 读测量值功能码 | **FC 04**(输入寄存器 3X | **FC 03**(保持寄存器,无 FC04 |
| 测量值起始地址 | `0x0000` 起(30001 | 瞬时量 `0x2000` 起;电量 `0x4000`/`0x400A` |
| 有功功率单位 | **W**(瓦) | **kW(千瓦)** —— 入库前注意换算 / 单位标注 |
| 无功功率单位 | VAr | kvar |
| 配置寄存器格式 | FloatFC03/16 | **16-bit 有符号整数**FC03/10 |
| 写功能码 | 16 / 0x10 | 10H(同 0x10 |
| 串口默认格式 | 8N1(1 停止位) | **8N22 停止位)** |
| 多协议 | 仅 Modbus | Modbus **与 DL/T 645-2007 可切换**(需确保在 Modbus 模式) |
| 浮点字序 | 大端、高寄存器在前(手册有实例佐证) | 字节序大端已确认;**字序手册无实例,需上机实测** |
## 7. 给本项目采集驱动的要点小结
1.**Modbus TCP 网关**`ModbusTcpClient(host, port)``slave=<Addr>`;电表须在 **Modbus 协议模式**
2. 所有读取(测量 + 电量 + 配置)都用 **FC 03H**——**没有 FC04**。
3. 测量值在 `0x2000` 段、电量在 `0x4000`/`0x400A`,均为 **float32**;解码大端字节序,**字序默认高寄存器在前但务必用电压实测校验**。
4. **有功功率单位是 kW、无功是 kvar**——与 SDM120 的 W/VAr 不同,新建 profile / 入库映射时单位别抄错。
5. 配置寄存器(`0x0000``0x0010`)是 **16-bit 有符号整数**,不是 float。
6. 默认**只读**`0002H` 写 1 会**清空累计电量**、`0006H/000CH/0005H` 会改通信参数,自动化链路里一律不写。
7. 新建 profile 时这是 `ddsu666` 这一个 register profile 的定义;建议 `function_code: 3``word_order: big`(先按大端字序,接入后用电压读数验证)、瞬时量与电量分块读。
## 8. 实测记录(真机验证,2026-06-30)
首次接入一台 **DDSU666 直接接入版(5(80)A** 实测,确认以下几点:
- **字序大端,确认无误**:电压 / 电流 / 频率 / 电能用「大端、高寄存器在前」解码全部得到合理值(如 233.9 V / 0.055 A / 49.99 Hz),与 §3 的假设一致。`ddsu666.yaml``word_order: big` / `byte_order: big` **无需修改**,§3 里「字序需上机实测」一项可视为已关闭。
- **FC03 读通**:所有量走 FC03profile `ddsu666` 在采集链路(CLI `read` / 设备 `/test` / 后台轮询)中工作正常。
- **低电流下「瞬时功率读 0、但电能照常累加」**:测试负载仅为一台 PoE 交换机(≈230 V / 0.05 A,真实有功仅几瓦),**远低于本表测量量程下限 Imin≈0.25 A**。此工况下:
- 瞬时 `active_power``0x2004`)与 `power_factor``0x200A`)寄存器返回**全零**(原始 hex `0x0000 0x0000`);电表 LCD 上功率在 0~3.3 W、PF 在 0~0.6 之间抖动。
- 抖动成因 = **低电流测量噪声 + 开关电源(SMPS,无 PFC)畸变电流**(电流为电压峰值附近的窄脉冲、谐波重 → 畸变功率因数天然偏低)。**不是**「采样率与开关频率拍频」:计量芯片 SH79F7019 采样在 kHz 量级,远低于开关电源 50–200 kHz 的开关频率,两者不在一个频段。
- 但累计电能 `import_energy``0x4000`)**正常累加**(实测 0 → 0.01 kWh)——电表内部积分器在防潜动起始电流(`0.004·Ib`≈0.02 A)之上照常计量。
- **结论**:低于量程下限时**瞬时功率 / PF 不可信,但电能计量不丢**;电流进入量程(正常负载)后瞬时量即稳定可信。这是 5(80)A 大量程表对极小负载的固有特性,**非缺陷、非解码问题**。
- **排错提示**:若日后看到 DDSU666「功率一直 0」,先确认负载电流是否在 Imin 以上——多半是负载太轻而非链路故障;可用 `scripts/modbus_cli probe --fc 3 --address 0x2000 --count 16``0x2004/0x2005` 原始寄存器是否真为全零佐证。
@@ -113,16 +113,26 @@ curl -s -X POST https://api.tibber.com/v1-beta/gql \
> "De verkoopvergoeding van 2,48 cent is gelijk aan de inkoopvergoeding die je bij je afgenomen stroom betaalt."
> (卖侧 verkoopvergoeding 2.48 分 = 买侧 inkoopvergoeding。)
**买卖服务费相等(均 €0.0248/kWh)**,在买卖里一进一出**相互抵消**。
**买卖服务费金额相等(均 €0.0248/kWh,但两者对住户都是成本、不互相抵消**
- 买侧 inkoopvergoeding 已经**包含在 Tibber API 的 `total` 里**(见下 §3.1 的实证拆解),买电按 `total` 计价即已含它。
- 卖侧 verkoopvergoeding 则是从回送价里**额外扣掉**的一笔——所以回送价 = `total 0.0248`,比买价低 0.0248/kWh。
- ⚠️ **早期版本误判为"一进一出抵消 → 回送=total"**,这是错的:`total` 里那笔 inkoopvergoeding 不会退回来充抵 verkoopvergoeding。代码里用 `energy.sell_fee`(默认 0.0248)建模这笔卖侧费用。
---
## 3. 净计量(saldering)、回送(teruglevering)、负电价、2027
### 3.1 回送价(净计量期内,文档原文)
### 3.1 回送价(净计量期内,文档原文 + 实证
> "Op het moment dat je teruglevert geven we je per kWh de beursprijs die op dat moment geldt …, inclusief energiebelasting en inkoopvergoeding plus de btw minus de verkoopvergoeding."
即净计量期内回送价 = `beursprijs + energiebelasting + inkoopvergoeding + btw verkoopvergoeding`。因 inkoopvergoeding = verkoopvergoeding 抵消 → **= 全额零售价**spot+能源税+VAT),正是 saldering "回送 1 度 = 用 1 度"的本质。
> **Worked exampleTibber NL 原文)**"Stel dat tussen 14:00 en 14:15 de totale stroomprijs €0,28 per kWh incl. is, dan krijg je €0,28 €0,0248 verkoopvergoeding = **€0,2552** per teruggeleverde kWh terug."
即净计量期内回送价 = `beursprijs + energiebelasting + inkoopvergoeding + btw verkoopvergoeding`,而官方例子直接写成 **`回送价 = totale stroomprijs verkoopvergoeding = total 0.0248`**。能源税**退回**(留在 total 里没动),只有 verkoopvergoeding 这 0.0248 被扣。
**✅ 实证(本项目生产库,2026-07-20 三个刻钟)**:按 21% VAT 拆 `total``total = 现货×1.21 + energiebelasting(0.11085) + inkoopvergoeding(0.0248)`,三段解出的 inkoop 都精确等于 **0.0248**。→ **我们存的 `tibber_price.total` 就是官方 "totale stroomprijs"(含 inkoopvergoeding 的买价)**,因此:
- 买价 `buy = total`(已含 inkoopvergoeding,正确)。
- 净计量回送价 `sell = total verkoopvergoeding = total 0.0248`
- ⚠️ 所以 saldering 下"回送 1 度"仍比"用 1 度"少 0.0248——**不是完全 1:1**。代码用 `sell_fee` 建模这笔扣减,`sell_adjust` 只负责在净计量期把能源税补回(`sell_adjust = energy_tax`)。
### 3.2 年末盈余 / 取消净计量后(文档原文,Scenario 2)
> "Voor de overproductie van 500 kWh heb je recht op de beursprijs en de inkoopvergoeding, maar heb je geen recht op de energiebelasting. … ontvang je nog een factuur van ons voor de te veel uitgekeerde belastingen …"
@@ -146,9 +156,11 @@ curl -s -X POST https://api.tibber.com/v1-beta/gql \
> spot 取 API `energy``total = energy + tax`(全包)。**买价直接用 `total`**,卖价从 `total` 扣掉卖电不交的能源税。
- **Tibber 动态合同**post-2027 口径)
- 买价 `buy = price.total`
- 卖价 `sell = price.total energy_tax_per_kwh sell_adjust``sell_adjust` 默认 0;含 VAT 归己;买卖费抵消已隐含在 total 里)
- **Tibber 动态合同**
- 买价 `buy = price.total`(含 energy_tax + VAT + inkoopvergoeding
- 卖价 `sell = price.total energy_tax sell_fee sell_adjust`
- `sell_fee`verkoopvergoeding(卖侧上网费,默认 **0.0248**,含 VAT),**始终扣除**——即使净计量期也扣(见 §3.1)。
- `sell_adjust`:手动修正项(默认 0)。**净计量期**设为 `energy_tax`(把能源税补回),得 `sell = total sell_fee`;**2027 取消净计量后**设为 0,得 `sell = total energy_tax sell_fee`(无能源税、纯市场价再扣上网费)。
- **固定合同(manual,双费率)**:
- 买价 `buy_档 = energy_buy_档 + energy_tax`(档 ∈ {normal, dal}
- 卖价 `sell_档 = sell_档`(回送价,**无能源税**
@@ -242,7 +254,7 @@ extra_device_timestamp, extra_device_delivered # 燃气表(m³,每
## 8. 待真实数据核对(合同生效后用真实 token / 账单)
1. **真实 token 复核**:跑 §1.4 的 15 分钟 curl,确认 NL 返回**真** 15 分钟价(非重复小时价)+ 币种 EUR。
2. **卖价残差**:确认 `total` 里 purchase fee 是否被卖侧 sales fee 完全抵掉、回送 VAT 口径 → 调 `sell_adjust`(默认 0
2. ~~**卖价残差**:确认 `total` 里 purchase fee 是否被卖侧 sales fee 完全抵掉~~**已核实(2026-07**`total` 含 inkoopvergoeding0.0248),净计量回送价 = `total verkoopvergoeding(0.0248)`,两费**不抵消**;代码以 `sell_fee`(默认 0.0248)建模。仍待真实账单核对 `sell_fee` / VAT 口径的最终残差
3. **双费率寄存器映射**:确认 `_1`=dal/`_2`=normal 没接反(差价小但要对)。
4. **能源税年值**:按当年实际值与年用电档位核 `energy_tax`
5. **固定合同数值**:回送两档价、电网费、heffingskorting 待用户从账单填。
+91 -2
View File
@@ -2,7 +2,7 @@
本文档记录 `home-automation``v1.0.3` 之后的下一阶段规划。这一阶段不是小修补,而是几次较大的结构性改动:单库化、前端重写、以及远期的移动端试水。
> 每个里程碑的**可执行原子任务**展开在 [`docs/design/`](./design/README.md)M1 [`m1-db-consolidation.md`](./design/m1-db-consolidation.md)、M2 [`m2-frontend-v2.md`](./design/m2-frontend-v2.md)、M3 [`m3-token-mobile.md`](./design/m3-token-mobile.md)、M4 [`m4-login-hardening.md`](./design/m4-login-hardening.md)、M5 [`m5-iot-energy.md`](./design/m5-iot-energy.md)、M6 [`m6-tibber-dynamic-energy.md`](./design/m6-tibber-dynamic-energy.md)、M7 [`m7-meter-epochs-archival.md`](./design/m7-meter-epochs-archival.md)。这些文档为 Orchestrator→Implementer→Reviewer 的多模型流水线设计
> 每个里程碑的设计与**可执行原子任务**展开在 [`docs/design/`](./design/README.md)M1 [`m1-db-consolidation.md`](./design/m1-db-consolidation.md)、M2 [`m2-frontend-v2.md`](./design/m2-frontend-v2.md)、M3 [`m3-token-mobile.md`](./design/m3-token-mobile.md)、M4 [`m4-login-hardening.md`](./design/m4-login-hardening.md)、M5 [`m5-iot-energy.md`](./design/m5-iot-energy.md)、M6 [`m6-tibber-dynamic-energy.md`](./design/m6-tibber-dynamic-energy.md)、M7 [`m7-meter-epochs-archival.md`](./design/m7-meter-epochs-archival.md)、Pre-M8 [`pre-m8-warmtelink-p1-poc.md`](./design/pre-m8-warmtelink-p1-poc.md)、M8 [`m8-warmtelink-energy.md`](./design/m8-warmtelink-energy.md)。Pre-M8 已完成;M8 Planning 也已完成并拆成 M8-T01~M8-T20,等待后续由编排器按依赖逐张实现
## 当前基线(v1.0.3
@@ -40,6 +40,8 @@
| **M5** ✅ | IoT / 能耗采集 | 通用 Modbus 采集(YAML profile + JSON readings+ MQTT/HA Discovery + 前端侧边栏 + Energy 视图 |
| **M6** ✅ | 通用电价层 + DSMR 接入 + 实时电费计算 | 通用电价层(manual/tibber profile + 合同版本)+ DSMR 实时电表接入 + 每 15min 寄存器差×价计量电费(不可变快照)+ 日/月/年汇总 + 反哺 HA Energy + 前端合同/价格/费用视图 |
| **M7** ✅ | 电表生命周期 / 换表归档 | 引入 Meter epoch,计费永不跨表算 delta,跨表/无表/异常 delta 一律降级,累计按当前表归零,追溯换表可重算,Meter CRUD API + 前端管理 UI |
| **Pre-M8** ✅ | WarmteLink P1 真机概念验证 | 正式只读 CLI 长测通过;人工开启供暖后累计量 `0.017 → 0.018 GJ` 且与物理表一致,所有 frame 的 CRC 状态仍为 `unverifiable` |
| **M8** 📋 | WarmteLink P1、多数据源 Meter 与热力计费 | Planning 已完成:统一 Source/Channel/Binding、WarmteLink 双 channel、DSMR 迁移、thermal 合同/成本、HA/UI/部署;M8-T01M8-T20 待实现 |
| **M3** | 开放与移动端(远期试水) | token 鉴权 + React Native 移动端 |
排序原则:**先清地基,再在干净结构上盖楼。** M2 的新 API 和 React 必须建立在合并后的单库之上;M4 是公网安全加固,在 M5 IoT 集成之前先堵住裸密码这个洞;M5 在安全基座就绪后再做 IoT 接入。
@@ -257,6 +259,68 @@ httpx / paho-mqtt / pyyaml / apscheduler 均为 M5 已有依赖,M6 复用,
---
## Pre-M8 — WarmteLink P1 真机概念验证(✅ 已完成)
### 目标
2026-08-22 的正式仓库 CLI 以 `115200 7N1` 连续运行 10 分钟,正常退出且没有 I/O error、未处理
异常或断连。一个 WarmteLink source 在 60/60 完整 frame 中暴露 channel 1 的生活热水累计量
`5.900 m³` 和 channel 2 的区域供暖累计量 `0.017 GJ`;二者再次与同日物理表一致。没有流量、
热功率或温度字段。
所有完整 frame 都缺少标准 `/`CRC 因而为 `unverifiable`;这项异常由正式 probe 原样报告,
没有伪装成校验通过。frame 平均 256 bytes、范围 237275,设备 timestamp 严格每 10 秒递进。
10 分钟内累计值无变化只能证明稳定基线,不能证明消费时更新频率或 reset/wrap 行为。
最终人工走查又在终端连续运行 probe 十几分钟并开启供暖;区域供暖累计量从 `0.017 GJ` 增至
`0.018 GJ`,物理热量表同步显示 `0.018 GJ`。这确认了实际消费时的累计递增与 `0.001 GJ`
精度,但不改变 CRC `unverifiable` 结论,也不证明精确更新延迟或 reset/wrap 行为。
本阶段不落库、不接 API/前端/HA,也不决定正式 Device/Source/Meter 关系。它只向 M8 提供脱敏的真机事实,避免在未知 firmware/字段语义上提前设计。
> 验证计划与通过条件:[`docs/design/pre-m8-warmtelink-p1-poc.md`](./design/pre-m8-warmtelink-p1-poc.md)
---
## M8 — WarmteLink P1、多数据源 Meter 与热力计费(📋 Planning 已完成,待实现)
### 目标
以统一的 `MeterSource → MeterSourceChannel → MeterSourceBinding → Meter epoch` 链路承载现有
DSMR MQTT 与新的 WarmteLink serial source;从一个 WarmteLink source 只读采集已由真机确认的
区域供暖累计 `GJ` 和生活热水累计 `m³`,并完成历史、计费、API、前端、HA 与部署闭环。
### 已锁定范围
- Source、Modbus Device、Meter 分离;channel 先发现、再由用户确认绑定,source 切换与物理换表
使用两条独立半开时间线。
- DSMR JSON 和 WarmteLink Decimal scalar 分表存储,但都显式关联 source/binding;既有 DSMR
历史和电费安全回填,正常电费数字保持不变,跨 source/binding 周期降级。
- WarmteLink `unverifiable` frame 需要连续双帧确认后接纳,质量标签不漂白;约 10 秒更新 latest、
每分钟保存 history,不保存 raw telegram 或 raw equipment identifier。
- 串口 worker 使用 `115200 7N1`、只读、短 DB session、可热更新和 `1…60s` 退避重连;以可选
compose overlay、稳定 by-id、serial GID 和非 root 容器部署。
- 合同增加 electricity/thermal scope;一份 district-heating 合同覆盖 heating GJ 与 hot-water m³,
生成按 commodity 的 15 分钟成本和 contract-level 每日固定费。仓库不硬编码真实 tariff。
- UI 分成 Sources / Modbus Devices / Meters,并让 Contracts / Prices / Costs 按 scope 切换;新增
source、meter 与 thermal cost HA entities,开关默认关闭,旧 DSMR latest API 保持兼容。
### 原子实施链
- **M8-T01T06**:统一 source/channel/binding schema、DSMR 历史/runtime/电费迁移和管理 API。
- **M8-T07T11**:共享 P1 parser、WarmteLink 标量存储、质量接纳、serial worker、发现与历史 API。
- **M8-T12T16**:合同 scope、district-heating profile、thermal cost 账本/引擎/API。
- **M8-T17T20**HA、Sources/Meters UI、scope-aware 计费 UI、compose/文档/全链收尾。
完成判据不仅是单元闸门全绿,还包括历史副本迁移对账、OpenAPI/codegen、全部前端闸门、真实
`docker build`、非 root 串口部署和真机端到端 walkthrough。任何任务都不得删除旧数据库、历史
读数、旧 config 行或 volumepush/tag 仍需用户单独授权。
> 完整架构、HTTP 契约、质量/计费规则、依赖图与 M8-T01~M8-T20 任务卡:
> [`docs/design/m8-warmtelink-energy.md`](./design/m8-warmtelink-energy.md)
---
## M3 — 开放与移动端(远期试水)
### 目标
@@ -288,11 +352,36 @@ httpx / paho-mqtt / pyyaml / apscheduler 均为 M5 已有依赖,M6 复用,
**动机**:浏览器端走 session cookie 即可,但**脚本 / 设备 / 外部程序调用 API** 需要一种长期有效、可随身携带的凭据。在设置页加一组功能,由 admin **手动签发 long-lived token**,之后用它来调 API。
**本次明确的首要目标 = 给现在裸奔的 ingestion 端点上鉴权**2026-06-27 与用户确认):
- `POST /location/record``app/api/routes/location.py:18`)——位置记录上报。**目前无任何鉴权**。当前数据经 Home Assistant 转发进来,上 token 后 **HA 侧需携带该 token**;也可由其他客户端直接上报。
- `POST /poo/record``app/api/routes/poo.py:21`+ `GET /poo/latest``poo.py:57`)——小狗排便记录上报 / 最新查询。**目前无任何鉴权**。
- 这些是设备 / 脚本(非浏览器)端点,session cookie 不适用,正是 long-lived token 的用武之地。(浏览器 CRUD `/api/data/*` 已由 session 保护,不在此列。)
**范围(粗略,待细化)**
- 设置页新增「API Token」区:生成 / 命名 / 吊销 long-lived token;明文只在**生成时展示一次**,此后只存哈希。
- 后端支持用该 token 鉴权访问 API(与现有 session cookie 并存,互不影响)。
- 后端支持用该 token 鉴权访问 API(与现有 session cookie 并存,互不影响);给上述 ingestion 端点加 token 鉴权依赖
- 与 [M3](#m3--开放与移动端远期试水) 的 token 主题相关,但**这条是 Web 设置页手动签发的 PAT 风格**,不依赖移动端 OAuth 流程;两者实现时可复用同一套 token 存储 / 校验。
- 与下面第 3 条「Session 滑动续期」同属 Authentication 主题(一个是设备/脚本的长期凭据,一个是浏览器短会话体验),实现时鉴权层可一并梳理。
### 3. Session 滑动自动续期(Authentication
**动机**2026-06-27 与用户确认):当前 session 是**绝对过期**——登录即定死、活动不续期,满 TTL 必须重新登录,体验割裂。希望改成**滑动续期(sliding / rolling)**:只要用户还在活动就自动延长,提供"在用就不掉线"的体验。
**现状(实现起点,便于快速拾起)**
- TTL 默认 **12 小时**`auth_session_ttl_hours``app/config.py:38`;配置页 `app/services/config_page.py:45` 可运行时改)。
- 登录时**一次性写死**`create_session``expires_at = now + ttl``app/services/auth.py:94`+ cookie `max_age = ttl``app/api/routes/api/session.py:153`)。
- 每请求**只读校验、从不延长**`get_authenticated_session``app/services/auth.py:103`)只判断 `expires_at <= now`,过期时仅顺手标 `revoked``set_cookie` 只在登录路由调用一次,**无 per-request 中间件**。→ 所以是绝对过期,不是滑动。
**设计要点(待写设计文档时展开)**
- 校验通过时 bump `expires_at = now + ttl` 并**重发 cookie**(滑动窗口)。
- **写节流**:不要每个请求都写 DB——仅当剩余寿命已过半(或距上次续期 > N 分钟)才续期,避免高频写放大。
- **绝对寿命硬顶**:除滑动 TTL 外再设 `created_at + max_lifetime` 上限,防止"永不过期"的会话(安全考量)。
- 新增配置项:滑动 TTL、绝对寿命上限、续期节流阈值。
- 注意:改动只对**新逻辑生效**,已存在 session 的 `expires_at` 行为按新校验路径走即可;上线前过校验闸门。
## Future Ideas(暂不排期,想到先记下)
+17 -5
View File
@@ -261,7 +261,7 @@ export interface paths {
*
* Response ``points`` carries per-slot:
* - ``buy = total`` (Tibber all-inclusive price)
* - ``sell = total energy_tax sell_adjust`` (from active version values)
* - ``sell = total energy_tax sell_fee sell_adjust`` (from active version values)
* - ``level`` (Tibber price level, may be null)
*
* ``tariff`` is null.
@@ -2220,7 +2220,9 @@ export interface components {
* SummaryResponse
* @description Response for GET /api/energy/costs/summary.
*
* All monetary values are in ``currency``.
* Monetary values are in ``currency``; the ``*_kwh`` fields are energy totals
* in kWh. ``metered_import``/``metered_export`` are **money**, not energy —
* only the ``_kwh``-suffixed fields carry kWh.
*
* ``total_payable = metered_net + fixed_costs credits``
*/
@@ -2229,19 +2231,29 @@ export interface components {
currency: string;
/**
* Metered Import
* @description Σ import_cost for non-degraded periods.
* @description Σ import_cost for non-degraded periods (money, in `currency`).
*/
metered_import: number;
/**
* Metered Export
* @description Σ export_revenue for non-degraded periods.
* @description Σ export_revenue for non-degraded periods (money, in `currency`).
*/
metered_export: number;
/**
* Metered Net
* @description Σ net_cost for non-degraded periods.
* @description Σ net_cost for non-degraded periods (money, in `currency`).
*/
metered_net: number;
/**
* Metered Import Kwh
* @description Σ (d1_kwh + d2_kwh) for non-degraded periods (energy imported, kWh).
*/
metered_import_kwh: number;
/**
* Metered Export Kwh
* @description Σ (r1_kwh + r2_kwh) for non-degraded periods (energy exported, kWh).
*/
metered_export_kwh: number;
/**
* Fixed Costs
* @description Standing charges (network_fee + management_fee) apportioned over the interval.
+11 -2
View File
@@ -61,9 +61,13 @@ const COST_PERIOD = {
const SUMMARY = {
currency: 'EUR',
// Money totals and kWh totals are deliberately distinct so the assertions
// below prove the cards read the *_kwh fields, not the monetary ones.
metered_import: 10.5,
metered_export: 2.3,
metered_net: 8.2,
metered_import_kwh: 33.3,
metered_export_kwh: 44.4,
fixed_costs: 5.0,
credits: 50.0,
total_payable: 12.5,
@@ -145,9 +149,14 @@ describe('CostView', () => {
expect(screen.getByTestId('summary-import')).toBeInTheDocument()
})
expect(screen.getByTestId('summary-import')).toHaveTextContent('10.500')
expect(screen.getByTestId('summary-export')).toHaveTextContent('2.300')
// Main figure is energy (kWh), taken from the *_kwh fields.
expect(screen.getByTestId('summary-import')).toHaveTextContent('33.300')
expect(screen.getByTestId('summary-export')).toHaveTextContent('44.400')
expect(screen.getByTestId('summary-total')).toHaveTextContent('12.50')
// Sub-line carries the monetary equivalent, so money is still visible.
expect(screen.getByTestId('summary-import-sub')).toHaveTextContent('10.50 EUR')
expect(screen.getByTestId('summary-export-sub')).toHaveTextContent('2.30 EUR')
})
it('shows recompute confirmation modal on button click', async () => {
+12 -3
View File
@@ -77,10 +77,12 @@ function getThisMonthRange(): { start: string; end: string } {
interface SummaryCardProps {
label: string
value: string
/** Optional secondary line, e.g. the monetary equivalent of an energy figure. */
sub?: string
testId?: string
}
function SummaryCard({ label, value, testId }: SummaryCardProps) {
function SummaryCard({ label, value, sub, testId }: SummaryCardProps) {
return (
<Paper withBorder p="sm" data-testid={testId}>
<Stack gap={4}>
@@ -90,6 +92,11 @@ function SummaryCard({ label, value, testId }: SummaryCardProps) {
<Text fw={600} size="lg">
{value}
</Text>
{sub !== undefined && (
<Text size="xs" c="dimmed" data-testid={testId ? `${testId}-sub` : undefined}>
{sub}
</Text>
)}
</Stack>
</Paper>
)
@@ -207,12 +214,14 @@ export function CostView() {
<SimpleGrid cols={{ base: 2, sm: 3 }} spacing="sm" data-testid="cost-summary">
<SummaryCard
label="Import (kWh)"
value={summaryQuery.data.metered_import.toFixed(3)}
value={summaryQuery.data.metered_import_kwh.toFixed(3)}
sub={`${summaryQuery.data.metered_import.toFixed(2)} ${currency}`}
testId="summary-import"
/>
<SummaryCard
label="Export (kWh)"
value={summaryQuery.data.metered_export.toFixed(3)}
value={summaryQuery.data.metered_export_kwh.toFixed(3)}
sub={`${summaryQuery.data.metered_export.toFixed(2)} ${currency}`}
testId="summary-export"
/>
<SummaryCard
+253 -3
View File
@@ -6,10 +6,14 @@
* 2. Empty state (no active contract / no kind).
* 3. Renders tibber chart when tibber kind data is available.
* 4. Shows tariff table for manual kind.
* 5. Marks the currently active price slot (dot + caption).
* 6. Hovering past midnight resolves tomorrow's slot, not today's (regression).
* 7. buildChartRows: unique X keys across midnight.
* 8. findActiveSlotIndex: which slot is currently active.
*/
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { screen, waitFor } from '@testing-library/react'
import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest'
import { screen, waitFor, fireEvent } from '@testing-library/react'
import { renderWithProviders } from '../test-utils'
// ---------------------------------------------------------------------------
@@ -42,7 +46,107 @@ vi.mock('../api/client', () => ({
// Import component
// ---------------------------------------------------------------------------
import { TibberPrices } from './TibberPrices'
import { TibberPrices, buildChartRows, findActiveSlotIndex } from './TibberPrices'
// ---------------------------------------------------------------------------
// Chart size harness
//
// jsdom reports every element as 0x0, so Recharts renders an empty plot and no
// pointer interaction is possible. These helpers hand the chart a fixed size:
// - the ResponsiveContainer gets 800x300 from its bounding rect + a ResizeObserver
// that reports the same size,
// - the chart wrapper reports 800x260 (the height the component asks for), which
// is what Recharts uses to translate clientX/clientY into chart coordinates,
// - everything else stays 0x0 so the legend does not eat the whole plot area.
// ---------------------------------------------------------------------------
const CHART_W = 800
const CONTAINER_H = 300
const CHART_H = 260
function fakeRect(width: number, height: number): DOMRect {
return {
x: 0,
y: 0,
left: 0,
top: 0,
right: width,
bottom: height,
width,
height,
toJSON: () => {},
} as DOMRect
}
const originalResizeObserver = globalThis.ResizeObserver
const offsetWidthDescriptor = Object.getOwnPropertyDescriptor(
HTMLElement.prototype,
'offsetWidth',
)
const offsetHeightDescriptor = Object.getOwnPropertyDescriptor(
HTMLElement.prototype,
'offsetHeight',
)
function installChartSize() {
vi.spyOn(Element.prototype, 'getBoundingClientRect').mockImplementation(function (this: Element) {
if (this.classList.contains('recharts-responsive-container')) {
return fakeRect(CHART_W, CONTAINER_H)
}
if (this.classList.contains('recharts-wrapper')) return fakeRect(CHART_W, CHART_H)
return fakeRect(0, 0)
})
// Recharts divides rect size by offset size to undo CSS transform scaling;
// matching them keeps the scale factor at 1.
Object.defineProperty(HTMLElement.prototype, 'offsetWidth', {
configurable: true,
value: CHART_W,
})
Object.defineProperty(HTMLElement.prototype, 'offsetHeight', {
configurable: true,
value: CHART_H,
})
globalThis.ResizeObserver = class implements ResizeObserver {
private readonly cb: ResizeObserverCallback
constructor(cb: ResizeObserverCallback) {
this.cb = cb
}
observe() {
this.cb(
[{ contentRect: { width: CHART_W, height: CONTAINER_H } } as ResizeObserverEntry],
this,
)
}
unobserve() {}
disconnect() {}
}
}
function restoreChartSize() {
globalThis.ResizeObserver = originalResizeObserver
if (offsetWidthDescriptor) {
Object.defineProperty(HTMLElement.prototype, 'offsetWidth', offsetWidthDescriptor)
}
if (offsetHeightDescriptor) {
Object.defineProperty(HTMLElement.prototype, 'offsetHeight', offsetHeightDescriptor)
}
}
/** Hourly price points, one per hour starting at `startUtc`, with unique prices. */
function hourlyPoints(startUtc: number, count: number) {
return Array.from({ length: count }, (_, i) => ({
starts_at: new Date(startUtc + i * 3600_000).toISOString(),
buy: 0.1 + i / 1000,
sell: 0.05 + i / 1000,
level: 'NORMAL',
}))
}
function tooltipText(): string {
return document.querySelector('.recharts-tooltip-wrapper')?.textContent ?? ''
}
// ---------------------------------------------------------------------------
// Tests
@@ -50,6 +154,10 @@ import { TibberPrices } from './TibberPrices'
describe('TibberPrices', () => {
beforeEach(() => vi.clearAllMocks())
afterEach(() => {
vi.restoreAllMocks()
restoreChartSize()
})
it('renders loading state initially', () => {
mockGet.mockImplementation(() => new Promise(() => {}))
@@ -135,4 +243,146 @@ describe('TibberPrices', () => {
expect(screen.getByTestId('tariff-sell-normal')).toHaveTextContent('0.0900')
expect(screen.getByTestId('tariff-sell-dal')).toHaveTextContent('0.0900')
})
it('marks the currently active price slot with a dot and a caption', async () => {
installChartSize()
const SLOT_MS = 15 * 60 * 1000
// Start of the quarter-hour slot that contains "now".
const currentSlot = Math.floor(Date.now() / SLOT_MS) * SLOT_MS
mockGet.mockResolvedValue({
data: {
kind: 'tibber',
currency: 'EUR',
points: [
{ starts_at: new Date(currentSlot - SLOT_MS).toISOString(), buy: 0.11, sell: 0.05 },
{ starts_at: new Date(currentSlot).toISOString(), buy: 0.2431, sell: 0.1102 },
{ starts_at: new Date(currentSlot + SLOT_MS).toISOString(), buy: 0.31, sell: 0.15 },
],
tariff: null,
},
})
renderWithProviders(<TibberPrices />)
await waitFor(() => {
expect(screen.getByTestId('tibber-current-price')).toBeInTheDocument()
})
const marker = screen.getByTestId('tibber-current-price')
expect(marker).toHaveTextContent('0.2431')
expect(marker).toHaveTextContent('0.1102')
// One dot on the buy line, one on the sell line — visible without hovering.
await waitFor(() => {
expect(document.querySelectorAll('.recharts-reference-dot')).toHaveLength(2)
})
})
it('resolves the hovered slot past midnight to tomorrow, not today', async () => {
installChartSize()
// 26 hourly points starting at 2020-01-01T00:00Z, so "00:00" and "01:00"
// each appear twice. Fixed past dates keep the "now" marker out of range.
const points = hourlyPoints(Date.UTC(2020, 0, 1), 26)
mockGet.mockResolvedValue({
data: { kind: 'tibber', currency: 'EUR', points, tariff: null },
})
renderWithProviders(<TibberPrices />)
await waitFor(() => expect(screen.getByTestId('tibber-chart')).toBeInTheDocument())
expect(screen.queryByTestId('tibber-current-price')).not.toBeInTheDocument()
const wrapper = document.querySelector('.recharts-wrapper')
expect(wrapper).not.toBeNull()
// Right edge of the plot area = the last slot (day 2, 01:00, buy 0.1250).
fireEvent.mouseMove(wrapper!, { clientX: 770, clientY: CHART_H / 2 })
await waitFor(() => expect(tooltipText()).toContain('0.1250'))
// Label carries the date, so day 2 is distinguishable from day 1.
expect(tooltipText()).toContain('1/2/2020')
expect(tooltipText()).toContain('0.0750')
// The active dots must sit on the hovered point (right half of the plot).
// The bug put them on day 1's identically-labelled slot near the left edge.
const dots = Array.from(document.querySelectorAll('.recharts-active-dot circle'))
expect(dots).toHaveLength(2)
for (const dot of dots) {
expect(Number(dot.getAttribute('cx'))).toBeGreaterThan(CHART_W / 2)
}
})
})
// ---------------------------------------------------------------------------
// buildChartRows
// ---------------------------------------------------------------------------
describe('buildChartRows', () => {
it('keeps X-axis keys unique across midnight', () => {
// Same local time-of-day on two consecutive days: as "HH:mm" labels these
// collided, which made Recharts resolve the hovered point to the first match
// (today) instead of the hovered one (tomorrow).
const rows = buildChartRows([
{ starts_at: '2026-07-26T22:00:00Z', buy: 0.1, sell: 0.05 },
{ starts_at: '2026-07-27T22:00:00Z', buy: 0.2, sell: 0.06 },
])
expect(rows).toHaveLength(2)
expect(new Set(rows.map((r) => r.ts)).size).toBe(2)
})
it('sorts rows by slot start and parses naive timestamps as UTC', () => {
const rows = buildChartRows([
{ starts_at: '2026-07-27T02:00:00', buy: 0.3, sell: 0.07 },
{ starts_at: '2026-07-27T01:00:00Z', buy: 0.2, sell: 0.06 },
{ starts_at: '2026-07-27T00:00:00Z', buy: 0.1, sell: 0.05 },
])
expect(rows.map((r) => r.buy)).toEqual([0.1, 0.2, 0.3])
expect(rows.map((r) => r.ts)).toEqual([
'2026-07-27T00:00:00.000Z',
'2026-07-27T01:00:00.000Z',
'2026-07-27T02:00:00.000Z',
])
})
})
// ---------------------------------------------------------------------------
// findActiveSlotIndex
// ---------------------------------------------------------------------------
describe('findActiveSlotIndex', () => {
const rows = buildChartRows([
{ starts_at: '2026-07-27T00:00:00Z', buy: 0.1, sell: 0.05 },
{ starts_at: '2026-07-27T00:15:00Z', buy: 0.2, sell: 0.06 },
{ starts_at: '2026-07-27T00:30:00Z', buy: 0.3, sell: 0.07 },
])
const at = (iso: string) => new Date(iso).getTime()
it('returns the slot containing now', () => {
expect(findActiveSlotIndex(rows, at('2026-07-27T00:20:00Z'))).toBe(1)
})
it('returns the slot at its exact start boundary', () => {
expect(findActiveSlotIndex(rows, at('2026-07-27T00:15:00Z'))).toBe(1)
})
it('returns null before the first slot', () => {
expect(findActiveSlotIndex(rows, at('2026-07-26T23:59:00Z'))).toBeNull()
})
it('stays on the last slot until its inferred end, then returns null', () => {
expect(findActiveSlotIndex(rows, at('2026-07-27T00:44:00Z'))).toBe(2)
expect(findActiveSlotIndex(rows, at('2026-07-27T00:45:00Z'))).toBeNull()
})
it('returns null for empty data', () => {
expect(findActiveSlotIndex([], Date.now())).toBeNull()
})
})
+136 -11
View File
@@ -2,13 +2,15 @@
* TibberPrices — price curve visualization.
*
* - Fetches today + tomorrow price range using useEnergyPrices.
* - For tibber kind: Recharts LineChart showing buy/sell prices over time.
* - For tibber kind: Recharts LineChart showing buy/sell prices over time,
* with the currently active price slot marked by a dot.
* - For manual kind: shows tariff table (buy_dal, buy_normal, sell_dal, sell_normal).
* - Handles: no active contract, empty data, loading, error.
*
* Recharts imports are isolated to this file only.
*/
import { useEffect, useMemo, useState } from 'react'
import {
Stack,
Text,
@@ -29,10 +31,20 @@ import {
CartesianGrid,
Tooltip,
Legend,
ReferenceDot,
ResponsiveContainer,
} from 'recharts'
import { useEnergyPrices } from './hooks'
import { formatLocalTime } from '../utils/datetime'
import { formatLocalDate, formatLocalTime, parseBackendTimestamp } from '../utils/datetime'
const BUY_COLOR = '#2196f3'
const SELL_COLOR = '#4caf50'
/** Slot length assumed for the very last point, when no next point bounds it. */
const FALLBACK_SLOT_MS = 60 * 60 * 1000
/** How often the "current price" marker re-evaluates which slot is active. */
const NOW_TICK_MS = 30 * 1000
// ---------------------------------------------------------------------------
// Time range helpers
@@ -51,34 +63,121 @@ function getTomorrowEnd(): string {
return d.toISOString()
}
// ---------------------------------------------------------------------------
// Chart data helpers
// ---------------------------------------------------------------------------
export interface PricePoint {
starts_at: string
buy: number
sell: number
level?: string | null
}
export interface ChartRow {
/**
* X-axis category key — the full instant, NOT a "HH:mm" label.
*
* Must be unique per slot: Recharts resolves the hovered point by *value*
* (findEntryInArray on the axis dataKey), so a repeated key makes the tooltip
* and the active dot snap back to the first match. With "HH:mm" labels, every
* time of day appears twice in a today+tomorrow range, which pinned the dot on
* today once the cursor passed midnight. Formatting to HH:mm happens in the
* tick / tooltip formatters instead.
*/
ts: string
/** Slot start as epoch ms; NaN when starts_at is unparseable. */
tsMs: number
buy: number
sell: number
}
/** Map API price points to chart rows with unique X keys, sorted by slot start. */
export function buildChartRows(points: PricePoint[]): ChartRow[] {
return points
.map((p) => {
const d = parseBackendTimestamp(p.starts_at)
const tsMs = d.getTime()
return {
ts: Number.isFinite(tsMs) ? d.toISOString() : p.starts_at,
tsMs,
buy: p.buy,
sell: p.sell,
}
})
.sort((a, b) => {
// Unparseable timestamps sort last so the ascending scan below can stop early.
if (!Number.isFinite(a.tsMs)) return Number.isFinite(b.tsMs) ? 1 : 0
if (!Number.isFinite(b.tsMs)) return -1
return a.tsMs - b.tsMs
})
}
/**
* Index of the row whose slot contains `nowMs`, or null when now is outside the
* fetched range. A slot ends where the next one starts; the last row has no next
* slot, so it falls back to the series spacing (quarter-hourly for Tibber).
*/
export function findActiveSlotIndex(rows: ChartRow[], nowMs: number): number | null {
let idx = -1
for (let i = 0; i < rows.length; i += 1) {
if (!Number.isFinite(rows[i].tsMs) || rows[i].tsMs > nowMs) break
idx = i
}
if (idx < 0) return null
const spacing = rows.length > 1 ? rows[1].tsMs - rows[0].tsMs : NaN
const slotMs = Number.isFinite(spacing) && spacing > 0 ? spacing : FALLBACK_SLOT_MS
const slotEnd = idx + 1 < rows.length ? rows[idx + 1].tsMs : rows[idx].tsMs + slotMs
return nowMs < slotEnd ? idx : null
}
/** Ticking clock so the active-slot marker follows slot boundaries while open. */
function useNowMs(intervalMs = NOW_TICK_MS): number {
const [now, setNow] = useState(() => Date.now())
useEffect(() => {
const id = setInterval(() => setNow(Date.now()), intervalMs)
return () => clearInterval(id)
}, [intervalMs])
return now
}
// ---------------------------------------------------------------------------
// Tibber chart
// ---------------------------------------------------------------------------
interface TibberChartProps {
points: Array<{ starts_at: string; buy: number; sell: number; level?: string | null }>
points: PricePoint[]
currency: string
}
function TibberChart({ points, currency }: TibberChartProps) {
const data = points.map((p) => ({
time: formatLocalTime(p.starts_at),
buy: p.buy,
sell: p.sell,
}))
const data = useMemo(() => buildChartRows(points), [points])
const nowMs = useNowMs()
const activeIndex = findActiveSlotIndex(data, nowMs)
const activeRow = activeIndex == null ? null : data[activeIndex]
return (
<Stack gap="xs" data-testid="tibber-chart">
<Group gap="xs" justify="space-between" align="baseline">
<Title order={6} c="dimmed">
Price curve ({currency})
</Title>
{activeRow && (
<Text size="xs" c="dimmed" data-testid="tibber-current-price">
Now {formatLocalTime(activeRow.ts)} · buy {activeRow.buy.toFixed(4)} · sell{' '}
{activeRow.sell.toFixed(4)}
</Text>
)}
</Group>
<ResponsiveContainer width="100%" height={260}>
<LineChart data={data} margin={{ top: 4, right: 16, left: 0, bottom: 4 }}>
<CartesianGrid strokeDasharray="3 3" />
<XAxis
dataKey="time"
dataKey="ts"
tick={{ fontSize: 10 }}
interval="preserveStartEnd"
tickFormatter={(v: string) => formatLocalTime(v)}
/>
<YAxis
tick={{ fontSize: 10 }}
@@ -89,12 +188,17 @@ function TibberChart({ points, currency }: TibberChartProps) {
formatter={(val: any) =>
[`${typeof val === 'number' ? val.toFixed(4) : String(val)} ${currency}`, undefined]
}
labelFormatter={(label) =>
typeof label === 'string'
? `${formatLocalDate(label)} ${formatLocalTime(label)}`
: label
}
/>
<Legend />
<Line
type="monotone"
dataKey="buy"
stroke="#2196f3"
stroke={BUY_COLOR}
dot={false}
strokeWidth={2}
name="Buy"
@@ -102,11 +206,32 @@ function TibberChart({ points, currency }: TibberChartProps) {
<Line
type="monotone"
dataKey="sell"
stroke="#4caf50"
stroke={SELL_COLOR}
dot={false}
strokeWidth={2}
name="Sell"
/>
{/* Currently active price slot, marked by default (no hover needed). */}
{activeRow && (
<ReferenceDot
x={activeRow.ts}
y={activeRow.buy}
r={4}
fill={BUY_COLOR}
stroke="#fff"
strokeWidth={2}
/>
)}
{activeRow && (
<ReferenceDot
x={activeRow.ts}
y={activeRow.sell}
r={4}
fill={SELL_COLOR}
stroke="#fff"
strokeWidth={2}
/>
)}
</LineChart>
</ResponsiveContainer>
</Stack>
@@ -83,6 +83,8 @@ const SUMMARY = {
metered_import: 10.5,
metered_export: 2.3,
metered_net: 8.2,
metered_import_kwh: 33.3,
metered_export_kwh: 44.4,
fixed_costs: 5.0,
credits: 50.0,
total_payable: 12.5,
+17 -5
View File
@@ -701,7 +701,7 @@
"api-energy"
],
"summary": "Get Prices",
"description": "Return the price curve for the active contract.\n\n**Tibber contracts** (kind=\"tibber\"):\n Fetches ``tibber_price`` rows within ``[start, end]``, ordered ascending\n by ``starts_at``. At most ``limit`` rows are returned (most recent first\n within the window, then reversed to ascending order — identical to the\n modbus readings pattern).\n\n Response ``points`` carries per-slot:\n - ``buy = total`` (Tibber all-inclusive price)\n - ``sell = total energy_tax sell_adjust`` (from active version values)\n - ``level`` (Tibber price level, may be null)\n\n ``tariff`` is null.\n\n**Manual contracts** (kind=\"manual\"):\n ``points`` is empty. ``tariff`` carries the four effective prices\n derived using the billing engine formula:\n - ``buy_dal = energy.buy.dal + energy_tax + ode``\n - ``buy_normal = energy.buy.normal + energy_tax + ode``\n - ``sell_dal = energy.sell.dal``\n - ``sell_normal = energy.sell.normal``\n\n**No active contract**: returns kind=null, currency=\"EUR\", points=[], tariff=null (200).",
"description": "Return the price curve for the active contract.\n\n**Tibber contracts** (kind=\"tibber\"):\n Fetches ``tibber_price`` rows within ``[start, end]``, ordered ascending\n by ``starts_at``. At most ``limit`` rows are returned (most recent first\n within the window, then reversed to ascending order — identical to the\n modbus readings pattern).\n\n Response ``points`` carries per-slot:\n - ``buy = total`` (Tibber all-inclusive price)\n - ``sell = total energy_tax sell_fee sell_adjust`` (from active version values)\n - ``level`` (Tibber price level, may be null)\n\n ``tariff`` is null.\n\n**Manual contracts** (kind=\"manual\"):\n ``points`` is empty. ``tariff`` carries the four effective prices\n derived using the billing engine formula:\n - ``buy_dal = energy.buy.dal + energy_tax + ode``\n - ``buy_normal = energy.buy.normal + energy_tax + ode``\n - ``sell_dal = energy.sell.dal``\n - ``sell_normal = energy.sell.normal``\n\n**No active contract**: returns kind=null, currency=\"EUR\", points=[], tariff=null (200).",
"operationId": "get_prices_api_energy_prices_get",
"parameters": [
{
@@ -4727,17 +4727,27 @@
"metered_import": {
"type": "number",
"title": "Metered Import",
"description": "Σ import_cost for non-degraded periods."
"description": "Σ import_cost for non-degraded periods (money, in `currency`)."
},
"metered_export": {
"type": "number",
"title": "Metered Export",
"description": "Σ export_revenue for non-degraded periods."
"description": "Σ export_revenue for non-degraded periods (money, in `currency`)."
},
"metered_net": {
"type": "number",
"title": "Metered Net",
"description": "Σ net_cost for non-degraded periods."
"description": "Σ net_cost for non-degraded periods (money, in `currency`)."
},
"metered_import_kwh": {
"type": "number",
"title": "Metered Import Kwh",
"description": "Σ (d1_kwh + d2_kwh) for non-degraded periods (energy imported, kWh)."
},
"metered_export_kwh": {
"type": "number",
"title": "Metered Export Kwh",
"description": "Σ (r1_kwh + r2_kwh) for non-degraded periods (energy exported, kWh)."
},
"fixed_costs": {
"type": "number",
@@ -4776,6 +4786,8 @@
"metered_import",
"metered_export",
"metered_net",
"metered_import_kwh",
"metered_export_kwh",
"fixed_costs",
"credits",
"total_payable",
@@ -4784,7 +4796,7 @@
"days"
],
"title": "SummaryResponse",
"description": "Response for GET /api/energy/costs/summary.\n\nAll monetary values are in ``currency``.\n\n``total_payable = metered_net + fixed_costs credits``"
"description": "Response for GET /api/energy/costs/summary.\n\nMonetary values are in ``currency``; the ``*_kwh`` fields are energy totals\nin kWh. ``metered_import``/``metered_export`` are **money**, not energy —\nonly the ``_kwh``-suffixed fields carry kWh.\n\n``total_payable = metered_net + fixed_costs credits``"
},
"TibberTestPriceSchema": {
"properties": {
+23 -7
View File
@@ -527,9 +527,9 @@ paths:
\ (most recent first\n within the window, then reversed to ascending order\
\ — identical to the\n modbus readings pattern).\n\n Response ``points``\
\ carries per-slot:\n - ``buy = total`` (Tibber all-inclusive\
\ price)\n - ``sell = total energy_tax sell_adjust`` (from active\
\ version values)\n - ``level`` (Tibber price level,\
\ may be null)\n\n ``tariff`` is null.\n\n**Manual contracts** (kind=\"\
\ price)\n - ``sell = total energy_tax sell_fee sell_adjust`` (from\
\ active version values)\n - ``level`` (Tibber price\
\ level, may be null)\n\n ``tariff`` is null.\n\n**Manual contracts** (kind=\"\
manual\"):\n ``points`` is empty. ``tariff`` carries the four effective\
\ prices\n derived using the billing engine formula:\n - ``buy_dal \
\ = energy.buy.dal + energy_tax + ode``\n - ``buy_normal = energy.buy.normal\
@@ -3659,15 +3659,25 @@ components:
metered_import:
type: number
title: Metered Import
description: Σ import_cost for non-degraded periods.
description: Σ import_cost for non-degraded periods (money, in `currency`).
metered_export:
type: number
title: Metered Export
description: Σ export_revenue for non-degraded periods.
description: Σ export_revenue for non-degraded periods (money, in `currency`).
metered_net:
type: number
title: Metered Net
description: Σ net_cost for non-degraded periods.
description: Σ net_cost for non-degraded periods (money, in `currency`).
metered_import_kwh:
type: number
title: Metered Import Kwh
description: Σ (d1_kwh + d2_kwh) for non-degraded periods (energy imported,
kWh).
metered_export_kwh:
type: number
title: Metered Export Kwh
description: Σ (r1_kwh + r2_kwh) for non-degraded periods (energy exported,
kWh).
fixed_costs:
type: number
title: Fixed Costs
@@ -3699,6 +3709,8 @@ components:
- metered_import
- metered_export
- metered_net
- metered_import_kwh
- metered_export_kwh
- fixed_costs
- credits
- total_payable
@@ -3709,7 +3721,11 @@ components:
description: 'Response for GET /api/energy/costs/summary.
All monetary values are in ``currency``.
Monetary values are in ``currency``; the ``*_kwh`` fields are energy totals
in kWh. ``metered_import``/``metered_export`` are **money**, not energy —
only the ``_kwh``-suffixed fields carry kWh.
``total_payable = metered_net + fixed_costs credits``'
+1
View File
@@ -7,6 +7,7 @@ paho-mqtt>=2.0,<3.0
pymodbus>=3.6,<4.0
pydantic-settings>=2.6,<3.0
pyotp>=2.9,<3.0
pyserial>=3.5,<4.0
python-multipart>=0.0.12,<1.0
pyyaml>=6.0,<7.0
sqlalchemy>=2.0,<3.0
+2
View File
@@ -65,6 +65,8 @@ pymodbus==3.13.1
# via -r requirements.in
pyotp==2.10.0
# via -r requirements.in
pyserial==3.5
# via -r requirements.in
python-dotenv==1.2.2
# via
# pydantic-settings
+2 -1
View File
@@ -131,6 +131,7 @@ def cmd_read(args: argparse.Namespace) -> None:
print(f"Profile : {profile.name}{profile.description}")
print(f"Gateway : {host}:{port} unit_id={unit_id}")
print(f"Function : FC{profile.function_code:02d}")
print(f"Blocks : {[(b.start, b.count) for b in profile.blocks]}")
print()
@@ -141,7 +142,7 @@ def cmd_read(args: argparse.Namespace) -> None:
from app.integrations.modbus.driver import read_blocks
try:
registers = read_blocks(host, port, unit_id, blocks)
registers = read_blocks(host, port, unit_id, blocks, function_code=profile.function_code)
except ModbusConnectionError as exc:
print(f"Connection error: {exc}", file=sys.stderr)
sys.exit(1)
+388
View File
@@ -0,0 +1,388 @@
"""Read-only parser and command-line probe for DSMR and WarmteLink P1 telegrams."""
from __future__ import annotations
import argparse
from dataclasses import dataclass
from decimal import Decimal, InvalidOperation
from enum import StrEnum
import errno
import re
import sys
import time
from typing import BinaryIO, Callable, TextIO
import serial
_OBIS_LINE = re.compile(r"^(?P<code>\d+-\d+:\d+\.\d+\.\d+)(?P<values>(?:\([^)]*\))*)$")
_NUMBER_WITH_UNIT = re.compile(r"^(?P<number>[+-]?\d+(?:\.\d+)?)(?:\*(?P<unit>.+))?$")
_CHANNEL_OBIS = re.compile(r"^0-(?P<channel>[1-9]\d*):(24|96)\.")
class IntegrityStatus(StrEnum):
"""Whether a frame has a verifiable standard DSMR checksum."""
VALID = "valid"
INVALID = "invalid"
UNVERIFIABLE = "unverifiable"
@dataclass(frozen=True)
class ObisField:
"""One OBIS line, including values not understood by this proof of concept."""
code: str
raw_values: tuple[str, ...]
value: Decimal | None = None
unit: str | None = None
@dataclass(frozen=True)
class P1Channel:
"""Fields associated with one M-Bus channel, discovered from its OBIS code."""
number: int
device_type: str | None
equipment_id: str | None
readings: tuple[ObisField, ...]
@dataclass(frozen=True)
class P1Telegram:
"""A parsed telegram while retaining its raw framing and all OBIS fields."""
raw: bytes
header: bytes
footer: bytes
integrity: IntegrityStatus
integrity_reason: str
timestamp: str | None
fields: tuple[ObisField, ...]
channels: tuple[P1Channel, ...]
def dsmr_crc16(data: bytes) -> int:
"""Return the DSMR CRC-16 over *data* (normally from ``/`` through ``!``)."""
crc = 0
for byte in data:
crc ^= byte
for _ in range(8):
crc = (crc >> 1) ^ 0xA001 if crc & 1 else crc >> 1
return crc & 0xFFFF
class TelegramFramer:
"""Incrementally extract newline-terminated telegrams from byte chunks.
A standard telegram starts with ``/``. WarmteLink's observed telegrams do
not, so a non-standard frame is retained from the current buffer start until
its ``!`` footer line instead of fabricating a standard header.
"""
def __init__(self) -> None:
self._buffer = bytearray()
def feed(self, chunk: bytes) -> list[bytes]:
"""Append *chunk* and return every complete frame now available."""
self._buffer.extend(chunk)
frames: list[bytes] = []
while (bang := self._buffer.find(b"!")) >= 0:
newline = self._buffer.find(b"\n", bang)
if newline < 0:
break
standard_start = self._buffer.find(b"/")
start = standard_start if 0 <= standard_start < bang else 0
frames.append(bytes(self._buffer[start : newline + 1]))
del self._buffer[: newline + 1]
return frames
def parse_telegram(frame: bytes) -> P1Telegram:
"""Parse a complete frame without guessing missing DSMR framing bytes."""
bang = frame.find(b"!")
if bang < 0:
raise ValueError("telegram has no footer marker '!'")
body = frame[:bang]
footer = frame[bang + 1 :].rstrip(b"\r\n")
header = body.splitlines()[0] if body else b""
integrity, reason = _integrity(frame, bang, footer)
fields = _parse_obis_fields(body)
timestamp = _field_value(fields, "0-0:1.0.0")
channels = _parse_channels(fields)
return P1Telegram(
raw=frame,
header=header,
footer=footer,
integrity=integrity,
integrity_reason=reason,
timestamp=timestamp,
fields=tuple(fields),
channels=channels,
)
def _integrity(frame: bytes, bang: int, footer: bytes) -> tuple[IntegrityStatus, str]:
if not frame.startswith(b"/"):
return IntegrityStatus.UNVERIFIABLE, "missing standard DSMR '/' header"
if len(footer) != 4 or not all(chr(byte) in "0123456789abcdefABCDEF" for byte in footer):
return IntegrityStatus.UNVERIFIABLE, "footer is not a four-digit hexadecimal CRC"
expected = int(footer, 16)
actual = dsmr_crc16(frame[: bang + 1])
if actual == expected:
return IntegrityStatus.VALID, "CRC16 verified from '/' through '!'"
return IntegrityStatus.INVALID, f"CRC16 mismatch: expected {expected:04X}, calculated {actual:04X}"
def _parse_obis_fields(body: bytes) -> list[ObisField]:
fields: list[ObisField] = []
for line in body.decode("ascii", errors="replace").splitlines()[1:]:
match = _OBIS_LINE.fullmatch(line)
if not match:
continue
raw_values = tuple(re.findall(r"\(([^)]*)\)", match.group("values")))
value, unit = _numeric_value(raw_values)
fields.append(ObisField(match.group("code"), raw_values, value, unit))
return fields
def _numeric_value(raw_values: tuple[str, ...]) -> tuple[Decimal | None, str | None]:
if not raw_values:
return None, None
match = _NUMBER_WITH_UNIT.fullmatch(raw_values[-1])
if not match:
return None, None
try:
return Decimal(match.group("number")), match.group("unit")
except InvalidOperation:
return None, None
def _field_value(fields: list[ObisField], code: str) -> str | None:
field = next((item for item in fields if item.code == code), None)
return field.raw_values[-1] if field and field.raw_values else None
def _parse_channels(fields: list[ObisField]) -> tuple[P1Channel, ...]:
by_channel: dict[int, list[ObisField]] = {}
for field in fields:
match = _CHANNEL_OBIS.match(field.code)
if match:
by_channel.setdefault(int(match.group("channel")), []).append(field)
return tuple(
P1Channel(
number=number,
device_type=_field_value(channel_fields, f"0-{number}:24.1.0"),
equipment_id=_field_value(channel_fields, f"0-{number}:96.1.0"),
readings=tuple(
field
for field in channel_fields
if field.code == f"0-{number}:24.2.1" and field.value is not None
),
)
for number, channel_fields in sorted(by_channel.items())
)
def build_parser() -> argparse.ArgumentParser:
"""Build the CLI parser for an explicitly selected, read-only serial device."""
parser = argparse.ArgumentParser(
description="Read-only WarmteLink P1 serial probe; it never writes to the device.",
epilog="Defaults are the locally measured WarmteLink 115200 7N1 framing, not DSMR defaults.",
)
parser.add_argument("--device", required=True, help="Explicit serial path, preferably /dev/serial/by-id/..."
)
parser.add_argument(
"--baudrate",
type=int,
default=115200,
help="Baud rate (default: 115200, the locally measured WarmteLink value).",
)
parser.add_argument(
"--bytesize",
type=int,
choices=(5, 6, 7, 8),
default=7,
help="Data bits (default: 7, locally measured; not the DSMR standard default).",
)
parser.add_argument(
"--parity",
choices=("N", "E", "O", "M", "S"),
type=lambda value: value.upper(),
default="N",
help="Parity (default: N, locally measured; not the DSMR standard default).",
)
parser.add_argument(
"--stopbits",
type=float,
choices=(1, 1.5, 2),
default=1,
help="Stop bits (default: 1, locally measured; not the DSMR standard default).",
)
parser.add_argument(
"--duration",
type=_positive_duration,
default=600.0,
help="Maximum capture time in seconds (default: 600).",
)
parser.add_argument(
"--show-changes",
action="store_true",
help="After the first frame, print only OBIS fields whose values changed.",
)
parser.add_argument(
"--raw-output",
type=argparse.FileType("wb"),
help="Optional path for the exact raw bytes read from the serial device.",
)
return parser
def _positive_duration(value: str) -> float:
try:
duration = float(value)
except ValueError as exc:
raise argparse.ArgumentTypeError("duration must be a positive number") from exc
if duration <= 0:
raise argparse.ArgumentTypeError("duration must be greater than zero")
return duration
def _serial_error_message(exc: BaseException) -> str:
"""Return a practical, non-root diagnostic for a serial open/read failure."""
error_number = getattr(exc, "errno", None)
message = str(exc)
if error_number == errno.EACCES or "permission denied" in message.lower():
return f"serial permission denied: {message}. Add your user to the dialout group; do not run it as root."
if error_number == errno.EBUSY or "resource busy" in message.lower():
return f"serial device is busy: {message}. Close the program currently using this device and retry."
if error_number in {errno.ENODEV, errno.ENOENT, errno.EIO}:
return f"serial device disconnected or unavailable: {message}. Check the cable and --device path."
return f"serial I/O failed: {message}. Check the cable, device path, and serial framing settings."
def _format_field(field: ObisField) -> str:
raw_values = ", ".join(field.raw_values) or "<no values>"
value = f" parsed={field.value} {field.unit or ''}" if field.value is not None else ""
return f" {field.code}: {raw_values}{value}".rstrip()
def _print_telegram(
telegram: P1Telegram,
frame_number: int,
cadence: float | None,
previous_fields: dict[str, tuple[str, ...]],
show_changes: bool,
output: TextIO,
) -> dict[str, tuple[str, ...]]:
cadence_text = "first frame" if cadence is None else f"cadence={cadence:.1f}s"
print(
f"frame {frame_number}: {telegram.integrity.value}; bytes={len(telegram.raw)}; {cadence_text}",
file=output,
)
print(f" integrity: {telegram.integrity_reason}", file=output)
current_fields = {field.code: field.raw_values for field in telegram.fields}
fields = telegram.fields
if show_changes and previous_fields:
fields = tuple(field for field in fields if previous_fields.get(field.code) != field.raw_values)
print(f" changed fields: {len(fields)}", file=output)
for field in fields:
print(_format_field(field), file=output)
for channel in telegram.channels:
print(
f" channel {channel.number}: device_type={channel.device_type or '<unknown>'}; "
f"readings={len(channel.readings)}",
file=output,
)
return current_fields
SerialFactory = Callable[..., serial.Serial]
def run_probe(
args: argparse.Namespace,
*,
serial_factory: SerialFactory = serial.Serial,
clock: Callable[[], float] = time.monotonic,
output: TextIO = sys.stdout,
error_output: TextIO = sys.stderr,
) -> int:
"""Capture and report frames until duration elapses or the user interrupts.
The only operation on ``serial_port`` is ``read``. It is always closed,
including after an interrupt, timeout, or a read error.
"""
raw_output: BinaryIO | None = args.raw_output
serial_port: serial.Serial | None = None
try:
serial_port = serial_factory(
port=args.device,
baudrate=args.baudrate,
bytesize=args.bytesize,
parity=args.parity,
stopbits=args.stopbits,
timeout=1,
)
framer = TelegramFramer()
deadline = clock() + args.duration
frame_number = 0
last_frame_at: float | None = None
previous_fields: dict[str, tuple[str, ...]] = {}
while clock() < deadline:
chunk = serial_port.read(4096)
if not chunk:
continue
if raw_output is not None:
raw_output.write(chunk)
raw_output.flush()
for frame in framer.feed(chunk):
now = clock()
telegram = parse_telegram(frame)
frame_number += 1
cadence = None if last_frame_at is None else now - last_frame_at
previous_fields = _print_telegram(
telegram,
frame_number,
cadence,
previous_fields,
args.show_changes,
output,
)
last_frame_at = now
return 0
except KeyboardInterrupt:
print("capture interrupted; serial device closed", file=output)
return 0
except (serial.SerialException, OSError) as exc:
print(_serial_error_message(exc), file=error_output)
return 1
finally:
if serial_port is not None:
serial_port.close()
if raw_output is not None:
raw_output.close()
def main(argv: list[str] | None = None) -> int:
"""Run the command-line probe."""
args = build_parser().parse_args(argv)
return run_probe(args)
if __name__ == "__main__": # pragma: no cover - exercised through main()
raise SystemExit(main())
+12
View File
@@ -0,0 +1,12 @@
/ISk5\2MT382-1000
1-3:0.2.8(50)
0-0:1.0.0(240822120000S)
0-0:96.1.1(TEST-GATEWAY)
0-1:24.1.0(006)
0-1:96.1.0(TEST-DHW)
0-1:24.2.1(240822120000S)(5.900*m3)
0-2:24.1.0(012)
0-2:96.1.0(TEST-HEAT)
0-2:24.2.1(240822120000S)(0.017*GJ)
!D18A
+11
View File
@@ -0,0 +1,11 @@
)TU)2NWA-MYRSKY
0-2:24.2.1(240822120000S)(0.017*GJ)
0-1:96.1.0(KAM-REDACTED)
0-0:1.0.0(240822120000S)
0-2:24.1.0(012)
0-1:24.2.1(240822120000S)(5.900*m3)
0-0:96.1.1(WARMTE-REDACTED)
0-1:24.1.0(006)
0-2:96.1.0(KAM-REDACTED)
1-3:0.2.8(50)
!x7z?
+53 -1
View File
@@ -363,13 +363,65 @@ def test_prices_tibber_contract_returns_points(energy_client):
starts_at_list = [p["starts_at"] for p in body["points"]]
assert starts_at_list == sorted(starts_at_list)
# Check buy/sell calculations: buy=total=0.245, sell=total-energy_tax-sell_adjust=0.245-0.1108-0.0
# Check buy/sell calculations: buy=total=0.245, sell=total-energy_tax-sell_fee-sell_adjust
# (this version has no sell_fee/sell_adjust → both default to 0 at read time).
for p in body["points"]:
assert abs(p["buy"] - 0.245) < 1e-6
assert abs(p["sell"] - (0.245 - 0.1108)) < 1e-4
assert p["level"] == "NORMAL"
def test_prices_tibber_sell_reflects_sell_fee(energy_client):
"""/prices sell price deducts sell_fee (verkoopvergoeding), net-metering config."""
client, engine, _app = energy_client
_login(client)
# Net-metering version: sell_adjust = energy_tax (refund tax), sell_fee = 0.0248.
now = datetime.now(UTC)
with Session(engine) as session:
contract = EnergyContract(
name="Tibber NetMeter",
kind="tibber",
active=True,
currency="EUR",
created_at=now,
updated_at=now,
)
session.add(contract)
session.flush()
session.add(
EnergyContractVersion(
contract_id=contract.id,
effective_from=now - timedelta(days=30),
effective_to=None,
values={
"energy": {
"energy_tax": 0.1108,
"sell_fee": 0.0248,
"sell_adjust": -0.1108,
},
"standing": {"management_fee": 5.99, "network_fee": 25.0},
"credits": {"heffingskorting": 600.0},
},
created_at=now,
)
)
session.commit()
_make_tibber_prices(engine, count=3)
start = (datetime.now(UTC) - timedelta(hours=2)).isoformat()
end = (datetime.now(UTC) + timedelta(hours=2)).isoformat()
resp = client.get("/api/energy/prices", params={"start": start, "end": end})
assert resp.status_code == 200
body = resp.json()
assert body["kind"] == "tibber"
assert len(body["points"]) == 3
# sell = 0.245 0.1108 0.0248 (0.1108) = 0.245 0.0248 = 0.2202
for p in body["points"]:
assert abs(p["buy"] - 0.245) < 1e-6
assert abs(p["sell"] - 0.2202) < 1e-4
def test_prices_tibber_limit_caps_results(energy_client):
client, engine, _app = energy_client
_login(client)
+39 -1
View File
@@ -990,6 +990,38 @@ class TestSummarize:
# Σnet ≈ 2 × 0.4051 = 0.8102
assert abs(result["metered_net"] - 0.8102) < 1e-6
def test_metered_kwh_sums(self, energy_db: Session) -> None:
"""The *_kwh totals sum both tariff registers and are distinct from the money totals."""
self._setup_two_periods(energy_db)
result = summarize(energy_db, _ts(10, 0), _ts(10, 30))
# Per period: d1=0.5, d2=1.2 → import 1.7 kWh; r1=0.0, r2=0.1 → export 0.1 kWh.
assert abs(result["metered_import_kwh"] - 3.4) < 1e-6, (
f"expected Σ(d1+d2) = 2 × 1.7 = 3.4 kWh, got {result['metered_import_kwh']}"
)
assert abs(result["metered_export_kwh"] - 0.2) < 1e-6, (
f"expected Σ(r1+r2) = 2 × 0.1 = 0.2 kWh, got {result['metered_export_kwh']}"
)
# Regression guard for the mislabelled-unit bug: energy and money totals
# must never be conflated (import 3.4 kWh vs 0.8202 EUR of import cost).
assert result["metered_import_kwh"] != result["metered_import"]
assert result["metered_export_kwh"] != result["metered_export"]
def test_metered_kwh_excludes_degraded_periods(self, energy_db: Session) -> None:
"""Degraded periods contribute no kWh, mirroring the money totals."""
self._setup_two_periods(energy_db)
# Degrade the first period; its kWh must drop out of the totals.
row = energy_db.execute(
select(EnergyCostPeriod).where(EnergyCostPeriod.period_start == _T0)
).scalar_one()
row.degraded = True
energy_db.commit()
result = summarize(energy_db, _ts(10, 0), _ts(10, 30))
assert result["degraded_count"] == 1
assert abs(result["metered_import_kwh"] - 1.7) < 1e-6
assert abs(result["metered_export_kwh"] - 0.1) < 1e-6
def test_period_count(self, energy_db: Session) -> None:
self._setup_two_periods(energy_db)
result = summarize(energy_db, _ts(10, 0), _ts(10, 30))
@@ -1263,6 +1295,9 @@ class TestSummarizePrincipleC:
def test_future_window_counts_0_days(self, energy_db: Session) -> None:
"""A fully future window (all local dates > today) counts 0 days.
Pins ``local_now`` to June 25 2026 noon AMS so the 7/18/1 window is
genuinely in the future regardless of the actual wall-clock date
(mirrors the sibling window tests, which all pin ``local_now``).
Matches table row: 7/18/1 (all future) 0 days.
"""
eff_utc = _ams_midnight(2026, 6, 1)
@@ -1270,7 +1305,9 @@ class TestSummarizePrincipleC:
start = _ams_midnight(2026, 7, 1)
end = _ams_midnight(2026, 8, 1)
result = self._run_summarize_ams(energy_db, start, end)
# Pin local_now to June 25 2026 noon AMS so 7/1→8/1 stays fully future.
pinned_now = datetime(2026, 6, 25, 12, 0, 0, tzinfo=_ams())
result = self._run_summarize_ams(energy_db, start, end, pinned_now=pinned_now)
assert result["fixed_costs"] == 0.0, (
f"All-future window must count 0 days; got fixed_costs={result['fixed_costs']}"
@@ -2703,6 +2740,7 @@ class TestSummarizeSettlementOffset:
expected_keys = {
"currency", "metered_import", "metered_export", "metered_net",
"metered_import_kwh", "metered_export_kwh",
"fixed_costs", "credits", "total_payable", "period_count",
"degraded_count", "days",
}
+51
View File
@@ -217,6 +217,57 @@ class TestReadBlocks:
mock_client.close.assert_called_once()
@patch("app.integrations.modbus.driver.ModbusTcpClient")
def test_default_function_code_uses_fc04_input_registers(
self, mock_client_cls: MagicMock
) -> None:
"""With no function_code given, read_blocks uses FC04 (read_input_registers)."""
mock_client = MagicMock()
mock_client_cls.return_value = mock_client
mock_client.connect.return_value = True
mock_client.read_input_registers.return_value = _make_ok_response([0x4366, 0x3334])
read_blocks("127.0.0.1", 502, 1, [{"start": 0x0000, "count": 2}])
mock_client.read_input_registers.assert_called_once_with(0x0000, count=2, device_id=1)
mock_client.read_holding_registers.assert_not_called()
@patch("app.integrations.modbus.driver.ModbusTcpClient")
def test_function_code_3_uses_fc03_holding_registers(
self, mock_client_cls: MagicMock
) -> None:
"""function_code=3 dispatches FC03 (read_holding_registers), e.g. DDSU666."""
mock_client = MagicMock()
mock_client_cls.return_value = mock_client
mock_client.connect.return_value = True
mock_client.read_holding_registers.return_value = _make_ok_response(
[0x4366, 0x3334, 0x3F80, 0x0000]
)
blocks = [{"start": 0x2000, "count": 4}]
result = read_blocks("127.0.0.1", 502, 1, blocks, function_code=3)
assert result == {0x2000: 0x4366, 0x2001: 0x3334, 0x2002: 0x3F80, 0x2003: 0x0000}
mock_client.read_holding_registers.assert_called_once_with(
0x2000, count=4, device_id=1
)
mock_client.read_input_registers.assert_not_called()
@patch("app.integrations.modbus.driver.ModbusTcpClient")
def test_invalid_function_code_raises_before_connecting(
self, mock_client_cls: MagicMock
) -> None:
"""An unsupported function code is rejected without opening a connection."""
mock_client = MagicMock()
mock_client_cls.return_value = mock_client
with pytest.raises(ModbusDriverError, match="Unsupported read function code"):
read_blocks("127.0.0.1", 502, 1, [{"start": 0, "count": 2}], function_code=16)
# No client should have been constructed or connected for a bad FC.
mock_client_cls.assert_not_called()
mock_client.connect.assert_not_called()
@patch("app.integrations.modbus.driver.ModbusTcpClient")
def test_unit_id_passed_as_device_id(self, mock_client_cls: MagicMock) -> None:
"""unit_id is forwarded as device_id= keyword argument (pymodbus 3.13.x)."""
+234
View File
@@ -0,0 +1,234 @@
from decimal import Decimal
import io
from pathlib import Path
import serial
from scripts.p1_probe import (
IntegrityStatus,
TelegramFramer,
build_parser,
parse_telegram,
run_probe,
)
FIXTURES = Path(__file__).parent / "fixtures"
def _fixture(name: str) -> bytes:
return (FIXTURES / name).read_bytes()
def test_standard_dsmr_fixture_has_a_valid_frame_and_crc():
frame = _fixture("dsmr_p1_valid.txt")
framer = TelegramFramer()
assert framer.feed(frame[:23]) == []
assert framer.feed(frame[23:]) == [frame]
telegram = parse_telegram(frame)
assert telegram.integrity is IntegrityStatus.VALID
assert telegram.header == b"/ISk5\\2MT382-1000"
assert telegram.timestamp == "240822120000S"
def test_warmtelink_fixture_is_unverifiable_but_parses_channels_by_obis_code():
telegram = parse_telegram(_fixture("warmtelink_p1_7n1.txt"))
assert telegram.integrity is IntegrityStatus.UNVERIFIABLE
assert "missing standard" in telegram.integrity_reason
assert [channel.number for channel in telegram.channels] == [1, 2]
values = {
channel.number: [(field.value, field.unit) for field in channel.readings]
for channel in telegram.channels
}
assert values[1] == [(Decimal("5.900"), "m3")]
assert values[2] == [(Decimal("0.017"), "GJ")]
def test_arbitrary_chunk_boundaries_and_field_order_do_not_change_parsing():
frame = _fixture("warmtelink_p1_7n1.txt")
framer = TelegramFramer()
frames = []
for byte in frame:
frames.extend(framer.feed(bytes([byte])))
assert frames == [frame]
assert parse_telegram(frames[0]).channels == parse_telegram(frame).channels
def test_unknown_fields_are_preserved_verbatim():
frame = b")HEADER\n9-9:9.9.9(opaque)(still-opaque)\n!nope\n"
telegram = parse_telegram(frame)
assert len(telegram.fields) == 1
assert telegram.fields[0].code == "9-9:9.9.9"
assert telegram.fields[0].raw_values == ("opaque", "still-opaque")
assert telegram.fields[0].value is None
def test_missing_header_and_non_hex_footer_are_unverifiable():
telegram = parse_telegram(b")HEADER\n0-0:1.0.0(240822120000S)\n!zzzz\n")
assert telegram.integrity is IntegrityStatus.UNVERIFIABLE
assert telegram.footer == b"zzzz"
def test_crc_mismatch_is_invalid_when_standard_framing_is_present():
frame = _fixture("dsmr_p1_valid.txt")
invalid = frame[:-5] + b"0000\n"
telegram = parse_telegram(invalid)
assert telegram.integrity is IntegrityStatus.INVALID
class FakeSerial:
def __init__(self, chunks: list[bytes]) -> None:
self.chunks = iter(chunks)
self.closed = False
def read(self, _size: int) -> bytes:
return next(self.chunks, b"")
def close(self) -> None:
self.closed = True
def _clock(values: list[float]):
ticks = iter(values)
return lambda: next(ticks, values[-1])
def test_cli_uses_measured_7n1_defaults_and_allows_overrides():
parser = build_parser()
defaults = parser.parse_args(["--device", "/dev/serial/by-id/example"])
overridden = parser.parse_args(
[
"--device",
"/dev/test",
"--baudrate",
"9600",
"--bytesize",
"8",
"--parity",
"E",
"--stopbits",
"2",
"--duration",
"1",
]
)
assert (defaults.baudrate, defaults.bytesize, defaults.parity, defaults.stopbits) == (115200, 7, "N", 1)
assert (overridden.baudrate, overridden.bytesize, overridden.parity, overridden.stopbits) == (
9600,
8,
"E",
2,
)
assert "locally measured" in parser.format_help()
def test_probe_reads_fake_chunks_writes_exact_raw_bytes_and_closes_device(tmp_path):
frame = _fixture("warmtelink_p1_7n1.txt")
fake = FakeSerial([frame[:17], frame[17:]])
raw_path = tmp_path / "capture.bin"
args = build_parser().parse_args(
["--device", "/dev/serial/by-id/fake", "--duration", "3", "--raw-output", str(raw_path)]
)
created: dict[str, object] = {}
def serial_factory(**kwargs):
created.update(kwargs)
return fake
output = io.StringIO()
result = run_probe(
args,
serial_factory=serial_factory,
clock=_clock([0, 0.1, 0.2, 0.3, 4]),
output=output,
)
assert result == 0
assert fake.closed
assert raw_path.read_bytes() == frame
assert created == {
"port": "/dev/serial/by-id/fake",
"baudrate": 115200,
"bytesize": 7,
"parity": "N",
"stopbits": 1,
"timeout": 1,
}
assert "unverifiable" in output.getvalue()
assert "0-1:24.2.1" in output.getvalue()
def test_show_changes_filters_unchanged_fields_and_reports_cadence():
frame = _fixture("warmtelink_p1_7n1.txt")
changed = frame.replace(b"(5.900*m3)", b"(5.901*m3)")
fake = FakeSerial([frame + changed])
args = build_parser().parse_args(["--device", "/dev/fake", "--duration", "2", "--show-changes"])
output = io.StringIO()
result = run_probe(
args,
serial_factory=lambda **_kwargs: fake,
clock=_clock([0, 0.1, 0.2, 1.2, 3]),
output=output,
)
assert result == 0
assert "cadence=1.0s" in output.getvalue()
assert "changed fields: 1" in output.getvalue()
assert fake.closed
def test_probe_diagnoses_permission_errors_without_suggesting_root():
args = build_parser().parse_args(["--device", "/dev/fake", "--duration", "1"])
errors = io.StringIO()
def serial_factory(**_kwargs):
raise serial.SerialException("[Errno 13] Permission denied: '/dev/fake'")
assert run_probe(args, serial_factory=serial_factory, error_output=errors) == 1
assert "dialout" in errors.getvalue()
assert "run it as root" in errors.getvalue()
def test_probe_diagnoses_busy_or_disconnected_device_and_closes_after_read_error():
args = build_parser().parse_args(["--device", "/dev/fake", "--duration", "1"])
errors = io.StringIO()
def busy_factory(**_kwargs):
raise serial.SerialException("[Errno 16] Device or resource busy: '/dev/fake'")
assert run_probe(args, serial_factory=busy_factory, error_output=errors) == 1
assert "Close the program" in errors.getvalue()
class DisconnectingSerial(FakeSerial):
def read(self, _size: int) -> bytes:
raise serial.SerialException("[Errno 5] device disconnected")
fake = DisconnectingSerial([])
errors = io.StringIO()
assert run_probe(args, serial_factory=lambda **_kwargs: fake, error_output=errors) == 1
assert fake.closed
assert "Check the cable" in errors.getvalue()
def test_probe_closes_device_when_interrupted():
class InterruptingSerial(FakeSerial):
def read(self, _size: int) -> bytes:
raise KeyboardInterrupt
fake = InterruptingSerial([])
args = build_parser().parse_args(["--device", "/dev/fake", "--duration", "1"])
assert run_probe(args, serial_factory=lambda **_kwargs: fake) == 0
assert fake.closed
+18
View File
@@ -137,6 +137,11 @@ class TestLoadProfileTibber:
profile = load_profile("tibber")
assert profile.energy.sell_adjust.default == 0
def test_sell_fee_has_default_verkoopvergoeding(self) -> None:
profile = load_profile("tibber")
assert profile.energy.sell_fee.unit == "EUR/kWh"
assert profile.energy.sell_fee.default == 0.0248
def test_management_fee_has_default(self) -> None:
profile = load_profile("tibber")
assert profile.standing.management_fee.default is not None
@@ -350,6 +355,19 @@ class TestValidateValuesTibber:
filled = validate_values("tibber", values)
assert filled["energy"]["sell_adjust"] == 0
def test_sell_fee_default_applied_when_absent(self) -> None:
values = {
"energy": {
"energy_tax": 0.1108,
"sell_adjust": 0.0,
# sell_fee absent — has default 0.0248 (verkoopvergoeding)
},
"standing": {"management_fee": 5.99, "network_fee": 9.87},
"credits": {"heffingskorting": 600.0},
}
filled = validate_values("tibber", values)
assert filled["energy"]["sell_fee"] == 0.0248
def test_management_fee_default_applied_when_absent(self) -> None:
values = {
"energy": {"energy_tax": 0.1108, "sell_adjust": 0.0},
+63 -1
View File
@@ -6,7 +6,7 @@ Acceptance criteria covered
2. Manual strategy: dual-tariff import/export/net calculated correctly (hand-verified).
3. Manual strategy: Decimal precision no float binary rounding errors.
4. Tibber strategy: queries the most recent TibberPrice with starts_at t0.
5. Tibber strategy: buy=total, sell=totalenergy_taxsell_adjust.
5. Tibber strategy: buy=total, sell=totalenergy_taxsell_feesell_adjust.
6. Tibber strategy: negative total negative export_revenue (not clamped).
7. Tibber strategy: raises TibberPriceNotFoundError when no matching row exists.
8. ``register_strategy`` / ``get_strategy`` round-trip works.
@@ -371,6 +371,68 @@ class TestTibberStrategy:
# sell = 0.25 - 0.10 - 0.02 = 0.13; export_revenue = 2 × 0.13 = 0.26
assert result["export_revenue"] == Decimal("2") * Decimal("0.13")
def test_sell_deducts_sell_fee(self, tibber_db) -> None:
"""verkoopvergoeding (sell_fee) is subtracted from the feed-in price.
Under net metering the energy tax is refunded (sell_adjust = energy_tax),
so sell should equal total sell_fee. Verifies the fee is a first-class,
always-deducted term and does NOT cancel against the buy-side inkoopvergoeding
that is already baked into total.
"""
t0 = _ts(10, 0)
with Session(tibber_db) as session:
_insert_tibber_price(session, starts_at=_ts(9, 45), total=0.3073)
session.commit()
# Net-metering config: sell_adjust = energy_tax refunds the tax;
# sell_fee = 0.0248 (Tibber verkoopvergoeding) is still deducted.
values = {
"energy": {
"energy_tax": 0.11085,
"sell_fee": 0.0248,
"sell_adjust": -0.11085,
},
"standing": {"management_fee": 5.99, "network_fee": 9.87},
"credits": {"heffingskorting": 600.0},
}
with Session(tibber_db) as session:
deltas = PeriodDeltas(
d1=Decimal("0"), d2=Decimal("0"),
r1=Decimal("0"), r2=Decimal("1"),
)
result = self._call(deltas, t0, session, values=values)
# sell = 0.3073 0.11085 0.0248 (0.11085) = 0.3073 0.0248 = 0.2825
expected_sell = (
Decimal("0.3073") - Decimal("0.11085") - Decimal("0.0248") - Decimal("-0.11085")
)
assert expected_sell == Decimal("0.2825")
assert result["export_revenue"] == Decimal("1") * expected_sell
assert result["pricing"]["sell_fee"] == "0.0248"
assert Decimal(result["pricing"]["sell"]) == Decimal("0.2825")
def test_sell_fee_absent_defaults_to_zero(self, tibber_db) -> None:
"""A version without sell_fee (pre-migration) reads it as 0 — no silent deduction."""
t0 = _ts(10, 0)
with Session(tibber_db) as session:
_insert_tibber_price(session, starts_at=_ts(9, 45), total=0.25)
session.commit()
values = {
"energy": {"energy_tax": 0.10, "sell_adjust": 0.0}, # no sell_fee key
"standing": {"management_fee": 5.99, "network_fee": 9.87},
"credits": {"heffingskorting": 600.0},
}
with Session(tibber_db) as session:
deltas = PeriodDeltas(
d1=Decimal("0"), d2=Decimal("0"),
r1=Decimal("0"), r2=Decimal("1"),
)
result = self._call(deltas, t0, session, values=values)
# sell = 0.25 0.10 0 0 = 0.15
assert result["export_revenue"] == Decimal("0.15")
assert result["pricing"]["sell_fee"] == "0"
def test_uses_most_recent_price_before_t0(self, tibber_db) -> None:
"""Correct row: starts_at ≤ t0, most recent wins."""
t0 = _ts(10, 0)
+80 -25
View File
@@ -69,22 +69,28 @@ _THREE_NODES = [
},
]
_PRICE_RANGE_RESPONSE = {
def _price_info_response(today: list[dict], tomorrow: list[dict] | None = None) -> dict:
"""Build a priceInfo(resolution: QUARTER_HOURLY) { today tomorrow } response."""
return {
"data": {
"viewer": {
"homes": [
{
"id": "home-id-1",
"currentSubscription": {
"priceInfoRange": {
"nodes": _THREE_NODES,
"priceInfo": {
"today": today,
"tomorrow": tomorrow if tomorrow is not None else [],
}
},
}
]
}
}
}
}
_PRICE_RANGE_RESPONSE = _price_info_response(_THREE_NODES)
_CURRENT_PRICE_RESPONSE = {
"data": {
@@ -173,23 +179,8 @@ def test_fetch_price_range_parses_nodes(monkeypatch):
def test_fetch_price_range_does_not_assume_node_count(monkeypatch):
"""Parser handles an arbitrary number of nodes (not hardcoded to 96)."""
# Build a response with a single node only.
one_node_response = {
"data": {
"viewer": {
"homes": [
{
"id": "home-id-1",
"currentSubscription": {
"priceInfoRange": {
"nodes": [_THREE_NODES[0]],
}
},
}
]
}
}
}
# Build a response with a single today node and no tomorrow yet.
one_node_response = _price_info_response([_THREE_NODES[0]])
transport = _make_transport(200, one_node_response)
def _patched_post(url, *, json, headers, timeout): # noqa: A002
@@ -202,6 +193,68 @@ def test_fetch_price_range_does_not_assume_node_count(monkeypatch):
assert len(points) == 1
def test_fetch_price_range_concatenates_today_and_tomorrow(monkeypatch):
"""today and tomorrow node lists are both parsed (today first, then tomorrow)."""
tomorrow_nodes = [
{
"startsAt": "2026-06-24T00:00:00.000+02:00",
"total": 0.40,
"energy": 0.32,
"tax": 0.08,
"currency": "EUR",
"level": "EXPENSIVE",
},
]
response = _price_info_response(_THREE_NODES, tomorrow_nodes)
transport = _make_transport(200, response)
def _patched_post(url, *, json, headers, timeout): # noqa: A002
client = httpx.Client(transport=transport)
return client.post(url, json=json, headers=headers, timeout=timeout)
monkeypatch.setattr("app.integrations.tibber.client.httpx.post", _patched_post)
points = fetch_price_range(_FAKE_TOKEN)
# 3 today + 1 tomorrow, in order.
assert len(points) == 4
# First node is today's first; last node is tomorrow's.
assert points[0].starts_at == datetime(2026, 6, 22, 22, 0, 0, tzinfo=UTC)
# 2026-06-24T00:00:00+02:00 → 2026-06-23T22:00:00Z
assert points[-1].starts_at == datetime(2026, 6, 23, 22, 0, 0, tzinfo=UTC)
assert points[-1].total == pytest.approx(0.40)
def test_fetch_price_range_tomorrow_null_returns_today_only(monkeypatch):
"""A null tomorrow (before day-ahead publication) yields today's nodes only."""
response = {
"data": {
"viewer": {
"homes": [
{
"id": "home-id-1",
"currentSubscription": {
"priceInfo": {
"today": _THREE_NODES,
"tomorrow": None,
}
},
}
]
}
}
}
transport = _make_transport(200, response)
def _patched_post(url, *, json, headers, timeout): # noqa: A002
client = httpx.Client(transport=transport)
return client.post(url, json=json, headers=headers, timeout=timeout)
monkeypatch.setattr("app.integrations.tibber.client.httpx.post", _patched_post)
points = fetch_price_range(_FAKE_TOKEN)
assert len(points) == 3
def test_fetch_price_range_home_id_selection(monkeypatch):
"""When home_id is specified, the matching home is selected."""
two_homes_response = {
@@ -211,16 +264,18 @@ def test_fetch_price_range_home_id_selection(monkeypatch):
{
"id": "home-id-first",
"currentSubscription": {
"priceInfoRange": {
"nodes": [_THREE_NODES[0]],
"priceInfo": {
"today": [_THREE_NODES[0]],
"tomorrow": [],
}
},
},
{
"id": "home-id-second",
"currentSubscription": {
"priceInfoRange": {
"nodes": [_THREE_NODES[1], _THREE_NODES[2]],
"priceInfo": {
"today": [_THREE_NODES[1], _THREE_NODES[2]],
"tomorrow": [],
}
},
},