45 Commits
Author SHA1 Message Date
tliu93 f4cea3874b test(energy-cost): pin local_now in future-window summarize test
frontend / frontend (push) Failing after 6m38s
pytest / test (push) Successful in 20m16s
docker-image / build-and-push (push) Successful in 13m32s
test_future_window_counts_0_days relied on the real wall-clock date and assumed
7/1→8/1 2026 was entirely in the future; once that range started elapsing the
window counted fixed-fee days and the assertion failed. Pin local_now to
June 25 2026 (as the sibling window tests already do) so the case stays
deterministic.
2026-07-17 18:49:26 +02:00
tliu93 134f0abb5f fix(tibber): fetch newest price slots via priceInfoRange last:192
priceInfoRange is a Relay-style cursor connection over the subscription's
entire price history. With no cursor, first:96 returned the OLDEST 96 slots,
anchored at the subscription start date — a fixed window that never advanced.
For a contract added mid-period this meant refresh_prices kept re-upserting the
same day-one slots forever, so GET /api/energy/prices found nothing in the
today+tomorrow window and the UI showed "No Tibber price points available".

Use last:192 to return the newest 192 quarter-hourly slots (2 days), which ends
at the latest published slot and advances daily, fully covering the prices
endpoint's today+tomorrow window.
2026-07-17 18:49:26 +02:00
tliu93 3e04b15656 feat(modbus): add DDSU666 profile and select read function code per profile
frontend / frontend (push) Successful in 2m21s
pytest / test (push) Successful in 9m50s
docker-image / build-and-push (push) Successful in 4m24s
- Add CHINT DDSU666 profile (ddsu666.yaml): FC03 holding registers, voltage/
  current/active power (kW)/reactive power (kvar)/PF/frequency, import+export
  active energy. Word/byte order left at big-endian as a documented best guess
  (manual has no float example) — to be confirmed on-device.
- Add DDSU666-Modbus-Protocol.md reference extracted from the official manual,
  plus the source PDF (parity with the SDM120 reference).
- Generalize driver.read_blocks to dispatch FC03 (holding) or FC04 (input)
  based on a function_code argument (default 4, SDM120 behaviour unchanged);
  the code is validated before any connection is attempted.
- Wire profile.function_code through the CLI read command, the background
  poller, and the device /test endpoint — previously the profile field was
  declared but never honoured (read path was hardcoded to FC04).
- Tests: default -> FC04, function_code=3 -> FC03 holding, invalid FC rejected
  before connecting.
2026-06-30 17:16:25 +02:00
tliu93 f2e8f6a8e7 docs(roadmap): queue Authentication next-steps — sliding session renewal + long-lived token targets (location/poo ingestion)
frontend / frontend (push) Successful in 2m16s
pytest / test (push) Successful in 10m40s
2026-06-27 22:16:02 +02:00
tliu93 90a03e7fd6 FUE-T09: grace-shift *_today daily reset past local midnight + dedicated 00:00:10 publish
docker-image / build-and-push (push) Successful in 4m29s
frontend / frontend (push) Successful in 2m17s
pytest / test (push) Successful in 11m2s
2026-06-26 12:21:51 +02:00
tliu93 d3fc90b320 FUE-T08: defer daily fixed-fee/heffingskorting settlement to local 01:05 in summarize() 2026-06-26 11:44:34 +02:00
tliu93 d4acfc438a FUE-T07: use meter label directly as energy-cost HA device name (drop 'Energy Cost' prefix)
frontend / frontend (push) Successful in 2m14s
pytest / test (push) Successful in 10m49s
2026-06-25 21:28:13 +02:00
tliu93 c26160b10b FUE-T06: trigger HA discovery republish after meter declare/update
frontend / frontend (push) Successful in 2m11s
pytest / test (push) Successful in 10m36s
2026-06-25 20:54:42 +02:00
tliu93 efbe36d7c0 FUE-T05: anchor energy-cost HA device identity to active meter (uuid id, label name, empty when none) 2026-06-25 20:43:07 +02:00
tliu93 f663981cdb FUE-T04: add Meter.uuid (stable HA identity anchor) + backfill migration 2026-06-25 20:20:01 +02:00
tliu93 da05fd2f09 FUE-T03: correct meters POST docstring (no recomputed_periods field) and inaccurate test comments
frontend / frontend (push) Successful in 2m11s
pytest / test (push) Successful in 8m51s
2026-06-25 18:36:46 +02:00
tliu93 188168b16a FUE-T02: fix EditMeterForm timestamp parsing to use shared parseBackendTimestamp (handles offset form) 2026-06-25 18:33:42 +02:00
tliu93 f1e7ce3133 FUE-T01: count billing window start day in full for fixed-fee/credit accrual (symmetric with end day) 2026-06-25 18:33:42 +02:00
tliu93 e3d0d17cac M7-T07: finalize M7 docs (roadmap entry, design index, status done, meter-epochs module doc)
frontend / frontend (push) Successful in 2m10s
pytest / test (push) Successful in 8m34s
2026-06-25 17:01:54 +02:00
tliu93 33f5de0591 M7-T06: add meter management UI (timeline, declare swap, edit) to Energy view 2026-06-25 17:01:54 +02:00
tliu93 839faaf95b M7-T05: add meter CRUD API + retroactive recompute + OpenAPI regen 2026-06-25 17:01:54 +02:00
tliu93 82bb329500 M7-T04: anchor cumulative cost entities to active meter start (per-meter reset) 2026-06-25 17:01:54 +02:00
tliu93 17057ed41e M7-T03: make billing engine meter-aware (no cross-meter delta, delta sanity guard, period.meter_id) 2026-06-25 17:01:54 +02:00
tliu93 227b146c2c M7-T02: add meter service layer (declare/swap/close/edit, meter_at, mutual exclusion) 2026-06-25 17:01:54 +02:00
tliu93 32a20785ee M7-T01: add meter table + model + energy_cost_period.meter_id + backfill migration 2026-06-25 14:48:39 +02:00
tliu93 2544514f52 M7: add meter-epochs / meter-swap-archival milestone design doc
frontend / frontend (push) Successful in 2m6s
pytest / test (push) Successful in 7m51s
Design for explicit meter lifecycle (epochs) so the register-difference
billing engine never computes a delta across a meter swap (2G->4G or house
move). Meter-centric (no Home entity); per-meter cumulative reset; retroactive
declaration with recompute; delta sanity guard as backstop. 7 atomic task cards.
2026-06-25 14:23:25 +02:00
tliu93 96e85fb48e FU11: anchor cumulative cost at recording start; add daily-reset cost entities; prefill add-version form
pytest / test (push) Successful in 7m32s
frontend / frontend (push) Successful in 2m6s
- MQTT import_cost_total / export_revenue_total now anchor fixed-fee / tax-credit
  accrual at max(contract start, first recorded period), so standing charges for
  untracked pre-recording days are no longer folded into the running total.
- Add energy.import_cost_today / energy.export_revenue_today (monetary,
  total_increasing) that reset at server-local midnight, for per-day cost
  aggregation in Home Assistant alongside the running totals.
- ContractForm add-version mode prefills the contract's open version values, so
  only the fields that actually change need editing.
2026-06-25 12:45:24 +02:00
tliu93 b71009620a compose add tz 2026-06-25 11:48:45 +02:00
tliu93 8b91146c29 FU10: localize energy time math to server tz; accrue fixed/credit per elapsed local day
frontend / frontend (push) Successful in 2m5s
pytest / test (push) Successful in 7m34s
- Store UTC, compute day boundaries in server local timezone (new app/services/timezone.py).
- summarize: fixed fee / tax credit accrue only for elapsed whole local days,
  integrated across contract versions — no future-day inflation, no version-switch collapse.
- MQTT import_cost_total / export_revenue_total getters delegate to summarize (no inline
  apportionment), anchored at the active contract's earliest version effective_from so the
  total sensors stay monotonic and never jump backward.
- effective_from: naive datetimes interpreted as server-local wall-clock, then stored UTC.
- frontend ContractForm sends local-midnight naive datetime (was UTC midnight).
- get_costs_summary default window uses server-local "today".
2026-06-25 11:13:30 +02:00
tliu93 682e06d256 modbus: add explicit cascade delete (device + readings + toggles + HA cleanup)
frontend / frontend (push) Successful in 2m4s
pytest / test (push) Successful in 7m32s
A disabled device with historical readings still cannot be deleted by default
(409 guard stays, to prevent accidental data loss). A new explicit opt-in lets
the user remove it for real:

- DELETE /api/modbus/devices/{uuid}?cascade=true removes the device's readings
  and exposed_entity_toggle rows, then the device, in one transaction (FK order:
  readings before device). Best-effort clears the HA discovery configs first
  (MQTT not connected -> no-op, never blocks the delete). Default (no cascade)
  unchanged. Returns 200 with deletion counts.
- Frontend: a 'Force Delete (all data)' button appears only after the 409, with
  a red irreversible warning; cascade is sent only on that explicit action.
- Also corrected a stale comment: FK RESTRICT IS enforced at runtime now.
2026-06-24 17:38:52 +02:00
tliu93 d7f0731d1c ha_discovery: publish state/availability under home_automation prefix
Follow the standard MQTT Discovery split: discovery config stays under the HA
discovery prefix (homeassistant/.../config, required), but the actual state and
availability are published under our own prefix (default home_automation/). The
config payload's state_topic/availability fields point at the new prefix so HA
reads state from there.

- new config ha_state_topic_prefix (default home_automation).
- build_discovery_payload + all publish_* split discovery_prefix (config topic)
  from state_prefix (state/availability). energy-cost still omits availability;
  modbus availability structure unchanged. Tests updated.
2026-06-24 16:21:15 +02:00
tliu93 9bc46f8d2d expose: buy/sell_price_now follow the live dual-tariff slot
Subscribe the separate DSMR tariff topic (dsmr/meter-stats/electricity_tariff,
1=dal/off-peak, 2=normal/peak), keep the latest slot in memory, and make the
manual-contract buy/sell_price_now sensors show the matching tariff's price
(tariff 1 -> dal, 2/unknown -> normal). Tibber (single 15-min price) unchanged.
Billing is untouched (register-split already handles tariffs correctly).

- new config dsmr_tariff_topic; subscribe managed by apply_dsmr_subscription
  (restart-free on config save). In-memory tariff is lock-guarded.
- import_cost_total / export_revenue_total cumulative getters unchanged.
2026-06-24 15:39:21 +02:00
tliu93 6f8cb05eab expose: fold standing fee into import cost, tax credit into export revenue
So HA can show the full payable: net = import - export = Total Payable, with
both entities staying monotonic-positive (no negative-value/reset pitfalls).

- import_cost_total = SUM(import_cost) + days*(network_fee+management_fee)/30
- export_revenue_total = SUM(export_revenue) + days*heffingskorting/365
  (days = calendar days since the active version's effective_from, incl. today)
- state_class total_increasing -> total (monetary-compatible; values still only
  increase). device_class/key/names: key unchanged, names clarified.
- buy/sell_price_now and DeviceInfo untouched. Decimal money math. Tests added.
2026-06-24 14:26:32 +02:00
tliu93 effe434637 frontend: show Energy/DSMR/meter times in local timezone, 24h
Backend serializes SQLite-read (naive) UTC datetimes without a Z, so the browser
was parsing them as local time -> the UI effectively showed UTC values.

- New src/utils/datetime.ts: treats a tz-less timestamp as UTC, then formats in
  the browser's local timezone with hour12:false (no AM/PM).
- Applied to Energy tabs (contracts/prices/costs/DSMR), per-device readings &
  trends, and the meter readings table on the Records page.
- CostView Today/This month now use local day boundaries (queries stay UTC ISO).
- Untouched: location/poo (raw Zulu strings), the map, and query-window params.
- Tests are timezone-independent (verified under TZ=America/Los_Angeles).
2026-06-24 13:22:07 +02:00
tliu93 9d3e28a529 Billing fix: don't bill periods from stale/out-of-range DSMR readings
Root cause (found in the live DB): register_at() returned the latest reading
for any boundary beyond the DSMR data range (a future instant or a long gap),
so future periods got start==end -> delta 0 and were written as degraded=False
'ok' rows with cost 0. recompute_range() over a future end then filled the whole
month with such fake rows, which then occupied the slots so the per-minute tick
(immutability guard skips non-degraded rows) never recomputed the real data.

- register_at: ignore a reading older than one period (15min) before the
  boundary -> returns None -> period is marked degraded instead of a fake 0.
- recompute_range: only process closed periods (t1 <= now); never future ones.
- Tests cover staleness window, out-of-range -> degraded, recompute skips
  future, and a regression that normal closed periods are unaffected.
2026-06-24 12:44:05 +02:00
tliu93 7e61b0a938 HA Discovery fix: energy-cost sensors are always-available
The energy-cost device has only sensors (no online/offline heartbeat), but the
discovery config still declared an availability topic that nothing ever publishes
'online' to — so Home Assistant marked every energy-cost entity 'unavailable'
even though the state topic was being published (visible in MQTT Explorer).

- DeviceInfo gains provides_availability (default True, keeps Modbus behavior).
- _energy_cost_provider sets provides_availability=False.
- build_discovery_payload only emits availability/availability_mode for devices
  that publish a heartbeat; HA treats the rest as always-available.
- Test asserts energy-cost configs omit availability.
2026-06-24 11:54:49 +02:00
tliu93 e8521351d7 frontend: add DSMR data panel to Energy page
- New 'DSMR' tab showing the latest parsed telegram (GET /api/energy/dsmr/latest)
  as a key/value table — the source-of-truth view so you can see ingest is
  landing data without reading the DB.
- Auto-refresh toggle (~10s); loading/error/empty states (empty explains the
  likely cause); null phase values shown as a dash.
- useDsmrLatest extended with optional auto-refresh. Tests added.
2026-06-24 11:02:42 +02:00
tliu93 8d4f496ff4 DSMR hardening: restart-free config + decouple from telegram id
Config hot-reload (no restart):
- MqttManager.unsubscribe(); apply_dsmr_subscription(settings) re-applies the
  DSMR subscription (subscribe/unsubscribe/topic-change, fresh snapshot so the
  sample interval also takes effect).
- Called from lifespan AND PUT /api/config, so toggling DSMR ingest in the UI
  takes effect immediately without restarting the app.

Decouple ingest from the DSMR telegram id (overflows / resets to zero):
- Migration 20260624_12: drop UNIQUE(source_id), make recorded_at UNIQUE.
- Idempotency now keys on recorded_at (telegram timestamp); the table's own
  autoincrement PK is the stable identity. source_id kept only as a reference.
- Regression test: colliding telegram ids no longer drop new data.
2026-06-24 11:02:32 +02:00
tliu93 78f3cc4776 M6-T11: document M6 (energy pricing + DSMR + Tibber) across README/roadmap/arch
- README: M6 section + feature list + new tables/config items.
- roadmap: M6 row (graduated) + detail section; M6 in milestone index.
- architecture-overview: M6 models/services/integrations/routes/jobs/config.
- design/README: index m6, drop stale '三个里程碑' wording.
- OpenAPI already in sync (no diff).
2026-06-23 23:52:07 +02:00
tliu93 96e88861d4 M6-T10: frontend contract management + price/cost views + Tibber test
- EnergyPage: Mantine Tabs (Devices kept intact + Contracts/Prices/Costs).
- ContractManager/ContractForm: list/activate/add-version + version history;
  form fields rendered dynamically from /api/energy/profiles structure.
- TibberPrices + CostView: Recharts price curve, cost trend/detail/summary,
  recompute; window-bounded, currency/units from API, empty/error/loading states.
- ConfigPage: tri-state Tibber test button (token never shown).
- hooks.ts: typed energy hooks; schema.d.ts regenerated via codegen.
2026-06-23 23:32:39 +02:00
tliu93 57f2459f60 M6-T09: add Energy data API (prices/costs/summary/dsmr-latest/recompute/tibber-test)
- routes/api/energy.py + schemas/energy.py: GET prices (tibber points or manual
  tariff, buy/sell consistent with the pricing strategies), GET costs (time
  window + limit, ascending), GET costs/summary (summarize passthrough), GET
  dsmr/latest, POST costs/recompute (idempotent, <=366d guard, CSRF), POST
  tibber/test (three-state, token never echoed, CSRF).
- main.py registers router; OpenAPI re-exported; tests for all endpoints.
2026-06-23 23:05:45 +02:00
tliu93 c61fc2d4ba M6-T08: add energy_cost expose provider + post-billing state publish
- expose.py: _energy_cost_provider yields buy/sell_price_now + import_cost_total
  /export_revenue_total (total_increasing, monetary), value_getters read from
  energy_cost_period; 2-element identifiers so the existing HA Discovery builder
  (which indexes identifiers[1]) works. Other providers/mechanism untouched.
- main.py: energy-cost job calls publish_states after computing periods (no-op
  when MQTT/Discovery off). Tests added.
Note: energy-cost sensors lack an availability heartbeat (ha_discovery mechanism
limitation, out of scope here).
2026-06-23 22:44:41 +02:00
tliu93 d35ac7d752 M6-T07: add metering billing engine + period job + summary + recompute
- app/services/energy_cost.py: register_at (register values at a UTC boundary),
  compute_period (register deltas x active-version strategy, snapshot price +
  contract_version_id, immutable unless overwrite), compute_closed_periods
  (bounded lookback), recompute_range (explicit overwrite), summarize
  (sum net + standing fees month/30 - heffingskorting year/365).
- Missing tibber price -> skip; missing reading -> degraded (zeroed amounts,
  null version), consistently on both create and recompute paths.
- main.py: 1-minute energy-cost job (existing jobs/MQTT untouched). Tests added.
2026-06-23 22:26:56 +02:00
tliu93 ef48032adb M6-T06: extend MqttManager with subscribe + DSMR ingest
- mqtt.py: subscription registry, on_message dispatch (swallows handler
  errors), re-subscribe on (re)connect; publish/connect/disconnect/reconnect
  unchanged.
- app/services/dsmr_ingest.py: handle_message stores the full telegram frame
  (no allowlist; gas/phases/null preserved), 10s downsample by timestamp.second,
  source_id idempotency (select + IntegrityError rollback). Net-thread safe.
- main.py registers the DSMR subscription only when dsmr_ingest_enabled.
2026-06-23 21:52:57 +02:00
tliu93 7e266a21eb M6-T05: add Tibber GraphQL client + price refresh service + schedule
- app/integrations/tibber/client.py: fetch_price_range/fetch_current_price
  (priceInfoRange QUARTER_HOURLY, startsAt->UTC, no fixed node count assumption),
  TibberError/TibberAuthError; token never logged or put in exception text.
- app/services/tibber_prices.py: refresh_prices upserts by starts_at (idempotent),
  no-op unless an active kind=tibber contract + token exist.
- main.py: hourly tibber-refresh job (existing jobs/MQTT untouched). Tests added.
2026-06-23 21:35:29 +02:00
tliu93 4284bd40a7 M6-T04: add EnergyContract CRUD + versions + profile validation API
- app/services/contracts.py: create_contract, add_version (append-only,
  auto-closes prior version), activate_contract (mutex), and
  active_contract_version_at (half-open [from, to)) for the billing engine.
- app/schemas/energy_contract.py + routes/api/energy_contracts.py: 6 endpoints
  (GET/POST contracts, GET/PATCH contracts/{id}, POST .../versions, GET profiles).
  Values validated against pricing profiles (422 on mismatch); no DELETE.
- main.py registers router; OpenAPI re-exported; tests for CRUD/version/activate.
2026-06-23 21:19:25 +02:00
tliu93 bedea196c3 M6-T03: add pricing profile framework + strategy registry
- app/integrations/pricing: profiles.py (pydantic ManualProfile/TibberProfile,
  load_profile/list_profiles/validate_values), manual.yaml + tibber.yaml
  (structure-only, no price values), strategies.py (register/get_strategy,
  manual dual-tariff + tibber strategies, all Decimal).
- tibber strategy matches nearest tibber_price with starts_at <= t0.
- tests for profiles + strategies (hand-checked dual-tariff & tibber math).
2026-06-23 20:58:34 +02:00
tliu93 df54f5518b M6-T02: add DSMR + Tibber flat config fields
- Settings: dsmr_ingest_enabled, dsmr_mqtt_topic, dsmr_sample_interval_s,
  tibber_api_token (secret), tibber_home_id (all opt-in / off by default).
- CONFIG_FIELDS DSMR + Tibber sections; _settings_payload extended.
- .env.example examples; test_api_config coverage (sections, secret, 422).
2026-06-23 20:43:46 +02:00
tliu93 18fe18281b M6-T01: add 5 energy tables, models, and migration
- app/models/energy.py: DsmrReading, EnergyContract, EnergyContractVersion,
  TibberPrice, EnergyCostPeriod (per design §3.5; JSON blobs, RESTRICT FKs).
- alembic_app migration 20260623_11_energy_tables (head), env.py imports.
- app_db_adopt APP_BASELINE_REVISION bumped to new head.
- tests/test_energy_models.py + expose_catalog test fixups for new head.
2026-06-23 20:33:35 +02:00
tliu93 355c8eb8b8 fix: add paho-mqtt and pymodbus to dev-requirements lock (CI install was missing M5 deps)
pytest / test (push) Successful in 5m13s
frontend / frontend (push) Successful in 1m48s
2026-06-23 20:06:13 +02:00
98 changed files with 30929 additions and 169 deletions
+18
View File
@@ -43,5 +43,23 @@ MQTT_TLS_ENABLED=false
# Optional: Home Assistant MQTT Discovery.
# Requires MQTT_ENABLED=true and a running MQTT broker.
# HA_DISCOVERY_PREFIX is the config topic prefix (must be "homeassistant" for HA auto-discovery).
# HA_STATE_TOPIC_PREFIX is the prefix for state/availability topics (separate from discovery).
HA_DISCOVERY_ENABLED=false
HA_DISCOVERY_PREFIX=homeassistant
HA_STATE_TOPIC_PREFIX=home_automation
# Optional: DSMR smart-meter ingest via MQTT (M6; default off — opt-in).
# Set DSMR_INGEST_ENABLED=true to subscribe to the DSMR MQTT topic and
# store 10-second downsampled telegram frames in dsmr_reading.
# DSMR_INGEST_ENABLED=false
# DSMR_MQTT_TOPIC=dsmr/json
# DSMR_SAMPLE_INTERVAL_S=10
# DSMR tariff topic: publishes "1" (dal/off-peak) or "2" (normal/peak).
# Empty string disables tariff-aware pricing (buy/sell_price_now always show normal rate).
# DSMR_TARIFF_TOPIC=dsmr/meter-stats/electricity_tariff
# Optional: Tibber dynamic pricing credentials (M6).
# Only used when an active energy contract with kind=tibber is configured.
# TIBBER_API_TOKEN=
# TIBBER_HOME_ID=
+70 -2
View File
@@ -17,6 +17,10 @@
- **Modbus 设备采集**:通过 YAML profile(首个:SDM120 电表)按设备周期轮询 Modbus-TCP 网关,解码工程量并落通用读数表(`modbus_device` + `modbus_reading`
- **MQTT + Home Assistant Discovery**:以可勾选方式把 Modbus 设备/工程量注册为 HA device/entity(含 binary_sensor online),state 周期发布;配置变更可重连重发
- **前端侧边栏 + Energy 视图**:侧边导航替换顶栏;Energy 页管理 Modbus 设备、展示最新读数与 Recharts 走势图;Config 页 Accordion 分区展开;Expose 设置勾选 HA 可暴露实体
- **DSMR 实时电表接入**:订阅 DSMR Reader 的 `dsmr/json`(每秒一帧)、整帧 JSON blob 按 10 秒降采样落库(`dsmr_reading`
- **通用电价合同层**YAML profile 定合同结构(manual 固定/双费率 / tibber 动态电价);`EnergyContract`+`EnergyContractVersion` 存 UI 可填的数值,改价加新版本旧版本保留;price strategy 按 kind 出价
- **实时买卖电费计算**:每 15 分钟按寄存器差值(`_1`=dal/低、`_2`=normal/高)× 买/卖价算计量电费,快照不可变;日/月/年汇总加固定费减 heffingskorting
- **反哺 Home Assistant Energy**:当前买/卖价 + 累计买电支出/卖电收入(`total_increasing`)发成 HA 实体,可直接挂 HA Energy 仪表盘
- pytest 测试与 OpenAPI 导出脚本
- Docker / Compose 部署入口
@@ -36,6 +40,10 @@
- Modbus 设备定义(`modbus_device` 表)
- Modbus 通用读数(`modbus_reading` 表,JSON payload
- HA 实体暴露开关(`exposed_entity_toggle` 表)
- DSMR 电表实时读数(`dsmr_reading` 表,整帧 JSON blob10s 降采样)
- 电价合同(`energy_contract` 表)与版本(`energy_contract_version` 表,values JSON
- Tibber 15 分钟电价缓存(`tibber_price` 表,不可变)
- 每 15 分钟计量电费(`energy_cost_period` 表,快照价,不可变)
配置层只保留一个数据库环境变量:
@@ -47,7 +55,7 @@
python -m scripts.run_migrations
```
该命令会通过 Alembic 将 `app.db` 初始化或升级到最新 head(含全部表,包括 M5 新增的 `modbus_device``modbus_reading``exposed_entity_toggle`)。
该命令会通过 Alembic 将 `app.db` 初始化或升级到最新 head(含全部表,包括 M5 新增的 `modbus_device``modbus_reading``exposed_entity_toggle`,以及 M6 新增的 `dsmr_reading``energy_contract``energy_contract_version``tibber_price``energy_cost_period`)。
## 当前目录
@@ -55,7 +63,7 @@ python -m scripts.run_migrations
- `app/`: FastAPI 应用代码(包含 JSON API、业务服务、数据模型)
- `frontend/`: React SPA 前端(Vite + React + TypeScript + Mantine
- `alembic_app/`: App DB 的 Alembic migration 环境(管理所有表,含 M5 新增的 `modbus_device``modbus_reading``exposed_entity_toggle`
- `alembic_app/`: App DB 的 Alembic migration 环境(管理所有表,含 M5 新增的 `modbus_device``modbus_reading``exposed_entity_toggle`,以及 M6 新增的 `dsmr_reading``energy_contract``energy_contract_version``tibber_price``energy_cost_period`
- `tests/`: pytest 测试
- `docs/`: 当前系统说明文档
- `scripts/`: 辅助脚本,例如 OpenAPI 导出
@@ -366,6 +374,64 @@ CLI 工具为受控手工验证而设(设备需接市电),仅暴露读功
- **Config 页 Accordion**:各大 config section 可独立折叠/展开;「Home Assistant Expose」面板按设备分组勾选可暴露实体、显示 MQTT/Discovery 连接状态、「重新发布 discovery」按钮。
- SPA 路由新增 `/energy`
## M6 DSMR 接入 / 电价合同 / 实时电费计算 / HA Energy 反哺
M6 在 M5 IoT 基建之上接入 DSMR 实时智能电表数据,建立通用电价合同层,按每 15 分钟算出实际买卖电费并反哺 HA Energy。
### 依赖
M6 **不新增任何 Python 依赖**,复用 M5 已有的 `httpx`Tibber GraphQL)、`paho-mqtt`DSMR 订阅)、`pyyaml`pricing profile 加载)、`apscheduler`(抓价 job、计费 job)。
### DSMR 实时电表接入
订阅 DSMR Reader 的 `dsmr/json` topic(每秒一帧完整 telegram),整帧存为 JSON blob、按 `dsmr_sample_interval_s`(默认 10 秒)降采样落 `dsmr_reading``source_id` 幂等去重)。`dsmr_ingest_enabled`(默认 falseopt-in)。
### 电价合同层
- **YAML profile 定结构**(仓库内,不放数值):`manual.yaml`(固定/双费率:buy_normal/dal、sell_normal/dal、energy_tax、ode、固定费、heffingskorting);`tibber.yaml`(动态:source=tibber_apienergy_tax、sell_adjust
- **`EnergyContract` + `EnergyContractVersion`**(UI 填数值):改价 = 加新版本行(带 `effective_from`),旧版本保留(审计链);一次只有一个 active 合同
- **price strategy**`manual` 用双费率常数(`buy = energy_buy_档 + energy_tax``sell = sell_档`);`tibber``tibber_price.total` 作买价(已含税,demo 确认 `total=energy+tax`),`total energy_tax sell_adjust` 作卖价(卖价残差 `sell_adjust` 默认 0,待真实账单核定)
### 每 15 分钟计量电费(不可变)
APScheduler 1 分钟 tick,取每个闭合 15 分钟窗口的 DSMR 寄存器差值(`delivered_1/2``returned_1/2``_1`=dal/低,`_2`=normal/高,NL 惯例)× 当时合同版本的 strategy 出价,upsert `energy_cost_period`(快照当时价 + `contract_version_id`)。缺价/缺数据时标 `degraded``POST /api/energy/costs/recompute` 显式重算。
日/月/年汇总 = Σnet + 固定费(network_fee + management_fee 按月→天 × 天数)- heffingskorting(按年→天 × 天数),读时计算、不落表。能源税 `energy_tax` 参考值约 0.1108 EUR/kWh2026 第一档含 VAT,待真实账单核定;该值由 UI 填入合同版本,YAML profile 仅声明字段 unit,代码无写死默认数值)。
### Tibber 动态电价
`app/integrations/tibber/client.py` httpx POST GraphQL`priceInfoRange(QUARTER_HOURLY, first=96)`),解析 `startsAt`/`total`/`energy`/`tax`/`level`,按 `starts_at` upsert `tibber_price`(幂等)。启动 + 每小时抓取今明两天 15 分钟价、幂等 upserthourly trigger,确保每日刷新且可补重试);仅当 active 合同 kind=tibber 且 `tibber_api_token` 存在时运行。`POST /api/energy/tibber/test` 试连三态(success 带当前价 / config-error / failed)。
### 反哺 Home Assistant Energy
`_energy_cost_provider` 向 expose 框架注册 4 个实体:`buy_price_now``sell_price_now`(€/kWh sensor)、`import_cost_total``export_revenue_total``total_increasing` monetary,可直接挂 HA Energy 仪表盘)。默认未勾选,在 Expose 面板启用。
### API 端点(M6 新增)
| 端点 | 用途 |
| --- | --- |
| `GET /api/energy/contracts` | 列出合同 + active 标记 |
| `POST /api/energy/contracts` | 新建合同(kind + 首版本值,按 profile 校验)|
| `GET /api/energy/contracts/{id}` | 单个合同 + 版本历史 |
| `PATCH /api/energy/contracts/{id}` | 改名 / 激活 |
| `POST /api/energy/contracts/{id}/versions` | 加新版本(改价,带生效日期)|
| `GET /api/energy/profiles` | 列出 pricing profile 结构(前端按它渲染表单)|
| `GET /api/energy/prices` | 区间价格点(曲线)|
| `GET /api/energy/costs` | 区间 `energy_cost_period`(走势/明细)|
| `GET /api/energy/costs/summary` | 区间汇总(计量电费 + 固定费 − 抵扣)|
| `POST /api/energy/costs/recompute` | 幂等重算 |
| `GET /api/energy/dsmr/latest` | 最新 `dsmr_reading` |
| `POST /api/energy/tibber/test` | 试连 Tibber + 拉当前价,三态 |
DSMR/Tibber 标量配置复用现有 `GET/PUT /api/config`(新增 `dsmr_ingest_enabled``dsmr_mqtt_topic``dsmr_sample_interval_s``tibber_api_token`secret)、`tibber_home_id`)。
### 前端视图(M6 新增,并入 Energy 视图)
- **Contracts Tab**:合同列表 + 新建/编辑(表单按 `/api/energy/profiles` 结构渲染,不 hardcode 字段)+ 激活 + 改价加版本 + 版本历史只读。
- **Prices Tab**15 分钟价格曲线(tibber 动态或 manual 档位),复用 Recharts。
- **Costs Tab**:费用走势/明细 + 汇总卡片(含固定费/抵扣)。
- **Config 页 Tibber 测试**:三态(success/config-error/failed)。
## Config 持久化
当前 config 页面不会把修改写回 `.env`
@@ -394,6 +460,8 @@ CLI 工具为受控手工验证而设(设备需接市电),仅暴露读功
- MQTT broker 配置(`MQTT_ENABLED``MQTT_BROKER_HOST/PORT/USERNAME/PASSWORD``MQTT_TLS_ENABLED`
- Home Assistant Discovery 配置(`HA_DISCOVERY_ENABLED``HA_DISCOVERY_PREFIX`
- Modbus 采集配置(`MODBUS_POLLING_ENABLED`
- DSMR 接入配置(`DSMR_INGEST_ENABLED``DSMR_MQTT_TOPIC``DSMR_SAMPLE_INTERVAL_S`
- Tibber 凭据(`TIBBER_API_TOKEN`secret)、`TIBBER_HOME_ID`
其中 SMTP password 与其他 secret 字段一致:
+7
View File
@@ -13,6 +13,13 @@ from app.models.location import Location # noqa: F401
from app.models.poo import PooRecord # noqa: F401
from app.models.modbus import ModbusDevice, ModbusReading # noqa: F401
from app.models.expose import ExposedEntityToggle # noqa: F401
from app.models.energy import ( # noqa: F401
DsmrReading,
EnergyContract,
EnergyContractVersion,
TibberPrice,
EnergyCostPeriod,
)
config = context.config
@@ -0,0 +1,132 @@
"""add energy pricing and DSMR metering tables
Revision ID: 20260623_11_energy_tables
Revises: 20260622_10_exposed_entities
Create Date: 2026-06-23 00:00:00.000000
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
revision: str = "20260623_11_energy_tables"
down_revision: Union[str, None] = "20260622_10_exposed_entities"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# dsmr_reading — raw DSMR telegram blobs (10-second down-sampled).
# No device table: single P1 smart meter; a ``source`` column can be added later
# if a second meter is introduced.
op.create_table(
"dsmr_reading",
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
sa.Column("recorded_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("source_id", sa.Integer(), nullable=True),
sa.Column("payload", sa.JSON(), nullable=False),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("source_id", name="uq_dsmr_reading_source_id"),
)
op.create_index(
"ix_dsmr_reading_recorded_at",
"dsmr_reading",
["recorded_at"],
unique=False,
)
# energy_contract — contract head (manual or tibber, one active at a time).
op.create_table(
"energy_contract",
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
sa.Column("name", sa.String(length=255), nullable=False),
sa.Column("kind", sa.String(length=32), nullable=False),
sa.Column("active", sa.Boolean(), nullable=False),
sa.Column("currency", sa.String(length=8), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.PrimaryKeyConstraint("id"),
)
# energy_contract_version — versioned pricing values; append-only for auditability.
# Must be created after energy_contract because of the FK dependency.
op.create_table(
"energy_contract_version",
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
sa.Column("contract_id", sa.Integer(), nullable=False),
sa.Column("effective_from", sa.DateTime(timezone=True), nullable=False),
sa.Column("effective_to", sa.DateTime(timezone=True), nullable=True),
sa.Column("values", sa.JSON(), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(
["contract_id"],
["energy_contract.id"],
ondelete="RESTRICT",
),
sa.PrimaryKeyConstraint("id"),
)
# tibber_price — cached Tibber 15-minute spot prices (immutable once fetched).
op.create_table(
"tibber_price",
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
sa.Column("starts_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("resolution", sa.String(length=32), nullable=False),
sa.Column("energy", sa.Float(), nullable=False),
sa.Column("tax", sa.Float(), nullable=False),
sa.Column("total", sa.Float(), nullable=False),
sa.Column("level", sa.String(length=32), nullable=True),
sa.Column("currency", sa.String(length=8), nullable=False),
sa.Column("fetched_at", sa.DateTime(timezone=True), nullable=False),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("starts_at", name="uq_tibber_price_starts_at"),
)
# energy_cost_period — computed 15-minute billing periods (immutable snapshot).
# Must be created after energy_contract_version because of the FK dependency.
op.create_table(
"energy_cost_period",
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
sa.Column("period_start", sa.DateTime(timezone=True), nullable=False),
sa.Column("d1_kwh", sa.Float(), nullable=False),
sa.Column("d2_kwh", sa.Float(), nullable=False),
sa.Column("r1_kwh", sa.Float(), nullable=False),
sa.Column("r2_kwh", sa.Float(), nullable=False),
sa.Column("import_cost", sa.Float(), nullable=False),
sa.Column("export_revenue", sa.Float(), nullable=False),
sa.Column("net_cost", sa.Float(), nullable=False),
sa.Column("currency", sa.String(length=8), nullable=False),
sa.Column("pricing", sa.JSON(), nullable=False),
sa.Column("contract_version_id", sa.Integer(), nullable=True),
sa.Column("degraded", sa.Boolean(), nullable=False),
sa.Column("computed_at", sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(
["contract_version_id"],
["energy_contract_version.id"],
ondelete="RESTRICT",
),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("period_start", name="uq_energy_cost_period_period_start"),
)
def downgrade() -> None:
# Drop tables in reverse dependency order: tables with FKs first.
# energy_cost_period has a FK to energy_contract_version — drop it first.
op.drop_table("energy_cost_period")
# tibber_price has no FK dependencies on our new tables.
op.drop_table("tibber_price")
# energy_contract_version has a FK to energy_contract — drop it before energy_contract.
op.drop_table("energy_contract_version")
# energy_contract has no FK dependencies on our new tables.
op.drop_table("energy_contract")
# dsmr_reading has no FK dependencies.
op.drop_index("ix_dsmr_reading_recorded_at", table_name="dsmr_reading")
op.drop_table("dsmr_reading")
@@ -0,0 +1,58 @@
"""decouple dsmr_reading from the telegram id
The DSMR Reader's own ``id`` field is auto-incrementing but is known to overflow
and require a manual reset to zero (a long-standing DSMR firmware quirk). If we
keep a UNIQUE constraint on ``source_id`` (the telegram id) and use it for
idempotency, an overflow/reset would make legitimately-new telegrams collide with
old ids and be silently dropped as "duplicates" — data loss.
This migration removes that coupling:
- Drops the UNIQUE constraint on ``source_id`` (the column is kept as a plain,
nullable reference value; it is no longer relied upon for uniqueness/dedup).
- Makes ``recorded_at`` (the telegram timestamp) UNIQUE instead — a single P1
meter emits one telegram per timestamp, so this is a robust, telegram-id-
independent idempotency key. The table's own autoincrement ``id`` PK remains
the stable internal identity.
Revision ID: 20260624_12_dsmr_decouple_telegram_id
Revises: 20260623_11_energy_tables
Create Date: 2026-06-24 00:00:00.000000
"""
from typing import Sequence, Union
from alembic import op
revision: str = "20260624_12_dsmr_decouple_telegram_id"
down_revision: Union[str, None] = "20260623_11_energy_tables"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# SQLite cannot ALTER away a constraint in place, so recreate the table via
# Alembic's batch mode. The table is recreated and rows are copied; existing
# data (if any) is preserved. recorded_at must be unique for this to succeed
# — a single meter never emits two telegrams at the exact same timestamp.
with op.batch_alter_table("dsmr_reading", schema=None) as batch_op:
# Old non-unique index on recorded_at is replaced by the unique constraint.
batch_op.drop_index("ix_dsmr_reading_recorded_at")
# Telegram id is no longer a uniqueness/idempotency key.
batch_op.drop_constraint("uq_dsmr_reading_source_id", type_="unique")
# Timestamp becomes the telegram-id-independent dedup key.
batch_op.create_unique_constraint(
"uq_dsmr_reading_recorded_at", ["recorded_at"]
)
def downgrade() -> None:
with op.batch_alter_table("dsmr_reading", schema=None) as batch_op:
batch_op.drop_constraint("uq_dsmr_reading_recorded_at", type_="unique")
batch_op.create_unique_constraint(
"uq_dsmr_reading_source_id", ["source_id"]
)
batch_op.create_index(
"ix_dsmr_reading_recorded_at", ["recorded_at"], unique=False
)
@@ -0,0 +1,238 @@
"""add meter table and energy_cost_period.meter_id
Introduces the ``meter`` table (one row per physical meter installation epoch)
and a nullable FK column ``energy_cost_period.meter_id`` that attributes each
billing period to a specific physical meter.
**Backfill logic (§3.7 of the M7 design doc)**:
If the database already contains any ``dsmr_reading`` or
``energy_cost_period`` rows, one initial ``meter`` row is created:
label = "Initial meter"
commodity = "electricity"
started_at = earliest dsmr_reading.recorded_at
(or, if none, earliest energy_cost_period.period_start,
or, if still none, the migration timestamp)
ended_at = NULL (still active)
reason = "initial"
All existing ``energy_cost_period`` rows are then back-filled with that
initial meter's id.
**Idempotency**: the backfill is guarded with a check for any existing
``meter`` row whose ``reason = 'initial'`` and ``commodity = 'electricity'``
and ``ended_at IS NULL``, so repeating the upgrade does not create duplicate
meters or overwrite already-filled meter_id values.
**Audit (on-non-degraded periods only)**: after backfilling, the number of
non-degraded ``energy_cost_period`` rows with ``meter_id IS NULL`` must be
zero; if it is not, the migration raises a ``RuntimeError`` and rolls back.
**Data safety**: this migration is additive only — no existing rows are
deleted or overwritten; it only creates a new table, adds a nullable column,
and back-fills that column.
Revision ID: 20260625_13_meter_table
Revises: 20260624_12_dsmr_decouple_telegram_id
Create Date: 2026-06-25 00:00:00.000000
"""
from datetime import datetime, timezone
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
revision: str = "20260625_13_meter_table"
down_revision: Union[str, None] = "20260624_12_dsmr_decouple_telegram_id"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# ------------------------------------------------------------------ #
# 1. Create the meter table. #
# ------------------------------------------------------------------ #
op.create_table(
"meter",
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
sa.Column("label", sa.String(length=255), nullable=False),
sa.Column("commodity", sa.String(length=32), nullable=False),
sa.Column("started_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("ended_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("reason", sa.String(length=64), nullable=False),
sa.Column("note", sa.String(length=1024), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.PrimaryKeyConstraint("id"),
)
# ------------------------------------------------------------------ #
# 2. Add meter_id column to energy_cost_period (nullable FK). #
# ------------------------------------------------------------------ #
with op.batch_alter_table("energy_cost_period", schema=None) as batch_op:
batch_op.add_column(
sa.Column("meter_id", sa.Integer(), nullable=True)
)
batch_op.create_foreign_key(
"fk_energy_cost_period_meter_id",
"meter",
["meter_id"],
["id"],
ondelete="RESTRICT",
)
# ------------------------------------------------------------------ #
# 3. Backfill initial meter (idempotent). #
# ------------------------------------------------------------------ #
conn = op.get_bind()
# Check if there is already an initial meter (idempotency guard).
existing_initial = conn.execute(
sa.text(
"SELECT id FROM meter "
"WHERE reason = 'initial' AND commodity = 'electricity' AND ended_at IS NULL "
"LIMIT 1"
)
).fetchone()
if existing_initial is not None:
# Already backfilled — nothing to do.
return
# Determine whether there is any historical data to create a meter for.
has_readings = conn.execute(
sa.text("SELECT 1 FROM dsmr_reading LIMIT 1")
).fetchone()
has_periods = conn.execute(
sa.text("SELECT 1 FROM energy_cost_period LIMIT 1")
).fetchone()
if not has_readings and not has_periods:
# Empty database: no historical data, so no initial meter is needed.
# meter_id will remain NULL on any future rows until T02 service layer
# starts populating it.
return
# Determine started_at: earliest dsmr_reading.recorded_at, falling back to
# earliest energy_cost_period.period_start, and finally to now().
earliest_reading_row = conn.execute(
sa.text("SELECT MIN(recorded_at) AS ts FROM dsmr_reading")
).fetchone()
earliest_period_row = conn.execute(
sa.text("SELECT MIN(period_start) AS ts FROM energy_cost_period")
).fetchone()
started_at_value: datetime | None = None
if earliest_reading_row and earliest_reading_row[0] is not None:
# SQLite returns ISO strings for datetime columns; parse to datetime.
raw = earliest_reading_row[0]
started_at_value = _parse_sqlite_datetime(raw)
if started_at_value is None and earliest_period_row and earliest_period_row[0] is not None:
raw = earliest_period_row[0]
started_at_value = _parse_sqlite_datetime(raw)
if started_at_value is None:
started_at_value = datetime.now(tz=timezone.utc)
now_utc = datetime.now(tz=timezone.utc)
# Insert the initial meter row.
conn.execute(
sa.text(
"INSERT INTO meter (label, commodity, started_at, ended_at, reason, note, created_at) "
"VALUES (:label, :commodity, :started_at, NULL, :reason, NULL, :created_at)"
),
{
"label": "Initial meter",
"commodity": "electricity",
"started_at": _iso(started_at_value),
"reason": "initial",
"created_at": _iso(now_utc),
},
)
# Retrieve the newly created meter id.
meter_row = conn.execute(
sa.text(
"SELECT id FROM meter "
"WHERE reason = 'initial' AND commodity = 'electricity' AND ended_at IS NULL "
"LIMIT 1"
)
).fetchone()
assert meter_row is not None, "Initial meter row not found after insert"
meter_id: int = meter_row[0]
# Back-fill all existing energy_cost_period rows that have meter_id IS NULL.
conn.execute(
sa.text(
"UPDATE energy_cost_period SET meter_id = :mid WHERE meter_id IS NULL"
),
{"mid": meter_id},
)
# ------------------------------------------------------------------ #
# 4. Audit: verify all non-degraded periods have a meter_id. #
# ------------------------------------------------------------------ #
unmatched_row = conn.execute(
sa.text(
"SELECT COUNT(*) FROM energy_cost_period "
"WHERE meter_id IS NULL AND degraded = 0"
)
).fetchone()
unmatched_count: int = unmatched_row[0] if unmatched_row else 0
if unmatched_count != 0:
raise RuntimeError(
f"Meter backfill audit failed: {unmatched_count} non-degraded "
"energy_cost_period row(s) still have meter_id IS NULL after backfill. "
"Migration aborted to protect data integrity."
)
def downgrade() -> None:
# Remove the FK column from energy_cost_period first (references meter).
with op.batch_alter_table("energy_cost_period", schema=None) as batch_op:
batch_op.drop_constraint("fk_energy_cost_period_meter_id", type_="foreignkey")
batch_op.drop_column("meter_id")
# Drop the meter table.
op.drop_table("meter")
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _parse_sqlite_datetime(value: str | datetime) -> datetime:
"""Parse a SQLite datetime value into an aware UTC datetime.
SQLite stores datetimes as ISO 8601 strings. SQLAlchemy may return them
as plain strings or as naive datetimes (no tzinfo) depending on the driver
and column declaration. This helper normalises both forms to an aware UTC
``datetime``.
"""
if isinstance(value, datetime):
if value.tzinfo is None:
return value.replace(tzinfo=timezone.utc)
return value
# String form — strip trailing Z or +00:00 variants, then attach UTC.
s = str(value).strip()
for suffix in ("+00:00", "Z", " UTC"):
if s.endswith(suffix):
s = s[: -len(suffix)]
# SQLite uses space as the T separator in some formats.
s = s.replace(" ", "T")
try:
dt = datetime.fromisoformat(s)
except ValueError:
# Fallback: strip subseconds if present to handle unusual formats.
dt = datetime.strptime(s[:19], "%Y-%m-%dT%H:%M:%S")
return dt.replace(tzinfo=timezone.utc)
def _iso(dt: datetime) -> str:
"""Serialise a datetime to an ISO 8601 string for SQLite storage."""
if dt.tzinfo is not None:
dt = dt.astimezone(timezone.utc).replace(tzinfo=None)
return dt.strftime("%Y-%m-%dT%H:%M:%S")
@@ -0,0 +1,104 @@
"""add uuid column to meter table
Adds a stable ``uuid`` (UUID v4 string) column to the ``meter`` table so that
each meter epoch has a durable identity anchor suitable for use as an HA
Discovery ``unique_id``.
**Migration strategy (SQLite-safe)**:
SQLite does not support adding a NOT NULL + UNIQUE column to a non-empty table
in a single ``ALTER TABLE ADD COLUMN`` statement (adding a NOT NULL column
without a default value is rejected if the table already has rows). The
safe approach used here is:
1. Add ``uuid`` as a **nullable** column (SQLite allows this).
2. **Back-fill** every existing ``meter`` row with a distinct ``str(uuid4())``
value. Each row gets its *own* random UUID — not a shared value — so the
subsequent UNIQUE constraint is satisfied.
3. Use ``batch_alter_table`` (which re-creates the table under the hood in
SQLite) to alter the column to ``NOT NULL`` and add a UNIQUE constraint.
**Idempotency**: only rows where ``uuid IS NULL`` are back-filled; rows that
already have a uuid (e.g. from a repeated upgrade after a partial failure) are
left untouched.
**Audit**: after back-fill, the count of rows with ``uuid IS NULL`` must be
exactly zero; if not, the migration raises ``RuntimeError`` and rolls back.
**Data safety**: this migration is additive only — no existing rows are deleted
or overwritten; it only adds a new column and fills it in.
Revision ID: 20260625_14_meter_uuid
Revises: 20260625_13_meter_table
Create Date: 2026-06-25 00:00:00.000000
"""
import uuid as _uuid
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
revision: str = "20260625_14_meter_uuid"
down_revision: Union[str, None] = "20260625_13_meter_table"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
conn = op.get_bind()
# ------------------------------------------------------------------ #
# 1. Add uuid as a nullable column. #
# ------------------------------------------------------------------ #
with op.batch_alter_table("meter", schema=None) as batch_op:
batch_op.add_column(
sa.Column("uuid", sa.String(length=36), nullable=True)
)
# ------------------------------------------------------------------ #
# 2. Back-fill: assign a distinct UUID to every row that has #
# uuid IS NULL. Each row gets its own random value so that the #
# subsequent UNIQUE constraint is satisfied. #
# ------------------------------------------------------------------ #
rows = conn.execute(sa.text("SELECT id FROM meter WHERE uuid IS NULL")).fetchall()
for (meter_id,) in rows:
new_uuid = str(_uuid.uuid4())
conn.execute(
sa.text("UPDATE meter SET uuid = :uuid WHERE id = :mid"),
{"uuid": new_uuid, "mid": meter_id},
)
# ------------------------------------------------------------------ #
# 3. Audit: verify no rows remain with uuid IS NULL. #
# ------------------------------------------------------------------ #
null_count_row = conn.execute(
sa.text("SELECT COUNT(*) FROM meter WHERE uuid IS NULL")
).fetchone()
null_count: int = null_count_row[0] if null_count_row else 0
if null_count != 0:
raise RuntimeError(
f"meter.uuid back-fill audit failed: {null_count} meter row(s) still have "
"uuid IS NULL after back-fill. Migration aborted to protect data integrity."
)
# ------------------------------------------------------------------ #
# 4. Alter column to NOT NULL + UNIQUE (requires batch on SQLite). #
# batch_alter_table re-creates the table, so the UNIQUE constraint #
# and NOT NULL are applied atomically. #
# ------------------------------------------------------------------ #
with op.batch_alter_table("meter", schema=None) as batch_op:
batch_op.alter_column(
"uuid",
existing_type=sa.String(length=36),
nullable=False,
)
batch_op.create_unique_constraint("uq_meter_uuid", ["uuid"])
def downgrade() -> None:
# Drop the UNIQUE constraint and the uuid column (batch on SQLite).
with op.batch_alter_table("meter", schema=None) as batch_op:
batch_op.drop_constraint("uq_meter_uuid", type_="unique")
batch_op.drop_column("uuid")
+6
View File
@@ -86,6 +86,12 @@ def put_config(
logger.info("MQTT settings changed — triggering reconnect.")
mqtt_manager.reconnect(refreshed_settings)
# Re-apply the DSMR subscription so enabling/disabling DSMR ingest (or changing
# its topic / sample interval) takes effect immediately, without an app restart.
# Done after any MQTT reconnect so it operates on the current client.
from app.services.dsmr_ingest import apply_dsmr_subscription
apply_dsmr_subscription(refreshed_settings)
sections_raw = build_config_sections(db, refreshed_settings)
return ConfigUpdateResponse(sections=_sections_from_raw(sections_raw))
+584
View File
@@ -0,0 +1,584 @@
"""Energy data API: prices, costs, summary, DSMR, recompute, Tibber test (M6-T09).
All endpoints are under /api/energy, require an authenticated session, and
write endpoints (POST) additionally require a non-empty X-CSRF-Token header.
Route prefix note
-----------------
This router shares the ``/api/energy`` prefix with ``energy_contracts.py``
(which handles contract CRUD at /contracts/* and /profiles). The sub-paths
used here (/prices, /costs, /costs/summary, /costs/recompute, /dsmr/latest,
/tibber/test) are disjoint from the contract router's paths, so there is no
conflict.
Tibber token security
---------------------
``POST /api/energy/tibber/test`` calls the Tibber API but **never** echoes the
token in the response body or in log messages. Three-state logic mirrors the
MQTT test endpoint (M5-T10, app/api/routes/api/config.py::post_mqtt_test):
200 { result: "success", message: ..., price: {...} }
400 { result: "config-error", message: ... }
502 { result: "failed", message: ... }
Recompute safety
----------------
``POST /api/energy/costs/recompute`` is idempotent: it calls
``energy_cost.recompute_range`` which upserts existing rows without deleting
anything. The endpoint enforces a maximum time-window of 366 days to avoid
unbounded recomputation triggered by erroneous client requests.
Prices endpoint behaviour
-------------------------
``GET /api/energy/prices`` queries the ``tibber_price`` table for tibber
contracts, or derives the effective fixed-tariff prices for manual contracts
using the same formula as the billing engine (_manual_strategy in strategies.py):
buy_dal = energy.buy.dal + energy.energy_tax + energy.ode
buy_normal = energy.buy.normal + energy.energy_tax + energy.ode
sell_dal = energy.sell.dal (no tax added to sell price)
sell_normal = energy.sell.normal
"""
from __future__ import annotations
import logging
from datetime import UTC, datetime, timedelta
from typing import Any
from fastapi import APIRouter, Depends, Query, status
from fastapi.responses import JSONResponse
from sqlalchemy import select
from sqlalchemy.orm import Session
from app.api.routes.api.deps import require_csrf, require_session
from app.dependencies import get_app_settings, get_db
from app.config import Settings
from app.integrations.tibber.client import (
TibberAuthError,
TibberError,
fetch_current_price,
)
from app.models.energy import DsmrReading, EnergyCostPeriod, TibberPrice
from app.schemas.energy import (
CostPeriodSchema,
CostsResponse,
DsmrLatestResponse,
ManualTariffSchema,
PricePointSchema,
PricesResponse,
RecomputeResponse,
SummaryResponse,
TibberTestPriceSchema,
TibberTestResponse,
)
from app.services.auth import AuthenticatedSession
from app.services.contracts import active_contract_version_at
from app.services.energy_cost import recompute_range, summarize
from app.services.timezone import local_midnight_utc, local_now
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/energy", tags=["api-energy"])
# Maximum number of cost periods returned per request (mirrors modbus readings cap).
_COSTS_LIMIT_MAX = 5000
# Maximum allowed time-window for recompute to prevent unbounded computation.
_RECOMPUTE_MAX_DAYS = 366
# ---------------------------------------------------------------------------
# Internal helpers
# ---------------------------------------------------------------------------
def _as_utc(dt: datetime) -> datetime:
"""Attach UTC tzinfo to a naive datetime (SQLite read-back workaround)."""
if dt.tzinfo is None:
return dt.replace(tzinfo=UTC)
return dt
def _manual_tariff_from_values(values: dict[str, Any]) -> ManualTariffSchema:
"""Derive the effective fixed tariff from a manual contract version's values dict.
Mirrors the _manual_strategy formula (strategies.py):
buy_dal = energy.buy.dal + energy.energy_tax + energy.ode
buy_normal = energy.buy.normal + energy.energy_tax + energy.ode
sell_dal = energy.sell.dal (no tax added)
sell_normal = energy.sell.normal
"""
from decimal import Decimal
def _d(v: Any) -> Decimal:
return Decimal(str(v or 0))
energy = values.get("energy", {})
buy = energy.get("buy", {})
sell = energy.get("sell", {})
energy_tax = _d(energy.get("energy_tax", 0))
ode = _d(energy.get("ode", 0))
buy_dal = _d(buy.get("dal", 0)) + energy_tax + ode
buy_normal = _d(buy.get("normal", 0)) + energy_tax + ode
sell_dal = _d(sell.get("dal", 0))
sell_normal = _d(sell.get("normal", 0))
return ManualTariffSchema(
buy_dal=float(buy_dal),
buy_normal=float(buy_normal),
sell_dal=float(sell_dal),
sell_normal=float(sell_normal),
)
# ---------------------------------------------------------------------------
# GET /api/energy/prices
# ---------------------------------------------------------------------------
@router.get("/prices", response_model=PricesResponse)
def get_prices(
start: datetime | None = Query(
default=None,
description="Inclusive start of the time window (ISO 8601). "
"Defaults to the start of today UTC when omitted.",
),
end: datetime | None = Query(
default=None,
description="Inclusive end of the time window (ISO 8601). "
"Defaults to the end of tomorrow UTC when omitted.",
),
limit: int = Query(
default=500,
ge=1,
le=_COSTS_LIMIT_MAX,
description="Maximum number of Tibber price points to return.",
),
db: Session = Depends(get_db),
_auth: AuthenticatedSession = Depends(require_session),
) -> PricesResponse:
"""Return the price curve for the active contract.
**Tibber contracts** (kind="tibber"):
Fetches ``tibber_price`` rows within ``[start, end]``, ordered ascending
by ``starts_at``. At most ``limit`` rows are returned (most recent first
within the window, then reversed to ascending order — identical to the
modbus readings pattern).
Response ``points`` carries per-slot:
- ``buy = total`` (Tibber all-inclusive price)
- ``sell = total energy_tax sell_adjust`` (from active version values)
- ``level`` (Tibber price level, may be null)
``tariff`` is null.
**Manual contracts** (kind="manual"):
``points`` is empty. ``tariff`` carries the four effective prices
derived using the billing engine formula:
- ``buy_dal = energy.buy.dal + energy_tax + ode``
- ``buy_normal = energy.buy.normal + energy_tax + ode``
- ``sell_dal = energy.sell.dal``
- ``sell_normal = energy.sell.normal``
**No active contract**: returns kind=null, currency="EUR", points=[], tariff=null (200).
"""
now = datetime.now(UTC)
# Default window: today + tomorrow.
if start is None:
start = now.replace(hour=0, minute=0, second=0, microsecond=0)
if end is None:
end = (start + timedelta(days=2)).replace(hour=0, minute=0, second=0, microsecond=0)
start_utc = _as_utc(start)
end_utc = _as_utc(end)
# Resolve the active contract version at the start of the window.
version = active_contract_version_at(db, start_utc)
if version is None:
return PricesResponse(
kind=None,
currency="EUR",
points=[],
tariff=None,
)
contract = version.contract
currency = contract.currency
if contract.kind == "tibber":
# Fetch tibber_price rows in the window.
stmt = (
select(TibberPrice)
.where(
TibberPrice.starts_at >= start_utc,
TibberPrice.starts_at <= end_utc,
)
.order_by(TibberPrice.starts_at.desc())
.limit(limit)
)
rows = list(reversed(db.execute(stmt).scalars().all()))
# Derive sell price per-point using version values (energy_tax + sell_adjust).
from decimal import Decimal
def _d(v: Any) -> Decimal:
return Decimal(str(v or 0))
energy = version.values.get("energy", {}) if version.values else {}
energy_tax = _d(energy.get("energy_tax", 0))
sell_adjust = _d(energy.get("sell_adjust", 0))
points = []
for row in rows:
total = _d(row.total)
sell = float(total - energy_tax - sell_adjust)
points.append(
PricePointSchema(
starts_at=_as_utc(row.starts_at),
buy=row.total,
sell=sell,
level=row.level,
)
)
return PricesResponse(
kind="tibber",
currency=currency,
points=points,
tariff=None,
)
elif contract.kind == "manual":
tariff = _manual_tariff_from_values(version.values or {})
return PricesResponse(
kind="manual",
currency=currency,
points=[],
tariff=tariff,
)
else:
# Unknown kind — return empty response gracefully.
return PricesResponse(
kind=contract.kind,
currency=currency,
points=[],
tariff=None,
)
# ---------------------------------------------------------------------------
# GET /api/energy/costs
# ---------------------------------------------------------------------------
@router.get("/costs", response_model=CostsResponse)
def get_costs(
start: datetime | None = Query(
default=None,
description="Inclusive lower bound for period_start (ISO 8601).",
),
end: datetime | None = Query(
default=None,
description="Inclusive upper bound for period_start (ISO 8601).",
),
limit: int = Query(
default=500,
ge=1,
le=_COSTS_LIMIT_MAX,
description=f"Maximum number of cost periods to return (default 500, max {_COSTS_LIMIT_MAX}).",
),
db: Session = Depends(get_db),
_auth: AuthenticatedSession = Depends(require_session),
) -> CostsResponse:
"""Return energy_cost_period rows within a time window.
Rows are ordered by ``period_start`` ascending. When the window contains
more rows than ``limit``, the **most recent** N rows are returned (DESC LIMIT),
then reversed to ascending order — identical to the modbus readings pattern.
Query parameters:
- ``start``: inclusive lower bound on ``period_start`` (ISO 8601 datetime).
- ``end``: inclusive upper bound on ``period_start`` (ISO 8601 datetime).
- ``limit``: max rows to return (default 500, max {_COSTS_LIMIT_MAX}).
"""
stmt = (
select(EnergyCostPeriod)
.order_by(EnergyCostPeriod.period_start.desc())
.limit(limit)
)
if start is not None:
stmt = stmt.where(EnergyCostPeriod.period_start >= _as_utc(start))
if end is not None:
stmt = stmt.where(EnergyCostPeriod.period_start <= _as_utc(end))
rows = list(reversed(db.execute(stmt).scalars().all()))
items = [CostPeriodSchema.model_validate(r) for r in rows]
return CostsResponse(items=items, total=len(items))
# ---------------------------------------------------------------------------
# GET /api/energy/costs/summary
# ---------------------------------------------------------------------------
@router.get("/costs/summary", response_model=SummaryResponse)
def get_costs_summary(
start: datetime | None = Query(
default=None,
description=(
"Inclusive start of the summary interval (ISO 8601). "
"Defaults to the start of the current UTC day."
),
),
end: datetime | None = Query(
default=None,
description=(
"Exclusive end of the summary interval (ISO 8601). "
"Defaults to the start of the next UTC day (i.e. today's full data)."
),
),
db: Session = Depends(get_db),
_auth: AuthenticatedSession = Depends(require_session),
) -> SummaryResponse:
"""Aggregate billing for a time interval.
Calls ``energy_cost.summarize(session, start, end)`` which computes:
total_payable = Σ(net_cost) + fixed_costs credits
where ``fixed_costs`` is (network_fee + management_fee) apportioned to the
interval length in days (÷30 per month), and ``credits`` is heffingskorting
apportioned similarly (÷365 per year).
Both ``fixed_costs`` and ``credits`` are derived from the **currently active
contract version at ``end``**. When no active contract exists they are 0.
"""
if start is None or end is None:
# Default to the server's local today: [local_midnight, next_local_midnight).
# This ensures "today" aligns with the local calendar day (NL time) rather
# than UTC midnight.
local_today = local_now().date()
if start is None:
start = local_midnight_utc(local_today)
if end is None:
end = local_midnight_utc(local_today + timedelta(days=1))
result = summarize(db, _as_utc(start), _as_utc(end))
return SummaryResponse(**result)
# ---------------------------------------------------------------------------
# GET /api/energy/dsmr/latest
# ---------------------------------------------------------------------------
@router.get("/dsmr/latest", response_model=DsmrLatestResponse)
def get_dsmr_latest(
db: Session = Depends(get_db),
_auth: AuthenticatedSession = Depends(require_session),
) -> DsmrLatestResponse:
"""Return the most recent dsmr_reading row.
Returns ``{"found": false, "recorded_at": null, "payload": null}`` (200, not
404) when no rows exist yet, so the front-end can distinguish "no data" from
a server error.
"""
row = db.execute(
select(DsmrReading)
.order_by(DsmrReading.recorded_at.desc())
.limit(1)
).scalar_one_or_none()
if row is None:
return DsmrLatestResponse(found=False)
return DsmrLatestResponse(
found=True,
recorded_at=_as_utc(row.recorded_at),
payload=row.payload,
)
# ---------------------------------------------------------------------------
# POST /api/energy/costs/recompute
# ---------------------------------------------------------------------------
@router.post(
"/costs/recompute",
responses={
200: {"model": RecomputeResponse},
422: {"description": "Validation error (missing window or range too large)"},
},
)
def post_recompute(
start: datetime = Query(
...,
description="Inclusive start of the recompute window (ISO 8601). Required.",
),
end: datetime = Query(
...,
description="Exclusive end of the recompute window (ISO 8601). Required.",
),
db: Session = Depends(get_db),
_auth: AuthenticatedSession = Depends(require_session),
_csrf: None = Depends(require_csrf),
) -> RecomputeResponse:
"""Idempotently recompute billing records in a time window.
Calls ``energy_cost.recompute_range(session, start, end)`` which overwrites
existing rows (including successful ones) for every UTC quarter-hour boundary
in ``[start, end)``.
**Idempotency**: repeated calls with the same window produce the same
outcome. No rows are deleted; only upserted.
**Window constraint**: the maximum allowed range is {_RECOMPUTE_MAX_DAYS} days.
Requests exceeding this return 422.
Returns the number of periods for which a billing record was written.
Periods skipped due to missing contract or missing Tibber price are not counted.
"""
start_utc = _as_utc(start)
end_utc = _as_utc(end)
if end_utc <= start_utc:
from fastapi import HTTPException
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
detail="'end' must be strictly after 'start'.",
)
span_days = (end_utc - start_utc).total_seconds() / 86400
if span_days > _RECOMPUTE_MAX_DAYS:
from fastapi import HTTPException
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
detail=(
f"Time window is {span_days:.1f} days which exceeds the maximum of "
f"{_RECOMPUTE_MAX_DAYS} days. Use a smaller window."
),
)
n = recompute_range(db, start_utc, end_utc)
logger.info(
"POST /api/energy/costs/recompute [%s, %s): wrote %d period(s).",
start_utc.isoformat(),
end_utc.isoformat(),
n,
)
return RecomputeResponse(recomputed=n)
# ---------------------------------------------------------------------------
# POST /api/energy/tibber/test
# ---------------------------------------------------------------------------
@router.post(
"/tibber/test",
responses={
200: {"model": TibberTestResponse},
400: {"model": TibberTestResponse},
502: {"model": TibberTestResponse},
},
)
def post_tibber_test(
settings: Settings = Depends(get_app_settings),
_auth: AuthenticatedSession = Depends(require_session),
_csrf: None = Depends(require_csrf),
) -> JSONResponse:
"""Test Tibber API connectivity by fetching the current price point.
Three possible outcomes:
- **200** ``{ result: "success", message: ..., price: {...} }``
The Tibber API responded with a valid current price. ``price`` contains
starts_at, total, energy, tax, currency, and level.
- **400** ``{ result: "config-error", message: ... }``
The Tibber API token is empty or not configured.
- **502** ``{ result: "failed", message: ... }``
The API call failed (authentication rejected, network error, timeout,
unexpected response, etc.).
The API token is **never** included in the response body or logged.
"""
token = settings.tibber_api_token
home_id = settings.tibber_home_id or None # treat empty string as None
if not token:
logger.info("POST /api/energy/tibber/test: no token configured.")
return JSONResponse(
status_code=status.HTTP_400_BAD_REQUEST,
content=TibberTestResponse(
result="config-error",
message=(
"Tibber API token is not configured. "
"Set TIBBER_API_TOKEN in the Config page."
),
price=None,
).model_dump(mode="json"),
)
try:
price_point = fetch_current_price(token, home_id)
except TibberAuthError:
logger.warning("POST /api/energy/tibber/test: authentication failed.")
return JSONResponse(
status_code=status.HTTP_502_BAD_GATEWAY,
content=TibberTestResponse(
result="failed",
message=(
"Tibber API authentication failed. "
"Check that your API token is correct."
),
price=None,
).model_dump(mode="json"),
)
except TibberError as exc:
logger.warning("POST /api/energy/tibber/test: API call failed — %s", exc)
return JSONResponse(
status_code=status.HTTP_502_BAD_GATEWAY,
content=TibberTestResponse(
result="failed",
message=f"Tibber API call failed: {exc}",
price=None,
).model_dump(mode="json"),
)
price_schema = TibberTestPriceSchema(
starts_at=price_point.starts_at,
total=price_point.total,
energy=price_point.energy,
tax=price_point.tax,
currency=price_point.currency,
level=price_point.level,
)
logger.info(
"POST /api/energy/tibber/test: success (starts_at=%s, total=%s %s).",
price_point.starts_at.isoformat(),
price_point.total,
price_point.currency,
)
return JSONResponse(
status_code=status.HTTP_200_OK,
content=TibberTestResponse(
result="success",
message=(
f"Tibber API connected. Current price: "
f"{price_point.total} {price_point.currency}/kWh "
f"(starts {price_point.starts_at.isoformat()})."
),
price=price_schema,
).model_dump(mode="json"),
)
+331
View File
@@ -0,0 +1,331 @@
"""EnergyContract CRUD, versioning, and pricing-profile API (M6-T04).
All endpoints are under /api/energy, require an authenticated session, and
write endpoints (POST/PATCH) additionally require a non-empty X-CSRF-Token
header.
There is deliberately no DELETE endpoint for contracts: the FK RESTRICT
constraint on energy_contract_version.contract_id and
energy_cost_period.contract_version_id prevents accidental deletion of
contracts that have billing history. Users should instead deactivate a
contract (PATCH active=false) to stop it from being used.
Route ordering note
-------------------
GET /api/energy/profiles and GET /api/energy/contracts/{id} are on separate
path prefixes (/energy/profiles vs /energy/contracts/{id}) so there is no
ambiguity even without extra ordering gymnastics.
"""
from __future__ import annotations
import logging
from datetime import UTC, datetime
from typing import Any
from fastapi import APIRouter, Depends, HTTPException, status
from sqlalchemy.orm import Session
from app.api.routes.api.deps import require_csrf, require_session
from app.dependencies import get_db
from app.integrations.pricing.profiles import (
ProfileNotFoundError,
ProfileValidationError,
list_profiles,
)
from app.schemas.energy_contract import (
ContractCreate,
ContractDetailResponse,
ContractListResponse,
ContractPatch,
ContractResponse,
ContractVersionResponse,
ProfilesResponse,
VersionCreate,
)
from app.services.auth import AuthenticatedSession
from app.services import timezone as _tz_mod
from app.services.contracts import (
ContractVersionError,
activate_contract,
add_version,
create_contract,
deactivate_contract,
get_contract_or_none,
list_contracts,
)
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/energy", tags=["api-energy-contracts"])
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _get_contract_or_404(db: Session, contract_id: int):
"""Return the contract with the given id or raise 404."""
contract = get_contract_or_none(db, contract_id)
if contract is None:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"Energy contract {contract_id!r} not found.",
)
return contract
def _contract_detail(db: Session, contract) -> ContractDetailResponse:
"""Build a ContractDetailResponse for *contract*, loading versions."""
# Eager-load versions ordered by effective_from for consistent display.
from sqlalchemy import select
from app.models.energy import EnergyContractVersion
versions = list(
db.execute(
select(EnergyContractVersion)
.where(EnergyContractVersion.contract_id == contract.id)
.order_by(EnergyContractVersion.effective_from)
)
.scalars()
.all()
)
version_schemas = [ContractVersionResponse.model_validate(v) for v in versions]
return ContractDetailResponse(
id=contract.id,
name=contract.name,
kind=contract.kind,
active=contract.active,
currency=contract.currency,
created_at=contract.created_at,
updated_at=contract.updated_at,
versions=version_schemas,
)
def _raise_422_for_profile_error(exc: Exception) -> Any:
"""Convert a ProfileValidationError or ProfileNotFoundError into HTTP 422."""
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
detail=str(exc),
)
def _localize_effective_from(dt: datetime | None) -> datetime:
"""Resolve *dt* to an aware UTC datetime for storage.
Rules (Principle A):
- If *dt* is None → use ``datetime.now(UTC)`` (unchanged from before).
- If *dt* is timezone-aware → convert to UTC as-is.
- If *dt* is timezone-naive → interpret as server local wall-clock time,
localize with ``local_tz()``, then convert to UTC.
This means a front-end that sends ``"2026-06-25T00:00:00"`` (no Z) has it
interpreted as local midnight (e.g. CEST = UTC+2 → stored as 2026-06-24T22:00:00Z),
not as UTC midnight.
"""
if dt is None:
return datetime.now(UTC)
if dt.tzinfo is not None:
# Already aware: convert to UTC.
return dt.astimezone(UTC)
# Naive: assume server local wall-clock.
tz = _tz_mod.local_tz()
local_dt = dt.replace(tzinfo=tz)
return local_dt.astimezone(UTC)
# ---------------------------------------------------------------------------
# GET /api/energy/profiles
# ---------------------------------------------------------------------------
@router.get("/profiles", response_model=ProfilesResponse)
def get_profiles(
_auth: AuthenticatedSession = Depends(require_session),
) -> ProfilesResponse:
"""List all available pricing profile structures.
Returns the full profile structure for each supported contract kind
(``manual`` and ``tibber``). The front-end uses this to dynamically
render the correct fields and labels for the contract creation/editing form.
"""
return ProfilesResponse(profiles=list_profiles())
# ---------------------------------------------------------------------------
# GET /api/energy/contracts
# ---------------------------------------------------------------------------
@router.get("/contracts", response_model=ContractListResponse)
def list_energy_contracts(
db: Session = Depends(get_db),
_auth: AuthenticatedSession = Depends(require_session),
) -> ContractListResponse:
"""List all energy contracts with their active status.
Returns a flat list (no embedded version history); use
GET /api/energy/contracts/{id} to fetch the full version history for a
specific contract.
"""
contracts = list_contracts(db)
items = [ContractResponse.model_validate(c) for c in contracts]
return ContractListResponse(items=items, total=len(items))
# ---------------------------------------------------------------------------
# POST /api/energy/contracts
# ---------------------------------------------------------------------------
@router.post(
"/contracts",
response_model=ContractDetailResponse,
status_code=status.HTTP_201_CREATED,
)
def create_energy_contract(
body: ContractCreate,
db: Session = Depends(get_db),
_auth: AuthenticatedSession = Depends(require_session),
_csrf: None = Depends(require_csrf),
) -> ContractDetailResponse:
"""Create a new energy contract with an initial pricing version.
The ``values`` dict is validated against the YAML profile for the given
``kind``; non-conforming values result in 422 Unprocessable Entity.
The new contract is created with ``active=False``; use
PATCH /api/energy/contracts/{id} with ``active=true`` to activate it.
"""
effective_from = _localize_effective_from(body.effective_from)
try:
contract = create_contract(
db,
name=body.name,
kind=body.kind,
currency=body.currency,
values=body.values,
effective_from=effective_from,
)
except (ProfileNotFoundError, ProfileValidationError) as exc:
_raise_422_for_profile_error(exc)
db.commit()
db.refresh(contract)
logger.info("Created energy contract id=%d name=%r kind=%s", contract.id, contract.name, contract.kind)
return _contract_detail(db, contract)
# ---------------------------------------------------------------------------
# GET /api/energy/contracts/{id}
# ---------------------------------------------------------------------------
@router.get("/contracts/{contract_id}", response_model=ContractDetailResponse)
def get_energy_contract(
contract_id: int,
db: Session = Depends(get_db),
_auth: AuthenticatedSession = Depends(require_session),
) -> ContractDetailResponse:
"""Return a single energy contract with its full version history.
Versions are ordered by ``effective_from`` ascending so the caller can
easily inspect the pricing timeline.
"""
contract = _get_contract_or_404(db, contract_id)
return _contract_detail(db, contract)
# ---------------------------------------------------------------------------
# PATCH /api/energy/contracts/{id}
# ---------------------------------------------------------------------------
@router.patch("/contracts/{contract_id}", response_model=ContractDetailResponse)
def patch_energy_contract(
contract_id: int,
body: ContractPatch,
db: Session = Depends(get_db),
_auth: AuthenticatedSession = Depends(require_session),
_csrf: None = Depends(require_csrf),
) -> ContractDetailResponse:
"""Partially update a contract: rename or change activation status.
- ``name``: updates the human-readable label.
- ``active=true``: activates this contract (all others are deactivated).
- ``active=false``: deactivates this contract (no effect on others).
At most one contract may be active at any time; the service layer enforces
mutual exclusion.
"""
contract = _get_contract_or_404(db, contract_id)
if body.name is not None:
contract.name = body.name
contract.updated_at = datetime.now(UTC)
if body.active is True:
activate_contract(db, contract)
elif body.active is False:
deactivate_contract(db, contract)
db.commit()
db.refresh(contract)
return _contract_detail(db, contract)
# ---------------------------------------------------------------------------
# POST /api/energy/contracts/{id}/versions
# ---------------------------------------------------------------------------
@router.post(
"/contracts/{contract_id}/versions",
response_model=ContractDetailResponse,
status_code=status.HTTP_201_CREATED,
)
def add_contract_version(
contract_id: int,
body: VersionCreate,
db: Session = Depends(get_db),
_auth: AuthenticatedSession = Depends(require_session),
_csrf: None = Depends(require_csrf),
) -> ContractDetailResponse:
"""Add a new pricing version to an existing contract.
This is how price changes are recorded: the current open version is
automatically closed (its ``effective_to`` is set to ``body.effective_from``)
and a new version is created starting at ``body.effective_from``.
The ``values`` dict must conform to the contract's pricing profile.
Non-conforming values return 422. If ``effective_from`` is not strictly
after the previous version's ``effective_from``, 422 is returned without
writing any rows.
Historical versions are never modified; this endpoint is append-only.
"""
contract = _get_contract_or_404(db, contract_id)
effective_from = _localize_effective_from(body.effective_from)
try:
add_version(
db,
contract,
effective_from=effective_from,
values=body.values,
)
except ContractVersionError as exc:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
detail=str(exc),
)
except (ProfileNotFoundError, ProfileValidationError) as exc:
_raise_422_for_profile_error(exc)
db.commit()
db.refresh(contract)
return _contract_detail(db, contract)
+331
View File
@@ -0,0 +1,331 @@
"""Meter CRUD, swap declaration, and retroactive recompute API (M7-T05).
All endpoints are under /api/energy/meters, require an authenticated session,
and write endpoints (POST/PATCH) additionally require a non-empty X-CSRF-Token
header.
Route semantics
---------------
GET /api/energy/meters — list all meter epochs (ascending started_at)
POST /api/energy/meters — declare a meter swap / initial meter epoch
PATCH /api/energy/meters/{id} — edit label / note, or correct started_at (retroactive)
Retroactive recompute
---------------------
Whenever a write operation changes a meter's ``started_at`` (new declaration
or PATCH correction), the affected billing window is re-judged via
``recompute_range``:
- **POST** (new meter, possibly retroactive):
window = [new_meter.started_at, now)
Rationale: the new meter's ``started_at`` closes the previous meter at that
point; all periods from that boundary forward may have a different meter
attribution. Using ``now`` as the upper bound is safe because
``recompute_range`` only processes closed quarters and the operation is
idempotent.
- **PATCH started_at** (retroactive correction):
window = [min(old_started_at, new_started_at), now)
Rationale: shifting the boundary in either direction affects all periods
between the old and new boundary (and potentially beyond if re-attribution
cascades). Using the minimum of the two timestamps guarantees the entire
affected range is covered; using ``now`` as the upper bound is safe and
idempotent.
``started_at`` localisation (Principle A, FU10 convention)
----------------------------------------------------------
If the client sends a timezone-naive ``started_at`` value, it is interpreted as
the **server's local wall-clock time** and converted to UTC before storage.
Timezone-aware values are converted to UTC as-is. This is identical to the
``_localize_effective_from`` convention used in ``energy_contracts.py``.
"""
from __future__ import annotations
import logging
from datetime import UTC, datetime
from typing import Optional
from fastapi import APIRouter, Depends, HTTPException, status
from sqlalchemy.orm import Session
from app.api.routes.api.deps import require_csrf, require_session
from app.dependencies import get_db
from app.models.energy import Meter
from app.schemas.meter import (
MeterDeclareRequest,
MeterListResponse,
MeterPatchRequest,
MeterResponse,
)
from app.services import timezone as _tz_mod
from app.services.auth import AuthenticatedSession
from app.services.energy_cost import recompute_range
from app.services.meters import (
MeterIntervalError,
MeterOverlapError,
declare_meter,
list_meters,
update_meter,
)
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/energy", tags=["api-energy-meters"])
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _trigger_discovery_republish(session: Session) -> None:
"""Call publish_discovery after a meter write operation (best-effort).
No-op if MQTT / discovery is not enabled or the broker is not connected
(publish_discovery guards internally). All errors are swallowed so that a
discovery failure never breaks the API response.
Must be called **after** db.commit() so that publish_discovery sees the
final committed state of the meter table when it rebuilds the catalog.
"""
try:
from app.services.ha_discovery import publish_discovery
publish_discovery(session)
except Exception:
logger.exception("_trigger_discovery_republish: publish_discovery raised an error")
def _get_meter_or_404(db: Session, meter_id: int) -> Meter:
"""Return the meter with the given id or raise 404."""
meter: Optional[Meter] = db.get(Meter, meter_id)
if meter is None:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"Meter {meter_id!r} not found.",
)
return meter
def _localize_started_at(dt: datetime) -> datetime:
"""Resolve *dt* to an aware UTC datetime for storage.
Follows the same Principle-A convention as ``_localize_effective_from``
in ``energy_contracts.py`` (FU10):
- Timezone-aware → convert to UTC as-is.
- Timezone-naive → interpret as server local wall-clock time, localize
with ``local_tz()``, then convert to UTC.
A front-end sending ``"2026-06-25T00:00:00"`` (no Z) has it interpreted
as local midnight (e.g. CEST = UTC+2 → stored as 2026-06-24T22:00:00Z),
not as UTC midnight.
"""
if dt.tzinfo is not None:
return dt.astimezone(UTC)
tz = _tz_mod.local_tz()
local_dt = dt.replace(tzinfo=tz)
return local_dt.astimezone(UTC)
def _trigger_recompute(db: Session, start: datetime, label: str) -> int:
"""Trigger recompute_range from *start* to now (UTC).
This is the standard "retroactive window" call: everything from the
affected boundary up to the current moment needs re-attribution.
Using ``now`` as the upper bound is safe because ``recompute_range``
only touches closed quarter-hour periods and the operation is idempotent.
"""
end = datetime.now(UTC)
if start >= end:
# started_at is in the future — nothing to recompute.
logger.info("%s: started_at (%s) is in the future, skipping recompute.", label, start)
return 0
n = recompute_range(db, start, end)
logger.info(
"%s: recomputed %d period(s) in window [%s, %s).",
label,
n,
start.isoformat(),
end.isoformat(),
)
return n
# ---------------------------------------------------------------------------
# GET /api/energy/meters
# ---------------------------------------------------------------------------
@router.get("/meters", response_model=MeterListResponse)
def list_energy_meters(
db: Session = Depends(get_db),
_auth: AuthenticatedSession = Depends(require_session),
) -> MeterListResponse:
"""List all meter epochs in ascending ``started_at`` order.
Returns the full historical sequence of meter installations across all
commodities. The active meter (``ended_at=null``) appears last because it
has the latest ``started_at``.
"""
meters = list_meters(db)
items = [MeterResponse.model_validate(m) for m in meters]
return MeterListResponse(items=items, total=len(items))
# ---------------------------------------------------------------------------
# POST /api/energy/meters
# ---------------------------------------------------------------------------
@router.post(
"/meters",
response_model=MeterResponse,
status_code=status.HTTP_201_CREATED,
)
def declare_energy_meter(
body: MeterDeclareRequest,
db: Session = Depends(get_db),
_auth: AuthenticatedSession = Depends(require_session),
_csrf: None = Depends(require_csrf),
) -> MeterResponse:
"""Declare a new meter epoch (swap, home move, or initial declaration).
Closes the current active meter for the given commodity at ``started_at``
and opens a new active meter. If no active meter exists, the new meter is
simply created without closing anything.
**Validation**: ``started_at`` must be **≥** the current active meter's
own ``started_at`` (no chronological backdate below the active epoch's
start). Equal timestamps are allowed (replaces the current meter at the
same logical moment). Violation → 422.
**Retroactive recompute**: if ``started_at`` is in the past, billing
records from that point forward are re-judged via ``recompute_range`` to
reflect the new meter attribution. The recompute is transparent — the
response body is the created meter (``MeterResponse``) only and does **not**
include a recompute count; callers should re-fetch costs if they need the
updated totals.
"""
started_at_utc = _localize_started_at(body.started_at)
try:
new_meter = declare_meter(
db,
label=body.label,
started_at=started_at_utc,
reason=body.reason.value,
commodity=body.commodity,
note=body.note,
)
except MeterOverlapError as exc:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
detail=str(exc),
)
db.flush() # assign PK before recompute (recompute uses session, needs meter in DB)
# Retroactive recompute: re-judge attribution from the new boundary onward.
now = datetime.now(UTC)
if started_at_utc < now:
_trigger_recompute(db, started_at_utc, "POST /api/energy/meters")
db.commit()
db.refresh(new_meter)
# Trigger HA discovery re-publish so the new active meter's energy-cost
# device/sensor configuration is pushed to Home Assistant. Best-effort:
# failures are logged and swallowed; the API response is not affected.
_trigger_discovery_republish(db)
logger.info(
"POST /api/energy/meters: declared %r meter id=%d label=%r started_at=%s",
body.commodity,
new_meter.id,
new_meter.label,
started_at_utc.isoformat(),
)
return MeterResponse.model_validate(new_meter)
# ---------------------------------------------------------------------------
# PATCH /api/energy/meters/{id}
# ---------------------------------------------------------------------------
@router.patch("/meters/{meter_id}", response_model=MeterResponse)
def patch_energy_meter(
meter_id: int,
body: MeterPatchRequest,
db: Session = Depends(get_db),
_auth: AuthenticatedSession = Depends(require_session),
_csrf: None = Depends(require_csrf),
) -> MeterResponse:
"""Partially update a meter epoch: rename, edit note, or correct started_at.
- ``label``: updates the human-readable label.
- ``note``: updates the free-form note.
- ``started_at``: **retroactive correction** — shifts this meter's start
boundary. The service layer maintains timeline continuity by also
updating the preceding meter's ``ended_at``. Validation:
* Must be strictly after the previous meter's own ``started_at``.
* Must be strictly before this meter's ``ended_at`` (if closed).
Violation → 422.
**Retroactive recompute when ``started_at`` changes**: billing records in
the window ``[min(old, new), now)`` are re-judged to reflect the corrected
meter attribution.
Not found → 404.
"""
meter = _get_meter_or_404(db, meter_id)
# Capture old started_at before mutation (needed for recompute window).
old_started_at: Optional[datetime] = meter.started_at
# Localise started_at if provided.
new_started_at_utc: Optional[datetime] = None
if body.started_at is not None:
new_started_at_utc = _localize_started_at(body.started_at)
try:
update_meter(
db,
meter,
label=body.label,
note=body.note,
started_at=new_started_at_utc,
)
except MeterIntervalError as exc:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
detail=str(exc),
)
# Retroactive recompute if started_at was changed.
if new_started_at_utc is not None and old_started_at is not None:
# Normalise old_started_at to UTC-aware for comparison.
if old_started_at.tzinfo is None:
old_started_at = old_started_at.replace(tzinfo=UTC)
# Window = [min(old, new), now) — covers all periods whose attribution
# may have changed due to the boundary shift in either direction.
window_start = min(old_started_at, new_started_at_utc)
_trigger_recompute(db, window_start, f"PATCH /api/energy/meters/{meter_id}")
db.commit()
db.refresh(meter)
# Trigger HA discovery re-publish so label renames on the active meter
# propagate to the HA device name. Best-effort: failures are logged and
# swallowed; the API response is not affected.
_trigger_discovery_republish(db)
logger.info(
"PATCH /api/energy/meters/%d: updated meter label=%r started_at=%s",
meter_id,
meter.label,
meter.started_at,
)
return MeterResponse.model_validate(meter)
+85 -16
View File
@@ -4,14 +4,14 @@ All endpoints are under /api/modbus, require an authenticated session, and
write endpoints (POST/PATCH/DELETE + test-read) additionally require a
non-empty X-CSRF-Token header.
Deletion safety (red-line):
DELETE /api/modbus/devices/{uuid} explicitly queries the application
layer for existing modbus_reading rows before deleting. If any exist
the endpoint returns 409 and suggests disabling the device instead.
We do NOT rely on the SQLite FK RESTRICT constraint because SQLite does
not enforce foreign-key constraints at runtime unless
PRAGMA foreign_keys=ON is set per connection (and it is not in this
project — see M5-T02 review note OBS-1).
Deletion safety:
DELETE /api/modbus/devices/{uuid} queries the application layer for
existing modbus_reading rows before deleting. Without ``cascade`` it
returns 409 and suggests disabling the device instead — a friendly guard
rather than letting the DB raise. With ``cascade=true`` it removes the
readings (and expose toggles) first, then the device. SQLite FK RESTRICT
*is* enforced at runtime here (the app sets PRAGMA foreign_keys=ON; see
app/db.py), so the cascade delete order matters.
"""
from __future__ import annotations
@@ -21,7 +21,7 @@ from datetime import UTC, datetime
from typing import Any
from fastapi import APIRouter, Depends, HTTPException, Query, status
from sqlalchemy import func, select
from sqlalchemy import delete as sa_delete, func, select
from sqlalchemy.orm import Session
from app.api.routes.api.deps import require_csrf, require_session
@@ -34,9 +34,11 @@ from app.integrations.modbus.profiles import (
list_profiles,
load_profile,
)
from app.models.expose import ExposedEntityToggle
from app.models.modbus import ModbusDevice, ModbusReading
from app.schemas.modbus import (
MetricInfo,
ModbusDeleteResponse,
ModbusDeviceCreate,
ModbusDeviceListResponse,
ModbusDeviceResponse,
@@ -220,29 +222,41 @@ def patch_device(
)
def delete_device(
uuid: str,
cascade: bool = Query(default=False),
db: Session = Depends(get_db),
_auth: AuthenticatedSession = Depends(require_session),
_csrf: None = Depends(require_csrf),
) -> None:
) -> None | ModbusDeleteResponse:
"""Delete a Modbus device.
**Default behaviour (cascade=false)**:
Returns 409 Conflict if the device has any associated readings; use
``enabled=false`` to disable it instead.
**Application-layer safety**: the check is performed via an explicit
SELECT COUNT query — not by relying on SQLite's FK RESTRICT constraint,
which is not enforced at runtime in this project (no PRAGMA foreign_keys=ON).
**Cascade delete (cascade=true)**:
Permanently deletes the device together with all its readings and any
``ExposedEntityToggle`` rows whose key matches ``modbus.<uuid>.*``.
Also makes a best-effort attempt to clear the device's HA Discovery
config topics from MQTT (empty retained payload) before the DB rows
are removed. MQTT failures are swallowed — the DB deletion proceeds
regardless.
Returns HTTP 200 with a ``ModbusDeleteResponse`` JSON body on success.
**Application-layer safety**: the 409 guard uses an explicit SELECT COUNT
query to return a friendly message. FK RESTRICT is enforced at runtime
(the app sets ``PRAGMA foreign_keys=ON``), so the cascade path deletes
readings before the device.
"""
from app.services.ha_discovery import clear_device_discovery
device = _get_device_or_404(db, uuid)
# Application-layer guard: refuse deletion if any readings exist.
# We do NOT rely on SQLite FK RESTRICT — it is not enforced at runtime
# without PRAGMA foreign_keys=ON, which is not set in this project.
reading_count: int = db.execute(
select(func.count(ModbusReading.id)).where(ModbusReading.device_id == device.id)
).scalar_one()
if reading_count > 0:
if reading_count > 0 and not cascade:
raise HTTPException(
status_code=status.HTTP_409_CONFLICT,
detail=(
@@ -252,6 +266,60 @@ def delete_device(
),
)
if cascade:
# --- Cascade deletion path ---
logger.info(
"Cascade-deleting Modbus device %r (uuid=%s): %d reading(s)",
device.friendly_name,
uuid,
reading_count,
)
# 1. Best-effort HA Discovery cleanup (before DB rows are gone).
try:
clear_device_discovery(db, uuid)
except Exception: # noqa: BLE001
logger.exception(
"delete_device(cascade): HA discovery cleanup raised for uuid=%s; continuing",
uuid,
)
# 2. Delete all readings for this device (must happen before device row).
deleted_readings_result = db.execute(
sa_delete(ModbusReading).where(ModbusReading.device_id == device.id)
)
readings_deleted: int = deleted_readings_result.rowcount
# 3. Delete all ExposedEntityToggle rows for this device.
# Keys follow the pattern "modbus.<uuid>.<metric>".
toggle_prefix = f"modbus.{uuid}.%"
deleted_toggles_result = db.execute(
sa_delete(ExposedEntityToggle).where(ExposedEntityToggle.key.like(toggle_prefix))
)
toggles_deleted: int = deleted_toggles_result.rowcount
# 4. Delete the device itself.
db.delete(device)
db.commit()
logger.info(
"Cascade-delete complete for device uuid=%s: %d reading(s), %d toggle(s) removed",
uuid,
readings_deleted,
toggles_deleted,
)
from fastapi.responses import JSONResponse
return JSONResponse(
status_code=status.HTTP_200_OK,
content={
"deleted": True,
"readings_deleted": readings_deleted,
"toggles_deleted": toggles_deleted,
},
)
# --- Non-cascade path (no readings exist at this point) ---
logger.info("Deleting Modbus device %r (uuid=%s)", device.friendly_name, uuid)
db.delete(device)
db.commit()
@@ -420,6 +488,7 @@ def test_read(
device.port,
device.unit_id,
[{"start": b.start, "count": b.count} for b in profile.blocks],
function_code=profile.function_code,
)
payload: dict[str, Any] = decode_profile(profile, registers)
return ModbusTestReadResponse(ok=True, payload=payload)
+16
View File
@@ -57,6 +57,22 @@ class Settings(BaseSettings):
# Home Assistant MQTT Discovery (T08 wires into CONFIG_FIELDS/UI; T11 does publishing).
ha_discovery_enabled: bool = False
ha_discovery_prefix: str = "homeassistant"
# State/availability topics use a separate prefix so they live outside the HA discovery
# namespace. HA still receives state via the state_topic declared in the discovery config.
ha_state_topic_prefix: str = "home_automation"
# DSMR smart-meter ingest via MQTT (M6; default off — opt-in).
# Subscribe to dsmr_mqtt_topic when dsmr_ingest_enabled=True.
dsmr_ingest_enabled: bool = False
dsmr_mqtt_topic: str = "dsmr/json"
dsmr_sample_interval_s: int = 10
# DSMR dual-tariff topic: publishes "1" (dal/off-peak) or "2" (normal/peak).
# Empty string = do not subscribe (tariff-aware pricing disabled).
dsmr_tariff_topic: str = "dsmr/meter-stats/electricity_tariff"
# Tibber dynamic pricing credentials (M6; only used when active contract kind=tibber).
tibber_api_token: str = ""
tibber_home_id: str = ""
model_config = SettingsConfigDict(
env_file=".env",
+529
View File
@@ -27,6 +27,7 @@ from __future__ import annotations
import logging
from dataclasses import dataclass, field
from datetime import timedelta
from typing import Any, Callable, Optional, Protocol
from sqlalchemy.orm import Session
@@ -58,6 +59,21 @@ class DeviceInfo:
name: str
"""Human-readable device name (may change; triggers HA discovery re-publish)."""
provides_availability: bool = True
"""Whether this device publishes an availability ("online"/"offline") heartbeat.
When True (the default — e.g. Modbus devices, which have an ``online``
binary_sensor driving availability), the HA Discovery config for each entity
declares an ``availability`` topic and HA marks the entity unavailable until
"online" is published.
When False (e.g. the energy-cost device, which has only sensors and no
heartbeat source), the config omits ``availability`` so HA treats the entity
as *always available* and shows its state as soon as one arrives. Without
this, such entities would stay perpetually ``unavailable`` in HA even though
their state is being published.
"""
@dataclass
class ExposableEntity:
@@ -348,3 +364,516 @@ def _modbus_provider(session: Session) -> list[ExposableEntity]:
# Register the modbus provider at module load time.
register_provider(_modbus_provider)
# ---------------------------------------------------------------------------
# Energy Cost provider
# ---------------------------------------------------------------------------
# *_today 当日窗口翻天的宽限:本地午夜后 5 秒才切到新的一天,避免在 00:00:0x 把
# 归零后的值发出去、被慢几秒的 HA 钟记成前一天的 23:59:59(归错小时桶)。
_TODAY_RESET_GRACE = timedelta(seconds=5)
def _energy_cost_provider(session: Session) -> list[ExposableEntity]:
"""Enumerate ExposableEntity objects for the energy cost subsystem.
Produces 6 sensor entities grouped under a single HA device whose identity
is anchored to the **current active electricity meter**:
- ``buy_price_now`` — current effective buy price (EUR/kWh or local currency).
- ``sell_price_now`` — current effective sell price (EUR/kWh or local currency).
- ``import_cost_total`` — cumulative import cost (total, monetary).
- ``export_revenue_total`` — cumulative export revenue (total, monetary).
- ``import_cost_today`` — today's import cost (total_increasing, monetary).
- ``export_revenue_today`` — today's export revenue (total_increasing, monetary).
Active meter requirement
------------------------
**If no active electricity meter exists, the provider returns ``[]``.**
No energy-cost entities are exposed to HA until a meter has been declared.
This prevents spurious sensor creation with an undefined device identity.
HA device identity (换表 → 新 sensor)
--------------------------------------
``identifiers[1]`` is set to the active meter's **uuid** (not the fixed
string ``"energy-cost"``). ``ha_discovery.py`` uses ``identifiers[1]`` as
the MQTT node_id and as part of the ``unique_id`` for every entity.
Declaring a new active electricity meter produces a new uuid → new node_id /
unique_id → HA creates a brand-new sensor, cleanly isolating post-swap data.
Entity key stability
--------------------
Entity keys remain the fixed stable strings (``"energy.buy_price_now"`` etc.),
**not** derived from the meter uuid. The ``exposed_entity_toggle`` table uses
keys as its primary handle; keeping them stable means toggled-on entities stay
enabled after a meter swap without requiring the user to re-tick them.
Current-price algorithm (source-agnostic, with fallback)
---------------------------------------------------------
Strategy A: read the ``pricing`` snapshot from the most recent non-degraded
``energy_cost_period`` row.
- For ``kind="tibber"``: the snapshot contains ``"buy"`` and ``"sell"`` keys
(per-unit prices in the contract currency).
- For ``kind="manual"``: ``"buy_normal"`` and ``"sell_normal"`` are used as
representative effective per-unit prices. (Both tariff-slot prices differ
only by the base rate; energy_tax and ODE are the same for both, so
buy_normal is the higher/conservative single representative value.)
When no non-degraded period exists or the pricing snapshot lacks the
expected keys, ``value_getter`` returns ``None`` (``publish_states``
skips None values automatically).
Cumulative totals
-----------------
``SUM(import_cost)`` and ``SUM(export_revenue)`` over **all non-degraded**
``energy_cost_period`` rows within the current meter's window. Degraded rows
carry 0 costs and are excluded to avoid double-counting when they are later
overwritten by real values.
Currency
--------
Taken from the most recent non-degraded period's ``currency`` column.
Falls back to ``"EUR"`` when no such row exists.
Key convention
--------------
Fixed stable string keys (not derived from any mutable field or DB id):
- ``"energy.buy_price_now"``
- ``"energy.sell_price_now"``
- ``"energy.import_cost_total"``
- ``"energy.export_revenue_total"``
- ``"energy.import_cost_today"``
- ``"energy.export_revenue_today"``
DeviceInfo identifiers
----------------------
**Two-element tuple** ``("energy-cost", meter.uuid)`` so that
``ha_discovery.py``'s ``entity.device.identifiers[1]`` resolves to the
meter uuid (used as the MQTT node_id and unique_id seed throughout).
"""
from app.models.energy import EnergyCostPeriod, Meter # local import to avoid circular
from sqlalchemy import select
# --- Require an active electricity meter; return [] if none exists ---
# Using an inline query (ended_at IS NULL) rather than a service-layer helper
# to avoid a new public dependency and remain consistent with the value_getter
# implementations below (which use the same inline pattern).
active_meter: Meter | None = session.execute(
select(Meter)
.where(
Meter.commodity == "electricity",
Meter.ended_at.is_(None),
)
.limit(1)
).scalar_one_or_none()
if active_meter is None:
# No active electricity meter → do not expose any energy-cost entities.
# HA will not see these sensors until a meter is declared.
return []
# --- Determine currency from the latest non-degraded row ---
latest_period: EnergyCostPeriod | None = (
session.query(EnergyCostPeriod)
.filter(EnergyCostPeriod.degraded.is_(False))
.order_by(EnergyCostPeriod.period_start.desc())
.first()
)
currency: str = "EUR"
if latest_period is not None and latest_period.currency:
currency = latest_period.currency
# --- Shared DeviceInfo anchored to the active meter's uuid ---
# identifiers[1] = meter.uuid drives the MQTT node_id and unique_id in
# ha_discovery.py. Swapping the meter produces a new uuid → new HA sensor.
# provides_availability=False: the energy-cost device has only sensors and no
# online/offline heartbeat, so its entities must be "always available" in HA.
# (Otherwise HA shows them unavailable despite state being published.)
device_info = DeviceInfo(
identifiers=("energy-cost", active_meter.uuid),
name=active_meter.label,
provides_availability=False,
)
# --- value_getter: current buy price ---
def _make_buy_price_getter() -> Callable[["Session"], Any]:
"""Return a getter for the current buy price per kWh.
For ``kind="manual"`` contracts the getter selects the price slot that
matches the current DSMR electricity tariff:
- tariff == 1 (dal / off-peak) → ``buy_dal``
- tariff == 2 (normal / peak) → ``buy_normal``
- tariff is None / unknown → ``buy_normal`` (safe fallback = current status quo)
For ``kind="tibber"`` (hourly dynamic pricing) the tariff slot is not
applicable; the getter always uses the ``buy`` key from the snapshot.
"""
def _getter(sess: "Session") -> Any:
from app.models.energy import EnergyCostPeriod as _ECP
from app.services.dsmr_ingest import get_current_tariff
period = (
sess.query(_ECP)
.filter(_ECP.degraded.is_(False))
.order_by(_ECP.period_start.desc())
.first()
)
if period is None or not period.pricing:
return None
snap = period.pricing
kind = snap.get("kind")
if kind == "tibber":
raw = snap.get("buy")
elif kind == "manual":
tariff = get_current_tariff()
if tariff == 1:
raw = snap.get("buy_dal")
else:
# tariff == 2, None, or any unexpected value → normal (peak) rate.
raw = snap.get("buy_normal")
else:
# Unknown kind — attempt common keys gracefully.
raw = snap.get("buy") or snap.get("buy_normal")
if raw is None:
return None
try:
return float(raw)
except (TypeError, ValueError):
return None
return _getter
# --- value_getter: current sell price ---
def _make_sell_price_getter() -> Callable[["Session"], Any]:
"""Return a getter for the current sell price per kWh.
For ``kind="manual"`` contracts the getter selects the price slot that
matches the current DSMR electricity tariff:
- tariff == 1 (dal / off-peak) → ``sell_dal``
- tariff == 2 (normal / peak) → ``sell_normal``
- tariff is None / unknown → ``sell_normal`` (safe fallback = current status quo)
For ``kind="tibber"`` the tariff slot is not applicable; always uses ``sell``.
"""
def _getter(sess: "Session") -> Any:
from app.models.energy import EnergyCostPeriod as _ECP
from app.services.dsmr_ingest import get_current_tariff
period = (
sess.query(_ECP)
.filter(_ECP.degraded.is_(False))
.order_by(_ECP.period_start.desc())
.first()
)
if period is None or not period.pricing:
return None
snap = period.pricing
kind = snap.get("kind")
if kind == "tibber":
raw = snap.get("sell")
elif kind == "manual":
tariff = get_current_tariff()
if tariff == 1:
raw = snap.get("sell_dal")
else:
# tariff == 2, None, or any unexpected value → normal (peak) rate.
raw = snap.get("sell_normal")
else:
raw = snap.get("sell") or snap.get("sell_normal")
if raw is None:
return None
try:
return float(raw)
except (TypeError, ValueError):
return None
return _getter
# --- value_getter: cumulative import cost (incl. standing charges) ---
def _make_import_cost_getter() -> Callable[["Session"], Any]:
"""Return a getter for the cumulative import cost plus prorated standing charges.
D2 (M7): anchor = current active electricity meter's ``started_at``.
After a meter swap the cumulative resets to zero for the new meter —
old-meter periods have ``period_start < new_meter.started_at`` and fall
outside the [anchor, now) window, so they are naturally excluded.
Cross-meter boundary periods are already degraded and also excluded.
No active electricity meter → returns None (cannot anchor; safer than
returning a stale or wrong value). The check is done via an inline
query (``ended_at IS NULL``) rather than ``meter_at(now)`` to be robust
against the edge case where the active meter's ``started_at`` is in the
future (``meter_at(now)`` would return None in that scenario).
No non-degraded periods at all → returns None (has_data guard).
No active contract → ``summarize`` still runs but fixed_costs = 0;
the return value is the pure metered sum within the current meter window.
This is consistent with D2: the anchor is the meter, not the contract.
value = summarize(anchor → now).metered_import + summarize(anchor → now).fixed_costs
"""
def _getter(sess: "Session") -> Any:
from datetime import UTC, datetime as _dt
from app.models.energy import EnergyCostPeriod as _ECP, Meter as _Meter
from app.services.energy_cost import summarize as _summarize
from sqlalchemy import func as _func, select as _select
# Quick check: any non-degraded period at all?
has_data = sess.query(_func.sum(_ECP.import_cost)).filter(
_ECP.degraded.is_(False)
).scalar()
if has_data is None:
return None
# D2: anchor = active electricity meter's started_at.
# Inline query (ended_at IS NULL) is more robust than meter_at(now)
# because it avoids the edge case where started_at is in the future.
active_meter = sess.execute(
_select(_Meter)
.where(
_Meter.commodity == "electricity",
_Meter.ended_at.is_(None),
)
.limit(1)
).scalar_one_or_none()
if active_meter is None:
# No active electricity meter — cannot anchor; return None.
return None
from app.services.contracts import _as_utc as _cu
anchor_utc = _cu(active_meter.started_at)
now_utc = _dt.now(UTC)
result = _summarize(sess, anchor_utc, now_utc)
return result["metered_import"] + result["fixed_costs"]
return _getter
# --- value_getter: cumulative export revenue (incl. tax credit) ---
def _make_export_revenue_getter() -> Callable[["Session"], Any]:
"""Return a getter for the cumulative export revenue plus prorated tax credit.
D2 (M7): anchor = current active electricity meter's ``started_at``.
Same reasoning as the import cost getter — see its docstring.
value = summarize(anchor → now).metered_export + summarize(anchor → now).credits
Returns None when no non-degraded period exists or no active electricity meter.
"""
def _getter(sess: "Session") -> Any:
from datetime import UTC, datetime as _dt
from app.models.energy import EnergyCostPeriod as _ECP, Meter as _Meter
from app.services.energy_cost import summarize as _summarize
from sqlalchemy import func as _func, select as _select
# Quick check: any non-degraded period at all?
has_data = sess.query(_func.sum(_ECP.export_revenue)).filter(
_ECP.degraded.is_(False)
).scalar()
if has_data is None:
return None
# D2: anchor = active electricity meter's started_at.
active_meter = sess.execute(
_select(_Meter)
.where(
_Meter.commodity == "electricity",
_Meter.ended_at.is_(None),
)
.limit(1)
).scalar_one_or_none()
if active_meter is None:
# No active electricity meter — cannot anchor; return None.
return None
from app.services.contracts import _as_utc as _cu
anchor_utc = _cu(active_meter.started_at)
now_utc = _dt.now(UTC)
result = _summarize(sess, anchor_utc, now_utc)
return result["metered_export"] + result["credits"]
return _getter
# --- value_getter: today's import cost (local-day window, resets at local midnight) ---
def _make_import_cost_today_getter() -> Callable[["Session"], Any]:
"""Return a getter for today's import cost (metered + standing for today).
Window: [local today 00:00, local tomorrow 00:00) in UTC.
After local midnight the window rolls to the new day → value resets to that
day's running cost, implementing "resets at local midnight" semantics for HA.
Uses timezone module via module-attribute access to preserve monkeypatch safety.
Returns None when no non-degraded period exists or no active contract.
"""
def _getter(sess: "Session") -> Any:
from app.services.contracts import active_contract_versions
from app.models.energy import EnergyCostPeriod as _ECP
from app.services.energy_cost import summarize as _summarize
from app.services import timezone as _tz_mod
from sqlalchemy import func as _func
from datetime import timedelta as _td
# Quick check: any non-degraded period at all?
has_data = sess.query(_func.sum(_ECP.import_cost)).filter(
_ECP.degraded.is_(False)
).scalar()
if has_data is None:
return None
versions = active_contract_versions(sess)
if not versions:
return None
# Today's window in UTC, using monkeypatch-safe module attribute calls.
# Grace: subtract _TODAY_RESET_GRACE so that in the first 5 seconds after
# local midnight the getter still returns yesterday's window. This prevents
# a "归零后的值" from being published while HA's clock (which may lag a few
# seconds) would stamp it as 23:59:59 of the previous day.
today_local = (_tz_mod.local_now() - _TODAY_RESET_GRACE).date()
tomorrow_local = today_local + _td(days=1)
today_start_utc = _tz_mod.local_midnight_utc(today_local)
tomorrow_start_utc = _tz_mod.local_midnight_utc(tomorrow_local)
result = _summarize(sess, today_start_utc, tomorrow_start_utc)
return result["metered_import"] + result["fixed_costs"]
return _getter
# --- value_getter: today's export revenue (local-day window, resets at local midnight) ---
def _make_export_revenue_today_getter() -> Callable[["Session"], Any]:
"""Return a getter for today's export revenue (metered + tax credit for today).
Same window semantics as import_cost_today.
Returns None when no non-degraded period exists or no active contract.
"""
def _getter(sess: "Session") -> Any:
from app.services.contracts import active_contract_versions
from app.models.energy import EnergyCostPeriod as _ECP
from app.services.energy_cost import summarize as _summarize
from app.services import timezone as _tz_mod
from sqlalchemy import func as _func
from datetime import timedelta as _td
# Quick check: any non-degraded period at all?
has_data = sess.query(_func.sum(_ECP.export_revenue)).filter(
_ECP.degraded.is_(False)
).scalar()
if has_data is None:
return None
versions = active_contract_versions(sess)
if not versions:
return None
# Grace: same logic as import_cost_today — see that getter's comment.
today_local = (_tz_mod.local_now() - _TODAY_RESET_GRACE).date()
tomorrow_local = today_local + _td(days=1)
today_start_utc = _tz_mod.local_midnight_utc(today_local)
tomorrow_start_utc = _tz_mod.local_midnight_utc(tomorrow_local)
result = _summarize(sess, today_start_utc, tomorrow_start_utc)
return result["metered_export"] + result["credits"]
return _getter
# Price unit string: "<currency>/kWh"
price_unit = f"{currency}/kWh"
entities: list[ExposableEntity] = [
ExposableEntity(
key="energy.buy_price_now",
component="sensor",
device=device_info,
device_class=None,
unit=price_unit,
name="Energy Buy Price Now",
value_getter=_make_buy_price_getter(),
state_class="measurement",
),
ExposableEntity(
key="energy.sell_price_now",
component="sensor",
device=device_info,
device_class=None,
unit=price_unit,
name="Energy Sell Price Now",
value_getter=_make_sell_price_getter(),
state_class="measurement",
),
ExposableEntity(
key="energy.import_cost_total",
component="sensor",
device=device_info,
device_class="monetary",
unit=currency,
name="Energy Import Cost (incl. standing)",
value_getter=_make_import_cost_getter(),
state_class="total",
),
ExposableEntity(
key="energy.export_revenue_total",
component="sensor",
device=device_info,
device_class="monetary",
unit=currency,
name="Energy Export Revenue (incl. tax credit)",
value_getter=_make_export_revenue_getter(),
state_class="total",
),
# Daily entities: reset at local midnight, state_class=total_increasing.
#
# device_class=monetary is correct here. The HA developer docs (long-term
# statistics section) prohibit MONETARY from being combined with
# state_class=measurement — they do NOT prohibit it with total or
# total_increasing. monetary+total_increasing is valid (the cumulative
# entities above use monetary+total with no issue).
#
# total_increasing is appropriate because these daily quantities are
# non-negative (import/export metered cost ≥ 0, fixed standing fee/tax
# credit ≥ 0) and monotonically non-decreasing within any given local day.
# When the local midnight rolls over the value drops back to ~1 day's fixed
# fee; HA interprets that decrease as a new cycle, giving correct daily
# aggregation without any explicit last_reset pipeline.
ExposableEntity(
key="energy.import_cost_today",
component="sensor",
device=device_info,
device_class="monetary",
unit=currency,
name="Energy Import Cost Today (incl. standing)",
value_getter=_make_import_cost_today_getter(),
state_class="total_increasing",
),
ExposableEntity(
key="energy.export_revenue_today",
component="sensor",
device=device_info,
device_class="monetary",
unit=currency,
name="Energy Export Revenue Today (incl. tax credit)",
value_getter=_make_export_revenue_today_getter(),
state_class="total_increasing",
),
]
return entities
# Register the energy cost provider at module load time.
register_provider(_energy_cost_provider)
+32 -5
View File
@@ -1,8 +1,11 @@
"""Modbus TCP driver — thin wrapper around pymodbus.
This module provides a single public function ``read_blocks`` that performs
one or more FC04 (Read Input Registers) block reads against a Modbus TCP
gateway and returns a flat ``dict[register_address -> 16-bit_value]`` map.
one or more block reads against a Modbus TCP gateway — using either FC04
(Read Input Registers) or FC03 (Read Holding Registers), selected per call
via the ``function_code`` argument — and returns a flat
``dict[register_address -> 16-bit_value]`` map. The function code comes from
the device profile (e.g. SDM120 uses FC04, DDSU666 uses FC03).
Design decisions
----------------
@@ -80,9 +83,10 @@ def read_blocks(
unit_id: int,
blocks: Sequence[Block],
*,
function_code: int = 4,
timeout: float = 3.0,
) -> dict[int, int]:
"""Read one or more contiguous register blocks via FC04 (input registers).
"""Read one or more contiguous register blocks via FC03 or FC04.
Parameters
----------
@@ -96,6 +100,11 @@ def read_blocks(
Sequence of ``{"start": int, "count": int}`` dicts describing the
contiguous register ranges to read. ``count`` is the number of
16-bit registers (not bytes).
function_code:
Modbus read function code: ``4`` for FC04 (Read Input Registers,
default — SDM120) or ``3`` for FC03 (Read Holding Registers —
DDSU666 and other devices that expose measurements as holding
registers). Comes from the device profile's ``function_code`` field.
timeout:
TCP connect/read timeout in seconds (default 3 s).
@@ -107,12 +116,21 @@ def read_blocks(
Raises
------
ModbusDriverError
If ``function_code`` is neither 3 nor 4 (validated before any
connection is attempted).
ModbusConnectionError
If the TCP connection to the gateway fails.
ModbusResponseError
If the gateway returns a Modbus exception frame or an unexpected
number of registers.
"""
if function_code not in (3, 4):
raise ModbusDriverError(
f"Unsupported read function code FC{function_code:02d} "
f"(only FC03 holding-register and FC04 input-register reads are supported)"
)
client = ModbusTcpClient(host, port=port, timeout=timeout)
try:
connected = client.connect()
@@ -125,7 +143,7 @@ def read_blocks(
for block in blocks:
start: int = block["start"]
count: int = block["count"]
_read_block(client, unit_id, start, count, registers)
_read_block(client, unit_id, start, count, registers, function_code=function_code)
return registers
@@ -145,9 +163,18 @@ def _read_block(
start: int,
count: int,
result: dict[int, int],
*,
function_code: int,
) -> None:
"""Read one block and merge into *result*. Raises on any error."""
"""Read one block and merge into *result*. Raises on any error.
``function_code`` is assumed already validated to be 3 or 4 by the caller
(``read_blocks``); 3 dispatches FC03 (holding) and 4 dispatches FC04 (input).
"""
try:
if function_code == 3:
response = client.read_holding_registers(start, count=count, device_id=unit_id)
else: # function_code == 4 (input registers)
response = client.read_input_registers(start, count=count, device_id=unit_id)
except ConnectionException as exc:
raise ModbusConnectionError(
@@ -0,0 +1,82 @@
name: ddsu666
description: CHINT DDSU666 single-phase smart meter
function_code: 3 # holding registers (FC03) — DDSU666 has NO input registers (no FC04)
word_order: big # high register first — ASSUMED; verify with a known voltage reading
byte_order: big # high byte first within each register (confirmed by manual CRC example)
# NOTE: the manual gives no worked float-decode example, so word_order is a best-guess
# (standard big-endian, high register first, matching sdm120). After wiring the meter,
# read 0x2000 (voltage) — it should decode to ~230 V. If it decodes to garbage, the
# device uses the opposite word order and this profile (and the decoder) need adjusting.
# Byte order is confirmed big-endian from the manual (Appendix A, Table A.4: 0x1388 -> 13 88).
blocks:
# Instantaneous quantities 0x20000x200F: voltage, current, P, Q, (rsv), PF, (rsv), Freq.
# 16 contiguous registers — single bulk read. (DDSU666 manual Table 9.)
- { start: 0x2000, count: 0x0010 }
# Active energy — import (0x4000) and export (0x400A) read as two small blocks rather
# than one span, to avoid touching the undocumented/reserved 0x40020x4009 gap.
- { start: 0x4000, count: 0x0002 }
- { start: 0x400A, count: 0x0002 }
metrics:
# Addresses are the raw Modbus protocol addresses (hex) from DDSU666 manual Table 9,
# read via FC03. Each float32 occupies two consecutive 16-bit registers.
- key: voltage
address: 0x2000 # U — Voltage (V)
type: float32
unit: "V"
device_class: voltage
ha_component: sensor
- key: current
address: 0x2002 # I — Current (A)
type: float32
unit: "A"
device_class: current
ha_component: sensor
- key: active_power
address: 0x2004 # P — Active power. Manual unit is kW (NOT W like sdm120).
type: float32
unit: "kW"
device_class: power
ha_component: sensor
- key: reactive_power
address: 0x2006 # Q — Reactive power (kvar)
type: float32
unit: "kvar"
device_class: reactive_power
ha_component: sensor
- key: power_factor
address: 0x200A # PF — Power factor (dimensionless)
type: float32
unit: ""
device_class: power_factor
ha_component: sensor
- key: frequency
address: 0x200E # Freq — Frequency (Hz)
type: float32
unit: "Hz"
device_class: frequency
ha_component: sensor
- key: import_energy
address: 0x4000 # Ep — positive/forward active energy (kWh)
type: float32
unit: "kWh"
device_class: energy
state_class: total_increasing
ha_component: sensor
- key: export_energy
address: 0x400A # -Ep — reverse active energy (kWh)
type: float32
unit: "kWh"
device_class: energy
state_class: total_increasing
ha_component: sensor
+84
View File
@@ -18,12 +18,18 @@ Key design decisions
- **Reconnect**: ``reconnect(settings)`` tears down the old client and
establishes a fresh connection with the new settings. Callers (e.g. PUT
/api/config) call this after saving MQTT-related config values.
- **Subscribe**: ``subscribe(topic, handler)`` registers a topic → handler
mapping. Subscriptions are re-established automatically on reconnect.
The ``on_message`` callback dispatches incoming payloads to the registered
handler; handler exceptions are caught and logged so they never crash the
paho loop thread or the network connection.
"""
from __future__ import annotations
import logging
import threading
from collections.abc import Callable
from typing import TYPE_CHECKING
import paho.mqtt.client as mqtt
@@ -68,6 +74,9 @@ class MqttManager:
self._client: mqtt.Client | None = None
self._lock = threading.Lock()
self._connected = False
# topic → handler registry; persists across reconnects so subscriptions
# are automatically re-established when the client reconnects.
self._subscriptions: dict[str, Callable[[bytes], None]] = {}
# ------------------------------------------------------------------
# Public properties
@@ -142,6 +151,51 @@ class MqttManager:
except Exception:
logger.exception("MQTT publish error (topic=%s).", topic)
def subscribe(self, topic: str, handler: Callable[[bytes], None]) -> None:
"""Register *handler* to be called when a message arrives on *topic*.
If the client is already connected, the subscription is sent to the
broker immediately. Otherwise it is queued and will be established
the next time ``_on_connect`` fires (including after a reconnect).
Handler exceptions are swallowed by the ``on_message`` dispatcher so
that a buggy handler can never crash the paho loop thread.
"""
self._subscriptions[topic] = handler
# Grab the client reference outside the lock to avoid holding the lock
# while calling back into paho (which may itself acquire internal locks).
with self._lock:
client = self._client
connected = self._connected
if client is not None and connected:
try:
client.subscribe(topic)
logger.debug("MQTT subscribed to topic=%s (immediate).", topic)
except Exception:
logger.exception("MQTT subscribe error (topic=%s).", topic)
def unsubscribe(self, topic: str) -> None:
"""Remove the handler for *topic* and unsubscribe from the broker.
Idempotent: unknown topics are ignored. Used to apply config changes
without a restart (e.g. when DSMR ingest is turned off or its topic
changes). If the client is connected, the broker unsubscribe is sent
immediately; either way the handler is removed from the registry so it
will not be re-subscribed on the next ``_on_connect``.
"""
self._subscriptions.pop(topic, None)
with self._lock:
client = self._client
connected = self._connected
if client is not None and connected:
try:
client.unsubscribe(topic)
logger.debug("MQTT unsubscribed from topic=%s.", topic)
except Exception:
logger.exception("MQTT unsubscribe error (topic=%s).", topic)
# ------------------------------------------------------------------
# Internal helpers — must be called with self._lock held
# ------------------------------------------------------------------
@@ -168,6 +222,15 @@ class MqttManager:
else:
logger.info("MQTT connected (reason_code=%s).", reason_code)
self._connected = True
# Re-establish all registered subscriptions after (re-)connect.
for sub_topic in self._subscriptions:
try:
_client.subscribe(sub_topic)
logger.debug("MQTT re-subscribed to topic=%s.", sub_topic)
except Exception:
logger.exception(
"MQTT re-subscribe failed (topic=%s).", sub_topic
)
# VERSION2 on_disconnect signature:
# (client, userdata, disconnect_flags, reason_code, properties)
@@ -184,8 +247,29 @@ class MqttManager:
else:
logger.info("MQTT disconnected cleanly.")
# VERSION2 on_message signature: (client, userdata, message)
def _on_message(
_client: mqtt.Client,
_userdata: object,
message: mqtt.MQTTMessage,
) -> None:
handler = self._subscriptions.get(message.topic)
if handler is None:
logger.debug(
"MQTT on_message: no handler for topic=%s.", message.topic
)
return
try:
handler(message.payload)
except Exception:
logger.exception(
"MQTT message handler raised for topic=%s (swallowed).",
message.topic,
)
client.on_connect = _on_connect
client.on_disconnect = _on_disconnect
client.on_message = _on_message
# TLS
if settings.mqtt_tls_enabled:
+11
View File
@@ -0,0 +1,11 @@
"""Pricing profile framework and strategy registry.
This package provides:
- ``profiles``: Pydantic models describing the structure of pricing profiles
(``ManualProfile`` / ``TibberProfile``), YAML loaders, and contract-values
validation (``load_profile`` / ``list_profiles`` / ``validate_values``).
- ``strategies``: A lightweight registry of price-calculation strategies
(``register_strategy`` / ``get_strategy``), with built-in implementations for
the ``manual`` (fixed dual-tariff) and ``tibber`` (dynamic API) kinds.
"""
+393
View File
@@ -0,0 +1,393 @@
"""Pricing profile loader, validator, and contract-values checker.
A *pricing profile* is a YAML file that describes the **structure** of an energy
contract — which fields exist, their units, and which have defaults. Actual
pricing values (the numbers the user fills in via the UI) are stored in the DB
as ``EnergyContractVersion.values`` (a JSON blob) and must conform to the
corresponding profile's structure.
Profiles live in ``app/integrations/pricing/profiles/<kind>.yaml`` and are
located at runtime relative to *this file* (not CWD), matching the pattern
established by the Modbus profile loader.
Design notes
------------
- **Purely data + functions** — no abstract base classes or inheritance.
- Two Pydantic models — ``ManualProfile`` and ``TibberProfile`` — capture the
different structures of the two supported kinds. A thin union dispatcher in
``load_profile`` picks the right one.
- ``validate_values(kind, values)`` fills in fields that carry a ``default`` and
raises ``ProfileValidationError`` for missing required fields or wrong types.
- All errors are subclasses of built-ins so callers need not import this module
just to catch them.
"""
from __future__ import annotations
import logging
from pathlib import Path
from typing import Any, Optional
import yaml
from pydantic import BaseModel, ValidationError, model_validator
logger = logging.getLogger(__name__)
# Directory containing the YAML profiles, located relative to *this* file.
_PROFILES_DIR = Path(__file__).parent / "profiles"
# ---------------------------------------------------------------------------
# Custom exceptions
# ---------------------------------------------------------------------------
class ProfileNotFoundError(FileNotFoundError):
"""Raised when the requested profile YAML file does not exist."""
class ProfileValidationError(ValueError):
"""Raised when a profile YAML fails Pydantic validation or when contract
values do not conform to the profile structure."""
# ---------------------------------------------------------------------------
# Leaf-node models (a single pricing field: unit + optional default)
# ---------------------------------------------------------------------------
class FieldSpec(BaseModel):
"""Specification for a single numeric pricing field."""
unit: str
"""Physical / monetary unit string (e.g. ``"EUR/kWh"``, ``"EUR/month"``)."""
default: Optional[float] = None
"""Default value used when the field is absent from contract values.
``None`` means the field is required (no default)."""
# ---------------------------------------------------------------------------
# ManualProfile — fixed / variable dual-tariff contract structure
# ---------------------------------------------------------------------------
class ManualBuySpec(BaseModel):
normal: FieldSpec # high-tariff buy price (delivered_2 registers)
dal: FieldSpec # low-tariff buy price (delivered_1 registers)
class ManualSellSpec(BaseModel):
normal: FieldSpec # high-tariff sell / return price
dal: FieldSpec # low-tariff sell / return price
class ManualEnergySpec(BaseModel):
dual_tariff: bool # always True for manual profiles
buy: ManualBuySpec
sell: ManualSellSpec
energy_tax: FieldSpec # added to buy price; includes VAT
ode: FieldSpec # currently merged into energy_tax; default 0
class ManualStandingSpec(BaseModel):
network_fee: FieldSpec # EUR/month
management_fee: FieldSpec # EUR/month
class ManualCreditsSpec(BaseModel):
heffingskorting: FieldSpec # EUR/year — energy-tax credit deducted at summary
class ManualProfile(BaseModel):
"""Complete structure description for a ``manual`` pricing contract."""
kind: str
label: str
energy: ManualEnergySpec
standing: ManualStandingSpec
credits: ManualCreditsSpec
@model_validator(mode="after")
def _check_kind(self) -> "ManualProfile":
if self.kind != "manual":
raise ValueError(f"ManualProfile requires kind='manual', got {self.kind!r}")
return self
# ---------------------------------------------------------------------------
# TibberProfile — dynamic Tibber API contract structure
# ---------------------------------------------------------------------------
class TibberEnergySpec(BaseModel):
source: str # must be "tibber_api"
energy_tax: FieldSpec # subtracted from total to derive sell price
sell_adjust: FieldSpec # additional sell-price adjustment; default 0
class TibberStandingSpec(BaseModel):
management_fee: FieldSpec # EUR/month; has a default
network_fee: FieldSpec # EUR/month
class TibberCreditsSpec(BaseModel):
heffingskorting: FieldSpec # EUR/year
class TibberProfile(BaseModel):
"""Complete structure description for a ``tibber`` pricing contract."""
kind: str
label: str
energy: TibberEnergySpec
standing: TibberStandingSpec
credits: TibberCreditsSpec
@model_validator(mode="after")
def _check_kind(self) -> "TibberProfile":
if self.kind != "tibber":
raise ValueError(f"TibberProfile requires kind='tibber', got {self.kind!r}")
return self
# A union type for type hints where either profile is acceptable.
AnyProfile = ManualProfile | TibberProfile
# Map kind → Pydantic model class used for validation.
_PROFILE_MODELS: dict[str, type[ManualProfile] | type[TibberProfile]] = {
"manual": ManualProfile,
"tibber": TibberProfile,
}
# ---------------------------------------------------------------------------
# Public API
# ---------------------------------------------------------------------------
def load_profile(kind: str) -> AnyProfile:
"""Load and validate a pricing profile by kind name.
The profile file is expected at
``app/integrations/pricing/profiles/<kind>.yaml``.
Parameters
----------
kind:
Profile kind without extension (``"manual"`` or ``"tibber"``).
Returns
-------
ManualProfile | TibberProfile
Validated profile model.
Raises
------
ProfileNotFoundError
If ``profiles/<kind>.yaml`` does not exist.
ProfileValidationError
If the YAML is syntactically valid but fails schema validation.
"""
path = _PROFILES_DIR / f"{kind}.yaml"
if not path.exists():
raise ProfileNotFoundError(
f"Pricing profile '{kind}' not found (looked for {path})"
)
with path.open("r", encoding="utf-8") as fh:
raw = yaml.safe_load(fh)
if not isinstance(raw, dict):
raise ProfileValidationError(
f"Profile '{kind}': expected a YAML mapping, got {type(raw).__name__}"
)
# Choose the right Pydantic model based on the ``kind`` field in the YAML.
yaml_kind = raw.get("kind", kind)
model_cls = _PROFILE_MODELS.get(yaml_kind)
if model_cls is None:
raise ProfileValidationError(
f"Profile '{kind}' has unknown kind={yaml_kind!r}; "
f"supported: {list(_PROFILE_MODELS)}"
)
try:
return model_cls.model_validate(raw)
except ValidationError as exc:
raise ProfileValidationError(
f"Profile '{kind}' failed validation: {exc}"
) from exc
def list_profiles() -> list[dict[str, Any]]:
"""Return structural data for all available pricing profiles.
Scans ``app/integrations/pricing/profiles/*.yaml``, loads each, and returns
a list of dicts (Pydantic model dumps). Profiles that fail to load are
skipped with a warning.
Returns
-------
list[dict[str, Any]]
One entry per valid profile, suitable for JSON serialisation and
front-end form rendering.
"""
results: list[dict[str, Any]] = []
if not _PROFILES_DIR.exists():
return results
for path in sorted(_PROFILES_DIR.glob("*.yaml")):
kind = path.stem
try:
profile = load_profile(kind)
results.append(profile.model_dump())
except (ProfileNotFoundError, ProfileValidationError, Exception) as exc:
logger.warning("Skipping malformed pricing profile '%s': %s", kind, exc)
return results
# ---------------------------------------------------------------------------
# Contract-values validation
# ---------------------------------------------------------------------------
def _fill_defaults_manual(values: dict[str, Any], profile: ManualProfile) -> dict[str, Any]:
"""Return a copy of *values* with any ``ode`` default applied if missing."""
filled = dict(values)
energy = dict(filled.get("energy", {}))
# Apply default for ode (default=0) if absent.
if "ode" not in energy and profile.energy.ode.default is not None:
energy["ode"] = profile.energy.ode.default
filled["energy"] = energy
return filled
def _fill_defaults_tibber(values: dict[str, Any], profile: TibberProfile) -> dict[str, Any]:
"""Return a copy of *values* with sell_adjust and management_fee defaults applied."""
filled = dict(values)
energy = dict(filled.get("energy", {}))
# Apply default for sell_adjust (default=0) if absent.
if "sell_adjust" not in energy and profile.energy.sell_adjust.default is not None:
energy["sell_adjust"] = profile.energy.sell_adjust.default
filled["energy"] = energy
standing = dict(filled.get("standing", {}))
# Apply default for management_fee if absent.
if (
"management_fee" not in standing
and profile.standing.management_fee.default is not None
):
standing["management_fee"] = profile.standing.management_fee.default
filled["standing"] = standing
return filled
def _require_numeric(section: str, key: str, container: dict[str, Any]) -> None:
"""Assert that *container[key]* exists and is a number; raise ProfileValidationError."""
if key not in container:
raise ProfileValidationError(
f"Contract values missing required field '{section}.{key}'"
)
val = container[key]
if not isinstance(val, (int, float)):
raise ProfileValidationError(
f"Contract values field '{section}.{key}' must be a number, "
f"got {type(val).__name__!r}"
)
def _validate_manual_values(values: dict[str, Any], profile: ManualProfile) -> dict[str, Any]:
"""Validate and fill-defaults for manual contract values.
Returns the filled values dict on success. Raises ProfileValidationError
on missing required fields or wrong types.
"""
filled = _fill_defaults_manual(values, profile)
energy = filled.get("energy", {})
buy = energy.get("buy", {})
sell = energy.get("sell", {})
standing = filled.get("standing", {})
credits = filled.get("credits", {})
# Required energy.buy fields.
_require_numeric("energy.buy", "normal", buy)
_require_numeric("energy.buy", "dal", buy)
# Required energy.sell fields.
_require_numeric("energy.sell", "normal", sell)
_require_numeric("energy.sell", "dal", sell)
# Required energy fields.
_require_numeric("energy", "energy_tax", energy)
_require_numeric("energy", "ode", energy)
# Required standing fields.
_require_numeric("standing", "network_fee", standing)
_require_numeric("standing", "management_fee", standing)
# Required credits fields.
_require_numeric("credits", "heffingskorting", credits)
return filled
def _validate_tibber_values(values: dict[str, Any], profile: TibberProfile) -> dict[str, Any]:
"""Validate and fill-defaults for tibber contract values.
Returns the filled values dict on success. Raises ProfileValidationError
on missing required fields or wrong types.
"""
filled = _fill_defaults_tibber(values, profile)
energy = filled.get("energy", {})
standing = filled.get("standing", {})
credits = filled.get("credits", {})
# Required energy fields.
_require_numeric("energy", "energy_tax", energy)
_require_numeric("energy", "sell_adjust", energy)
# Required standing fields.
_require_numeric("standing", "management_fee", standing)
_require_numeric("standing", "network_fee", standing)
# Required credits fields.
_require_numeric("credits", "heffingskorting", credits)
return filled
def validate_values(kind: str, values: dict[str, Any]) -> dict[str, Any]:
"""Validate a contract-values dict against the named profile structure.
Fields that carry a ``default`` in the profile (e.g. ``ode``,
``sell_adjust``, tibber ``management_fee``) are silently filled in when
absent from *values*. Fields with no default that are absent, or fields
whose value is not a number, cause a ``ProfileValidationError``.
Parameters
----------
kind:
Profile kind (``"manual"`` or ``"tibber"``).
values:
Contract values dict as stored in ``EnergyContractVersion.values``.
Returns
-------
dict[str, Any]
A (possibly mutated) copy of *values* with defaults applied.
Raises
------
ProfileNotFoundError
If the profile YAML for *kind* does not exist.
ProfileValidationError
If required fields are missing or have wrong types.
"""
profile = load_profile(kind)
if isinstance(profile, ManualProfile):
return _validate_manual_values(values, profile)
if isinstance(profile, TibberProfile):
return _validate_tibber_values(values, profile)
# Unreachable with current kinds, but guard for future extensions.
raise ProfileValidationError(f"No validator implemented for kind={kind!r}")
@@ -0,0 +1,20 @@
kind: manual
label: 固定 / 可变费率(NL,双费率)
energy:
dual_tariff: true # use delivered_1/2, returned_1/2 for low/high tariffs
buy:
normal: { unit: EUR/kWh } # high-tariff buy price (delivered_2 register)
dal: { unit: EUR/kWh } # low-tariff buy price (delivered_1 register)
sell:
normal: { unit: EUR/kWh } # high-tariff sell / return price
dal: { unit: EUR/kWh } # low-tariff sell / return price (currently equal to normal)
energy_tax: { unit: EUR/kWh } # energy tax added to buy price (incl. VAT)
ode: { unit: EUR/kWh, default: 0 } # currently merged into energy_tax
standing: # fixed charges; UI fills per month, engine prorates to days
network_fee: { unit: EUR/month }
management_fee: { unit: EUR/month }
credits:
heffingskorting: { unit: EUR/year } # energy-tax credit; deducted at summary layer
@@ -0,0 +1,14 @@
kind: tibber
label: Tibber 动态电价(15 分钟)
energy:
source: tibber_api # buy = total (from API); sell = total energy_tax sell_adjust
energy_tax: { unit: EUR/kWh } # subtracted from total to derive sell price (incl. VAT)
sell_adjust: { unit: EUR/kWh, default: 0 } # additional sell-price adjustment (residual spread)
standing: # fixed charges; UI fills per month, engine prorates to days
management_fee: { unit: EUR/month, default: 5.99 }
network_fee: { unit: EUR/month }
credits:
heffingskorting: { unit: EUR/year } # energy-tax credit; deducted at summary layer
+288
View File
@@ -0,0 +1,288 @@
"""Price-calculation strategy registry and built-in implementations.
A *strategy* is a plain function registered under a ``kind`` string. Given
per-register kWh deltas, the period start time, the active contract-version
values, and a DB session, it returns a result dict with:
{
"import_cost": Decimal, # total cost of electricity drawn from grid
"export_revenue": Decimal, # total revenue from electricity fed to grid
"net_cost": Decimal, # import_cost export_revenue
"pricing": dict, # snapshot of the price inputs used (for auditing)
}
**Import and export are kept separate throughout** — net_cost is only derived
at the end. This supports the Dutch no-netting rule (saldering afgebouwd).
**All monetary arithmetic uses Decimal** converted via ``Decimal(str(x))`` to
avoid float binary rounding errors.
Design notes
------------
- **Purely data + functions**: no ABC, no class hierarchy. A ``dict``-based
registry is the simplest structure that supports the two current kinds and
leaves the door open for future additions (e.g. ``octopus``, ``frank``).
- ``deltas`` is a plain dataclass ``PeriodDeltas`` — typed, but no ORM.
- Tibber strategy queries ``TibberPrice`` directly from the session; it does not
call the Tibber API (that is T05's job).
"""
from __future__ import annotations
import logging
from dataclasses import dataclass
from datetime import datetime
from decimal import Decimal
from typing import Any, Callable
from sqlalchemy.orm import Session
logger = logging.getLogger(__name__)
# ---------------------------------------------------------------------------
# Shared data types
# ---------------------------------------------------------------------------
@dataclass
class PeriodDeltas:
"""Per-register kWh deltas for one 15-minute billing period.
All values are ``Decimal`` and represent ``end_reading start_reading``
for the corresponding cumulative energy register.
Attributes
----------
d1: Decimal
Delivered (consumed) kWh on the low/dal tariff register (delivered_1).
d2: Decimal
Delivered (consumed) kWh on the normal/high tariff register (delivered_2).
r1: Decimal
Returned (fed-to-grid) kWh on the low/dal tariff register (returned_1).
r2: Decimal
Returned (fed-to-grid) kWh on the normal/high tariff register (returned_2).
"""
d1: Decimal # delivered low-tariff
d2: Decimal # delivered high-tariff
r1: Decimal # returned low-tariff
r2: Decimal # returned high-tariff
# Strategy callable signature.
StrategyFn = Callable[
[PeriodDeltas, datetime, dict[str, Any], Session],
dict[str, Any],
]
# ---------------------------------------------------------------------------
# Registry
# ---------------------------------------------------------------------------
_REGISTRY: dict[str, StrategyFn] = {}
def register_strategy(kind: str, fn: StrategyFn) -> None:
"""Register *fn* as the price-calculation strategy for *kind*.
Overwrites any previously registered strategy for the same kind (allows
monkey-patching in tests).
"""
_REGISTRY[kind] = fn
def get_strategy(kind: str) -> StrategyFn:
"""Return the registered strategy for *kind*.
Raises
------
KeyError
If no strategy is registered for *kind*.
"""
if kind not in _REGISTRY:
raise KeyError(
f"No price strategy registered for kind={kind!r}. "
f"Available: {sorted(_REGISTRY)}"
)
return _REGISTRY[kind]
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _to_decimal(value: Any) -> Decimal:
"""Convert *value* to Decimal via str() to avoid float binary rounding."""
return Decimal(str(value))
# ---------------------------------------------------------------------------
# Manual strategy — fixed dual-tariff
# ---------------------------------------------------------------------------
def _manual_strategy(
deltas: PeriodDeltas,
t0: datetime,
values: dict[str, Any],
session: Session,
) -> dict[str, Any]:
"""Calculate billing costs for one period using fixed dual-tariff rates.
Formula (§3.4):
- ``buy_dal = energy.buy.dal + energy.energy_tax + energy.ode``
- ``buy_normal = energy.buy.normal + energy.energy_tax + energy.ode``
- ``import_cost = Δd1 × buy_dal + Δd2 × buy_normal``
- ``sell_dal = energy.sell.dal`` (no tax on return price)
- ``sell_normal = energy.sell.normal``
- ``export_revenue = Δr1 × sell_dal + Δr2 × sell_normal``
- ``net_cost = import_cost export_revenue``
The ``pricing`` snapshot contains all per-unit prices used so that the
result is fully auditable without re-querying the contract version.
"""
energy = values.get("energy", {})
buy = energy.get("buy", {})
sell = energy.get("sell", {})
energy_tax = _to_decimal(energy.get("energy_tax", 0))
ode = _to_decimal(energy.get("ode", 0))
buy_dal_base = _to_decimal(buy.get("dal", 0))
buy_normal_base = _to_decimal(buy.get("normal", 0))
sell_dal = _to_decimal(sell.get("dal", 0))
sell_normal = _to_decimal(sell.get("normal", 0))
# Effective buy prices including all taxes.
buy_dal = buy_dal_base + energy_tax + ode
buy_normal = buy_normal_base + energy_tax + ode
import_cost = deltas.d1 * buy_dal + deltas.d2 * buy_normal
export_revenue = deltas.r1 * sell_dal + deltas.r2 * sell_normal
net_cost = import_cost - export_revenue
pricing_snapshot = {
"kind": "manual",
"buy_dal": str(buy_dal),
"buy_normal": str(buy_normal),
"sell_dal": str(sell_dal),
"sell_normal": str(sell_normal),
"energy_tax": str(energy_tax),
"ode": str(ode),
}
return {
"import_cost": import_cost,
"export_revenue": export_revenue,
"net_cost": net_cost,
"pricing": pricing_snapshot,
}
# ---------------------------------------------------------------------------
# Tibber strategy — dynamic 15-minute spot price
# ---------------------------------------------------------------------------
class TibberPriceNotFoundError(LookupError):
"""Raised when no TibberPrice row covers the requested period start.
The billing engine (T07) catches this to mark the period as ``degraded``
or skip it until a price becomes available.
"""
def _tibber_strategy(
deltas: PeriodDeltas,
t0: datetime,
values: dict[str, Any],
session: Session,
) -> dict[str, Any]:
"""Calculate billing costs for one period using Tibber dynamic pricing.
Price lookup:
Takes the most recent ``TibberPrice`` row where ``starts_at ≤ t0``
(i.e. the slot that was in effect at *t0*). This is a DESC LIMIT 1
query on ``starts_at``.
Formula (§3.4):
- ``buy = total`` (Tibber's all-inclusive price, already includes tax)
- ``sell = total energy_tax sell_adjust``
- ``import_cost = (Δd1 + Δd2) × buy``
- ``export_revenue = (Δr1 + Δr2) × sell``
- ``net_cost = import_cost export_revenue``
Tibber does not differentiate tariff slots (dal vs normal) — the 15-minute
API price applies to the full delivered/returned volume.
Negative ``total`` (extreme negative spot prices):
When ``total`` is negative the ``sell`` price will also be negative,
meaning ``export_revenue`` becomes negative (feeding to grid *costs*
money). This is the mathematically correct outcome and is left as-is.
Raises
------
TibberPriceNotFoundError
If no ``TibberPrice`` row exists with ``starts_at ≤ t0``. The caller
(T07 billing engine) should catch this and mark the period as
``degraded`` or skip it for later recomputation.
"""
from app.models.energy import TibberPrice # local import to avoid circular
from sqlalchemy import desc
price_row: TibberPrice | None = (
session.query(TibberPrice)
.filter(TibberPrice.starts_at <= t0)
.order_by(desc(TibberPrice.starts_at))
.first()
)
if price_row is None:
raise TibberPriceNotFoundError(
f"No TibberPrice found covering t0={t0.isoformat()!r}; "
"period cannot be billed until price data is available."
)
energy = values.get("energy", {})
energy_tax = _to_decimal(energy.get("energy_tax", 0))
sell_adjust = _to_decimal(energy.get("sell_adjust", 0))
total = _to_decimal(price_row.total)
buy = total
sell = total - energy_tax - sell_adjust
total_delivered = deltas.d1 + deltas.d2
total_returned = deltas.r1 + deltas.r2
import_cost = total_delivered * buy
export_revenue = total_returned * sell
net_cost = import_cost - export_revenue
pricing_snapshot = {
"kind": "tibber",
"tibber_price_starts_at": price_row.starts_at.isoformat(),
"tibber_price_id": price_row.id,
"total": str(total),
"buy": str(buy),
"sell": str(sell),
"energy_tax": str(energy_tax),
"sell_adjust": str(sell_adjust),
}
return {
"import_cost": import_cost,
"export_revenue": export_revenue,
"net_cost": net_cost,
"pricing": pricing_snapshot,
}
# ---------------------------------------------------------------------------
# Register built-in strategies at module import time
# ---------------------------------------------------------------------------
register_strategy("manual", _manual_strategy)
register_strategy("tibber", _tibber_strategy)
+10
View File
@@ -0,0 +1,10 @@
"""Tibber dynamic electricity pricing integration.
This package provides:
- ``client``: HTTP client for the Tibber GraphQL API (price fetching).
- Custom exceptions: ``TibberError`` (base) and ``TibberAuthError`` (auth failure).
Usage::
from app.integrations.tibber.client import fetch_price_range, TibberAuthError
"""
+337
View File
@@ -0,0 +1,337 @@
"""Tibber GraphQL API client for fetching electricity price data.
Design decisions
----------------
- Uses ``httpx`` (already a project dependency) for all HTTP requests.
- Token is **never** written to log messages or exception strings to prevent
credential leakage into log aggregators.
- ``fetch_price_range`` returns a list of ``PricePoint`` objects with UTC-aware
``starts_at`` datetimes; the number of nodes is not assumed — all returned
nodes are parsed regardless of count.
- ``fetch_current_price`` uses a separate, simpler query that asks for the
*current* price point only; used by the connection-test endpoint (T09) to
produce a fast three-state result (success / auth-error / network-error).
- Authentication failures (HTTP 401/403) are raised as ``TibberAuthError`` so
that callers (the test endpoint) can distinguish them from generic network or
API errors.
"""
from __future__ import annotations
import logging
from dataclasses import dataclass
from datetime import UTC, datetime
from typing import Any
import httpx
logger = logging.getLogger(__name__)
_TIBBER_API_URL = "https://api.tibber.com/v1-beta/gql"
_DEFAULT_TIMEOUT = 15.0
# GraphQL query to fetch a range of 15-minute price nodes.
# ``priceInfoRange`` is a Relay-style cursor connection over the subscription's
# entire available price history. With no before/after cursor:
# * ``first: N`` returns the OLDEST N nodes (anchored at the subscription
# start date) — a fixed window that never advances, so it is WRONG for
# "current + upcoming" prices.
# * ``last: N`` returns the NEWEST N nodes (ending at the latest published
# slot: tomorrow 23:45 once day-ahead prices are out, else today 23:45).
# We want the newest slots, so we use ``last``. 192 = 2 days × 24 h × 4 slots,
# which fully covers the "today + tomorrow" window that GET /api/energy/prices
# queries (the earliest of the 192 newest slots reaches back past today 00:00
# UTC in either publish state).
_PRICE_RANGE_QUERY = """
{
viewer {
homes {
id
currentSubscription {
priceInfoRange(resolution: QUARTER_HOURLY, last: 192) {
nodes {
startsAt
total
energy
tax
currency
level
}
}
}
}
}
}
"""
# GraphQL query to fetch the single *current* price point.
_CURRENT_PRICE_QUERY = """
{
viewer {
homes {
id
currentSubscription {
priceInfo {
current {
startsAt
total
energy
tax
currency
level
}
}
}
}
}
}
"""
# ---------------------------------------------------------------------------
# Custom exceptions
# ---------------------------------------------------------------------------
class TibberError(Exception):
"""Base class for all Tibber client errors.
Raised for network failures, timeouts, unexpected HTTP status codes, and
malformed API responses. The exception message will **never** contain the
API token.
"""
class TibberAuthError(TibberError):
"""Raised when the Tibber API rejects the provided token (HTTP 401/403).
Callers that implement a three-state connection test should catch this
exception separately from ``TibberError`` to distinguish auth problems
from network / API problems.
"""
# ---------------------------------------------------------------------------
# Data model
# ---------------------------------------------------------------------------
@dataclass(slots=True)
class PricePoint:
"""One 15-minute price slot returned by the Tibber API.
All monetary values are in ``currency`` and include VAT (user-facing).
``starts_at`` is always a timezone-aware UTC datetime regardless of the
timezone offset that Tibber returns in ``startsAt``.
"""
starts_at: datetime # UTC-aware
total: float # full all-in price (energy + tax)
energy: float # spot energy component
tax: float # tax component
currency: str # ISO 4217 (typically "EUR")
level: str | None # e.g. "CHEAP", "NORMAL", "EXPENSIVE"; None if absent
resolution: str # label for the slot resolution (e.g. "QUARTER_HOURLY")
# ---------------------------------------------------------------------------
# Internal helpers
# ---------------------------------------------------------------------------
def _parse_starts_at(raw: str) -> datetime:
"""Parse an ISO 8601 timestamp with timezone offset and convert to UTC.
Tibber returns timestamps like ``2026-06-23T00:00:00.000+02:00``.
``datetime.fromisoformat`` handles this format in Python 3.11+; the result
is then converted to UTC via ``.astimezone(UTC)``.
"""
return datetime.fromisoformat(raw).astimezone(UTC)
def _parse_node(node: dict[str, Any], resolution: str) -> PricePoint:
"""Parse a single price node dict into a ``PricePoint``."""
return PricePoint(
starts_at=_parse_starts_at(node["startsAt"]),
total=float(node["total"]),
energy=float(node["energy"]),
tax=float(node["tax"]),
currency=str(node["currency"]),
level=node.get("level") or None,
resolution=resolution,
)
def _pick_home(homes: list[dict[str, Any]], home_id: str | None) -> dict[str, Any]:
"""Return the home dict matching *home_id*, or the first home if None."""
if not homes:
raise TibberError("Tibber API returned no homes")
if home_id is not None:
for h in homes:
if h.get("id") == home_id:
return h
raise TibberError("Tibber home id not found in API response")
return homes[0]
def _post_graphql(token: str, query: str, timeout: float) -> dict[str, Any]:
"""POST the GraphQL *query* and return the parsed ``data`` dict.
Raises
------
TibberAuthError
On HTTP 401 or 403.
TibberError
On timeouts, network errors, non-2xx responses, or unexpected body shape.
"""
headers = {
"Authorization": f"Bearer {token}",
"Content-Type": "application/json",
}
try:
response = httpx.post(
_TIBBER_API_URL,
json={"query": query},
headers=headers,
timeout=timeout,
)
except httpx.TimeoutException as exc:
raise TibberError("Tibber API request timed out") from exc
except httpx.HTTPError as exc:
raise TibberError("Tibber API request failed") from exc
if response.status_code in (401, 403):
raise TibberAuthError("Tibber API authentication failed")
try:
response.raise_for_status()
except httpx.HTTPStatusError as exc:
raise TibberError(f"Tibber API returned unexpected status {response.status_code}") from exc
try:
body = response.json()
except Exception as exc:
raise TibberError("Tibber API returned non-JSON response") from exc
if "errors" in body:
# GraphQL errors are not HTTP errors; surface them as TibberError.
# Do not include token in the message.
raise TibberError("Tibber GraphQL returned errors")
data = body.get("data")
if data is None:
raise TibberError("Tibber API response missing 'data' field")
return data
# ---------------------------------------------------------------------------
# Public API
# ---------------------------------------------------------------------------
def fetch_price_range(
token: str,
home_id: str | None = None,
*,
timeout: float = _DEFAULT_TIMEOUT,
) -> list[PricePoint]:
"""Fetch a range of 15-minute price nodes from the Tibber API.
Sends the ``priceInfoRange(resolution: QUARTER_HOURLY, last: 192)`` query
and parses every returned node into a ``PricePoint``. ``last`` (not
``first``) is used so the newest slots are returned; ``first`` would anchor
at the subscription start date and never advance. The number of nodes is
not assumed — all returned nodes are parsed regardless of count.
Parameters
----------
token:
Tibber API token. **Never** logged or included in exception messages.
home_id:
If given, the home with this Tibber home ID is selected. If ``None``,
the first home in the account is used.
timeout:
HTTP request timeout in seconds (default 15 s).
Returns
-------
list[PricePoint]
List of price points with UTC-aware ``starts_at`` values.
Raises
------
TibberAuthError
If the token is rejected (HTTP 401/403).
TibberError
For network failures, timeouts, or unexpected API responses.
"""
data = _post_graphql(token, _PRICE_RANGE_QUERY, timeout)
try:
homes = data["viewer"]["homes"]
except (KeyError, TypeError) as exc:
raise TibberError("Tibber API response has unexpected shape") from exc
home = _pick_home(homes, home_id)
try:
nodes = home["currentSubscription"]["priceInfoRange"]["nodes"]
except (KeyError, TypeError) as exc:
raise TibberError("Tibber API response missing priceInfoRange nodes") from exc
return [_parse_node(node, "QUARTER_HOURLY") for node in nodes]
def fetch_current_price(
token: str,
home_id: str | None = None,
*,
timeout: float = _DEFAULT_TIMEOUT,
) -> PricePoint:
"""Fetch the *current* price point from the Tibber API.
Uses the lighter ``priceInfo { current { ... } }`` query rather than the
full range query, making it suitable for a fast connection test.
Parameters
----------
token:
Tibber API token. **Never** logged or included in exception messages.
home_id:
If given, the home with this Tibber home ID is selected. If ``None``,
the first home in the account is used.
timeout:
HTTP request timeout in seconds (default 15 s).
Returns
-------
PricePoint
The current price point with a UTC-aware ``starts_at``.
Raises
------
TibberAuthError
If the token is rejected (HTTP 401/403).
TibberError
For network failures, timeouts, missing current price, or unexpected
API responses.
"""
data = _post_graphql(token, _CURRENT_PRICE_QUERY, timeout)
try:
homes = data["viewer"]["homes"]
except (KeyError, TypeError) as exc:
raise TibberError("Tibber API response has unexpected shape") from exc
home = _pick_home(homes, home_id)
try:
current = home["currentSubscription"]["priceInfo"]["current"]
except (KeyError, TypeError) as exc:
raise TibberError("Tibber API response missing current price") from exc
if current is None:
raise TibberError("Tibber API returned null for current price")
return _parse_node(current, "QUARTER_HOURLY")
+121
View File
@@ -7,13 +7,17 @@ from fastapi import FastAPI, HTTPException, Request
from fastapi.responses import FileResponse
from fastapi.staticfiles import StaticFiles
from apscheduler.schedulers.background import BackgroundScheduler
from apscheduler.triggers.cron import CronTrigger
from apscheduler.triggers.interval import IntervalTrigger
from sqlalchemy.orm import Session
from app import models # noqa: F401
from app.api.routes.api.config import router as api_config_router
from app.api.routes.api.data import router as api_data_router
from app.api.routes.api.energy import router as api_energy_router
from app.api.routes.api.energy_contracts import router as api_energy_contracts_router
from app.api.routes.api.expose import router as api_expose_router
from app.api.routes.api.meters import router as api_meters_router
from app.api.routes.api.modbus import router as api_modbus_router
from app.api.routes.api.session import router as api_session_router
from app.api.routes import status
@@ -27,9 +31,13 @@ from app.config import get_settings
from app.integrations.mqtt import mqtt_manager
from app.services.auth import AuthBootstrapError, initialize_auth_schema
from app.services.config_page import build_runtime_settings, seed_missing_config_from_bootstrap, sync_app_hostname_from_bootstrap
from app.services.dsmr_ingest import apply_dsmr_subscription
from app.services.public_ip import check_public_ipv4_and_notify
from app.services.modbus_poll import poll_all_enabled_devices, BASE_POLL_TICK_SECONDS
from app.services.ha_discovery import publish_discovery, publish_states
from app.services.tibber_prices import refresh_prices
from app.services.energy_cost import compute_closed_periods
from app.services.timezone import local_tz
from scripts.app_db_adopt import AppDatabaseAdoptionError, validate_app_runtime_db
logger = logging.getLogger(__name__)
@@ -62,6 +70,64 @@ def _run_scheduled_modbus_poll() -> None:
session.close()
def _run_scheduled_tibber_refresh() -> None:
"""Scheduled job: fetch Tibber 15-minute prices and upsert into tibber_price.
Runs every hour so that:
- Today's prices are available from startup.
- Tomorrow's prices (published by Tibber around 13:00 CET / 11:00 UTC) are
picked up within an hour of publication without requiring a server restart.
The job is a no-op when:
- No active energy contract with kind="tibber" exists.
- The Tibber API token is empty in the runtime settings.
Any client exceptions (auth failures, network errors) are caught and logged
so that a single failed fetch does not crash the scheduler or affect the
other background jobs.
"""
session_local = get_session_local()
session: Session = session_local()
try:
runtime_settings = build_runtime_settings(session, get_settings())
refresh_prices(session, runtime_settings)
except Exception:
logger.exception("_run_scheduled_tibber_refresh: unexpected error")
finally:
session.close()
def _run_scheduled_energy_cost() -> None:
"""Scheduled job: compute billing records for all uncalculated closed 15-minute periods.
Runs every minute so that a new period is picked up within 1 minute of
closing. The job is a no-op when:
- No active energy contract with a version covering the period exists.
- DSMR data has not yet arrived for the period boundaries.
- The period's billing record already exists and is not degraded.
Any unexpected exceptions are caught and logged so that a single failure
does not crash the scheduler or affect the other background jobs.
"""
session_local = get_session_local()
session: Session = session_local()
try:
compute_closed_periods(session)
# After billing periods are computed, push fresh energy-cost state values
# to MQTT/HA. publish_states is internally guarded by _should_publish
# (MQTT disabled / not connected → no-op), so this never raises due to
# unconfigured MQTT and does not block the billing job.
try:
from app.services.ha_discovery import publish_states
publish_states(session)
except Exception:
logger.exception("_run_scheduled_energy_cost: publish_states failed (non-fatal)")
except Exception:
logger.exception("_run_scheduled_energy_cost: unexpected error")
finally:
session.close()
def _run_scheduled_ha_state_publish() -> None:
"""Periodic job: publish discovery configs + state + availability for all enabled exposed entities.
@@ -95,6 +161,20 @@ def _run_scheduled_ha_state_publish() -> None:
session.close()
def _run_midnight_state_publish() -> None:
"""本地午夜后不久专门发布一次状态,让 *_today 的每日归零稳稳落在午夜之后
(对 HA 钟慢几秒鲁棒)。best-effort:失败仅记日志,不影响调度器。"""
session_local = get_session_local()
session = session_local()
try:
from app.services.ha_discovery import publish_states
publish_states(session)
except Exception:
logger.exception("_run_midnight_state_publish: failed (non-fatal)")
finally:
session.close()
def ensure_auth_db_ready() -> None:
session_local = get_session_local()
session: Session = session_local()
@@ -147,6 +227,39 @@ async def lifespan(_: FastAPI):
max_instances=1,
coalesce=True,
)
# Tibber price refresh: fetch today + tomorrow every hour.
# The job is a no-op when no active tibber contract or token is configured,
# so it is safe to register unconditionally.
scheduler.add_job(
_run_scheduled_tibber_refresh,
trigger=IntervalTrigger(hours=1),
id="tibber-refresh",
replace_existing=True,
max_instances=1,
coalesce=True,
)
# Energy cost billing: compute uncalculated closed 15-minute periods every minute.
# The job is a no-op when no active contract or DSMR data is present, so it is
# safe to register unconditionally.
scheduler.add_job(
_run_scheduled_energy_cost,
trigger=IntervalTrigger(minutes=1),
id="energy-cost",
replace_existing=True,
max_instances=1,
coalesce=True,
)
# Dedicated midnight publish: fire at local 00:00:10 so *_today grace (5 s) has
# already elapsed and the day-rolled value is pushed to HA immediately, rather
# than waiting for the next 60-second ha-state-publish sweep.
scheduler.add_job(
_run_midnight_state_publish,
trigger=CronTrigger(hour=0, minute=0, second=10, timezone=local_tz()),
id="midnight-today-publish",
replace_existing=True,
max_instances=1,
coalesce=True,
)
scheduler.start()
# MQTT: connect using DB-merged runtime settings so broker configured via UI
@@ -161,6 +274,11 @@ async def lifespan(_: FastAPI):
_startup_session.close()
mqtt_manager.connect(_startup_runtime_settings)
# DSMR ingest: subscribe to the configured MQTT topic when enabled. The same
# applier is called from PUT /api/config, so toggling DSMR via the UI takes
# effect without an app restart.
apply_dsmr_subscription(_startup_runtime_settings)
yield
# MQTT: clean shutdown before the process exits.
@@ -188,6 +306,9 @@ def create_app() -> FastAPI:
app.include_router(status.router)
app.include_router(api_config_router)
app.include_router(api_data_router)
app.include_router(api_energy_router)
app.include_router(api_energy_contracts_router)
app.include_router(api_meters_router)
app.include_router(api_expose_router)
app.include_router(api_modbus_router)
app.include_router(api_session_router)
+318
View File
@@ -0,0 +1,318 @@
"""SQLAlchemy models for the energy pricing and DSMR metering subsystem.
Six tables:
- meter: physical electricity meter lifecycle epoch.
- dsmr_reading: raw DSMR telegram blobs (10-second down-sampled).
- energy_contract: contract head (manual or tibber, one active at a time).
- energy_contract_version: versioned pricing values; append-only for auditability.
- tibber_price: cached Tibber 15-minute spot prices (immutable).
- energy_cost_period: computed 15-minute billing periods (immutable snapshot).
"""
from __future__ import annotations
import uuid as _uuid
from datetime import datetime
from sqlalchemy import Boolean, DateTime, Float, ForeignKey, Integer, String
from sqlalchemy.orm import Mapped, mapped_column, relationship
from sqlalchemy.types import JSON
from app.db import Base
def _uuid4_str() -> str:
return str(_uuid.uuid4())
class Meter(Base):
"""One physical electricity meter's installation epoch.
A ``meter`` record represents the period ``[started_at, ended_at)`` during
which a particular physical meter was installed and active. Replacing a meter
(swap, home move, etc.) is modelled by closing the current record
(``ended_at = swap_timestamp``) and opening a new one
(``started_at = swap_timestamp``).
**Invariant**: for each ``commodity`` there is at most one active meter
(``ended_at IS NULL``) at any point in time. The service layer enforces
this — no DB-level constraint is added to keep the migration simple and to
allow the application to return a meaningful error message.
``commodity`` defaults to ``"electricity"``; the field is a free-form string
(no CHECK constraint) so future commodities (``gas``, ``heating``) can be
added without a schema change.
``reason`` captures why this epoch started — one of ``initial``,
``meter_swap``, ``home_move``, or ``other`` — stored as a plain string so
the application layer controls the allowed set.
"""
__tablename__ = "meter"
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
# Stable internal identity — used as HA Discovery unique_id anchor.
uuid: Mapped[str] = mapped_column(
String(36), unique=True, nullable=False, default=_uuid4_str
)
# Human-readable label for this physical meter (e.g. address, serial, tariff zone).
label: Mapped[str] = mapped_column(String(255), nullable=False)
# Energy commodity this meter measures. Defaults to "electricity".
commodity: Mapped[str] = mapped_column(String(32), nullable=False, default="electricity")
# UTC timestamp when this meter epoch starts (inclusive). May be in the past
# (retroactive declaration); effective billing start = max(started_at, data start).
started_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
# UTC timestamp when this meter epoch ends (exclusive). NULL = currently active.
ended_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
# Why this epoch was created. Application-layer validation enforces the
# allowed set; no CHECK constraint to keep migrations simple.
reason: Mapped[str] = mapped_column(String(64), nullable=False)
# Free-form note (e.g. location, physical meter id, reason details).
note: Mapped[str | None] = mapped_column(String(1024), nullable=True)
# UTC timestamp of when this row was created.
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
# Relationship to cost periods attributed to this meter epoch (not loaded eagerly).
cost_periods: Mapped[list["EnergyCostPeriod"]] = relationship(
back_populates="meter", cascade="save-update, merge"
)
class DsmrReading(Base):
"""One down-sampled DSMR telegram stored as a full JSON blob.
Identity & idempotency are **independent of the DSMR Reader's telegram id**
(that field overflows and must be manually reset to zero — a known DSMR
quirk — so relying on it for uniqueness risks silently dropping new data).
The table's own autoincrement ``id`` PK is the stable internal identity, and
``recorded_at`` (the telegram timestamp) is the UNIQUE de-duplication key: a
single P1 meter emits exactly one telegram per timestamp.
``recorded_at`` is a real column (not inside the payload) so time-range
queries are efficient. The entire telegram frame is stored verbatim in
``payload``; no field allow-list is applied so future commodities (gas,
heating, three-phase) are accommodated without a schema change.
"""
__tablename__ = "dsmr_reading"
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
# UTC timestamp of the sample — real column, UNIQUE (telegram-id-independent
# idempotency key). The unique index also serves time-range queries.
recorded_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), nullable=False, unique=True
)
# Telegram's own id (DSMR Reader assigns it). Stored only as a reference /
# debugging aid — NOT used for uniqueness or idempotency (it overflows and
# gets reset to zero). Nullable because some DSMR sources may not emit one.
source_id: Mapped[int | None] = mapped_column(Integer, nullable=True)
# Full telegram frame as a JSON object; values are typically JSON strings
# (e.g. "20915.154") — callers must cast to Decimal before arithmetic.
payload: Mapped[dict] = mapped_column(JSON, nullable=False)
class EnergyContract(Base):
"""Contract head: a named energy contract with a chosen pricing strategy.
``kind`` determines which price strategy is used (``manual`` for fixed
dual-tariff rates entered by the user, ``tibber`` for dynamic API prices).
Only one contract may be ``active`` at a time; the service layer enforces
mutual exclusion. Specific pricing values live in ``EnergyContractVersion``
so that price changes can be tracked without modifying historical records.
"""
__tablename__ = "energy_contract"
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
# Human-readable label; freely editable by the user.
name: Mapped[str] = mapped_column(String(255), nullable=False)
# Strategy selector: "manual" or "tibber". Application-layer validation
# enforces the allowed set; no DB CHECK constraint is added to keep the
# migration simple and the strategy registry extensible.
kind: Mapped[str] = mapped_column(String(32), nullable=False)
# Whether this is the currently active contract (at most one should be True).
active: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
# ISO 4217 currency code for all monetary values in this contract.
currency: Mapped[str] = mapped_column(String(8), nullable=False, default="EUR")
# Audit timestamps.
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
# Relationship to versions (back-reference; not loaded eagerly).
versions: Mapped[list["EnergyContractVersion"]] = relationship(
back_populates="contract", cascade="save-update, merge"
)
class EnergyContractVersion(Base):
"""One time-bounded version of an energy contract's pricing values.
Pricing changes are modelled as new versions (append-only); existing versions
are never modified so that historical ``EnergyCostPeriod`` records remain
fully auditable. ``effective_to`` is ``NULL`` for the currently open version.
"""
__tablename__ = "energy_contract_version"
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
# FK to the parent contract. RESTRICT prevents deletion of a contract that
# still has versioned pricing rows attached to it.
contract_id: Mapped[int] = mapped_column(
ForeignKey("energy_contract.id", ondelete="RESTRICT"), nullable=False
)
# Start of this version's validity window (inclusive, UTC).
effective_from: Mapped[datetime] = mapped_column(
DateTime(timezone=True), nullable=False
)
# End of this version's validity window (exclusive, UTC). NULL means open-ended
# (i.e. this is the most recent / current version).
effective_to: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True), nullable=True
)
# Pricing values as a JSON object conforming to the profile structure for
# ``contract.kind`` (validated by the application layer against the YAML profile).
values: Mapped[dict] = mapped_column(JSON, nullable=False)
# Creation timestamp.
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
# Relationship back to the parent contract.
contract: Mapped["EnergyContract"] = relationship(back_populates="versions")
# Relationship to cost periods that reference this version.
cost_periods: Mapped[list["EnergyCostPeriod"]] = relationship(
back_populates="contract_version", cascade="save-update, merge"
)
class TibberPrice(Base):
"""Cached Tibber 15-minute spot price point (immutable once fetched).
``starts_at`` is unique so that upserts are idempotent. Past prices are
never overwritten; the fetch job only adds rows for future time slots.
"""
__tablename__ = "tibber_price"
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
# UTC start of the 15-minute slot; unique so upsert is idempotent.
starts_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), nullable=False, unique=True
)
# Resolution label as returned by the Tibber API (e.g. "QUARTER_HOURLY").
resolution: Mapped[str] = mapped_column(String(32), nullable=False)
# Price components in the contract currency (all include VAT, user-facing).
energy: Mapped[float] = mapped_column(Float, nullable=False)
tax: Mapped[float] = mapped_column(Float, nullable=False)
total: Mapped[float] = mapped_column(Float, nullable=False)
# Tibber price level (e.g. "NORMAL", "CHEAP", "EXPENSIVE"); may be absent.
level: Mapped[str | None] = mapped_column(String(32), nullable=True)
# ISO 4217 currency code as returned by the API.
currency: Mapped[str] = mapped_column(String(8), nullable=False)
# UTC timestamp of when this row was fetched/inserted.
fetched_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
class EnergyCostPeriod(Base):
"""Computed billing record for one 15-minute metering period (immutable snapshot).
Each row captures the per-register kWh deltas, the resulting import cost and
export revenue, and a full snapshot of the pricing values used so that the
calculation is fully auditable and reproducible without re-querying the
contract version. Rows are written once and never modified; explicit
recomputation via the API is the only way to overwrite a period.
"""
__tablename__ = "energy_cost_period"
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
# UTC start of the 15-minute period; unique so upsert is idempotent.
period_start: Mapped[datetime] = mapped_column(
DateTime(timezone=True), nullable=False, unique=True
)
# Per-register kWh deltas for the period (end minus start of cumulative registers).
# _1 = dal/low-tariff, _2 = normal/high-tariff (NL convention).
d1_kwh: Mapped[float] = mapped_column(Float, nullable=False) # delivered low
d2_kwh: Mapped[float] = mapped_column(Float, nullable=False) # delivered high
r1_kwh: Mapped[float] = mapped_column(Float, nullable=False) # returned low
r2_kwh: Mapped[float] = mapped_column(Float, nullable=False) # returned high
# Computed monetary amounts for the period (in ``currency``).
import_cost: Mapped[float] = mapped_column(Float, nullable=False)
export_revenue: Mapped[float] = mapped_column(Float, nullable=False)
net_cost: Mapped[float] = mapped_column(Float, nullable=False)
# ISO 4217 currency code matching the contract.
currency: Mapped[str] = mapped_column(String(8), nullable=False)
# Full snapshot of the pricing inputs used during computation. This makes
# each row self-contained and auditable even if the contract is later changed.
pricing: Mapped[dict] = mapped_column(JSON, nullable=False)
# FK to the exact contract version whose values were used. RESTRICT prevents
# deletion of a version that has cost records attached. Nullable to support
# periods computed in ``degraded`` mode (missing price data).
contract_version_id: Mapped[int | None] = mapped_column(
ForeignKey("energy_contract_version.id", ondelete="RESTRICT"), nullable=True
)
# FK to the meter epoch this period belongs to. RESTRICT prevents deletion of
# a meter that still has attributed cost periods. Nullable for backwards
# compatibility (pre-M7 rows) and degraded periods where the meter was not
# determinable.
meter_id: Mapped[int | None] = mapped_column(
ForeignKey("meter.id", ondelete="RESTRICT"), nullable=True
)
# True when the period was computed with incomplete data (missing readings or
# missing price); serves as a flag for later recomputation.
degraded: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
# UTC timestamp of when this row was computed/inserted.
computed_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
# Relationship back to the contract version.
contract_version: Mapped["EnergyContractVersion | None"] = relationship(
back_populates="cost_periods"
)
# Relationship back to the meter epoch.
meter: Mapped["Meter | None"] = relationship(back_populates="cost_periods")
# Index on recorded_at for efficient time-range queries on DSMR readings.
# (The ORM-level index=True on recorded_at already creates ix_dsmr_reading_recorded_at;
# no composite index is needed for single-meter deployments.)
# Index on period_start is covered by the unique constraint (SQLite creates an
# implicit index for UNIQUE columns), so no additional index is required.
# Index on starts_at for TibberPrice is covered by the unique constraint similarly.
+216
View File
@@ -0,0 +1,216 @@
"""Pydantic schemas for the Energy data API (M6-T09).
Covers six endpoint groups under /api/energy:
- GET /prices — price curve (tibber 15min points or manual tariff)
- GET /costs — energy_cost_period rows (time-range, paginated)
- GET /costs/summary — aggregated metered + standing charges credits
- GET /dsmr/latest — most recent dsmr_reading blob
- POST /costs/recompute — explicit idempotent recompute (returns count)
- POST /tibber/test — three-state Tibber connection test
"""
from __future__ import annotations
from datetime import datetime
from typing import Any, Literal
from pydantic import BaseModel, Field
# ---------------------------------------------------------------------------
# GET /api/energy/prices
# ---------------------------------------------------------------------------
class PricePointSchema(BaseModel):
"""A single 15-minute price point (tibber) or placeholder entry."""
starts_at: datetime
buy: float = Field(description="All-in buy price in EUR/kWh (including taxes).")
sell: float = Field(description="Net sell price in EUR/kWh.")
level: str | None = Field(
default=None,
description="Tibber price level (CHEAP / NORMAL / EXPENSIVE); null for manual.",
)
class ManualTariffSchema(BaseModel):
"""Fixed-tariff breakdown for manual contracts.
Prices are the *effective* buy prices as used by the billing engine
(energy_buy_x + energy_tax + ode) and the raw sell prices.
"""
buy_dal: float = Field(description="Effective buy price, low-tariff / dal (EUR/kWh).")
buy_normal: float = Field(description="Effective buy price, normal / high-tariff (EUR/kWh).")
sell_dal: float = Field(description="Sell price, low-tariff / dal (EUR/kWh).")
sell_normal: float = Field(description="Sell price, normal / high-tariff (EUR/kWh).")
class PricesResponse(BaseModel):
"""Response for GET /api/energy/prices.
``kind`` mirrors the active contract kind:
- ``"tibber"`` → ``points`` has actual 15-min price entries; ``tariff`` is null.
- ``"manual"`` → ``points`` is empty; ``tariff`` carries the fixed-rate table.
- ``None`` → no active contract; both ``points`` and ``tariff`` are empty/null.
``currency`` comes from the active contract (or "EUR" fallback).
``points`` is always ascending by ``starts_at``.
"""
kind: str | None = Field(
description="Active contract kind ('tibber' or 'manual'), or null if no active contract."
)
currency: str = Field(description="ISO 4217 currency code.")
points: list[PricePointSchema] = Field(
description=(
"15-minute price points for tibber contracts (ascending by starts_at). "
"Empty for manual contracts or when no active contract exists."
)
)
tariff: ManualTariffSchema | None = Field(
default=None,
description=(
"Fixed tariff table for manual contracts. "
"Null for tibber contracts and when no active contract exists."
),
)
# ---------------------------------------------------------------------------
# GET /api/energy/costs
# ---------------------------------------------------------------------------
class CostPeriodSchema(BaseModel):
"""One 15-minute billing record from the energy_cost_period table."""
period_start: datetime
d1_kwh: float = Field(description="Delivered low-tariff kWh for this period.")
d2_kwh: float = Field(description="Delivered normal-tariff kWh for this period.")
r1_kwh: float = Field(description="Returned low-tariff kWh for this period.")
r2_kwh: float = Field(description="Returned normal-tariff kWh for this period.")
import_cost: float = Field(description="Cost of electricity drawn from grid (EUR).")
export_revenue: float = Field(description="Revenue from electricity fed to grid (EUR).")
net_cost: float = Field(description="import_cost export_revenue (EUR).")
currency: str = Field(description="ISO 4217 currency code.")
degraded: bool = Field(
description="True when the period was computed with incomplete data."
)
contract_version_id: int | None = Field(
default=None,
description="FK to the contract version used for this billing period (null when degraded).",
)
model_config = {"from_attributes": True}
class CostsResponse(BaseModel):
"""Response for GET /api/energy/costs."""
items: list[CostPeriodSchema]
total: int = Field(description="Number of items returned.")
# ---------------------------------------------------------------------------
# GET /api/energy/costs/summary
# ---------------------------------------------------------------------------
class SummaryResponse(BaseModel):
"""Response for GET /api/energy/costs/summary.
All monetary values are in ``currency``.
``total_payable = metered_net + fixed_costs credits``
"""
currency: str
metered_import: float = Field(description="Σ import_cost for non-degraded periods.")
metered_export: float = Field(description="Σ export_revenue for non-degraded periods.")
metered_net: float = Field(description="Σ net_cost for non-degraded periods.")
fixed_costs: float = Field(
description="Standing charges (network_fee + management_fee) apportioned over the interval."
)
credits: float = Field(
description="Energy-tax credit (heffingskorting) apportioned over the interval."
)
total_payable: float = Field(
description="metered_net + fixed_costs credits (actual amount owed)."
)
period_count: int = Field(description="Number of non-degraded billing periods in range.")
degraded_count: int = Field(description="Number of degraded billing periods in range.")
days: float = Field(description="Interval length in days.")
# ---------------------------------------------------------------------------
# GET /api/energy/dsmr/latest
# ---------------------------------------------------------------------------
class DsmrLatestResponse(BaseModel):
"""Response for GET /api/energy/dsmr/latest.
``found`` is False when no dsmr_reading rows exist yet. The front-end
should check ``found`` before reading ``recorded_at`` or ``payload``.
"""
found: bool
recorded_at: datetime | None = None
payload: dict[str, Any] | None = None
# ---------------------------------------------------------------------------
# POST /api/energy/costs/recompute
# ---------------------------------------------------------------------------
class RecomputeResponse(BaseModel):
"""Response for POST /api/energy/costs/recompute."""
recomputed: int = Field(
description=(
"Number of 15-minute periods for which a billing record was written "
"(inserted or updated). Periods skipped due to missing contract or "
"missing Tibber price are not counted."
)
)
# ---------------------------------------------------------------------------
# POST /api/energy/tibber/test
# ---------------------------------------------------------------------------
class TibberTestPriceSchema(BaseModel):
"""Current Tibber price point returned on a successful test.
Carries enough fields for the front-end to confirm the API is working and
display the live price. The API token is **never** included.
"""
starts_at: datetime
total: float
energy: float
tax: float
currency: str
level: str | None = None
class TibberTestResponse(BaseModel):
"""Three-state response for POST /api/energy/tibber/test.
Possible ``result`` values:
- ``"success"`` — Tibber API responded with a valid price.
- ``"config-error"`` — Token is missing or not configured.
- ``"failed"`` — API call failed (auth rejected, network error, timeout, etc.).
``price`` is populated only on ``"success"``; it is null otherwise.
``message`` always contains a human-readable explanation.
"""
result: Literal["success", "config-error", "failed"]
message: str
price: TibberTestPriceSchema | None = None
+148
View File
@@ -0,0 +1,148 @@
"""Pydantic schemas for the EnergyContract CRUD + versioning API (M6-T04).
Schema hierarchy
----------------
ContractVersionResponse — single version row (id, dates, values, created_at)
ContractResponse — contract head (id, name, kind, active, currency, timestamps)
ContractDetailResponse — contract head + embedded versions list (for GET{id})
ContractListResponse — paginated list of ContractResponse items
ContractCreate — POST /api/energy/contracts body
ContractPatch — PATCH /api/energy/contracts/{id} body (all fields optional)
VersionCreate — POST /api/energy/contracts/{id}/versions body
ProfilesResponse — GET /api/energy/profiles response
"""
from __future__ import annotations
from datetime import datetime
from typing import Any
from pydantic import BaseModel, Field
# ---------------------------------------------------------------------------
# Version schemas
# ---------------------------------------------------------------------------
class ContractVersionResponse(BaseModel):
"""Response schema for a single EnergyContractVersion row."""
id: int
effective_from: datetime
effective_to: datetime | None
values: dict[str, Any]
created_at: datetime
model_config = {"from_attributes": True}
# ---------------------------------------------------------------------------
# Contract schemas
# ---------------------------------------------------------------------------
class ContractResponse(BaseModel):
"""Response schema for a single EnergyContract (without embedded versions).
Used for list responses where embedding all versions would be expensive.
"""
id: int
name: str
kind: str
active: bool
currency: str
created_at: datetime
updated_at: datetime
model_config = {"from_attributes": True}
class ContractDetailResponse(BaseModel):
"""Response schema for a single EnergyContract with full version history.
Returned by GET /api/energy/contracts/{id} and by successful POST / PATCH
operations where the caller needs to see all version data.
"""
id: int
name: str
kind: str
active: bool
currency: str
created_at: datetime
updated_at: datetime
versions: list[ContractVersionResponse]
model_config = {"from_attributes": True}
class ContractListResponse(BaseModel):
"""Response schema for GET /api/energy/contracts."""
items: list[ContractResponse]
total: int
# ---------------------------------------------------------------------------
# Request schemas
# ---------------------------------------------------------------------------
class ContractCreate(BaseModel):
"""Request body for POST /api/energy/contracts.
``effective_from`` defaults to the current UTC time if not provided,
giving the first version an open-ended start from "now".
``kind`` is validated at the application layer against the profile registry;
clients should send ``"manual"`` or ``"tibber"``.
"""
name: str = Field(..., min_length=1, max_length=255)
kind: str = Field(..., min_length=1, max_length=32)
currency: str = Field(default="EUR", min_length=1, max_length=8)
values: dict[str, Any]
effective_from: datetime | None = Field(
default=None,
description=(
"UTC datetime from which the first pricing version is effective. "
"Defaults to the current UTC time when omitted."
),
)
class ContractPatch(BaseModel):
"""Request body for PATCH /api/energy/contracts/{id}.
All fields are optional. Sending ``active=true`` activates this contract
(deactivating all others); ``active=false`` deactivates it without affecting
other contracts.
"""
name: str | None = Field(default=None, min_length=1, max_length=255)
active: bool | None = None
class VersionCreate(BaseModel):
"""Request body for POST /api/energy/contracts/{id}/versions."""
effective_from: datetime
values: dict[str, Any]
# ---------------------------------------------------------------------------
# Profile response schemas
# ---------------------------------------------------------------------------
class ProfilesResponse(BaseModel):
"""Response schema for GET /api/energy/profiles.
``profiles`` is a list of raw profile dicts as produced by
``list_profiles()`` (Pydantic model dumps). Each entry contains at minimum
``kind`` and ``label``; the full nested structure allows the front-end to
render a type-appropriate form for each pricing profile.
"""
profiles: list[dict[str, Any]]
+131
View File
@@ -0,0 +1,131 @@
"""Pydantic schemas for the Meter CRUD + swap declaration API (M7-T05).
Schema hierarchy
----------------
MeterResponse — single meter row (id/label/commodity/started_at/ended_at/reason/note/created_at)
MeterListResponse — ordered list of MeterResponse items
MeterDeclareRequest — POST /api/energy/meters body (declare a swap or initial meter)
MeterPatchRequest — PATCH /api/energy/meters/{id} body (all fields optional)
"""
from __future__ import annotations
from datetime import datetime
from enum import Enum
from pydantic import BaseModel, Field
# ---------------------------------------------------------------------------
# Enums
# ---------------------------------------------------------------------------
class MeterReason(str, Enum):
"""Allowed values for the meter epoch creation reason."""
initial = "initial"
meter_swap = "meter_swap"
home_move = "home_move"
other = "other"
# ---------------------------------------------------------------------------
# Response schemas
# ---------------------------------------------------------------------------
class MeterResponse(BaseModel):
"""Response schema for a single Meter epoch row.
``ended_at`` is ``null`` for the currently active meter.
"""
id: int
label: str
commodity: str
started_at: datetime
ended_at: datetime | None
reason: str
note: str | None
created_at: datetime
model_config = {"from_attributes": True}
class MeterListResponse(BaseModel):
"""Response schema for GET /api/energy/meters.
Meters are returned in ascending ``started_at`` order so the caller sees
the historical installation sequence.
"""
items: list[MeterResponse]
total: int
# ---------------------------------------------------------------------------
# Request schemas
# ---------------------------------------------------------------------------
_VALID_REASONS = ", ".join(r.value for r in MeterReason)
class MeterDeclareRequest(BaseModel):
"""Request body for POST /api/energy/meters.
Declares a new meter epoch (swap, home move, or initial declaration). The
service layer closes the current active meter for the given commodity at
``started_at`` and opens a new one.
``started_at`` follows the Principle-A localisation convention: a
timezone-naive value is interpreted as the **server's local wall-clock time**
(e.g. CEST midnight → stored as UTC the night before); a timezone-aware
value is converted to UTC as-is. Omitting ``started_at`` is not allowed —
every meter declaration must carry an explicit start timestamp.
``commodity`` defaults to ``"electricity"``; the field is available for
future use with ``gas`` or ``heating``.
"""
label: str = Field(..., min_length=1, max_length=255)
started_at: datetime = Field(
...,
description=(
"UTC (or server-local naive) datetime from which this meter epoch starts. "
"May be in the past (retroactive declaration)."
),
)
reason: MeterReason = Field(
...,
description=f"Why this epoch was created. One of: {_VALID_REASONS}.",
)
note: str | None = Field(default=None, max_length=1024)
commodity: str = Field(
default="electricity",
min_length=1,
max_length=32,
description="Energy commodity this meter measures. Defaults to 'electricity'.",
)
class MeterPatchRequest(BaseModel):
"""Request body for PATCH /api/energy/meters/{id}.
All fields are optional. Only non-``None`` values are applied.
Updating ``started_at`` is a **retroactive correction**: the service layer
maintains timeline continuity (adjusting the preceding meter's ``ended_at``)
and the API layer triggers ``recompute_range`` over the affected window so
that billing attribution is re-judged.
"""
label: str | None = Field(default=None, min_length=1, max_length=255)
note: str | None = Field(default=None, max_length=1024)
started_at: datetime | None = Field(
default=None,
description=(
"Retroactive correction of the meter epoch start timestamp. "
"Triggers billing recompute over the affected window."
),
)
+17
View File
@@ -152,3 +152,20 @@ class ModbusTestReadResponse(BaseModel):
ok: bool
payload: dict[str, Any] | None = None
error: str | None = None
# ---------------------------------------------------------------------------
# Cascade-delete schema
# ---------------------------------------------------------------------------
class ModbusDeleteResponse(BaseModel):
"""Response for DELETE /api/modbus/devices/{uuid}?cascade=true.
Returned only when cascade deletion succeeds (HTTP 200). Non-cascade
successful deletes continue to return HTTP 204 (no body).
"""
deleted: bool
readings_deleted: int
toggles_deleted: int
+42
View File
@@ -122,7 +122,42 @@ CONFIG_FIELDS: tuple[ConfigField, ...] = (
"ha_discovery_prefix",
"HA Discovery Prefix",
),
ConfigField(
"Home Assistant Discovery",
"HA_STATE_TOPIC_PREFIX",
"ha_state_topic_prefix",
"HA State Topic Prefix",
),
ConfigField("Modbus", "MODBUS_POLLING_ENABLED", "modbus_polling_enabled", "Modbus Polling Enabled", input_type="checkbox"),
ConfigField(
"DSMR",
"DSMR_INGEST_ENABLED",
"dsmr_ingest_enabled",
"DSMR Ingest Enabled",
input_type="checkbox",
),
ConfigField("DSMR", "DSMR_MQTT_TOPIC", "dsmr_mqtt_topic", "DSMR MQTT Topic"),
ConfigField(
"DSMR",
"DSMR_SAMPLE_INTERVAL_S",
"dsmr_sample_interval_s",
"DSMR Sample Interval (s)",
input_type="number",
),
ConfigField(
"DSMR",
"DSMR_TARIFF_TOPIC",
"dsmr_tariff_topic",
"DSMR Tariff Topic",
),
ConfigField(
"Tibber",
"TIBBER_API_TOKEN",
"tibber_api_token",
"Tibber API Token",
secret=True,
),
ConfigField("Tibber", "TIBBER_HOME_ID", "tibber_home_id", "Tibber Home ID"),
)
@@ -322,4 +357,11 @@ def _settings_payload(settings: Settings) -> dict[str, Any]:
"mqtt_tls_enabled": settings.mqtt_tls_enabled,
"ha_discovery_enabled": settings.ha_discovery_enabled,
"ha_discovery_prefix": settings.ha_discovery_prefix,
"ha_state_topic_prefix": settings.ha_state_topic_prefix,
"dsmr_ingest_enabled": settings.dsmr_ingest_enabled,
"dsmr_mqtt_topic": settings.dsmr_mqtt_topic,
"dsmr_sample_interval_s": settings.dsmr_sample_interval_s,
"dsmr_tariff_topic": settings.dsmr_tariff_topic,
"tibber_api_token": settings.tibber_api_token,
"tibber_home_id": settings.tibber_home_id,
}
+364
View File
@@ -0,0 +1,364 @@
"""Service layer for EnergyContract CRUD, versioning, and activation.
All functions accept an explicit SQLAlchemy Session; callers are responsible
for committing or rolling back the transaction.
Design decisions
----------------
- ``create_contract``: validates values against the pricing profile *before*
writing any rows; raises ``ProfileValidationError`` on non-compliance.
- ``add_version``: append-only; closes the previous open version by
setting its ``effective_to`` to the new version's ``effective_from``; raises
``ContractVersionError`` if the new date is strictly earlier than the previous
version's ``effective_from``.
- ``activate_contract``: mutual-exclusion; sets all other contracts' ``active``
to False, then sets the given contract's ``active`` to True.
- ``active_contract_version_at``: returns the single version of the currently
active contract that covers *ts* (``effective_from ≤ ts < effective_to``,
or open-ended when ``effective_to`` is None).
SQLite timezone note
--------------------
SQLite stores ``DateTime(timezone=True)`` columns as naive UTC strings; on
read-back they come out as **timezone-naive** datetimes. Wherever this code
compares timestamps from the DB against timezone-aware values (e.g. from
Pydantic or ``datetime.now(UTC)``), it calls ``_as_utc()`` to make both sides
comparable without tripping on "offset-naive vs offset-aware" TypeErrors.
"""
from __future__ import annotations
import logging
from datetime import UTC, datetime
from typing import Any
from sqlalchemy import select
from sqlalchemy.orm import Session
from app.integrations.pricing.profiles import validate_values
from app.models.energy import EnergyContract, EnergyContractVersion
logger = logging.getLogger(__name__)
# ---------------------------------------------------------------------------
# Internal helpers
# ---------------------------------------------------------------------------
def _as_utc(dt: datetime) -> datetime:
"""Return *dt* as a timezone-aware UTC datetime.
SQLite's DateTime(timezone=True) column type stores datetimes as naive UTC
strings and gives them back as naive datetimes on read. This helper
re-attaches the UTC timezone info when it is missing, making cross-origin
comparisons safe.
"""
if dt.tzinfo is None:
return dt.replace(tzinfo=UTC)
return dt
# ---------------------------------------------------------------------------
# Custom exception
# ---------------------------------------------------------------------------
class ContractVersionError(ValueError):
"""Raised when a new contract version has an invalid effective date.
Specifically: the new version's ``effective_from`` must be greater than or
equal to the previous open version's ``effective_from``. Allowing equal
timestamps would cause ambiguous overlap; the service therefore also rejects
strictly-equal values (same second) to avoid silent data loss.
"""
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def get_contract_or_none(session: Session, contract_id: int) -> EnergyContract | None:
"""Return the contract with the given id, or None if not found."""
return session.execute(
select(EnergyContract).where(EnergyContract.id == contract_id)
).scalar_one_or_none()
def list_contracts(session: Session) -> list[EnergyContract]:
"""Return all contracts ordered by id (ascending)."""
return list(
session.execute(select(EnergyContract).order_by(EnergyContract.id)).scalars().all()
)
def _open_version(session: Session, contract: EnergyContract) -> EnergyContractVersion | None:
"""Return the current open version (effective_to IS NULL) for *contract*, or None."""
return session.execute(
select(EnergyContractVersion)
.where(
EnergyContractVersion.contract_id == contract.id,
EnergyContractVersion.effective_to.is_(None),
)
.order_by(EnergyContractVersion.effective_from.desc())
.limit(1)
).scalar_one_or_none()
# ---------------------------------------------------------------------------
# Core service functions
# ---------------------------------------------------------------------------
def create_contract(
session: Session,
*,
name: str,
kind: str,
currency: str = "EUR",
values: dict[str, Any],
effective_from: datetime,
) -> EnergyContract:
"""Create a new energy contract with its first pricing version.
Parameters
----------
session:
Active SQLAlchemy session. Caller must commit after this returns.
name:
Human-readable label for the contract.
kind:
Pricing strategy identifier (``"manual"`` or ``"tibber"``).
currency:
ISO 4217 currency code (default ``"EUR"``).
values:
Pricing values dict conforming to the named profile's structure.
Validated via ``validate_values(kind, values)`` before any writes.
effective_from:
UTC datetime at which the first pricing version takes effect.
Returns
-------
EnergyContract
The newly created contract (not yet committed).
Raises
------
ProfileNotFoundError
If no YAML profile exists for *kind*.
ProfileValidationError
If *values* does not conform to the profile structure.
"""
# Validate (and fill defaults) before any DB write.
filled_values = validate_values(kind, values)
now = datetime.now(UTC)
contract = EnergyContract(
name=name,
kind=kind,
currency=currency,
active=False, # New contracts are inactive; caller must explicitly activate.
created_at=now,
updated_at=now,
)
session.add(contract)
session.flush() # Assign contract.id so we can reference it in the version FK.
version = EnergyContractVersion(
contract_id=contract.id,
effective_from=effective_from,
effective_to=None,
values=filled_values,
created_at=now,
)
session.add(version)
logger.info("Created energy contract %r (kind=%s, id=%d)", name, kind, contract.id)
return contract
def add_version(
session: Session,
contract: EnergyContract,
*,
effective_from: datetime,
values: dict[str, Any],
) -> EnergyContractVersion:
"""Add a new pricing version to an existing contract (append-only).
The previous open version's ``effective_to`` is automatically set to the
new version's ``effective_from`` (version closure), ensuring there is never
a gap or overlap between consecutive versions.
The new ``effective_from`` **must be strictly greater than** the previous
open version's ``effective_from``. Equal timestamps are rejected because
they would produce two versions starting at the same instant, making it
impossible to determine which is current. If this constraint is not met,
``ContractVersionError`` is raised and **no rows are written**.
Parameters
----------
session:
Active SQLAlchemy session. Caller must commit after this returns.
contract:
The parent ``EnergyContract`` to append a version to.
effective_from:
UTC datetime at which this version's pricing takes effect.
values:
New pricing values dict conforming to the contract's profile.
Returns
-------
EnergyContractVersion
The newly created version (not yet committed).
Raises
------
ContractVersionError
If *effective_from* is not strictly after the previous open version's
``effective_from``.
ProfileNotFoundError
If no YAML profile exists for the contract's kind.
ProfileValidationError
If *values* does not conform to the profile structure.
"""
# Validate (and fill defaults) before any DB write.
filled_values = validate_values(contract.kind, values)
prev = _open_version(session, contract)
if prev is not None:
# Enforce strictly-after constraint to prevent ambiguous overlap.
# Use _as_utc() on both sides: the incoming value may be tz-aware
# while the DB-read value is tz-naive (SQLite limitation).
if _as_utc(effective_from) <= _as_utc(prev.effective_from):
raise ContractVersionError(
f"New version effective_from ({effective_from.isoformat()}) must be strictly "
f"after the previous open version's effective_from "
f"({prev.effective_from.isoformat()})."
)
# Close the previous open version.
prev.effective_to = effective_from
now = datetime.now(UTC)
new_version = EnergyContractVersion(
contract_id=contract.id,
effective_from=effective_from,
effective_to=None,
values=filled_values,
created_at=now,
)
session.add(new_version)
logger.info(
"Added version to contract id=%d (kind=%s, effective_from=%s)",
contract.id,
contract.kind,
effective_from.isoformat(),
)
return new_version
def activate_contract(session: Session, contract: EnergyContract) -> None:
"""Activate a contract with mutual exclusion.
Sets every other contract's ``active`` flag to False, then sets the given
contract's ``active`` to True. This guarantees at most one active contract
at any time.
Caller must commit after this returns.
"""
# Deactivate all contracts (including the target; we re-activate below).
for other in session.execute(select(EnergyContract)).scalars().all():
other.active = False
contract.active = True
contract.updated_at = datetime.now(UTC)
logger.info("Activated contract %r (id=%d)", contract.name, contract.id)
def deactivate_contract(session: Session, contract: EnergyContract) -> None:
"""Deactivate a contract without touching other contracts.
Caller must commit after this returns.
"""
contract.active = False
contract.updated_at = datetime.now(UTC)
logger.info("Deactivated contract %r (id=%d)", contract.name, contract.id)
def active_contract_versions(session: Session) -> list[EnergyContractVersion]:
"""Return all versions of the currently active contract, ordered by effective_from ascending.
Returns an empty list when there is no active contract. The list spans the
full history of the active contract (all closed + the current open version)
and is used to iterate over pricing-rate segments for cross-version fixed-
cost / credit accumulation (Principle C).
"""
active = session.execute(
select(EnergyContract).where(EnergyContract.active.is_(True)).limit(1)
).scalar_one_or_none()
if active is None:
return []
return list(
session.execute(
select(EnergyContractVersion)
.where(EnergyContractVersion.contract_id == active.id)
.order_by(EnergyContractVersion.effective_from)
)
.scalars()
.all()
)
def active_contract_version_at(
session: Session, ts: datetime
) -> EnergyContractVersion | None:
"""Return the active contract's version that covers *ts*.
A version covers *ts* when:
``effective_from ≤ ts`` AND (``effective_to IS NULL`` OR ``ts < effective_to``)
Returns None when:
- There is no active contract.
- The active contract has no version covering *ts*.
Parameters
----------
session:
Active SQLAlchemy session (read-only usage).
ts:
UTC datetime to look up.
Returns
-------
EnergyContractVersion | None
"""
active = session.execute(
select(EnergyContract).where(EnergyContract.active.is_(True)).limit(1)
).scalar_one_or_none()
if active is None:
return None
# Build a query for all versions of the active contract covering ts.
stmt = (
select(EnergyContractVersion)
.where(
EnergyContractVersion.contract_id == active.id,
EnergyContractVersion.effective_from <= ts,
)
.order_by(EnergyContractVersion.effective_from.desc())
.limit(1)
)
version = session.execute(stmt).scalar_one_or_none()
if version is None:
return None
# Exclude versions whose effective window has already closed before ts.
if version.effective_to is not None and _as_utc(ts) >= _as_utc(version.effective_to):
return None
return version
+297
View File
@@ -0,0 +1,297 @@
"""DSMR telegram ingest service.
Subscribes to the DSMR Reader MQTT topic (``dsmr/json``) and persists
down-sampled DSMR telegram frames to the ``dsmr_reading`` table.
Design decisions
----------------
- **Whole-frame storage**: the entire parsed telegram dict is stored as a JSON
blob in ``DsmrReading.payload``; no field allow-list is applied. This lets
future commodities (gas, heating, three-phase) be accommodated without a
table-schema change.
- **10-second down-sampling** (configurable via ``dsmr_sample_interval_s``):
only telegrams whose ``timestamp`` second falls on an exact multiple of the
interval are persisted. This reduces write volume from ~60 rows/min to ~6
rows/min while guaranteeing that every 15-minute boundary (second=00) is
captured.
- **Idempotency**: the telegram's own ``id`` field is stored as ``source_id``
with a UNIQUE constraint. A second delivery of the same telegram (e.g. after
a broker reconnect) is silently skipped.
- **Network-thread safety**: ``handle_message`` is called from paho's background
loop thread. It opens and closes its own short-lived SQLAlchemy session and
swallows all exceptions so that a buggy payload or transient DB error never
crashes the paho loop or drops the MQTT connection.
- **Numeric values kept as strings**: the DSMR Reader emits all numeric readings
as JSON strings (e.g. ``"20915.154"``). They are stored verbatim; conversion
to ``Decimal`` is deferred to the billing engine (T07) where precision matters.
- **Null phases**: some telegrams omit certain phase readings (``null`` in JSON);
these are stored as-is without special handling.
"""
from __future__ import annotations
import json
import logging
import threading
from datetime import datetime, timezone
from typing import TYPE_CHECKING
import sqlalchemy.exc
from sqlalchemy import select
from app.db import get_session_local
from app.models.energy import DsmrReading
if TYPE_CHECKING:
from app.config import Settings
logger = logging.getLogger(__name__)
# Tracks the DSMR topic currently subscribed via the MQTT manager, so a config
# change can unsubscribe the old topic before subscribing the new one.
_current_dsmr_topic: str | None = None
# Tracks the DSMR tariff topic currently subscribed via the MQTT manager.
_current_tariff_topic: str | None = None
# Current electricity tariff: 1 = dal/off-peak, 2 = normal/peak.
# Written by the paho network thread, read by the publish job — guarded by a lock.
_current_tariff: int | None = None
_tariff_lock = threading.Lock()
def get_current_tariff() -> int | None:
"""Return the most recently received electricity tariff (1 or 2), or None."""
with _tariff_lock:
return _current_tariff
def set_current_tariff(value: int | None) -> None:
"""Set the current electricity tariff (1 or 2), or clear it with None."""
with _tariff_lock:
global _current_tariff
_current_tariff = value
def handle_tariff_message(payload_bytes: bytes) -> None:
"""Parse one DSMR tariff MQTT payload and update the in-memory tariff state.
Called from the paho network thread; *must* swallow all exceptions so that
a bad payload never crashes the loop or drops the broker connection.
Accepts payload as bytes or str (paho can deliver either). Ignores
whitespace. Only accepts integer values 1 or 2; anything else is discarded
and the previous known tariff is preserved.
Parameters
----------
payload_bytes:
Raw bytes from the MQTT message (may also be a str in some paho versions).
"""
try:
# Decode bytes → str if needed; strip surrounding whitespace.
if isinstance(payload_bytes, (bytes, bytearray)):
raw = payload_bytes.decode("utf-8", errors="replace").strip()
else:
raw = str(payload_bytes).strip()
value = int(raw)
if value not in (1, 2):
logger.debug(
"dsmr_ingest.handle_tariff_message: unexpected tariff value %d (expected 1 or 2, ignored).",
value,
)
return
set_current_tariff(value)
logger.debug("dsmr_ingest.handle_tariff_message: tariff updated to %d.", value)
except Exception:
# Malformed payload (e.g. non-numeric); swallow silently to protect network thread.
logger.debug(
"dsmr_ingest.handle_tariff_message: could not parse payload %r (ignored).",
payload_bytes,
)
def apply_dsmr_subscription(settings: "Settings") -> None:
"""(Re)apply the DSMR MQTT subscriptions to match *settings* — restart-free.
Call this at startup and after every config save. It makes the live MQTT
subscriptions reflect the current ``dsmr_ingest_enabled`` / ``dsmr_mqtt_topic``
/ ``dsmr_sample_interval_s`` / ``dsmr_tariff_topic`` settings without an app
restart:
- **Disabled** → unsubscribe any active DSMR and tariff subscriptions.
- **Enabled** → (re)subscribe to ``dsmr_mqtt_topic`` with a handler bound to
a *fresh* settings snapshot, so a changed sample interval also takes effect.
Also subscribe to ``dsmr_tariff_topic`` when non-empty.
- **Topic changed** → unsubscribe the old topic before subscribing the new one.
Idempotent and safe to call when MQTT is not connected (the subscription is
queued in the manager and established on the next connect).
"""
# Imported here (not at module top) to avoid a circular import at app start.
from app.integrations.mqtt import mqtt_manager
global _current_dsmr_topic, _current_tariff_topic
if not settings.dsmr_ingest_enabled:
if _current_dsmr_topic is not None:
mqtt_manager.unsubscribe(_current_dsmr_topic)
logger.info("DSMR ingest disabled — unsubscribed from topic=%s.", _current_dsmr_topic)
_current_dsmr_topic = None
if _current_tariff_topic is not None:
mqtt_manager.unsubscribe(_current_tariff_topic)
logger.info(
"DSMR ingest disabled — unsubscribed from tariff topic=%s.",
_current_tariff_topic,
)
_current_tariff_topic = None
return
# --- Main DSMR telegram topic ---
topic = settings.dsmr_mqtt_topic
if _current_dsmr_topic is not None and _current_dsmr_topic != topic:
mqtt_manager.unsubscribe(_current_dsmr_topic)
# Re-subscribe (overwrites any existing handler for this topic) with a fresh
# settings snapshot so dsmr_sample_interval_s changes take effect too.
snapshot = settings
mqtt_manager.subscribe(topic, lambda payload: handle_message(payload, snapshot))
_current_dsmr_topic = topic
logger.info("DSMR ingest enabled — subscribed to topic=%s.", topic)
# --- DSMR tariff topic (dual-tariff slot indicator) ---
tariff_topic = settings.dsmr_tariff_topic if settings.dsmr_tariff_topic else ""
if tariff_topic:
if _current_tariff_topic is not None and _current_tariff_topic != tariff_topic:
mqtt_manager.unsubscribe(_current_tariff_topic)
mqtt_manager.subscribe(tariff_topic, lambda payload: handle_tariff_message(payload))
_current_tariff_topic = tariff_topic
logger.info("DSMR tariff topic — subscribed to topic=%s.", tariff_topic)
else:
# tariff_topic is empty → unsubscribe any existing tariff subscription.
if _current_tariff_topic is not None:
mqtt_manager.unsubscribe(_current_tariff_topic)
logger.info(
"DSMR tariff topic cleared — unsubscribed from topic=%s.",
_current_tariff_topic,
)
_current_tariff_topic = None
def handle_message(payload_bytes: bytes, settings: "Settings") -> None:
"""Parse one DSMR MQTT payload and persist it if it passes the sample filter.
Called from the paho network thread; *must* swallow all exceptions so that
a bad payload or transient error does not crash the loop or drop the broker
connection.
Parameters
----------
payload_bytes:
Raw bytes from the MQTT message.
settings:
Runtime settings snapshot (captured at subscription time). Used for
``dsmr_sample_interval_s``.
"""
try:
_handle_message_inner(payload_bytes, settings)
except Exception:
logger.exception("dsmr_ingest.handle_message: unexpected error (swallowed).")
def _handle_message_inner(payload_bytes: bytes, settings: "Settings") -> None:
"""Inner implementation — may raise; caller wraps in try/except."""
# --- 1. Parse JSON ---
try:
data: dict = json.loads(payload_bytes)
except (json.JSONDecodeError, ValueError):
logger.debug("dsmr_ingest: invalid JSON payload (skipped).")
return
if not isinstance(data, dict):
logger.debug("dsmr_ingest: payload is not a JSON object (skipped).")
return
# --- 2. Parse timestamp ---
raw_ts = data.get("timestamp")
if raw_ts is None:
logger.debug("dsmr_ingest: missing 'timestamp' field (skipped).")
return
try:
# Python 3.11+ accepts the trailing 'Z' directly; for 3.10 compat we
# replace 'Z' with '+00:00' before parsing.
ts_str = raw_ts if not isinstance(raw_ts, str) else raw_ts.replace("Z", "+00:00")
ts_utc: datetime = datetime.fromisoformat(ts_str)
# Ensure it is timezone-aware UTC.
if ts_utc.tzinfo is None:
ts_utc = ts_utc.replace(tzinfo=timezone.utc)
except (ValueError, TypeError, AttributeError):
logger.debug(
"dsmr_ingest: cannot parse 'timestamp' value %r (skipped).", raw_ts
)
return
# --- 3. Down-sample: only persist if second falls on interval boundary ---
interval = settings.dsmr_sample_interval_s
if interval > 0 and (ts_utc.second % interval) != 0:
# This telegram is between sample points; discard silently.
return
# --- 4. Extract source_id (telegram's own id) — stored only as a reference,
# NOT used for uniqueness/idempotency (it overflows and gets reset). ---
source_id: int | None = data.get("id")
if source_id is not None and not isinstance(source_id, int):
# Unexpected type — treat as missing rather than raising.
logger.debug(
"dsmr_ingest: 'id' field has unexpected type %s (ignoring).",
type(source_id).__name__,
)
source_id = None
# --- 5. Persist to database ---
# Idempotency is keyed on recorded_at (the telegram timestamp), which is
# telegram-id-independent: a single P1 meter emits one telegram per second,
# and down-sampling keeps at most one per interval-aligned second. The
# UNIQUE(recorded_at) constraint is the backstop for the IntegrityError race.
session_local = get_session_local()
session = session_local()
try:
existing = session.scalar(
select(DsmrReading).where(DsmrReading.recorded_at == ts_utc)
)
if existing is not None:
logger.debug(
"dsmr_ingest: recorded_at=%s already in DB, skipping.",
ts_utc.isoformat(),
)
return
reading = DsmrReading(
recorded_at=ts_utc,
source_id=source_id,
payload=data, # full frame, verbatim
)
session.add(reading)
session.commit()
logger.debug(
"dsmr_ingest: persisted reading recorded_at=%s source_id=%s.",
ts_utc.isoformat(),
source_id,
)
except sqlalchemy.exc.IntegrityError:
# Race / duplicate: another insert beat us to the same recorded_at.
session.rollback()
logger.debug(
"dsmr_ingest: IntegrityError for recorded_at=%s (duplicate, skipped).",
ts_utc.isoformat(),
)
except Exception:
session.rollback()
logger.exception("dsmr_ingest: DB error (swallowed).")
finally:
session.close()
+894
View File
@@ -0,0 +1,894 @@
"""Billing engine for DSMR 15-minute energy metering periods.
This module implements the two-layer billing model described in §3.4 of the
M6 design document, extended in M7-T03 to be meter-aware:
**Layer 1 — per-period metering cost (immutable, price-snapshot)**
``compute_period(session, t0)`` computes the import cost, export revenue, and
net cost for the 15-minute period ``[t0, t0+15min)``. The result is written
to ``energy_cost_period`` with a full pricing snapshot so each row is
self-contained and auditable. Existing *successful* rows are never overwritten
by the normal tick path; only an explicit ``recompute_range`` call passes
``overwrite=True``.
**Layer 2 — summary (computed at read time, not stored)**
``summarize(session, start, end)`` aggregates all non-degraded
``energy_cost_period`` rows in ``[start, end)``, then adds the daily
standing charges (network_fee + management_fee, apportioned at EUR/month
÷ 30 per day) and subtracts the energy-tax credit (heffingskorting,
apportioned at EUR/year ÷ 365 per day).
Design notes
------------
- **Decimal arithmetic throughout**: all monetary computations use
``decimal.Decimal`` to avoid float binary rounding errors. Only when
writing to ``EnergyCostPeriod`` columns (Float) are values converted to
float. ``summarize`` converts back to Decimal for summation.
- **UTC quarter-hour grid**: period boundaries are aligned to UTC 00/15/30/45
minutes (``floor_to_quarter``). NL local time (CET/CEST) is always a whole
number of hours from UTC, so the quarter-hour grid is the same in both
timezone representations.
- **Register keys**: DSMR payload uses JSON strings like ``"20915.154"``
for cumulative kWh registers. ``register_at`` converts them to Decimal.
- **Degraded vs skip semantics**:
- *No meter coverage* (``meter_at`` returns None for t0): write a
``degraded=True`` row with ``meter_id=None``.
- *Cross-meter boundary* (m0.id != m1.id for t0/t1): write a ``degraded=True``
row with ``meter_id=m0.id``; losing this one period at the swap boundary is
acceptable (D5 decision).
- *Missing readings* (``register_at`` returns None for start or end
boundary within the meter window): write a ``degraded=True`` row with
``meter_id=m0.id`` so the period is tracked and can be retried by
``compute_closed_periods``.
- *Negative or excessively large delta* (delta sanity guard D6): write a
``degraded=True`` row with ``meter_id=m0.id``; prevents negative costs and
grossly inflated costs from meter resets, DSMR rollover, or data spikes.
- *Missing Tibber price* (``TibberPriceNotFoundError``): skip entirely (do
not write a row); the period will be retried once prices arrive.
- *Missing active contract version*: skip (no contract to compute against).
- **Meter-aware register lookup**: ``register_at`` now accepts a ``meter``
parameter and restricts the DSMR reading query to readings within
``[meter.started_at, meter.ended_at)`` (half-open), preventing old-meter
readings from leaking into a new-meter epoch.
- **Lookback window in ``compute_closed_periods``**: to avoid scanning all
historical DSMR data on every tick, the function looks back at most 7 days
from the current time. This covers typical short outages (no data / no
contract) while staying bounded. Periods older than 7 days must be
recovered via an explicit ``recompute_range`` call.
Meter-aware compute_period ordering rationale (M7-T03)
-------------------------------------------------------
The order of checks inside ``compute_period`` is:
1. **Immutability guard** (existing non-degraded row, overwrite=False) → return False.
2. **Meter determination** (m0 = meter_at(t0), m1 = meter_at(t1)):
- No meter (m0 is None) → write degraded, meter_id=None.
- Cross-meter boundary (m0.id != m1.id) → write degraded, meter_id=m0.id.
3. **Active contract version check** → skip (no write) if absent.
4. **Boundary register readings** within m0's window → write degraded if missing.
5. **Delta sanity guard** → write degraded if any delta < 0 or > _MAX_DELTA_KWH.
6. **Price strategy** → skip (no write) if Tibber price missing.
7. **Upsert billing record** with meter_id=m0.id.
Why meter before contract? The meter is a *structural* prerequisite: without a
known meter epoch we cannot trust the delta at all, so we commit a degraded row
immediately. The contract skip, by contrast, is transient (the period can be
re-computed once a contract is configured), so it produces no row.
"""
from __future__ import annotations
import logging
from datetime import UTC, datetime, timedelta
from decimal import Decimal
from typing import Any
from sqlalchemy import select
from sqlalchemy.orm import Session
from app.integrations.pricing.strategies import (
PeriodDeltas,
TibberPriceNotFoundError,
get_strategy,
)
from app.models.energy import DsmrReading, EnergyCostPeriod, Meter
from app.services.contracts import active_contract_version_at, active_contract_versions
from app.services.meters import meter_at
from app.services.timezone import local_date, local_now
logger = logging.getLogger(__name__)
# ---------------------------------------------------------------------------
# Constants
# ---------------------------------------------------------------------------
_PERIOD_MINUTES = 15
_LOOKBACK_DAYS = 7 # maximum lookback window for compute_closed_periods
# Maximum age a DSMR reading may have relative to the boundary being queried.
# Under normal operation DSMR readings arrive every ~10 seconds, so a reading
# more than one period (15 minutes) old at the boundary indicates either a data
# gap or — critically — a *future* boundary being resolved against the last
# historical reading. In both cases the reading is considered stale and
# ``register_at`` returns None, letting the period be marked degraded instead of
# producing a spurious zero-delta "successful" row.
_READING_MAX_STALENESS = timedelta(minutes=_PERIOD_MINUTES)
# Maximum plausible kWh delta for a single 15-minute period (D6 sanity guard).
# A typical Dutch household uses well under 5 kWh per quarter hour even under
# heavy load. 100 kWh per 15 minutes corresponds to ~400 kW — far beyond any
# residential consumption — but is lenient enough to never fire on legitimate
# data. Any delta at or above this threshold indicates a meter reset, DSMR
# rollover, sign error, or other data anomaly, and the period is marked
# degraded to prevent negative costs or grossly inflated charges.
_MAX_DELTA_KWH = Decimal("100")
# 每日固定费/税补在"本地午夜后多久"才结算入账。延后到 01:05 是为了让累计成本的
# 整天阶跃落在新一天、且避开 01:00 整点(HA 长期统计的小时桶边界)。
_SETTLEMENT_OFFSET = timedelta(hours=1, minutes=5)
# DSMR payload register keys (cumulative kWh, JSON string values).
_KEY_D1 = "electricity_delivered_1" # delivered low-tariff (dal / _1)
_KEY_D2 = "electricity_delivered_2" # delivered high-tariff (normal / _2)
_KEY_R1 = "electricity_returned_1" # returned low-tariff
_KEY_R2 = "electricity_returned_2" # returned high-tariff
# ---------------------------------------------------------------------------
# Internal helpers
# ---------------------------------------------------------------------------
def floor_to_quarter(dt: datetime) -> datetime:
"""Return *dt* floored to the nearest UTC quarter-hour boundary.
The result always has seconds=0 and microseconds=0, and minutes in
{0, 15, 30, 45}. Timezone info is preserved if present.
"""
floored_minute = (dt.minute // _PERIOD_MINUTES) * _PERIOD_MINUTES
return dt.replace(minute=floored_minute, second=0, microsecond=0)
def _to_decimal(value: Any) -> Decimal:
"""Convert *value* to Decimal via str() to avoid float binary rounding."""
return Decimal(str(value))
def _as_utc(dt: datetime) -> datetime:
"""Attach UTC tzinfo to a naive datetime (SQLite read-back workaround)."""
if dt.tzinfo is None:
return dt.replace(tzinfo=UTC)
return dt
def _existing_period(session: Session, t0: datetime) -> EnergyCostPeriod | None:
"""Return the EnergyCostPeriod row for period_start=t0, or None."""
return session.execute(
select(EnergyCostPeriod).where(EnergyCostPeriod.period_start == t0)
).scalar_one_or_none()
# ---------------------------------------------------------------------------
# register_at — boundary reading lookup (meter-aware)
# ---------------------------------------------------------------------------
def register_at(
session: Session,
boundary: datetime,
meter: Meter,
) -> dict[str, Decimal] | None:
"""Return the four cumulative kWh register values at *boundary*, within *meter*'s window.
Queries the most recent ``DsmrReading`` with:
``recorded_at ≤ boundary``
AND ``recorded_at ≥ meter.started_at``
AND (``meter.ended_at IS NULL`` OR ``recorded_at < meter.ended_at``)
The meter window constraint (half-open ``[started_at, ended_at)``) ensures
that readings from a previous meter epoch are never used to anchor a new
meter's computation. Without this guard, the final reading of the old meter
would be visible at the start of the new meter's epoch and produce a
cross-meter delta, defeating the isolation guarantee.
Extracts the four energy registers from ``payload``:
d1 — electricity_delivered_1 (delivered low-tariff / dal)
d2 — electricity_delivered_2 (delivered high-tariff / normal)
r1 — electricity_returned_1 (returned low-tariff)
r2 — electricity_returned_2 (returned high-tariff)
Returns
-------
dict[str, Decimal] with keys ``d1``, ``d2``, ``r1``, ``r2``, or ``None``
when:
- No ``DsmrReading`` row exists with ``recorded_at ≤ boundary`` within
*meter*'s epoch window.
- The most recent such reading is older than ``_READING_MAX_STALENESS``
relative to *boundary* (freshness guard).
- Any of the four register keys is absent from the payload.
- Any of the four register values is ``None`` (null in JSON).
SQLite naive datetime note
--------------------------
``recorded_at`` is stored as a naive UTC datetime in SQLite. Comparisons
against *boundary* (always tz-aware UTC) use ``_as_utc()`` for the
freshness check. The SQL ``WHERE`` clause comparisons work correctly
because SQLAlchemy's SQLite dialect strips tzinfo when binding parameters
(leaving the wall-clock UTC value unchanged), consistent with the storage
format.
"""
# Build the meter-window constraints: [started_at, ended_at).
meter_lower = meter.started_at # DsmrReading.recorded_at >= meter.started_at
meter_upper = meter.ended_at # DsmrReading.recorded_at < meter.ended_at (if set)
stmt = (
select(DsmrReading)
.where(
DsmrReading.recorded_at <= boundary,
DsmrReading.recorded_at >= meter_lower,
)
.order_by(DsmrReading.recorded_at.desc())
.limit(1)
)
# Apply the upper bound only when the meter is closed (ended_at is not None).
if meter_upper is not None:
stmt = stmt.where(DsmrReading.recorded_at < meter_upper)
row: DsmrReading | None = session.execute(stmt).scalar_one_or_none()
if row is None:
return None
# Freshness guard: reject readings that are too old relative to *boundary*.
# ``recorded_at`` is stored as a naive UTC datetime in SQLite; attach UTC
# tzinfo before comparing with *boundary* (which is always tz-aware UTC) to
# avoid an "offset-naive vs offset-aware" TypeError.
if _as_utc(row.recorded_at) < _as_utc(boundary) - _READING_MAX_STALENESS:
return None
payload = row.payload or {}
try:
d1_raw = payload[_KEY_D1]
d2_raw = payload[_KEY_D2]
r1_raw = payload[_KEY_R1]
r2_raw = payload[_KEY_R2]
except KeyError:
return None
if any(v is None for v in (d1_raw, d2_raw, r1_raw, r2_raw)):
return None
return {
"d1": _to_decimal(d1_raw),
"d2": _to_decimal(d2_raw),
"r1": _to_decimal(r1_raw),
"r2": _to_decimal(r2_raw),
}
# ---------------------------------------------------------------------------
# compute_period — single 15-minute period
# ---------------------------------------------------------------------------
def compute_period(session: Session, t0: datetime, *, overwrite: bool = False) -> bool:
"""Compute and upsert the billing record for the period ``[t0, t0+15min)``.
Parameters
----------
session:
Active SQLAlchemy session. Caller is responsible for committing.
t0:
UTC start of the 15-minute period. **Must** lie on a quarter-hour
grid boundary (minutes ∈ {0, 15, 30, 45}, seconds=0, microseconds=0).
overwrite:
If ``True``, overwrite an existing *successful* row (i.e. re-compute
even when a non-degraded record already exists). The normal tick path
always passes ``False``; only ``recompute_range`` passes ``True``.
Returns
-------
bool
``True`` if a record was written (inserted or updated), ``False`` if
the period was skipped (missing contract or missing Tibber price).
Side-effects
------------
- Inserts or updates an ``EnergyCostPeriod`` row keyed on ``period_start=t0``.
- If no meter covers t0 (``meter_at`` returns None for t0): inserts/updates
a degraded row with ``meter_id=None``.
- If the period spans a meter boundary (``meter_at(t0).id != meter_at(t1).id``):
inserts/updates a degraded row with ``meter_id=m0.id`` (D5 decision).
- If readings are missing at either boundary within the meter window:
inserts/updates a degraded row with ``meter_id=m0.id``.
- If any delta is negative or exceeds ``_MAX_DELTA_KWH`` (D6 sanity guard):
inserts/updates a degraded row with ``meter_id=m0.id``.
- If the active contract version is missing: **skips** (returns False, no write).
- If the Tibber price is missing (TibberPriceNotFoundError): **skips**
(returns False, no write).
"""
t1 = t0 + timedelta(minutes=_PERIOD_MINUTES)
now = datetime.now(UTC)
# Immutability guard: skip if a successful record already exists and we
# are not in overwrite mode.
existing = _existing_period(session, t0)
if existing is not None and not existing.degraded and not overwrite:
return False
# --- Meter determination (structural prerequisite, checked before contract) ---
#
# A missing or cross-boundary meter is a structural problem: we cannot trust
# the delta at all, so we write a degraded row immediately. This is different
# from the contract skip (transient, no write): the degraded row ensures the
# period appears in the history and can be revisited once the meter timeline
# is corrected and a recompute_range is triggered.
#
# Ordering rationale:
# 1. No meter (m0 is None) → degraded(meter_id=None): no epoch for t0.
# 2. Cross-meter boundary (m0.id != m1.id) → degraded(meter_id=m0.id): D5.
# 3. (Single meter, proceed) → contract check → readings → delta guard → price.
#
# We place meter before contract so that "cross-table period" is always
# marked degraded regardless of contract state. If we checked contract
# first, a missing-contract skip would silently discard the cross-table
# evidence; once a contract is added and recompute runs, the engine would
# incorrectly use cross-table reads.
m0 = meter_at(session, t0)
m1 = meter_at(session, t1)
if m0 is None:
# No meter epoch covers t0 — degraded with no meter attribution.
logger.debug(
"compute_period(%s): no active meter at t0 — writing degraded (meter_id=None).",
t0.isoformat(),
)
_upsert_degraded(session, t0, now, existing, meter_id=None)
return True
if m1 is None or m0.id != m1.id:
# Period spans a meter boundary or t1 has no meter. Degrade with m0's id
# (t0's meter attribution): the period's start belongs to m0's epoch.
logger.debug(
"compute_period(%s): period crosses meter boundary "
"(m0.id=%s, m1.id=%s) — writing degraded.",
t0.isoformat(),
m0.id,
m1.id if m1 is not None else None,
)
_upsert_degraded(session, t0, now, existing, meter_id=m0.id)
return True
# --- Active contract version at t0 ---
# If there is no active contract covering t0, skip the period entirely.
# We do not write a degraded row — there is no meaningful state to recover
# without a contract (we would not know which strategy to apply once data
# arrives). The period can be recovered via an explicit recompute_range once
# a contract is configured and activated.
version = active_contract_version_at(session, t0)
if version is None:
logger.debug("compute_period(%s): no active contract version — skipping.", t0.isoformat())
return False
# --- Boundary readings within m0's meter window ---
start_regs = register_at(session, t0, m0)
end_regs = register_at(session, t1, m0)
if start_regs is None or end_regs is None:
# Missing readings within the meter window → degraded with m0 attribution.
_upsert_degraded(session, t0, now, existing, meter_id=m0.id)
return True # a record was written (degraded)
# --- Compute deltas (end start) ---
deltas = PeriodDeltas(
d1=end_regs["d1"] - start_regs["d1"],
d2=end_regs["d2"] - start_regs["d2"],
r1=end_regs["r1"] - start_regs["r1"],
r2=end_regs["r2"] - start_regs["r2"],
)
# --- Delta sanity guard (D6) ---
# Any negative delta indicates a meter reset, DSMR rollover, or data error.
# Any delta exceeding _MAX_DELTA_KWH (100 kWh per 15 min = 400 kW average)
# is implausible for residential use and indicates an anomaly.
# Both cases produce a degraded row so no negative or grossly inflated cost
# is ever written to the billing record.
all_deltas = (deltas.d1, deltas.d2, deltas.r1, deltas.r2)
if any(d < Decimal("0") for d in all_deltas) or any(d > _MAX_DELTA_KWH for d in all_deltas):
logger.debug(
"compute_period(%s): delta sanity guard triggered "
"(d1=%s, d2=%s, r1=%s, r2=%s) — writing degraded.",
t0.isoformat(),
deltas.d1,
deltas.d2,
deltas.r1,
deltas.r2,
)
_upsert_degraded(session, t0, now, existing, meter_id=m0.id)
return True
# --- Price strategy ---
strategy = get_strategy(version.contract.kind)
try:
result = strategy(deltas, t0, version.values, session)
except TibberPriceNotFoundError:
# Missing Tibber price → skip the period; it will be retried once the
# price arrives (e.g. after the next Tibber refresh job runs).
logger.debug(
"compute_period(%s): no Tibber price found — skipping.", t0.isoformat()
)
return False
# --- Upsert the billing record ---
import_cost: Decimal = result["import_cost"]
export_revenue: Decimal = result["export_revenue"]
net_cost: Decimal = result["net_cost"]
pricing: dict = result["pricing"]
if existing is not None:
# Update in-place (overwrite=True or previous record was degraded).
existing.d1_kwh = float(deltas.d1)
existing.d2_kwh = float(deltas.d2)
existing.r1_kwh = float(deltas.r1)
existing.r2_kwh = float(deltas.r2)
existing.import_cost = float(import_cost)
existing.export_revenue = float(export_revenue)
existing.net_cost = float(net_cost)
existing.currency = version.contract.currency
existing.pricing = pricing
existing.contract_version_id = version.id
existing.meter_id = m0.id
existing.degraded = False
existing.computed_at = now
else:
period = EnergyCostPeriod(
period_start=t0,
d1_kwh=float(deltas.d1),
d2_kwh=float(deltas.d2),
r1_kwh=float(deltas.r1),
r2_kwh=float(deltas.r2),
import_cost=float(import_cost),
export_revenue=float(export_revenue),
net_cost=float(net_cost),
currency=version.contract.currency,
pricing=pricing,
contract_version_id=version.id,
meter_id=m0.id,
degraded=False,
computed_at=now,
)
session.add(period)
return True
def _upsert_degraded(
session: Session,
t0: datetime,
now: datetime,
existing: EnergyCostPeriod | None,
*,
meter_id: int | None,
) -> None:
"""Insert or update a degraded placeholder for period *t0*.
Parameters
----------
session:
Active SQLAlchemy session.
t0:
UTC start of the 15-minute period.
now:
Current UTC timestamp for the ``computed_at`` field.
existing:
The existing ``EnergyCostPeriod`` row for this period, or ``None``.
meter_id:
The meter ID to attribute this degraded period to, or ``None`` when
no meter epoch covers the period (no-meter degraded case).
When *existing* is not None (row was previously written — either degraded
or successful), the row is explicitly reset to the standard degraded state.
This is required for the ``recompute_range`` (overwrite=True) path: if the
row was previously a *successful* computation and the boundary readings have
since disappeared, the stale non-zero costs must be cleared so the row
accurately reflects the current "missing readings" state rather than
masquerading as a valid result.
The ``meter_id`` is always updated to reflect the current meter attribution
judgment (the result of ``meter_at`` at the time of recompute). This
ensures that a retroactive ``started_at`` change + ``recompute_range`` will
re-attribute historical degraded periods to the correct meter epoch.
"""
if existing is not None:
# Explicitly reset to degraded state — identical field values to the
# new-row path below. This covers the recompute-over-successful-row
# case where old non-zero costs must not survive the downgrade.
existing.d1_kwh = 0.0
existing.d2_kwh = 0.0
existing.r1_kwh = 0.0
existing.r2_kwh = 0.0
existing.import_cost = 0.0
existing.export_revenue = 0.0
existing.net_cost = 0.0
existing.pricing = {}
existing.contract_version_id = None
existing.meter_id = meter_id
existing.degraded = True
existing.computed_at = now
else:
period = EnergyCostPeriod(
period_start=t0,
d1_kwh=0.0,
d2_kwh=0.0,
r1_kwh=0.0,
r2_kwh=0.0,
import_cost=0.0,
export_revenue=0.0,
net_cost=0.0,
currency="EUR", # placeholder; real currency known after contract lookup
pricing={},
contract_version_id=None,
meter_id=meter_id,
degraded=True,
computed_at=now,
)
session.add(period)
# ---------------------------------------------------------------------------
# compute_closed_periods — periodic tick
# ---------------------------------------------------------------------------
def compute_closed_periods(session: Session) -> int:
"""Find and compute all uncalculated closed 15-minute periods.
A period ``[t0, t1)`` is *closed* when ``t1 ≤ now``. This function:
1. Determines the lookback window: from ``now LOOKBACK_DAYS`` to ``now``,
floored to the nearest quarter-hour. This avoids an unbounded full
historical scan on every tick while still covering the typical recovery
window (short outages, missing contract, etc.). Periods older than
``LOOKBACK_DAYS`` must be recovered via an explicit ``recompute_range``.
2. Iterates over all quarter-hour boundaries in that window where
``t1 ≤ now`` (i.e. the period has already closed).
3. For each boundary, calls ``compute_period(overwrite=False)``, which:
- Skips periods that already have a *successful* (non-degraded) record.
- Retries periods that have a *degraded* record.
- Writes degraded rows for periods with no meter or cross-meter boundaries.
- Skips periods for which no active contract version exists or the
Tibber price is unavailable (without writing a degraded row).
Returns
-------
int
Number of periods for which a record was written (inserted or updated).
Does not count skipped periods.
"""
now = datetime.now(UTC)
# Current period boundary (the one whose t1 has not yet passed).
current_t0 = floor_to_quarter(now)
# Earliest boundary to consider.
earliest_t0 = floor_to_quarter(now - timedelta(days=_LOOKBACK_DAYS))
written = 0
t0 = earliest_t0
while t0 < current_t0:
t1 = t0 + timedelta(minutes=_PERIOD_MINUTES)
if t1 <= now:
try:
did_write = compute_period(session, t0, overwrite=False)
if did_write:
written += 1
except Exception:
logger.exception(
"compute_closed_periods: unexpected error for t0=%s — continuing.",
t0.isoformat(),
)
t0 += timedelta(minutes=_PERIOD_MINUTES)
if written:
session.commit()
logger.info("compute_closed_periods: wrote %d period(s).", written)
return written
# ---------------------------------------------------------------------------
# recompute_range — explicit full recompute
# ---------------------------------------------------------------------------
def recompute_range(session: Session, start: datetime, end: datetime) -> int:
"""Recompute (overwrite) all 15-minute periods in ``[start, end)``.
This is the *explicit opt-in* path for recovering from:
- Periods where readings or prices arrived late.
- Price corrections (new contract version retroactively applied).
- Retroactive meter changes (``update_meter`` with new ``started_at``) —
re-running this function will re-judge meter attribution and re-compute
costs using the corrected epoch boundaries.
- Any other reason to override the immutability guard.
The function iterates over every UTC quarter-hour boundary in
``[floor(start), end)`` and calls ``compute_period(overwrite=True)``.
Existing rows (including successful ones) are overwritten; their
``meter_id`` fields will reflect the *current* ``meter_at`` judgment for
each period's start timestamp, naturally re-attributing periods when meter
``started_at`` values have been retroactively corrected.
Parameters
----------
session:
Active SQLAlchemy session. The function commits after all periods
have been processed.
start:
Inclusive start datetime (floored to the nearest quarter-hour internally).
end:
Exclusive end datetime.
Returns
-------
int
Number of periods for which a record was written (inserted or updated).
Periods skipped due to missing contract or missing Tibber price are
*not* counted.
"""
t0 = floor_to_quarter(_as_utc(start))
end_utc = _as_utc(end)
now = datetime.now(UTC)
written = 0
while t0 < end_utc:
t1 = t0 + timedelta(minutes=_PERIOD_MINUTES)
# Only recompute periods that have already closed (t1 ≤ now). Even
# when the caller passes a future *end*, we must not write speculative
# "future" rows: register_at would resolve to the latest historical
# reading for both boundaries, producing a zero-delta fake-success row
# that blocks the real computation once the period actually closes.
if t1 <= now:
try:
did_write = compute_period(session, t0, overwrite=True)
if did_write:
written += 1
except Exception:
logger.exception(
"recompute_range: unexpected error for t0=%s — continuing.",
t0.isoformat(),
)
t0 += timedelta(minutes=_PERIOD_MINUTES)
session.commit()
logger.info(
"recompute_range(%s, %s): wrote %d period(s).",
start.isoformat(),
end.isoformat(),
written,
)
return written
# ---------------------------------------------------------------------------
# summarize — layer-2 aggregation (read-time, not stored)
# ---------------------------------------------------------------------------
def summarize(session: Session, start: datetime, end: datetime) -> dict[str, Any]:
"""Aggregate billing for the interval ``[start, end)``.
Computes the total payable as:
total_payable = Σ(net_cost) -- metered electricity
+ fixed_costs -- per-day standing charges, cross-version
- credits -- per-day heffingskorting, cross-version
**Fixed-cost / credit counting — Principle C (symmetric begin/end)**:
Both the local calendar day in which *start* falls and the local calendar
day in which *end* falls are counted as full days. Fixed charges
(network_fee, management_fee) and the energy-tax credit (heffingskorting)
are assessed on a "service-is-active" basis — if the meter was online on a
given calendar day, the full day's charge/credit applies, regardless of
whether the window starts at midnight or mid-morning.
For each local calendar date D in the range
``[local_date(start), min(local_date(end), today_local)]``:
- D ≤ today_local (only elapsed / today days count as "whole days").
- The contract version whose effective_from local-date ≤ D is used.
- This is cross-version: if V1 is from June 1 and V2 from June 25,
querying June 130 uses V1 for days 1-24 and V2 for day 25.
The end-day (last_counted) is included when the end's local midnight falls
strictly before end_utc; combined with the always-counted start day this
makes the begin/end handling symmetric. A short same-day window therefore
counts its single local day. A window contributes 0 days only when the
counted range is empty (first_counted > last_counted) — e.g. a window lying
entirely in the future, since last_counted is capped at today_local.
Daily getters (``*_today``) use windows exactly aligned to local midnight,
so their ``first_counted`` is always today — unaffected by this fix.
Days that have not yet started in local time (D > today_local) are
never counted. Switching versions never resets the counter.
**Timezone note**: the ``days`` field in the returned dict still represents
the window length in calendar days (total_seconds / 86400), for backward
compatibility with existing API consumers. The fixed/credit calculation
independently counts whole local days as described above.
All arithmetic uses Decimal; the returned dict contains Python floats for
JSON-serialisation convenience.
Parameters
----------
session:
Active read-only SQLAlchemy session.
start:
Inclusive start of the summary interval (UTC or naive-UTC).
end:
Exclusive end of the summary interval (UTC or naive-UTC).
Returns
-------
dict with keys:
currency str ISO 4217 currency (from contract, or "EUR" fallback)
metered_import float Σ import_cost from non-degraded periods
metered_export float Σ export_revenue from non-degraded periods
metered_net float Σ net_cost from non-degraded periods
fixed_costs float standing charges for elapsed whole local days
credits float energy-tax credit for elapsed whole local days
total_payable float metered_net + fixed_costs credits
period_count int number of non-degraded periods in range
degraded_count int number of degraded periods in range
days float interval length in days (total_seconds / 86400)
"""
from datetime import timedelta as _td, date as _date
start_utc = _as_utc(start)
end_utc = _as_utc(end)
# --- Fetch all EnergyCostPeriod rows in [start, end) ---
rows = session.execute(
select(EnergyCostPeriod).where(
EnergyCostPeriod.period_start >= start_utc,
EnergyCostPeriod.period_start < end_utc,
)
).scalars().all()
good_rows = [r for r in rows if not r.degraded]
degraded_rows = [r for r in rows if r.degraded]
# Σ monetary amounts (Decimal arithmetic).
sum_import = sum((_to_decimal(r.import_cost) for r in good_rows), Decimal("0"))
sum_export = sum((_to_decimal(r.export_revenue) for r in good_rows), Decimal("0"))
sum_net = sum((_to_decimal(r.net_cost) for r in good_rows), Decimal("0"))
# --- Interval length in days (window, not elapsed — kept for API compat) ---
total_seconds = (end_utc - start_utc).total_seconds()
days = _to_decimal(str(total_seconds)) / _to_decimal("86400")
# --- Fixed costs and credits: Principle C cross-version whole-day counting ---
_now_local = local_now()
today_local: _date = _now_local.date()
# --- Compute [first_counted, last_counted] local date range (inclusive) ---
#
# Both the window-start day and the window-end day are counted as complete
# local calendar days, regardless of whether the window starts/ends at midnight.
#
# Principle C (symmetric begin/end):
# • first_counted = local calendar date of start_utc (start day always counted)
# • last_counted = local calendar date of end_utc (end day counted if its
# local midnight is strictly before end_utc)
# • Both are then capped at today_local (only elapsed / today days count).
#
# Why symmetric? Fixed charges (network_fee, management_fee) and energy-tax credits
# (heffingskorting) are assessed on a "service-is-active" basis, not on how many
# hours the service was actually running within that calendar day. If the meter
# anchor (started_at) falls at 09:18 on June 24, the full June 24 standing charge
# and credit still apply because the service was online for that day.
#
# Previous asymmetric behaviour: a start_utc that was *later* than the local
# midnight of local_start_date caused first_counted to be bumped to the *next*
# day, silently dropping the anchor day's charges/credits. This was incorrect
# for cumulative entities (import_cost_total / export_revenue_total) whose anchor
# is often an above-midnight started_at. The end-side had always been symmetric
# (counted if local midnight < end_utc), creating an inconsistency.
#
# Daily getters (window = [local today 00:00, local tomorrow 00:00)) are
# unaffected: local_date(local_midnight_utc(today)) == today, so first_counted
# is still today regardless of the fix.
#
# A short same-day window now counts its single local day (the start day is
# always counted, symmetric with the end side). A window contributes 0 days
# only when the counted range is empty (first_counted > last_counted) — e.g. a
# window lying entirely in the future, where last_counted is capped at
# today_local while first_counted is later.
from app.services.timezone import local_midnight_utc as _lmu
# Start side: always count the local calendar day in which start_utc falls.
first_counted: _date = local_date(start_utc)
local_end_date: _date = local_date(end_utc)
# Is the midnight of local_end_date < end_utc? If yes, that day's midnight is in range.
if _lmu(local_end_date) < end_utc:
last_counted: _date = local_end_date
else:
last_counted = local_end_date - _td(days=1)
# Settlement cap: "today" is only counted once the local clock has passed the
# settlement offset since midnight (01:05). This defers the daily standing-charge
# step from 00:00 to 01:05, ensuring the cumulative-cost adiabatic jump lands
# inside the new calendar day and avoids the HA 01:00 hourly-bucket boundary.
_today_midnight_utc = _lmu(today_local)
if _now_local >= _today_midnight_utc + _SETTLEMENT_OFFSET:
settled_cap = today_local
else:
settled_cap = today_local - _td(days=1)
last_counted = min(last_counted, settled_cap)
# If the range is empty (first_counted > last_counted), no days are counted.
# Fetch all versions of the active contract once (single DB call).
versions = active_contract_versions(session)
fixed_dec = Decimal("0")
credits_dec = Decimal("0")
currency = "EUR"
if versions:
# Currency comes from the contract regardless of day count.
currency = versions[0].contract.currency
if versions and first_counted <= last_counted:
# For each version segment, find its overlap with [first_counted, last_counted]
# and accumulate whole days.
version_segments: list[tuple[_date, _date | None, dict]] = []
for v in versions:
v_start_local = local_date(_as_utc(v.effective_from))
v_end_local = local_date(_as_utc(v.effective_to)) if v.effective_to is not None else None
version_segments.append((v_start_local, v_end_local, v.values or {}))
for v_start, v_end_excl, v_values in version_segments:
# The version covers [v_start, v_end_excl) in local dates,
# where v_end_excl=None means open-ended (no upper bound).
# Intersection with [first_counted, last_counted]:
seg_start = max(first_counted, v_start)
if v_end_excl is not None:
seg_end_excl = min(last_counted + _td(days=1), v_end_excl)
else:
seg_end_excl = last_counted + _td(days=1)
if seg_start >= seg_end_excl:
continue # no overlap
n_days = (seg_end_excl - seg_start).days
if n_days <= 0:
continue
standing: dict = v_values.get("standing", {})
creds: dict = v_values.get("credits", {})
network_fee = _to_decimal(standing.get("network_fee", 0))
management_fee = _to_decimal(standing.get("management_fee", 0))
heffingskorting = _to_decimal(creds.get("heffingskorting", 0))
# Standing charges: EUR/month → EUR/day (÷ 30) × n_days.
fixed_dec += (network_fee + management_fee) / Decimal("30") * Decimal(str(n_days))
# Energy-tax credit: EUR/year → EUR/day (÷ 365) × n_days.
credits_dec += heffingskorting / Decimal("365") * Decimal(str(n_days))
total_payable = sum_net + fixed_dec - credits_dec
return {
"currency": currency,
"metered_import": float(sum_import),
"metered_export": float(sum_export),
"metered_net": float(sum_net),
"fixed_costs": float(fixed_dec),
"credits": float(credits_dec),
"total_payable": float(total_payable),
"period_count": len(good_rows),
"degraded_count": len(degraded_rows),
"days": float(days),
}
+114 -23
View File
@@ -15,12 +15,17 @@ Design
doing any work. Callers never need to guard these conditions.
- **Stable unique_id**: derived from device ``uuid`` + metric ``key``, never
from mutable fields like ``friendly_name`` or auto-increment DB ids.
- **Discovery topic format**: ``<prefix>/<component>/<node_id>/<object_id>/config``
- **Discovery topic format**: ``<discovery_prefix>/<component>/<node_id>/<object_id>/config``
where ``node_id`` is the device uuid (slugified to be safe) and
``object_id`` is a uuid-prefixed stable string.
- **State topic format**: ``<prefix>/<component>/<node_id>/<object_id>/state``
- **Availability topic**: ``<prefix>/modbus/<node_id>/availability`` (shared
across all entities of the same device; publishes "online"/"offline").
``object_id`` is a uuid-prefixed stable string. Uses ``ha_discovery_prefix``
(default ``"homeassistant"``), which must stay under the HA discovery namespace.
- **State topic format**: ``<state_prefix>/<component>/<node_id>/<object_id>/state``
Uses ``ha_state_topic_prefix`` (default ``"home_automation"``), separate from
the discovery namespace so HA discovery and state/availability topics live under
different prefixes.
- **Availability topic**: ``<state_prefix>/modbus/<node_id>/availability`` (shared
across all entities of the same device; publishes "online"/"offline"). Also uses
the state prefix, not the discovery prefix.
- **best-effort**: functions catch all exceptions internally so that callers
(e.g. the Modbus poll loop) never crash due to MQTT publish errors.
"""
@@ -107,7 +112,8 @@ def _unique_id(entity: ExposableEntity) -> str:
def build_discovery_payload(
entity: ExposableEntity,
prefix: str,
discovery_prefix: str,
state_prefix: str | None = None,
) -> tuple[str, dict[str, Any]]:
"""Build the HA MQTT Discovery config topic and payload dict for *entity*.
@@ -115,8 +121,14 @@ def build_discovery_payload(
----------
entity:
An ``ExposableEntity`` (from ``build_catalog``).
prefix:
The discovery prefix (e.g. ``"homeassistant"``).
discovery_prefix:
The HA Discovery config topic prefix (e.g. ``"homeassistant"``).
This determines where HA looks for the discovery config topic.
state_prefix:
The prefix used for state and availability topics (e.g.
``"home_automation"``). When omitted or ``None``, falls back to
``discovery_prefix`` for backward compatibility (e.g. in tests that
pass only one prefix).
Returns
-------
@@ -124,23 +136,33 @@ def build_discovery_payload(
``(topic, config_dict)`` — the config topic and the HA Discovery
payload (not yet JSON-serialised).
"""
if state_prefix is None:
state_prefix = discovery_prefix
device_uuid = entity.device.identifiers[1]
avail_topic = _availability_topic(device_uuid, prefix)
state_t = _state_topic(entity, prefix)
topic = _discovery_topic(entity, prefix)
avail_topic = _availability_topic(device_uuid, state_prefix)
state_t = _state_topic(entity, state_prefix)
topic = _discovery_topic(entity, discovery_prefix)
config: dict[str, Any] = {
"unique_id": _unique_id(entity),
"name": entity.name,
"state_topic": state_t,
"availability": [{"topic": avail_topic}],
"availability_mode": "all",
"device": {
"identifiers": list(entity.device.identifiers),
"name": entity.device.name,
},
}
# Only declare an availability topic for devices that actually publish an
# online/offline heartbeat (e.g. Modbus, via its "online" binary_sensor).
# Devices without a heartbeat (e.g. energy-cost) omit availability so HA
# treats their entities as always-available — otherwise HA would mark them
# ``unavailable`` forever even though their state is being published.
if entity.device.provides_availability:
config["availability"] = [{"topic": avail_topic}]
config["availability_mode"] = "all"
# Component-specific fields
if entity.component == "binary_sensor":
# "online" binary sensor: payload ON/OFF
@@ -181,7 +203,8 @@ def publish_discovery(session: Session) -> None:
logger.debug("publish_discovery: skipped (MQTT/Discovery not enabled or not connected)")
return
prefix = settings.ha_discovery_prefix
discovery_prefix = settings.ha_discovery_prefix
state_prefix = settings.ha_state_topic_prefix
try:
catalog = build_catalog(session)
@@ -192,7 +215,7 @@ def publish_discovery(session: Session) -> None:
for entry in catalog:
entity = entry.entity
try:
topic, config = build_discovery_payload(entity, prefix)
topic, config = build_discovery_payload(entity, discovery_prefix, state_prefix)
if entry.enabled:
payload = json.dumps(config)
mqtt_manager.publish(topic, payload, retain=True)
@@ -234,7 +257,7 @@ def publish_states(session: Session) -> None:
logger.debug("publish_states: skipped (MQTT/Discovery not enabled or not connected)")
return
prefix = settings.ha_discovery_prefix
state_prefix = settings.ha_state_topic_prefix
try:
catalog = build_catalog(session)
@@ -248,7 +271,7 @@ def publish_states(session: Session) -> None:
continue
entity = entry.entity
try:
_publish_entity_state(entity, prefix, session)
_publish_entity_state(entity, state_prefix, session)
except Exception:
logger.exception(
"publish_states: error publishing state for %r; continuing", entity.key
@@ -322,13 +345,13 @@ def publish_device_state(session: Session, device: Any) -> None:
if not _should_publish(settings):
return
prefix = settings.ha_discovery_prefix
state_prefix = settings.ha_state_topic_prefix
device_uuid = device.uuid
online = bool(device.last_poll_ok)
# Publish availability topic first.
try:
avail_topic = _availability_topic(device_uuid, prefix)
avail_topic = _availability_topic(device_uuid, state_prefix)
avail_payload = "online" if online else "offline"
mqtt_manager.publish(avail_topic, avail_payload, retain=False)
except Exception:
@@ -360,7 +383,7 @@ def publish_device_state(session: Session, device: Any) -> None:
if entity.component == "binary_sensor" and "online" in entity.key:
# Publish the binary_sensor state too.
try:
state_t = _state_topic(entity, prefix)
state_t = _state_topic(entity, state_prefix)
mqtt_manager.publish(state_t, "ON", retain=False)
except Exception:
logger.exception(
@@ -376,7 +399,7 @@ def publish_device_state(session: Session, device: Any) -> None:
value = entity.value_getter(session)
if value is None:
continue
state_t = _state_topic(entity, prefix)
state_t = _state_topic(entity, state_prefix)
mqtt_manager.publish(state_t, str(value), retain=False)
except Exception:
logger.exception(
@@ -384,6 +407,74 @@ def publish_device_state(session: Session, device: Any) -> None:
)
def clear_device_discovery(session: Session, device_uuid: str) -> None:
"""Best-effort: send empty retained payloads to all HA Discovery config topics
for the given device, effectively removing the device's entities from HA.
This must be called **before** the device rows are deleted from the DB, so
that ``build_catalog`` can still enumerate the device's entities.
Behaviour
---------
- No-op if MQTT / HA Discovery is not enabled or the MQTT client is not
connected.
- All exceptions are caught internally; this function never raises.
The caller proceeds with the DB deletion regardless of MQTT outcome.
Parameters
----------
session:
Active SQLAlchemy session (device must still exist in DB at call time).
device_uuid:
UUID string of the device being deleted.
"""
from app.config import get_settings
settings = build_runtime_settings(session, get_settings())
if not _should_publish(settings):
logger.debug(
"clear_device_discovery: skipped (MQTT/Discovery not enabled or not connected)"
)
return
discovery_prefix = settings.ha_discovery_prefix
state_prefix = settings.ha_state_topic_prefix
try:
catalog = build_catalog(session)
except Exception:
logger.exception(
"clear_device_discovery: failed to build catalog for device uuid=%s; skipping HA cleanup",
device_uuid,
)
return
cleared = 0
for entry in catalog:
entity = entry.entity
# Only clear entities belonging to this device.
if entity.device.identifiers[1] != device_uuid:
continue
try:
topic, _config = build_discovery_payload(entity, discovery_prefix, state_prefix)
mqtt_manager.publish(topic, b"", retain=True)
cleared += 1
logger.debug(
"clear_device_discovery: cleared config topic for entity %r%s",
entity.key,
topic,
)
except Exception:
logger.exception(
"clear_device_discovery: error clearing entity %r; continuing", entity.key
)
logger.info(
"clear_device_discovery: cleared %d HA discovery topic(s) for device uuid=%s",
cleared,
device_uuid,
)
def publish_device_offline(session: Session, device_uuid: str) -> None:
"""Publish an "offline" availability payload for *device_uuid*.
@@ -399,8 +490,8 @@ def publish_device_offline(session: Session, device_uuid: str) -> None:
return
try:
prefix = settings.ha_discovery_prefix
avail_topic = _availability_topic(device_uuid, prefix)
state_prefix = settings.ha_state_topic_prefix
avail_topic = _availability_topic(device_uuid, state_prefix)
mqtt_manager.publish(avail_topic, "offline", retain=False)
except Exception:
logger.exception(
+480
View File
@@ -0,0 +1,480 @@
"""Service layer for Meter epoch CRUD, swap/close, and time-range lookup.
All functions accept an explicit SQLAlchemy Session; callers are responsible
for committing or rolling back the transaction.
Design decisions
----------------
- ``meter_at``: half-open interval ``[started_at, ended_at)`` lookup;
returns the meter whose epoch covers *ts* for the given commodity.
SQLite naive datetime is normalised via ``_as_utc()`` before comparison.
- ``declare_meter``: validates that the new ``started_at`` is **not earlier
than** the current active meter's ``started_at`` (rejects back-dating below
the active meter's own start). Equal timestamps are allowed because the
typical "swap now" use-case sets ``started_at`` to the current moment, which
coincides with the active meter's ``started_at`` only in degenerate test
scenarios — but blocking equal values would make that workflow impossible.
The old active meter is closed (``ended_at = started_at``) and a new active
meter is opened in the same operation, guaranteeing continuity: the old
meter's ``ended_at`` equals the new meter's ``started_at`` (contiguous,
no gap, no overlap).
- ``update_meter``: when ``started_at`` is modified, the service keeps the
timeline contiguous by also updating the **previous** meter's ``ended_at``
(the one whose ``ended_at`` matched the old ``started_at``) to the new
``started_at``. Validation ensures the new ``started_at``:
* is strictly after the previous meter's own ``started_at``
(cannot push the boundary before the previous meter even started);
* is strictly before the current meter's ``ended_at``, if set
(cannot push the boundary past where the current meter was already
closed).
- Mutual exclusion (at most one active meter per commodity) is enforced by the
service layer; no DB-level unique partial index is added to keep migrations
simple and to allow the application to return a meaningful error message.
SQLite timezone note
--------------------
SQLite stores ``DateTime(timezone=True)`` columns as naive UTC strings; on
read-back they come out as **timezone-naive** datetimes. Wherever this code
compares timestamps from the DB against timezone-aware values, it calls
``_as_utc()`` to make both sides comparable without tripping on
"offset-naive vs offset-aware" TypeErrors.
"""
from __future__ import annotations
import logging
from datetime import UTC, datetime
from typing import Optional
from sqlalchemy import select
from sqlalchemy.orm import Session
from app.models.energy import Meter
logger = logging.getLogger(__name__)
# ---------------------------------------------------------------------------
# Internal helpers
# ---------------------------------------------------------------------------
def _as_utc(dt: datetime) -> datetime:
"""Return *dt* as a timezone-aware UTC datetime.
SQLite's ``DateTime(timezone=True)`` column type stores datetimes as naive
UTC strings and gives them back as naive datetimes on read. This helper
re-attaches the UTC timezone info when it is missing, making cross-origin
comparisons safe.
"""
if dt.tzinfo is None:
return dt.replace(tzinfo=UTC)
return dt
# ---------------------------------------------------------------------------
# Custom exceptions
# ---------------------------------------------------------------------------
class MeterError(ValueError):
"""Base class for meter service validation errors."""
class MeterOverlapError(MeterError):
"""Raised when a new meter's ``started_at`` would create an overlap or backdate.
Specifically: the new meter's ``started_at`` must be greater than or equal
to the current active meter's ``started_at``. Allowing a value strictly
earlier than the active meter's start would mean the new epoch begins before
the current one, which is chronologically inconsistent.
"""
class MeterIntervalError(MeterError):
"""Raised when an ``update_meter`` call would produce an inconsistent interval.
Examples of inconsistent intervals:
- New ``started_at`` ≥ this meter's ``ended_at`` (epoch would be empty/inverted).
- New ``started_at`` ≤ the previous meter's own ``started_at`` (previous meter
would become empty/inverted after its ``ended_at`` is updated).
"""
# ---------------------------------------------------------------------------
# Internal query helpers
# ---------------------------------------------------------------------------
def _active_meter(session: Session, commodity: str) -> Optional[Meter]:
"""Return the current active (``ended_at IS NULL``) meter for *commodity*, or None."""
return session.execute(
select(Meter)
.where(
Meter.commodity == commodity,
Meter.ended_at.is_(None),
)
.limit(1)
).scalar_one_or_none()
def _meter_before(session: Session, meter: Meter) -> Optional[Meter]:
"""Return the meter whose ``ended_at`` equals *meter*'s ``started_at``.
This is the meter that was closed when *meter* was opened; its ``ended_at``
needs to stay equal to *meter*'s ``started_at`` to maintain timeline
continuity. Returns None if *meter* is the first epoch for its commodity.
The lookup compares naive/aware datetimes via string to avoid SQLite timezone
quirks: both are formatted as ISO 8601 UTC strings for the WHERE clause.
We rely on the fact that the service layer always stores the same timestamp
object as both ``prev.ended_at`` and ``new.started_at``, so their string
representations are identical.
"""
# Normalise the target to a tz-aware UTC datetime for comparison.
# We scan in Python (rather than SQL) to avoid SQLite naive-vs-aware
# mismatch issues when comparing DateTime columns against tz-aware values.
target = _as_utc(meter.started_at)
# Fetch all closed meters of the same commodity and find the one whose
# ended_at equals this meter's started_at (the standard contiguous handoff).
candidates = session.execute(
select(Meter)
.where(
Meter.commodity == meter.commodity,
Meter.ended_at.is_not(None),
)
).scalars().all()
for candidate in candidates:
if candidate.id == meter.id:
continue
candidate_ended = _as_utc(candidate.ended_at)
if candidate_ended == target:
return candidate
return None
# ---------------------------------------------------------------------------
# Core service functions
# ---------------------------------------------------------------------------
def meter_at(
session: Session,
ts: datetime,
commodity: str = "electricity",
) -> Optional[Meter]:
"""Return the meter epoch that covers *ts* for *commodity*.
A meter covers *ts* when:
``started_at ≤ ts`` AND (``ended_at IS NULL`` OR ``ts < ended_at``)
This is the standard half-open interval ``[started_at, ended_at)`` lookup,
consistent with ``active_contract_version_at`` in ``contracts.py``.
Returns ``None`` when no meter covers *ts* (e.g. before the first epoch
was declared, or after a gap).
Parameters
----------
session:
Active SQLAlchemy session (read-only usage).
ts:
UTC datetime to look up.
commodity:
Energy commodity (default ``"electricity"``).
Returns
-------
Meter | None
Implementation note — why the upper bound is pushed into SQL
------------------------------------------------------------
When ``declare_meter`` is called with ``started_at == active.started_at``
(the "equal-timestamp swap" allowed by §3.5), the old meter is closed as a
zero-width epoch ``[T0, T0)`` and the new active meter also has
``started_at == T0``. Two rows share the same ``started_at``; SQLite's
row-ordering for ``ORDER BY started_at DESC LIMIT 1`` is then determined by
rowid (i.e. insertion order), which returns the older zero-width row first.
If the upper bound were checked in Python *after* fetching that one row, the
condition ``ts < ended_at`` would be False for **any** ``ts >= T0`` (because
``ended_at == T0``), causing the function to return ``None`` and making the
new active meter permanently invisible.
Pushing both bounds into the SQL ``WHERE`` clause eliminates the ambiguity:
the zero-width row is excluded by ``ts < ended_at`` before ``LIMIT 1`` is
applied, so only the genuinely covering row survives.
SQLite naive-vs-aware datetime note: SQLAlchemy's SQLite dialect strips the
``tzinfo`` from aware datetimes when binding parameters (it does *not*
convert to UTC first). Since all datetimes in this codebase are UTC (either
naive-UTC from the DB or aware-UTC from ``datetime.now(UTC)``), stripping
the tzinfo leaves the wall-clock value unchanged and comparisons remain
correct. This is consistent with how ``contracts.py`` handles the same
situation (see OBS 2 in the M7-T02 review notes).
"""
# Push both the lower and upper bounds into the SQL WHERE clause so that
# zero-width epochs (ended_at == started_at) are excluded *before* LIMIT 1
# is applied. This prevents an equal-timestamp swap from making the new
# active meter invisible (see implementation note above).
stmt = (
select(Meter)
.where(
Meter.commodity == commodity,
Meter.started_at <= ts,
(Meter.ended_at.is_(None)) | (Meter.ended_at > ts),
)
.order_by(Meter.started_at.desc())
.limit(1)
)
return session.execute(stmt).scalar_one_or_none()
def declare_meter(
session: Session,
*,
label: str,
started_at: datetime,
reason: str,
commodity: str = "electricity",
note: Optional[str] = None,
) -> Meter:
"""Declare a meter swap or initial meter epoch.
Closes the current active meter for *commodity* (if one exists) by setting
its ``ended_at`` to *started_at*, then opens a new active meter. The
resulting timeline is **contiguous**: old meter's ``ended_at`` equals new
meter's ``started_at``.
If there is no current active meter (first-ever declaration for this
commodity), the new meter is simply opened without closing anything.
Validation
----------
- If a current active meter exists, *started_at* must be **≥** that meter's
own ``started_at``. A value strictly earlier would place the new epoch
entirely before the current active meter, which is a chronological
contradiction ("back-dating before the active epoch's start"). Raises
``MeterOverlapError`` when this constraint is violated.
Note: equal timestamps (``started_at == active.started_at``) are allowed
because that scenario effectively replaces the current meter at the same
logical moment (e.g. correcting a mis-entry), which is a valid use-case.
The old meter is then closed with ``ended_at == started_at`` (a zero-width
epoch), which is intentional and auditable.
Parameters
----------
session:
Active SQLAlchemy session. Caller must commit after this returns.
label:
Human-readable label for the new meter.
started_at:
UTC datetime at which this meter epoch starts. May be in the past.
reason:
Why this epoch was created (``"initial"``, ``"meter_swap"``,
``"home_move"``, or ``"other"``).
commodity:
Energy commodity (default ``"electricity"``).
note:
Optional free-form note.
Returns
-------
Meter
The newly created, not-yet-committed active meter.
Raises
------
MeterOverlapError
If *started_at* is strictly earlier than the current active meter's
``started_at`` (chronological backdate below the active epoch's start).
"""
active = _active_meter(session, commodity)
if active is not None:
# Reject back-dating: new started_at must be ≥ current active's started_at.
if _as_utc(started_at) < _as_utc(active.started_at):
raise MeterOverlapError(
f"New meter started_at ({started_at.isoformat()}) must be ≥ the current "
f"active {commodity!r} meter's started_at "
f"({active.started_at.isoformat()}). "
"Declare a started_at on or after the active meter's start to avoid "
"a chronologically inconsistent epoch ordering."
)
# Close the current active meter at the swap point (contiguous handoff).
active.ended_at = started_at
logger.info(
"Closed active %r meter id=%d (ended_at=%s)",
commodity,
active.id,
started_at.isoformat(),
)
now = datetime.now(UTC)
new_meter = Meter(
label=label,
commodity=commodity,
started_at=started_at,
ended_at=None,
reason=reason,
note=note,
created_at=now,
)
session.add(new_meter)
logger.info(
"Declared new %r meter %r (started_at=%s, reason=%s)",
commodity,
label,
started_at.isoformat(),
reason,
)
return new_meter
def list_meters(
session: Session,
commodity: Optional[str] = None,
) -> list[Meter]:
"""List all meter epochs, ordered by started_at ascending.
Active meters (``ended_at IS NULL``) sort naturally to the end of the
timeline since they have the latest ``started_at``. Within a single
commodity, the ascending ``started_at`` order reflects the historical
sequence of installed meters.
Parameters
----------
session:
Active SQLAlchemy session (read-only usage).
commodity:
If provided, filter to this commodity only. If ``None``, return all
meters across all commodities.
Returns
-------
list[Meter]
Meters ordered by (started_at ASC).
"""
stmt = select(Meter).order_by(Meter.started_at.asc())
if commodity is not None:
stmt = stmt.where(Meter.commodity == commodity)
return list(session.execute(stmt).scalars().all())
def update_meter(
session: Session,
meter: Meter,
*,
label: Optional[str] = None,
note: Optional[str] = None,
started_at: Optional[datetime] = None,
) -> Meter:
"""Update a meter's mutable fields (label, note, started_at).
Passing ``None`` for a field leaves it unchanged. At least one keyword
argument must be non-``None``; calling with all-``None`` is a no-op but
is not an error.
Updating ``started_at`` (retroactive correction)
------------------------------------------------
When ``started_at`` is provided the service maintains **timeline
continuity** across the adjacent meter boundaries:
1. **Previous meter's ``ended_at``** — if the meter immediately before
this one has ``ended_at == meter.started_at`` (the standard contiguous
handoff), its ``ended_at`` is updated to the new ``started_at`` so the
boundary between the two epochs stays seamless.
2. **Validation** — the new ``started_at`` is checked for consistency:
a. It must be **strictly after** the previous meter's own ``started_at``
(otherwise the previous meter's epoch would collapse to zero or
invert).
b. It must be **strictly before** this meter's ``ended_at`` (if set),
so this meter's epoch remains non-empty.
Note: triggering a billing recompute (``recompute_range``) after a
retroactive ``started_at`` change is **out of scope** for this service
layer; that is the API layer's responsibility (M7-T05).
Parameters
----------
session:
Active SQLAlchemy session. Caller must commit after this returns.
meter:
The ``Meter`` ORM instance to update (already loaded from the session).
label:
New human-readable label; ``None`` = keep existing.
note:
New free-form note; ``None`` = keep existing.
started_at:
New start timestamp for this meter epoch; ``None`` = keep existing.
Returns
-------
Meter
The updated ``Meter`` instance (not yet committed).
Raises
------
MeterIntervalError
If the new ``started_at`` would produce an invalid (empty or inverted)
epoch for this meter or the immediately preceding one.
"""
if label is not None:
meter.label = label
logger.info("Updated meter id=%d label=%r", meter.id, label)
if note is not None:
meter.note = note
logger.info("Updated meter id=%d note=%r", meter.id, note)
if started_at is not None:
old_started_at = meter.started_at
# --- Validate upper bound: new started_at must be < this meter's ended_at (if set).
if meter.ended_at is not None:
if _as_utc(started_at) >= _as_utc(meter.ended_at):
raise MeterIntervalError(
f"New started_at ({started_at.isoformat()}) must be strictly before "
f"this meter's ended_at ({meter.ended_at.isoformat()}). "
"The meter epoch would become empty or inverted."
)
# --- Find the immediately preceding meter (its ended_at == meter's old started_at).
prev = _meter_before(session, meter)
# --- Validate lower bound: new started_at must be strictly after prev's started_at.
if prev is not None:
if _as_utc(started_at) <= _as_utc(prev.started_at):
raise MeterIntervalError(
f"New started_at ({started_at.isoformat()}) must be strictly after "
f"the previous meter's started_at ({prev.started_at.isoformat()}). "
"Moving the boundary that far back would collapse the previous meter's epoch."
)
# Maintain continuity: update the previous meter's ended_at to match the new start.
prev.ended_at = started_at
logger.info(
"Updated previous meter id=%d ended_at=%s (boundary shift from %s)",
prev.id,
started_at.isoformat(),
old_started_at.isoformat(),
)
meter.started_at = started_at
logger.info(
"Updated meter id=%d started_at=%s (was %s)",
meter.id,
started_at.isoformat(),
old_started_at.isoformat(),
)
return meter
+1
View File
@@ -64,6 +64,7 @@ def poll_device(session: Session, device: ModbusDevice) -> ModbusReading | None:
device.port,
device.unit_id,
[{"start": b.start, "count": b.count} for b in profile.blocks],
function_code=profile.function_code,
)
payload = profiles.decode(profile, registers)
+139
View File
@@ -0,0 +1,139 @@
"""Service layer for fetching and persisting Tibber 15-minute electricity prices.
``refresh_prices`` is the main entry point. It is designed to be called from a
scheduled background job (see ``app/main.py``) and from tests.
Design decisions
----------------
- **Guard clause**: if no active contract with ``kind="tibber"`` exists, or if
the Tibber API token is empty in the runtime settings, the function is a
complete no-op (returns 0) and does not raise. This means the scheduler can
call ``refresh_prices`` unconditionally; the service itself decides whether to
do anything based on current configuration.
- **Upsert idempotency**: rows are matched by ``starts_at`` (the unique
constraint on ``tibber_price``). If a row already exists, its price fields
and ``fetched_at`` are updated in-place; if it does not exist, a new row is
inserted. Running ``refresh_prices`` twice in a row must not double-insert.
- **No destructive operations**: this service never deletes ``tibber_price``
rows. Only INSERT or UPDATE.
- **Exception propagation**: exceptions from the Tibber client are *not*
swallowed here. The scheduled job wrapper in ``main.py`` is responsible for
catching and logging errors so that a single fetch failure does not crash the
scheduler.
- **SQLite timezone note**: ``DateTime(timezone=True)`` columns come back as
timezone-naive UTC datetimes on read. We store UTC-aware datetimes on write
(``starts_at`` from ``PricePoint`` is already UTC-aware; ``fetched_at`` is
``datetime.now(UTC)``). Reads elsewhere that compare against these values
must normalise accordingly.
"""
from __future__ import annotations
import logging
from datetime import UTC, datetime
from sqlalchemy import select
from sqlalchemy.orm import Session
from app.integrations.tibber.client import fetch_price_range
from app.models.energy import EnergyContract, TibberPrice
logger = logging.getLogger(__name__)
def _active_tibber_contract_exists(session: Session) -> bool:
"""Return True if there is an active contract with kind='tibber'."""
row = session.execute(
select(EnergyContract).where(
EnergyContract.active.is_(True),
EnergyContract.kind == "tibber",
).limit(1)
).scalar_one_or_none()
return row is not None
def refresh_prices(session: Session, settings: object) -> int:
"""Fetch today-and-tomorrow Tibber prices and upsert them into ``tibber_price``.
Parameters
----------
session:
An active SQLAlchemy session. The caller is responsible for closing it;
this function commits each upserted row individually to keep transactions
short.
settings:
A runtime settings object that exposes ``tibber_api_token`` and
``tibber_home_id`` attributes (typically a ``Settings`` or merged
runtime-settings instance from ``build_runtime_settings``).
Returns
-------
int
Number of rows upserted (inserted or updated). Returns 0 for a no-op.
Notes
-----
- No-op (returns 0) when:
* No active contract exists with ``kind="tibber"``, OR
* ``settings.tibber_api_token`` is empty or whitespace.
- Exceptions from the Tibber client are *not* caught here; they propagate to
the caller (the scheduled job wrapper swallows them and logs).
"""
token: str = getattr(settings, "tibber_api_token", "") or ""
if not token.strip():
logger.debug("refresh_prices: tibber_api_token is empty — no-op")
return 0
if not _active_tibber_contract_exists(session):
logger.debug("refresh_prices: no active tibber contract — no-op")
return 0
home_id: str = getattr(settings, "tibber_home_id", "") or ""
home_id_or_none: str | None = home_id.strip() or None
logger.info("refresh_prices: fetching Tibber price range (home_id=%r)", home_id_or_none)
# May raise TibberError or TibberAuthError — let them propagate.
price_points = fetch_price_range(token, home_id_or_none)
if not price_points:
logger.info("refresh_prices: API returned zero price points — nothing to upsert")
return 0
fetched_at = datetime.now(UTC)
upserted = 0
for point in price_points:
existing = session.execute(
select(TibberPrice).where(TibberPrice.starts_at == point.starts_at)
).scalar_one_or_none()
if existing is not None:
# Update in-place — price fields may change (e.g. Tibber corrects a
# forecast), but starts_at and resolution stay the same.
existing.total = point.total
existing.energy = point.energy
existing.tax = point.tax
existing.level = point.level
existing.currency = point.currency
existing.fetched_at = fetched_at
else:
session.add(
TibberPrice(
starts_at=point.starts_at,
resolution=point.resolution,
total=point.total,
energy=point.energy,
tax=point.tax,
level=point.level,
currency=point.currency,
fetched_at=fetched_at,
)
)
upserted += 1
session.commit()
logger.info("refresh_prices: upserted %d price points", upserted)
return upserted
+100
View File
@@ -0,0 +1,100 @@
"""Server local-timezone helpers.
All time is stored as UTC (no change). This module provides a single,
monkeypatch-friendly entry point for converting UTC moments to the server's
local timezone — used for business-day boundary calculations (standing charges,
effective-from date interpretation, etc.).
Design goals
------------
- Zero third-party dependencies (no tzlocal, no pytz).
- Testable without depending on CI host timezone:
monkeypatch ``local_tz`` to a fixed ``ZoneInfo`` in unit tests.
- DST-correct: conversions use ``astimezone(local_tz())`` per instant, so each
moment is independently correct even during DST transitions.
- Consistent with the existing ``.astimezone()`` pattern already used in
``homeassistant_inbound.py`` and ``poo.py``.
Priority for resolving the local timezone
-----------------------------------------
1. ``TZ`` environment variable — ``ZoneInfo(os.environ["TZ"])``.
Set ``TZ=Europe/Amsterdam`` in the deployment env for correct NL handling.
2. System local timezone fallback: ``datetime.now().astimezone().tzinfo``.
This matches the behaviour callers already relied on implicitly.
"""
from __future__ import annotations
import os
from datetime import date, datetime, timezone
from typing import TYPE_CHECKING
from zoneinfo import ZoneInfo
if TYPE_CHECKING:
from datetime import tzinfo
def local_tz() -> "tzinfo":
"""Return the server's local timezone.
Resolution order:
1. ``TZ`` environment variable (``ZoneInfo(TZ)``). Set
``TZ=Europe/Amsterdam`` in production for correct NL/DST handling.
2. System local timezone via ``datetime.now().astimezone().tzinfo``.
**Monkeypatch this function in tests** to get deterministic timezone
behaviour regardless of CI host configuration::
monkeypatch.setattr("app.services.timezone.local_tz",
lambda: ZoneInfo("Europe/Amsterdam"))
"""
tz_env = os.environ.get("TZ", "").strip()
if tz_env:
return ZoneInfo(tz_env)
# System fallback — identical to the .astimezone() pattern already used
# in homeassistant_inbound.py and poo.py.
return datetime.now().astimezone().tzinfo # type: ignore[return-value]
def to_local(dt: datetime) -> datetime:
"""Convert *dt* to the server local timezone.
Handles three input cases:
- timezone-aware (any zone): converted via ``astimezone``.
- timezone-naive: assumed UTC (SQLite read-back convention), then converted.
Returns a timezone-aware datetime in ``local_tz()``.
"""
if dt.tzinfo is None:
dt = dt.replace(tzinfo=timezone.utc)
return dt.astimezone(local_tz())
def local_now() -> datetime:
"""Return the current moment as a timezone-aware local datetime."""
return datetime.now(local_tz())
def local_date(dt: datetime) -> date:
"""Return the local calendar date for *dt*.
Equivalent to ``to_local(dt).date()``. A distinct helper to make
call sites readable.
"""
return to_local(dt).date()
def local_midnight_utc(d: date) -> datetime:
"""Return the UTC instant that corresponds to local midnight on *d*.
Example (Europe/Amsterdam, CEST=UTC+2):
local_midnight_utc(date(2026, 6, 25))
→ datetime(2026, 6, 24, 22, 0, 0, tzinfo=UTC)
This is used to convert a local calendar date back to a UTC boundary, e.g.
to compute the start/end of a local day for DB queries.
"""
tz = local_tz()
# Build a naive local datetime at midnight, then localize and convert to UTC.
local_midnight = datetime(d.year, d.month, d.day, 0, 0, 0, tzinfo=tz)
return local_midnight.astimezone(timezone.utc)
+4
View File
@@ -62,6 +62,8 @@ packaging==26.1
# build
# pytest
# wheel
paho-mqtt==2.1.0
# via -r requirements.in
pip-tools==7.5.3
# via -r dev-requirements.in
pluggy==1.6.0
@@ -78,6 +80,8 @@ pydantic-settings==2.13.1
# via -r requirements.in
pygments==2.20.0
# via pytest
pymodbus==3.13.1
# via -r requirements.in
pyotp==2.10.0
# via -r requirements.in
pyproject-hooks==1.2.0
+2
View File
@@ -9,6 +9,7 @@ services:
volumes:
- ./data:/app/data
- ./.env:/app/.env:ro
- /etc/localtime:/etc/localtime:ro
app:
container_name: home-automation-app
@@ -24,4 +25,5 @@ services:
volumes:
- ./data:/app/data
- ./.env:/app/.env:ro
- /etc/localtime:/etc/localtime:ro
+11 -5
View File
@@ -19,36 +19,42 @@
- `main.py`
- FastAPI app factory
- lifespanAPScheduler 启停、MQTT 客户端起停、连接后触发 HA Discovery 发布)
- lifespanAPScheduler 启停、MQTT 客户端起停、连接后触发 HA Discovery 发布M6 新增 `tibber-refresh` 抓价 job + `energy-cost` 1 分钟计费 tick jobM6 启用时注册 DSMR MQTT 订阅
- 基础路由注册
- `config.py`
- 环境变量驱动的 settings(含 M5 新增的 MQTT/HA Discovery/Modbus 配置项)
- 环境变量驱动的 settings(含 M5 新增的 MQTT/HA Discovery/Modbus 配置项M6 新增 `dsmr_ingest_enabled``dsmr_mqtt_topic``dsmr_sample_interval_s``tibber_api_token`secret)、`tibber_home_id`
- `db.py`
- 统一数据层:一个 `Base`、一个绑定 `app_database_url` 的 cached engineSQLite WAL)、`get_engine` / `get_session_local` / `reset_db_caches` / `get_db_session`
- `dependencies.py`
- 通用依赖注入
- `api/`
- HTTP routes
- `api/routes/api/`JSON API`/api/*` 前缀),供 React SPA 调用:会话/鉴权、配置读写、数据查询、记录 CRUD、Modbus 设备 CRUD + readings + metrics + test`/api/modbus/*`)、Expose 勾选 + 重发 discovery`/api/expose`)、MQTT 测试连接(`/api/config/mqtt/test`
- `api/routes/api/`JSON API`/api/*` 前缀),供 React SPA 调用:会话/鉴权、配置读写、数据查询、记录 CRUD、Modbus 设备 CRUD + readings + metrics + test`/api/modbus/*`)、Expose 勾选 + 重发 discovery`/api/expose`)、MQTT 测试连接(`/api/config/mqtt/test`、M6 新增合同 CRUD + 版本(`/api/energy/contracts*`)、pricing profile 列表(`/api/energy/profiles`)、价格/费用/汇总/DSMR 最新/重算/Tibber 测试(`/api/energy/prices``/api/energy/costs``/api/energy/costs/summary``/api/energy/dsmr/latest``/api/energy/costs/recompute``/api/energy/tibber/test`
- 裸 ingestion 端点:`GET /public-ip/check``POST /homeassistant/publish``POST /poo/record``GET /poo/latest`、TickTick OAuth 等
- `models/`
- SQLAlchemy models
- 所有模型(auth / config / public_ip / location / poo / modbus / expose)共用同一个 `Base`,均落在单一 `app.db`
- 所有模型(auth / config / public_ip / location / poo / modbus / expose / energy)共用同一个 `Base`,均落在单一 `app.db`
- M5 新增:`ModbusDevice`(设备部署层)、`ModbusReading`(通用遥测,JSON payload)、`ExposedEntityToggle`HA 实体暴露开关)
- M6 新增:`DsmrReading`(整帧 DSMR telegram10s 降采样)、`EnergyContract`(合同头,含 active 标记)、`EnergyContractVersion`(版本/时段,values JSON,只增不改)、`TibberPrice`15 分钟价缓存,不可变)、`EnergyCostPeriod`(每 15 分钟计量电费,快照价,不可变)
- `schemas/`
- Pydantic schemasM5 新增 `modbus.py``expose.py`
- Pydantic schemasM5 新增 `modbus.py``expose.py`M6 新增 `energy_contract.py``energy.py`
- `services/`
- 业务服务层
- 当前已迁入 config page 的 DB 持久化逻辑
- 当前已迁入 public IPv4 检查、状态持久化与变化通知逻辑
- 当前已迁入 SMTP 发信与测试发信逻辑
- M5 新增:`modbus_poll.py`(采集 service,逐设备 poll + 落库 + 推 MQTT state)、`ha_discovery.py`(构建 HA Discovery payload、发布 retained config、发布 state
- M6 新增:`tibber_prices.py`httpx GraphQL 抓 15 分钟价,upsert `tibber_price`,幂等;仅 active=tibber 且 token 存在时运行)、`dsmr_ingest.py`MQTT handler,整帧 JSON blob + 10s 降采样落库,`source_id` 幂等)、`energy_cost.py`(计费引擎:每 15 分钟寄存器差 × strategy 出价 → `energy_cost_period` 不可变快照;汇总 Σnet + 固定费 heffingskorting;重算显式 opt-in
- `integrations/`
- 外部系统适配层
- Home Assistant outbound adapterREST 通道,原有)
- M5 新增:`modbus/`pymodbus 薄封装:`driver.py` 块读 + float32 解码;`profiles.py` YAML profile 加载/校验/解码;`profiles/sdm120.yaml` SDM120 协议声明)
- M5 新增:`mqtt.py`paho-mqtt 长连接 `MqttManager`:lifespan 起/停、配置变更重连、`publish(topic, payload, retain)`
- M5 新增:`expose.py`(通用 expose 框架:`ExposableEntity`、provider 注册表、`build_catalog`Modbus provider 从 YAML profile 派生 sensor/binary_sensor 实体目录)
- M6 新增:`pricing/`(通用电价层:`profiles.py` pydantic 加载/校验 YAML profile + `validate_values``strategies.py` manual/tibber 出价策略注册表;`profiles/manual.yaml` 固定/双费率结构;`profiles/tibber.yaml` 动态电价结构)
- M6 新增:`tibber/``client.py`httpx GraphQL 客户端,`priceInfoRange(QUARTER_HOURLY)` 查询,按 `starts_at` 解析,不假设固定节点数)
- M6 扩展:`mqtt.py` 新增订阅端(`subscribe(topic, handler)``on_connect` 里 subscribe`on_message` 按 topic 分发,handler 异常吞掉不崩连接)
- M6 扩展:`expose.py` 新增 `_energy_cost_provider``buy_price_now``sell_price_now``import_cost_total`/`export_revenue_total``total_increasing`,反哺 HA Energy
- `static/`
- 极简静态资源
+4 -2
View File
@@ -1,14 +1,16 @@
# 设计文档与多模型协作约定
本目录把 `docs/roadmap.md` 里的个里程碑展开成**可被 coding agent 流水线执行**的详细设计文档。
本目录把 `docs/roadmap.md` 里的个里程碑展开成**可被 coding agent 流水线执行**的详细设计文档。
- [`m1-db-consolidation.md`](./m1-db-consolidation.md) — 单库化地基
- [`m2-frontend-v2.md`](./m2-frontend-v2.md) — React SPA 前端 v2
- [`m3-token-mobile.md`](./m3-token-mobile.md) — token 鉴权与移动端(远期)
- [`m4-login-hardening.md`](./m4-login-hardening.md) — 登录加固(防爆破/指数退避 + CLI 逃生 + 可选 TOTP**先做**
- [`m5-iot-energy.md`](./m5-iot-energy.md) — IoT 集成与能耗采集(Modbus/Energy + MQTT/HA Discovery + 前端侧边栏)
- [`m6-tibber-dynamic-energy.md`](./m6-tibber-dynamic-energy.md) — 通用电价层 + DSMR 实时电表接入 + 实时买卖电费计算 + HA Energy 反哺
- [`m7-meter-epochs-archival.md`](./m7-meter-epochs-archival.md) — 电表生命周期 / 换表归档(Meter epochs
本文件定义**所有任务共用的格式与协作规则**,个里程碑文档不再重复这些约定。
本文件定义**所有任务共用的格式与协作规则**,个里程碑文档不再重复这些约定。
---
+11 -11
View File
@@ -289,7 +289,7 @@ Phase DAPI + 前端)
> 后端任务沿用校验闸门(`pytest` / `ruff` / 改路由或 schema 则 `export_openapi` 重导出入库);前端闸门见 §8。**不新增依赖**。纯新增、不删/移文件;改 `app/main.py` lifespan 时不得破坏既有 job/连接。
### M6-T01 — 5 张表与模型 `[schema]`
- **Status**: `todo` · **Depends**: none
- **Status**: `done` · **Depends**: none
- **Context**: 单库 app 链新增 5 张表(§3.5)。只建 schema + 模型,不写采集/计算/接口。
- **Files**: `create app/models/energy.py``DsmrReading`/`EnergyContract`/`EnergyContractVersion`/`TibberPrice`/`EnergyCostPeriod`);`create alembic_app/versions/20260623_11_energy_tables.py``modify alembic_app/env.py`、`scripts/app_db_adopt.py``APP_BASELINE_REVISION`→新 head);`create tests/test_energy_models.py`
- **Steps**:
@@ -304,7 +304,7 @@ Phase DAPI + 前端)
- **Reviewer checklist**: 列/约束/FK 与 §3.5 一致;`recorded_at`/`period_start`/`starts_at` 真实索引或唯一列;`env.py` 已 import 新模型;迁移 `down_revision` 指当前真实 head;无破坏性操作。
### M6-T02 — flat 配置(DSMR + Tibber 凭据)
- **Status**: `todo` · **Depends**: none
- **Status**: `done` · **Depends**: none
- **Context**: DSMR 订阅设置 + Tibber 凭据接入扁平配置(前端自动渲染)。**价格数值不在这里**(在合同里)。
- **Files**: `modify app/config.py`、`app/services/config_page.py`CONFIG_FIELDS + `_settings_payload`);`modify .env.example`、`tests/test_api_config.py`
- **Steps**: `Settings` 加 `dsmr_ingest_enabled: bool=False`、`dsmr_mqtt_topic: str="dsmr/json"`、`dsmr_sample_interval_s: int=10`、`tibber_api_token: str=""`、`tibber_home_id: str=""`CONFIG_FIELDS 归「DSMR」「Tibber」section`tibber_api_token` 为 secretbool=checkbox、数值=number`_settings_payload` 补齐;`.env.example` 加注释样例(默认 off)。
@@ -315,7 +315,7 @@ Phase DAPI + 前端)
- **Reviewer checklist**: secret 不回显/不进 OpenAPI 示例;`_settings_payload` 无遗漏;默认 off。
### M6-T03 — pricing profile 框架 + strategy 注册表
- **Status**: `todo` · **Depends**: none
- **Status**: `done` · **Depends**: none
- **Context**: 仓库内 `manual.yaml`/`tibber.yaml` 定结构(pydantic 校验),加 price strategy 注册表(manual/tibber 出价)。纯模块,单测。
- **Files**: `create app/integrations/pricing/__init__.py`、`pricing/profiles.py`pydantic 模型 + `load_profile`/`list_profiles`/`validate_values`)、`pricing/profiles/manual.yaml`、`pricing/profiles/tibber.yaml`、`pricing/strategies.py``register_strategy`/`get_strategy`、`manual`/`tibber` 实现);`create tests/test_pricing_profiles.py`、`tests/test_pricing_strategies.py`
- **Steps**:
@@ -333,7 +333,7 @@ Phase DAPI + 前端)
- **Reviewer checklist**: profile 纯结构、无具体数值;策略用 Decimal;进出口分开(不相抵);tibber 价匹配用 `starts_at ≤ t0` 最近一条;无 OOP 过度抽象(数据+函数,仿 M5 profiles.py)。
### M6-T04 — EnergyContract CRUD + 版本 + 校验(API
- **Status**: `todo` · **Depends**: M6-T01, M6-T03
- **Status**: `done` · **Depends**: M6-T01, M6-T03
- **Context**: 合同增删改、版本(改价加新版本、生效日期)、激活(一次一个)、按 profile 校验数值。
- **Files**: `create app/api/routes/api/energy_contracts.py`、`app/schemas/energy_contract.py`、`app/services/contracts.py``modify app/main.py`(注册路由);`create tests/test_api_energy_contracts.py`
- **Steps**:
@@ -347,7 +347,7 @@ Phase DAPI + 前端)
- **Reviewer checklist**: 版本只增不改(不覆盖旧版本);激活互斥;删除受 FK RESTRICT(有版本/有费用记录不可裸删);数值校验走 T03 `validate_values`;路径键用合同 id。
### M6-T05 — Tibber 客户端 + 抓价 service + 调度 + tibber/test
- **Status**: `todo` · **Depends**: M6-T01, M6-T02, M6-T03
- **Status**: `done` · **Depends**: M6-T01, M6-T02, M6-T03
- **Context**: httpx 拉 15min 价 upsert `tibber_price`;启动+每日抓(仅 active=tibber);连接测试。
- **Files**: `create app/integrations/tibber/__init__.py`、`tibber/client.py`、`app/services/tibber_prices.py``modify app/main.py`(抓价 job);`create tests/test_tibber_client.py`、`tests/test_tibber_prices.py`
- **Steps**:
@@ -362,7 +362,7 @@ Phase DAPI + 前端)
- **Reviewer checklist**: token 不进日志;httpx 超时;upsert 幂等;时间存 UTC;无对 `tibber_price` 的破坏性批删。
### M6-T06 — MqttManager 扩订阅 + DSMR ingest
- **Status**: `todo` · **Depends**: M6-T01, M6-T02
- **Status**: `done` · **Depends**: M6-T01, M6-T02
- **Context**: 给只发不收的 `MqttManager` 扩订阅;DSMR ingest 整帧 blob + 10s 降采样落库。
- **Files**: `modify app/integrations/mqtt.py`(订阅注册 + on_message 分发);`create app/services/dsmr_ingest.py``modify app/main.py`(启用时注册订阅);`create tests/test_mqtt_subscribe.py`、`tests/test_dsmr_ingest.py`
- **Steps**:
@@ -378,7 +378,7 @@ Phase DAPI + 前端)
- **Reviewer checklist**: on_message 网络线程内只做短事务、吞异常不崩连接;整帧存(含 gas/各相);null 容错;`source_id` 幂等;10s 降采样正确。
### M6-T07 — 计费引擎 + 周期 job + 汇总 + 重算
- **Status**: `todo` · **Depends**: M6-T01, M6-T03, M6-T04, M6-T05, M6-T06
- **Status**: `done` · **Depends**: M6-T01, M6-T03, M6-T04, M6-T05, M6-T06
- **Context**: 每 15min 用寄存器差 × active 合同 strategy 算计量电费(快照、不可变);汇总加固定费 − 抵扣;周期 job + 重算。
- **Files**: `create app/services/energy_cost.py``modify app/main.py`(周期 job);`create tests/test_energy_cost.py`
- **Steps**:
@@ -393,7 +393,7 @@ Phase DAPI + 前端)
- **Reviewer checklist**: Decimal 算钱;寄存器差为"末−初";进出口分开;周期边界 UTC 刻钟;快照不可变(重算才覆盖,且显式);版本选择按 `effective_from≤t0<effective_to`job 不崩。
### M6-T08 — energy_cost expose provider + state 发布
- **Status**: `todo` · **Depends**: M6-T07
- **Status**: `done` · **Depends**: M6-T07
- **Context**: 复用 M5 expose/Discovery,发当前买/卖价 + 累计成本/收入。
- **Files**: `modify app/integrations/expose.py``_energy_cost_provider` + 注册);`modify app/services/energy_cost.py` 或 `app/main.py`(算完顺带 `publish_states`);`create tests/test_energy_expose.py`
- **Steps**: provider 产出 §3.7 实体(`buy_price_now`/`sell_price_now`/`import_cost_total`/`export_revenue_total`),累计项 `total_increasing`+`monetary`+currency`value_getter` 从 `energy_cost_period` 取;计费 job 后 `publish_states`。
@@ -405,7 +405,7 @@ Phase DAPI + 前端)
- **Reviewer checklist**: `key` 稳定(固定字符串);复用既有 provider 协议、未改其它 provider;累计 `total_increasing` 语义正确;不阻塞计费 job。
### M6-T09 — Energy 数据 API
- **Status**: `todo` · **Depends**: M6-T05, M6-T07
- **Status**: `done` · **Depends**: M6-T05, M6-T07
- **Context**: 价格/费用/汇总/最新 DSMR/重算/Tibber 测试端点(合同 CRUD 在 T04)。
- **Files**: `create app/api/routes/api/energy.py`、`app/schemas/energy.py``modify app/main.py`(注册);`create tests/test_api_energy.py`
- **Steps**: `GET /prices`、`GET /costs`、`GET /costs/summary`、`GET /dsmr/latest`、`POST /costs/recompute`(调 T07)、`POST /tibber/test`(调 T05 client,三态);session+CSRF;查询走索引 + 窗口/上限。
@@ -416,7 +416,7 @@ Phase DAPI + 前端)
- **Reviewer checklist**: 查询有时间窗+上限;recompute 复用 T07、无破坏删;test 不泄 token。
### M6-T10 — 前端:合同管理 + 价格/费用视图 + Tibber 测试
- **Status**: `todo` · **Depends**: M6-T04, M6-T09
- **Status**: `done` · **Depends**: M6-T04, M6-T09
- **Files**: `modify frontend/src/pages/EnergyPage.tsx``create frontend/src/energy/ContractManager.tsx`、`ContractForm.tsx`、`TibberPrices.tsx`、`CostView.tsx`、扩 `hooks.ts``modify` Config 页 Tibber 测试入口;`create` 对应 `*.test.tsx`
- **Steps**: 合同列表/新建/编辑/激活/加版本(**表单字段按 `/profiles` 结构渲染**,manual 双费率/税/固定费/抵扣)+ 版本历史只读;价格曲线 + 费用走势/明细 + 汇总卡片;Tibber 测试三态;全走类型化 client;空/加载/错误态 + 缺 key 容错。
- **Out of scope**: 不做服务端降采样;不改后端。
@@ -427,7 +427,7 @@ Phase DAPI + 前端)
- **Reviewer checklist**: 全走类型化 client;图表组件隔离复用;表单按结构渲染、不 hardcode 字段;窗口/上限;空/错/加载/缺 key 容错。
### M6-T11 — 文档 + OpenAPI + roadmap 收尾
- **Status**: `todo` · **Depends**: 全部
- **Status**: `done` · **Depends**: 全部
- **Files**: `modify README.md`、`docs/roadmap.md`M6 行 + 毕业)、`docs/architecture-overview.md`、`docs/design/README.md`(列入 m6);`run python scripts/export_openapi.py` 并提交 `openapi/`
- **Acceptance criteria**:
- [ ] 文档反映新链路;`git diff --exit-code openapi/` 无未提交差异;校验闸门全绿。
+322
View File
@@ -0,0 +1,322 @@
# M7 — 电表生命周期 / 换表归档(Meter epochs
> 协作格式、任务卡结构、校验闸门、数据安全红线见 [`README.md`](./README.md),本文不再重复。
## 1. 目标
让计费系统正确处理**电表更换**这一**必然事件**:
- 荷兰 2G 智能电表退网后,电网公司**一定**会把表换成 4G 表(同址换表);搬家继承新表也是同类。
- 计费是**寄存器差值(delta)模型**:每个 15 分钟周期成本 =(周期末读数 − 周期初读数)× 单价。换表后新表寄存器基数与旧表**无关**(可能更高也可能更低),若跨表算 delta 会产出**负成本或巨额假成本**。
- 目标:引入显式的 **Meter(电表)** 概念,标记"某时刻起属于哪一块物理表",使引擎**永不跨表算 delta**,并让累计量按表归零、历史可追溯可查。
非目标(本里程碑不做,§9 留痕):gas / 区域供暖的**计费**"家庭(home)"分组实体;多合同时间线积分;自动识别换表(DSMR 帧无电表序列号,无法自动识别,靠用户显式声明)。
## 2. 现状(实现者可据此工作,不必通读全仓库)
- 计费数据模型(M6):
- `dsmr_reading`:整帧 JSON`recorded_at`(UNIQUE) 去重;含 4 个累计寄存器 `electricity_delivered_1/2``electricity_returned_1/2`。**帧内无电表序列号字段**。
- `energy_cost_period`:每 15 分钟一行,存 `d1/d2/r1/r2_kwh`(delta)、`import_cost/export_revenue/net_cost``pricing` 快照、`contract_version_id``degraded``computed_at``period_start`(unique)。
- 计费引擎 `app/services/energy_cost.py`
- `register_at(session, boundary)`:取 `recorded_at ≤ boundary` 的最近一条;有 15 分钟 staleness 护栏(超过则返回 None → 周期降级)。
- `compute_period(session, t0)`:取 t0/t1 两端寄存器,算 delta,按 `contract` 的 strategy 计价;缺读数→降级行(成本 0)。
- `compute_closed_periods` / `recompute_range`:遍历刻钟边界批量算。
- `summarize(session, start, end)`:聚合非降级周期电费 + 固定费/抵扣(FU10 已按本地日跨版本积分)。
- 计价策略 `app/integrations/pricing/strategies.py``import_cost = Δd1×buy_dal + Δd2×buy_normal` 等,**对 delta 无任何 clamp/sanity 检查**(负 delta → 负成本,超大 delta → 巨额成本)。
- 暴露 `app/integrations/expose.py`:累计 `import_cost_total`/`export_revenue_total` 锚点(FU11= `max(合同最早 effective_from, 最早非降级 period_start)``*_today` 日归零实体(FU11)。
- 合同 `app/services/contracts.py``active` 互斥 + 版本时间线 + `active_contract_version_at(ts)`(半开 `[from,to)`)。合同**不引用电表**。
- 配置/迁移:单库 appAlembic 链 `alembic_app/``PRAGMA foreign_keys=ON` 已开(`app/db.py`)。
- 时区:FU10 的 `app/services/timezone.py``local_*`,可 monkeypatch)。
## 3. 目标架构
### 3.1 核心概念:Meter= epoch
一条 `meter` 记录 = **一块物理电表的一段安装期** `[started_at, ended_at)`。换表 = 关闭当前 active 表(`ended_at=T`+ 新建一条 `started_at=T` 的表。计费**永不跨 meter 算 delta**:跨表的那个周期按设计降级。
- **每种 `commodity` 至多一个 active 表**`ended_at IS NULL`)。本里程碑只 `electricity` 参与计费,`commodity` 字段为未来 gas/heating 预留。
- `meter_at(session, ts, commodity="electricity")`:返回 `started_at ≤ ts AND (ended_at IS NULL OR ts < ended_at)` 的那块表(半开区间,仿 `active_contract_version_at`)。
- 计费基准:某表窗口内**最早的一条 `dsmr_reading`**`recorded_at ≥ started_at`)即基准;用户**只需填 `started_at` 日期,无需知道当前读数**。追溯(`started_at` 在过去)时,有效起点自然 = `max(started_at, 数据起点)`
- **隔离的前提 = 显式声明**:系统无法自动识别换表(DSMR 帧无序列号),搬家后新旧读数同落 `dsmr_reading`,靠用户**显式新建那块表**才隔离。万一忘了声明:搬家必有的数据空档(staleness)+ delta 护栏(§3.3)保证**不会产出垃圾成本**,只是新数据暂混在旧表 epoch、累计未归零;**事后补声明 + 追溯 `recompute_range`** 即可纠正。
### 3.2 数据模型(新增 1 张表 + 1 列)
`meter` 表(`app/models/energy.py`,单库 app 链):
| 列 | 类型 | 说明 |
| --- | --- | --- |
| `id` | int PK | 内置自增 ID |
| `label` | str | 人读标签,编址/标识(如 "旧 2G 表 @ Dorpsstraat 1" |
| `commodity` | str | 默认 `"electricity"`;预留 `gas`/`heating` |
| `started_at` | datetime(UTC) | 安装/继承/搬家起点(可在过去) |
| `ended_at` | datetime(UTC) \| null | null = 当前 active |
| `reason` | str | `initial` / `meter_swap` / `home_move` / `other` |
| `note` | str \| null | 备注 |
| `created_at` | datetime(UTC) | |
`energy_cost_period` 新增 `meter_id`nullable FK → `meter.id`):记录该周期归属哪块表,便于审计/按表查询/归档。
### 3.3 计费引擎改造(`energy_cost.py` / `strategies.py` 调用处)
1. `register_at(session, boundary, meter)`:**仅在该 meter 窗口内**取读数(`started_at ≤ recorded_at < ended_at``≤ boundary` 且过 staleness 护栏)。绝不把旧表读数拉进新表周期。
2. `compute_period(session, t0)`
- 查 `m0 = meter_at(t0)``m1 = meter_at(t1)`
- `m0 is None`(无表覆盖)→ **降级**(不计费)。
- `m0.id != m1.id`(周期跨表边界)→ **降级**D5:丢这一个周期可接受)。
- 否则在 `m0` 窗口内 `register_at` 两端、算 delta、计价;新增写 `period.meter_id = m0.id`
3. **delta sanity 护栏(D6**:算出 delta 后,若 `d1/d2/r1/r2` 任一 `< 0`,或任一 `> _MAX_DELTA_KWH`(常量,给一个远超住宅的宽松上限,如每 15 分钟 100 kWh)→ **降级**(兜底表重置/DSMR 回绕/数据毛刺,与换表无关也防住)。
4. `compute_closed_periods` / `recompute_range`:逐周期按 `meter_at` 判定,无需额外结构。
### 3.4 累计语义(per-meter 归零,D2
- `expose.py` 累计 `import_cost_total`/`export_revenue_total` 锚点改为**当前 active electricity 表的起点**`anchor = active electricity meter.started_at`(窗口内首条读数自然 ≥ 它)。换表后累计**从零起一段新序列**,不维护偏移、不跨表累加。
- `*_today` 日归零实体不变(其本地今天窗口必落在当前表内)。
- (可选,非核心)暴露"当前电表 label"作为一个文本 sensor,便于 HA 侧识别当前表。
- **已知行为(可接受)**:累计 `_total``state_class: total`,换表归零时其值会下台阶,HA 长期统计可能在换表那一刻记一次性负 blip。HA 侧自己存旧值做 down-sample——已与用户确认**先这样、观察后按需处理**(`last_reset` 信号见 §9 杠杆,本里程碑不做)。
### 3.5 API`app/api/routes/api/` + schemas
- `GET /api/energy/meters` — 列出全部表(时间线,active 在前/或按 started_at)。
- `POST /api/energy/meters` — 声明换表/继承:`{label, started_at, reason, note?}`;服务层关闭当前 active 同 commodity 表(`ended_at=started_at`+ 建新表。`started_at` 必须 `≥` 当前 active 表的 `started_at`(拒绝倒挂,仿合同版本"strictly after")。
- `PATCH /api/energy/meters/{id}` — 改 `label`/`note`,或修正 `started_at`(追溯)。
- `POST /api/energy/meters/{id}/recompute`(或复用现有 recompute)— 追溯变更后对受影响窗口 `recompute_range`,重判跨表周期与归属。
- 改了路由/schema → 重导出 OpenAPI 并提交。
### 3.6 前端(并入 Energy 视图)
- 一个"电表(Meters"管理区:表的时间线(label / 区间 / active 标记 / reason),"我换了表 / 继承了新表"表单(只填 label + 日期 + reason),编辑 label/日期。
- 追溯保存后给出"已重算受影响周期"的反馈(或提供 recompute 按钮)。
- 前端闸门见 §8 引用。
### 3.7 迁移与回填(数据安全 runbook)
- Alembic`alembic_app/`):建 `meter` 表 + 给 `energy_cost_period``meter_id` 列(nullable)。
- **回填(只增不删、幂等、对账)**
1. 若库内已有任何 `dsmr_reading`/`energy_cost_period`,创建一条**初始表** `meter(label="Initial meter", commodity="electricity", started_at = 最早 dsmr_reading.recorded_at(无则最早 period_start,再无则迁移时刻), ended_at=NULL, reason="initial")`
2. 把现有 `energy_cost_period.meter_id` 全部回填为这条初始表的 id。
3. **对账**:回填后 `meter_id IS NULL` 的非降级周期数必须为 0;对不上立即中止、非零退出。
- **红线**:迁移**不删除/不覆盖**任何 `dsmr_reading`/`energy_cost_period`/旧 `.db`/备份;纯新增表+列+回填。先在备份副本演练再对真实库执行。
## 4. 已锁定决策(讨论后拍板)
- **D1 粒度**:Meter 为一等公民,**不绑定 home**;`label` 编址 + `commodity` 区分品类(每 commodity 一个 active)。
- **D2 累计**:换表后累计量**归零**(锚当前表起点),直接吃 DSMR 累计 delta,不维护偏移。
- **D3 合同耦合**:**不引入 home 实体**。合同与电表是两条独立时间线,合同不引用电表 → 同址换表时合同自动照常套用,无需上层抽象。搬家若换合同:用"给现有合同加 version"保持历史连续(多合同时间线积分留作后续杠杆)。
- **D4 追溯**`started_at` 可在过去;有效计费起点 = `max(started_at, 数据起点)`;追溯变更触发 `recompute_range`
- **D5 边界周期**:跨表周期判 **degraded**(可接受,换表常伴随长时间无数据)。
- **D6 delta 护栏**:加(负/异常大 delta → degraded),通用兜底。
- **D7 扩展性**gas/heating **计费**不在 scope`commodity` 字段 + 每品类 active 表使未来加表低成本。
## 5. 任务依赖图
```
M7-T01 (meter 表+模型+列+迁移回填)
├──► M7-T02 (meter 服务层: swap/close/edit, meter_at, 互斥/校验)
│ ├──► M7-T03 (计费引擎 meter-aware + delta 护栏 + period.meter_id)
│ │ └──► M7-T04 (累计 per-meter 归零, expose 锚点)
│ └──► M7-T05 (meter API + 追溯 recompute + OpenAPI)
│ └──► M7-T06 (前端电表管理 UI)
└──────────────────────────────► M7-T07 (文档/roadmap/收尾)
```
## 6. 原子任务(任务卡)
### M7-T01 — `meter` 表 + 模型 + `energy_cost_period.meter_id` + 迁移回填 `[schema]`
- **Status**: `done`
- **Depends**: none
- **Context**: 引入 Meter 实体的数据地基;为现有数据回填初始表。
**Files**
- `modify app/models/energy.py`(新增 `Meter``EnergyCostPeriod``meter_id`
- `create alembic_app/versions/<rev>_meter_table.py`
- `modify tests/test_energy_models.py`
**Steps**
1. 加 `Meter` 模型(字段见 §3.2);`EnergyCostPeriod` 加 nullable `meter_id` FK。
2. 写迁移:建表 + 加列;按 §3.7 回填初始表并回填 `meter_id`;带对账(不一致则 raise)。
3. 测试:模型字段、active-per-commodity 约束的服务层留给 T02;这里测建表/列/回填幂等。
**Out of scope / 不要碰**:计费引擎、expose、API(后续任务)。
**Acceptance criteria**
- [ ] `meter` 表与 `energy_cost_period.meter_id` 建出;迁移可升降级。
- [ ] 有历史数据时回填出恰好一条 `reason="initial"` 的 active 表,且非降级周期 `meter_id` 全部非空。
- [ ] 迁移**不删除/覆盖**任何既有数据;回填幂等。
- [ ] 校验闸门全绿。
**Reviewer checklist**:回填对账逻辑;空库/有数据两种路径;数据安全红线(无删/覆盖)。
### M7-T02 — Meter 服务层(swap/close/edit + `meter_at` + 互斥/校验)
- **Status**: `done`
- **Depends**: M7-T01
- **Context**: 换表/继承的业务逻辑与按时刻查表。
**Files**
- `create app/services/meters.py`
- `create tests/test_meters.py`
**Steps**
1. `meter_at(session, ts, commodity="electricity")`(半开区间)。
2. `declare_meter(session, *, label, started_at, reason, commodity="electricity", note=None)`:校验 `started_at ≥ 当前 active 同 commodity 表的 started_at`(拒绝倒挂);关闭旧 active`ended_at=started_at`+ 建新 active。
3. `list_meters` / `update_meter`label/note/started_at);改 started_at 时维持区间自洽。
4. 测试:互斥(每 commodity 单 active)、倒挂拒绝、追溯、`meter_at` 边界。
**Out of scope**API、引擎、前端。
**Acceptance criteria**
- [ ] 每 commodity 至多一个 activeswap 正确续接区间。
- [ ] `meter_at` 半开区间正确(含边界)。
- [ ] 校验闸门全绿。
**Reviewer checklist**:互斥与续接的事务正确性;追溯改 started_at 的区间一致性。
### M7-T03 — 计费引擎 meter-aware + delta 护栏 + `period.meter_id`
- **Status**: `done`
- **Depends**: M7-T02
- **Context**: 让计费永不跨表算 delta,并兜底异常 delta。
**Files**
- `modify app/services/energy_cost.py`
- `modify tests/test_energy_cost.py`
**Steps**
1. `register_at` 限定在传入 meter 的窗口内取读数。
2. `compute_period`:按 §3.3 判定 `m0/m1`;无表/跨表 → 降级;否则算 delta、写 `meter_id`
3. 加 `_MAX_DELTA_KWH` 常量与护栏:任一寄存器 delta `<0``>上限` → 降级。
4. `compute_closed_periods`/`recompute_range` 逐周期 meter 判定。
5. 测试:同表内正确;跨表周期降级;负/超大 delta 降级;无 active 表降级。
**Out of scope**expose 锚点(T04)、APIT05)。
**Acceptance criteria**
- [ ] 跨 meter 边界周期 = degraded;无表覆盖 = degraded。
- [ ] 负 delta / 超 `_MAX_DELTA_KWH` = degraded,绝不产负/巨额成本。
- [ ] 同表内 delta 计费与归属 `meter_id` 正确。
- [ ] 校验闸门全绿。
**Reviewer checklist**staleness × meter 窗口的交互;护栏阈值是否远超住宅、不会误伤真实用量;recompute 重判归属。
### M7-T04 — 累计 per-meter 归零(expose 锚点)
- **Status**: `done`
- **Depends**: M7-T03
- **Context**: 换表后 MQTT 累计量从零起算。
**Files**
- `modify app/integrations/expose.py`
- `modify tests/test_energy_expose.py`
**Steps**
1. 累计 `import_cost_total`/`export_revenue_total` 锚点改为**当前 active electricity 表的 `started_at`**(替换 FU11 的 `max(合同, 首周期)`)。
2. `*_today` 不变;确认无非降级周期/无 active 表时仍 `None` 保护。
3.(可选)暴露当前电表 label 文本实体。
4. 测试:换表后累计仅含当前表;anchor 取当前表起点。
**Out of scope**API/前端。
**Acceptance criteria**
- [ ] 累计量锚 = 当前 active electricity meter 起点;换表归零。
- [ ] None 保护分支保留;daily 实体不受影响。
- [ ] 校验闸门全绿。
**Reviewer checklist**:与 FU11 行为差异是否符合 D2;无表/空数据分支。
### M7-T05 — Meter API + 追溯 recompute + OpenAPI
- **Status**: `done`
- **Depends**: M7-T02
- **Context**: 暴露电表 CRUD 与换表声明;追溯后重算。
**Files**
- `create app/api/routes/api/meters.py`
- `create app/schemas/meter.py`
- `modify app/api/routes/...`(注册路由)
- `modify openapi/openapi.json`, `openapi/openapi.yaml`
- `create tests/test_api_meters.py`
**Steps**
1. `GET/POST/PATCH /api/energy/meters`+ 追溯触发 `recompute_range`)。
2. 鉴权沿用 `require_session`
3. `python scripts/export_openapi.py` 并提交 `openapi/`
4. 测试:列出/声明换表/编辑/倒挂拒绝/追溯重算。
**Out of scope**:前端。
**Acceptance criteria**
- [ ] 端点行为符合 §3.5;追溯变更触发受影响窗口重算。
- [ ] `git diff --exit-code openapi/` 干净(已重导出提交)。
- [ ] 校验闸门全绿。
**Reviewer checklist**:错误码(倒挂/不存在);recompute 范围是否覆盖受影响周期。
### M7-T06 — 前端电表管理 UI
- **Status**: `done`
- **Depends**: M7-T05
- **Context**: 让用户在 Energy 页声明换表、查看电表时间线。
**Files**
- `create frontend/src/energy/MeterManager.tsx`
- `modify frontend/src/energy/EnergyPage.tsx`, `frontend/src/energy/hooks.ts`
- `modify frontend/src/energy/*.test.tsx`
- `modify frontend/src/api/schema.d.ts``npm run codegen`
**Steps**
1. `npm run codegen` 拉新端点类型。
2. 电表时间线 + "换表/继承"表单(label + 日期 + reason+ 编辑。
3. 追溯保存后反馈"已重算"。
4. 前端闸门(§8)。
**Out of scope**:后端。
**Acceptance criteria**
- [ ] 能列出/声明换表/编辑;UI 合理。
- [ ] 前端 lint/typecheck/test/build 全绿;`schema.d.ts` 已 codegen。
**Reviewer checklist**:日期→后端时区语义(沿用 FU10 本地午夜 naive 约定);空状态/加载/错误。
### M7-T07 — 文档 / OpenAPI / roadmap 收尾
- **Status**: `done`
- **Depends**: M7-T01..T06
- **Context**: 把 M7 落档、更新 roadmap 与索引。
**Files**
- `modify docs/roadmap.md`(新增 M7 条目,标完成)
- `modify docs/design/README.md`(索引加 m7
- `modify docs/design/m7-meter-epochs-archival.md`Status 收尾)
- `modify docs/*`(如新增 meter 模块说明,按需)
**Acceptance criteria**
- [ ] roadmap/README/本文 Status 一致;OpenAPI 已是最新并提交。
- [ ] 校验闸门全绿。
## 7. 构建上下文完整性(M1 教训)
- 本里程碑只**新增** `app/services/meters.py``app/api/routes/api/meters.py``app/schemas/meter.py``alembic_app/versions/<rev>_*.py` 与前端文件;均落在现有 `Dockerfile``COPY app ./app` / `COPY alembic_app ./alembic_app` / 前端构建阶段覆盖范围内,**无新增顶层目录**。
- `tests/test_deployment.py::test_dockerfile_copy_sources_exist` 应仍绿;删/移文件时按 README 规则 grep 构建清单(本里程碑预期无删除)。
## 8. 前端校验闸门
`frontend/` 下:`npm run lint && npm run typecheck && npm run test && npm run build`;改了端点/schema 还需 `npm run codegen` 并提交 `schema.d.ts`
## 9. 后续杠杆(本里程碑不做,留痕)
- **Home/Site 分组**:若将来要"一个地址下多品类表合并出账 + 整屋归档",再加薄分组层(可仅 `site_label` 字段,不必新实体)。
- **多合同时间线积分**:让 `summarize` 跨多个非重叠合同积分固定费/抵扣,免去搬家"加 version"的折中。
- **换表时给累计实体发 `last_reset` 信号**:消除 HA 长期统计在归零时刻的一次性负 blip(需给 expose/discovery 加 `last_reset` 管线,FU11 当时为省事未做)。已与用户确认本里程碑**先不做**,先观察、需要时再加。
- **Gas / 区域供暖计费**`commodity!="electricity"` 的 strategy 与寄存器映射。
- **当前电表 label 暴露给 HA**、按表/按地址的成本报表分段。
## 10. 里程碑完成定义(DoD
**已完成**M7-T01..T07 全部 done
- `meter` 数据地基 + 回填上线;计费引擎永不跨表算 delta,跨表/无表/异常 delta 一律降级。
- 累计量按当前表归零;追溯换表可重算。
- Meter CRUD API + 前端管理 UI 可用;OpenAPI/schema 已同步。
- 全程数据安全红线无违反;pytest/ruff/前端闸门全绿。
- 人工 walkthrough:声明一次(含追溯)换表 → 旧表周期保留、跨表周期降级、新表从零累计、HA 累计实体不出现负/巨额跳变。
+114
View File
@@ -0,0 +1,114 @@
# Meter Epochs(电表生命周期 / 换表归档)
本文档说明 **Meter epoch** 概念、换表声明流程、计费隔离行为、累计量归零、追溯重算,以及与 HA 累计 blip 的已知行为。
## 为什么需要 Meter epoch
计费引擎采用**寄存器差值(delta)模型**:每 15 分钟成本 =(周期末读数 − 周期初读数)× 单价。荷兰 2G 智能电表退网后,电网公司会把表换成 4G 表(同址换表);搬家继承新表也是同类场景。
问题在于:**新表的寄存器基数与旧表无关**,若跨表算 delta,会产出负成本或巨额假成本。Meter epoch 让引擎知道"某时刻起属于哪一块物理表",从而永不跨表算 delta。
## 核心概念
一条 `meter` 记录 = **一块物理电表的一段安装期** `[started_at, ended_at)`
- `started_at`:表安装/继承/搬家的起点(可在过去,追溯声明)。
- `ended_at`null = 当前 active;非 null = 已退役表的结束时刻。
- `label`:人读标签,便于识别(如 "旧 2G 表 @ Dorpsstraat 1")。
- `reason``initial`(初始建档)/ `meter_swap`(换表)/ `home_move`(搬家)/ `other`
- `commodity`:默认 `electricity`;为未来 gas/heating 预留,每 commodity 至多一个 active 表。
**换表** = 关闭旧 active 表(`ended_at = T`+ 新建 `started_at = T` 的表。系统**无法自动识别换表**(DSMR 帧无电表序列号),靠用户**显式声明**才能隔离。
## 计费隔离行为
每个 15 分钟计费周期,引擎先查该周期两端(t0/t1)分别属于哪块表:
| 情况 | 结果 |
| --- | --- |
| 同一块表(正常)| 在表窗口内取读数、算 delta、正常计费 |
| 无表覆盖(未声明)| **降级**(成本置 0`degraded=true` |
| 跨表边界(t0 ≠ t1 的表)| **降级**(丢这一个周期,可接受;换表常伴随长时间无数据) |
| delta < 0 或 > 100 kWh/15min | **降级**(兜底异常,如 DSMR 回绕/毛刺,与换表无关也防住) |
**忘了声明换表怎么办**:搬家必有的数据空档(staleness)+ delta 护栏保证不会产出垃圾成本,只是新数据暂混在旧表 epoch、累计未归零。事后补声明 + 追溯重算即可纠正(见下方"追溯换表")。
## 换表声明流程
### 通过前端(Energy 页 → Meters tab
1. 在 Energy 页切到 **Meters** tab,查看当前电表时间线。
2. 点击"声明换表 / 继承新表",填写:
- **Label**:新表的人读标签(如 "4G 新表 2026"
- **安装日期**:新表的 `started_at`(可选择历史日期追溯)
- **Reason**`meter_swap` / `home_move` / `other`
3. 保存后,旧表自动关闭(`ended_at = 填写日期`),新表成为 active。
4. 如填写的是过去日期,系统自动触发受影响窗口的重算(`recompute_range`),跨表周期变为降级,新表内周期重新归属。
### 通过 API
```http
POST /api/energy/meters
Content-Type: application/json
{
"label": "4G 新表 2026",
"started_at": "2026-07-01T00:00:00",
"reason": "meter_swap",
"note": "荷兰电网公司换装"
}
```
`started_at` 必须 `≥` 当前 active 表的 `started_at`(拒绝倒挂)。
查询所有表(时间线):
```http
GET /api/energy/meters
```
编辑 label / 备注 / 修正日期:
```http
PATCH /api/energy/meters/{id}
```
## 追溯换表(started_at 在过去)
若换表日期在过去但当时忘了声明:
1. 声明新表,填写过去的安装日期。
2. 系统自动对 `[started_at, now]` 范围内的周期触发 `recompute_range`
- 跨表边界周期 → 降级。
- 归属新表窗口内的周期 → `meter_id` 更新为新表。
3. MQTT 累计量锚点切换至新表起点,下一次 expose 推送时累计量从新表起点重算。
## 累计量归零(per-meter
MQTT 上报的累计成本实体(`import_cost_total` / `export_revenue_total`)锚点 = **当前 active electricity 表的 `started_at`**。换表后:
- 累计量从新表安装时刻起重新累加,不跨表维护历史偏移。
- 日归零实体(`*_today`)不受影响(其窗口必落在当前表内)。
### 已知行为:HA 长期统计的一次性负 blip
累计实体 `state_class: total`,换表归零时其值会**下台阶**。Home Assistant 长期统计(energy dashboard / statistics)可能在换表那一刻记录一次性负 blip。
这是**已知、可接受的行为**:HA 自身存旧值做 down-sample,短期 blip 不影响日常电费读数。已与用户确认先这样观察。如需消除 blip,可在未来通过 `last_reset` 信号通知 HA(见设计文档 §9 后续杠杆,本里程碑不做)。
## 回填(迁移时的初始表)
首次升级到含 M7 迁移的版本时:
- 若库内已有任何 `dsmr_reading` / `energy_cost_period`,迁移自动创建一条 `reason="initial"` 的初始表(`started_at = 最早 dsmr_reading.recorded_at`),并回填现有 `energy_cost_period.meter_id`
- 若是全新空库,初始表不创建(无历史数据)。
- 回填幂等:重复跑迁移不会创建多条初始表;回填后对账(非降级周期 `meter_id IS NULL` 数必须为 0)。
## 非目标(本里程碑不做)
- Gas / 区域供暖的计费(`commodity != "electricity"` 的 strategy)。
- "家庭(home)"分组实体;多合同时间线积分。
- 自动识别换表(DSMR 帧无电表序列号,无法自动识别,靠用户显式声明)。
- `last_reset` 信号(消除 HA 长期统计 blip)。
> 详细设计与任务卡:[`docs/design/m7-meter-epochs-archival.md`](./design/m7-meter-epochs-archival.md)
Binary file not shown.
+198
View File
@@ -0,0 +1,198 @@
# DDSU666 Modbus 协议(从官方 PDF 提取)
> 来源:`docs/references/DDSU666 Single phase Smart Meter.pdf`
> CHINT / 正泰仪表 **DDSU666 Single phase Smart Meter — Operation Manual**,文档号 `ZTY0.464.1224`,版本 **V2**2020 年 8 月;厂商 Zhejiang Chint Instrument & Meter Co., Ltd.
> 本文件是 PDF 的可读化提取,供本项目的 Modbus 采集驱动设计参考。**以官方 PDF 为准**,本文件如有出入以 PDF 为准。
> 同类文档见 SDM120 的 `SDM120-Modbus-Protocol.md`;两表差异较大,见下方 §6「与 SDM120 的关键差异」。
## 设备速览(来自手册 Table 1 / Table 5
| 项 | DDSU666(直接接入) | DDSU666-CT(经互感器) |
| --- | --- | --- |
| 精度等级 | Active Class B | Active Class C |
| 参考电压 | 230 V | 230 V |
| 电流规格 | 0.255(80) A | 0.0151.5(6) A |
| 表常数 | 800 imp/kWh | 6400 imp/kWh |
| 接入方式 | 直接接入 | 经电流互感器 |
- 单相电子式电能表,DIN35mm 导轨安装;测量电压、电流、有功/无功功率、频率、功率因数、正/反向有功电能。
- 电能测量范围 `0999999.99 kWh`(LCD 只显示 6 位,自动移动小数点)。
- 通信:RS485**Modbus-RTU**(也支持 DL/T 645-2007,可切换,见 §5 `0005H ChangeProtocol`)。
- 手册 Table 1 标注 Frequency Reference = 60Hz,但 LCD 示例又写 `F=50.00Hz`(手册自身不一致);**实际频率以寄存器 `200EH` 读数为准**,不要把 50/60 写死。
## 0. 本项目的接入方式(重要)
DDSU666 物理层是 **Modbus RTURS-485 串口)**,半双工。和 SDM120 一样,本项目通过一个 **Modbus-TCP 网关**接入:
- 后端用 **Modbus TCP**`IP:port`)连到网关,网关在串口侧转成 RTU 与电表通信。
- TCP 帧用 MBAP header、**无 CRC**CRC 由网关在 RTU 侧处理)。本文档里 RTU 帧的 `CRC (Lo/Hi)` 字段在 TCP 模式下不需要我们关心。
- **Slave Address / Unit ID = 电表的通信地址 Addr**(范围 1–247;面板按键只能设 1–99;见 §5 `0006H`),在 TCP 请求里作为 unit id 传入。
- 若以后直连串口(RTU),才需要管波特率 / 数据格式 / CRC:**默认串口格式是 8 数据位、无校验、2 停止位(8N2)**,与 SDM120 的 8N1 不同——直连时务必对齐。
- 电表必须处于 **Modbus 协议模式**(而非 DL/T 645)才能用本协议;可经面板长按切换,或写 `0005H = 2`(见 §5)。
## 1. 协议帧格式(Appendix A
异步传输,按字节为单位。一帧 10 位字符 = **1 起始位(0) + 8 数据位(无校验) + 2 停止位(1)**(其它格式可定制)。
### 信息帧结构(Table A.1
| 字段 | 长度 | 说明 |
| --- | --- | --- |
| Start(起始) | >3.5 字符静默 | 帧间至少 3.5 字符空闲时间作为分隔 |
| Address code(地址码) | 1 字节 | 目标从机地址 1–247;每个从机在总线上地址唯一 |
| Function code(功能码) | 1 字节 | 仅支持 **03H / 10H**(见 §2 |
| Data(数据域) | n 字节 | 随功能码不同而不同(起始地址、寄存器数、寄存器数据等) |
| CRC check codeCRC 校验) | 2 字节 | 16-bit CRC(**低字节在前、高字节在后**;多项式 `A001` |
| End(结束) | >3.5 字符静默 | 帧间静默 |
> TCP 网关模式下 Start/End 静默与 CRC 由网关处理,本项目不关心。
### 功能码 03H 示例(读寄存器,Table A.3/A.4
读从机 `01H`、起始地址 `0CH`、2 个寄存器:
- 主机发送:`01 03 00 0C 00 02 04 08`(最后 `04 08` 是 CRC,低字节 `04` 在前)。
- 从机返回(设 `0CH/0DH` 内容为 `0000H``1388H`):`01 03 04 00 00 13 88 F7 65`
- `04` = 字节数;`00 00` = `0CH` 数据;`13 88` = `0DH` 数据;`F7 65` = CRC(低字节 `F7` 在前)。
> **注意**:单个 16-bit 寄存器内是「高字节在前、低字节在后」(Table A.4 里 `0DH` 数据返回 `13 88` = `0x1388`)。这一点对解码浮点的字节序很关键,见 §3。
### 功能码 10H 示例(写多个寄存器,Table A.5/A.6
向从机 `01H`、起始地址 `00H` 连续写 3 个寄存器 `0002H,1388H,000AH`
- 主机发送:`01 10 00 00 00 03 06 00 02 13 88 00 0A 9B E9`
- `06` = 写入字节数(3 寄存器 × 2 字节);随后是 3 个寄存器的数据;末尾 `9B E9` CRC。
- 从机返回:`01 10 00 00 00 03 80 08`(回显起始地址 + 寄存器数 + CRC `80 08`)。
### 异常响应(Table A.7/A.8
- 异常时返回的 Function Code = **原功能码 + 128**(即最高位置 1`03H→83H``10H→90H`)。
- 数据为单字节 Error Code
| Error Code | 含义 | 说明 |
| --- | --- | --- |
| `01H` | Illegal function code | 收到的功能码本表不支持 |
| `02H` | Illegal register address | 寄存器地址超出有效范围 |
| `03H` | Illegal data value | 数据值超出对应地址的取值范围 |
## 2. 功能码
DDSU666 **只支持两个功能码**Table A.2):
| 功能码 | 作用 | 说明 |
| --- | --- | --- |
| **03H** | Read register(读寄存器) | 读一个或多个寄存器——**测量值、电量、配置全部走它** |
| **10H** | Write multiple registers(写多个寄存器) | 向 n 个连续寄存器写 n 个 16-bit 数据(改配置 / 清电量) |
> ⚠️ **DDSU666 没有「输入寄存器 / FC04」概念**——所有数据(包括电压电流功率)都用 **FC 03H** 读保持寄存器。这是它和 SDM120(测量值走 FC04)最大的踩坑差异,见 §6。
## 3. 数据编码
DDSU666 有两类数据:
1. **配置 / 参数寄存器(§5`0000H``0010H`**:每个 1 个寄存器、`16-bit with symbols`**16 位有符号整数**)。
2. **测量 / 电量寄存器(§4`2000H`+ / `4000H`+**:每个参数 = **32-bit IEEE-754 单精度浮点**(手册写 “single precision floating decimal”),占 **2 个相邻寄存器**Length = 2 Word)。
**浮点字节序 / 字序**
- **字节序(byte order)= 大端:寄存器内高字节在前** —— 由 Table A.4 的 `0x1388` 返回为 `13 88` 确认。
- **字序(word order,两个寄存器谁是高 16 位)**:手册**没有给出浮点解码的实例**,未明确标注。按标准 Modbus 浮点惯例应为**大端字序(高寄存器在前,`ABCD`)**,与本项目 SDM120 驱动一致(`registers_to_float``>f`,高寄存器在前)。
- ⚠️ **需上机实测确认**:读 `2000H`(电压)应解出 ~230V 这样的合理值;若解出乱数,多半是字序相反,改成「低寄存器在前」再试。CHINT 同系列(DTSU/DDSU666)现场固件偶有字序差异,**首次接入务必用一个已知量(电压)校验**,不要凭手册想当然。
> Python 解码(大端、高寄存器在前):`struct.unpack('>f', struct.pack('>HH', hi_reg, lo_reg))[0]`
> pymodbus`BinaryPayloadDecoder.fromRegisters(regs, byteorder=Endian.BIG, wordorder=Endian.BIG)`
## 4. 测量 / 电量寄存器表(FC 03H 读)
全部为只读、`Float`32-bit),每项占 **2 个寄存器**。地址为 Modbus 协议原始地址(即帧里的 Start Register Address Hi/Lo),手册用十六进制。
### 4.1 瞬时量(“Electric quantity of the secondary side”,`2000H` 段)
| 地址(hex) | 参数 | 代号 | 单位 | 备注 |
| --- | --- | --- | --- | --- |
| `2000H` | 电压 Voltage | U | V | |
| `2002H` | 电流 Current | I | A | |
| `2004H` | 有功功率 Active power | P | **kW** | 手册标注 “the unit is KW”——**不是 W** |
| `2006H` | 无功功率 Reactive power | Q | **kvar** | |
| `2008H` | (保留 RESERVED | — | — | 占 2 寄存器,跳过 |
| `200AH` | 功率因数 Power factor | PF | — | 无量纲 |
| `200CH` | (保留 RESERVED | — | — | 占 2 寄存器,跳过 |
| `200EH` | 频率 Frequency | Freq | Hz | |
### 4.2 电量(“Electrical data of the secondary side”,`4000H` 段)
| 地址(hex) | 参数 | 代号 | 单位 | 备注 |
| --- | --- | --- | --- | --- |
| `4000H` | 正向(导入)有功电能 Active in electricity | Ep | kWh | 正向 / forward active energy |
| `400AH` | 反向(导出)有功电能 Reverse in electricity | -Ep | kWh | 反向 / reverse active energy |
> 手册里 `4000H``400AH` 之间(`4002H``4009H`)未列出,视为保留/未文档化。
**读取分块建议**
- 瞬时量:`2000H``200FH`**16 个寄存器连续**,一次块读即可覆盖 U…Freq(含两段 RESERVED,解码时跳过)。
- 电量:`4000H`2 寄存器)与 `400AH`(2 寄存器)相距较远,分两小块读,或读 `4000H``400BH`(12 寄存器)一次取出后挑用——以网关 / 电表是否允许跨保留地址块读为准,谨慎起见分开读更稳。
### 常用核心子集(日常监控够用)
电压 `2000H`、电流 `2002H`、有功功率 `2004H`kW)、功率因数 `200AH`、频率 `200EH`、正向有功电能 `4000H`、反向有功电能 `400AH`
## 5. 配置 / 参数寄存器表(FC 03H 读 / FC 10H 写)(Table 9
每个 1 个寄存器、**16-bit 有符号整数**。`R/W` 列来自手册。
| 地址(hex) | 代号 | 含义 | R/W | 取值 / 说明 |
| --- | --- | --- | --- | --- |
| `0000H` | UCode | 编程密码 Programming password code | R/W | 写配置前的密码字 |
| `0001H` | REV. | 保留;**实际读出的是版本号** | R | |
| `0002H` | ClrE | 电能清零 CLr.E | R/W | **写 `1` 清除总电量**(不可逆,慎用) |
| `0003H``0004H` | RESERVED | 保留 | — | |
| `0005H` | ChangeProtocol | 协议切换 | R/W | **`2` = Modbus-RTU**`1` = DL/T 645-2007 |
| `0006H` | Addr | 通信地址 | R/W | 1247(面板按键仅 199 |
| `0007H``000AH` | RESERVED | 保留 | — | |
| `000BH` | Meter type | 表型 Meter type | R | 只读设备类型标识 |
| `000CH` | BAud | 通信波特率 | R/W | **`1`=2400bps`2`=4800bps`3`=9600bps**(手册寄存器仅列这三档;通信章另提到也支持 1200bps) |
| `000DH``0010H` | RESERVED | 保留 | — | |
> ⚠️ 写 `0002H`(清电量)、`0006H`(改地址)、`000CH`(改波特率)、`0005H`(切协议)都会改变电表状态或通信参数,配错可能**清空累计电量**或**导致通信中断**。本项目默认**只读采集**,不在自动化链路里写电表配置寄存器。
> 写配置通常需先经 `0000H UCode` 密码字校验;具体密码值手册正文未给出,需向厂商确认或经面板操作。
## 6. 与 SDM120 的关键差异(迁移 / 复用驱动时必看)
本项目已有 SDM120 profile`app/integrations/modbus/profiles/sdm120.yaml`)。DDSU666 **不能照搬**,主要差异:
| 维度 | SDM120 (Eastron) | DDSU666 (CHINT) |
| --- | --- | --- |
| 读测量值功能码 | **FC 04**(输入寄存器 3X | **FC 03**(保持寄存器,无 FC04 |
| 测量值起始地址 | `0x0000` 起(30001 | 瞬时量 `0x2000` 起;电量 `0x4000`/`0x400A` |
| 有功功率单位 | **W**(瓦) | **kW(千瓦)** —— 入库前注意换算 / 单位标注 |
| 无功功率单位 | VAr | kvar |
| 配置寄存器格式 | FloatFC03/16 | **16-bit 有符号整数**FC03/10 |
| 写功能码 | 16 / 0x10 | 10H(同 0x10 |
| 串口默认格式 | 8N1(1 停止位) | **8N22 停止位)** |
| 多协议 | 仅 Modbus | Modbus **与 DL/T 645-2007 可切换**(需确保在 Modbus 模式) |
| 浮点字序 | 大端、高寄存器在前(手册有实例佐证) | 字节序大端已确认;**字序手册无实例,需上机实测** |
## 7. 给本项目采集驱动的要点小结
1. 走 **Modbus TCP 网关**`ModbusTcpClient(host, port)``slave=<Addr>`;电表须在 **Modbus 协议模式**
2. 所有读取(测量 + 电量 + 配置)都用 **FC 03H**——**没有 FC04**。
3. 测量值在 `0x2000` 段、电量在 `0x4000`/`0x400A`,均为 **float32**;解码大端字节序,**字序默认高寄存器在前但务必用电压实测校验**。
4. **有功功率单位是 kW、无功是 kvar**——与 SDM120 的 W/VAr 不同,新建 profile / 入库映射时单位别抄错。
5. 配置寄存器(`0x0000``0x0010`)是 **16-bit 有符号整数**,不是 float。
6. 默认**只读**`0002H` 写 1 会**清空累计电量**、`0006H/000CH/0005H` 会改通信参数,自动化链路里一律不写。
7. 新建 profile 时这是 `ddsu666` 这一个 register profile 的定义;建议 `function_code: 3``word_order: big`(先按大端字序,接入后用电压读数验证)、瞬时量与电量分块读。
## 8. 实测记录(真机验证,2026-06-30)
首次接入一台 **DDSU666 直接接入版(5(80)A** 实测,确认以下几点:
- **字序大端,确认无误**:电压 / 电流 / 频率 / 电能用「大端、高寄存器在前」解码全部得到合理值(如 233.9 V / 0.055 A / 49.99 Hz),与 §3 的假设一致。`ddsu666.yaml``word_order: big` / `byte_order: big` **无需修改**,§3 里「字序需上机实测」一项可视为已关闭。
- **FC03 读通**:所有量走 FC03profile `ddsu666` 在采集链路(CLI `read` / 设备 `/test` / 后台轮询)中工作正常。
- **低电流下「瞬时功率读 0、但电能照常累加」**:测试负载仅为一台 PoE 交换机(≈230 V / 0.05 A,真实有功仅几瓦),**远低于本表测量量程下限 Imin≈0.25 A**。此工况下:
- 瞬时 `active_power``0x2004`)与 `power_factor``0x200A`)寄存器返回**全零**(原始 hex `0x0000 0x0000`);电表 LCD 上功率在 0~3.3 W、PF 在 0~0.6 之间抖动。
- 抖动成因 = **低电流测量噪声 + 开关电源(SMPS,无 PFC)畸变电流**(电流为电压峰值附近的窄脉冲、谐波重 → 畸变功率因数天然偏低)。**不是**「采样率与开关频率拍频」:计量芯片 SH79F7019 采样在 kHz 量级,远低于开关电源 50–200 kHz 的开关频率,两者不在一个频段。
- 但累计电能 `import_energy``0x4000`)**正常累加**(实测 0 → 0.01 kWh)——电表内部积分器在防潜动起始电流(`0.004·Ib`≈0.02 A)之上照常计量。
- **结论**:低于量程下限时**瞬时功率 / PF 不可信,但电能计量不丢**;电流进入量程(正常负载)后瞬时量即稳定可信。这是 5(80)A 大量程表对极小负载的固有特性,**非缺陷、非解码问题**。
- **排错提示**:若日后看到 DDSU666「功率一直 0」,先确认负载电流是否在 Imin 以上——多半是负载太轻而非链路故障;可用 `scripts/modbus_cli probe --fc 3 --address 0x2000 --count 16``0x2004/0x2005` 原始寄存器是否真为全零佐证。
+108 -2
View File
@@ -2,7 +2,7 @@
本文档记录 `home-automation``v1.0.3` 之后的下一阶段规划。这一阶段不是小修补,而是几次较大的结构性改动:单库化、前端重写、以及远期的移动端试水。
> 每个里程碑的**可执行原子任务**展开在 [`docs/design/`](./design/README.md)M1 [`m1-db-consolidation.md`](./design/m1-db-consolidation.md)、M2 [`m2-frontend-v2.md`](./design/m2-frontend-v2.md)、M3 [`m3-token-mobile.md`](./design/m3-token-mobile.md)、M4 [`m4-login-hardening.md`](./design/m4-login-hardening.md)、M5 [`m5-iot-energy.md`](./design/m5-iot-energy.md)。这些文档为 Orchestrator→Implementer→Reviewer 的多模型流水线设计。
> 每个里程碑的**可执行原子任务**展开在 [`docs/design/`](./design/README.md)M1 [`m1-db-consolidation.md`](./design/m1-db-consolidation.md)、M2 [`m2-frontend-v2.md`](./design/m2-frontend-v2.md)、M3 [`m3-token-mobile.md`](./design/m3-token-mobile.md)、M4 [`m4-login-hardening.md`](./design/m4-login-hardening.md)、M5 [`m5-iot-energy.md`](./design/m5-iot-energy.md)、M6 [`m6-tibber-dynamic-energy.md`](./design/m6-tibber-dynamic-energy.md)、M7 [`m7-meter-epochs-archival.md`](./design/m7-meter-epochs-archival.md)。这些文档为 Orchestrator→Implementer→Reviewer 的多模型流水线设计。
## 当前基线(v1.0.3
@@ -38,6 +38,8 @@
| **M2** ✅ | 前端 v2 | React SPA 取代 Jinja,承载 config + 可视化 + 记录增删改 |
| **M4** ✅ | 登录加固 | 防爆破/指数退避 + CLI 逃生通道 + 可选 TOTP 二次验证(**先于 M5** |
| **M5** ✅ | IoT / 能耗采集 | 通用 Modbus 采集(YAML profile + JSON readings+ MQTT/HA Discovery + 前端侧边栏 + Energy 视图 |
| **M6** ✅ | 通用电价层 + DSMR 接入 + 实时电费计算 | 通用电价层(manual/tibber profile + 合同版本)+ DSMR 实时电表接入 + 每 15min 寄存器差×价计量电费(不可变快照)+ 日/月/年汇总 + 反哺 HA Energy + 前端合同/价格/费用视图 |
| **M7** ✅ | 电表生命周期 / 换表归档 | 引入 Meter epoch,计费永不跨表算 delta,跨表/无表/异常 delta 一律降级,累计按当前表归零,追溯换表可重算,Meter CRUD API + 前端管理 UI |
| **M3** | 开放与移动端(远期试水) | token 鉴权 + React Native 移动端 |
排序原则:**先清地基,再在干净结构上盖楼。** M2 的新 API 和 React 必须建立在合并后的单库之上;M4 是公网安全加固,在 M5 IoT 集成之前先堵住裸密码这个洞;M5 在安全基座就绪后再做 IoT 接入。
@@ -176,6 +178,85 @@
---
## M6 — 通用电价层 + DSMR 实时数据 + 实时买卖电费计算(✅ 已完成)
### 目标
在 M5 IoT 基建之上,把"电价合同"与"DSMR 实时电表数据"接起来,按 **15 分钟**周期算出**实际买电支出 / 卖电收入**,自己落库留底(不可变、可审计),并通过 MQTT + HA Discovery 反哺 Home Assistant 的 Energy 仪表盘。
### 关键能力
- **通用电价层(两层模型)**:仓库内 YAML profile 定结构(`manual` 固定/双费率、`tibber` 动态电价);`EnergyContract`(+ 版本)存 UI 可改的数值;price strategy 按 `kind` 出价。一次激活一个合同。改价 = 加新版本,旧版本保留(审计链)。
- **DSMR 实时数据接入**:扩 `MqttManager` 订阅 `dsmr/json`(每秒一帧),整帧 JSON blob 按 10 秒降采样落库(电、气、各相全存;字段 allowlist 不做,blob 天然容纳未来字段)。
- **实时买卖电费计算(两层)**:每 15 分钟用电表累计寄存器差值(`delivered_1/2``returned_1/2`)× 当时合同的买/卖价算计量电费(不可变、快照价)。买价:tibber = API `total`(含税全包),manual = `energy_buy_档 + energy_tax`;卖价:tibber = `total energy_tax sell_adjust`manual = `sell_档`(无能源税)。双费率:`_1`=dal/低、`_2`=normal/高(NL 惯例)。日/月/年汇总再加固定费(按月→天)减 heffingskorting(按年→天)。不做净计量。卖价残差(sell_adjust)及能源税等当前数值在真实账单确认后钉死。
- **反哺 Home Assistant**`_energy_cost_provider``buy_price_now``sell_price_now``import_cost_total``total_increasing`)、`export_revenue_total``total_increasing`)—— 直接挂 HA Energy 仪表盘。
- **Tibber 动态电价**httpx GraphQL 客户端抓 `QUARTER_HOURLY` 15 分钟价(已 demo 验证 `total=energy+tax`);启动 + 每小时抓取今明两天价格、幂等 upsert `tibber_price`hourly trigger,确保每日刷新且可补重试);仅当 active 合同 kind=tibber 且 token 存在时运行。
- **前端**:Energy 视图新增三个 Tab——Contracts(合同管理,表单按 profile 结构渲染)、Prices15 分钟价格曲线)、Costs(费用走势/明细/汇总卡片);Config 页 Tibber 测试三态入口。
### 命名分层(延续 M5
存储/采集/计算层中性命名(`dsmr_*` / `tibber_price` / `energy_cost_period` / `energy_contract`);面向用户并入既有 **Energy** 视图。
### 新增 5 张表(单库 app 链,migration `20260623_11_energy_tables`
| 表 | 关键列 | 说明 |
| --- | --- | --- |
| `dsmr_reading` | `recorded_at`(idx)、`source_id`(unique)、`payload`(JSON) | 整帧 DSMR telegram blob10 秒降采样 |
| `energy_contract` | `name``kind``active``currency`、时间戳 | 合同头;一次一个 active |
| `energy_contract_version` | `contract_id`(FK)、`effective_from``effective_to`(null)、`values`(JSON)、`created_at` | 版本/时段;改价加新版本、旧版本保留 |
| `tibber_price` | `starts_at`(unique)、`resolution``energy/tax/total``level``currency``fetched_at` | 15 分钟价缓存,不可变 |
| `energy_cost_period` | `period_start`(unique)、`d1/d2/r1/r2_kwh``import_cost/export_revenue/net_cost``currency``pricing`(JSON 快照)、`contract_version_id`(FK)、`degraded``computed_at` | 每 15 分钟计量电费,不可变(快照价+版本) |
### 不新增依赖
httpx / paho-mqtt / pyyaml / apscheduler 均为 M5 已有依赖,M6 复用,不新增任何 Python 包。
> 详细设计与任务卡:[`docs/design/m6-tibber-dynamic-energy.md`](./design/m6-tibber-dynamic-energy.md)
---
## M7 — 电表生命周期 / 换表归档(✅ 已完成)
### 目标
让计费系统正确处理**电表更换**这一必然事件:荷兰 2G 智能电表退网后,电网公司会把表换成 4G 表(同址换表);搬家继承新表也是同类。引入显式的 **Meter(电表)epoch** 概念,标记"某时刻起属于哪一块物理表",使引擎**永不跨表算 delta**,并让累计量按表归零、历史可追溯可查。
### 关键能力
- **Meter epoch 数据模型**:一条 `meter` 记录 = 一块物理电表的一段安装期 `[started_at, ended_at)`。换表 = 关闭旧 active 表(`ended_at=T`+ 新建 `started_at=T` 的表。每 `commodity` 至多一个 active 表;`commodity` 字段预留 `gas`/`heating`
- **计费永不跨表**`compute_period` 先查 t0/t1 两端的 `meter_at`;无表覆盖或跨表边界 → **降级**(成本置 0),绝不产负成本或巨额假成本。
- **delta 护栏**:任一寄存器 delta `< 0``> _MAX_DELTA_KWH`100 kWh/15min,远超住宅用量)→ 降级。兜底表重置 / DSMR 回绕 / 数据毛刺,与换表无关的异常也一并防住。
- **累计按当前表归零(D2**`expose.py` 累计 `import_cost_total`/`export_revenue_total` 锚点 = 当前 active electricity meter 的 `started_at`;换表后累计从零起新序列,不维护跨表偏移。
- **追溯换表可重算**`started_at` 可在过去;PATCH 修正日期后,触发受影响窗口 `recompute_range` 重判跨表周期归属。
- **Meter CRUD API + 前端管理 UI**`GET/POST/PATCH /api/energy/meters`+ 追溯 recompute);Energy 页新增 Meters tab,展示电表时间线,支持"换表/继承"表单与编辑。
### 新增表与列(单库 app 链,migration `20260625_13_meter_table`
| 表/列 | 关键设计 |
| --- | --- |
| `meter` | `id``label``commodity`(默认 `electricity`)、`started_at``ended_at`null=active)、`reason``initial`/`meter_swap`/`home_move`/`other`)、`note``created_at` |
| `energy_cost_period.meter_id` | nullable FK → `meter.id`;记录每周期归属,便于审计/按表查询/归档 |
迁移含回填:有历史数据时自动创建一条 `reason="initial"` 的初始表,并回填现有 `energy_cost_period.meter_id`;幂等 + 对账(非降级周期回填后 `meter_id IS NULL` 数必须为 0)。
### 已锁定决策摘要
- **D1**Meter 不绑定 home`label` 编址 + `commodity` 区分品类。
- **D2**:换表后累计量归零(锚当前表起点),不维护跨表偏移。
- **D3**:合同与电表是独立时间线,合同不引用电表;同址换表时合同自动沿用。
- **D4**`started_at` 可在过去;有效计费起点 = `max(started_at, 数据起点)`
- **D5**:跨表边界周期判 degraded(换表常伴随长时间无数据,可接受)。
- **D6**:负/异常大 delta → degraded,通用兜底。
- **D7**gas/heating 计费不在本里程碑;`commodity` 字段为未来扩展预留。
**已知行为(可接受)**:累计 `_total``state_class: total`,换表归零时 HA 长期统计可能在换表那一刻记一次性负 blip。已与用户确认先这样、观察后按需处理(`last_reset` 信号见设计文档 §9 留痕,本里程碑不做)。
> 详细设计与任务卡:[`docs/design/m7-meter-epochs-archival.md`](./design/m7-meter-epochs-archival.md)
>
> 模块概念说明:[`docs/meter-epochs.md`](./meter-epochs.md)
---
## M3 — 开放与移动端(远期试水)
### 目标
@@ -207,11 +288,36 @@
**动机**:浏览器端走 session cookie 即可,但**脚本 / 设备 / 外部程序调用 API** 需要一种长期有效、可随身携带的凭据。在设置页加一组功能,由 admin **手动签发 long-lived token**,之后用它来调 API。
**本次明确的首要目标 = 给现在裸奔的 ingestion 端点上鉴权**2026-06-27 与用户确认):
- `POST /location/record``app/api/routes/location.py:18`)——位置记录上报。**目前无任何鉴权**。当前数据经 Home Assistant 转发进来,上 token 后 **HA 侧需携带该 token**;也可由其他客户端直接上报。
- `POST /poo/record``app/api/routes/poo.py:21`+ `GET /poo/latest``poo.py:57`)——小狗排便记录上报 / 最新查询。**目前无任何鉴权**。
- 这些是设备 / 脚本(非浏览器)端点,session cookie 不适用,正是 long-lived token 的用武之地。(浏览器 CRUD `/api/data/*` 已由 session 保护,不在此列。)
**范围(粗略,待细化)**
- 设置页新增「API Token」区:生成 / 命名 / 吊销 long-lived token;明文只在**生成时展示一次**,此后只存哈希。
- 后端支持用该 token 鉴权访问 API(与现有 session cookie 并存,互不影响)。
- 后端支持用该 token 鉴权访问 API(与现有 session cookie 并存,互不影响);给上述 ingestion 端点加 token 鉴权依赖
- 与 [M3](#m3--开放与移动端远期试水) 的 token 主题相关,但**这条是 Web 设置页手动签发的 PAT 风格**,不依赖移动端 OAuth 流程;两者实现时可复用同一套 token 存储 / 校验。
- 与下面第 3 条「Session 滑动续期」同属 Authentication 主题(一个是设备/脚本的长期凭据,一个是浏览器短会话体验),实现时鉴权层可一并梳理。
### 3. Session 滑动自动续期(Authentication
**动机**2026-06-27 与用户确认):当前 session 是**绝对过期**——登录即定死、活动不续期,满 TTL 必须重新登录,体验割裂。希望改成**滑动续期(sliding / rolling)**:只要用户还在活动就自动延长,提供"在用就不掉线"的体验。
**现状(实现起点,便于快速拾起)**:
- TTL 默认 **12 小时**`auth_session_ttl_hours``app/config.py:38`;配置页 `app/services/config_page.py:45` 可运行时改)。
- 登录时**一次性写死**`create_session``expires_at = now + ttl``app/services/auth.py:94`+ cookie `max_age = ttl``app/api/routes/api/session.py:153`)。
- 每请求**只读校验、从不延长**`get_authenticated_session``app/services/auth.py:103`)只判断 `expires_at <= now`,过期时仅顺手标 `revoked``set_cookie` 只在登录路由调用一次,**无 per-request 中间件**。→ 所以是绝对过期,不是滑动。
**设计要点(待写设计文档时展开)**:
- 校验通过时 bump `expires_at = now + ttl` 并**重发 cookie**(滑动窗口)。
- **写节流**:不要每个请求都写 DB——仅当剩余寿命已过半(或距上次续期 > N 分钟)才续期,避免高频写放大。
- **绝对寿命硬顶**:除滑动 TTL 外再设 `created_at + max_lifetime` 上限,防止"永不过期"的会话(安全考量)。
- 新增配置项:滑动 TTL、绝对寿命上限、续期节流阈值。
- 注意:改动只对**新逻辑生效**,已存在 session 的 `expires_at` 行为按新校验路径走即可;上线前过校验闸门。
## Future Ideas(暂不排期,想到先记下)
+1460 -4
View File
File diff suppressed because it is too large Load Diff
+338
View File
@@ -0,0 +1,338 @@
/**
* Tests for ContractForm component.
*
* Coverage:
* 1. Renders profile fields dynamically from profile structure (not hardcoded)
* verified by passing defaultKind so fields render automatically.
* 2. Submit in create mode calls POST /api/energy/contracts.
* 3. Submit in add-version mode calls POST /api/energy/contracts/{id}/versions.
* 4. Cancel closes modal.
*/
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { screen, waitFor } from '@testing-library/react'
import userEvent from '@testing-library/user-event'
import { renderWithProviders } from '../test-utils'
// ---------------------------------------------------------------------------
// Mock apiClient
// ---------------------------------------------------------------------------
const mockGet = vi.fn()
const mockPost = vi.fn()
vi.mock('../api/client', () => ({
default: {
GET: (...args: unknown[]) => mockGet(...args),
POST: (...args: unknown[]) => mockPost(...args),
PATCH: vi.fn(),
DELETE: vi.fn(),
},
ApiError: class ApiError extends Error {
status: number
body: unknown
constructor(status: number, body: unknown) {
super(`API error ${status}`)
this.name = 'ApiError'
this.status = status
this.body = body
}
},
registerLoginRedirect: vi.fn(),
}))
// ---------------------------------------------------------------------------
// Fixtures
// ---------------------------------------------------------------------------
const PROFILES_RESPONSE = {
profiles: [
{
kind: 'manual',
label: 'Fixed / Variable Rate (NL, dual-tariff)',
energy: {
dual_tariff: true,
buy: { normal: { unit: 'EUR/kWh' }, dal: { unit: 'EUR/kWh' } },
sell: { normal: { unit: 'EUR/kWh' }, dal: { unit: 'EUR/kWh' } },
energy_tax: { unit: 'EUR/kWh' },
ode: { unit: 'EUR/kWh', default: 0 },
},
standing: {
network_fee: { unit: 'EUR/month' },
management_fee: { unit: 'EUR/month' },
},
credits: {
heffingskorting: { unit: 'EUR/year' },
},
},
],
}
const CREATED_CONTRACT = {
id: 1,
name: 'Test Contract',
kind: 'manual',
active: false,
currency: 'EUR',
created_at: '2026-06-01T00:00:00Z',
updated_at: '2026-06-01T00:00:00Z',
versions: [],
}
// ---------------------------------------------------------------------------
// Import component
// ---------------------------------------------------------------------------
import { ContractForm } from './ContractForm'
// ---------------------------------------------------------------------------
// Tests
// ---------------------------------------------------------------------------
describe('ContractForm', () => {
beforeEach(() => vi.clearAllMocks())
it('renders profile fields dynamically from profile structure (not hardcoded)', async () => {
mockGet.mockImplementation((path: string) => {
if (path === '/api/energy/profiles') {
return Promise.resolve({ data: PROFILES_RESPONSE })
}
return Promise.resolve({ data: null })
})
const onClose = vi.fn()
const onSaved = vi.fn()
// Pass defaultKind so the profile fields render automatically in create mode
renderWithProviders(
<ContractForm defaultKind="manual" onClose={onClose} onSaved={onSaved} />,
)
// Wait for profiles to load and fields to appear
await waitFor(() => {
expect(screen.getByTestId('contract-section-energy')).toBeInTheDocument()
}, { timeout: 3000 })
// Verify sections are rendered from profile structure (not hardcoded)
expect(screen.getByTestId('contract-section-energy')).toBeInTheDocument()
expect(screen.getByTestId('contract-section-standing')).toBeInTheDocument()
expect(screen.getByTestId('contract-section-credits')).toBeInTheDocument()
// Verify specific fields exist from the profile structure
// "energy.buy.normal" → label "Buy Normal"
expect(screen.getByTestId('contract-field-energy.buy.normal')).toBeInTheDocument()
expect(screen.getByTestId('contract-field-energy.buy.dal')).toBeInTheDocument()
expect(screen.getByTestId('contract-field-standing.network_fee')).toBeInTheDocument()
})
it('calls POST /api/energy/contracts in create mode', async () => {
const user = userEvent.setup()
mockGet.mockImplementation((path: string) => {
if (path === '/api/energy/profiles') {
return Promise.resolve({ data: PROFILES_RESPONSE })
}
return Promise.resolve({ data: null })
})
mockPost.mockResolvedValue({ data: CREATED_CONTRACT })
const onClose = vi.fn()
const onSaved = vi.fn()
// Use defaultKind to ensure fields load without needing to interact with Select
renderWithProviders(
<ContractForm defaultKind="manual" onClose={onClose} onSaved={onSaved} />,
)
// Wait for form to render
await waitFor(() => {
expect(screen.getByTestId('contract-form')).toBeInTheDocument()
})
// Wait for profile fields to appear
await waitFor(() => {
expect(screen.getByTestId('contract-name')).toBeInTheDocument()
})
// Fill in contract name
await user.type(screen.getByTestId('contract-name'), 'Test Contract')
// Submit form
await user.click(screen.getByTestId('contract-form-submit'))
await waitFor(() => {
expect(mockPost).toHaveBeenCalledWith(
'/api/energy/contracts',
expect.objectContaining({
body: expect.objectContaining({
name: 'Test Contract',
kind: 'manual',
currency: 'EUR',
}),
}),
)
}, { timeout: 3000 })
})
it('calls POST /api/energy/contracts/{id}/versions in add-version mode', async () => {
const user = userEvent.setup()
mockGet.mockImplementation((path: string) => {
if (path === '/api/energy/profiles') {
return Promise.resolve({ data: PROFILES_RESPONSE })
}
return Promise.resolve({ data: null })
})
mockPost.mockResolvedValue({ data: CREATED_CONTRACT })
const onClose = vi.fn()
const onSaved = vi.fn()
renderWithProviders(
<ContractForm contractId={1} defaultKind="manual" onClose={onClose} onSaved={onSaved} />,
)
// Wait for form to be ready (add-version mode)
await waitFor(() => {
expect(screen.getByTestId('contract-form')).toBeInTheDocument()
})
// In add-version mode, no name or kind fields shown
expect(screen.queryByTestId('contract-name')).not.toBeInTheDocument()
expect(screen.queryByTestId('contract-kind')).not.toBeInTheDocument()
// Fill in effective_from (required in add-version mode)
const effectiveDateInput = screen.getByTestId('contract-effective-from')
await user.type(effectiveDateInput, '2026-07-01')
// Submit form
await user.click(screen.getByTestId('contract-form-submit'))
await waitFor(() => {
expect(mockPost).toHaveBeenCalledWith(
'/api/energy/contracts/{contract_id}/versions',
expect.objectContaining({
params: { path: { contract_id: 1 } },
body: expect.objectContaining({
values: expect.any(Object),
}),
}),
)
}, { timeout: 3000 })
})
it('calls onClose when Cancel button is clicked', async () => {
const user = userEvent.setup()
mockGet.mockImplementation((path: string) => {
if (path === '/api/energy/profiles') {
return Promise.resolve({ data: PROFILES_RESPONSE })
}
return Promise.resolve({ data: null })
})
const onClose = vi.fn()
const onSaved = vi.fn()
renderWithProviders(
<ContractForm onClose={onClose} onSaved={onSaved} />,
)
await waitFor(() => {
expect(screen.getByTestId('contract-form-cancel')).toBeInTheDocument()
})
await user.click(screen.getByTestId('contract-form-cancel'))
expect(onClose).toHaveBeenCalled()
})
it('prefills fields from the open version when in add-version mode', async () => {
// FU11: add-version mode should auto-prefill rate fields from the contract's
// current open version so the user only has to change what's different.
const CONTRACT_DETAIL = {
id: 42,
name: 'My Contract',
kind: 'manual',
active: true,
currency: 'EUR',
created_at: '2026-01-01T00:00:00Z',
updated_at: '2026-01-01T00:00:00Z',
versions: [
{
id: 1,
contract_id: 42,
effective_from: '2026-01-01T00:00:00Z',
effective_to: null, // open version
values: {
energy: {
buy: { normal: 0.133, dal: 0.127 },
sell: { normal: 0.05, dal: 0.05 },
energy_tax: 0.11,
ode: 0.0,
},
standing: {
network_fee: 30.0,
management_fee: 60.0,
},
credits: {
heffingskorting: 365.0,
},
},
created_at: '2026-01-01T00:00:00Z',
},
],
}
mockGet.mockImplementation((path: string) => {
if (path === '/api/energy/profiles') {
return Promise.resolve({ data: PROFILES_RESPONSE })
}
if (path === '/api/energy/contracts/{contract_id}') {
return Promise.resolve({ data: CONTRACT_DETAIL })
}
return Promise.resolve({ data: null })
})
const onClose = vi.fn()
const onSaved = vi.fn()
renderWithProviders(
<ContractForm contractId={42} defaultKind="manual" onClose={onClose} onSaved={onSaved} />,
)
const user = userEvent.setup()
// Wait for the form to render and profile fields to appear
await waitFor(() => {
expect(screen.getByTestId('contract-section-energy')).toBeInTheDocument()
}, { timeout: 3000 })
// Wait for prefill to apply: both profile and contract detail queries must resolve.
// Fill in the required effective_from date and submit; the submitted values
// body.values should contain the prefilled rates from the open version.
mockPost.mockResolvedValue({ data: {} })
const effectiveInput = screen.getByTestId('contract-effective-from')
await user.type(effectiveInput, '2026-07-01')
await user.click(screen.getByTestId('contract-form-submit'))
// The submitted body should contain the prefilled values from CONTRACT_DETAIL
await waitFor(() => {
expect(mockPost).toHaveBeenCalledWith(
'/api/energy/contracts/{contract_id}/versions',
expect.objectContaining({
body: expect.objectContaining({
values: expect.objectContaining({
standing: expect.objectContaining({
network_fee: 30,
management_fee: 60,
}),
}),
}),
}),
)
}, { timeout: 3000 })
})
})
+466
View File
@@ -0,0 +1,466 @@
/**
* ContractForm create a new energy contract OR add a version to an existing one.
*
* Form fields are dynamically rendered from the profile structure returned by
* GET /api/energy/profiles never hardcoded. The profile structure provides
* section names (energy, standing, credits), leaf keys, and their units.
*
* Create mode: show Name, Kind (Select from profiles), Currency, optional
* effective_from, then dynamically rendered profile value fields.
* Add-version mode (contractId provided): show only effective_from (required)
* + profile value fields for the contract's kind.
*/
import { useState, useMemo } from 'react'
import {
Modal,
Stack,
TextInput,
NumberInput,
Select,
Button,
Group,
Alert,
Loader,
Center,
Text,
Divider,
Title,
} from '@mantine/core'
import { useEnergyProfiles, useCreateContract, useAddContractVersion, useContractDetail } from './hooks'
// ---------------------------------------------------------------------------
// Props
// ---------------------------------------------------------------------------
export interface ContractFormProps {
/** When provided: add-version mode; contract must exist. */
contractId?: number
/** Existing contract kind (for add-version mode or edit). */
defaultKind?: string
onClose: () => void
onSaved: () => void
}
// ---------------------------------------------------------------------------
// Helpers for dynamic profile field rendering
// ---------------------------------------------------------------------------
/**
* A leaf in the profile structure is any object with a `unit` key.
* Returns true if the value is a leaf (renderable field).
*/
function isLeaf(val: unknown): val is { unit: string; default?: unknown } {
return typeof val === 'object' && val !== null && 'unit' in val
}
/**
* Extract all leaf fields from a profile section, returning
* [dotPath, unit, defaultValue] tuples. E.g.:
* "energy.buy.normal" "EUR/kWh"
* "energy.energy_tax" "EUR/kWh"
*/
interface LeafField {
/** Dot-separated path within the section, e.g. "buy.normal" */
fieldPath: string
unit: string
defaultValue?: number
}
function extractLeafFields(obj: Record<string, unknown>, prefix = ''): LeafField[] {
const fields: LeafField[] = []
for (const [key, val] of Object.entries(obj)) {
// Skip non-object values (e.g. dual_tariff: true)
if (typeof val !== 'object' || val === null) continue
const path = prefix ? `${prefix}.${key}` : key
if (isLeaf(val)) {
fields.push({
fieldPath: path,
unit: val.unit,
defaultValue: typeof val.default === 'number' ? val.default : undefined,
})
} else {
fields.push(...extractLeafFields(val as Record<string, unknown>, path))
}
}
return fields
}
/**
* Build a nested values object from flat field values.
* E.g. { "buy.normal": 0.133, "buy.dal": 0.127 } { buy: { normal: 0.133, dal: 0.127 } }
*/
function buildNestedValues(
sectionFields: Record<string, LeafField[]>,
fieldValues: Record<string, number | string>,
): Record<string, unknown> {
const result: Record<string, unknown> = {}
for (const [section, fields] of Object.entries(sectionFields)) {
const sectionObj: Record<string, unknown> = {}
for (const field of fields) {
const raw = fieldValues[`${section}.${field.fieldPath}`]
const numVal = typeof raw === 'number' ? raw : parseFloat(String(raw))
// Set nested path
const parts = field.fieldPath.split('.')
let current = sectionObj
for (let i = 0; i < parts.length - 1; i++) {
if (!(parts[i] in current)) current[parts[i]] = {}
current = current[parts[i]] as Record<string, unknown>
}
current[parts[parts.length - 1]] = isNaN(numVal) ? 0 : numVal
}
result[section] = sectionObj
}
return result
}
/**
* Label formatter: converts "buy.normal" "Buy Normal", "energy_tax" "Energy Tax"
*/
function formatLabel(path: string): string {
return path
.replace(/\./g, ' ')
.replace(/_/g, ' ')
.replace(/\b\w/g, (c) => c.toUpperCase())
}
// ---------------------------------------------------------------------------
// Component
// ---------------------------------------------------------------------------
export function ContractForm({ contractId, defaultKind, onClose, onSaved }: ContractFormProps) {
const isAddVersion = contractId != null
// Profiles query
const profilesQuery = useEnergyProfiles()
// Contract detail query — only used in add-version mode to prefill from the open version.
const contractDetailQuery = useContractDetail(contractId ?? 0)
// Create mode fields
const [name, setName] = useState('')
const [selectedKind, setSelectedKind] = useState<string | null>(defaultKind ?? null)
const [currency, setCurrency] = useState('EUR')
// ISO date string "YYYY-MM-DD" for effective_from; empty = default to now
const [effectiveFromStr, setEffectiveFromStr] = useState('')
// Dynamic profile field values: key = "<section>.<fieldPath>", value = number.
// User edits are stored here; for add-version mode this is merged with prefillValues.
const [userEdits, setUserEdits] = useState<Record<string, number | string>>({})
const [error, setError] = useState<string | null>(null)
const createMutation = useCreateContract()
const addVersionMutation = useAddContractVersion()
const isBusy = createMutation.isPending || addVersionMutation.isPending
// Resolve the effective kind (add-version mode uses defaultKind)
const effectiveKind = isAddVersion ? (defaultKind ?? null) : selectedKind
// Build profile options from API response
const profiles = profilesQuery.data?.profiles ?? []
const profileOptions = profiles.map((p: Record<string, unknown>) => ({
value: p.kind as string,
label: (p.label as string | undefined) ?? (p.kind as string),
}))
// Find the selected profile structure
const selectedProfile = profiles.find((p: Record<string, unknown>) => p.kind === effectiveKind)
// Extract section → leaf fields mapping from the selected profile (memoized for stable reference).
const sectionFields = useMemo<Record<string, LeafField[]>>(() => {
if (!selectedProfile) return {}
const profileObj = selectedProfile as Record<string, unknown>
const result: Record<string, LeafField[]> = {}
for (const [key, val] of Object.entries(profileObj)) {
if (key === 'kind' || key === 'label') continue
if (typeof val === 'object' && val !== null && !isLeaf(val)) {
result[key] = extractLeafFields(val as Record<string, unknown>)
}
}
return result
}, [selectedProfile])
// Add-version prefill: compute the seeded values from the contract's open version.
// Returns the flat fieldValues dict or null when data is not ready.
const prefillValues = useMemo<Record<string, number | string> | null>(() => {
if (!isAddVersion) return null
if (contractDetailQuery.isLoading || contractDetailQuery.isError) return null
if (Object.keys(sectionFields).length === 0) return null
const detail = contractDetailQuery.data
if (!detail || !detail.versions || detail.versions.length === 0) return null
const versions = detail.versions as Array<{
effective_from: string
effective_to: string | null
values: Record<string, unknown>
}>
let openVersion = versions.find((v) => v.effective_to == null)
if (!openVersion) {
openVersion = [...versions].sort((a, b) =>
a.effective_from < b.effective_from ? 1 : -1
)[0]
}
if (!openVersion?.values) return null
const seeded: Record<string, number | string> = {}
for (const [section, fields] of Object.entries(sectionFields)) {
const sectionVal = (openVersion.values as Record<string, unknown>)[section]
if (typeof sectionVal !== 'object' || sectionVal === null) continue
for (const leaf of fields) {
const parts = leaf.fieldPath.split('.')
let cursor: unknown = sectionVal
for (const part of parts) {
if (typeof cursor !== 'object' || cursor === null) { cursor = undefined; break }
cursor = (cursor as Record<string, unknown>)[part]
}
if (cursor != null && (typeof cursor === 'number' || typeof cursor === 'string')) {
const numVal = typeof cursor === 'number' ? cursor : parseFloat(String(cursor))
seeded[`${section}.${leaf.fieldPath}`] = isNaN(numVal) ? 0 : numVal
}
}
}
return Object.keys(seeded).length > 0 ? seeded : null
}, [
isAddVersion,
contractDetailQuery.isLoading,
contractDetailQuery.isError,
contractDetailQuery.data,
sectionFields,
])
// The effective field values: user edits override prefill; prefill is the base.
// In add-version mode: when the user has not yet edited any field (userEdits is empty),
// prefillValues is used as the initial base. Once the user edits any field, their
// full merged value (prefill + override) is stored in userEdits.
const fieldValues: Record<string, number | string> =
isAddVersion && prefillValues !== null && Object.keys(userEdits).length === 0
? prefillValues
: userEdits
function handleFieldChange(key: string, val: number | string) {
// On first edit in add-version mode, copy prefill into userEdits so we don't
// lose the other prefilled values when the user changes one field.
setUserEdits((prev) => {
const base = isAddVersion && prefillValues !== null && Object.keys(prev).length === 0
? { ...prefillValues }
: prev
return { ...base, [key]: val }
})
}
// Initialize default values when profile changes (create mode only)
function initDefaults(kind: string | null) {
if (!kind) return
const profile = profiles.find((p: Record<string, unknown>) => p.kind === kind)
if (!profile) return
const profileObj = profile as Record<string, unknown>
const defaults: Record<string, number | string> = {}
for (const [sectionKey, sectionVal] of Object.entries(profileObj)) {
if (sectionKey === 'kind' || sectionKey === 'label') continue
if (typeof sectionVal === 'object' && sectionVal !== null && !isLeaf(sectionVal)) {
const leaves = extractLeafFields(sectionVal as Record<string, unknown>)
for (const leaf of leaves) {
defaults[`${sectionKey}.${leaf.fieldPath}`] = leaf.defaultValue ?? 0
}
}
}
setUserEdits(defaults)
}
async function handleSubmit(e: React.FormEvent) {
e.preventDefault()
setError(null)
if (!isAddVersion && !name.trim()) {
setError('Contract name is required.')
return
}
if (!effectiveKind) {
setError('Contract kind is required.')
return
}
const values = buildNestedValues(sectionFields, fieldValues)
try {
// Convert local date string to a naive local-midnight datetime string (no Z).
// The backend interprets naive datetimes as server local wall-clock time,
// so "2026-06-25T00:00:00" → CEST local midnight → stored as UTC 22:00Z.
// When effectiveFromStr is empty, fall back to the current instant (aware,
// sent as ISO with Z) so the backend stores it unchanged.
const effectiveFromISO = effectiveFromStr
? `${effectiveFromStr}T00:00:00`
: undefined
if (isAddVersion) {
const body = {
effective_from: effectiveFromISO ?? new Date().toISOString(),
values,
}
await addVersionMutation.mutateAsync({ id: contractId, body })
} else {
const body = {
name: name.trim(),
kind: effectiveKind,
currency,
values,
...(effectiveFromISO ? { effective_from: effectiveFromISO } : {}),
}
await createMutation.mutateAsync(body)
}
onSaved()
onClose()
} catch {
setError('Failed to save. Please try again.')
}
}
// ---------------------------------------------------------------------------
// Render
// ---------------------------------------------------------------------------
const title = isAddVersion ? 'Add Contract Version' : 'New Energy Contract'
return (
<Modal
opened
onClose={onClose}
title={title}
size="md"
data-testid="contract-form-modal"
>
{(profilesQuery.isLoading || (isAddVersion && contractDetailQuery.isLoading)) && (
<Center py="md">
<Loader size="sm" data-testid="contract-form-loading" />
</Center>
)}
{profilesQuery.isError && (
<Alert color="red" mb="sm" data-testid="contract-form-profiles-error">
Failed to load pricing profiles. Cannot continue.
</Alert>
)}
{isAddVersion && contractDetailQuery.isError && !contractDetailQuery.isLoading && (
<Alert color="yellow" mb="sm" data-testid="contract-form-detail-error">
Could not load existing contract values for prefill fields start at 0.
</Alert>
)}
{!profilesQuery.isLoading && !(isAddVersion && contractDetailQuery.isLoading) && (
<form onSubmit={handleSubmit} data-testid="contract-form">
<Stack gap="sm">
{/* Create mode fields */}
{!isAddVersion && (
<>
<TextInput
label="Contract name"
required
value={name}
onChange={(e) => setName(e.currentTarget.value)}
data-testid="contract-name"
/>
<Select
label="Pricing kind"
required
data={profileOptions}
value={selectedKind}
onChange={(val) => {
setSelectedKind(val)
initDefaults(val)
}}
data-testid="contract-kind"
/>
<TextInput
label="Currency"
value={currency}
onChange={(e) => setCurrency(e.currentTarget.value)}
data-testid="contract-currency"
/>
</>
)}
{/* Effective from — required for add-version, optional for create */}
<TextInput
label={isAddVersion ? 'Effective from (required)' : 'Effective from (optional, defaults to now)'}
description="Date in YYYY-MM-DD format"
type="date"
required={isAddVersion}
value={effectiveFromStr}
onChange={(e) => setEffectiveFromStr(e.currentTarget.value)}
data-testid="contract-effective-from"
/>
{/* Dynamic profile fields */}
{effectiveKind && Object.keys(sectionFields).length > 0 && (
<>
<Divider mt="xs" />
{Object.entries(sectionFields).map(([section, fields]) => (
<Stack key={section} gap="xs" data-testid={`contract-section-${section}`}>
<Title order={6} tt="capitalize" c="dimmed">
{section.replace(/_/g, ' ')}
</Title>
{fields.map((field) => {
const key = `${section}.${field.fieldPath}`
const raw = fieldValues[key]
const numVal = typeof raw === 'number' ? raw : parseFloat(String(raw))
return (
<NumberInput
key={key}
label={formatLabel(field.fieldPath)}
description={field.unit}
value={isNaN(numVal) ? 0 : numVal}
onChange={(val) => handleFieldChange(key, val)}
decimalScale={6}
step={0.001}
data-testid={`contract-field-${key}`}
/>
)
})}
</Stack>
))}
</>
)}
{!effectiveKind && !isAddVersion && (
<Text size="sm" c="dimmed" data-testid="contract-form-kind-hint">
Select a pricing kind to see the rate fields.
</Text>
)}
{error && (
<Alert color="red" data-testid="contract-form-error">
{error}
</Alert>
)}
<Group justify="flex-end" gap="sm">
<Button
type="button"
variant="default"
onClick={onClose}
data-testid="contract-form-cancel"
>
Cancel
</Button>
<Button
type="submit"
loading={isBusy}
data-testid="contract-form-submit"
>
{isAddVersion ? 'Add Version' : 'Create Contract'}
</Button>
</Group>
</Stack>
</form>
)}
</Modal>
)
}
@@ -0,0 +1,205 @@
/**
* Tests for ContractManager component.
*
* Coverage:
* 1. Loading state rendering.
* 2. Contract list rendering.
* 3. Empty state rendering.
* 4. Activate button calls PATCH with {active: true}.
* 5. "New Contract" button opens ContractForm.
*/
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { screen, waitFor } from '@testing-library/react'
import userEvent from '@testing-library/user-event'
import { renderWithProviders } from '../test-utils'
// ---------------------------------------------------------------------------
// Mock apiClient
// ---------------------------------------------------------------------------
const mockGet = vi.fn()
const mockPost = vi.fn()
const mockPatch = vi.fn()
vi.mock('../api/client', () => ({
default: {
GET: (...args: unknown[]) => mockGet(...args),
POST: (...args: unknown[]) => mockPost(...args),
PATCH: (...args: unknown[]) => mockPatch(...args),
DELETE: vi.fn(),
},
ApiError: class ApiError extends Error {
status: number
body: unknown
constructor(status: number, body: unknown) {
super(`API error ${status}`)
this.name = 'ApiError'
this.status = status
this.body = body
}
},
registerLoginRedirect: vi.fn(),
}))
// ---------------------------------------------------------------------------
// Fixtures
// ---------------------------------------------------------------------------
const ACTIVE_CONTRACT = {
id: 1,
name: 'My Active Contract',
kind: 'manual',
active: true,
currency: 'EUR',
created_at: '2026-06-01T00:00:00Z',
updated_at: '2026-06-01T00:00:00Z',
}
const INACTIVE_CONTRACT = {
id: 2,
name: 'Inactive Contract',
kind: 'tibber',
active: false,
currency: 'EUR',
created_at: '2026-06-02T00:00:00Z',
updated_at: '2026-06-02T00:00:00Z',
}
const PROFILES_RESPONSE = {
profiles: [
{
kind: 'manual',
label: 'Fixed / Variable Rate (NL, dual-tariff)',
energy: {
dual_tariff: true,
buy: { normal: { unit: 'EUR/kWh' }, dal: { unit: 'EUR/kWh' } },
sell: { normal: { unit: 'EUR/kWh' }, dal: { unit: 'EUR/kWh' } },
energy_tax: { unit: 'EUR/kWh' },
ode: { unit: 'EUR/kWh', default: 0 },
},
standing: {
network_fee: { unit: 'EUR/month' },
management_fee: { unit: 'EUR/month' },
},
credits: {
heffingskorting: { unit: 'EUR/year' },
},
},
],
}
// ---------------------------------------------------------------------------
// Import component
// ---------------------------------------------------------------------------
import { ContractManager } from './ContractManager'
// ---------------------------------------------------------------------------
// Tests
// ---------------------------------------------------------------------------
describe('ContractManager', () => {
beforeEach(() => vi.clearAllMocks())
it('renders loading state initially', () => {
// Never resolve so we stay in loading state
mockGet.mockImplementation(() => new Promise(() => {}))
renderWithProviders(<ContractManager />)
expect(screen.getByTestId('contracts-loading')).toBeInTheDocument()
})
it('renders contract list when data loads', async () => {
mockGet.mockImplementation((path: string) => {
if (path === '/api/energy/contracts') {
return Promise.resolve({
data: { items: [ACTIVE_CONTRACT, INACTIVE_CONTRACT], total: 2 },
})
}
return Promise.resolve({ data: null })
})
renderWithProviders(<ContractManager />)
await waitFor(() => {
expect(screen.getByTestId('contracts-table')).toBeInTheDocument()
})
expect(screen.getByText('My Active Contract')).toBeInTheDocument()
expect(screen.getByText('Inactive Contract')).toBeInTheDocument()
expect(screen.getByTestId('contract-active-badge-1')).toHaveTextContent('active')
expect(screen.getByTestId('contract-active-badge-2')).toHaveTextContent('inactive')
})
it('renders empty state when no contracts exist', async () => {
mockGet.mockResolvedValue({ data: { items: [], total: 0 } })
renderWithProviders(<ContractManager />)
await waitFor(() => {
expect(screen.getByTestId('contracts-empty')).toBeInTheDocument()
})
})
it('calls PATCH with active=true when Activate button clicked', async () => {
const user = userEvent.setup()
mockGet.mockImplementation((path: string) => {
if (path === '/api/energy/contracts') {
return Promise.resolve({
data: { items: [INACTIVE_CONTRACT], total: 1 },
})
}
return Promise.resolve({ data: null })
})
mockPatch.mockResolvedValue({
data: { ...INACTIVE_CONTRACT, active: true, versions: [] },
})
renderWithProviders(<ContractManager />)
await waitFor(() => {
expect(screen.getByTestId(`contract-activate-${INACTIVE_CONTRACT.id}`)).toBeInTheDocument()
})
await user.click(screen.getByTestId(`contract-activate-${INACTIVE_CONTRACT.id}`))
await waitFor(() => {
expect(mockPatch).toHaveBeenCalledWith(
'/api/energy/contracts/{contract_id}',
expect.objectContaining({
params: { path: { contract_id: INACTIVE_CONTRACT.id } },
body: { active: true },
}),
)
})
})
it('opens ContractForm when "New Contract" button is clicked', async () => {
const user = userEvent.setup()
mockGet.mockImplementation((path: string) => {
if (path === '/api/energy/contracts') {
return Promise.resolve({ data: { items: [], total: 0 } })
}
if (path === '/api/energy/profiles') {
return Promise.resolve({ data: PROFILES_RESPONSE })
}
return Promise.resolve({ data: null })
})
renderWithProviders(<ContractManager />)
await waitFor(() => {
expect(screen.getByTestId('contract-new-button')).toBeInTheDocument()
})
await user.click(screen.getByTestId('contract-new-button'))
await waitFor(() => {
expect(screen.getByTestId('contract-form-modal')).toBeInTheDocument()
})
})
})
+331
View File
@@ -0,0 +1,331 @@
/**
* ContractManager energy contract list UI.
*
* Features:
* - Table of contracts: name, kind, active badge, currency, created date, actions.
* - Actions per row: Activate (PATCH active=true), Add Version, View History.
* - "New Contract" button at top.
* - Version history modal showing all versions with effective dates and values.
* - Loading / error / empty states.
*/
import { useState } from 'react'
import {
Table,
Button,
Group,
Text,
Loader,
Center,
Alert,
Stack,
Badge,
ScrollArea,
Modal,
Accordion,
Code,
} from '@mantine/core'
import {
useContracts,
useUpdateContract,
type ContractResponse,
type ContractDetailResponse,
} from './hooks'
import { ContractForm } from './ContractForm'
import apiClient from '../api/client'
import { formatLocalDate } from '../utils/datetime'
// ---------------------------------------------------------------------------
// Version history modal
// ---------------------------------------------------------------------------
interface VersionHistoryModalProps {
contractId: number
contractName: string
onClose: () => void
}
function VersionHistoryModal({ contractId, contractName, onClose }: VersionHistoryModalProps) {
const [loading, setLoading] = useState(true)
const [detail, setDetail] = useState<ContractDetailResponse | null>(null)
const [fetchError, setFetchError] = useState<string | null>(null)
// Fetch detail on first render
useState(() => {
apiClient
.GET('/api/energy/contracts/{contract_id}', {
params: { path: { contract_id: contractId } },
})
.then((res) => {
setDetail(res.data ?? null)
setLoading(false)
})
.catch(() => {
setFetchError('Failed to load contract details.')
setLoading(false)
})
})
return (
<Modal
opened
onClose={onClose}
title={`Version history — ${contractName}`}
size="lg"
data-testid="version-history-modal"
>
{loading && (
<Center py="md">
<Loader size="sm" />
</Center>
)}
{fetchError && (
<Alert color="red" data-testid="version-history-error">
{fetchError}
</Alert>
)}
{!loading && !fetchError && detail && (
<>
{detail.versions.length === 0 ? (
<Text c="dimmed" ta="center" size="sm" data-testid="version-history-empty">
No versions found.
</Text>
) : (
<Accordion variant="separated" data-testid="version-history-accordion">
{detail.versions.map((v) => (
<Accordion.Item key={v.id} value={String(v.id)} data-testid={`version-item-${v.id}`}>
<Accordion.Control>
<Group gap="sm">
<Text size="sm" fw={500}>
From: {formatLocalDate(v.effective_from)}
</Text>
{v.effective_to ? (
<Text size="xs" c="dimmed">
To: {formatLocalDate(v.effective_to)}
</Text>
) : (
<Badge color="green" size="xs">
current
</Badge>
)}
</Group>
</Accordion.Control>
<Accordion.Panel>
<Code block style={{ fontSize: 11 }}>
{JSON.stringify(v.values, null, 2)}
</Code>
</Accordion.Panel>
</Accordion.Item>
))}
</Accordion>
)}
</>
)}
<Group justify="flex-end" mt="md">
<Button variant="default" onClick={onClose}>
Close
</Button>
</Group>
</Modal>
)
}
// ---------------------------------------------------------------------------
// Contract table
// ---------------------------------------------------------------------------
interface ContractTableProps {
contracts: ContractResponse[]
onActivate: (id: number) => void
onAddVersion: (contract: ContractResponse) => void
onViewHistory: (contract: ContractResponse) => void
activatingId: number | null
}
function ContractTable({
contracts,
onActivate,
onAddVersion,
onViewHistory,
activatingId,
}: ContractTableProps) {
if (contracts.length === 0) {
return (
<Text c="dimmed" ta="center" size="sm" data-testid="contracts-empty">
No contracts configured yet. Click "New Contract" to add one.
</Text>
)
}
return (
<ScrollArea>
<Table striped highlightOnHover withTableBorder data-testid="contracts-table">
<Table.Thead>
<Table.Tr>
<Table.Th>Name</Table.Th>
<Table.Th>Kind</Table.Th>
<Table.Th>Active</Table.Th>
<Table.Th>Currency</Table.Th>
<Table.Th>Created</Table.Th>
<Table.Th style={{ textAlign: 'right' }}>Actions</Table.Th>
</Table.Tr>
</Table.Thead>
<Table.Tbody>
{contracts.map((contract) => (
<Table.Tr key={contract.id} data-testid={`contract-row-${contract.id}`}>
<Table.Td>
<Text fw={500} size="sm">
{contract.name}
</Text>
</Table.Td>
<Table.Td>
<Badge variant="outline" size="sm">
{contract.kind}
</Badge>
</Table.Td>
<Table.Td>
<Badge
color={contract.active ? 'green' : 'gray'}
variant="light"
size="sm"
data-testid={`contract-active-badge-${contract.id}`}
>
{contract.active ? 'active' : 'inactive'}
</Badge>
</Table.Td>
<Table.Td>{contract.currency}</Table.Td>
<Table.Td>
<Text size="xs" c="dimmed">
{formatLocalDate(contract.created_at)}
</Text>
</Table.Td>
<Table.Td>
<Group justify="flex-end" gap="xs">
{!contract.active && (
<Button
size="xs"
variant="outline"
color="green"
loading={activatingId === contract.id}
onClick={() => onActivate(contract.id)}
data-testid={`contract-activate-${contract.id}`}
>
Activate
</Button>
)}
<Button
size="xs"
variant="outline"
onClick={() => onAddVersion(contract)}
data-testid={`contract-add-version-${contract.id}`}
>
Add Version
</Button>
<Button
size="xs"
variant="subtle"
onClick={() => onViewHistory(contract)}
data-testid={`contract-history-${contract.id}`}
>
History
</Button>
</Group>
</Table.Td>
</Table.Tr>
))}
</Table.Tbody>
</Table>
</ScrollArea>
)
}
// ---------------------------------------------------------------------------
// ContractManager — top-level
// ---------------------------------------------------------------------------
export function ContractManager() {
const contractsQuery = useContracts()
const updateMutation = useUpdateContract()
const [showCreateForm, setShowCreateForm] = useState(false)
const [addVersionContract, setAddVersionContract] = useState<ContractResponse | null>(null)
const [historyContract, setHistoryContract] = useState<ContractResponse | null>(null)
const [activatingId, setActivatingId] = useState<number | null>(null)
async function handleActivate(id: number) {
setActivatingId(id)
try {
await updateMutation.mutateAsync({ id, body: { active: true } })
} finally {
setActivatingId(null)
}
}
// ---------------------------------------------------------------------------
// Render states
// ---------------------------------------------------------------------------
if (contractsQuery.isLoading) {
return (
<Center py="xl" data-testid="contracts-loading">
<Loader />
</Center>
)
}
if (contractsQuery.isError || !contractsQuery.data) {
return (
<Alert color="red" data-testid="contracts-load-error">
Failed to load contracts. Please refresh.
</Alert>
)
}
const contracts = contractsQuery.data.items
return (
<Stack gap="lg" data-testid="contract-manager">
<Group justify="space-between" align="center">
<Text fw={500}>Energy Contracts</Text>
<Button onClick={() => setShowCreateForm(true)} data-testid="contract-new-button">
New Contract
</Button>
</Group>
<ContractTable
contracts={contracts}
onActivate={handleActivate}
onAddVersion={(c) => setAddVersionContract(c)}
onViewHistory={(c) => setHistoryContract(c)}
activatingId={activatingId}
/>
{/* Create new contract */}
{showCreateForm && (
<ContractForm
onClose={() => setShowCreateForm(false)}
onSaved={() => setShowCreateForm(false)}
/>
)}
{/* Add version to existing contract */}
{addVersionContract && (
<ContractForm
contractId={addVersionContract.id}
defaultKind={addVersionContract.kind}
onClose={() => setAddVersionContract(null)}
onSaved={() => setAddVersionContract(null)}
/>
)}
{/* Version history modal */}
{historyContract && (
<VersionHistoryModal
contractId={historyContract.id}
contractName={historyContract.name}
onClose={() => setHistoryContract(null)}
/>
)}
</Stack>
)
}
+214
View File
@@ -0,0 +1,214 @@
/**
* Tests for CostView component.
*
* Coverage:
* 1. Loading state.
* 2. Empty state.
* 3. Renders costs table with data.
* 4. Shows summary values.
* 5. Recompute button triggers confirmation modal and then recompute call.
*/
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { screen, waitFor } from '@testing-library/react'
import userEvent from '@testing-library/user-event'
import { renderWithProviders } from '../test-utils'
// ---------------------------------------------------------------------------
// Mock apiClient
// ---------------------------------------------------------------------------
const mockGet = vi.fn()
const mockPost = vi.fn()
vi.mock('../api/client', () => ({
default: {
GET: (...args: unknown[]) => mockGet(...args),
POST: (...args: unknown[]) => mockPost(...args),
PATCH: vi.fn(),
DELETE: vi.fn(),
},
ApiError: class ApiError extends Error {
status: number
body: unknown
constructor(status: number, body: unknown) {
super(`API error ${status}`)
this.name = 'ApiError'
this.status = status
this.body = body
}
},
registerLoginRedirect: vi.fn(),
}))
// ---------------------------------------------------------------------------
// Fixtures
// ---------------------------------------------------------------------------
const COST_PERIOD = {
period_start: '2026-06-22T10:00:00Z',
d1_kwh: 0.1,
d2_kwh: 0.2,
r1_kwh: 0.0,
r2_kwh: 0.05,
import_cost: 0.044,
export_revenue: 0.005,
net_cost: 0.039,
currency: 'EUR',
degraded: false,
contract_version_id: 1,
}
const SUMMARY = {
currency: 'EUR',
metered_import: 10.5,
metered_export: 2.3,
metered_net: 8.2,
fixed_costs: 5.0,
credits: 50.0,
total_payable: 12.5,
}
// ---------------------------------------------------------------------------
// Import component
// ---------------------------------------------------------------------------
import { CostView } from './CostView'
// ---------------------------------------------------------------------------
// Tests
// ---------------------------------------------------------------------------
describe('CostView', () => {
beforeEach(() => vi.clearAllMocks())
it('renders loading state initially', () => {
mockGet.mockImplementation(() => new Promise(() => {}))
renderWithProviders(<CostView />)
expect(screen.getByTestId('costs-loading')).toBeInTheDocument()
})
it('renders empty state when no cost periods available', async () => {
mockGet.mockImplementation((path: string) => {
if (path === '/api/energy/costs') {
return Promise.resolve({ data: { items: [], total: 0 } })
}
if (path === '/api/energy/costs/summary') {
return Promise.resolve({ data: SUMMARY })
}
return Promise.resolve({ data: null })
})
renderWithProviders(<CostView />)
await waitFor(() => {
expect(screen.getByTestId('costs-empty')).toBeInTheDocument()
})
})
it('renders costs table when data is available', async () => {
mockGet.mockImplementation((path: string) => {
if (path === '/api/energy/costs') {
return Promise.resolve({ data: { items: [COST_PERIOD], total: 1 } })
}
if (path === '/api/energy/costs/summary') {
return Promise.resolve({ data: SUMMARY })
}
return Promise.resolve({ data: null })
})
renderWithProviders(<CostView />)
await waitFor(() => {
expect(screen.getByTestId('costs-table')).toBeInTheDocument()
})
expect(screen.getByTestId('cost-row-0')).toBeInTheDocument()
})
it('shows summary values correctly', async () => {
mockGet.mockImplementation((path: string) => {
if (path === '/api/energy/costs') {
return Promise.resolve({ data: { items: [], total: 0 } })
}
if (path === '/api/energy/costs/summary') {
return Promise.resolve({ data: SUMMARY })
}
return Promise.resolve({ data: null })
})
renderWithProviders(<CostView />)
await waitFor(() => {
expect(screen.getByTestId('summary-import')).toBeInTheDocument()
})
expect(screen.getByTestId('summary-import')).toHaveTextContent('10.500')
expect(screen.getByTestId('summary-export')).toHaveTextContent('2.300')
expect(screen.getByTestId('summary-total')).toHaveTextContent('12.50')
})
it('shows recompute confirmation modal on button click', async () => {
const user = userEvent.setup()
mockGet.mockImplementation((path: string) => {
if (path === '/api/energy/costs') {
return Promise.resolve({ data: { items: [], total: 0 } })
}
if (path === '/api/energy/costs/summary') {
return Promise.resolve({ data: SUMMARY })
}
return Promise.resolve({ data: null })
})
renderWithProviders(<CostView />)
await waitFor(() => {
expect(screen.getByTestId('cost-recompute-button')).toBeInTheDocument()
})
await user.click(screen.getByTestId('cost-recompute-button'))
await waitFor(() => {
expect(screen.getByTestId('recompute-confirm-modal')).toBeInTheDocument()
})
})
it('calls recompute mutation when confirmed', async () => {
const user = userEvent.setup()
mockGet.mockImplementation((path: string) => {
if (path === '/api/energy/costs') {
return Promise.resolve({ data: { items: [], total: 0 } })
}
if (path === '/api/energy/costs/summary') {
return Promise.resolve({ data: SUMMARY })
}
return Promise.resolve({ data: null })
})
mockPost.mockResolvedValue({ data: { recomputed: 0 } })
renderWithProviders(<CostView />)
await waitFor(() => {
expect(screen.getByTestId('cost-recompute-button')).toBeInTheDocument()
})
await user.click(screen.getByTestId('cost-recompute-button'))
await waitFor(() => {
expect(screen.getByTestId('recompute-confirm')).toBeInTheDocument()
})
await user.click(screen.getByTestId('recompute-confirm'))
await waitFor(() => {
expect(mockPost).toHaveBeenCalledWith(
'/api/energy/costs/recompute',
expect.any(Object),
)
})
})
})
+403
View File
@@ -0,0 +1,403 @@
/**
* CostView cost trends + detail + summary.
*
* Features:
* - Date range selector: Today / This month / Custom (SegmentedControl + DateInput).
* - Recharts AreaChart showing import_cost, export_revenue, net_cost per period.
* - Table showing per-15min periods with time, kWh values, costs, degraded badge.
* - Summary cards: metered import/export, fixed costs, credits, total payable.
* - "Recompute" button with confirmation.
* - Loading/error/empty states, degraded period highlighting.
*
* Recharts imports are isolated to this file only.
*/
import { useState } from 'react'
import {
Stack,
Text,
TextInput,
Loader,
Center,
Alert,
Table,
Group,
Badge,
ScrollArea,
SegmentedControl,
Button,
Paper,
SimpleGrid,
Modal,
Title,
} from '@mantine/core'
import {
AreaChart,
Area,
XAxis,
YAxis,
CartesianGrid,
Tooltip,
Legend,
ResponsiveContainer,
} from 'recharts'
import { useEnergyCosts, useEnergyCostSummary, useRecomputeCosts } from './hooks'
import { formatLocalTime } from '../utils/datetime'
// ---------------------------------------------------------------------------
// Cost limit — prevent accidental full-table pulls
// ---------------------------------------------------------------------------
const COSTS_MAX_LIMIT = 500
// ---------------------------------------------------------------------------
// Date range helpers
// ---------------------------------------------------------------------------
function getTodayRange(): { start: string; end: string } {
// Use local date boundary so "today" matches what the user sees in the display.
const now = new Date()
const start = new Date(now.getFullYear(), now.getMonth(), now.getDate())
const end = new Date(now.getFullYear(), now.getMonth(), now.getDate() + 1)
return { start: start.toISOString(), end: end.toISOString() }
}
function getThisMonthRange(): { start: string; end: string } {
// Use local date boundary so "this month" matches what the user sees in the display.
const now = new Date()
const start = new Date(now.getFullYear(), now.getMonth(), 1)
const end = new Date(now.getFullYear(), now.getMonth() + 1, 1)
return { start: start.toISOString(), end: end.toISOString() }
}
// ---------------------------------------------------------------------------
// Summary cards
// ---------------------------------------------------------------------------
interface SummaryCardProps {
label: string
value: string
testId?: string
}
function SummaryCard({ label, value, testId }: SummaryCardProps) {
return (
<Paper withBorder p="sm" data-testid={testId}>
<Stack gap={4}>
<Text size="xs" c="dimmed">
{label}
</Text>
<Text fw={600} size="lg">
{value}
</Text>
</Stack>
</Paper>
)
}
// ---------------------------------------------------------------------------
// CostView — main component
// ---------------------------------------------------------------------------
type RangePreset = 'today' | 'month' | 'custom'
export function CostView() {
const [rangePreset, setRangePreset] = useState<RangePreset>('today')
// Date strings in YYYY-MM-DD format for custom range
const [customStartStr, setCustomStartStr] = useState('')
const [customEndStr, setCustomEndStr] = useState('')
const [showRecomputeConfirm, setShowRecomputeConfirm] = useState(false)
// Compute effective date range
const { start, end } = (() => {
if (rangePreset === 'today') return getTodayRange()
if (rangePreset === 'month') return getThisMonthRange()
return {
start: customStartStr ? new Date(customStartStr).toISOString() : undefined,
end: customEndStr ? new Date(customEndStr).toISOString() : undefined,
}
})()
const costsQuery = useEnergyCosts(start, end, COSTS_MAX_LIMIT)
const summaryQuery = useEnergyCostSummary(start, end)
const recomputeMutation = useRecomputeCosts()
async function handleRecompute() {
setShowRecomputeConfirm(false)
await recomputeMutation.mutateAsync({ start, end })
}
// ---------------------------------------------------------------------------
// Render
// ---------------------------------------------------------------------------
const currency = costsQuery.data?.items[0]?.currency ?? summaryQuery.data?.currency ?? 'EUR'
return (
<Stack gap="lg" data-testid="cost-view">
{/* Date range selector */}
<Group align="flex-start" gap="md" wrap="wrap">
<Stack gap="xs">
<Text size="sm" fw={500}>
Date range
</Text>
<SegmentedControl
value={rangePreset}
onChange={(v) => setRangePreset(v as RangePreset)}
data={[
{ label: 'Today', value: 'today' },
{ label: 'This month', value: 'month' },
{ label: 'Custom', value: 'custom' },
]}
data-testid="cost-range-control"
/>
</Stack>
{rangePreset === 'custom' && (
<Group gap="sm" align="flex-end">
<TextInput
label="From"
type="date"
value={customStartStr}
onChange={(e) => setCustomStartStr(e.currentTarget.value)}
data-testid="cost-custom-start"
/>
<TextInput
label="To"
type="date"
value={customEndStr}
onChange={(e) => setCustomEndStr(e.currentTarget.value)}
data-testid="cost-custom-end"
/>
</Group>
)}
<Group gap="sm" style={{ marginLeft: 'auto' }} align="flex-end">
<Button
variant="outline"
color="orange"
size="sm"
onClick={() => setShowRecomputeConfirm(true)}
loading={recomputeMutation.isPending}
data-testid="cost-recompute-button"
>
Recompute
</Button>
</Group>
</Group>
{/* Summary cards */}
{summaryQuery.isLoading && (
<Center>
<Loader size="sm" />
</Center>
)}
{summaryQuery.isError && (
<Alert color="red" data-testid="summary-error">
Failed to load cost summary.
</Alert>
)}
{summaryQuery.data && (
<Stack gap="xs">
<Title order={6} c="dimmed">
Summary
</Title>
<SimpleGrid cols={{ base: 2, sm: 3 }} spacing="sm" data-testid="cost-summary">
<SummaryCard
label="Import (kWh)"
value={summaryQuery.data.metered_import.toFixed(3)}
testId="summary-import"
/>
<SummaryCard
label="Export (kWh)"
value={summaryQuery.data.metered_export.toFixed(3)}
testId="summary-export"
/>
<SummaryCard
label={`Fixed costs (${currency})`}
value={summaryQuery.data.fixed_costs.toFixed(2)}
testId="summary-fixed"
/>
<SummaryCard
label={`Credits (${currency})`}
value={summaryQuery.data.credits.toFixed(2)}
testId="summary-credits"
/>
<SummaryCard
label={`Total payable (${currency})`}
value={summaryQuery.data.total_payable.toFixed(2)}
testId="summary-total"
/>
</SimpleGrid>
</Stack>
)}
{/* Cost chart */}
{costsQuery.isLoading && (
<Center py="xl" data-testid="costs-loading">
<Loader />
</Center>
)}
{costsQuery.isError && (
<Alert color="red" data-testid="costs-error">
Failed to load cost periods. Please refresh.
</Alert>
)}
{costsQuery.data && costsQuery.data.items.length === 0 && (
<Alert color="gray" data-testid="costs-empty">
No cost data available for the selected period.
</Alert>
)}
{costsQuery.data && costsQuery.data.items.length > 0 && (
<>
{/* Area chart */}
<Stack gap="xs" data-testid="cost-chart">
<Title order={6} c="dimmed">
Cost trends ({currency})
</Title>
<ResponsiveContainer width="100%" height={220}>
<AreaChart
data={costsQuery.data.items.map((item) => ({
time: formatLocalTime(item.period_start),
import_cost: item.import_cost,
export_revenue: item.export_revenue,
net_cost: item.net_cost,
}))}
margin={{ top: 4, right: 16, left: 0, bottom: 4 }}
>
<CartesianGrid strokeDasharray="3 3" />
<XAxis dataKey="time" tick={{ fontSize: 10 }} interval="preserveStartEnd" />
<YAxis tick={{ fontSize: 10 }} tickFormatter={(v: number) => v.toFixed(2)} />
<Tooltip
// eslint-disable-next-line @typescript-eslint/no-explicit-any
formatter={(val: any) =>
[`${typeof val === 'number' ? val.toFixed(4) : String(val)} ${currency}`, undefined]
}
/>
<Legend />
<Area
type="monotone"
dataKey="import_cost"
stroke="#2196f3"
fill="#bbdefb"
name="Import cost"
dot={false}
/>
<Area
type="monotone"
dataKey="export_revenue"
stroke="#4caf50"
fill="#c8e6c9"
name="Export revenue"
dot={false}
/>
<Area
type="monotone"
dataKey="net_cost"
stroke="#ff9800"
fill="#ffe0b2"
name="Net cost"
dot={false}
/>
</AreaChart>
</ResponsiveContainer>
</Stack>
{/* Detail table */}
<Stack gap="xs">
<Title order={6} c="dimmed">
Period detail
</Title>
<ScrollArea>
<Table
striped
highlightOnHover
withTableBorder
withColumnBorders
style={{ fontSize: 12 }}
data-testid="costs-table"
>
<Table.Thead>
<Table.Tr>
<Table.Th>Time</Table.Th>
<Table.Th>Import kWh</Table.Th>
<Table.Th>Export kWh</Table.Th>
<Table.Th>Import cost</Table.Th>
<Table.Th>Export rev.</Table.Th>
<Table.Th>Net cost</Table.Th>
<Table.Th></Table.Th>
</Table.Tr>
</Table.Thead>
<Table.Tbody>
{costsQuery.data.items.map((item, idx) => (
<Table.Tr
key={idx}
style={item.degraded ? { opacity: 0.6 } : undefined}
data-testid={`cost-row-${idx}`}
>
<Table.Td>
<Text size="xs">
{formatLocalTime(item.period_start)}
</Text>
</Table.Td>
<Table.Td>{(item.d1_kwh + item.d2_kwh).toFixed(3)}</Table.Td>
<Table.Td>{(item.r1_kwh + item.r2_kwh).toFixed(3)}</Table.Td>
<Table.Td>{item.import_cost.toFixed(4)}</Table.Td>
<Table.Td>{item.export_revenue.toFixed(4)}</Table.Td>
<Table.Td>{item.net_cost.toFixed(4)}</Table.Td>
<Table.Td>
{item.degraded && (
<Badge color="orange" size="xs" data-testid={`cost-degraded-${idx}`}>
degraded
</Badge>
)}
</Table.Td>
</Table.Tr>
))}
</Table.Tbody>
</Table>
</ScrollArea>
</Stack>
</>
)}
{/* Recompute confirmation modal */}
{showRecomputeConfirm && (
<Modal
opened
onClose={() => setShowRecomputeConfirm(false)}
title="Recompute costs?"
size="sm"
data-testid="recompute-confirm-modal"
>
<Stack gap="md">
<Text size="sm">
This will recompute all cost periods for the selected date range. Continue?
</Text>
<Group justify="flex-end" gap="sm">
<Button
variant="default"
onClick={() => setShowRecomputeConfirm(false)}
data-testid="recompute-cancel"
>
Cancel
</Button>
<Button
color="orange"
onClick={handleRecompute}
data-testid="recompute-confirm"
>
Recompute
</Button>
</Group>
</Stack>
</Modal>
)}
</Stack>
)
}
+82
View File
@@ -0,0 +1,82 @@
/**
* Tests for DsmrPanel latest parsed DSMR telegram view.
*
* Coverage:
* 1. Loading state.
* 2. Empty state (found=false) with a helpful hint.
* 3. Renders the payload as a key/value table; null values shown as "—".
* 4. Error state.
*/
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { screen, waitFor } from '@testing-library/react'
import { renderWithProviders } from '../test-utils'
const mockGet = vi.fn()
vi.mock('../api/client', () => ({
default: {
GET: (...args: unknown[]) => mockGet(...args),
POST: vi.fn(),
PATCH: vi.fn(),
DELETE: vi.fn(),
},
ApiError: class ApiError extends Error {
status: number
body: unknown
constructor(status: number, body: unknown) {
super(`API error ${status}`)
this.name = 'ApiError'
this.status = status
this.body = body
}
},
registerLoginRedirect: vi.fn(),
}))
import { DsmrPanel } from './DsmrPanel'
describe('DsmrPanel', () => {
beforeEach(() => vi.clearAllMocks())
it('renders loading state initially', () => {
mockGet.mockImplementation(() => new Promise(() => {}))
renderWithProviders(<DsmrPanel />)
expect(screen.getByTestId('dsmr-loading')).toBeInTheDocument()
})
it('shows an empty hint when no DSMR data has been ingested', async () => {
mockGet.mockResolvedValue({ data: { found: false, recorded_at: null, payload: null } })
renderWithProviders(<DsmrPanel />)
await waitFor(() => expect(screen.getByTestId('dsmr-empty')).toBeInTheDocument())
expect(screen.queryByTestId('dsmr-table')).not.toBeInTheDocument()
})
it('renders the latest telegram as a key/value table; null shown as dash', async () => {
mockGet.mockResolvedValue({
data: {
found: true,
recorded_at: '2026-06-23T12:16:00Z',
payload: {
electricity_delivered_1: '20915.154',
phase_voltage_l2: null,
},
},
})
renderWithProviders(<DsmrPanel />)
await waitFor(() => expect(screen.getByTestId('dsmr-table')).toBeInTheDocument())
// Field rows present (keys are sorted).
expect(screen.getByTestId('dsmr-row-electricity_delivered_1')).toHaveTextContent('20915.154')
// Null phase value rendered as an em dash, not omitted.
expect(screen.getByTestId('dsmr-row-phase_voltage_l2')).toHaveTextContent('—')
expect(screen.getByTestId('dsmr-recorded-at')).toBeInTheDocument()
})
it('renders an error state when the request fails', async () => {
mockGet.mockRejectedValue(new Error('network down'))
renderWithProviders(<DsmrPanel />)
await waitFor(() => expect(screen.getByTestId('dsmr-error')).toBeInTheDocument())
})
})
+154
View File
@@ -0,0 +1,154 @@
/**
* DsmrPanel shows the latest parsed DSMR telegram (the source of our metering data).
*
* This is the "ground truth" view: it fetches GET /api/energy/dsmr/latest and
* renders the full parsed frame as a key/value table, so you can confirm at a
* glance whether DSMR ingest is actually landing data (without reading the DB).
*
* - Auto-refresh toggle (default on) so new telegrams appear without a manual reload.
* - Loading / error / empty states; "empty" explains the likely cause.
* - Null phase values are shown as "—" (kept verbatim in the payload).
*/
import { useState } from 'react'
import {
Stack,
Group,
Text,
Loader,
Center,
Alert,
Paper,
Table,
ScrollArea,
Switch,
Divider,
Badge,
} from '@mantine/core'
import { useDsmrLatest } from './hooks'
import { formatLocalDateTime } from '../utils/datetime'
/** Default auto-refresh interval (ms) — DSMR ingest stores ~one row per 10 s. */
const AUTO_REFRESH_INTERVAL_MS = 10_000
function formatValue(value: unknown): string {
if (value === null || value === undefined) return '—'
if (typeof value === 'object') return JSON.stringify(value)
return String(value)
}
export function DsmrPanel() {
const [autoRefresh, setAutoRefresh] = useState(true)
const latestQuery = useDsmrLatest(
autoRefresh ? { refetchIntervalMs: AUTO_REFRESH_INTERVAL_MS } : undefined,
)
return (
<Stack gap="md" data-testid="dsmr-panel">
<Group justify="space-between" align="center">
<div>
<Text fw={600}>Latest DSMR reading</Text>
<Text size="xs" c="dimmed">
The most recent parsed telegram persisted to <code>dsmr_reading</code>.
</Text>
</div>
<Switch
label="Auto-refresh"
checked={autoRefresh}
onChange={(e) => setAutoRefresh(e.currentTarget.checked)}
size="sm"
data-testid="dsmr-auto-refresh-switch"
/>
</Group>
<Divider />
<DsmrContent
isLoading={latestQuery.isLoading}
isError={latestQuery.isError}
data={latestQuery.data}
/>
</Stack>
)
}
interface DsmrContentProps {
isLoading: boolean
isError: boolean
data:
| { found: boolean; recorded_at?: string | null; payload?: Record<string, unknown> | null }
| undefined
}
function DsmrContent({ isLoading, isError, data }: DsmrContentProps) {
if (isLoading) {
return (
<Center py="xl" data-testid="dsmr-loading">
<Loader />
</Center>
)
}
if (isError || !data) {
return (
<Alert color="red" data-testid="dsmr-error">
Failed to load the latest DSMR reading.
</Alert>
)
}
if (!data.found || !data.payload) {
return (
<Alert color="gray" data-testid="dsmr-empty">
No DSMR data yet. Enable <strong>DSMR ingest</strong> in Config, make sure MQTT
is connected, and confirm the DSMR Reader is publishing to the configured topic
(default <code>dsmr/json</code>). Rows are stored about once every 10 seconds.
</Alert>
)
}
const payload = data.payload
const keys = Object.keys(payload).sort()
return (
<Stack gap="sm" data-testid="dsmr-data">
<Group gap="xs">
<Badge color="teal" variant="light">
{keys.length} fields
</Badge>
{data.recorded_at && (
<Text size="sm" c="dimmed" data-testid="dsmr-recorded-at">
recorded at {formatLocalDateTime(data.recorded_at)}
</Text>
)}
</Group>
<Paper withBorder>
<ScrollArea.Autosize mah={480}>
<Table striped highlightOnHover withColumnBorders data-testid="dsmr-table">
<Table.Thead>
<Table.Tr>
<Table.Th>Field</Table.Th>
<Table.Th>Value</Table.Th>
</Table.Tr>
</Table.Thead>
<Table.Tbody>
{keys.map((key) => (
<Table.Tr key={key} data-testid={`dsmr-row-${key}`}>
<Table.Td>
<Text size="sm" ff="monospace">
{key}
</Text>
</Table.Td>
<Table.Td>
<Text size="sm">{formatValue(payload[key])}</Text>
</Table.Td>
</Table.Tr>
))}
</Table.Tbody>
</Table>
</ScrollArea.Autosize>
</Paper>
</Stack>
)
}
+3 -11
View File
@@ -43,6 +43,7 @@ import {
import { useReadings, useMetrics } from './hooks'
import type { MetricInfo, AutoRefreshOptions } from './hooks'
import { formatMetricValue } from './format'
import { formatLocalTime, formatLocalDateTime } from '../utils/datetime'
// ---------------------------------------------------------------------------
// Constants
@@ -107,12 +108,7 @@ function lineColor(index: number): string {
// ---------------------------------------------------------------------------
function formatTimeTick(isoString: string): string {
try {
const d = new Date(isoString)
return d.toLocaleTimeString([], { hour: '2-digit', minute: '2-digit' })
} catch {
return isoString
}
return formatLocalTime(isoString)
}
// ---------------------------------------------------------------------------
@@ -300,11 +296,7 @@ export function EnergyCharts({ uuid, deviceName, autoRefresh }: EnergyChartsProp
] as [string, string]
}}
labelFormatter={(label) => {
try {
return new Date(String(label)).toLocaleString()
} catch {
return String(label)
}
return formatLocalDateTime(String(label))
}}
/>
<Legend />
+446
View File
@@ -0,0 +1,446 @@
/**
* Tests for MeterManager component.
*
* Coverage:
* 1. Loading state rendering.
* 2. Error state rendering.
* 3. Empty state when no meters exist.
* 4. Meter timeline list rendering (label, dates, active badge, reason).
* 5. "Declare New Meter" button opens form modal.
* 6. Declare meter form submit calls POST /api/energy/meters.
* 7. Declare meter 422 () error is displayed.
* 8. Edit button opens edit form modal.
* 9. Edit meter saves label/note via PATCH /api/energy/meters/{meter_id}.
* 10. Edit meter retroactive started_at triggers recompute notice.
*/
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { screen, waitFor } from '@testing-library/react'
import userEvent from '@testing-library/user-event'
import { renderWithProviders } from '../test-utils'
import { MeterManager } from './MeterManager'
// ---------------------------------------------------------------------------
// Mock apiClient
// ---------------------------------------------------------------------------
const mockGet = vi.fn()
const mockPost = vi.fn()
const mockPatch = vi.fn()
vi.mock('../api/client', () => ({
default: {
GET: (...args: unknown[]) => mockGet(...args),
POST: (...args: unknown[]) => mockPost(...args),
PATCH: (...args: unknown[]) => mockPatch(...args),
DELETE: vi.fn(),
},
ApiError: class ApiError extends Error {
status: number
body: unknown
constructor(status: number, body: unknown) {
super(`API error ${status}`)
this.name = 'ApiError'
this.status = status
this.body = body
}
},
registerLoginRedirect: vi.fn(),
}))
// ---------------------------------------------------------------------------
// Fixtures
// ---------------------------------------------------------------------------
const ACTIVE_METER = {
id: 1,
label: 'Initial 2G meter',
commodity: 'electricity',
started_at: '2024-01-15T00:00:00Z',
ended_at: null,
reason: 'initial',
note: null,
created_at: '2024-01-15T00:00:00Z',
}
const CLOSED_METER = {
id: 2,
label: 'Old 4G meter',
commodity: 'electricity',
started_at: '2023-06-01T00:00:00Z',
ended_at: '2024-01-15T00:00:00Z',
reason: 'meter_swap',
note: 'Replaced by grid company',
created_at: '2023-06-01T00:00:00Z',
}
const METERS_RESPONSE = {
items: [CLOSED_METER, ACTIVE_METER],
total: 2,
}
// ---------------------------------------------------------------------------
// Tests
// ---------------------------------------------------------------------------
describe('MeterManager — loading / error / empty states', () => {
beforeEach(() => vi.clearAllMocks())
it('renders loading state initially', () => {
mockGet.mockImplementation(() => new Promise(() => {}))
renderWithProviders(<MeterManager />)
expect(screen.getByTestId('meters-loading')).toBeInTheDocument()
})
it('renders error state when GET fails', async () => {
mockGet.mockRejectedValue(new Error('Network error'))
renderWithProviders(<MeterManager />)
await waitFor(() => {
expect(screen.getByTestId('meters-load-error')).toBeInTheDocument()
})
})
it('renders empty state when no meters exist', async () => {
mockGet.mockResolvedValue({ data: { items: [], total: 0 } })
renderWithProviders(<MeterManager />)
await waitFor(() => {
expect(screen.getByTestId('meters-empty')).toBeInTheDocument()
})
})
})
describe('MeterManager — meter list', () => {
beforeEach(() => vi.clearAllMocks())
it('renders meter timeline with label, dates, status badge, reason', async () => {
mockGet.mockResolvedValue({ data: METERS_RESPONSE })
renderWithProviders(<MeterManager />)
await waitFor(() => {
expect(screen.getByTestId('meters-table')).toBeInTheDocument()
})
// Labels
expect(screen.getByText('Initial 2G meter')).toBeInTheDocument()
expect(screen.getByText('Old 4G meter')).toBeInTheDocument()
// Status badges
expect(screen.getByTestId(`meter-status-${ACTIVE_METER.id}`)).toHaveTextContent('active')
expect(screen.getByTestId(`meter-status-${CLOSED_METER.id}`)).toHaveTextContent('closed')
// Reason badges
expect(screen.getByText('initial')).toBeInTheDocument()
expect(screen.getByText('meter_swap')).toBeInTheDocument()
})
it('renders "Declare New Meter" button', async () => {
mockGet.mockResolvedValue({ data: METERS_RESPONSE })
renderWithProviders(<MeterManager />)
await waitFor(() => {
expect(screen.getByTestId('meter-declare-button')).toBeInTheDocument()
})
})
})
describe('MeterManager — declare new meter', () => {
beforeEach(() => vi.clearAllMocks())
it('opens declare modal when "Declare New Meter" is clicked', async () => {
const user = userEvent.setup()
mockGet.mockResolvedValue({ data: { items: [], total: 0 } })
renderWithProviders(<MeterManager />)
await waitFor(() => expect(screen.getByTestId('meter-declare-button')).toBeInTheDocument())
await user.click(screen.getByTestId('meter-declare-button'))
await waitFor(() => {
expect(screen.getByTestId('declare-meter-modal')).toBeInTheDocument()
})
expect(screen.getByTestId('declare-meter-form')).toBeInTheDocument()
})
it('calls POST /api/energy/meters with correct payload including local-midnight naive datetime', async () => {
const user = userEvent.setup()
mockGet.mockResolvedValue({ data: { items: [], total: 0 } })
mockPost.mockResolvedValue({ data: ACTIVE_METER })
renderWithProviders(<MeterManager />)
await waitFor(() => expect(screen.getByTestId('meter-declare-button')).toBeInTheDocument())
await user.click(screen.getByTestId('meter-declare-button'))
await waitFor(() => expect(screen.getByTestId('declare-meter-form')).toBeInTheDocument())
// Fill form
await user.type(screen.getByTestId('meter-label'), 'New meter label')
await user.type(screen.getByTestId('meter-started-at'), '2026-01-01')
// Select reason via the combobox (Mantine Select renders a combobox)
await user.click(screen.getByTestId('meter-reason'))
await waitFor(() => screen.getByText('Initial installation'))
await user.click(screen.getByText('Initial installation'))
await user.click(screen.getByTestId('declare-meter-submit'))
await waitFor(() => {
expect(mockPost).toHaveBeenCalledWith(
'/api/energy/meters',
expect.objectContaining({
body: expect.objectContaining({
label: 'New meter label',
// FU10 local-midnight naive convention: no Z suffix
started_at: '2026-01-01T00:00:00',
reason: 'initial',
commodity: 'electricity',
}),
}),
)
})
})
it('displays error when POST fails with 422 (倒挂 / validation error)', async () => {
const user = userEvent.setup()
mockGet.mockResolvedValue({ data: { items: [ACTIVE_METER], total: 1 } })
const { ApiError } = await import('../api/client')
mockPost.mockRejectedValue(
new ApiError(422, { detail: 'started_at must be ≥ current active meter started_at' }),
)
renderWithProviders(<MeterManager />)
await waitFor(() => expect(screen.getByTestId('meter-declare-button')).toBeInTheDocument())
await user.click(screen.getByTestId('meter-declare-button'))
await waitFor(() => expect(screen.getByTestId('declare-meter-form')).toBeInTheDocument())
await user.type(screen.getByTestId('meter-label'), 'Bad meter')
await user.type(screen.getByTestId('meter-started-at'), '2020-01-01')
await user.click(screen.getByTestId('meter-reason'))
await waitFor(() => screen.getByText('Meter swap (same address)'))
await user.click(screen.getByText('Meter swap (same address)'))
await user.click(screen.getByTestId('declare-meter-submit'))
await waitFor(() => {
expect(screen.getByTestId('declare-meter-error')).toBeInTheDocument()
})
expect(screen.getByTestId('declare-meter-error').textContent).toContain(
'started_at must be ≥',
)
})
it('cancel button closes the declare modal', async () => {
const user = userEvent.setup()
mockGet.mockResolvedValue({ data: { items: [], total: 0 } })
renderWithProviders(<MeterManager />)
await waitFor(() => expect(screen.getByTestId('meter-declare-button')).toBeInTheDocument())
await user.click(screen.getByTestId('meter-declare-button'))
await waitFor(() => expect(screen.getByTestId('declare-meter-modal')).toBeInTheDocument())
await user.click(screen.getByTestId('declare-meter-cancel'))
await waitFor(() => {
expect(screen.queryByTestId('declare-meter-modal')).not.toBeInTheDocument()
})
})
})
describe('MeterManager — edit meter date initialisation', () => {
beforeEach(() => vi.clearAllMocks())
it('initialises date input from started_at (Z-suffix, UTC midnight → local date)', async () => {
// ACTIVE_METER.started_at = '2024-01-15T00:00:00Z' (UTC midnight).
// The test suite is pinned to TZ=UTC (via vite.config.ts test.env), so
// the local date is deterministically '2024-01-15' on any CI runner.
const user = userEvent.setup()
mockGet.mockResolvedValue({ data: { items: [ACTIVE_METER], total: 1 } })
renderWithProviders(<MeterManager />)
await waitFor(() => expect(screen.getByTestId(`meter-edit-${ACTIVE_METER.id}`)).toBeInTheDocument())
await user.click(screen.getByTestId(`meter-edit-${ACTIVE_METER.id}`))
await waitFor(() => expect(screen.getByTestId('edit-meter-form')).toBeInTheDocument())
const dateInput = screen.getByTestId('edit-meter-started-at') as HTMLInputElement
expect(dateInput.value).toBe('2024-01-15')
})
it('initialises date input from started_at (naive, no tz marker)', async () => {
// A naive timestamp without timezone marker — parseBackendTimestamp appends 'Z'
// so it is treated as UTC. With TZ=UTC (pinned in vite.config.ts), the local date
// equals the UTC date exactly.
const naiveMeter = { ...ACTIVE_METER, started_at: '2024-03-20T00:00:00' }
const user = userEvent.setup()
mockGet.mockResolvedValue({ data: { items: [naiveMeter], total: 1 } })
renderWithProviders(<MeterManager />)
await waitFor(() => expect(screen.getByTestId(`meter-edit-${naiveMeter.id}`)).toBeInTheDocument())
await user.click(screen.getByTestId(`meter-edit-${naiveMeter.id}`))
await waitFor(() => expect(screen.getByTestId('edit-meter-form')).toBeInTheDocument())
const dateInput = screen.getByTestId('edit-meter-started-at') as HTMLInputElement
expect(dateInput.value).toBe('2024-03-20')
})
it('initialises date input from started_at with explicit UTC offset (+02:00) — regression for old buggy regex', async () => {
// The old hand-written regex /[zZ+-]\d*$/ would fail to match '+02:00' (the ':00'
// suffix broke the pattern) and would incorrectly append 'Z', producing an Invalid Date.
// The new code uses parseBackendTimestamp which uses the correct TZ_MARKER_RE regex
// and handles explicit offsets properly.
const offsetMeter = { ...ACTIVE_METER, started_at: '2024-01-15T02:00:00+02:00' }
// UTC equivalent: 2024-01-15T00:00:00Z → with TZ=UTC (pinned) local date = '2024-01-15'
const user = userEvent.setup()
mockGet.mockResolvedValue({ data: { items: [offsetMeter], total: 1 } })
renderWithProviders(<MeterManager />)
await waitFor(() => expect(screen.getByTestId(`meter-edit-${offsetMeter.id}`)).toBeInTheDocument())
await user.click(screen.getByTestId(`meter-edit-${offsetMeter.id}`))
await waitFor(() => expect(screen.getByTestId('edit-meter-form')).toBeInTheDocument())
const dateInput = screen.getByTestId('edit-meter-started-at') as HTMLInputElement
// Must not be empty (which would indicate Invalid Date from the old buggy path)
expect(dateInput.value).not.toBe('')
expect(dateInput.value).toBe('2024-01-15')
})
it('does not include started_at in PATCH body when date is unchanged (round-trip idempotence)', async () => {
// Open the edit form and immediately submit without changing any fields except label.
// The date should be considered unchanged → no started_at in the PATCH body.
const user = userEvent.setup()
mockGet.mockResolvedValue({ data: { items: [ACTIVE_METER], total: 1 } })
mockPatch.mockResolvedValue({ data: { ...ACTIVE_METER, label: 'New label' } })
renderWithProviders(<MeterManager />)
await waitFor(() => expect(screen.getByTestId(`meter-edit-${ACTIVE_METER.id}`)).toBeInTheDocument())
await user.click(screen.getByTestId(`meter-edit-${ACTIVE_METER.id}`))
await waitFor(() => expect(screen.getByTestId('edit-meter-form')).toBeInTheDocument())
// Change only label; leave date untouched
const labelInput = screen.getByTestId('edit-meter-label')
await user.clear(labelInput)
await user.type(labelInput, 'New label')
await user.click(screen.getByTestId('edit-meter-submit'))
await waitFor(() => {
expect(mockPatch).toHaveBeenCalled()
})
const patchBody = mockPatch.mock.calls[0][1].body
expect(patchBody).not.toHaveProperty('started_at')
})
})
describe('MeterManager — edit meter', () => {
beforeEach(() => vi.clearAllMocks())
it('opens edit modal when Edit button is clicked', async () => {
const user = userEvent.setup()
mockGet.mockResolvedValue({ data: METERS_RESPONSE })
renderWithProviders(<MeterManager />)
await waitFor(() => expect(screen.getByTestId(`meter-edit-${ACTIVE_METER.id}`)).toBeInTheDocument())
await user.click(screen.getByTestId(`meter-edit-${ACTIVE_METER.id}`))
await waitFor(() => {
expect(screen.getByTestId('edit-meter-modal')).toBeInTheDocument()
})
expect(screen.getByTestId('edit-meter-form')).toBeInTheDocument()
})
it('calls PATCH /api/energy/meters/{meter_id} when label is changed', async () => {
const user = userEvent.setup()
mockGet.mockResolvedValue({ data: { items: [ACTIVE_METER], total: 1 } })
mockPatch.mockResolvedValue({ data: { ...ACTIVE_METER, label: 'Renamed meter' } })
renderWithProviders(<MeterManager />)
await waitFor(() => expect(screen.getByTestId(`meter-edit-${ACTIVE_METER.id}`)).toBeInTheDocument())
await user.click(screen.getByTestId(`meter-edit-${ACTIVE_METER.id}`))
await waitFor(() => expect(screen.getByTestId('edit-meter-form')).toBeInTheDocument())
// Clear label and type new one
const labelInput = screen.getByTestId('edit-meter-label')
await user.clear(labelInput)
await user.type(labelInput, 'Renamed meter')
await user.click(screen.getByTestId('edit-meter-submit'))
await waitFor(() => {
expect(mockPatch).toHaveBeenCalledWith(
'/api/energy/meters/{meter_id}',
expect.objectContaining({
params: { path: { meter_id: ACTIVE_METER.id } },
body: expect.objectContaining({ label: 'Renamed meter' }),
}),
)
})
})
it('shows recompute notice when started_at is changed (retroactive correction)', async () => {
const user = userEvent.setup()
mockGet.mockResolvedValue({ data: { items: [ACTIVE_METER], total: 1 } })
mockPatch.mockResolvedValue({ data: { ...ACTIVE_METER, started_at: '2024-02-01T00:00:00' } })
renderWithProviders(<MeterManager />)
await waitFor(() => expect(screen.getByTestId(`meter-edit-${ACTIVE_METER.id}`)).toBeInTheDocument())
await user.click(screen.getByTestId(`meter-edit-${ACTIVE_METER.id}`))
await waitFor(() => expect(screen.getByTestId('edit-meter-form')).toBeInTheDocument())
// Change the date field
const dateInput = screen.getByTestId('edit-meter-started-at')
await user.clear(dateInput)
await user.type(dateInput, '2024-02-01')
await user.click(screen.getByTestId('edit-meter-submit'))
await waitFor(() => {
expect(screen.getByTestId('meter-recompute-notice')).toBeInTheDocument()
})
})
it('displays error when PATCH fails', async () => {
const user = userEvent.setup()
mockGet.mockResolvedValue({ data: { items: [ACTIVE_METER], total: 1 } })
const { ApiError } = await import('../api/client')
mockPatch.mockRejectedValue(new ApiError(422, { detail: 'started_at conflict' }))
renderWithProviders(<MeterManager />)
await waitFor(() => expect(screen.getByTestId(`meter-edit-${ACTIVE_METER.id}`)).toBeInTheDocument())
await user.click(screen.getByTestId(`meter-edit-${ACTIVE_METER.id}`))
await waitFor(() => expect(screen.getByTestId('edit-meter-form')).toBeInTheDocument())
// Change label so there's something to patch
const labelInput = screen.getByTestId('edit-meter-label')
await user.clear(labelInput)
await user.type(labelInput, 'Different label')
await user.click(screen.getByTestId('edit-meter-submit'))
await waitFor(() => {
expect(screen.getByTestId('edit-meter-error')).toBeInTheDocument()
})
expect(screen.getByTestId('edit-meter-error').textContent).toContain('started_at conflict')
})
})
+503
View File
@@ -0,0 +1,503 @@
/**
* MeterManager electricity meter timeline UI.
*
* Features:
* - Table of meter epochs: label / interval (started_at ended_at or "active") /
* active badge / reason.
* - "Declare New Meter" button: form with label + date (started_at) + reason +
* optional note. Sends local-midnight naive datetime per FU10 convention.
* - Edit modal: update label, note, or correct started_at (retroactive).
* - Retroactive feedback: if started_at is changed, a success notice mentions
* that affected billing periods have been recomputed.
* - Loading / error / empty states.
*/
import { useState } from 'react'
import {
Table,
Button,
Group,
Text,
Loader,
Center,
Alert,
Stack,
Badge,
ScrollArea,
Modal,
TextInput,
Textarea,
Select,
Notification,
} from '@mantine/core'
import {
useMeters,
useDeclareMeter,
useUpdateMeter,
type MeterResponse,
type MeterReason,
} from './hooks'
import { ApiError } from '../api/client'
import { formatLocalDate, parseBackendTimestamp } from '../utils/datetime'
// ---------------------------------------------------------------------------
// Helpers
// ---------------------------------------------------------------------------
const REASON_OPTIONS: { value: MeterReason; label: string }[] = [
{ value: 'initial', label: 'Initial installation' },
{ value: 'meter_swap', label: 'Meter swap (same address)' },
{ value: 'home_move', label: 'Home / address move' },
{ value: 'other', label: 'Other' },
]
/**
* Convert a local date string "YYYY-MM-DD" to a naive local-midnight datetime
* string (no Z suffix) following the FU10 / ContractForm convention.
* The backend interprets naive datetimes as server local wall-clock time.
*/
function toLocalMidnightNaive(dateStr: string): string {
return `${dateStr}T00:00:00`
}
/**
* Format a Date object as a "YYYY-MM-DD" string using the browser's local timezone.
* Used to populate <input type="date"> fields.
* Returns '' if the Date is invalid.
*/
function toLocalDateInputString(d: Date): string {
if (isNaN(d.getTime())) return ''
const y = d.getFullYear()
const m = String(d.getMonth() + 1).padStart(2, '0')
const day = String(d.getDate()).padStart(2, '0')
return `${y}-${m}-${day}`
}
// ---------------------------------------------------------------------------
// Declare meter form (modal)
// ---------------------------------------------------------------------------
interface DeclareMeterFormProps {
onClose: () => void
onSaved: () => void
}
function DeclareMeterForm({ onClose, onSaved }: DeclareMeterFormProps) {
const [label, setLabel] = useState('')
const [dateStr, setDateStr] = useState('')
const [reason, setReason] = useState<string | null>(null)
const [note, setNote] = useState('')
const [error, setError] = useState<string | null>(null)
const declareMutation = useDeclareMeter()
async function handleSubmit(e: React.FormEvent) {
e.preventDefault()
setError(null)
if (!label.trim()) {
setError('Label is required.')
return
}
if (!dateStr) {
setError('Start date is required.')
return
}
if (!reason) {
setError('Reason is required.')
return
}
try {
await declareMutation.mutateAsync({
label: label.trim(),
started_at: toLocalMidnightNaive(dateStr),
reason: reason as MeterReason,
note: note.trim() || undefined,
commodity: 'electricity',
})
onSaved()
onClose()
} catch (err) {
if (err instanceof ApiError) {
const detail = (err.body as { detail?: string } | null)?.detail
setError(detail ?? `Error ${err.status}: failed to declare meter.`)
} else {
setError('Failed to declare meter. Please try again.')
}
}
}
return (
<Modal
opened
onClose={onClose}
title="Declare New Meter"
size="md"
data-testid="declare-meter-modal"
>
<form onSubmit={handleSubmit} data-testid="declare-meter-form">
<Stack gap="sm">
<TextInput
label="Label"
description={'Human-readable identifier, e.g. "2G meter @ Dorpsstraat 1"'}
required
value={label}
onChange={(e) => setLabel(e.currentTarget.value)}
data-testid="meter-label"
/>
<TextInput
label="Start date"
description="Date in YYYY-MM-DD format (interpreted as local midnight)"
type="date"
required
value={dateStr}
onChange={(e) => setDateStr(e.currentTarget.value)}
data-testid="meter-started-at"
/>
<Select
label="Reason"
required
data={REASON_OPTIONS}
value={reason}
onChange={setReason}
data-testid="meter-reason"
/>
<Textarea
label="Note (optional)"
value={note}
onChange={(e) => setNote(e.currentTarget.value)}
autosize
minRows={2}
data-testid="meter-note"
/>
{error && (
<Alert color="red" data-testid="declare-meter-error">
{error}
</Alert>
)}
<Group justify="flex-end" gap="sm">
<Button
type="button"
variant="default"
onClick={onClose}
data-testid="declare-meter-cancel"
>
Cancel
</Button>
<Button
type="submit"
loading={declareMutation.isPending}
data-testid="declare-meter-submit"
>
Declare Meter
</Button>
</Group>
</Stack>
</form>
</Modal>
)
}
// ---------------------------------------------------------------------------
// Edit meter form (modal)
// ---------------------------------------------------------------------------
interface EditMeterFormProps {
meter: MeterResponse
onClose: () => void
onSaved: (retroactive: boolean) => void
}
function EditMeterForm({ meter, onClose, onSaved }: EditMeterFormProps) {
const [label, setLabel] = useState(meter.label)
const [note, setNote] = useState(meter.note ?? '')
// Convert started_at to a local date string for the <input type="date">.
// parseBackendTimestamp correctly handles naive strings (no tz marker → treated as UTC,
// matching the backend's storage convention), Z-suffixed strings, and strings with
// explicit offsets like +02:00. We then extract the local-timezone date components
// so the displayed date matches the local wall-clock date of the meter start.
const initialDateStr = toLocalDateInputString(parseBackendTimestamp(meter.started_at))
const [dateStr, setDateStr] = useState(initialDateStr)
const [error, setError] = useState<string | null>(null)
const updateMutation = useUpdateMeter()
// Detect if the user changed started_at (retroactive correction).
const startedAtChanged = dateStr !== initialDateStr
async function handleSubmit(e: React.FormEvent) {
e.preventDefault()
setError(null)
const patchBody: { label?: string | null; note?: string | null; started_at?: string | null } = {}
if (label.trim() !== meter.label) patchBody.label = label.trim()
const noteVal = note.trim() || null
if (noteVal !== meter.note) patchBody.note = noteVal
if (startedAtChanged && dateStr) {
patchBody.started_at = toLocalMidnightNaive(dateStr)
}
if (Object.keys(patchBody).length === 0) {
onClose()
return
}
try {
await updateMutation.mutateAsync({ id: meter.id, body: patchBody })
onSaved(startedAtChanged)
onClose()
} catch (err) {
if (err instanceof ApiError) {
const detail = (err.body as { detail?: string } | null)?.detail
setError(detail ?? `Error ${err.status}: failed to update meter.`)
} else {
setError('Failed to update meter. Please try again.')
}
}
}
return (
<Modal
opened
onClose={onClose}
title={`Edit Meter — ${meter.label}`}
size="md"
data-testid="edit-meter-modal"
>
<form onSubmit={handleSubmit} data-testid="edit-meter-form">
<Stack gap="sm">
<TextInput
label="Label"
required
value={label}
onChange={(e) => setLabel(e.currentTarget.value)}
data-testid="edit-meter-label"
/>
<TextInput
label="Start date"
description="Retroactive correction: shifts the epoch boundary and re-judges billing periods"
type="date"
value={dateStr}
onChange={(e) => setDateStr(e.currentTarget.value)}
data-testid="edit-meter-started-at"
/>
<Textarea
label="Note (optional)"
value={note}
onChange={(e) => setNote(e.currentTarget.value)}
autosize
minRows={2}
data-testid="edit-meter-note"
/>
{error && (
<Alert color="red" data-testid="edit-meter-error">
{error}
</Alert>
)}
<Group justify="flex-end" gap="sm">
<Button
type="button"
variant="default"
onClick={onClose}
data-testid="edit-meter-cancel"
>
Cancel
</Button>
<Button
type="submit"
loading={updateMutation.isPending}
data-testid="edit-meter-submit"
>
Save
</Button>
</Group>
</Stack>
</form>
</Modal>
)
}
// ---------------------------------------------------------------------------
// Meter timeline table
// ---------------------------------------------------------------------------
interface MeterTableProps {
meters: MeterResponse[]
onEdit: (meter: MeterResponse) => void
}
function MeterTable({ meters, onEdit }: MeterTableProps) {
if (meters.length === 0) {
return (
<Text c="dimmed" ta="center" size="sm" data-testid="meters-empty">
No meters declared yet. Click "Declare New Meter" to add one.
</Text>
)
}
return (
<ScrollArea>
<Table striped highlightOnHover withTableBorder data-testid="meters-table">
<Table.Thead>
<Table.Tr>
<Table.Th>Label</Table.Th>
<Table.Th>Commodity</Table.Th>
<Table.Th>From</Table.Th>
<Table.Th>To</Table.Th>
<Table.Th>Status</Table.Th>
<Table.Th>Reason</Table.Th>
<Table.Th style={{ textAlign: 'right' }}>Actions</Table.Th>
</Table.Tr>
</Table.Thead>
<Table.Tbody>
{meters.map((meter) => {
const isActive = meter.ended_at === null
return (
<Table.Tr key={meter.id} data-testid={`meter-row-${meter.id}`}>
<Table.Td>
<Text fw={500} size="sm">
{meter.label}
</Text>
</Table.Td>
<Table.Td>
<Badge variant="outline" size="sm">
{meter.commodity}
</Badge>
</Table.Td>
<Table.Td>
<Text size="xs" c="dimmed">
{formatLocalDate(meter.started_at)}
</Text>
</Table.Td>
<Table.Td>
<Text size="xs" c="dimmed">
{meter.ended_at ? formatLocalDate(meter.ended_at) : '—'}
</Text>
</Table.Td>
<Table.Td>
<Badge
color={isActive ? 'green' : 'gray'}
variant="light"
size="sm"
data-testid={`meter-status-${meter.id}`}
>
{isActive ? 'active' : 'closed'}
</Badge>
</Table.Td>
<Table.Td>
<Badge variant="outline" size="sm" color="blue">
{meter.reason}
</Badge>
</Table.Td>
<Table.Td>
<Group justify="flex-end" gap="xs">
<Button
size="xs"
variant="outline"
onClick={() => onEdit(meter)}
data-testid={`meter-edit-${meter.id}`}
>
Edit
</Button>
</Group>
</Table.Td>
</Table.Tr>
)
})}
</Table.Tbody>
</Table>
</ScrollArea>
)
}
// ---------------------------------------------------------------------------
// MeterManager — top-level
// ---------------------------------------------------------------------------
export function MeterManager() {
const metersQuery = useMeters()
const [showDeclareForm, setShowDeclareForm] = useState(false)
const [editMeter, setEditMeter] = useState<MeterResponse | null>(null)
const [recomputeNotice, setRecomputeNotice] = useState(false)
// ---------------------------------------------------------------------------
// Render states
// ---------------------------------------------------------------------------
if (metersQuery.isLoading) {
return (
<Center py="xl" data-testid="meters-loading">
<Loader />
</Center>
)
}
if (metersQuery.isError || !metersQuery.data) {
return (
<Alert color="red" data-testid="meters-load-error">
Failed to load meters. Please refresh.
</Alert>
)
}
const meters = metersQuery.data.items
return (
<Stack gap="lg" data-testid="meter-manager">
<Group justify="space-between" align="center">
<Text fw={500}>Electricity Meters</Text>
<Button
onClick={() => setShowDeclareForm(true)}
data-testid="meter-declare-button"
>
Declare New Meter
</Button>
</Group>
{recomputeNotice && (
<Notification
color="teal"
title="Billing periods recomputed"
onClose={() => setRecomputeNotice(false)}
data-testid="meter-recompute-notice"
>
The start date was corrected. Affected billing periods have been
re-judged and cost attributions updated.
</Notification>
)}
<MeterTable meters={meters} onEdit={(m) => setEditMeter(m)} />
{/* Declare new meter */}
{showDeclareForm && (
<DeclareMeterForm
onClose={() => setShowDeclareForm(false)}
onSaved={() => setShowDeclareForm(false)}
/>
)}
{/* Edit existing meter */}
{editMeter && (
<EditMeterForm
meter={editMeter}
onClose={() => setEditMeter(null)}
onSaved={(retroactive) => {
setEditMeter(null)
if (retroactive) setRecomputeNotice(true)
}}
/>
)}
</Stack>
)
}
+138
View File
@@ -0,0 +1,138 @@
/**
* Tests for TibberPrices component.
*
* Coverage:
* 1. Loading state.
* 2. Empty state (no active contract / no kind).
* 3. Renders tibber chart when tibber kind data is available.
* 4. Shows tariff table for manual kind.
*/
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { screen, waitFor } from '@testing-library/react'
import { renderWithProviders } from '../test-utils'
// ---------------------------------------------------------------------------
// Mock apiClient
// ---------------------------------------------------------------------------
const mockGet = vi.fn()
vi.mock('../api/client', () => ({
default: {
GET: (...args: unknown[]) => mockGet(...args),
POST: vi.fn(),
PATCH: vi.fn(),
DELETE: vi.fn(),
},
ApiError: class ApiError extends Error {
status: number
body: unknown
constructor(status: number, body: unknown) {
super(`API error ${status}`)
this.name = 'ApiError'
this.status = status
this.body = body
}
},
registerLoginRedirect: vi.fn(),
}))
// ---------------------------------------------------------------------------
// Import component
// ---------------------------------------------------------------------------
import { TibberPrices } from './TibberPrices'
// ---------------------------------------------------------------------------
// Tests
// ---------------------------------------------------------------------------
describe('TibberPrices', () => {
beforeEach(() => vi.clearAllMocks())
it('renders loading state initially', () => {
mockGet.mockImplementation(() => new Promise(() => {}))
renderWithProviders(<TibberPrices />)
expect(screen.getByTestId('prices-loading')).toBeInTheDocument()
})
it('renders "no active contract" when kind is missing/null', async () => {
mockGet.mockResolvedValue({
data: {
kind: null,
currency: 'EUR',
points: [],
tariff: null,
},
})
renderWithProviders(<TibberPrices />)
await waitFor(() => {
expect(screen.getByTestId('prices-no-contract')).toBeInTheDocument()
})
})
it('renders error state when fetch fails', async () => {
mockGet.mockRejectedValue(new Error('Network error'))
renderWithProviders(<TibberPrices />)
await waitFor(() => {
expect(screen.getByTestId('prices-error')).toBeInTheDocument()
})
})
it('renders tibber chart when tibber data is available', async () => {
mockGet.mockResolvedValue({
data: {
kind: 'tibber',
currency: 'EUR',
points: [
{ starts_at: '2026-06-22T10:00:00Z', buy: 0.133, sell: 0.09, level: 'NORMAL' },
{ starts_at: '2026-06-22T10:15:00Z', buy: 0.140, sell: 0.092, level: 'NORMAL' },
],
tariff: null,
},
})
renderWithProviders(<TibberPrices />)
await waitFor(() => {
expect(screen.getByTestId('tibber-chart')).toBeInTheDocument()
})
// Check badge shows kind
expect(screen.getByText('tibber')).toBeInTheDocument()
})
it('shows tariff table for manual kind', async () => {
mockGet.mockResolvedValue({
data: {
kind: 'manual',
currency: 'EUR',
points: [],
tariff: {
buy_dal: 0.127,
buy_normal: 0.133,
sell_dal: 0.09,
sell_normal: 0.09,
},
},
})
renderWithProviders(<TibberPrices />)
await waitFor(() => {
expect(screen.getByTestId('manual-tariff-table')).toBeInTheDocument()
})
expect(screen.getByTestId('tariff-buy-normal')).toHaveTextContent('0.1330')
expect(screen.getByTestId('tariff-buy-dal')).toHaveTextContent('0.1270')
expect(screen.getByTestId('tariff-sell-normal')).toHaveTextContent('0.0900')
expect(screen.getByTestId('tariff-sell-dal')).toHaveTextContent('0.0900')
})
})
+244
View File
@@ -0,0 +1,244 @@
/**
* TibberPrices price curve visualization.
*
* - Fetches today + tomorrow price range using useEnergyPrices.
* - For tibber kind: Recharts LineChart showing buy/sell prices over time.
* - For manual kind: shows tariff table (buy_dal, buy_normal, sell_dal, sell_normal).
* - Handles: no active contract, empty data, loading, error.
*
* Recharts imports are isolated to this file only.
*/
import {
Stack,
Text,
Loader,
Center,
Alert,
Table,
Title,
Badge,
Group,
Paper,
} from '@mantine/core'
import {
LineChart,
Line,
XAxis,
YAxis,
CartesianGrid,
Tooltip,
Legend,
ResponsiveContainer,
} from 'recharts'
import { useEnergyPrices } from './hooks'
import { formatLocalTime } from '../utils/datetime'
// ---------------------------------------------------------------------------
// Time range helpers
// ---------------------------------------------------------------------------
function getTodayStart(): string {
const d = new Date()
d.setUTCHours(0, 0, 0, 0)
return d.toISOString()
}
function getTomorrowEnd(): string {
const d = new Date()
d.setUTCHours(0, 0, 0, 0)
d.setUTCDate(d.getUTCDate() + 2)
return d.toISOString()
}
// ---------------------------------------------------------------------------
// Tibber chart
// ---------------------------------------------------------------------------
interface TibberChartProps {
points: Array<{ starts_at: string; buy: number; sell: number; level?: string | null }>
currency: string
}
function TibberChart({ points, currency }: TibberChartProps) {
const data = points.map((p) => ({
time: formatLocalTime(p.starts_at),
buy: p.buy,
sell: p.sell,
}))
return (
<Stack gap="xs" data-testid="tibber-chart">
<Title order={6} c="dimmed">
Price curve ({currency})
</Title>
<ResponsiveContainer width="100%" height={260}>
<LineChart data={data} margin={{ top: 4, right: 16, left: 0, bottom: 4 }}>
<CartesianGrid strokeDasharray="3 3" />
<XAxis
dataKey="time"
tick={{ fontSize: 10 }}
interval="preserveStartEnd"
/>
<YAxis
tick={{ fontSize: 10 }}
tickFormatter={(v: number) => v.toFixed(3)}
/>
<Tooltip
// eslint-disable-next-line @typescript-eslint/no-explicit-any
formatter={(val: any) =>
[`${typeof val === 'number' ? val.toFixed(4) : String(val)} ${currency}`, undefined]
}
/>
<Legend />
<Line
type="monotone"
dataKey="buy"
stroke="#2196f3"
dot={false}
strokeWidth={2}
name="Buy"
/>
<Line
type="monotone"
dataKey="sell"
stroke="#4caf50"
dot={false}
strokeWidth={2}
name="Sell"
/>
</LineChart>
</ResponsiveContainer>
</Stack>
)
}
// ---------------------------------------------------------------------------
// Manual tariff table
// ---------------------------------------------------------------------------
interface ManualTariffTableProps {
tariff: {
buy_dal: number
buy_normal: number
sell_dal: number
sell_normal: number
}
currency: string
}
function ManualTariffTable({ tariff, currency }: ManualTariffTableProps) {
return (
<Stack gap="xs" data-testid="manual-tariff-table">
<Title order={6} c="dimmed">
Fixed tariff ({currency}/kWh)
</Title>
<Table withTableBorder withColumnBorders>
<Table.Thead>
<Table.Tr>
<Table.Th>Tariff</Table.Th>
<Table.Th>Buy</Table.Th>
<Table.Th>Sell</Table.Th>
</Table.Tr>
</Table.Thead>
<Table.Tbody>
<Table.Tr>
<Table.Td>Normal (peak)</Table.Td>
<Table.Td data-testid="tariff-buy-normal">{tariff.buy_normal.toFixed(4)}</Table.Td>
<Table.Td data-testid="tariff-sell-normal">{tariff.sell_normal.toFixed(4)}</Table.Td>
</Table.Tr>
<Table.Tr>
<Table.Td>Dal (off-peak)</Table.Td>
<Table.Td data-testid="tariff-buy-dal">{tariff.buy_dal.toFixed(4)}</Table.Td>
<Table.Td data-testid="tariff-sell-dal">{tariff.sell_dal.toFixed(4)}</Table.Td>
</Table.Tr>
</Table.Tbody>
</Table>
</Stack>
)
}
// ---------------------------------------------------------------------------
// TibberPrices — main component
// ---------------------------------------------------------------------------
export function TibberPrices() {
const start = getTodayStart()
const end = getTomorrowEnd()
const { data, isLoading, isError } = useEnergyPrices(start, end)
if (isLoading) {
return (
<Center py="xl" data-testid="prices-loading">
<Loader />
</Center>
)
}
if (isError) {
return (
<Alert color="red" data-testid="prices-error">
Failed to load energy prices. Please refresh.
</Alert>
)
}
if (!data) {
return (
<Alert color="gray" data-testid="prices-no-data">
No pricing data available.
</Alert>
)
}
// No active contract
if (!data.kind) {
return (
<Paper withBorder p="md" data-testid="prices-no-contract">
<Stack gap="xs">
<Text fw={500}>No active contract</Text>
<Text size="sm" c="dimmed">
Activate an energy contract on the Contracts tab to see pricing data.
</Text>
</Stack>
</Paper>
)
}
const currency = data.currency
return (
<Stack gap="lg" data-testid="tibber-prices">
<Group gap="sm" align="center">
<Text fw={500}>Energy Prices</Text>
<Badge variant="outline" size="sm">
{data.kind}
</Badge>
<Text size="xs" c="dimmed">
{currency}
</Text>
</Group>
{data.kind === 'tibber' && data.points && data.points.length > 0 && (
<TibberChart points={data.points} currency={currency} />
)}
{data.kind === 'tibber' && (!data.points || data.points.length === 0) && (
<Alert color="yellow" data-testid="tibber-no-prices">
No Tibber price points available for the selected time range. Check Tibber configuration.
</Alert>
)}
{data.kind === 'manual' && data.tariff && (
<ManualTariffTable tariff={data.tariff} currency={currency} />
)}
{data.kind === 'manual' && !data.tariff && (
<Alert color="yellow" data-testid="manual-no-tariff">
Manual tariff data not available.
</Alert>
)}
</Stack>
)
}
+293
View File
@@ -0,0 +1,293 @@
/**
* Tests for energy hooks in hooks.ts energy pricing API wrappers.
*
* Coverage:
* 1. useContracts GET /api/energy/contracts
* 2. useCreateContract POST /api/energy/contracts
* 3. useEnergyPrices GET /api/energy/prices
* 4. useEnergyCosts GET /api/energy/costs
* 5. useEnergyCostSummary GET /api/energy/costs/summary
* 6. useRecomputeCosts POST /api/energy/costs/recompute
*/
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { renderHook, waitFor, act } from '@testing-library/react'
import { QueryClient, QueryClientProvider } from '@tanstack/react-query'
import type { ReactNode } from 'react'
// ---------------------------------------------------------------------------
// Mock apiClient
// ---------------------------------------------------------------------------
const mockGet = vi.fn()
const mockPost = vi.fn()
const mockPatch = vi.fn()
vi.mock('../api/client', () => ({
default: {
GET: (...args: unknown[]) => mockGet(...args),
POST: (...args: unknown[]) => mockPost(...args),
PATCH: (...args: unknown[]) => mockPatch(...args),
DELETE: vi.fn(),
},
ApiError: class ApiError extends Error {
status: number
body: unknown
constructor(status: number, body: unknown) {
super(`API error ${status}`)
this.name = 'ApiError'
this.status = status
this.body = body
}
},
registerLoginRedirect: vi.fn(),
}))
// ---------------------------------------------------------------------------
// Fixtures
// ---------------------------------------------------------------------------
const CONTRACT = {
id: 1,
name: 'Test Contract',
kind: 'manual',
active: true,
currency: 'EUR',
created_at: '2026-06-01T00:00:00Z',
updated_at: '2026-06-01T00:00:00Z',
}
const PRICE_POINT = {
starts_at: '2026-06-22T10:00:00Z',
buy: 0.133,
sell: 0.09,
level: 'NORMAL',
}
const COST_PERIOD = {
period_start: '2026-06-22T10:00:00Z',
d1_kwh: 0.1,
d2_kwh: 0.2,
r1_kwh: 0.0,
r2_kwh: 0.05,
import_cost: 0.044,
export_revenue: 0.005,
net_cost: 0.039,
currency: 'EUR',
degraded: false,
contract_version_id: 1,
}
const SUMMARY = {
currency: 'EUR',
metered_import: 10.5,
metered_export: 2.3,
metered_net: 8.2,
fixed_costs: 5.0,
credits: 50.0,
total_payable: 12.5,
}
// ---------------------------------------------------------------------------
// Wrapper
// ---------------------------------------------------------------------------
function makeWrapper() {
const qc = new QueryClient({
defaultOptions: { queries: { retry: false }, mutations: { retry: false } },
})
function Wrapper({ children }: { children: ReactNode }) {
return <QueryClientProvider client={qc}>{children}</QueryClientProvider>
}
return { qc, Wrapper }
}
// ---------------------------------------------------------------------------
// Tests
// ---------------------------------------------------------------------------
describe('useContracts', () => {
beforeEach(() => vi.clearAllMocks())
it('calls GET /api/energy/contracts and returns contract list', async () => {
mockGet.mockResolvedValue({ data: { items: [CONTRACT], total: 1 } })
const { Wrapper } = makeWrapper()
const { useContracts } = await import('./hooks')
const { result } = renderHook(() => useContracts(), { wrapper: Wrapper })
await waitFor(() => expect(result.current.isSuccess).toBe(true))
expect(mockGet).toHaveBeenCalledWith('/api/energy/contracts')
expect(result.current.data?.items).toHaveLength(1)
expect(result.current.data?.items[0].id).toBe(1)
})
})
describe('useCreateContract', () => {
beforeEach(() => vi.clearAllMocks())
it('calls POST /api/energy/contracts with the provided body', async () => {
mockPost.mockResolvedValue({ data: { ...CONTRACT, versions: [] } })
mockGet.mockResolvedValue({ data: { items: [], total: 0 } })
const { Wrapper } = makeWrapper()
const { useCreateContract } = await import('./hooks')
const { result } = renderHook(() => useCreateContract(), { wrapper: Wrapper })
const body = {
name: 'New Contract',
kind: 'manual',
currency: 'EUR',
values: { energy: { buy: { normal: 0.133 } } },
}
await act(async () => {
await result.current.mutateAsync(body)
})
expect(mockPost).toHaveBeenCalledWith('/api/energy/contracts', { body })
})
})
describe('useEnergyPrices', () => {
beforeEach(() => vi.clearAllMocks())
it('calls GET /api/energy/prices with start and end params', async () => {
mockGet.mockResolvedValue({
data: {
kind: 'tibber',
currency: 'EUR',
points: [PRICE_POINT],
tariff: null,
},
})
const { Wrapper } = makeWrapper()
const { useEnergyPrices } = await import('./hooks')
const start = '2026-06-22T00:00:00Z'
const end = '2026-06-23T00:00:00Z'
const { result } = renderHook(() => useEnergyPrices(start, end), { wrapper: Wrapper })
await waitFor(() => expect(result.current.isSuccess).toBe(true))
expect(mockGet).toHaveBeenCalledWith('/api/energy/prices', {
params: { query: { start, end } },
})
expect(result.current.data?.points).toHaveLength(1)
expect(result.current.data?.kind).toBe('tibber')
})
it('calls GET /api/energy/prices without params when none provided', async () => {
mockGet.mockResolvedValue({
data: { kind: 'manual', currency: 'EUR', points: [], tariff: null },
})
const { Wrapper } = makeWrapper()
const { useEnergyPrices } = await import('./hooks')
const { result } = renderHook(() => useEnergyPrices(), { wrapper: Wrapper })
await waitFor(() => expect(result.current.isSuccess).toBe(true))
expect(mockGet).toHaveBeenCalledWith('/api/energy/prices', {
params: { query: {} },
})
})
})
describe('useEnergyCosts', () => {
beforeEach(() => vi.clearAllMocks())
it('calls GET /api/energy/costs with start, end, and limit', async () => {
mockGet.mockResolvedValue({
data: { items: [COST_PERIOD], total: 1 },
})
const { Wrapper } = makeWrapper()
const { useEnergyCosts } = await import('./hooks')
const start = '2026-06-22T00:00:00Z'
const end = '2026-06-23T00:00:00Z'
const limit = 100
const { result } = renderHook(
() => useEnergyCosts(start, end, limit),
{ wrapper: Wrapper },
)
await waitFor(() => expect(result.current.isSuccess).toBe(true))
expect(mockGet).toHaveBeenCalledWith('/api/energy/costs', {
params: { query: { start, end, limit } },
})
expect(result.current.data?.items).toHaveLength(1)
})
})
describe('useEnergyCostSummary', () => {
beforeEach(() => vi.clearAllMocks())
it('calls GET /api/energy/costs/summary with date range', async () => {
mockGet.mockResolvedValue({ data: SUMMARY })
const { Wrapper } = makeWrapper()
const { useEnergyCostSummary } = await import('./hooks')
const start = '2026-06-01T00:00:00Z'
const end = '2026-06-30T23:59:59Z'
const { result } = renderHook(
() => useEnergyCostSummary(start, end),
{ wrapper: Wrapper },
)
await waitFor(() => expect(result.current.isSuccess).toBe(true))
expect(mockGet).toHaveBeenCalledWith('/api/energy/costs/summary', {
params: { query: { start, end } },
})
expect(result.current.data?.total_payable).toBe(12.5)
})
})
describe('useRecomputeCosts', () => {
beforeEach(() => vi.clearAllMocks())
it('calls POST /api/energy/costs/recompute and invalidates cost queries', async () => {
mockPost.mockResolvedValue({ data: { recomputed: 10 } })
// Mock GET for invalidation queries (costs and summary)
mockGet.mockResolvedValue({ data: { items: [], total: 0 } })
const { Wrapper } = makeWrapper()
const { useRecomputeCosts } = await import('./hooks')
const { result } = renderHook(() => useRecomputeCosts(), { wrapper: Wrapper })
await act(async () => {
await result.current.mutateAsync({
start: '2026-06-22T00:00:00Z',
end: '2026-06-23T00:00:00Z',
})
})
expect(mockPost).toHaveBeenCalledWith('/api/energy/costs/recompute', {
params: {
query: {
start: '2026-06-22T00:00:00Z',
end: '2026-06-23T00:00:00Z',
},
},
})
})
it('calls POST /api/energy/costs/recompute without params when none provided', async () => {
mockPost.mockResolvedValue({ data: { recomputed: 0 } })
const { Wrapper } = makeWrapper()
const { useRecomputeCosts } = await import('./hooks')
const { result } = renderHook(() => useRecomputeCosts(), { wrapper: Wrapper })
await act(async () => {
await result.current.mutateAsync({})
})
expect(mockPost).toHaveBeenCalledWith('/api/energy/costs/recompute', {
params: { query: {} },
})
})
})
+21 -2
View File
@@ -173,7 +173,7 @@ describe('useUpdateDevice', () => {
describe('useDeleteDevice', () => {
beforeEach(() => vi.clearAllMocks())
it('calls DELETE /api/modbus/devices/{uuid}', async () => {
it('calls DELETE /api/modbus/devices/{uuid} without query param when cascade omitted', async () => {
mockDelete.mockResolvedValue({ data: null })
mockGet.mockResolvedValue({ data: { items: [], total: 0 } })
@@ -182,13 +182,32 @@ describe('useDeleteDevice', () => {
const { result } = renderHook(() => useDeleteDevice(), { wrapper: Wrapper })
await act(async () => {
await result.current.mutateAsync('test-uuid-1')
await result.current.mutateAsync({ uuid: 'test-uuid-1' })
})
expect(mockDelete).toHaveBeenCalledWith('/api/modbus/devices/{uuid}', {
params: { path: { uuid: 'test-uuid-1' } },
})
})
it('calls DELETE /api/modbus/devices/{uuid} with cascade=true in query when cascade=true', async () => {
mockDelete.mockResolvedValue({
data: { deleted: true, readings_deleted: 5, toggles_deleted: 2 },
})
mockGet.mockResolvedValue({ data: { items: [], total: 0 } })
const { Wrapper } = makeWrapper()
const { useDeleteDevice } = await import('./hooks')
const { result } = renderHook(() => useDeleteDevice(), { wrapper: Wrapper })
await act(async () => {
await result.current.mutateAsync({ uuid: 'test-uuid-1', cascade: true })
})
expect(mockDelete).toHaveBeenCalledWith('/api/modbus/devices/{uuid}', {
params: { path: { uuid: 'test-uuid-1' }, query: { cascade: true } },
})
})
})
describe('useTestReadDevice', () => {
+311 -2
View File
@@ -115,12 +115,21 @@ export function useUpdateDevice() {
// Mutation: delete device
// ---------------------------------------------------------------------------
export interface DeleteDeviceParams {
uuid: string
/** When true, perform a cascade delete (removes readings + expose toggles). */
cascade?: boolean
}
export function useDeleteDevice() {
const qc = useQueryClient()
return useMutation({
mutationFn: (uuid: string) =>
mutationFn: ({ uuid, cascade }: DeleteDeviceParams) =>
apiClient.DELETE('/api/modbus/devices/{uuid}', {
params: { path: { uuid } },
params: {
path: { uuid },
...(cascade ? { query: { cascade: true } } : {}),
},
}),
onSuccess: () => qc.invalidateQueries({ queryKey: ['modbus-devices'] }),
})
@@ -191,6 +200,306 @@ export function useMetrics(uuid: string) {
})
}
// ===========================================================================
// Energy / Pricing hooks — typed TanStack Query wrappers for /api/energy/*.
//
// Query-key conventions:
// ['energy-contracts'] — contract list
// ['energy-contract', id] — single contract with versions
// ['energy-profiles'] — pricing profile structures
// ['energy-prices', start, end] — price curve
// ['energy-costs', start, end, limit] — cost periods
// ['energy-costs-summary', start, end] — summary
// ['dsmr-latest'] — DSMR latest
// ===========================================================================
// Re-exported energy types for consumers
export type MeterResponse = components['schemas']['MeterResponse']
export type MeterListResponse = components['schemas']['MeterListResponse']
export type MeterDeclareRequest = components['schemas']['MeterDeclareRequest']
export type MeterPatchRequest = components['schemas']['MeterPatchRequest']
export type MeterReason = components['schemas']['MeterReason']
export type ContractResponse = components['schemas']['ContractResponse']
export type ContractDetailResponse = components['schemas']['ContractDetailResponse']
export type ContractVersionResponse = components['schemas']['ContractVersionResponse']
export type ContractListResponse = components['schemas']['ContractListResponse']
export type ContractCreate = components['schemas']['ContractCreate']
export type ContractPatch = components['schemas']['ContractPatch']
export type VersionCreate = components['schemas']['VersionCreate']
export type ProfilesResponse = components['schemas']['ProfilesResponse']
export type PricesResponse = components['schemas']['PricesResponse']
export type PricePointSchema = components['schemas']['PricePointSchema']
export type ManualTariffSchema = components['schemas']['ManualTariffSchema']
export type CostsResponse = components['schemas']['CostsResponse']
export type CostPeriodSchema = components['schemas']['CostPeriodSchema']
export type SummaryResponse = components['schemas']['SummaryResponse']
export type DsmrLatestResponse = components['schemas']['DsmrLatestResponse']
export type TibberTestResponse = components['schemas']['TibberTestResponse']
export type TibberTestPriceSchema = components['schemas']['TibberTestPriceSchema']
// ---------------------------------------------------------------------------
// Query: list all energy contracts
// ---------------------------------------------------------------------------
export function useContracts() {
return useQuery({
queryKey: ['energy-contracts'],
queryFn: async () => {
const res = await apiClient.GET('/api/energy/contracts')
return res.data
},
})
}
// ---------------------------------------------------------------------------
// Query: single contract with full version history
// ---------------------------------------------------------------------------
export function useContractDetail(id: number) {
return useQuery({
queryKey: ['energy-contract', id],
queryFn: async () => {
const res = await apiClient.GET('/api/energy/contracts/{contract_id}', {
params: { path: { contract_id: id } },
})
return res.data
},
enabled: !!id,
})
}
// ---------------------------------------------------------------------------
// Query: energy pricing profile structures
// ---------------------------------------------------------------------------
export function useEnergyProfiles() {
return useQuery({
queryKey: ['energy-profiles'],
queryFn: async () => {
const res = await apiClient.GET('/api/energy/profiles')
return res.data
},
staleTime: 5 * 60 * 1000,
})
}
// ---------------------------------------------------------------------------
// Mutation: create energy contract
// ---------------------------------------------------------------------------
export function useCreateContract() {
const qc = useQueryClient()
return useMutation({
mutationFn: (body: ContractCreate) =>
apiClient.POST('/api/energy/contracts', { body }),
onSuccess: () => qc.invalidateQueries({ queryKey: ['energy-contracts'] }),
})
}
// ---------------------------------------------------------------------------
// Mutation: update (PATCH) energy contract
// ---------------------------------------------------------------------------
export function useUpdateContract() {
const qc = useQueryClient()
return useMutation({
mutationFn: ({ id, body }: { id: number; body: ContractPatch }) =>
apiClient.PATCH('/api/energy/contracts/{contract_id}', {
params: { path: { contract_id: id } },
body,
}),
onSuccess: () => qc.invalidateQueries({ queryKey: ['energy-contracts'] }),
})
}
// ---------------------------------------------------------------------------
// Mutation: add a new version to an existing contract
// ---------------------------------------------------------------------------
export function useAddContractVersion() {
const qc = useQueryClient()
return useMutation({
mutationFn: ({ id, body }: { id: number; body: VersionCreate }) =>
apiClient.POST('/api/energy/contracts/{contract_id}/versions', {
params: { path: { contract_id: id } },
body,
}),
onSuccess: (_data, vars) => {
void qc.invalidateQueries({ queryKey: ['energy-contracts'] })
void qc.invalidateQueries({ queryKey: ['energy-contract', vars.id] })
},
})
}
// ---------------------------------------------------------------------------
// Query: energy price curve (Tibber 15-min / manual tariff)
// ---------------------------------------------------------------------------
export function useEnergyPrices(start?: string, end?: string) {
return useQuery({
queryKey: ['energy-prices', start, end],
queryFn: async () => {
const res = await apiClient.GET('/api/energy/prices', {
params: {
query: {
...(start ? { start } : {}),
...(end ? { end } : {}),
},
},
})
return res.data
},
})
}
// ---------------------------------------------------------------------------
// Query: cost periods
// ---------------------------------------------------------------------------
export function useEnergyCosts(start?: string, end?: string, limit?: number) {
return useQuery({
queryKey: ['energy-costs', start, end, limit],
queryFn: async () => {
const res = await apiClient.GET('/api/energy/costs', {
params: {
query: {
...(start ? { start } : {}),
...(end ? { end } : {}),
...(limit != null ? { limit } : {}),
},
},
})
return res.data
},
})
}
// ---------------------------------------------------------------------------
// Query: cost summary
// ---------------------------------------------------------------------------
export function useEnergyCostSummary(start?: string, end?: string) {
return useQuery({
queryKey: ['energy-costs-summary', start, end],
queryFn: async () => {
const res = await apiClient.GET('/api/energy/costs/summary', {
params: {
query: {
...(start ? { start } : {}),
...(end ? { end } : {}),
},
},
})
return res.data
},
})
}
// ---------------------------------------------------------------------------
// Query: DSMR latest reading
// ---------------------------------------------------------------------------
export function useDsmrLatest(options?: AutoRefreshOptions) {
const refetchInterval = options?.refetchIntervalMs != null
? Math.max(2_000, options.refetchIntervalMs)
: undefined
return useQuery({
queryKey: ['dsmr-latest'],
queryFn: async () => {
const res = await apiClient.GET('/api/energy/dsmr/latest')
return res.data
},
refetchInterval,
})
}
// ---------------------------------------------------------------------------
// Mutation: recompute costs
// ---------------------------------------------------------------------------
export function useRecomputeCosts() {
const qc = useQueryClient()
return useMutation({
mutationFn: ({ start, end }: { start?: string; end?: string } = {}) => {
// The schema marks start/end as required, but the backend accepts them as
// optional query params; we spread only defined values.
const query = {
...(start ? { start } : {}),
...(end ? { end } : {}),
} as { start: string; end: string }
return apiClient.POST('/api/energy/costs/recompute', {
params: { query },
})
},
onSuccess: () => {
void qc.invalidateQueries({ queryKey: ['energy-costs'] })
void qc.invalidateQueries({ queryKey: ['energy-costs-summary'] })
},
})
}
// ===========================================================================
// Meter hooks — typed TanStack Query wrappers for /api/energy/meters.
//
// Query-key conventions:
// ['energy-meters'] — meter list
// ===========================================================================
// ---------------------------------------------------------------------------
// Query: list all meter epochs
// ---------------------------------------------------------------------------
export function useMeters() {
return useQuery({
queryKey: ['energy-meters'],
queryFn: async () => {
const res = await apiClient.GET('/api/energy/meters')
return res.data
},
})
}
// ---------------------------------------------------------------------------
// Mutation: declare a new meter epoch (swap / home move / initial)
// ---------------------------------------------------------------------------
export function useDeclareMeter() {
const qc = useQueryClient()
return useMutation({
mutationFn: (body: MeterDeclareRequest) =>
apiClient.POST('/api/energy/meters', { body }),
onSuccess: () => {
void qc.invalidateQueries({ queryKey: ['energy-meters'] })
// Invalidate cost-related queries: a new meter may trigger recompute server-side.
void qc.invalidateQueries({ queryKey: ['energy-costs'] })
void qc.invalidateQueries({ queryKey: ['energy-costs-summary'] })
},
})
}
// ---------------------------------------------------------------------------
// Mutation: update (PATCH) a meter epoch (label / note / started_at)
// ---------------------------------------------------------------------------
export function useUpdateMeter() {
const qc = useQueryClient()
return useMutation({
mutationFn: ({ id, body }: { id: number; body: MeterPatchRequest }) =>
apiClient.PATCH('/api/energy/meters/{meter_id}', {
params: { path: { meter_id: id } },
body,
}),
onSuccess: () => {
void qc.invalidateQueries({ queryKey: ['energy-meters'] })
// Retroactive started_at correction triggers recompute server-side.
void qc.invalidateQueries({ queryKey: ['energy-costs'] })
void qc.invalidateQueries({ queryKey: ['energy-costs-summary'] })
},
})
}
// ---------------------------------------------------------------------------
// Query: time-range readings for a device (window + limit — never full-table)
// ---------------------------------------------------------------------------
@@ -0,0 +1,212 @@
/**
* Tests for Tibber test button in ConfigPage.
*
* Coverage:
* 1. Tibber test button appears when Tibber section is in config.
* 2. Success tri-state shows green alert.
* 3. Config-error tri-state shows orange alert.
* 4. Failed tri-state shows red alert.
*/
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { screen, waitFor } from '@testing-library/react'
import userEvent from '@testing-library/user-event'
import { renderWithProviders } from '../test-utils'
// ---------------------------------------------------------------------------
// Mock apiClient
// ---------------------------------------------------------------------------
const mockGet = vi.fn()
const mockPost = vi.fn()
const mockPut = vi.fn()
vi.mock('../api/client', () => ({
default: {
GET: (...args: unknown[]) => mockGet(...args),
POST: (...args: unknown[]) => mockPost(...args),
PUT: (...args: unknown[]) => mockPut(...args),
PATCH: vi.fn(),
DELETE: vi.fn(),
},
ApiError: class ApiError extends Error {
status: number
body: unknown
constructor(status: number, body: unknown) {
super(`API error ${status}`)
this.name = 'ApiError'
this.status = status
this.body = body
}
},
registerLoginRedirect: vi.fn(),
}))
// ---------------------------------------------------------------------------
// Fixtures
// ---------------------------------------------------------------------------
const CONFIG_WITH_TIBBER = {
sections: [
{
name: 'Tibber',
fields: [
{
env_name: 'TIBBER_TOKEN',
label: 'Tibber API Token',
secret: true,
input_type: 'text',
configured: true,
value: null,
},
],
},
],
}
// ---------------------------------------------------------------------------
// Import component
// ---------------------------------------------------------------------------
import { ConfigPage } from './ConfigPage'
// ---------------------------------------------------------------------------
// Helper to suppress TOTP and Expose Settings sub-queries
// ---------------------------------------------------------------------------
function mockConfigDependencies() {
mockGet.mockImplementation((path: string) => {
if (path === '/api/config') {
return Promise.resolve({ data: CONFIG_WITH_TIBBER })
}
if (path === '/api/expose') {
return Promise.resolve({
data: {
catalog: [],
mqtt_status: { connected: false, broker: null },
},
})
}
if (path === '/api/auth/totp/status') {
return Promise.resolve({ data: { enabled: false } })
}
return Promise.resolve({ data: null })
})
}
// ---------------------------------------------------------------------------
// Tests
// ---------------------------------------------------------------------------
describe('ConfigPage — Tibber test button', () => {
beforeEach(() => vi.clearAllMocks())
it('renders Tibber test button when Tibber section is present', async () => {
mockConfigDependencies()
renderWithProviders(<ConfigPage />)
await waitFor(() => {
expect(screen.getByTestId('tibber-test-button')).toBeInTheDocument()
})
})
it('shows success alert on successful Tibber test', async () => {
const user = userEvent.setup()
mockConfigDependencies()
mockPost.mockImplementation((path: string) => {
if (path === '/api/energy/tibber/test') {
return Promise.resolve({
data: {
result: 'success',
message: 'Connected to Tibber API',
price: {
starts_at: '2026-06-22T10:00:00Z',
total: 0.1337,
energy: 0.08,
tax: 0.0537,
currency: 'EUR',
level: 'NORMAL',
},
},
})
}
return Promise.resolve({ data: null })
})
renderWithProviders(<ConfigPage />)
await waitFor(() => {
expect(screen.getByTestId('tibber-test-button')).toBeInTheDocument()
})
await user.click(screen.getByTestId('tibber-test-button'))
await waitFor(() => {
expect(screen.getByTestId('tibber-result-success')).toBeInTheDocument()
})
expect(screen.getByTestId('tibber-result-success')).toHaveTextContent(
'Tibber connection successful',
)
})
it('shows config-error alert when Tibber is misconfigured', async () => {
const user = userEvent.setup()
mockConfigDependencies()
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const ApiErrorClass = (await import('../api/client')).ApiError as any
mockPost.mockImplementation((path: string) => {
if (path === '/api/energy/tibber/test') {
throw new ApiErrorClass(400, {
result: 'config-error',
message: 'Tibber token not configured',
})
}
return Promise.resolve({ data: null })
})
renderWithProviders(<ConfigPage />)
await waitFor(() => {
expect(screen.getByTestId('tibber-test-button')).toBeInTheDocument()
})
await user.click(screen.getByTestId('tibber-test-button'))
await waitFor(() => {
expect(screen.getByTestId('tibber-result-config-error')).toBeInTheDocument()
})
})
it('shows failed alert when Tibber test fails', async () => {
const user = userEvent.setup()
mockConfigDependencies()
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const ApiErrorClass = (await import('../api/client')).ApiError as any
mockPost.mockImplementation((path: string) => {
if (path === '/api/energy/tibber/test') {
throw new ApiErrorClass(500, {
result: 'failed',
message: 'Connection refused',
})
}
return Promise.resolve({ data: null })
})
renderWithProviders(<ConfigPage />)
await waitFor(() => {
expect(screen.getByTestId('tibber-test-button')).toBeInTheDocument()
})
await user.click(screen.getByTestId('tibber-test-button'))
await waitFor(() => {
expect(screen.getByTestId('tibber-result-failed')).toBeInTheDocument()
})
})
})
+96
View File
@@ -49,6 +49,7 @@ import { TotpSettings } from './TotpSettings'
type ConfigField = components['schemas']['ConfigField']
type ConfigSection = components['schemas']['ConfigSection']
type TibberTestPriceSchema = components['schemas']['TibberTestPriceSchema']
/** SMTP test result tri-state. */
type SmtpResult =
@@ -64,6 +65,13 @@ type MqttResult =
| { kind: 'failed'; message: string }
| null
/** Tibber test result tri-state. */
type TibberResult =
| { kind: 'success'; message: string; price?: TibberTestPriceSchema | null }
| { kind: 'config-error'; message: string }
| { kind: 'failed'; message: string }
| null
// ---------------------------------------------------------------------------
// Hook: load config
// ---------------------------------------------------------------------------
@@ -339,6 +347,85 @@ function MqttTestButton({ mqttResult, setMqttResult }: MqttTestButtonProps) {
)
}
// ---------------------------------------------------------------------------
// TibberTestButton — sends POST /api/energy/tibber/test and displays tri-state result
// ---------------------------------------------------------------------------
interface TibberTestButtonProps {
tibberResult: TibberResult
setTibberResult: (r: TibberResult) => void
}
function TibberTestButton({ tibberResult, setTibberResult }: TibberTestButtonProps) {
const [testing, setTesting] = useState(false)
async function handleTest() {
setTibberResult(null)
setTesting(true)
try {
const res = await apiClient.POST('/api/energy/tibber/test')
if (res.data) {
setTibberResult({
kind: 'success',
message: res.data.message,
price: res.data.price,
})
}
} catch (err) {
if (err instanceof ApiError) {
const body = err.body as { result?: string; message?: string } | null
const result = body?.result
const message = body?.message ?? 'Unknown error'
if (result === 'config-error') {
setTibberResult({ kind: 'config-error', message })
} else {
setTibberResult({ kind: 'failed', message })
}
} else {
setTibberResult({ kind: 'failed', message: 'Unexpected error during Tibber test.' })
}
} finally {
setTesting(false)
}
}
return (
<Stack gap="xs">
<Button
type="button"
variant="outline"
onClick={handleTest}
loading={testing}
data-testid="tibber-test-button"
>
Test Tibber Connection
</Button>
{tibberResult?.kind === 'success' && (
<Alert color="green" data-testid="tibber-result-success">
Tibber connection successful. {tibberResult.message}
{tibberResult.price && (
<Text size="xs" mt={4}>
Current price: {tibberResult.price.total} {tibberResult.price.currency}/kWh
{tibberResult.price.level ? ` (${tibberResult.price.level})` : ''}
</Text>
)}
</Alert>
)}
{tibberResult?.kind === 'config-error' && (
<Alert color="orange" data-testid="tibber-result-config-error">
Tibber configuration error check your Tibber API token. {tibberResult.message}
</Alert>
)}
{tibberResult?.kind === 'failed' && (
<Alert color="red" data-testid="tibber-result-failed">
Tibber test failed. {tibberResult.message}
</Alert>
)}
</Stack>
)
}
// ---------------------------------------------------------------------------
// ConfigPage — main component
// ---------------------------------------------------------------------------
@@ -374,6 +461,9 @@ export function ConfigPage() {
// MQTT test tri-state
const [mqttResult, setMqttResult] = useState<MqttResult>(null)
// Tibber test tri-state
const [tibberResult, setTibberResult] = useState<TibberResult>(null)
function handleChange(envName: string, value: string) {
setLocalValues((prev) => ({ ...prev, [envName]: value }))
setSaveStatus(null)
@@ -432,6 +522,9 @@ export function ConfigPage() {
// Detect if there is an MQTT section (to show the MQTT test button).
const hasMqttSection = data.sections.some((s) => s.name.toLowerCase() === 'mqtt')
// Detect if there is a Tibber section (to show the Tibber test button).
const hasTibberSection = data.sections.some((s) => s.name.toLowerCase() === 'tibber')
// Default: open the first section so users immediately see content.
const defaultAccordionValue = data.sections[0]?.name ?? null
@@ -512,6 +605,9 @@ export function ConfigPage() {
{hasMqttSection && (
<MqttTestButton mqttResult={mqttResult} setMqttResult={setMqttResult} />
)}
{hasTibberSection && (
<TibberTestButton tibberResult={tibberResult} setTibberResult={setTibberResult} />
)}
</Group>
</Group>
</Stack>
+71
View File
@@ -288,6 +288,7 @@ describe('EnergyPage — delete device', () => {
await waitFor(() => expect(screen.getByTestId(`device-delete-${DEVICE.uuid}`)).toBeInTheDocument())
fireEvent.click(screen.getByTestId(`device-delete-${DEVICE.uuid}`))
// On first open (no 409 yet), the regular confirm button is visible.
await waitFor(() => expect(screen.getByTestId('device-delete-confirm')).toBeInTheDocument())
fireEvent.click(screen.getByTestId('device-delete-confirm'))
@@ -295,6 +296,38 @@ describe('EnergyPage — delete device', () => {
await waitFor(() => {
expect(screen.getByTestId('device-delete-409-hint')).toBeInTheDocument()
})
// After 409, the force-delete button appears instead of the normal confirm.
expect(screen.getByTestId('device-delete-force')).toBeInTheDocument()
expect(screen.queryByTestId('device-delete-confirm')).not.toBeInTheDocument()
})
it('force-delete button triggers cascade delete after 409', async () => {
const { ApiError } = await import('../api/client')
// First call → 409, second call (cascade) → success
mockDelete
.mockRejectedValueOnce(new ApiError(409, { detail: 'device has readings' }))
.mockResolvedValueOnce({ data: { deleted: true, readings_deleted: 3, toggles_deleted: 1 } })
renderEnergy()
await waitFor(() => expect(screen.getByTestId(`device-delete-${DEVICE.uuid}`)).toBeInTheDocument())
fireEvent.click(screen.getByTestId(`device-delete-${DEVICE.uuid}`))
await waitFor(() => expect(screen.getByTestId('device-delete-confirm')).toBeInTheDocument())
// First attempt → 409
fireEvent.click(screen.getByTestId('device-delete-confirm'))
// Force-delete button appears
await waitFor(() => expect(screen.getByTestId('device-delete-force')).toBeInTheDocument())
// Click force-delete → should call DELETE with cascade=true
fireEvent.click(screen.getByTestId('device-delete-force'))
await waitFor(() => {
expect(mockDelete).toHaveBeenCalledWith('/api/modbus/devices/{uuid}', {
params: { path: { uuid: DEVICE.uuid }, query: { cascade: true } },
})
})
})
})
@@ -348,6 +381,44 @@ describe('EnergyPage — test-read', () => {
})
})
describe('EnergyPage — meters tab', () => {
beforeEach(() => {
vi.clearAllMocks()
setupDefaultMocks()
})
it('renders Meters tab in the tab list', async () => {
renderEnergy()
await waitFor(() => {
expect(screen.getByTestId('tab-meters')).toBeInTheDocument()
})
expect(screen.getByTestId('tab-meters').textContent).toContain('Meters')
})
it('renders meters panel when Meters tab is clicked', async () => {
mockGet.mockImplementation((path: string) => {
if (path === '/api/modbus/devices') {
return Promise.resolve({ data: { items: [DEVICE], total: 1 } })
}
if (path === '/api/energy/meters') {
return Promise.resolve({ data: { items: [], total: 0 } })
}
return Promise.resolve({ data: null })
})
renderEnergy()
await waitFor(() => expect(screen.getByTestId('tab-meters')).toBeInTheDocument())
fireEvent.click(screen.getByTestId('tab-meters'))
await waitFor(() => {
expect(screen.getByTestId('panel-meters')).toBeInTheDocument()
})
})
})
describe('EnergyPage — auto-refresh switch', () => {
beforeEach(() => {
vi.clearAllMocks()
+110 -16
View File
@@ -11,6 +11,8 @@
* - Latest readings card per device (T07): fetches /latest and /metrics; tolerates
* missing payload keys.
* - Trend charts per device (T07): EnergyCharts component (Recharts isolated inside).
*
* M6-T10: added Tabs layout with Devices / Contracts / Prices / Costs tabs.
*/
import { useState } from 'react'
@@ -34,13 +36,20 @@ import {
SimpleGrid,
Divider,
Switch,
Tabs,
} from '@mantine/core'
import { useDevices, useDeleteDevice, useTestReadDevice, useLatestReading, useMetrics } from '../energy/hooks'
import { DeviceForm } from '../energy/DeviceForm'
import { EnergyCharts } from '../energy/EnergyCharts'
import { ContractManager } from '../energy/ContractManager'
import { MeterManager } from '../energy/MeterManager'
import { TibberPrices } from '../energy/TibberPrices'
import { CostView } from '../energy/CostView'
import { DsmrPanel } from '../energy/DsmrPanel'
import type { ModbusDevice, ModbusTestReadResponse, MetricInfo } from '../energy/hooks'
import { ApiError } from '../api/client'
import { formatMetricValue } from '../energy/format'
import { formatLocalDateTime } from '../utils/datetime'
// ---------------------------------------------------------------------------
// Delete confirmation modal (with 409 "disable instead" hint)
@@ -49,13 +58,22 @@ import { formatMetricValue } from '../energy/format'
interface ConfirmDeleteProps {
device: ModbusDevice
onConfirm: () => void
/** Called when the user explicitly opts into cascade (force) deletion. */
onForceDelete: () => void
onCancel: () => void
loading: boolean
/** Set when the delete failed with 409. */
has409Error: boolean
}
function ConfirmDeleteModal({ device, onConfirm, onCancel, loading, has409Error }: ConfirmDeleteProps) {
function ConfirmDeleteModal({
device,
onConfirm,
onForceDelete,
onCancel,
loading,
has409Error,
}: ConfirmDeleteProps) {
return (
<Modal
opened
@@ -70,16 +88,24 @@ function ConfirmDeleteModal({ device, onConfirm, onCancel, loading, has409Error
</Text>
{has409Error && (
<>
<Alert color="orange" data-testid="device-delete-409-hint">
This device has existing readings and cannot be deleted. Consider{' '}
<strong>disabling</strong> it instead (click Edit uncheck Enabled).
</Alert>
<Alert color="red" variant="light" data-testid="device-delete-force-warning">
Alternatively, you can <strong>permanently delete</strong> the device together with{' '}
all its historical readings and expose settings. This action{' '}
<strong>cannot be undone</strong>.
</Alert>
</>
)}
<Group justify="flex-end" gap="sm">
<Button variant="default" onClick={onCancel} data-testid="device-delete-cancel">
Cancel
</Button>
{!has409Error && (
<Button
color="red"
loading={loading}
@@ -88,6 +114,17 @@ function ConfirmDeleteModal({ device, onConfirm, onCancel, loading, has409Error
>
Delete
</Button>
)}
{has409Error && (
<Button
color="red"
loading={loading}
onClick={onForceDelete}
data-testid="device-delete-force"
>
Force Delete (all data)
</Button>
)}
</Group>
</Stack>
</Modal>
@@ -304,7 +341,7 @@ function LatestReadingsCard({ device, autoRefresh }: LatestReadingsCardProps) {
</Text>
{latest.recorded_at && (
<Text size="xs" c="dimmed">
{new Date(latest.recorded_at).toLocaleString()}
{formatLocalDateTime(latest.recorded_at)}
</Text>
)}
</Group>
@@ -488,10 +525,10 @@ function DeviceReadingsSection({ devices }: DeviceReadingsSectionProps) {
}
// ---------------------------------------------------------------------------
// EnergyPage — top-level
// DevicesTab — self-contained devices management tab (extracted from original EnergyPage)
// ---------------------------------------------------------------------------
export function EnergyPage() {
function DevicesTab() {
const devicesQuery = useDevices()
const deleteMutation = useDeleteDevice()
@@ -524,7 +561,7 @@ export function EnergyPage() {
if (!deleteDevice) return
setDelete409(false)
try {
await deleteMutation.mutateAsync(deleteDevice.uuid)
await deleteMutation.mutateAsync({ uuid: deleteDevice.uuid })
closeDelete()
} catch (err) {
if (err instanceof ApiError && err.status === 409) {
@@ -534,9 +571,16 @@ export function EnergyPage() {
}
}
// ---------------------------------------------------------------------------
// Loading / error states
// ---------------------------------------------------------------------------
async function handleForceDelete() {
if (!deleteDevice) return
try {
await deleteMutation.mutateAsync({ uuid: deleteDevice.uuid, cascade: true })
closeDelete()
} catch {
// If cascade delete also fails for some reason, keep the modal open.
// (Should be very rare — only if the device was concurrently deleted.)
}
}
if (devicesQuery.isLoading) {
return (
@@ -548,22 +592,15 @@ export function EnergyPage() {
if (devicesQuery.isError || !devicesQuery.data) {
return (
<Container size="xl" pt="xl">
<Alert color="red" data-testid="energy-load-error">
Failed to load devices. Please refresh.
</Alert>
</Container>
)
}
const devices = devicesQuery.data.items
// ---------------------------------------------------------------------------
// Main render
// ---------------------------------------------------------------------------
return (
<Container size="xl" pt="xl" pb="xl" data-testid="energy-page">
<Stack gap="lg">
<Group justify="space-between" align="center">
<Title order={2}>Energy Devices</Title>
@@ -580,7 +617,6 @@ export function EnergyPage() {
{/* Latest readings cards + trend charts (T07) */}
<DeviceReadingsSection devices={devices} />
</Stack>
{/* Create form */}
{showCreateForm && (
@@ -604,11 +640,69 @@ export function EnergyPage() {
<ConfirmDeleteModal
device={deleteDevice}
onConfirm={handleDeleteConfirm}
onForceDelete={handleForceDelete}
onCancel={closeDelete}
loading={deleteMutation.isPending}
has409Error={delete409}
/>
)}
</Stack>
)
}
// ---------------------------------------------------------------------------
// EnergyPage — top-level with Tabs
// ---------------------------------------------------------------------------
export function EnergyPage() {
return (
<Container size="xl" pt="xl" pb="xl" data-testid="energy-page">
<Tabs defaultValue="devices">
<Tabs.List mb="lg">
<Tabs.Tab value="devices" data-testid="tab-devices">
Devices
</Tabs.Tab>
<Tabs.Tab value="meters" data-testid="tab-meters">
Meters
</Tabs.Tab>
<Tabs.Tab value="contracts" data-testid="tab-contracts">
Contracts
</Tabs.Tab>
<Tabs.Tab value="prices" data-testid="tab-prices">
Prices
</Tabs.Tab>
<Tabs.Tab value="costs" data-testid="tab-costs">
Costs
</Tabs.Tab>
<Tabs.Tab value="dsmr" data-testid="tab-dsmr">
DSMR
</Tabs.Tab>
</Tabs.List>
<Tabs.Panel value="devices" data-testid="panel-devices">
<DevicesTab />
</Tabs.Panel>
<Tabs.Panel value="meters" data-testid="panel-meters">
<MeterManager />
</Tabs.Panel>
<Tabs.Panel value="contracts" data-testid="panel-contracts">
<ContractManager />
</Tabs.Panel>
<Tabs.Panel value="prices" data-testid="panel-prices">
<TibberPrices />
</Tabs.Panel>
<Tabs.Panel value="costs" data-testid="panel-costs">
<CostView />
</Tabs.Panel>
<Tabs.Panel value="dsmr" data-testid="panel-dsmr">
<DsmrPanel />
</Tabs.Panel>
</Tabs>
</Container>
)
}
+3 -6
View File
@@ -35,6 +35,7 @@ import type { PooRecord, LocationRecord } from '../records'
import { useDevices, useMetrics, useReadings } from '../energy/hooks'
import type { ModbusDevice, MetricInfo } from '../energy/hooks'
import { formatMetricValue } from '../energy/format'
import { formatLocalDateTime } from '../utils/datetime'
// ---------------------------------------------------------------------------
// Constants
@@ -359,13 +360,9 @@ interface MeterReadingsTableProps {
device: ModbusDevice
}
/** Format an ISO timestamp for display. */
/** Format an ISO timestamp for display (local timezone, 24-hour). */
function formatRecordedAt(iso: string): string {
try {
return new Date(iso).toLocaleString()
} catch {
return iso
}
return formatLocalDateTime(iso)
}
function MeterReadingsTable({ device }: MeterReadingsTableProps) {
+172
View File
@@ -0,0 +1,172 @@
/**
* Tests for src/utils/datetime.ts
*
* Key invariants verified (timezone-agnostic where possible):
* 1. A tz-less string is parsed identically to the same string with "Z" appended
* (i.e. treated as UTC regardless of the machine's local timezone).
* 2. Formatted output never contains AM / PM / am / pm (always 24-hour).
* 3. Strings that already carry a timezone marker are NOT double-offset.
* 4. Invalid / empty input does not throw returns original string or safe fallback.
* 5. Pure date strings (no T) are handled without crashing.
*/
import { describe, it, expect } from 'vitest'
import {
parseBackendTimestamp,
formatLocalDateTime,
formatLocalTime,
formatLocalDate,
} from './datetime'
// ---------------------------------------------------------------------------
// parseBackendTimestamp — core UTC-fallback logic
// ---------------------------------------------------------------------------
describe('parseBackendTimestamp', () => {
it('returns the same Date for tz-less and Z-suffixed strings (UTC equivalence)', () => {
const noTz = parseBackendTimestamp('2026-06-24T10:27:00')
const withZ = parseBackendTimestamp('2026-06-24T10:27:00Z')
// Both must represent the exact same UTC instant.
expect(noTz.getTime()).toBe(withZ.getTime())
})
it('does not double-offset a string that already has Z', () => {
const d1 = parseBackendTimestamp('2026-06-24T10:27:00Z')
const d2 = parseBackendTimestamp('2026-06-24T10:27:00')
expect(d1.getTime()).toBe(d2.getTime())
})
it('correctly parses a string with +HH:MM offset', () => {
// 2026-06-24T12:27:00+02:00 == 2026-06-24T10:27:00Z
const withOffset = parseBackendTimestamp('2026-06-24T12:27:00+02:00')
const utc = parseBackendTimestamp('2026-06-24T10:27:00Z')
expect(withOffset.getTime()).toBe(utc.getTime())
})
it('correctly parses a string with -HH:MM offset', () => {
// 2026-06-24T02:27:00-08:00 == 2026-06-24T10:27:00Z
const withOffset = parseBackendTimestamp('2026-06-24T02:27:00-08:00')
const utc = parseBackendTimestamp('2026-06-24T10:27:00Z')
expect(withOffset.getTime()).toBe(utc.getTime())
})
it('handles pure date strings (no T) without crashing', () => {
const d = parseBackendTimestamp('2026-06-24')
expect(isNaN(d.getTime())).toBe(false)
})
it('returns Invalid Date for empty string', () => {
const d = parseBackendTimestamp('')
expect(isNaN(d.getTime())).toBe(true)
})
it('returns Invalid Date for garbage input', () => {
const d = parseBackendTimestamp('not-a-date')
expect(isNaN(d.getTime())).toBe(true)
})
})
// ---------------------------------------------------------------------------
// formatLocalDateTime — no AM/PM, 24-hour
// ---------------------------------------------------------------------------
describe('formatLocalDateTime', () => {
it('produces the same output for tz-less and Z-suffixed strings (UTC equivalence)', () => {
const noTz = formatLocalDateTime('2026-06-24T10:27:00')
const withZ = formatLocalDateTime('2026-06-24T10:27:00Z')
// Both should format to the same local representation.
expect(noTz).toBe(withZ)
})
it('does not contain AM or PM (24-hour enforcement)', () => {
// Use noon UTC — a time where AM/PM distinction matters.
const result = formatLocalDateTime('2026-06-24T14:30:00Z')
expect(result).not.toMatch(/[Aa][Mm]|[Pp][Mm]/)
})
it('does not throw or return undefined for empty string', () => {
const result = formatLocalDateTime('')
expect(result).toBe('')
})
it('does not throw or return undefined for invalid string', () => {
const result = formatLocalDateTime('not-a-date')
// Should return the original string as a safe fallback.
expect(result).toBe('not-a-date')
})
it('returns a non-empty string for a valid timestamp', () => {
const result = formatLocalDateTime('2026-06-24T10:27:00Z')
expect(typeof result).toBe('string')
expect(result.length).toBeGreaterThan(0)
})
})
// ---------------------------------------------------------------------------
// formatLocalTime — compact HH:mm, no AM/PM
// ---------------------------------------------------------------------------
describe('formatLocalTime', () => {
it('produces the same output for tz-less and Z-suffixed strings', () => {
const noTz = formatLocalTime('2026-06-24T10:27:00')
const withZ = formatLocalTime('2026-06-24T10:27:00Z')
expect(noTz).toBe(withZ)
})
it('does not contain AM or PM', () => {
const result = formatLocalTime('2026-06-24T14:30:00Z')
expect(result).not.toMatch(/[Aa][Mm]|[Pp][Mm]/)
})
it('returns a non-empty string for a valid timestamp', () => {
const result = formatLocalTime('2026-06-24T10:27:00Z')
expect(typeof result).toBe('string')
expect(result.length).toBeGreaterThan(0)
})
it('does not throw for empty string', () => {
expect(() => formatLocalTime('')).not.toThrow()
expect(formatLocalTime('')).toBe('')
})
it('does not throw for invalid string', () => {
expect(() => formatLocalTime('bad')).not.toThrow()
expect(formatLocalTime('bad')).toBe('bad')
})
})
// ---------------------------------------------------------------------------
// formatLocalDate — date only
// ---------------------------------------------------------------------------
describe('formatLocalDate', () => {
it('produces the same output for tz-less and Z-suffixed strings', () => {
// Pick a time that won't cross a date boundary in any timezone
// (noon UTC — safe for UTC-12 through UTC+14).
const noTz = formatLocalDate('2026-06-24T12:00:00')
const withZ = formatLocalDate('2026-06-24T12:00:00Z')
expect(noTz).toBe(withZ)
})
it('handles pure date strings without crashing', () => {
expect(() => formatLocalDate('2026-06-24')).not.toThrow()
const result = formatLocalDate('2026-06-24')
expect(typeof result).toBe('string')
expect(result.length).toBeGreaterThan(0)
})
it('does not throw for empty string', () => {
expect(() => formatLocalDate('')).not.toThrow()
expect(formatLocalDate('')).toBe('')
})
it('does not throw for invalid string', () => {
expect(() => formatLocalDate('not-a-date')).not.toThrow()
expect(formatLocalDate('not-a-date')).toBe('not-a-date')
})
it('returns a non-empty string for a valid date', () => {
const result = formatLocalDate('2026-06-01T00:00:00Z')
expect(result.length).toBeGreaterThan(0)
})
})
+103
View File
@@ -0,0 +1,103 @@
/**
* datetime.ts shared time formatting utilities for Energy/Modbus displays.
*
* Root cause addressed: the backend stores timestamps as naive ISO strings without
* timezone info (e.g. "2026-06-24T10:27:00", no Z/offset). Browsers parse these as
* *local* time by default, but the values are actually UTC. We fix this by treating
* any string that lacks a timezone marker as UTC (appending "Z" before parsing).
*
* Scope: Energy domain + Modbus meter readings ONLY.
* Do NOT use these for Location/Poo timestamps those already use raw strings.
*/
/**
* Timezone-marker detection regex.
* Matches a trailing Z (any case) or a UTC offset like +05:30, -08:00, +0530, -0800.
*/
const TZ_MARKER_RE = /([zZ]|[+-]\d{2}:?\d{2})$/
/**
* Parse a backend ISO timestamp, treating strings without a timezone marker as UTC.
*
* - If the string already has Z / +HH:MM / -HHMM parse as-is.
* - If no timezone marker append "Z" so the browser reads it as UTC.
* - Pure date strings like "2026-06-24" (no 'T') parse as-is (Date handles them).
* - Invalid / empty input returns an Invalid Date (does NOT throw).
*/
export function parseBackendTimestamp(iso: string): Date {
if (!iso || typeof iso !== 'string') return new Date(NaN)
const hasTComponent = iso.includes('T')
if (hasTComponent && !TZ_MARKER_RE.test(iso)) {
// No timezone marker on a datetime string — treat as UTC.
return new Date(iso + 'Z')
}
return new Date(iso)
}
// ---------------------------------------------------------------------------
// Locale formatting helpers
// All use the browser's local timezone (no explicit timeZone option) and
// 24-hour clock (hour12: false).
// ---------------------------------------------------------------------------
/**
* Local date + time, 24-hour, e.g. "6/24/2026, 12:27:00".
* Used for: LatestReadingsCard recorded_at, DSMR recorded_at, chart tooltips,
* meter readings table (RecordsPage), cost period detail table.
*/
export function formatLocalDateTime(iso: string): string {
try {
const d = parseBackendTimestamp(iso)
if (isNaN(d.getTime())) return iso
return d.toLocaleString(undefined, {
hour12: false,
year: 'numeric',
month: 'numeric',
day: 'numeric',
hour: '2-digit',
minute: '2-digit',
second: '2-digit',
})
} catch {
return iso
}
}
/**
* Local time only, compact HH:mm (24-hour), e.g. "12:27".
* Used for: chart X-axis ticks, cost chart X-axis, Tibber price chart X-axis.
*/
export function formatLocalTime(iso: string): string {
try {
const d = parseBackendTimestamp(iso)
if (isNaN(d.getTime())) return iso
return d.toLocaleTimeString(undefined, {
hour12: false,
hour: '2-digit',
minute: '2-digit',
})
} catch {
return iso
}
}
/**
* Local date only, e.g. "6/1/2026" or "2026-06-01" depending on locale.
* Used for: contract effective_from / effective_to dates, contract created_at.
*/
export function formatLocalDate(iso: string): string {
try {
const d = parseBackendTimestamp(iso)
if (isNaN(d.getTime())) return iso
return d.toLocaleDateString(undefined, {
year: 'numeric',
month: 'numeric',
day: 'numeric',
})
} catch {
return iso
}
}
+8
View File
@@ -20,5 +20,13 @@ export default defineConfig({
environment: 'jsdom',
globals: true,
setupFiles: ['./src/test-setup.ts'],
env: {
// Lock the test timezone to UTC so that date-formatting assertions
// (which rely on toLocalDateInputString / getFullYear etc.) produce
// deterministic results regardless of the CI runner's local timezone.
// All fixture timestamps are UTC midnight, so the expected YYYY-MM-DD
// values in MeterManager.test.tsx are correct under UTC.
TZ: 'UTC',
},
},
})
+1747 -1
View File
File diff suppressed because it is too large Load Diff
+1509 -3
View File
File diff suppressed because it is too large Load Diff
+1 -1
View File
@@ -15,7 +15,7 @@ if str(PROJECT_ROOT) not in sys.path:
from app.config import get_settings
APP_BASELINE_REVISION = "20260622_10_exposed_entities"
APP_BASELINE_REVISION = "20260625_14_meter_uuid"
class AppDatabaseAdoptionError(RuntimeError):
+2 -1
View File
@@ -131,6 +131,7 @@ def cmd_read(args: argparse.Namespace) -> None:
print(f"Profile : {profile.name}{profile.description}")
print(f"Gateway : {host}:{port} unit_id={unit_id}")
print(f"Function : FC{profile.function_code:02d}")
print(f"Blocks : {[(b.start, b.count) for b in profile.blocks]}")
print()
@@ -141,7 +142,7 @@ def cmd_read(args: argparse.Namespace) -> None:
from app.integrations.modbus.driver import read_blocks
try:
registers = read_blocks(host, port, unit_id, blocks)
registers = read_blocks(host, port, unit_id, blocks, function_code=profile.function_code)
except ModbusConnectionError as exc:
print(f"Connection error: {exc}", file=sys.stderr)
sys.exit(1)
+276
View File
@@ -187,6 +187,28 @@ def test_put_config_with_csrf_header_updates_app_name(
assert app_name_field["value"] == "Updated via API"
def test_put_config_reapplies_dsmr_subscription(
client: TestClient, test_database_urls
) -> None:
"""Saving config must re-apply the DSMR subscription so enabling DSMR ingest
takes effect without an app restart (the route calls apply_dsmr_subscription
with the refreshed settings)."""
_login(client)
payload = _full_config_payload({"DSMR_INGEST_ENABLED": "true"})
with patch("app.services.dsmr_ingest.apply_dsmr_subscription") as spy:
response = client.put(
"/api/config",
json={"updates": payload},
headers={"X-CSRF-Token": "any-non-empty-value"},
)
assert response.status_code == 200
spy.assert_called_once()
applied_settings = spy.call_args.args[0]
assert applied_settings.dsmr_ingest_enabled is True
def test_put_config_blank_secret_keeps_existing_value(
client: TestClient, test_database_urls
) -> None:
@@ -572,6 +594,7 @@ EXPECTED_CHECKBOX_FIELDS = {
"MQTT_TLS_ENABLED",
"HA_DISCOVERY_ENABLED",
"MODBUS_POLLING_ENABLED",
"DSMR_INGEST_ENABLED",
}
@@ -703,6 +726,259 @@ def test_put_config_mqtt_reconnect_uses_db_merged_settings(
)
# ---------------------------------------------------------------------------
# M6-T02: DSMR + Tibber CONFIG_FIELDS
# ---------------------------------------------------------------------------
def test_get_config_includes_dsmr_section(client: TestClient) -> None:
"""GET /api/config must include a DSMR section with expected fields including DSMR_TARIFF_TOPIC."""
_login(client)
response = client.get("/api/config")
body = response.json()
section_names = {s["name"] for s in body["sections"]}
assert "DSMR" in section_names, f"DSMR section missing; got {section_names}"
dsmr_section = next(s for s in body["sections"] if s["name"] == "DSMR")
env_names = {f["env_name"] for f in dsmr_section["fields"]}
assert "DSMR_INGEST_ENABLED" in env_names
assert "DSMR_MQTT_TOPIC" in env_names
assert "DSMR_SAMPLE_INTERVAL_S" in env_names
assert "DSMR_TARIFF_TOPIC" in env_names, (
f"DSMR_TARIFF_TOPIC must be present in DSMR section; got {env_names}"
)
def test_get_config_includes_tibber_section(client: TestClient) -> None:
"""GET /api/config must include a Tibber section with expected fields."""
_login(client)
response = client.get("/api/config")
body = response.json()
section_names = {s["name"] for s in body["sections"]}
assert "Tibber" in section_names, f"Tibber section missing; got {section_names}"
tibber_section = next(s for s in body["sections"] if s["name"] == "Tibber")
env_names = {f["env_name"] for f in tibber_section["fields"]}
assert "TIBBER_API_TOKEN" in env_names
assert "TIBBER_HOME_ID" in env_names
def test_get_config_tibber_api_token_is_secret(client: TestClient) -> None:
"""TIBBER_API_TOKEN must be marked secret and return empty string in GET response."""
_login(client)
response = client.get("/api/config")
body = response.json()
tibber_section = next(s for s in body["sections"] if s["name"] == "Tibber")
token_field = next(f for f in tibber_section["fields"] if f["env_name"] == "TIBBER_API_TOKEN")
assert token_field["secret"] is True
assert token_field["value"] == "", (
f"TIBBER_API_TOKEN should be masked (empty string), got {token_field['value']!r}"
)
def test_get_config_dsmr_ingest_enabled_is_checkbox(client: TestClient) -> None:
"""DSMR_INGEST_ENABLED must have input_type='checkbox' for correct frontend rendering."""
_login(client)
response = client.get("/api/config")
body = response.json()
dsmr_section = next(s for s in body["sections"] if s["name"] == "DSMR")
enabled_field = next(f for f in dsmr_section["fields"] if f["env_name"] == "DSMR_INGEST_ENABLED")
assert enabled_field["input_type"] == "checkbox", (
f"DSMR_INGEST_ENABLED input_type should be 'checkbox', got {enabled_field['input_type']!r}"
)
def test_get_config_dsmr_sample_interval_input_type_is_number(client: TestClient) -> None:
"""DSMR_SAMPLE_INTERVAL_S must have input_type='number'."""
_login(client)
response = client.get("/api/config")
body = response.json()
dsmr_section = next(s for s in body["sections"] if s["name"] == "DSMR")
interval_field = next(
f for f in dsmr_section["fields"] if f["env_name"] == "DSMR_SAMPLE_INTERVAL_S"
)
assert interval_field["input_type"] == "number", (
f"DSMR_SAMPLE_INTERVAL_S input_type should be 'number', got {interval_field['input_type']!r}"
)
def test_put_config_blank_tibber_api_token_keeps_existing(
client: TestClient, test_database_urls
) -> None:
"""Submitting blank TIBBER_API_TOKEN must NOT overwrite the stored secret."""
_login(client)
# Store a secret via full PUT
payload_with_secret = _full_config_payload({"TIBBER_API_TOKEN": "tibber-secret-token"})
resp1 = client.put(
"/api/config",
json={"updates": payload_with_secret},
headers={"X-CSRF-Token": "token"},
)
assert resp1.status_code == 200, f"First PUT failed: {resp1.status_code} {resp1.text}"
# PUT with blank for that secret (keep-old semantics)
payload_blank_secret = _full_config_payload({"TIBBER_API_TOKEN": ""})
resp2 = client.put(
"/api/config",
json={"updates": payload_blank_secret},
headers={"X-CSRF-Token": "token"},
)
assert resp2.status_code == 200, f"Second PUT failed: {resp2.status_code} {resp2.text}"
# The stored value in the DB should still be the original secret
conn = sqlite3.connect(test_database_urls["app_path"])
try:
rows = dict(conn.execute("SELECT key, value FROM app_config").fetchall())
finally:
conn.close()
assert rows.get("TIBBER_API_TOKEN") == "tibber-secret-token"
def test_put_config_new_tibber_api_token_overwrites_existing(
client: TestClient, test_database_urls
) -> None:
"""Submitting a non-blank TIBBER_API_TOKEN must overwrite the stored secret."""
_login(client)
# Store initial secret
payload_initial = _full_config_payload({"TIBBER_API_TOKEN": "old-tibber-token"})
resp1 = client.put(
"/api/config",
json={"updates": payload_initial},
headers={"X-CSRF-Token": "token"},
)
assert resp1.status_code == 200
# Overwrite with a new non-blank secret
payload_new = _full_config_payload({"TIBBER_API_TOKEN": "new-tibber-token"})
resp2 = client.put(
"/api/config",
json={"updates": payload_new},
headers={"X-CSRF-Token": "token"},
)
assert resp2.status_code == 200
conn = sqlite3.connect(test_database_urls["app_path"])
try:
rows = dict(conn.execute("SELECT key, value FROM app_config").fetchall())
finally:
conn.close()
assert rows.get("TIBBER_API_TOKEN") == "new-tibber-token"
def test_put_config_invalid_dsmr_sample_interval_returns_422_and_does_not_write(
client: TestClient, test_database_urls
) -> None:
"""Non-integer DSMR_SAMPLE_INTERVAL_S must return 422 and not persist the bad value."""
_login(client)
payload = _full_config_payload({"DSMR_SAMPLE_INTERVAL_S": "not-a-number"})
response = client.put(
"/api/config",
json={"updates": payload},
headers={"X-CSRF-Token": "token"},
)
assert response.status_code == 422
conn = sqlite3.connect(test_database_urls["app_path"])
try:
rows = dict(conn.execute("SELECT key, value FROM app_config").fetchall())
finally:
conn.close()
assert rows.get("DSMR_SAMPLE_INTERVAL_S") != "not-a-number"
def test_put_config_dsmr_tariff_topic_persists_and_reflects_in_get(
client: TestClient, test_database_urls
) -> None:
"""DSMR_TARIFF_TOPIC must persist via PUT and be readable via GET /api/config."""
_login(client)
new_topic = "meter/tariff/slot"
payload = _full_config_payload({"DSMR_TARIFF_TOPIC": new_topic})
with patch("app.services.dsmr_ingest.apply_dsmr_subscription"):
response = client.put(
"/api/config",
json={"updates": payload},
headers={"X-CSRF-Token": "token"},
)
assert response.status_code == 200
# The updated value must appear in the GET response.
get_resp = client.get("/api/config")
body = get_resp.json()
dsmr_section = next(s for s in body["sections"] if s["name"] == "DSMR")
tariff_field = next(f for f in dsmr_section["fields"] if f["env_name"] == "DSMR_TARIFF_TOPIC")
assert tariff_field["value"] == new_topic, (
f"Expected DSMR_TARIFF_TOPIC to be {new_topic!r}, got {tariff_field['value']!r}"
)
def test_put_config_dsmr_tariff_topic_in_settings_payload(client: TestClient) -> None:
"""DSMR_TARIFF_TOPIC must appear in _settings_payload (GET /api/config returns it)."""
_login(client)
# The default value from Settings must appear in the DSMR section.
response = client.get("/api/config")
body = response.json()
dsmr_section = next(s for s in body["sections"] if s["name"] == "DSMR")
tariff_field = next(
(f for f in dsmr_section["fields"] if f["env_name"] == "DSMR_TARIFF_TOPIC"), None
)
assert tariff_field is not None, "DSMR_TARIFF_TOPIC must appear in DSMR config section"
# Default value should be the DSMR reader meter-stats topic.
assert tariff_field["value"] == "dsmr/meter-stats/electricity_tariff", (
f"Unexpected default for DSMR_TARIFF_TOPIC: {tariff_field['value']!r}"
)
def test_get_config_tibber_api_token_value_masked_after_save(
client: TestClient, test_database_urls
) -> None:
"""After saving a TIBBER_API_TOKEN, the value must remain masked (empty string) in GET response."""
_login(client)
# Save a non-blank token
payload = _full_config_payload({"TIBBER_API_TOKEN": "some-tibber-token"})
put_resp = client.put(
"/api/config",
json={"updates": payload},
headers={"X-CSRF-Token": "token"},
)
assert put_resp.status_code == 200
# After saving: value must still be masked (empty string), configured must be True
resp_after = client.get("/api/config")
body_after = resp_after.json()
tibber_section_after = next(s for s in body_after["sections"] if s["name"] == "Tibber")
token_field_after = next(
f for f in tibber_section_after["fields"] if f["env_name"] == "TIBBER_API_TOKEN"
)
assert token_field_after["configured"] is True
assert token_field_after["value"] == "", "Secret value must remain masked after save"
# The actual token must not appear anywhere in the response body
assert "some-tibber-token" not in resp_after.text
def test_post_mqtt_test_uses_db_broker_host(
client: TestClient, test_database_urls
) -> None:
+855
View File
@@ -0,0 +1,855 @@
"""Tests for M6-T09: Energy data API.
Coverage
--------
GET /api/energy/prices
- unauthenticated 401
- no active contract 200, kind=null, points=[], tariff=null
- tibber active contract 200, points from tibber_price, tariff=null
- manual active contract 200, points=[], tariff with effective prices
- limit parameter caps tibber points
GET /api/energy/costs
- unauthenticated 401
- empty DB 200, items=[]
- returns rows ordered by period_start ascending
- limit parameter caps results
GET /api/energy/costs/summary
- unauthenticated 401
- empty DB 200, all-zeros (no active contract, no periods)
- returns summarize() result structure
GET /api/energy/dsmr/latest
- unauthenticated 401
- no data 200, found=false
- with data 200, found=true, correct payload
POST /api/energy/costs/recompute
- unauthenticated 401
- missing CSRF 403
- end start 422
- window > 366 days 422
- valid call 200, recomputed=int (idempotent)
POST /api/energy/tibber/test
- unauthenticated 401
- missing CSRF 403
- token empty 400 config-error
- mock client success 200 success with price data
- mock client TibberAuthError 502 failed
- mock client TibberError 502 failed
- token never appears in success/failed responses
Auth/CSRF matrix: all write endpoints need session + CSRF.
"""
from __future__ import annotations
from datetime import UTC, datetime, timedelta
from typing import Any
from unittest.mock import patch
import pytest
from fastapi.testclient import TestClient
from sqlalchemy import create_engine
from sqlalchemy.orm import Session
from app.integrations.tibber.client import PricePoint, TibberAuthError, TibberError
from app.models.energy import (
DsmrReading,
EnergyCostPeriod,
EnergyContract,
EnergyContractVersion,
TibberPrice,
)
# ---------------------------------------------------------------------------
# Shared helpers
# ---------------------------------------------------------------------------
_CSRF = "test-csrf-token"
_MANUAL_VALUES: dict[str, Any] = {
"energy": {
"buy": {"normal": 0.40, "dal": 0.30},
"sell": {"normal": 0.10, "dal": 0.10},
"energy_tax": 0.1108,
"ode": 0.0,
},
"standing": {
"network_fee": 25.0,
"management_fee": 9.87,
},
"credits": {
"heffingskorting": 600.0,
},
}
_TIBBER_VALUES: dict[str, Any] = {
"energy": {
"energy_tax": 0.1108,
"sell_adjust": 0.0,
},
"standing": {
"management_fee": 5.99,
"network_fee": 25.0,
},
"credits": {
"heffingskorting": 600.0,
},
}
def _login(client: TestClient) -> None:
resp = client.post(
"/api/auth/login",
json={"username": "admin", "password": "test-password"},
)
assert resp.status_code == 200, f"Login failed: {resp.status_code} {resp.text}"
def _set_app_config(engine, key: str, value: str) -> None:
"""Upsert a key/value pair into the app_config table.
Used by tibber test helpers to inject a non-empty TIBBER_API_TOKEN without
needing to patch get_app_settings (which is shared with auth infrastructure).
"""
from sqlalchemy.orm import Session as _Session
from app.models.config import AppConfigEntry
now = datetime.now(UTC)
with _Session(engine) as session:
existing = session.execute(
__import__("sqlalchemy").select(AppConfigEntry).where(AppConfigEntry.key == key)
).scalar_one_or_none()
if existing is not None:
existing.value = value
existing.updated_at = now
else:
session.add(AppConfigEntry(key=key, value=value, updated_at=now))
session.commit()
# ---------------------------------------------------------------------------
# Fixtures
# ---------------------------------------------------------------------------
@pytest.fixture()
def energy_client(auth_database):
"""TestClient + SQLAlchemy engine + FastAPI app for Energy API tests."""
from app.main import create_app
app_url = auth_database["app_url"]
engine = create_engine(app_url, connect_args={"check_same_thread": False})
fastapi_app = create_app()
with TestClient(fastapi_app) as test_client:
yield test_client, engine, fastapi_app
engine.dispose()
def _make_active_manual_contract(engine) -> tuple[EnergyContract, EnergyContractVersion]:
"""Insert an active manual EnergyContract with one version and return both."""
now = datetime.now(UTC)
with Session(engine) as session:
contract = EnergyContract(
name="Test Manual",
kind="manual",
active=True,
currency="EUR",
created_at=now,
updated_at=now,
)
session.add(contract)
session.flush()
version = EnergyContractVersion(
contract_id=contract.id,
effective_from=now - timedelta(days=30),
effective_to=None,
values=_MANUAL_VALUES,
created_at=now,
)
session.add(version)
session.commit()
# capture ids before session closes
contract_id = contract.id
version_id = version.id
# Return plain data structs (not ORM objects tied to closed session)
class _ContractStub:
id = contract_id
class _VersionStub:
id = version_id
return _ContractStub(), _VersionStub() # type: ignore[return-value]
def _make_active_tibber_contract(engine) -> tuple[int, int]:
"""Insert an active tibber EnergyContract with one version and return (contract_id, version_id)."""
now = datetime.now(UTC)
with Session(engine) as session:
contract = EnergyContract(
name="Test Tibber",
kind="tibber",
active=True,
currency="EUR",
created_at=now,
updated_at=now,
)
session.add(contract)
session.flush()
version = EnergyContractVersion(
contract_id=contract.id,
effective_from=now - timedelta(days=30),
effective_to=None,
values=_TIBBER_VALUES,
created_at=now,
)
session.add(version)
session.commit()
return contract.id, version.id
def _make_tibber_prices(engine, count: int = 3) -> list[datetime]:
"""Insert ``count`` TibberPrice rows starting from one hour ago; return starts_at list."""
now = datetime.now(UTC).replace(second=0, microsecond=0)
base = now - timedelta(hours=1)
starts = [base + timedelta(minutes=15 * i) for i in range(count)]
with Session(engine) as session:
for s in starts:
row = TibberPrice(
starts_at=s,
resolution="QUARTER_HOURLY",
energy=0.18,
tax=0.065,
total=0.245,
level="NORMAL",
currency="EUR",
fetched_at=now,
)
session.add(row)
session.commit()
return starts
def _make_cost_periods(engine, count: int = 3, version_id: int | None = None) -> list[datetime]:
"""Insert ``count`` EnergyCostPeriod rows starting from one hour ago; return period_start list."""
now = datetime.now(UTC).replace(second=0, microsecond=0)
base = now - timedelta(hours=2)
starts = []
with Session(engine) as session:
for i in range(count):
t0 = base + timedelta(minutes=15 * i)
period = EnergyCostPeriod(
period_start=t0,
d1_kwh=0.1 * i,
d2_kwh=0.05 * i,
r1_kwh=0.0,
r2_kwh=0.0,
import_cost=0.05 * i,
export_revenue=0.0,
net_cost=0.05 * i,
currency="EUR",
pricing={"kind": "manual"},
contract_version_id=version_id,
degraded=False,
computed_at=now,
)
session.add(period)
starts.append(t0)
session.commit()
return starts
def _make_dsmr_reading(engine, recorded_at: datetime | None = None) -> datetime:
"""Insert one DsmrReading and return its recorded_at."""
now = recorded_at or datetime.now(UTC)
with Session(engine) as session:
row = DsmrReading(
recorded_at=now,
source_id=42,
payload={"electricity_delivered_1": "100.0", "test": True},
)
session.add(row)
session.commit()
return now
# ---------------------------------------------------------------------------
# GET /api/energy/prices — auth
# ---------------------------------------------------------------------------
def test_prices_unauthenticated_returns_401(energy_client):
client, _, _app = energy_client
resp = client.get("/api/energy/prices")
assert resp.status_code == 401
# ---------------------------------------------------------------------------
# GET /api/energy/prices — no active contract
# ---------------------------------------------------------------------------
def test_prices_no_contract_returns_empty(energy_client):
client, _, _app = energy_client
_login(client)
resp = client.get("/api/energy/prices")
assert resp.status_code == 200
body = resp.json()
assert body["kind"] is None
assert body["points"] == []
assert body["tariff"] is None
assert "currency" in body
# ---------------------------------------------------------------------------
# GET /api/energy/prices — manual contract
# ---------------------------------------------------------------------------
def test_prices_manual_contract_returns_tariff(energy_client):
client, engine, _app = energy_client
_login(client)
_make_active_manual_contract(engine)
resp = client.get("/api/energy/prices")
assert resp.status_code == 200
body = resp.json()
assert body["kind"] == "manual"
assert body["currency"] == "EUR"
assert body["points"] == []
assert body["tariff"] is not None
tariff = body["tariff"]
# buy_dal = 0.30 + 0.1108 + 0.0 = 0.4108
assert abs(tariff["buy_dal"] - 0.4108) < 1e-6, f"buy_dal wrong: {tariff['buy_dal']}"
# buy_normal = 0.40 + 0.1108 + 0.0 = 0.5108
assert abs(tariff["buy_normal"] - 0.5108) < 1e-6, f"buy_normal wrong: {tariff['buy_normal']}"
# sell_dal = 0.10 (no tax added)
assert abs(tariff["sell_dal"] - 0.10) < 1e-6
# sell_normal = 0.10
assert abs(tariff["sell_normal"] - 0.10) < 1e-6
# ---------------------------------------------------------------------------
# GET /api/energy/prices — tibber contract
# ---------------------------------------------------------------------------
def test_prices_tibber_contract_returns_points(energy_client):
client, engine, _app = energy_client
_login(client)
_make_active_tibber_contract(engine)
_make_tibber_prices(engine, count=3)
# Use a wide enough window to include our test prices.
start = (datetime.now(UTC) - timedelta(hours=2)).isoformat()
end = (datetime.now(UTC) + timedelta(hours=2)).isoformat()
resp = client.get("/api/energy/prices", params={"start": start, "end": end})
assert resp.status_code == 200
body = resp.json()
assert body["kind"] == "tibber"
assert body["tariff"] is None
assert isinstance(body["points"], list)
assert len(body["points"]) == 3
# Verify ascending order.
starts_at_list = [p["starts_at"] for p in body["points"]]
assert starts_at_list == sorted(starts_at_list)
# Check buy/sell calculations: buy=total=0.245, sell=total-energy_tax-sell_adjust=0.245-0.1108-0.0
for p in body["points"]:
assert abs(p["buy"] - 0.245) < 1e-6
assert abs(p["sell"] - (0.245 - 0.1108)) < 1e-4
assert p["level"] == "NORMAL"
def test_prices_tibber_limit_caps_results(energy_client):
client, engine, _app = energy_client
_login(client)
_make_active_tibber_contract(engine)
_make_tibber_prices(engine, count=5)
start = (datetime.now(UTC) - timedelta(hours=3)).isoformat()
end = (datetime.now(UTC) + timedelta(hours=2)).isoformat()
resp = client.get(
"/api/energy/prices", params={"start": start, "end": end, "limit": 2}
)
assert resp.status_code == 200
body = resp.json()
assert len(body["points"]) <= 2
# ---------------------------------------------------------------------------
# GET /api/energy/costs — auth
# ---------------------------------------------------------------------------
def test_costs_unauthenticated_returns_401(energy_client):
client, _, _app = energy_client
resp = client.get("/api/energy/costs")
assert resp.status_code == 401
# ---------------------------------------------------------------------------
# GET /api/energy/costs — empty
# ---------------------------------------------------------------------------
def test_costs_empty_returns_empty(energy_client):
client, _, _app = energy_client
_login(client)
resp = client.get("/api/energy/costs")
assert resp.status_code == 200
body = resp.json()
assert body["items"] == []
assert body["total"] == 0
# ---------------------------------------------------------------------------
# GET /api/energy/costs — with data
# ---------------------------------------------------------------------------
def test_costs_returns_sorted_ascending(energy_client):
client, engine, _app = energy_client
_login(client)
_make_active_manual_contract(engine)
_make_cost_periods(engine, count=3)
resp = client.get("/api/energy/costs")
assert resp.status_code == 200
body = resp.json()
assert body["total"] == 3
period_starts = [item["period_start"] for item in body["items"]]
assert period_starts == sorted(period_starts), "Items must be ascending by period_start"
def test_costs_limit_caps_results(energy_client):
client, engine, _app = energy_client
_login(client)
_make_active_manual_contract(engine)
_make_cost_periods(engine, count=5)
resp = client.get("/api/energy/costs?limit=2")
assert resp.status_code == 200
body = resp.json()
assert body["total"] <= 2
assert len(body["items"]) <= 2
def test_costs_schema_fields_present(energy_client):
client, engine, _app = energy_client
_login(client)
_make_active_manual_contract(engine)
_make_cost_periods(engine, count=1)
resp = client.get("/api/energy/costs")
assert resp.status_code == 200
item = resp.json()["items"][0]
for field in (
"period_start",
"d1_kwh", "d2_kwh", "r1_kwh", "r2_kwh",
"import_cost", "export_revenue", "net_cost",
"currency", "degraded",
):
assert field in item, f"Missing field: {field}"
# ---------------------------------------------------------------------------
# GET /api/energy/costs/summary — auth
# ---------------------------------------------------------------------------
def test_summary_unauthenticated_returns_401(energy_client):
client, _, _app = energy_client
resp = client.get("/api/energy/costs/summary")
assert resp.status_code == 401
# ---------------------------------------------------------------------------
# GET /api/energy/costs/summary — empty
# ---------------------------------------------------------------------------
def test_summary_empty_returns_zeros(energy_client):
client, _, _app = energy_client
_login(client)
resp = client.get("/api/energy/costs/summary")
assert resp.status_code == 200
body = resp.json()
assert body["period_count"] == 0
assert body["metered_net"] == 0.0
assert "total_payable" in body
assert "currency" in body
# ---------------------------------------------------------------------------
# GET /api/energy/costs/summary — with data
# ---------------------------------------------------------------------------
def test_summary_returns_correct_structure(energy_client):
client, engine, _app = energy_client
_login(client)
_make_active_manual_contract(engine)
_make_cost_periods(engine, count=2)
start = (datetime.now(UTC) - timedelta(hours=3)).isoformat()
end = datetime.now(UTC).isoformat()
resp = client.get(
"/api/energy/costs/summary", params={"start": start, "end": end}
)
assert resp.status_code == 200
body = resp.json()
for field in (
"currency",
"metered_import",
"metered_export",
"metered_net",
"fixed_costs",
"credits",
"total_payable",
"period_count",
"degraded_count",
"days",
):
assert field in body, f"Missing field in summary: {field}"
# ---------------------------------------------------------------------------
# GET /api/energy/dsmr/latest — auth
# ---------------------------------------------------------------------------
def test_dsmr_latest_unauthenticated_returns_401(energy_client):
client, _, _app = energy_client
resp = client.get("/api/energy/dsmr/latest")
assert resp.status_code == 401
# ---------------------------------------------------------------------------
# GET /api/energy/dsmr/latest — no data
# ---------------------------------------------------------------------------
def test_dsmr_latest_no_data_returns_not_found(energy_client):
client, _, _app = energy_client
_login(client)
resp = client.get("/api/energy/dsmr/latest")
assert resp.status_code == 200
body = resp.json()
assert body["found"] is False
assert body["recorded_at"] is None
assert body["payload"] is None
# ---------------------------------------------------------------------------
# GET /api/energy/dsmr/latest — with data
# ---------------------------------------------------------------------------
def test_dsmr_latest_returns_most_recent(energy_client):
client, engine, _app = energy_client
_login(client)
# Insert two readings; the more recent one should be returned.
older = datetime.now(UTC) - timedelta(minutes=5)
newer = datetime.now(UTC) - timedelta(seconds=30)
with Session(engine) as session:
session.add(DsmrReading(recorded_at=older, source_id=1, payload={"order": "first"}))
session.add(DsmrReading(recorded_at=newer, source_id=2, payload={"order": "second"}))
session.commit()
resp = client.get("/api/energy/dsmr/latest")
assert resp.status_code == 200
body = resp.json()
assert body["found"] is True
assert body["payload"]["order"] == "second"
assert body["recorded_at"] is not None
# ---------------------------------------------------------------------------
# POST /api/energy/costs/recompute — auth / CSRF
# ---------------------------------------------------------------------------
def test_recompute_unauthenticated_returns_401(energy_client):
client, _, _app = energy_client
now = datetime.now(UTC)
resp = client.post(
"/api/energy/costs/recompute",
params={"start": now.isoformat(), "end": (now + timedelta(hours=1)).isoformat()},
)
assert resp.status_code == 401
def test_recompute_missing_csrf_returns_403(energy_client):
client, _, _app = energy_client
_login(client)
now = datetime.now(UTC)
resp = client.post(
"/api/energy/costs/recompute",
params={"start": now.isoformat(), "end": (now + timedelta(hours=1)).isoformat()},
)
assert resp.status_code == 403
# ---------------------------------------------------------------------------
# POST /api/energy/costs/recompute — validation
# ---------------------------------------------------------------------------
def test_recompute_end_before_start_returns_422(energy_client):
client, _, _app = energy_client
_login(client)
now = datetime.now(UTC)
resp = client.post(
"/api/energy/costs/recompute",
params={
"start": now.isoformat(),
"end": (now - timedelta(hours=1)).isoformat(),
},
headers={"X-CSRF-Token": _CSRF},
)
assert resp.status_code == 422
def test_recompute_window_too_large_returns_422(energy_client):
client, _, _app = energy_client
_login(client)
now = datetime.now(UTC)
resp = client.post(
"/api/energy/costs/recompute",
params={
"start": (now - timedelta(days=400)).isoformat(),
"end": now.isoformat(),
},
headers={"X-CSRF-Token": _CSRF},
)
assert resp.status_code == 422
# ---------------------------------------------------------------------------
# POST /api/energy/costs/recompute — success (idempotent)
# ---------------------------------------------------------------------------
def test_recompute_returns_count(energy_client):
client, _, _app = energy_client
_login(client)
# A window with no data is fine — recompute returns 0 but is not an error.
now = datetime.now(UTC)
params = {
"start": (now - timedelta(hours=1)).isoformat(),
"end": now.isoformat(),
}
resp = client.post(
"/api/energy/costs/recompute",
params=params,
headers={"X-CSRF-Token": _CSRF},
)
assert resp.status_code == 200
body = resp.json()
assert "recomputed" in body
assert isinstance(body["recomputed"], int)
def test_recompute_is_idempotent(energy_client):
"""Calling recompute twice with the same window must produce the same result."""
client, _, _app = energy_client
_login(client)
now = datetime.now(UTC)
params = {
"start": (now - timedelta(hours=1)).isoformat(),
"end": now.isoformat(),
}
headers = {"X-CSRF-Token": _CSRF}
resp1 = client.post("/api/energy/costs/recompute", params=params, headers=headers)
resp2 = client.post("/api/energy/costs/recompute", params=params, headers=headers)
assert resp1.status_code == 200
assert resp2.status_code == 200
assert resp1.json()["recomputed"] == resp2.json()["recomputed"]
# ---------------------------------------------------------------------------
# POST /api/energy/tibber/test — auth / CSRF
# ---------------------------------------------------------------------------
def test_tibber_test_unauthenticated_returns_401(energy_client):
client, _, _app = energy_client
resp = client.post("/api/energy/tibber/test")
assert resp.status_code == 401
def test_tibber_test_missing_csrf_returns_403(energy_client):
client, _, _app = energy_client
_login(client)
resp = client.post("/api/energy/tibber/test")
assert resp.status_code == 403
# ---------------------------------------------------------------------------
# POST /api/energy/tibber/test — token empty → 400 config-error
# ---------------------------------------------------------------------------
def test_tibber_test_no_token_returns_config_error(energy_client):
"""When tibber_api_token is empty, expect 400 config-error regardless of CSRF."""
client, _, _app = energy_client
_login(client)
resp = client.post(
"/api/energy/tibber/test",
headers={"X-CSRF-Token": _CSRF},
)
# Token is empty by default (no config in test DB).
assert resp.status_code == 400
body = resp.json()
assert body["result"] == "config-error"
assert "message" in body
# ---------------------------------------------------------------------------
# POST /api/energy/tibber/test — mock success
# ---------------------------------------------------------------------------
def test_tibber_test_success(energy_client):
"""Token stored in DB app_config + mocked HTTP → 200 success with price data."""
client, engine, _app = energy_client
_login(client)
# Set TIBBER_API_TOKEN in the app_config table so get_app_settings returns it.
_set_app_config(engine, "TIBBER_API_TOKEN", "fake-tibber-token")
mock_price = PricePoint(
starts_at=datetime(2026, 6, 23, 14, 0, 0, tzinfo=UTC),
total=0.245,
energy=0.18,
tax=0.065,
currency="EUR",
level="NORMAL",
resolution="QUARTER_HOURLY",
)
with patch(
"app.api.routes.api.energy.fetch_current_price",
return_value=mock_price,
):
resp = client.post(
"/api/energy/tibber/test",
headers={"X-CSRF-Token": _CSRF},
)
assert resp.status_code == 200
body = resp.json()
assert body["result"] == "success"
assert body["price"] is not None
assert body["price"]["total"] == 0.245
assert body["price"]["currency"] == "EUR"
assert body["price"]["level"] == "NORMAL"
# Token must NOT appear anywhere in the response.
assert "fake-tibber-token" not in str(body)
# ---------------------------------------------------------------------------
# POST /api/energy/tibber/test — mock TibberAuthError → 502
# ---------------------------------------------------------------------------
def test_tibber_test_auth_error_returns_502(energy_client):
"""Bad token stored in DB + mocked TibberAuthError → 502 failed."""
client, engine, _app = energy_client
_login(client)
_set_app_config(engine, "TIBBER_API_TOKEN", "bad-tibber-token")
with patch(
"app.api.routes.api.energy.fetch_current_price",
side_effect=TibberAuthError("auth failed"),
):
resp = client.post(
"/api/energy/tibber/test",
headers={"X-CSRF-Token": _CSRF},
)
assert resp.status_code == 502
body = resp.json()
assert body["result"] == "failed"
assert "bad-tibber-token" not in str(body)
# ---------------------------------------------------------------------------
# POST /api/energy/tibber/test — mock TibberError → 502
# ---------------------------------------------------------------------------
def test_tibber_test_network_error_returns_502(energy_client):
"""Token in DB + mocked TibberError (network timeout) → 502 failed."""
client, engine, _app = energy_client
_login(client)
_set_app_config(engine, "TIBBER_API_TOKEN", "some-tibber-token")
with patch(
"app.api.routes.api.energy.fetch_current_price",
side_effect=TibberError("network timeout"),
):
resp = client.post(
"/api/energy/tibber/test",
headers={"X-CSRF-Token": _CSRF},
)
assert resp.status_code == 502
body = resp.json()
assert body["result"] == "failed"
assert "some-tibber-token" not in str(body)
# ---------------------------------------------------------------------------
# POST /api/energy/tibber/test — token never in response (extra check)
# ---------------------------------------------------------------------------
def test_tibber_test_token_not_in_response(energy_client):
"""Even in the success response, the Tibber token must not appear."""
client, engine, _app = energy_client
_login(client)
secret_token = "super-secret-tibber-token-xyz"
_set_app_config(engine, "TIBBER_API_TOKEN", secret_token)
mock_price = PricePoint(
starts_at=datetime(2026, 6, 23, 14, 0, 0, tzinfo=UTC),
total=0.30,
energy=0.22,
tax=0.08,
currency="EUR",
level="CHEAP",
resolution="QUARTER_HOURLY",
)
with patch(
"app.api.routes.api.energy.fetch_current_price",
return_value=mock_price,
):
resp = client.post(
"/api/energy/tibber/test",
headers={"X-CSRF-Token": _CSRF},
)
assert resp.status_code == 200
assert secret_token not in resp.text
+750
View File
@@ -0,0 +1,750 @@
"""Tests for M6-T04: EnergyContract CRUD + versioning + pricing-profile API.
Coverage matrix
---------------
GET /api/energy/profiles
- unauthenticated 401
- authenticated 200, both 'manual' and 'tibber' profiles present
GET /api/energy/contracts
- unauthenticated 401
- authenticated empty 200, items=[]
- after creating two contracts items with active flag correct
POST /api/energy/contracts
- unauthenticated 401
- missing CSRF 403
- invalid values (missing required field) 422, DB unchanged
- unknown kind 422
- valid manual 201, ContractDetailResponse with versions
- valid tibber 201
GET /api/energy/contracts/{id}
- unauthenticated 401
- not found 404
- success versions ordered by effective_from
PATCH /api/energy/contracts/{id}
- unauthenticated 401
- missing CSRF 403
- not found 404
- rename name updated
- activate mutual exclusion (A active activate B A.active=False)
- deactivate active=False, others unchanged
POST /api/energy/contracts/{id}/versions
- unauthenticated 401
- missing CSRF 403
- not found 404
- invalid values 422, no rows written, old version unchanged
- effective_from previous 422
- success old version closed (effective_to set), new open version added,
old version values unchanged (only-append semantics)
Auth/CSRF matrix tested for all write endpoints.
"""
from __future__ import annotations
from datetime import UTC, datetime, timedelta
from typing import Any
import pytest
from fastapi.testclient import TestClient
from sqlalchemy import create_engine, select
from sqlalchemy.orm import Session
from app.models.energy import EnergyContract, EnergyContractVersion
# ---------------------------------------------------------------------------
# Shared helpers
# ---------------------------------------------------------------------------
_CSRF = "test-csrf-token"
def _login(client: TestClient) -> None:
resp = client.post(
"/api/auth/login",
json={"username": "admin", "password": "test-password"},
)
assert resp.status_code == 200, f"Login failed: {resp.status_code} {resp.text}"
# Minimal valid values for each kind.
_MANUAL_VALUES: dict[str, Any] = {
"energy": {
"buy": {"normal": 0.40, "dal": 0.30},
"sell": {"normal": 0.10, "dal": 0.10},
"energy_tax": 0.1108,
"ode": 0.0,
},
"standing": {
"network_fee": 25.0,
"management_fee": 9.87,
},
"credits": {
"heffingskorting": 600.0,
},
}
_TIBBER_VALUES: dict[str, Any] = {
"energy": {
"energy_tax": 0.1108,
"sell_adjust": 0.0,
},
"standing": {
"management_fee": 5.99,
"network_fee": 25.0,
},
"credits": {
"heffingskorting": 600.0,
},
}
def _manual_payload(**overrides) -> dict[str, Any]:
base: dict[str, Any] = {
"name": "Test Manual Contract",
"kind": "manual",
"currency": "EUR",
"values": _MANUAL_VALUES,
}
base.update(overrides)
return base
def _tibber_payload(**overrides) -> dict[str, Any]:
base: dict[str, Any] = {
"name": "Test Tibber Contract",
"kind": "tibber",
"currency": "EUR",
"values": _TIBBER_VALUES,
}
base.update(overrides)
return base
# ---------------------------------------------------------------------------
# Fixtures
# ---------------------------------------------------------------------------
@pytest.fixture()
def contracts_client(auth_database):
"""TestClient + SQLAlchemy engine for EnergyContract API tests."""
from app.main import create_app
app_url = auth_database["app_url"]
engine = create_engine(app_url, connect_args={"check_same_thread": False})
fastapi_app = create_app()
with TestClient(fastapi_app) as test_client:
yield test_client, engine
engine.dispose()
# ---------------------------------------------------------------------------
# GET /api/energy/profiles
# ---------------------------------------------------------------------------
def test_profiles_unauthenticated_returns_401(contracts_client):
client, _ = contracts_client
resp = client.get("/api/energy/profiles")
assert resp.status_code == 401
def test_profiles_returns_both_kinds(contracts_client):
client, _ = contracts_client
_login(client)
resp = client.get("/api/energy/profiles")
assert resp.status_code == 200
body = resp.json()
assert "profiles" in body
kinds = {p["kind"] for p in body["profiles"]}
assert "manual" in kinds
assert "tibber" in kinds
def test_profiles_contain_structure(contracts_client):
"""Each profile entry must have the nested structure the front-end needs."""
client, _ = contracts_client
_login(client)
resp = client.get("/api/energy/profiles")
body = resp.json()
for profile in body["profiles"]:
assert "kind" in profile
assert "label" in profile
assert "energy" in profile
assert "standing" in profile
assert "credits" in profile
# ---------------------------------------------------------------------------
# GET /api/energy/contracts
# ---------------------------------------------------------------------------
def test_list_contracts_unauthenticated_returns_401(contracts_client):
client, _ = contracts_client
resp = client.get("/api/energy/contracts")
assert resp.status_code == 401
def test_list_contracts_empty(contracts_client):
client, _ = contracts_client
_login(client)
resp = client.get("/api/energy/contracts")
assert resp.status_code == 200
body = resp.json()
assert body["items"] == []
assert body["total"] == 0
def test_list_contracts_shows_active_flag(contracts_client):
"""After creating two contracts and activating one, active flag is correct."""
client, _ = contracts_client
_login(client)
# Create contract A
resp_a = client.post(
"/api/energy/contracts",
json=_manual_payload(name="Contract A"),
headers={"X-CSRF-Token": _CSRF},
)
assert resp_a.status_code == 201
id_a = resp_a.json()["id"]
# Create contract B
resp_b = client.post(
"/api/energy/contracts",
json=_manual_payload(name="Contract B"),
headers={"X-CSRF-Token": _CSRF},
)
assert resp_b.status_code == 201
id_b = resp_b.json()["id"]
# Activate A
client.patch(
f"/api/energy/contracts/{id_a}",
json={"active": True},
headers={"X-CSRF-Token": _CSRF},
)
resp = client.get("/api/energy/contracts")
items = {c["id"]: c for c in resp.json()["items"]}
assert items[id_a]["active"] is True
assert items[id_b]["active"] is False
# ---------------------------------------------------------------------------
# POST /api/energy/contracts
# ---------------------------------------------------------------------------
def test_create_contract_unauthenticated_returns_401(contracts_client):
client, _ = contracts_client
resp = client.post(
"/api/energy/contracts",
json=_manual_payload(),
headers={"X-CSRF-Token": _CSRF},
)
assert resp.status_code == 401
def test_create_contract_missing_csrf_returns_403(contracts_client):
client, _ = contracts_client
_login(client)
resp = client.post("/api/energy/contracts", json=_manual_payload())
assert resp.status_code == 403
def test_create_contract_invalid_values_returns_422_no_db_write(contracts_client):
"""Non-conforming values must return 422 without writing any rows."""
client, engine = contracts_client
_login(client)
# Missing required field energy.buy.normal
bad_values = {
"energy": {
"buy": {"dal": 0.30}, # missing "normal"
"sell": {"normal": 0.10, "dal": 0.10},
"energy_tax": 0.1108,
"ode": 0.0,
},
"standing": {"network_fee": 25.0, "management_fee": 9.87},
"credits": {"heffingskorting": 600.0},
}
resp = client.post(
"/api/energy/contracts",
json=_manual_payload(values=bad_values),
headers={"X-CSRF-Token": _CSRF},
)
assert resp.status_code == 422
# Confirm no rows were written.
with Session(engine) as s:
count = s.execute(select(EnergyContract)).scalars().all()
assert len(count) == 0
def test_create_contract_unknown_kind_returns_422(contracts_client):
client, _ = contracts_client
_login(client)
resp = client.post(
"/api/energy/contracts",
json={
"name": "Bad Kind",
"kind": "nonexistent_kind_xyz",
"values": {},
},
headers={"X-CSRF-Token": _CSRF},
)
assert resp.status_code == 422
def test_create_manual_contract_success(contracts_client):
client, engine = contracts_client
_login(client)
resp = client.post(
"/api/energy/contracts",
json=_manual_payload(),
headers={"X-CSRF-Token": _CSRF},
)
assert resp.status_code == 201
body = resp.json()
assert body["name"] == "Test Manual Contract"
assert body["kind"] == "manual"
assert body["active"] is False # new contracts are inactive by default
assert "versions" in body
assert len(body["versions"]) == 1
v = body["versions"][0]
assert v["effective_to"] is None # open-ended first version
# DB check
with Session(engine) as s:
contracts = s.execute(select(EnergyContract)).scalars().all()
assert len(contracts) == 1
assert contracts[0].name == "Test Manual Contract"
versions = s.execute(select(EnergyContractVersion)).scalars().all()
assert len(versions) == 1
def test_create_tibber_contract_success(contracts_client):
client, _ = contracts_client
_login(client)
resp = client.post(
"/api/energy/contracts",
json=_tibber_payload(),
headers={"X-CSRF-Token": _CSRF},
)
assert resp.status_code == 201
body = resp.json()
assert body["kind"] == "tibber"
assert len(body["versions"]) == 1
def test_create_contract_defaults_effective_from(contracts_client):
"""When effective_from is omitted, the version is created with a recent timestamp.
SQLite stores datetimes as naive UTC strings; we compare both sides as naive UTC.
"""
client, _ = contracts_client
_login(client)
# Use naive UTC (no tzinfo) to match what SQLite gives back via Pydantic.
# Remove tzinfo from datetime.now(UTC) to get a comparable naive datetime.
before = datetime.now(UTC).replace(tzinfo=None)
resp = client.post(
"/api/energy/contracts",
json=_manual_payload(),
headers={"X-CSRF-Token": _CSRF},
)
assert resp.status_code == 201
v = resp.json()["versions"][0]
# Parse as naive (SQLite round-trip strips tz)
raw_eff = datetime.fromisoformat(v["effective_from"])
# Strip tzinfo from raw_eff if present (shouldn't be, but guard anyway)
eff_naive = raw_eff.replace(tzinfo=None)
assert eff_naive >= before
# ---------------------------------------------------------------------------
# GET /api/energy/contracts/{id}
# ---------------------------------------------------------------------------
def test_get_contract_unauthenticated_returns_401(contracts_client):
client, _ = contracts_client
resp = client.get("/api/energy/contracts/1")
assert resp.status_code == 401
def test_get_contract_not_found_returns_404(contracts_client):
client, _ = contracts_client
_login(client)
resp = client.get("/api/energy/contracts/99999")
assert resp.status_code == 404
def test_get_contract_returns_versions_ordered(contracts_client):
"""GET {id} must return versions ordered by effective_from."""
client, _ = contracts_client
_login(client)
t0 = datetime(2026, 1, 1, 0, 0, 0, tzinfo=UTC)
resp = client.post(
"/api/energy/contracts",
json=_manual_payload(effective_from=t0.isoformat()),
headers={"X-CSRF-Token": _CSRF},
)
assert resp.status_code == 201
contract_id = resp.json()["id"]
t1 = datetime(2026, 6, 1, 0, 0, 0, tzinfo=UTC)
new_values = dict(_MANUAL_VALUES)
new_values = {
**_MANUAL_VALUES,
"energy": {**_MANUAL_VALUES["energy"], "buy": {"normal": 0.50, "dal": 0.40}},
}
client.post(
f"/api/energy/contracts/{contract_id}/versions",
json={"effective_from": t1.isoformat(), "values": new_values},
headers={"X-CSRF-Token": _CSRF},
)
resp = client.get(f"/api/energy/contracts/{contract_id}")
assert resp.status_code == 200
versions = resp.json()["versions"]
assert len(versions) == 2
# Ordered by effective_from ascending
eff0 = datetime.fromisoformat(versions[0]["effective_from"])
eff1 = datetime.fromisoformat(versions[1]["effective_from"])
assert eff0 < eff1
# ---------------------------------------------------------------------------
# PATCH /api/energy/contracts/{id}
# ---------------------------------------------------------------------------
def test_patch_contract_unauthenticated_returns_401(contracts_client):
client, _ = contracts_client
resp = client.patch(
"/api/energy/contracts/1",
json={"name": "Renamed"},
headers={"X-CSRF-Token": _CSRF},
)
assert resp.status_code == 401
def test_patch_contract_missing_csrf_returns_403(contracts_client):
client, _ = contracts_client
_login(client)
# Create a contract first
resp = client.post(
"/api/energy/contracts",
json=_manual_payload(),
headers={"X-CSRF-Token": _CSRF},
)
contract_id = resp.json()["id"]
# PATCH without CSRF
resp = client.patch(f"/api/energy/contracts/{contract_id}", json={"name": "Renamed"})
assert resp.status_code == 403
def test_patch_contract_not_found_returns_404(contracts_client):
client, _ = contracts_client
_login(client)
resp = client.patch(
"/api/energy/contracts/99999",
json={"name": "Does Not Exist"},
headers={"X-CSRF-Token": _CSRF},
)
assert resp.status_code == 404
def test_patch_contract_rename(contracts_client):
client, _ = contracts_client
_login(client)
resp = client.post(
"/api/energy/contracts",
json=_manual_payload(name="Original Name"),
headers={"X-CSRF-Token": _CSRF},
)
contract_id = resp.json()["id"]
resp = client.patch(
f"/api/energy/contracts/{contract_id}",
json={"name": "Renamed Contract"},
headers={"X-CSRF-Token": _CSRF},
)
assert resp.status_code == 200
assert resp.json()["name"] == "Renamed Contract"
def test_activate_contract_mutual_exclusion(contracts_client):
"""Activating B deactivates A; at most one contract is active."""
client, engine = contracts_client
_login(client)
# Create A and B
resp_a = client.post(
"/api/energy/contracts",
json=_manual_payload(name="A"),
headers={"X-CSRF-Token": _CSRF},
)
id_a = resp_a.json()["id"]
resp_b = client.post(
"/api/energy/contracts",
json=_manual_payload(name="B"),
headers={"X-CSRF-Token": _CSRF},
)
id_b = resp_b.json()["id"]
# Activate A
resp = client.patch(
f"/api/energy/contracts/{id_a}",
json={"active": True},
headers={"X-CSRF-Token": _CSRF},
)
assert resp.status_code == 200
assert resp.json()["active"] is True
# Now activate B — A must become inactive
resp = client.patch(
f"/api/energy/contracts/{id_b}",
json={"active": True},
headers={"X-CSRF-Token": _CSRF},
)
assert resp.status_code == 200
assert resp.json()["active"] is True
# Verify A is now inactive via list
list_resp = client.get("/api/energy/contracts")
items = {c["id"]: c for c in list_resp.json()["items"]}
assert items[id_a]["active"] is False
assert items[id_b]["active"] is True
# DB check: only one active
with Session(engine) as s:
active_contracts = (
s.execute(select(EnergyContract).where(EnergyContract.active.is_(True)))
.scalars()
.all()
)
assert len(active_contracts) == 1
assert active_contracts[0].id == id_b
def test_deactivate_contract(contracts_client):
"""PATCH active=false deactivates the contract without touching others."""
client, _ = contracts_client
_login(client)
resp = client.post(
"/api/energy/contracts",
json=_manual_payload(),
headers={"X-CSRF-Token": _CSRF},
)
contract_id = resp.json()["id"]
# Activate it first
client.patch(
f"/api/energy/contracts/{contract_id}",
json={"active": True},
headers={"X-CSRF-Token": _CSRF},
)
# Now deactivate
resp = client.patch(
f"/api/energy/contracts/{contract_id}",
json={"active": False},
headers={"X-CSRF-Token": _CSRF},
)
assert resp.status_code == 200
assert resp.json()["active"] is False
# ---------------------------------------------------------------------------
# POST /api/energy/contracts/{id}/versions
# ---------------------------------------------------------------------------
def test_add_version_unauthenticated_returns_401(contracts_client):
client, _ = contracts_client
resp = client.post(
"/api/energy/contracts/1/versions",
json={"effective_from": datetime.now(UTC).isoformat(), "values": _MANUAL_VALUES},
headers={"X-CSRF-Token": _CSRF},
)
assert resp.status_code == 401
def test_add_version_missing_csrf_returns_403(contracts_client):
client, _ = contracts_client
_login(client)
resp_c = client.post(
"/api/energy/contracts",
json=_manual_payload(),
headers={"X-CSRF-Token": _CSRF},
)
cid = resp_c.json()["id"]
# POST version without CSRF
t1 = (datetime.now(UTC) + timedelta(days=1)).isoformat()
resp = client.post(
f"/api/energy/contracts/{cid}/versions",
json={"effective_from": t1, "values": _MANUAL_VALUES},
)
assert resp.status_code == 403
def test_add_version_not_found_returns_404(contracts_client):
client, _ = contracts_client
_login(client)
t1 = (datetime.now(UTC) + timedelta(days=1)).isoformat()
resp = client.post(
"/api/energy/contracts/99999/versions",
json={"effective_from": t1, "values": _MANUAL_VALUES},
headers={"X-CSRF-Token": _CSRF},
)
assert resp.status_code == 404
def test_add_version_invalid_values_returns_422_no_db_write(contracts_client):
"""Non-conforming values for a new version must return 422; DB unchanged."""
client, engine = contracts_client
_login(client)
t0 = datetime(2026, 1, 1, tzinfo=UTC)
resp_c = client.post(
"/api/energy/contracts",
json=_manual_payload(effective_from=t0.isoformat()),
headers={"X-CSRF-Token": _CSRF},
)
cid = resp_c.json()["id"]
bad_values = {"energy": {}} # missing almost everything
t1 = datetime(2026, 6, 1, tzinfo=UTC)
resp = client.post(
f"/api/energy/contracts/{cid}/versions",
json={"effective_from": t1.isoformat(), "values": bad_values},
headers={"X-CSRF-Token": _CSRF},
)
assert resp.status_code == 422
# Only the original version should exist in DB.
with Session(engine) as s:
versions = (
s.execute(
select(EnergyContractVersion).where(EnergyContractVersion.contract_id == cid)
)
.scalars()
.all()
)
assert len(versions) == 1
def test_add_version_effective_from_not_after_previous_returns_422(contracts_client):
"""effective_from ≤ previous open version's effective_from must return 422."""
client, _ = contracts_client
_login(client)
t0 = datetime(2026, 6, 1, tzinfo=UTC)
resp_c = client.post(
"/api/energy/contracts",
json=_manual_payload(effective_from=t0.isoformat()),
headers={"X-CSRF-Token": _CSRF},
)
cid = resp_c.json()["id"]
# Attempt to add a version with t1 == t0 (not strictly after)
resp = client.post(
f"/api/energy/contracts/{cid}/versions",
json={"effective_from": t0.isoformat(), "values": _MANUAL_VALUES},
headers={"X-CSRF-Token": _CSRF},
)
assert resp.status_code == 422
# Attempt to add a version with t1 < t0
t_before = datetime(2026, 1, 1, tzinfo=UTC)
resp = client.post(
f"/api/energy/contracts/{cid}/versions",
json={"effective_from": t_before.isoformat(), "values": _MANUAL_VALUES},
headers={"X-CSRF-Token": _CSRF},
)
assert resp.status_code == 422
def test_add_version_closes_previous_and_appends(contracts_client):
"""Adding a new version must close the previous one and create a new open version.
Verification:
- Old version's effective_to == new version's effective_from.
- Old version's values are unchanged (append-only; never overwritten).
- New version has effective_to=None (open-ended).
"""
client, engine = contracts_client
_login(client)
t0 = datetime(2026, 1, 1, 0, 0, 0, tzinfo=UTC)
old_values = _MANUAL_VALUES
resp_c = client.post(
"/api/energy/contracts",
json=_manual_payload(effective_from=t0.isoformat(), values=old_values),
headers={"X-CSRF-Token": _CSRF},
)
assert resp_c.status_code == 201
cid = resp_c.json()["id"]
v1_id = resp_c.json()["versions"][0]["id"]
t1 = datetime(2026, 6, 1, 0, 0, 0, tzinfo=UTC)
new_values = {
**old_values,
"energy": {**old_values["energy"], "buy": {"normal": 0.50, "dal": 0.40}},
}
resp = client.post(
f"/api/energy/contracts/{cid}/versions",
json={"effective_from": t1.isoformat(), "values": new_values},
headers={"X-CSRF-Token": _CSRF},
)
assert resp.status_code == 201
versions = resp.json()["versions"]
assert len(versions) == 2
# Ordered ascending — first is the old version
ver_old = versions[0]
ver_new = versions[1]
# Old version must be closed at t1.
# SQLite round-trips datetimes as naive UTC strings; compare without tzinfo.
assert ver_old["id"] == v1_id
assert ver_old["effective_to"] is not None
eff_to = datetime.fromisoformat(ver_old["effective_to"]).replace(tzinfo=None)
assert eff_to == t1.replace(tzinfo=None)
# Old version values must be unchanged
assert ver_old["values"]["energy"]["buy"]["normal"] == old_values["energy"]["buy"]["normal"]
# New version must be open-ended
assert ver_new["effective_to"] is None
assert ver_new["values"]["energy"]["buy"]["normal"] == 0.50
# Double-check via DB
with Session(engine) as s:
v1 = s.get(EnergyContractVersion, v1_id)
assert v1 is not None
assert v1.effective_to is not None
assert v1.values["energy"]["buy"]["normal"] == old_values["energy"]["buy"]["normal"]
all_versions = (
s.execute(
select(EnergyContractVersion).where(EnergyContractVersion.contract_id == cid)
)
.scalars()
.all()
)
assert len(all_versions) == 2
+755
View File
@@ -0,0 +1,755 @@
"""Tests for M7-T05: Meter CRUD + swap declaration + retroactive recompute API.
Coverage matrix
---------------
GET /api/energy/meters
- unauthenticated 401
- authenticated, no meters 200, items=[]
- after declaring meters items ordered by started_at asc
POST /api/energy/meters
- unauthenticated 401
- missing CSRF 403
- valid (first meter, no active) 201, MeterResponse
- valid swap (active meter exists) 201, old meter closed, new meter active
- retroactive started_at 201, triggers recompute for affected window
- started_at before active meter's started_at (overlap) → 422
PATCH /api/energy/meters/{id}
- unauthenticated 401
- missing CSRF 403
- not found 404
- rename label 200, label updated
- edit note 200, note updated
- correct started_at (retroactive) 200, triggers recompute for affected window
- started_at interval violation 422
Retroactive recompute integration
- PATCH started_at change triggers recompute over min(old, new)..now window
- POST retroactive declaration triggers recompute from new started_at
"""
from __future__ import annotations
from datetime import UTC, datetime
from unittest.mock import patch
import pytest
from fastapi.testclient import TestClient
from sqlalchemy import create_engine, select
from sqlalchemy.orm import Session
from app.models.energy import Meter
# ---------------------------------------------------------------------------
# Shared helpers
# ---------------------------------------------------------------------------
_CSRF = "test-csrf-token"
def _login(client: TestClient) -> None:
resp = client.post(
"/api/auth/login",
json={"username": "admin", "password": "test-password"},
)
assert resp.status_code == 200, f"Login failed: {resp.status_code} {resp.text}"
def _declare_payload(**overrides) -> dict:
base = {
"label": "Test Meter",
"started_at": datetime(2025, 1, 1, 0, 0, 0, tzinfo=UTC).isoformat(),
"reason": "initial",
"commodity": "electricity",
}
base.update(overrides)
return base
# ---------------------------------------------------------------------------
# Fixtures
# ---------------------------------------------------------------------------
@pytest.fixture(autouse=True)
def mock_publish_discovery():
"""Auto-mock publish_discovery for all tests in this module.
The meters API now calls _trigger_discovery_republish (best-effort) after
every successful write. publish_discovery is lazy-imported inside that
helper, so we patch it at its canonical source path
(app.services.ha_discovery.publish_discovery). Tests that need to assert
the call receive this fixture explicitly; all others benefit from the
isolation it provides (no live MQTT broker required).
"""
with patch(
"app.services.ha_discovery.publish_discovery",
return_value=None,
) as mock:
yield mock
@pytest.fixture()
def meters_client(auth_database):
"""TestClient + SQLAlchemy engine for Meter API tests."""
from app.main import create_app
app_url = auth_database["app_url"]
engine = create_engine(app_url, connect_args={"check_same_thread": False})
fastapi_app = create_app()
with TestClient(fastapi_app) as test_client:
yield test_client, engine
engine.dispose()
# ---------------------------------------------------------------------------
# GET /api/energy/meters
# ---------------------------------------------------------------------------
def test_list_meters_unauthenticated_returns_401(meters_client):
client, _ = meters_client
resp = client.get("/api/energy/meters")
assert resp.status_code == 401
def test_list_meters_empty(meters_client):
client, _ = meters_client
_login(client)
resp = client.get("/api/energy/meters")
assert resp.status_code == 200
body = resp.json()
assert body["items"] == []
assert body["total"] == 0
def test_list_meters_ordered_by_started_at(meters_client):
"""After declaring two meters, list returns them in ascending started_at order."""
client, _ = meters_client
_login(client)
t0 = datetime(2024, 6, 1, 0, 0, 0, tzinfo=UTC)
t1 = datetime(2025, 1, 1, 0, 0, 0, tzinfo=UTC)
with patch("app.api.routes.api.meters.recompute_range", return_value=0):
# Declare first meter (initial)
resp1 = client.post(
"/api/energy/meters",
json=_declare_payload(label="Meter A", started_at=t0.isoformat(), reason="initial"),
headers={"X-CSRF-Token": _CSRF},
)
assert resp1.status_code == 201
# Declare swap meter
resp2 = client.post(
"/api/energy/meters",
json=_declare_payload(label="Meter B", started_at=t1.isoformat(), reason="meter_swap"),
headers={"X-CSRF-Token": _CSRF},
)
assert resp2.status_code == 201
resp = client.get("/api/energy/meters")
assert resp.status_code == 200
body = resp.json()
assert body["total"] == 2
items = body["items"]
# Ordered by started_at ascending: Meter A first, Meter B second
assert items[0]["label"] == "Meter A"
assert items[1]["label"] == "Meter B"
# Meter B is active (ended_at is null)
assert items[1]["ended_at"] is None
# Meter A is closed
assert items[0]["ended_at"] is not None
# ---------------------------------------------------------------------------
# POST /api/energy/meters
# ---------------------------------------------------------------------------
def test_declare_meter_unauthenticated_returns_401(meters_client):
client, _ = meters_client
resp = client.post(
"/api/energy/meters",
json=_declare_payload(),
headers={"X-CSRF-Token": _CSRF},
)
assert resp.status_code == 401
def test_declare_meter_missing_csrf_returns_403(meters_client):
client, _ = meters_client
_login(client)
resp = client.post("/api/energy/meters", json=_declare_payload())
assert resp.status_code == 403
def test_declare_meter_first_no_active(meters_client):
"""Declaring the first meter succeeds without closing any previous meter."""
client, engine = meters_client
_login(client)
t0 = datetime(2025, 1, 1, 0, 0, 0, tzinfo=UTC)
with patch("app.api.routes.api.meters.recompute_range", return_value=0):
resp = client.post(
"/api/energy/meters",
json=_declare_payload(label="First Meter", started_at=t0.isoformat(), reason="initial"),
headers={"X-CSRF-Token": _CSRF},
)
assert resp.status_code == 201
body = resp.json()
assert body["label"] == "First Meter"
assert body["commodity"] == "electricity"
assert body["reason"] == "initial"
assert body["ended_at"] is None # active
# DB check: one meter, active
with Session(engine) as s:
meters = s.execute(select(Meter)).scalars().all()
assert len(meters) == 1
assert meters[0].ended_at is None
def test_declare_meter_swap_closes_previous(meters_client):
"""Declaring a swap closes the previous active meter at started_at."""
client, engine = meters_client
_login(client)
t0 = datetime(2024, 6, 1, 0, 0, 0, tzinfo=UTC)
t1 = datetime(2025, 3, 15, 12, 0, 0, tzinfo=UTC)
with patch("app.api.routes.api.meters.recompute_range", return_value=0):
# First meter
resp1 = client.post(
"/api/energy/meters",
json=_declare_payload(label="Old Meter", started_at=t0.isoformat(), reason="initial"),
headers={"X-CSRF-Token": _CSRF},
)
assert resp1.status_code == 201
old_id = resp1.json()["id"]
# Swap
resp2 = client.post(
"/api/energy/meters",
json=_declare_payload(label="New Meter", started_at=t1.isoformat(), reason="meter_swap"),
headers={"X-CSRF-Token": _CSRF},
)
assert resp2.status_code == 201
body2 = resp2.json()
assert body2["label"] == "New Meter"
assert body2["ended_at"] is None # new meter is active
# DB check: old meter is closed at t1
with Session(engine) as s:
old_meter = s.get(Meter, old_id)
assert old_meter is not None
assert old_meter.ended_at is not None
# ended_at should equal t1 (modulo naive/aware round-trip)
ended_naive = old_meter.ended_at
if ended_naive.tzinfo is None:
ended_naive = ended_naive.replace(tzinfo=UTC)
assert ended_naive == t1
def test_declare_meter_overlap_returns_422(meters_client):
"""Declaring a meter with started_at before active meter's started_at → 422."""
client, _ = meters_client
_login(client)
t0 = datetime(2025, 6, 1, 0, 0, 0, tzinfo=UTC)
t_before = datetime(2025, 1, 1, 0, 0, 0, tzinfo=UTC)
with patch("app.api.routes.api.meters.recompute_range", return_value=0):
# Declare first meter
resp = client.post(
"/api/energy/meters",
json=_declare_payload(started_at=t0.isoformat(), reason="initial"),
headers={"X-CSRF-Token": _CSRF},
)
assert resp.status_code == 201
# Attempt to declare a meter before t0 → overlap error
# (t_before is in the past so would trigger recompute, but service layer raises first)
resp2 = client.post(
"/api/energy/meters",
json=_declare_payload(
label="Backdated Meter", started_at=t_before.isoformat(), reason="meter_swap"
),
headers={"X-CSRF-Token": _CSRF},
)
assert resp2.status_code == 422
assert "started_at" in resp2.json()["detail"].lower()
def test_declare_meter_missing_fields_returns_422(meters_client):
"""Missing required fields (started_at, reason) → 422 from Pydantic validation."""
client, _ = meters_client
_login(client)
resp = client.post(
"/api/energy/meters",
json={"label": "No Reason Meter"}, # missing started_at and reason
headers={"X-CSRF-Token": _CSRF},
)
assert resp.status_code == 422
def test_declare_meter_response_fields(meters_client):
"""POST response contains all expected MeterResponse fields."""
client, _ = meters_client
_login(client)
t0 = datetime(2025, 1, 1, 0, 0, 0, tzinfo=UTC)
with patch("app.api.routes.api.meters.recompute_range", return_value=0):
resp = client.post(
"/api/energy/meters",
json=_declare_payload(
label="Full Fields Meter",
started_at=t0.isoformat(),
reason="home_move",
note="Testing note",
),
headers={"X-CSRF-Token": _CSRF},
)
assert resp.status_code == 201
body = resp.json()
for field in ("id", "label", "commodity", "started_at", "ended_at", "reason", "note", "created_at"):
assert field in body, f"Missing field {field!r} in response"
assert body["note"] == "Testing note"
assert body["reason"] == "home_move"
def test_declare_meter_retroactive_triggers_recompute(meters_client):
"""Declaring a meter with started_at in the past triggers recompute_range.
Both POST calls are made with recompute_range mocked so the test does not
spend time iterating over thousands of empty quarter-hour periods.
"""
client, _ = meters_client
_login(client)
t0 = datetime(2024, 1, 1, 0, 0, 0, tzinfo=UTC)
t_past = datetime(2025, 3, 1, 0, 0, 0, tzinfo=UTC)
with patch(
"app.api.routes.api.meters.recompute_range", return_value=5
) as mock_recompute:
# First meter (initial); also in the past, so recompute is called here too.
client.post(
"/api/energy/meters",
json=_declare_payload(started_at=t0.isoformat(), reason="initial"),
headers={"X-CSRF-Token": _CSRF},
)
# Reset call count before the swap we are actually testing.
mock_recompute.reset_mock()
# Retroactive swap: started_at in the past → should trigger recompute
resp = client.post(
"/api/energy/meters",
json=_declare_payload(
label="Retroactive Swap", started_at=t_past.isoformat(), reason="meter_swap"
),
headers={"X-CSRF-Token": _CSRF},
)
assert resp.status_code == 201
# recompute_range should have been called with start == t_past
assert mock_recompute.called
call_args = mock_recompute.call_args
recompute_start = call_args[0][1] # positional arg index 1 (session is 0)
# Normalise for comparison
if recompute_start.tzinfo is None:
recompute_start = recompute_start.replace(tzinfo=UTC)
assert recompute_start <= t_past
# ---------------------------------------------------------------------------
# PATCH /api/energy/meters/{id}
# ---------------------------------------------------------------------------
def test_patch_meter_unauthenticated_returns_401(meters_client):
client, _ = meters_client
resp = client.patch(
"/api/energy/meters/1",
json={"label": "Renamed"},
headers={"X-CSRF-Token": _CSRF},
)
assert resp.status_code == 401
def test_patch_meter_missing_csrf_returns_403(meters_client):
client, _ = meters_client
_login(client)
t0 = datetime(2025, 1, 1, 0, 0, 0, tzinfo=UTC)
with patch("app.api.routes.api.meters.recompute_range", return_value=0):
resp = client.post(
"/api/energy/meters",
json=_declare_payload(started_at=t0.isoformat()),
headers={"X-CSRF-Token": _CSRF},
)
meter_id = resp.json()["id"]
resp = client.patch(f"/api/energy/meters/{meter_id}", json={"label": "Renamed"})
assert resp.status_code == 403
def test_patch_meter_not_found_returns_404(meters_client):
client, _ = meters_client
_login(client)
resp = client.patch(
"/api/energy/meters/99999",
json={"label": "Does Not Exist"},
headers={"X-CSRF-Token": _CSRF},
)
assert resp.status_code == 404
def test_patch_meter_rename_label(meters_client):
"""PATCH label updates the meter's human-readable label."""
client, engine = meters_client
_login(client)
t0 = datetime(2025, 1, 1, 0, 0, 0, tzinfo=UTC)
with patch("app.api.routes.api.meters.recompute_range", return_value=0):
resp = client.post(
"/api/energy/meters",
json=_declare_payload(label="Original Label", started_at=t0.isoformat()),
headers={"X-CSRF-Token": _CSRF},
)
meter_id = resp.json()["id"]
resp = client.patch(
f"/api/energy/meters/{meter_id}",
json={"label": "Updated Label"},
headers={"X-CSRF-Token": _CSRF},
)
assert resp.status_code == 200
assert resp.json()["label"] == "Updated Label"
# DB check
with Session(engine) as s:
m = s.get(Meter, meter_id)
assert m.label == "Updated Label"
def test_patch_meter_edit_note(meters_client):
"""PATCH note updates the meter's note field."""
client, _ = meters_client
_login(client)
t0 = datetime(2025, 1, 1, 0, 0, 0, tzinfo=UTC)
with patch("app.api.routes.api.meters.recompute_range", return_value=0):
resp = client.post(
"/api/energy/meters",
json=_declare_payload(started_at=t0.isoformat(), note=None),
headers={"X-CSRF-Token": _CSRF},
)
meter_id = resp.json()["id"]
resp = client.patch(
f"/api/energy/meters/{meter_id}",
json={"note": "Added a note"},
headers={"X-CSRF-Token": _CSRF},
)
assert resp.status_code == 200
assert resp.json()["note"] == "Added a note"
def test_patch_meter_started_at_retroactive_triggers_recompute(meters_client):
"""PATCH started_at triggers recompute over min(old, new)..now window."""
client, _ = meters_client
_login(client)
# Set up two meters: initial + swap. All POST calls are mocked to avoid
# running recompute over thousands of empty historical periods.
t0 = datetime(2024, 1, 1, 0, 0, 0, tzinfo=UTC)
t1 = datetime(2025, 1, 1, 0, 0, 0, tzinfo=UTC)
with patch("app.api.routes.api.meters.recompute_range", return_value=0):
client.post(
"/api/energy/meters",
json=_declare_payload(label="Meter A", started_at=t0.isoformat(), reason="initial"),
headers={"X-CSRF-Token": _CSRF},
)
resp_b = client.post(
"/api/energy/meters",
json=_declare_payload(label="Meter B", started_at=t1.isoformat(), reason="meter_swap"),
headers={"X-CSRF-Token": _CSRF},
)
meter_b_id = resp_b.json()["id"]
# Correct Meter B's started_at to a slightly different past timestamp
t1_corrected = datetime(2024, 12, 15, 0, 0, 0, tzinfo=UTC) # earlier than t1
with patch(
"app.api.routes.api.meters.recompute_range", return_value=10
) as mock_recompute:
resp = client.patch(
f"/api/energy/meters/{meter_b_id}",
json={"started_at": t1_corrected.isoformat()},
headers={"X-CSRF-Token": _CSRF},
)
assert resp.status_code == 200
# recompute should be triggered
assert mock_recompute.called
call_args = mock_recompute.call_args
recompute_start = call_args[0][1]
if recompute_start.tzinfo is None:
recompute_start = recompute_start.replace(tzinfo=UTC)
# Window start should be min(t1_corrected, t1) = t1_corrected
assert recompute_start <= t1_corrected
def test_patch_meter_started_at_interval_violation_returns_422(meters_client):
"""PATCH started_at that would create an invalid interval → 422."""
client, _ = meters_client
_login(client)
# Set up: initial meter A, then swap to B. POST calls mocked to avoid slow recompute.
t0 = datetime(2024, 1, 1, 0, 0, 0, tzinfo=UTC)
t1 = datetime(2025, 1, 1, 0, 0, 0, tzinfo=UTC)
with patch("app.api.routes.api.meters.recompute_range", return_value=0):
resp_a = client.post(
"/api/energy/meters",
json=_declare_payload(label="Meter A", started_at=t0.isoformat(), reason="initial"),
headers={"X-CSRF-Token": _CSRF},
)
meter_a_id = resp_a.json()["id"]
client.post(
"/api/energy/meters",
json=_declare_payload(label="Meter B", started_at=t1.isoformat(), reason="meter_swap"),
headers={"X-CSRF-Token": _CSRF},
)
# Try to set Meter A's started_at to after its ended_at (t1) → interval error
t_too_late = datetime(2025, 6, 1, 0, 0, 0, tzinfo=UTC)
resp = client.patch(
f"/api/energy/meters/{meter_a_id}",
json={"started_at": t_too_late.isoformat()},
headers={"X-CSRF-Token": _CSRF},
)
assert resp.status_code == 422
def test_patch_meter_no_recompute_when_started_at_not_changed(meters_client):
"""PATCH that only changes label does NOT trigger recompute."""
client, _ = meters_client
_login(client)
t0 = datetime(2025, 1, 1, 0, 0, 0, tzinfo=UTC)
with patch("app.api.routes.api.meters.recompute_range", return_value=0) as mock_recompute:
resp = client.post(
"/api/energy/meters",
json=_declare_payload(started_at=t0.isoformat()),
headers={"X-CSRF-Token": _CSRF},
)
meter_id = resp.json()["id"]
mock_recompute.reset_mock()
resp = client.patch(
f"/api/energy/meters/{meter_id}",
json={"label": "Renamed Only"},
headers={"X-CSRF-Token": _CSRF},
)
assert resp.status_code == 200
# recompute should NOT be triggered (no started_at change)
assert not mock_recompute.called
# ---------------------------------------------------------------------------
# Timeline continuity (recompute mocked to avoid slow computation over empty quarters)
# ---------------------------------------------------------------------------
def test_swap_timeline_continuity(meters_client):
"""After two swaps, meter timeline is contiguous and self-consistent."""
client, engine = meters_client
_login(client)
t0 = datetime(2023, 1, 1, 0, 0, 0, tzinfo=UTC)
t1 = datetime(2024, 1, 1, 0, 0, 0, tzinfo=UTC)
t2 = datetime(2025, 1, 1, 0, 0, 0, tzinfo=UTC)
# Declare 3 meters in sequence. POST calls mocked to avoid slow recompute over
# years of empty quarter-hour periods.
with patch("app.api.routes.api.meters.recompute_range", return_value=0):
for label, ts, reason in [
("M1", t0, "initial"),
("M2", t1, "meter_swap"),
("M3", t2, "meter_swap"),
]:
resp = client.post(
"/api/energy/meters",
json=_declare_payload(label=label, started_at=ts.isoformat(), reason=reason),
headers={"X-CSRF-Token": _CSRF},
)
assert resp.status_code == 201
# Verify timeline via list endpoint
resp = client.get("/api/energy/meters")
items = resp.json()["items"]
assert len(items) == 3
# M1: [t0, t1); M2: [t1, t2); M3: [t2, None)
m1 = next(i for i in items if i["label"] == "M1")
m2 = next(i for i in items if i["label"] == "M2")
m3 = next(i for i in items if i["label"] == "M3")
assert m1["ended_at"] is not None
assert m2["ended_at"] is not None
assert m3["ended_at"] is None # active
# ended_at of M1 == started_at of M2 (contiguous)
m1_ended = datetime.fromisoformat(m1["ended_at"]).replace(tzinfo=None)
m2_started = datetime.fromisoformat(m2["started_at"]).replace(tzinfo=None)
assert m1_ended == m2_started
m2_ended = datetime.fromisoformat(m2["ended_at"]).replace(tzinfo=None)
m3_started = datetime.fromisoformat(m3["started_at"]).replace(tzinfo=None)
assert m2_ended == m3_started
# ---------------------------------------------------------------------------
# Reason enum validation
# ---------------------------------------------------------------------------
def test_declare_meter_invalid_reason_returns_422(meters_client):
"""Unknown reason value → 422 from Pydantic enum validation."""
client, _ = meters_client
_login(client)
t0 = datetime(2025, 1, 1, 0, 0, 0, tzinfo=UTC)
resp = client.post(
"/api/energy/meters",
json=_declare_payload(started_at=t0.isoformat(), reason="unknown_reason_xyz"),
headers={"X-CSRF-Token": _CSRF},
)
assert resp.status_code == 422
# ---------------------------------------------------------------------------
# Retroactive recompute & boundary update (recompute mocked) — window coverage check
# ---------------------------------------------------------------------------
def test_patch_started_at_earlier_updates_boundary(meters_client):
"""Moving started_at earlier should update the previous meter's ended_at."""
client, engine = meters_client
_login(client)
t0 = datetime(2024, 6, 1, 0, 0, 0, tzinfo=UTC)
t1 = datetime(2025, 1, 1, 0, 0, 0, tzinfo=UTC)
t1_earlier = datetime(2024, 12, 1, 0, 0, 0, tzinfo=UTC)
with patch("app.api.routes.api.meters.recompute_range", return_value=0):
resp_a = client.post(
"/api/energy/meters",
json=_declare_payload(label="Meter A", started_at=t0.isoformat(), reason="initial"),
headers={"X-CSRF-Token": _CSRF},
)
meter_a_id = resp_a.json()["id"]
resp_b = client.post(
"/api/energy/meters",
json=_declare_payload(label="Meter B", started_at=t1.isoformat(), reason="meter_swap"),
headers={"X-CSRF-Token": _CSRF},
)
meter_b_id = resp_b.json()["id"]
# Correct Meter B's started_at to t1_earlier (moves boundary earlier)
resp = client.patch(
f"/api/energy/meters/{meter_b_id}",
json={"started_at": t1_earlier.isoformat()},
headers={"X-CSRF-Token": _CSRF},
)
assert resp.status_code == 200
assert resp.json()["id"] == meter_b_id
# DB check: Meter A's ended_at should now equal t1_earlier
with Session(engine) as s:
meter_a = s.get(Meter, meter_a_id)
assert meter_a is not None
ended = meter_a.ended_at
if ended is not None and ended.tzinfo is None:
ended = ended.replace(tzinfo=UTC)
assert ended == t1_earlier
# ---------------------------------------------------------------------------
# FUE-T06: HA discovery re-publish triggered after meter writes
# ---------------------------------------------------------------------------
def test_declare_meter_triggers_publish_discovery(meters_client, mock_publish_discovery):
"""POST /api/energy/meters triggers publish_discovery after successful commit."""
client, _ = meters_client
_login(client)
t0 = datetime(2025, 1, 1, 0, 0, 0, tzinfo=UTC)
with patch("app.api.routes.api.meters.recompute_range", return_value=0):
resp = client.post(
"/api/energy/meters",
json=_declare_payload(label="Discovery Meter", started_at=t0.isoformat()),
headers={"X-CSRF-Token": _CSRF},
)
assert resp.status_code == 201
# publish_discovery must have been called exactly once after the declare.
mock_publish_discovery.assert_called_once()
def test_patch_meter_triggers_publish_discovery(meters_client, mock_publish_discovery):
"""PATCH /api/energy/meters/{id} triggers publish_discovery after successful commit."""
client, _ = meters_client
_login(client)
t0 = datetime(2025, 1, 1, 0, 0, 0, tzinfo=UTC)
with patch("app.api.routes.api.meters.recompute_range", return_value=0):
resp = client.post(
"/api/energy/meters",
json=_declare_payload(label="Original Label", started_at=t0.isoformat()),
headers={"X-CSRF-Token": _CSRF},
)
meter_id = resp.json()["id"]
# Reset call count: the POST above also called publish_discovery.
mock_publish_discovery.reset_mock()
resp = client.patch(
f"/api/energy/meters/{meter_id}",
json={"label": "Renamed Label"},
headers={"X-CSRF-Token": _CSRF},
)
assert resp.status_code == 200
# publish_discovery must have been called exactly once after the PATCH.
mock_publish_discovery.assert_called_once()
def test_declare_meter_succeeds_when_publish_discovery_raises(meters_client):
"""publish_discovery raising an exception must NOT cause POST declare to return 500.
The _trigger_discovery_republish helper is best-effort: it swallows all
exceptions so that a broken MQTT / discovery layer never breaks the API.
"""
client, _ = meters_client
_login(client)
t0 = datetime(2025, 1, 1, 0, 0, 0, tzinfo=UTC)
with (
patch("app.api.routes.api.meters.recompute_range", return_value=0),
patch(
"app.services.ha_discovery.publish_discovery",
side_effect=RuntimeError("MQTT broker unreachable"),
),
):
resp = client.post(
"/api/energy/meters",
json=_declare_payload(label="Best Effort Meter", started_at=t0.isoformat()),
headers={"X-CSRF-Token": _CSRF},
)
# The meter must be created successfully despite the discovery failure.
assert resp.status_code == 201
assert resp.json()["label"] == "Best Effort Meter"
+152
View File
@@ -432,6 +432,158 @@ def test_delete_device_not_found_returns_404(modbus_client):
assert resp.status_code == 404
# ---------------------------------------------------------------------------
# DELETE /api/modbus/devices/{uuid}?cascade=true
# ---------------------------------------------------------------------------
def test_cascade_false_with_readings_still_returns_409(modbus_client):
"""cascade=false (default) + readings → 409, device and readings remain."""
client, engine = modbus_client
_login(client)
device = _make_device(engine)
_make_reading(engine, device.id, datetime.now(UTC), {"voltage": 230.0})
resp = client.delete(
f"/api/modbus/devices/{device.uuid}",
params={"cascade": "false"},
headers={"X-CSRF-Token": _CSRF},
)
assert resp.status_code == 409
# Device and reading still in DB
with Session(engine) as session:
assert session.query(ModbusDevice).count() == 1
assert session.query(ModbusReading).count() == 1
def test_cascade_true_with_readings_deletes_device_readings_and_toggles(modbus_client):
"""cascade=true + readings → device, readings, and toggle rows all deleted; 200 response."""
from datetime import UTC
from app.models.expose import ExposedEntityToggle
client, engine = modbus_client
_login(client)
device = _make_device(engine)
# Add two readings
now = datetime.now(UTC)
_make_reading(engine, device.id, now, {"voltage": 230.0})
_make_reading(engine, device.id, now - timedelta(minutes=1), {"voltage": 229.0})
# Add an ExposedEntityToggle row for this device
with Session(engine) as session:
toggle = ExposedEntityToggle(
key=f"modbus.{device.uuid}.voltage",
enabled=True,
updated_at=now,
)
session.add(toggle)
# Add one toggle for a different device to verify isolation
other_toggle = ExposedEntityToggle(
key="modbus.other-uuid.voltage",
enabled=True,
updated_at=now,
)
session.add(other_toggle)
session.commit()
resp = client.delete(
f"/api/modbus/devices/{device.uuid}",
params={"cascade": "true"},
headers={"X-CSRF-Token": _CSRF},
)
assert resp.status_code == 200
body = resp.json()
assert body["deleted"] is True
assert body["readings_deleted"] == 2
assert body["toggles_deleted"] == 1
# All rows for this device must be gone
with Session(engine) as session:
assert session.query(ModbusDevice).count() == 0
assert session.query(ModbusReading).count() == 0
# Only the other device's toggle remains
remaining_toggles = session.query(ExposedEntityToggle).all()
assert len(remaining_toggles) == 1
assert remaining_toggles[0].key == "modbus.other-uuid.voltage"
def test_cascade_true_no_readings_deletes_device(modbus_client):
"""cascade=true + no readings → device deleted, response shows 0 counts."""
client, engine = modbus_client
_login(client)
device = _make_device(engine)
resp = client.delete(
f"/api/modbus/devices/{device.uuid}",
params={"cascade": "true"},
headers={"X-CSRF-Token": _CSRF},
)
assert resp.status_code == 200
body = resp.json()
assert body["deleted"] is True
assert body["readings_deleted"] == 0
assert body["toggles_deleted"] == 0
with Session(engine) as session:
assert session.query(ModbusDevice).count() == 0
def test_cascade_unauthenticated_returns_401(modbus_client):
"""Unauthenticated cascade delete request → 401."""
client, engine = modbus_client
device = _make_device(engine)
resp = client.delete(
f"/api/modbus/devices/{device.uuid}",
params={"cascade": "true"},
headers={"X-CSRF-Token": _CSRF},
)
assert resp.status_code == 401
def test_cascade_missing_csrf_returns_403(modbus_client):
"""cascade=true without CSRF → 403."""
client, engine = modbus_client
_login(client)
device = _make_device(engine)
resp = client.delete(
f"/api/modbus/devices/{device.uuid}",
params={"cascade": "true"},
)
assert resp.status_code == 403
def test_cascade_mqtt_unavailable_still_deletes(modbus_client):
"""When MQTT is not connected, cascade delete must still succeed (HA cleanup is best-effort)."""
from unittest.mock import patch
client, engine = modbus_client
_login(client)
device = _make_device(engine)
_make_reading(engine, device.id, datetime.now(UTC), {"voltage": 230.0})
# Simulate MQTT not connected so clear_device_discovery is a no-op
with patch("app.services.ha_discovery.mqtt_manager") as mock_mqtt:
mock_mqtt.is_connected = False
resp = client.delete(
f"/api/modbus/devices/{device.uuid}",
params={"cascade": "true"},
headers={"X-CSRF-Token": _CSRF},
)
assert resp.status_code == 200
body = resp.json()
assert body["deleted"] is True
assert body["readings_deleted"] == 1
# DB rows gone
with Session(engine) as session:
assert session.query(ModbusDevice).count() == 0
assert session.query(ModbusReading).count() == 0
# ---------------------------------------------------------------------------
# GET /api/modbus/devices/{uuid}/latest
# ---------------------------------------------------------------------------
+504
View File
@@ -0,0 +1,504 @@
"""Tests for M6-T06: DSMR ingest service (handle_message).
All tests use a temporary SQLite database (upgraded to the full Alembic head)
so that the real DsmrReading model/constraints are exercised.
Covers:
- Sample telegram with second=00 (on interval) is persisted as a full-frame blob.
- Sample telegram with second=48 (off interval, 10s default) is discarded.
- Telegram with second=00 all fields including gas (extra_device_*) and null
phases are stored verbatim.
- Duplicate source_id not re-inserted (idempotency).
- source_id absent row still inserted with source_id=None.
- dsmr_sample_interval_s=0 no ZeroDivisionError; every telegram is persisted.
- Invalid JSON silently discarded (no exception propagated).
- Missing 'timestamp' field silently discarded.
- Unparseable 'timestamp' silently discarded.
- Handler exception (e.g. bad DB) does not propagate out of handle_message.
"""
from __future__ import annotations
import json
from pathlib import Path
from unittest.mock import MagicMock, patch
import pytest
from alembic import command
from alembic.config import Config
from sqlalchemy import create_engine, select
from sqlalchemy.orm import Session
from app.models.energy import DsmrReading
# ---------------------------------------------------------------------------
# Helpers / fixtures
# ---------------------------------------------------------------------------
def _alembic_config(database_url: str) -> Config:
cfg = Config("alembic_app.ini")
cfg.set_main_option("sqlalchemy.url", database_url)
return cfg
@pytest.fixture()
def dsmr_db(tmp_path: Path):
"""Temporary SQLite DB upgraded to the Alembic head; yields (engine, session_factory)."""
db_path = tmp_path / "dsmr_ingest_test.db"
db_url = f"sqlite:///{db_path}"
command.upgrade(_alembic_config(db_url), "head")
engine = create_engine(db_url, connect_args={"check_same_thread": False})
from sqlalchemy.orm import sessionmaker
SessionLocal = sessionmaker(bind=engine, autoflush=False, autocommit=False, class_=Session)
yield engine, SessionLocal
engine.dispose()
def _make_settings(
*,
dsmr_sample_interval_s: int = 10,
dsmr_ingest_enabled: bool = True,
dsmr_mqtt_topic: str = "dsmr/json",
):
s = MagicMock()
s.dsmr_sample_interval_s = dsmr_sample_interval_s
s.dsmr_ingest_enabled = dsmr_ingest_enabled
s.dsmr_mqtt_topic = dsmr_mqtt_topic
return s
# The reference telegram sample from §6.3 of the design doc.
# Numeric values are JSON strings; missing phases are null.
_SAMPLE_TELEGRAM = {
"id": 200086230,
"timestamp": "2026-06-23T12:16:00Z", # second=00, on 10s boundary
"electricity_delivered_1": "20915.154",
"electricity_returned_1": "2979.905",
"electricity_delivered_2": "15212.090",
"electricity_returned_2": "6786.406",
"electricity_currently_delivered": "0.000",
"phase_currently_delivered_l1": "0.000",
"phase_currently_delivered_l2": None, # absent phase
"extra_device_timestamp": "2026-06-23T12:15:00Z",
"extra_device_delivered": "6208.234",
"phase_voltage_l1": "237.0",
"phase_voltage_l2": None, # absent phase
}
# Telegram with second=48 (not divisible by 10) — should be discarded.
_SAMPLE_TELEGRAM_SECOND_48 = {
**_SAMPLE_TELEGRAM,
"id": 200086248,
"timestamp": "2026-06-23T12:16:48Z", # second=48
}
# Telegram with second=10 — another valid boundary.
_SAMPLE_TELEGRAM_SECOND_10 = {
**_SAMPLE_TELEGRAM,
"id": 200086210,
"timestamp": "2026-06-23T12:16:10Z", # second=10
}
def _payload(data: dict) -> bytes:
return json.dumps(data).encode()
# ---------------------------------------------------------------------------
# Helper: invoke handle_message with the test DB injected via patch
# ---------------------------------------------------------------------------
def _call_handle_message(
data: dict,
settings,
SessionLocal,
) -> None:
"""Call dsmr_ingest.handle_message with the test SessionLocal patched in."""
from app.services import dsmr_ingest
with patch.object(dsmr_ingest, "get_session_local", return_value=SessionLocal):
dsmr_ingest.handle_message(_payload(data), settings)
def _count_readings(SessionLocal) -> int:
with SessionLocal() as session:
return session.scalar(
__import__("sqlalchemy", fromlist=["func"]).func.count(DsmrReading.id)
)
def _get_readings(SessionLocal) -> list[DsmrReading]:
with SessionLocal() as session:
return session.scalars(select(DsmrReading)).all()
# ---------------------------------------------------------------------------
# 1. On-interval telegram is persisted (whole frame)
# ---------------------------------------------------------------------------
def test_second_00_persists_full_frame(dsmr_db):
"""A telegram with second=00 (10s boundary) must be persisted with the full payload."""
engine, SessionLocal = dsmr_db
settings = _make_settings(dsmr_sample_interval_s=10)
_call_handle_message(_SAMPLE_TELEGRAM, settings, SessionLocal)
with Session(engine) as session:
readings = session.scalars(select(DsmrReading)).all()
assert len(readings) == 1
row = readings[0]
# Recorded_at must be parsed from the telegram timestamp.
assert row.recorded_at is not None
assert row.recorded_at.second == 0
# source_id must equal the telegram id.
assert row.source_id == _SAMPLE_TELEGRAM["id"]
# Full frame preserved — spot-check several fields.
payload = row.payload
assert payload["electricity_delivered_1"] == "20915.154"
assert payload["electricity_returned_2"] == "6786.406"
assert payload["extra_device_delivered"] == "6208.234"
assert payload["extra_device_timestamp"] == "2026-06-23T12:15:00Z"
# Null phases stored verbatim (not omitted, not converted).
assert "phase_currently_delivered_l2" in payload
assert payload["phase_currently_delivered_l2"] is None
assert "phase_voltage_l2" in payload
assert payload["phase_voltage_l2"] is None
def test_second_10_persists(dsmr_db):
"""A telegram with second=10 (another 10s boundary) must also be persisted."""
_, SessionLocal = dsmr_db
settings = _make_settings(dsmr_sample_interval_s=10)
_call_handle_message(_SAMPLE_TELEGRAM_SECOND_10, settings, SessionLocal)
assert _count_readings(SessionLocal) == 1
# ---------------------------------------------------------------------------
# 2. Off-interval telegram is discarded
# ---------------------------------------------------------------------------
def test_second_48_discarded(dsmr_db):
"""A telegram with second=48 (not on 10s boundary) must be silently discarded."""
_, SessionLocal = dsmr_db
settings = _make_settings(dsmr_sample_interval_s=10)
_call_handle_message(_SAMPLE_TELEGRAM_SECOND_48, settings, SessionLocal)
assert _count_readings(SessionLocal) == 0
def test_off_interval_not_multiple_of_10(dsmr_db):
"""Seconds 19, 1119, etc. are all off-interval and must be discarded."""
_, SessionLocal = dsmr_db
settings = _make_settings(dsmr_sample_interval_s=10)
for second in (1, 3, 7, 9, 11, 23, 47, 59):
ts = f"2026-06-23T12:16:{second:02d}Z"
data = {**_SAMPLE_TELEGRAM, "id": 200000000 + second, "timestamp": ts}
_call_handle_message(data, settings, SessionLocal)
assert _count_readings(SessionLocal) == 0
# ---------------------------------------------------------------------------
# 3. Idempotency — keyed on recorded_at (telegram timestamp), NOT the telegram id
# ---------------------------------------------------------------------------
def test_same_timestamp_not_reinserted(dsmr_db):
"""Feeding the same telegram twice (same timestamp) must result in one row."""
_, SessionLocal = dsmr_db
settings = _make_settings(dsmr_sample_interval_s=10)
_call_handle_message(_SAMPLE_TELEGRAM, settings, SessionLocal)
_call_handle_message(_SAMPLE_TELEGRAM, settings, SessionLocal) # duplicate
assert _count_readings(SessionLocal) == 1
def test_different_timestamps_are_independent(dsmr_db):
"""Telegrams with different timestamps produce separate rows."""
_, SessionLocal = dsmr_db
settings = _make_settings(dsmr_sample_interval_s=10)
_call_handle_message(_SAMPLE_TELEGRAM, settings, SessionLocal)
_call_handle_message(_SAMPLE_TELEGRAM_SECOND_10, settings, SessionLocal)
assert _count_readings(SessionLocal) == 2
def test_telegram_id_collision_does_not_drop_new_data(dsmr_db):
"""Regression: the telegram id overflows / gets reset to zero in DSMR firmware.
Two DISTINCT telegrams (different timestamps) that happen to share the SAME
telegram id must BOTH be stored dedup must not depend on the telegram id."""
_, SessionLocal = dsmr_db
settings = _make_settings(dsmr_sample_interval_s=10)
first = {**_SAMPLE_TELEGRAM, "id": 0, "timestamp": "2026-06-23T12:16:00Z"}
# Later telegram, id reset back to the same value after an overflow.
second = {**_SAMPLE_TELEGRAM, "id": 0, "timestamp": "2026-06-23T12:16:10Z"}
_call_handle_message(first, settings, SessionLocal)
_call_handle_message(second, settings, SessionLocal)
# Both must persist — the colliding telegram id must not cause a drop.
assert _count_readings(SessionLocal) == 2
# ---------------------------------------------------------------------------
# 4. Missing source_id — still persisted with source_id=None
# ---------------------------------------------------------------------------
def test_missing_id_persisted_with_source_id_none(dsmr_db):
"""Telegram without an 'id' field must be stored with source_id=None."""
engine, SessionLocal = dsmr_db
settings = _make_settings(dsmr_sample_interval_s=10)
data = {k: v for k, v in _SAMPLE_TELEGRAM.items() if k != "id"}
_call_handle_message(data, settings, SessionLocal)
with Session(engine) as session:
readings = session.scalars(select(DsmrReading)).all()
assert len(readings) == 1
assert readings[0].source_id is None
# ---------------------------------------------------------------------------
# 5. interval=0 → no ZeroDivisionError; all telegrams persisted
# ---------------------------------------------------------------------------
def test_interval_zero_does_not_crash(dsmr_db):
"""When dsmr_sample_interval_s=0, no ZeroDivisionError; all telegrams kept."""
_, SessionLocal = dsmr_db
settings = _make_settings(dsmr_sample_interval_s=0)
for second in (0, 7, 13, 48, 59):
ts = f"2026-06-23T12:16:{second:02d}Z"
data = {**_SAMPLE_TELEGRAM, "id": 300000000 + second, "timestamp": ts}
_call_handle_message(data, settings, SessionLocal)
# All 5 should have been persisted (no sampling applied).
assert _count_readings(SessionLocal) == 5
# ---------------------------------------------------------------------------
# 6. Invalid / malformed payloads — silently discarded, no exception propagated
# ---------------------------------------------------------------------------
def test_invalid_json_discarded(dsmr_db):
"""Invalid JSON payload must be silently discarded without raising."""
_, SessionLocal = dsmr_db
settings = _make_settings()
from app.services import dsmr_ingest
with patch.object(dsmr_ingest, "get_session_local", return_value=SessionLocal):
# Must not raise
dsmr_ingest.handle_message(b"not-valid-json", settings)
assert _count_readings(SessionLocal) == 0
def test_missing_timestamp_discarded(dsmr_db):
"""Telegram without a 'timestamp' field must be discarded."""
_, SessionLocal = dsmr_db
settings = _make_settings()
data = {k: v for k, v in _SAMPLE_TELEGRAM.items() if k != "timestamp"}
_call_handle_message(data, settings, SessionLocal)
assert _count_readings(SessionLocal) == 0
def test_unparseable_timestamp_discarded(dsmr_db):
"""Telegram with a garbage 'timestamp' field must be discarded."""
_, SessionLocal = dsmr_db
settings = _make_settings()
data = {**_SAMPLE_TELEGRAM, "timestamp": "not-a-date"}
_call_handle_message(data, settings, SessionLocal)
assert _count_readings(SessionLocal) == 0
def test_handle_message_does_not_propagate_any_exception() -> None:
"""handle_message must never propagate any exception to the caller."""
from app.services import dsmr_ingest
settings = _make_settings()
def _explode():
raise RuntimeError("DB is on fire")
with patch.object(dsmr_ingest, "get_session_local", side_effect=_explode):
# Must not raise
dsmr_ingest.handle_message(_payload(_SAMPLE_TELEGRAM), settings)
# ---------------------------------------------------------------------------
# 7. Null phase values stored verbatim
# ---------------------------------------------------------------------------
def test_null_phases_stored_as_none(dsmr_db):
"""Null phase values must be preserved as JSON null (Python None) in payload."""
engine, SessionLocal = dsmr_db
settings = _make_settings(dsmr_sample_interval_s=10)
_call_handle_message(_SAMPLE_TELEGRAM, settings, SessionLocal)
with Session(engine) as session:
reading = session.scalars(select(DsmrReading)).first()
assert reading is not None
assert reading.payload.get("phase_currently_delivered_l2") is None
assert reading.payload.get("phase_voltage_l2") is None
# ---------------------------------------------------------------------------
# 8. Gas / extra_device fields stored
# ---------------------------------------------------------------------------
def test_gas_fields_stored(dsmr_db):
"""extra_device_* (gas) fields must be present in the stored payload."""
engine, SessionLocal = dsmr_db
settings = _make_settings(dsmr_sample_interval_s=10)
_call_handle_message(_SAMPLE_TELEGRAM, settings, SessionLocal)
with Session(engine) as session:
reading = session.scalars(select(DsmrReading)).first()
assert reading is not None
payload = reading.payload
assert "extra_device_delivered" in payload
assert payload["extra_device_delivered"] == "6208.234"
assert "extra_device_timestamp" in payload
# ---------------------------------------------------------------------------
# 9. Timestamp with +00:00 suffix (no Z) also works
# ---------------------------------------------------------------------------
def test_timestamp_with_utc_offset_suffix(dsmr_db):
"""Timestamps using '+00:00' instead of 'Z' must also be parsed correctly."""
_, SessionLocal = dsmr_db
settings = _make_settings(dsmr_sample_interval_s=10)
data = {
**_SAMPLE_TELEGRAM,
"id": 999999,
"timestamp": "2026-06-23T12:16:00+00:00", # no Z, uses +00:00
}
_call_handle_message(data, settings, SessionLocal)
assert _count_readings(SessionLocal) == 1
# ---------------------------------------------------------------------------
# 10. handle_tariff_message: parse, validate, update, reject invalid payloads
# ---------------------------------------------------------------------------
@pytest.fixture()
def reset_tariff(monkeypatch):
"""Reset _current_tariff to None before and after each tariff test."""
from app.services import dsmr_ingest as _di
monkeypatch.setattr(_di, "_current_tariff", None)
yield
# monkeypatch auto-restores on teardown
def test_tariff_message_value_2_sets_tariff(reset_tariff):
"""Payload b'2' must set the current tariff to 2 (normal/peak)."""
from app.services.dsmr_ingest import handle_tariff_message, get_current_tariff
handle_tariff_message(b"2")
assert get_current_tariff() == 2
def test_tariff_message_value_1_sets_tariff(reset_tariff):
"""Payload b'1' must set the current tariff to 1 (dal/off-peak)."""
from app.services.dsmr_ingest import handle_tariff_message, get_current_tariff
handle_tariff_message(b"1")
assert get_current_tariff() == 1
def test_tariff_message_updates_from_2_to_1(reset_tariff):
"""Subsequent payloads must overwrite the previous tariff value."""
from app.services.dsmr_ingest import handle_tariff_message, get_current_tariff
handle_tariff_message(b"2")
assert get_current_tariff() == 2
handle_tariff_message(b"1")
assert get_current_tariff() == 1
def test_tariff_message_strips_whitespace(reset_tariff):
"""Payloads with surrounding whitespace (e.g. b'2\\n') must be accepted."""
from app.services.dsmr_ingest import handle_tariff_message, get_current_tariff
handle_tariff_message(b"2\n")
assert get_current_tariff() == 2
handle_tariff_message(b" 1 ")
assert get_current_tariff() == 1
def test_tariff_message_invalid_non_numeric_does_not_update(reset_tariff):
"""Non-numeric payload must not update the tariff; previous value is preserved."""
from app.services.dsmr_ingest import handle_tariff_message, get_current_tariff, set_current_tariff
set_current_tariff(2)
handle_tariff_message(b"x")
# Must NOT raise and must NOT change the tariff.
assert get_current_tariff() == 2
def test_tariff_message_invalid_empty_does_not_update(reset_tariff):
"""Empty payload must not update the tariff; previous value is preserved."""
from app.services.dsmr_ingest import handle_tariff_message, get_current_tariff, set_current_tariff
set_current_tariff(1)
handle_tariff_message(b"")
assert get_current_tariff() == 1
def test_tariff_message_out_of_range_value_does_not_update(reset_tariff):
"""Payload with out-of-range integer (not 1 or 2) must not update the tariff."""
from app.services.dsmr_ingest import handle_tariff_message, get_current_tariff, set_current_tariff
set_current_tariff(2)
handle_tariff_message(b"3") # 3 is not a valid tariff
assert get_current_tariff() == 2
handle_tariff_message(b"0") # 0 is not a valid tariff
assert get_current_tariff() == 2
def test_tariff_message_does_not_raise_on_any_input(reset_tariff):
"""handle_tariff_message must never propagate any exception to the caller."""
from app.services.dsmr_ingest import handle_tariff_message
# All of these must complete without raising.
for payload in (b"", b"x", b"99", b"\xff\xfe", b"None", b"2.0"):
handle_tariff_message(payload) # must not raise
+173
View File
@@ -0,0 +1,173 @@
"""Tests for restart-free DSMR subscription management (apply_dsmr_subscription).
These verify that toggling DSMR ingest / changing its topic / changing its sample
interval via the config UI is reflected in the live MQTT subscription without an
app restart. A fake MQTT manager is injected so no real broker is touched.
"""
from __future__ import annotations
from unittest.mock import MagicMock
import pytest
from app.services import dsmr_ingest
class _FakeMqtt:
def __init__(self) -> None:
self.subscribe_calls: list[tuple[str, object]] = []
self.unsubscribe_calls: list[str] = []
def subscribe(self, topic: str, handler) -> None:
self.subscribe_calls.append((topic, handler))
def unsubscribe(self, topic: str) -> None:
self.unsubscribe_calls.append(topic)
def _settings(
*,
enabled: bool = True,
topic: str = "dsmr/json",
interval: int = 10,
tariff_topic: str = "",
):
s = MagicMock()
s.dsmr_ingest_enabled = enabled
s.dsmr_mqtt_topic = topic
s.dsmr_sample_interval_s = interval
s.dsmr_tariff_topic = tariff_topic
return s
@pytest.fixture()
def fake_mqtt(monkeypatch):
fake = _FakeMqtt()
# apply_dsmr_subscription does `from app.integrations.mqtt import mqtt_manager`
# at call time, so patching the module attribute is picked up.
monkeypatch.setattr("app.integrations.mqtt.mqtt_manager", fake)
# Reset (and auto-restore) the module-level "currently subscribed topics".
monkeypatch.setattr(dsmr_ingest, "_current_dsmr_topic", None)
monkeypatch.setattr(dsmr_ingest, "_current_tariff_topic", None)
return fake
def test_enabled_subscribes_to_topic(fake_mqtt):
dsmr_ingest.apply_dsmr_subscription(_settings(enabled=True, topic="dsmr/json"))
assert len(fake_mqtt.subscribe_calls) == 1
assert fake_mqtt.subscribe_calls[0][0] == "dsmr/json"
assert fake_mqtt.unsubscribe_calls == []
assert dsmr_ingest._current_dsmr_topic == "dsmr/json"
def test_disabled_after_enabled_unsubscribes(fake_mqtt):
dsmr_ingest.apply_dsmr_subscription(_settings(enabled=True, topic="dsmr/json"))
dsmr_ingest.apply_dsmr_subscription(_settings(enabled=False))
assert fake_mqtt.unsubscribe_calls == ["dsmr/json"]
assert dsmr_ingest._current_dsmr_topic is None
def test_topic_change_unsubscribes_old_subscribes_new(fake_mqtt):
dsmr_ingest.apply_dsmr_subscription(_settings(topic="dsmr/json"))
dsmr_ingest.apply_dsmr_subscription(_settings(topic="meter/dsmr"))
assert fake_mqtt.unsubscribe_calls == ["dsmr/json"]
assert [t for t, _ in fake_mqtt.subscribe_calls] == ["dsmr/json", "meter/dsmr"]
assert dsmr_ingest._current_dsmr_topic == "meter/dsmr"
def test_reapply_same_topic_resubscribes_fresh_handler(fake_mqtt):
# A changed sample interval must take effect — the handler is re-bound to a
# fresh settings snapshot, so re-applying the same topic re-subscribes.
dsmr_ingest.apply_dsmr_subscription(_settings(topic="dsmr/json", interval=10))
dsmr_ingest.apply_dsmr_subscription(_settings(topic="dsmr/json", interval=20))
assert len(fake_mqtt.subscribe_calls) == 2
assert fake_mqtt.unsubscribe_calls == [] # same topic, no churn
handler1 = fake_mqtt.subscribe_calls[0][1]
handler2 = fake_mqtt.subscribe_calls[1][1]
assert handler1 is not handler2 # fresh closure carrying the new settings
def test_disabled_when_never_enabled_is_noop(fake_mqtt):
dsmr_ingest.apply_dsmr_subscription(_settings(enabled=False))
assert fake_mqtt.subscribe_calls == []
assert fake_mqtt.unsubscribe_calls == []
assert dsmr_ingest._current_dsmr_topic is None
# ---------------------------------------------------------------------------
# Tariff topic subscription management
# ---------------------------------------------------------------------------
def test_enabled_with_tariff_topic_subscribes_both(fake_mqtt):
"""When enabled and tariff_topic is non-empty, both topics must be subscribed."""
dsmr_ingest.apply_dsmr_subscription(
_settings(enabled=True, topic="dsmr/json", tariff_topic="dsmr/meter-stats/electricity_tariff")
)
subscribed_topics = [t for t, _ in fake_mqtt.subscribe_calls]
assert "dsmr/json" in subscribed_topics
assert "dsmr/meter-stats/electricity_tariff" in subscribed_topics
assert len(fake_mqtt.subscribe_calls) == 2
assert dsmr_ingest._current_tariff_topic == "dsmr/meter-stats/electricity_tariff"
def test_enabled_with_empty_tariff_topic_subscribes_only_main(fake_mqtt):
"""When tariff_topic is empty, only the main DSMR topic is subscribed."""
dsmr_ingest.apply_dsmr_subscription(
_settings(enabled=True, topic="dsmr/json", tariff_topic="")
)
assert len(fake_mqtt.subscribe_calls) == 1
assert fake_mqtt.subscribe_calls[0][0] == "dsmr/json"
assert dsmr_ingest._current_tariff_topic is None
def test_disabled_after_tariff_subscription_unsubscribes_both(fake_mqtt):
"""Disabling ingest must also unsubscribe the tariff topic."""
dsmr_ingest.apply_dsmr_subscription(
_settings(enabled=True, topic="dsmr/json", tariff_topic="dsmr/meter-stats/electricity_tariff")
)
dsmr_ingest.apply_dsmr_subscription(_settings(enabled=False))
assert "dsmr/json" in fake_mqtt.unsubscribe_calls
assert "dsmr/meter-stats/electricity_tariff" in fake_mqtt.unsubscribe_calls
assert dsmr_ingest._current_dsmr_topic is None
assert dsmr_ingest._current_tariff_topic is None
def test_tariff_topic_change_resubscribes(fake_mqtt):
"""Changing the tariff topic must unsubscribe the old one and subscribe the new one."""
old_tariff = "dsmr/meter-stats/electricity_tariff"
new_tariff = "meter/tariff"
dsmr_ingest.apply_dsmr_subscription(
_settings(enabled=True, topic="dsmr/json", tariff_topic=old_tariff)
)
dsmr_ingest.apply_dsmr_subscription(
_settings(enabled=True, topic="dsmr/json", tariff_topic=new_tariff)
)
assert old_tariff in fake_mqtt.unsubscribe_calls
subscribed_topics = [t for t, _ in fake_mqtt.subscribe_calls]
assert new_tariff in subscribed_topics
assert dsmr_ingest._current_tariff_topic == new_tariff
def test_tariff_topic_cleared_unsubscribes(fake_mqtt):
"""Setting tariff_topic to empty after it was subscribed must unsubscribe it."""
dsmr_ingest.apply_dsmr_subscription(
_settings(enabled=True, topic="dsmr/json", tariff_topic="dsmr/meter-stats/electricity_tariff")
)
dsmr_ingest.apply_dsmr_subscription(
_settings(enabled=True, topic="dsmr/json", tariff_topic="")
)
assert "dsmr/meter-stats/electricity_tariff" in fake_mqtt.unsubscribe_calls
assert dsmr_ingest._current_tariff_topic is None
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+62 -14
View File
@@ -203,16 +203,61 @@ def test_register_provider_direct_call():
def test_build_catalog_empty_with_no_devices(expose_db):
"""build_catalog with no enabled devices must return an empty list."""
"""build_catalog with no enabled modbus devices must contain no modbus entities.
FUE-T05: the energy_cost provider now requires an active electricity meter.
Without one, it returns [] and the catalog contains no energy entities.
With one, it produces 6 entities.
This test verifies both cases:
1. No modbus devices no modbus entities.
2. No active meter no energy entities (provider returns []).
3. After inserting an active meter 6 energy entities present.
"""
from app.integrations.expose import build_catalog
# Case: no modbus devices, no active meter → catalog is empty.
with Session(expose_db) as session:
catalog = build_catalog(session)
# The modbus provider will find no enabled devices; other providers may
# add entries only if registered. Since we only register the modbus
# provider at module load, the result should be empty.
assert catalog == []
modbus_entities = [e for e in catalog if e.entity.key.startswith("modbus.")]
assert modbus_entities == [], (
"Expected no modbus entities when no modbus devices are enabled"
)
# No active electricity meter → energy-cost provider returns [] → no energy entities.
energy_keys_no_meter = {e.entity.key for e in catalog if e.entity.key.startswith("energy.")}
assert len(energy_keys_no_meter) == 0, (
f"Expected 0 energy_cost entities (no active meter), got {energy_keys_no_meter!r}"
)
# Insert an active electricity meter → provider should now produce 6 entities.
from datetime import datetime, timezone
now = datetime.now(tz=timezone.utc)
with Session(expose_db) as session:
from app.models.energy import Meter
m = Meter(
label="Test Meter for catalog",
commodity="electricity",
started_at=now,
ended_at=None,
reason="initial",
note=None,
created_at=now,
)
session.add(m)
session.commit()
with Session(expose_db) as session:
catalog_with_meter = build_catalog(session)
energy_keys_with_meter = {
e.entity.key for e in catalog_with_meter if e.entity.key.startswith("energy.")
}
assert len(energy_keys_with_meter) == 6, (
f"Expected exactly 6 energy_cost entities (4 original + 2 daily) with active meter, "
f"got {energy_keys_with_meter!r}"
)
def test_build_catalog_produces_entities_for_enabled_device(expose_db):
@@ -530,7 +575,13 @@ def test_exposed_entity_toggle_key_unique_index(expose_db):
def test_downgrade_removes_toggle_table(tmp_path: Path):
"""downgrade -1 from head must drop the exposed_entity_toggle table cleanly."""
"""Downgrading to the revision before exposed_entity_toggle must drop that table cleanly.
We downgrade to the explicit down_revision of the expose-toggle migration
(``20260622_09_modbus_tables``) rather than using ``-1`` from head, so the test
remains correct as new revisions are added on top. This is the same pattern
used in test_modbus_models.py for the modbus downgrade test.
"""
db_path = tmp_path / "downgrade_expose_test.db"
db_url = f"sqlite:///{db_path}"
alembic_cfg = _make_app_alembic_config(db_url)
@@ -542,14 +593,15 @@ def test_downgrade_removes_toggle_table(tmp_path: Path):
assert "exposed_entity_toggle" in inspector.get_table_names()
engine.dispose()
# Downgrade one step removes the exposed_entity_toggle revision.
command.downgrade(alembic_cfg, "-1")
# Downgrade to the explicit down_revision of the exposed_entity_toggle migration,
# consistent with how test_modbus_models.py handles its own downgrade test.
command.downgrade(alembic_cfg, "20260622_09_modbus_tables")
engine = create_engine(db_url, connect_args={"check_same_thread": False})
inspector = inspect(engine)
table_names = inspector.get_table_names()
assert "exposed_entity_toggle" not in table_names, (
"exposed_entity_toggle should be gone after downgrade -1"
"exposed_entity_toggle should be gone after downgrade to 20260622_09_modbus_tables"
)
# Previous tables must still exist.
assert "modbus_device" in table_names
@@ -668,7 +720,7 @@ def test_expose_toggle_update(expose_db):
def test_app_baseline_revision_matches_new_head(tmp_path: Path):
"""APP_BASELINE_REVISION must equal the Alembic head after the new migration."""
"""APP_BASELINE_REVISION must equal the Alembic head revision."""
from alembic.script import ScriptDirectory
from scripts.app_db_adopt import APP_BASELINE_REVISION
@@ -683,7 +735,3 @@ def test_app_baseline_revision_matches_new_head(tmp_path: Path):
f"APP_BASELINE_REVISION={APP_BASELINE_REVISION!r} does not match "
f"Alembic head={head!r}"
)
# Confirm the new revision is the head.
assert head == "20260622_10_exposed_entities", (
f"Expected new head to be '20260622_10_exposed_entities', got {head!r}"
)
+200 -12
View File
@@ -90,11 +90,13 @@ def _make_settings(
mqtt_enabled: bool = True,
ha_discovery_enabled: bool = True,
ha_discovery_prefix: str = "homeassistant",
ha_state_topic_prefix: str = "home_automation",
) -> MagicMock:
s = MagicMock()
s.mqtt_enabled = mqtt_enabled
s.ha_discovery_enabled = ha_discovery_enabled
s.ha_discovery_prefix = ha_discovery_prefix
s.ha_state_topic_prefix = ha_state_topic_prefix
return s
@@ -142,7 +144,7 @@ def test_build_discovery_payload_topic_format() -> None:
name="My Meter Voltage",
)
topic, config = build_discovery_payload(entity, prefix="homeassistant")
topic, config = build_discovery_payload(entity, discovery_prefix="homeassistant")
# node_id replaces hyphens with underscores
node_id = uuid_val.replace("-", "_")
@@ -168,7 +170,7 @@ def test_build_discovery_payload_unique_id_from_uuid() -> None:
name="Meter A Current",
)
_, config = build_discovery_payload(entity, prefix="homeassistant")
_, config = build_discovery_payload(entity, discovery_prefix="homeassistant")
# unique_id must contain the device uuid (normalised)
uid = config["unique_id"].replace("-", "").replace("_", "")
@@ -210,8 +212,8 @@ def test_build_discovery_payload_unique_id_is_stable_across_renames() -> None:
name="New Name Voltage",
)
_, config_a = build_discovery_payload(entity_a, prefix="homeassistant")
_, config_b = build_discovery_payload(entity_b, prefix="homeassistant")
_, config_a = build_discovery_payload(entity_a, discovery_prefix="homeassistant")
_, config_b = build_discovery_payload(entity_b, discovery_prefix="homeassistant")
assert config_a["unique_id"] == config_b["unique_id"], (
"unique_id must not change when friendly_name changes"
@@ -234,7 +236,7 @@ def test_build_discovery_payload_device_block() -> None:
name="Friendly Name Voltage",
)
_, config = build_discovery_payload(entity, prefix="homeassistant")
_, config = build_discovery_payload(entity, discovery_prefix="homeassistant")
assert "device" in config
assert "identifiers" in config["device"]
@@ -259,7 +261,7 @@ def test_build_discovery_payload_sensor_fields() -> None:
state_class="total_increasing",
)
_, config = build_discovery_payload(entity, prefix="homeassistant")
_, config = build_discovery_payload(entity, discovery_prefix="homeassistant")
assert config["device_class"] == "energy"
assert config["unit_of_measurement"] == "kWh"
@@ -284,7 +286,7 @@ def test_build_discovery_payload_binary_sensor() -> None:
name="Meter Online Online",
)
_, config = build_discovery_payload(entity, prefix="homeassistant")
_, config = build_discovery_payload(entity, discovery_prefix="homeassistant")
assert config.get("payload_on") == "ON"
assert config.get("payload_off") == "OFF"
@@ -307,7 +309,7 @@ def test_build_discovery_payload_state_topic_matches_node_and_object() -> None:
name="Meter Voltage",
)
_, config = build_discovery_payload(entity, prefix="homeassistant")
_, config = build_discovery_payload(entity, discovery_prefix="homeassistant")
state_topic = config["state_topic"]
assert "homeassistant/sensor/" in state_topic
@@ -991,7 +993,7 @@ def test_build_discovery_payload_includes_availability_topic() -> None:
name="Avail Meter Voltage",
)
_, config = build_discovery_payload(entity, prefix="homeassistant")
_, config = build_discovery_payload(entity, discovery_prefix="homeassistant")
expected_avail_topic = _availability_topic(uuid_val, "homeassistant")
assert "availability" in config
@@ -1142,7 +1144,9 @@ def test_real_provider_value_getter_returns_latest_reading_value(disco_db) -> No
topics_to_payloads: dict[str, Any] = {t: p for t, p in published}
# Availability topic must be "online".
avail_topic = f"homeassistant/modbus/{uuid_val.replace('-', '_')}/availability"
# State/availability topics now use ha_state_topic_prefix ("home_automation"), not
# ha_discovery_prefix ("homeassistant").
avail_topic = f"home_automation/modbus/{uuid_val.replace('-', '_')}/availability"
assert avail_topic in topics_to_payloads, (
f"Availability topic {avail_topic!r} not published. All topics: {list(topics_to_payloads)}"
)
@@ -1153,7 +1157,8 @@ def test_real_provider_value_getter_returns_latest_reading_value(disco_db) -> No
# Voltage state topic must carry the real value from the reading (231.5).
node_id = uuid_val.replace("-", "_")
voltage_obj_id = f"modbus_{uuid_val.replace('-', '_')}_voltage"
voltage_state_topic = f"homeassistant/sensor/{node_id}/{voltage_obj_id}/state"
# State topics use ha_state_topic_prefix ("home_automation").
voltage_state_topic = f"home_automation/sensor/{node_id}/{voltage_obj_id}/state"
assert voltage_state_topic in topics_to_payloads, (
f"Voltage state topic {voltage_state_topic!r} not published. "
f"Sensor state topics: {[t for t in topics_to_payloads if 'sensor' in t and 'state' in t]}"
@@ -1165,7 +1170,8 @@ def test_real_provider_value_getter_returns_latest_reading_value(disco_db) -> No
# Online binary_sensor state topic must be "ON" (last_poll_ok=True).
online_obj_id = f"modbus_{uuid_val.replace('-', '_')}_online"
online_state_topic = f"homeassistant/binary_sensor/{node_id}/{online_obj_id}/state"
# State topics use ha_state_topic_prefix ("home_automation").
online_state_topic = f"home_automation/binary_sensor/{node_id}/{online_obj_id}/state"
assert online_state_topic in topics_to_payloads, (
f"Online state topic {online_state_topic!r} not published. "
f"Topics: {list(topics_to_payloads)}"
@@ -1173,3 +1179,185 @@ def test_real_provider_value_getter_returns_latest_reading_value(disco_db) -> No
assert topics_to_payloads[online_state_topic] == "ON", (
f"Expected 'ON' for online sensor, got {topics_to_payloads[online_state_topic]!r}"
)
# ---------------------------------------------------------------------------
# 10. Topic prefix separation: config vs state/availability
# ---------------------------------------------------------------------------
def test_build_discovery_payload_state_topic_uses_state_prefix() -> None:
"""When discovery_prefix and state_prefix differ, state_topic must use state_prefix."""
from app.integrations.expose import DeviceInfo, ExposableEntity
from app.services.ha_discovery import build_discovery_payload
uuid_val = "aaaaaaaa-1111-2222-3333-444444444444"
device = DeviceInfo(identifiers=("modbus", uuid_val), name="Split Prefix Meter")
entity = ExposableEntity(
key=f"modbus.{uuid_val}.voltage",
component="sensor",
device=device,
device_class="voltage",
unit="V",
name="Split Prefix Meter Voltage",
)
topic, config = build_discovery_payload(
entity,
discovery_prefix="homeassistant",
state_prefix="home_automation",
)
# Config topic must use the discovery prefix.
assert topic.startswith("homeassistant/"), (
f"Config topic must start with 'homeassistant/', got {topic!r}"
)
assert topic.endswith("/config"), f"Config topic must end with /config, got {topic!r}"
# State topic in payload must use the state prefix.
state_topic = config["state_topic"]
assert state_topic.startswith("home_automation/"), (
f"state_topic must start with 'home_automation/', got {state_topic!r}"
)
assert state_topic.endswith("/state"), f"state_topic must end with /state, got {state_topic!r}"
def test_build_discovery_payload_availability_topic_uses_state_prefix() -> None:
"""When state_prefix differs from discovery_prefix, availability must use state_prefix."""
from app.integrations.expose import DeviceInfo, ExposableEntity
from app.services.ha_discovery import build_discovery_payload
uuid_val = "bbbbbbbb-1111-2222-3333-444444444444"
device = DeviceInfo(identifiers=("modbus", uuid_val), name="Avail Split Meter")
entity = ExposableEntity(
key=f"modbus.{uuid_val}.voltage",
component="sensor",
device=device,
device_class="voltage",
unit="V",
name="Avail Split Meter Voltage",
)
_, config = build_discovery_payload(
entity,
discovery_prefix="homeassistant",
state_prefix="home_automation",
)
assert "availability" in config, "Entity with provides_availability=True must have availability"
avail_topics = [a["topic"] for a in config["availability"]]
assert len(avail_topics) == 1
avail_topic = avail_topics[0]
assert avail_topic.startswith("home_automation/"), (
f"Availability topic must start with 'home_automation/', got {avail_topic!r}"
)
# The /modbus/ path segment must still be present (hardcoded structure preserved).
assert "/modbus/" in avail_topic, (
f"Availability topic must contain /modbus/, got {avail_topic!r}"
)
assert avail_topic.endswith("/availability"), (
f"Availability topic must end with /availability, got {avail_topic!r}"
)
def test_build_discovery_payload_custom_state_prefix_changes_state_not_config() -> None:
"""Custom ha_state_topic_prefix changes state/availability topics but NOT config topic."""
from app.integrations.expose import DeviceInfo, ExposableEntity
from app.services.ha_discovery import build_discovery_payload
uuid_val = "cccccccc-5555-6666-7777-888888888888"
device = DeviceInfo(identifiers=("modbus", uuid_val), name="Custom Prefix Meter")
entity = ExposableEntity(
key=f"modbus.{uuid_val}.current",
component="sensor",
device=device,
device_class="current",
unit="A",
name="Custom Prefix Meter Current",
)
topic_default, config_default = build_discovery_payload(
entity,
discovery_prefix="homeassistant",
state_prefix="home_automation",
)
topic_custom, config_custom = build_discovery_payload(
entity,
discovery_prefix="homeassistant",
state_prefix="custom_prefix",
)
# Config topic must be identical (discovery_prefix unchanged).
assert topic_default == topic_custom, (
f"Config topic must not change with different state_prefix. "
f"default={topic_default!r}, custom={topic_custom!r}"
)
# State topic must differ.
assert config_default["state_topic"] != config_custom["state_topic"], (
"state_topic must change when state_prefix changes"
)
assert config_custom["state_topic"].startswith("custom_prefix/"), (
f"Custom state_topic must start with 'custom_prefix/', got {config_custom['state_topic']!r}"
)
# Availability topic must differ.
avail_default = config_default["availability"][0]["topic"]
avail_custom = config_custom["availability"][0]["topic"]
assert avail_default != avail_custom, (
"Availability topic must change when state_prefix changes"
)
assert avail_custom.startswith("custom_prefix/"), (
f"Custom avail topic must start with 'custom_prefix/', got {avail_custom!r}"
)
def test_config_ha_state_topic_prefix_default_and_configurable() -> None:
"""ha_state_topic_prefix must default to 'home_automation' and be configurable."""
from app.config import Settings
# Default value.
s = Settings(_env_file=None)
assert s.ha_state_topic_prefix == "home_automation", (
f"Default ha_state_topic_prefix must be 'home_automation', got {s.ha_state_topic_prefix!r}"
)
# Configurable via constructor.
s2 = Settings(_env_file=None, ha_state_topic_prefix="custom_state")
assert s2.ha_state_topic_prefix == "custom_state", (
f"ha_state_topic_prefix must be overridable, got {s2.ha_state_topic_prefix!r}"
)
def test_config_page_ha_state_topic_prefix_in_discovery_section() -> None:
"""HA_STATE_TOPIC_PREFIX must appear in the 'Home Assistant Discovery' config section."""
from app.services.config_page import CONFIG_FIELDS
ha_state_field = next(
(f for f in CONFIG_FIELDS if f.env_name == "HA_STATE_TOPIC_PREFIX"), None
)
assert ha_state_field is not None, (
"HA_STATE_TOPIC_PREFIX must be registered in CONFIG_FIELDS"
)
assert ha_state_field.section == "Home Assistant Discovery", (
f"HA_STATE_TOPIC_PREFIX must be in 'Home Assistant Discovery' section, "
f"got {ha_state_field.section!r}"
)
assert ha_state_field.setting_attr == "ha_state_topic_prefix", (
f"setting_attr must be 'ha_state_topic_prefix', got {ha_state_field.setting_attr!r}"
)
def test_settings_payload_includes_ha_state_topic_prefix() -> None:
"""_settings_payload must include ha_state_topic_prefix so it can be round-tripped."""
from app.config import Settings
from app.services.config_page import _settings_payload
s = Settings(_env_file=None, ha_state_topic_prefix="my_prefix")
payload = _settings_payload(s)
assert "ha_state_topic_prefix" in payload, (
"_settings_payload must include 'ha_state_topic_prefix'"
)
assert payload["ha_state_topic_prefix"] == "my_prefix", (
f"Expected 'my_prefix', got {payload['ha_state_topic_prefix']!r}"
)
+639
View File
@@ -0,0 +1,639 @@
"""Tests for M7-T02: Meter service layer.
Coverage
--------
1. ``meter_at`` half-open interval boundary semantics.
2. ``declare_meter`` first declaration (no active meter), normal swap, backdate rejection.
3. Mutual exclusion each commodity has at most one active meter after swaps.
4. Interval continuity old meter's ended_at == new meter's started_at after swap.
5. Different commodities are independent (electricity swap doesn't touch gas meters).
6. ``list_meters`` ordering and commodity filtering.
7. ``update_meter`` label/note update, started_at retroactive correction with
interval consistency, and validation errors.
8. ``update_meter`` first-meter (no previous) retroactive started_at change.
"""
from __future__ import annotations
from datetime import UTC, datetime, timedelta
from pathlib import Path
import pytest
from alembic import command
from alembic.config import Config
from sqlalchemy import create_engine, event as sa_event
from sqlalchemy.orm import Session
from app.models.energy import Meter
from app.services.meters import (
MeterIntervalError,
MeterOverlapError,
declare_meter,
list_meters,
meter_at,
update_meter,
)
# ---------------------------------------------------------------------------
# Fixtures
# ---------------------------------------------------------------------------
def _make_app_alembic_config(database_url: str) -> Config:
cfg = Config("alembic_app.ini")
cfg.set_main_option("sqlalchemy.url", database_url)
return cfg
def _engine_with_fk(db_url: str):
"""Create a SQLAlchemy engine with SQLite FK enforcement enabled."""
engine = create_engine(db_url, connect_args={"check_same_thread": False})
@sa_event.listens_for(engine, "connect")
def _enable_fk(dbapi_conn, _rec):
cursor = dbapi_conn.cursor()
cursor.execute("PRAGMA foreign_keys = ON")
cursor.close()
return engine
@pytest.fixture()
def meter_db(tmp_path: Path):
"""Temporary SQLite DB upgraded to the current Alembic head with FK enforcement."""
db_path = tmp_path / "meter_service_test.db"
db_url = f"sqlite:///{db_path}"
alembic_cfg = _make_app_alembic_config(db_url)
command.upgrade(alembic_cfg, "head")
engine = _engine_with_fk(db_url)
yield engine
engine.dispose()
@pytest.fixture()
def session(meter_db):
"""Provide a single SQLAlchemy session for a test, auto-rolling back on exit."""
with Session(meter_db) as s:
yield s
# Tests that commit explicitly are fine; for read-only tests the context
# manager handles cleanup.
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
_T0 = datetime(2026, 1, 1, 12, 0, 0, tzinfo=UTC) # base timestamp for tests
def _make_meter(
session: Session,
*,
started_at: datetime,
ended_at: datetime | None = None,
label: str = "Test meter",
commodity: str = "electricity",
reason: str = "initial",
note: str | None = None,
) -> Meter:
"""Insert a Meter row directly (bypassing service logic) for test setup."""
m = Meter(
label=label,
commodity=commodity,
started_at=started_at,
ended_at=ended_at,
reason=reason,
note=note,
created_at=datetime.now(UTC),
)
session.add(m)
session.flush()
return m
# ---------------------------------------------------------------------------
# 1. meter_at — half-open interval semantics
# ---------------------------------------------------------------------------
class TestMeterAt:
def test_returns_none_when_no_meters(self, session: Session):
"""meter_at must return None when the table is empty."""
assert meter_at(session, _T0) is None
def test_returns_active_meter_for_ts_after_start(self, session: Session):
"""An active meter (ended_at IS NULL) covers any ts ≥ started_at."""
m = _make_meter(session, started_at=_T0, ended_at=None)
result = meter_at(session, _T0 + timedelta(hours=1))
assert result is not None
assert result.id == m.id
def test_exact_started_at_is_inclusive(self, session: Session):
"""ts == started_at must be covered by that meter (left-closed boundary)."""
m = _make_meter(session, started_at=_T0, ended_at=None)
result = meter_at(session, _T0)
assert result is not None
assert result.id == m.id
def test_ts_before_started_at_returns_none(self, session: Session):
"""ts < started_at must not be covered."""
_make_meter(session, started_at=_T0, ended_at=None)
result = meter_at(session, _T0 - timedelta(seconds=1))
assert result is None
def test_exact_ended_at_is_exclusive(self, session: Session):
"""ts == ended_at must NOT be covered by the closed meter (right-open boundary)."""
t1 = _T0 + timedelta(hours=2)
_make_meter(session, started_at=_T0, ended_at=t1, label="Old meter")
# The new active meter starts exactly at t1.
new = _make_meter(session, started_at=t1, ended_at=None, label="New meter")
result = meter_at(session, t1)
assert result is not None
assert result.id == new.id
def test_ts_just_before_ended_at_is_covered(self, session: Session):
"""ts just before ended_at must still be covered by the closing meter."""
t1 = _T0 + timedelta(hours=2)
m = _make_meter(session, started_at=_T0, ended_at=t1)
result = meter_at(session, t1 - timedelta(seconds=1))
assert result is not None
assert result.id == m.id
def test_commodity_filter(self, session: Session):
"""meter_at must only return the meter for the requested commodity."""
m_elec = _make_meter(session, started_at=_T0, commodity="electricity")
_make_meter(session, started_at=_T0, commodity="gas")
result = meter_at(session, _T0, commodity="electricity")
assert result is not None
assert result.id == m_elec.id
result_gas = meter_at(session, _T0, commodity="gas")
assert result_gas is not None
assert result_gas.commodity == "gas"
def test_no_meter_for_unknown_commodity(self, session: Session):
"""meter_at returns None when no meter exists for the requested commodity."""
_make_meter(session, started_at=_T0, commodity="electricity")
assert meter_at(session, _T0, commodity="heating") is None
def test_two_contiguous_epochs_correct_routing(self, session: Session):
"""With two contiguous meters, meter_at routes each ts to the correct epoch."""
t1 = _T0 + timedelta(hours=3)
m0 = _make_meter(session, started_at=_T0, ended_at=t1, label="Meter 0")
m1 = _make_meter(session, started_at=t1, ended_at=None, label="Meter 1")
# ts in first epoch
assert meter_at(session, _T0 + timedelta(hours=1)).id == m0.id
# ts exactly at boundary → second epoch
assert meter_at(session, t1).id == m1.id
# ts in second epoch
assert meter_at(session, t1 + timedelta(hours=1)).id == m1.id
def test_equal_started_at_swap_meter_at_still_returns_new_active(self, session: Session):
"""Regression: after equal-timestamp swap, meter_at must return the new active meter.
When declare_meter is called with started_at == active.started_at (the
"equal-timestamp replace" allowed by §3.5), the old meter becomes a
zero-width epoch [T0, T0). A previous bug caused meter_at to select
the zero-width row first (same started_at, lower rowid) and then fail
the upper-bound check, returning None for *any* ts >= T0. This test
pins the correct behaviour: meter_at(T0) and meter_at(T0+δ) must both
return the new active meter, not None.
"""
# Declare first meter at T0.
declare_meter(session, label="M1 (original)", started_at=_T0, reason="initial")
session.commit()
# Declare second meter at *the same* T0 — equal-timestamp swap.
new_m = declare_meter(
session, label="M2 (replacement)", started_at=_T0, reason="meter_swap"
)
session.commit()
# meter_at at exactly T0 must return the new active meter.
result_at_T0 = meter_at(session, _T0)
assert result_at_T0 is not None, (
"meter_at(T0) returned None after equal-timestamp swap; "
"the new active meter should cover T0"
)
assert result_at_T0.id == new_m.id, (
f"meter_at(T0) returned meter id={result_at_T0.id} (label={result_at_T0.label!r}), "
f"expected id={new_m.id} (the new active meter)"
)
# meter_at slightly after T0 must also return the new active meter.
result_after_T0 = meter_at(session, _T0 + timedelta(seconds=1))
assert result_after_T0 is not None, (
"meter_at(T0+1s) returned None after equal-timestamp swap"
)
assert result_after_T0.id == new_m.id, (
f"meter_at(T0+1s) returned meter id={result_after_T0.id}, "
f"expected id={new_m.id} (the new active meter)"
)
# ---------------------------------------------------------------------------
# 2 & 3. declare_meter — first declaration, swap, mutual exclusion
# ---------------------------------------------------------------------------
class TestDeclareMeter:
def test_first_declaration_no_active(self, session: Session):
"""Declaring the first meter must create an active meter with ended_at IS NULL."""
m = declare_meter(
session,
label="Initial meter",
started_at=_T0,
reason="initial",
)
session.commit()
fetched = session.get(Meter, m.id)
assert fetched is not None
assert fetched.ended_at is None
assert fetched.commodity == "electricity"
assert fetched.reason == "initial"
def test_swap_closes_old_meter(self, session: Session):
"""Declaring a second meter must close the previous active meter."""
first = declare_meter(
session, label="First meter", started_at=_T0, reason="initial"
)
session.commit()
first_id = first.id
t1 = _T0 + timedelta(days=30)
second = declare_meter(
session, label="Second meter", started_at=t1, reason="meter_swap"
)
session.commit()
# First meter must now be closed at exactly t1.
closed = session.get(Meter, first_id)
assert closed.ended_at is not None
from app.services.meters import _as_utc
assert _as_utc(closed.ended_at) == _as_utc(t1)
# Second meter must be active.
assert second.ended_at is None
def test_swap_interval_contiguous(self, session: Session):
"""Old meter's ended_at must exactly equal new meter's started_at after swap."""
declare_meter(session, label="M1", started_at=_T0, reason="initial")
session.commit()
t1 = _T0 + timedelta(days=10)
new_m = declare_meter(session, label="M2", started_at=t1, reason="meter_swap")
session.commit()
# Query the old (now-closed) meter.
meters = list_meters(session, commodity="electricity")
assert len(meters) == 2
old_m = next(m for m in meters if m.id != new_m.id)
from app.services.meters import _as_utc
assert _as_utc(old_m.ended_at) == _as_utc(new_m.started_at), (
"Timeline gap or overlap: old ended_at must == new started_at"
)
def test_at_most_one_active_per_commodity_after_multiple_swaps(self, session: Session):
"""After N swaps, exactly one meter per commodity must be active."""
declare_meter(session, label="M1", started_at=_T0, reason="initial")
session.commit()
declare_meter(
session, label="M2", started_at=_T0 + timedelta(days=10), reason="meter_swap"
)
session.commit()
declare_meter(
session, label="M3", started_at=_T0 + timedelta(days=20), reason="meter_swap"
)
session.commit()
active_meters = [m for m in list_meters(session, commodity="electricity") if m.ended_at is None]
assert len(active_meters) == 1, f"Expected 1 active meter, got {len(active_meters)}"
def test_backdate_rejected(self, session: Session):
"""started_at strictly before active meter's started_at must raise MeterOverlapError."""
declare_meter(session, label="Current", started_at=_T0, reason="initial")
session.commit()
with pytest.raises(MeterOverlapError):
declare_meter(
session,
label="Too early",
started_at=_T0 - timedelta(hours=1),
reason="meter_swap",
)
def test_backdate_raises_before_any_db_write(self, session: Session):
"""When backdate is rejected, no new meter row must be written."""
declare_meter(session, label="Current", started_at=_T0, reason="initial")
session.commit()
meters_before = list_meters(session, commodity="electricity")
count_before = len(meters_before)
with pytest.raises(MeterOverlapError):
declare_meter(
session,
label="Bad meter",
started_at=_T0 - timedelta(minutes=5),
reason="meter_swap",
)
# Rollback the failed operation explicitly (simulating what the caller would do).
session.rollback()
# Re-open session to verify state.
with Session(session.get_bind()) as s2:
meters_after = list_meters(s2, commodity="electricity")
assert len(meters_after) == count_before, (
"No extra meter must be written when backdate is rejected"
)
def test_equal_started_at_allowed(self, session: Session):
"""started_at == active meter's started_at must NOT raise (equal is allowed)."""
declare_meter(session, label="M1", started_at=_T0, reason="initial")
session.commit()
# Should not raise — equal timestamps are valid (replaces meter at same instant).
new_m = declare_meter(
session, label="M2", started_at=_T0, reason="meter_swap"
)
session.commit()
assert new_m.ended_at is None
def test_note_and_commodity_stored(self, session: Session):
"""declare_meter must persist note and commodity correctly."""
m = declare_meter(
session,
label="Gas meter",
started_at=_T0,
reason="initial",
commodity="gas",
note="Rotameter serial XYZ",
)
session.commit()
fetched = session.get(Meter, m.id)
assert fetched.commodity == "gas"
assert fetched.note == "Rotameter serial XYZ"
def test_declare_meter_generates_uuid(self, session: Session):
"""declare_meter must auto-generate a non-empty uuid via ORM default."""
import re
UUID4_RE = re.compile(
r"^[0-9a-f]{8}-[0-9a-f]{4}-4[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$",
re.IGNORECASE,
)
m = declare_meter(
session,
label="Meter with UUID",
started_at=_T0,
reason="initial",
)
session.commit()
fetched = session.get(Meter, m.id)
assert fetched is not None
assert fetched.uuid is not None, "uuid must not be None after declare_meter"
assert fetched.uuid != "", "uuid must not be empty"
assert UUID4_RE.match(fetched.uuid), (
f"uuid {fetched.uuid!r} does not look like a valid UUID v4"
)
def test_declare_meter_each_gets_distinct_uuid(self, session: Session):
"""Each declared meter must receive a distinct UUID (not duplicated)."""
m1 = declare_meter(session, label="M1", started_at=_T0, reason="initial")
session.commit()
t1 = _T0 + timedelta(days=10)
m2 = declare_meter(session, label="M2", started_at=t1, reason="meter_swap")
session.commit()
assert m1.uuid != m2.uuid, (
f"Two declared meters must have distinct UUIDs; both got {m1.uuid!r}"
)
# ---------------------------------------------------------------------------
# 5. Different commodities are independent
# ---------------------------------------------------------------------------
class TestCommodityIsolation:
def test_electricity_swap_does_not_affect_gas_meter(self, session: Session):
"""Swapping the electricity meter must not touch the gas meter's active state."""
declare_meter(session, label="Elec 1", started_at=_T0, reason="initial", commodity="electricity")
declare_meter(session, label="Gas 1", started_at=_T0, reason="initial", commodity="gas")
session.commit()
declare_meter(
session,
label="Elec 2",
started_at=_T0 + timedelta(days=5),
reason="meter_swap",
commodity="electricity",
)
session.commit()
# Gas meter must still be active.
gas_meters = list_meters(session, commodity="gas")
active_gas = [m for m in gas_meters if m.ended_at is None]
assert len(active_gas) == 1, "Gas meter must remain active after electricity swap"
assert active_gas[0].label == "Gas 1"
# Electricity: exactly one active.
elec_meters = list_meters(session, commodity="electricity")
active_elec = [m for m in elec_meters if m.ended_at is None]
assert len(active_elec) == 1
assert active_elec[0].label == "Elec 2"
def test_different_commodity_backdate_is_independent(self, session: Session):
"""Backdate validation is per-commodity: gas meter start does not constrain electricity."""
# Declare gas meter at a later time.
declare_meter(
session, label="Gas 1", started_at=_T0 + timedelta(days=10), reason="initial", commodity="gas"
)
session.commit()
# Declaring an electricity meter at an earlier time must succeed (no gas constraint).
m = declare_meter(
session, label="Elec 1", started_at=_T0, reason="initial", commodity="electricity"
)
session.commit()
assert m is not None
# ---------------------------------------------------------------------------
# 6. list_meters
# ---------------------------------------------------------------------------
class TestListMeters:
def test_returns_empty_list_when_no_meters(self, session: Session):
assert list_meters(session) == []
def test_ordered_by_started_at_asc(self, session: Session):
"""list_meters must return meters in ascending started_at order."""
t1 = _T0 + timedelta(days=10)
t2 = _T0 + timedelta(days=20)
declare_meter(session, label="M1", started_at=_T0, reason="initial")
session.commit()
declare_meter(session, label="M2", started_at=t1, reason="meter_swap")
session.commit()
declare_meter(session, label="M3", started_at=t2, reason="meter_swap")
session.commit()
meters = list_meters(session, commodity="electricity")
assert [m.label for m in meters] == ["M1", "M2", "M3"]
def test_commodity_filter_returns_only_matching(self, session: Session):
"""list_meters with commodity kwarg must filter correctly."""
declare_meter(session, label="Elec", started_at=_T0, reason="initial", commodity="electricity")
declare_meter(session, label="Gas", started_at=_T0, reason="initial", commodity="gas")
session.commit()
elec = list_meters(session, commodity="electricity")
gas = list_meters(session, commodity="gas")
all_meters = list_meters(session)
assert len(elec) == 1 and elec[0].commodity == "electricity"
assert len(gas) == 1 and gas[0].commodity == "gas"
assert len(all_meters) == 2
# ---------------------------------------------------------------------------
# 7 & 8. update_meter
# ---------------------------------------------------------------------------
class TestUpdateMeter:
def test_update_label(self, session: Session):
"""update_meter must update label without touching other fields."""
m = _make_meter(session, started_at=_T0)
update_meter(session, m, label="New label")
session.commit()
fetched = session.get(Meter, m.id)
assert fetched.label == "New label"
assert fetched.note is None # unchanged
def test_update_note(self, session: Session):
"""update_meter must update note without touching other fields."""
m = _make_meter(session, started_at=_T0)
update_meter(session, m, note="Some note")
session.commit()
fetched = session.get(Meter, m.id)
assert fetched.note == "Some note"
assert fetched.label == "Test meter" # unchanged
def test_update_label_and_note_simultaneously(self, session: Session):
"""update_meter must update both label and note in a single call."""
m = _make_meter(session, started_at=_T0)
update_meter(session, m, label="Updated label", note="Updated note")
session.commit()
fetched = session.get(Meter, m.id)
assert fetched.label == "Updated label"
assert fetched.note == "Updated note"
def test_update_started_at_first_meter(self, session: Session):
"""Changing started_at of the first (only) active meter must work without a previous meter."""
m = _make_meter(session, started_at=_T0, ended_at=None)
new_start = _T0 + timedelta(hours=2)
update_meter(session, m, started_at=new_start)
session.commit()
fetched = session.get(Meter, m.id)
from app.services.meters import _as_utc
assert _as_utc(fetched.started_at) == _as_utc(new_start)
def test_update_started_at_updates_previous_ended_at(self, session: Session):
"""Retroactive started_at change must propagate to the previous meter's ended_at."""
t1 = _T0 + timedelta(days=10)
prev = _make_meter(session, started_at=_T0, ended_at=t1, label="Prev meter")
curr = _make_meter(session, started_at=t1, ended_at=None, label="Curr meter")
session.commit()
new_start = _T0 + timedelta(days=7) # shift boundary 3 days earlier
update_meter(session, curr, started_at=new_start)
session.commit()
from app.services.meters import _as_utc
fetched_prev = session.get(Meter, prev.id)
fetched_curr = session.get(Meter, curr.id)
# The previous meter's ended_at must now equal the new started_at.
assert _as_utc(fetched_prev.ended_at) == _as_utc(new_start), (
"Previous meter's ended_at must be updated to maintain continuity"
)
# The current meter's started_at must reflect the change.
assert _as_utc(fetched_curr.started_at) == _as_utc(new_start)
def test_update_started_at_continuity_maintained(self, session: Session):
"""After retroactive started_at change, prev.ended_at == curr.started_at (no gap)."""
t1 = _T0 + timedelta(days=10)
prev = _make_meter(session, started_at=_T0, ended_at=t1, label="Prev")
curr = _make_meter(session, started_at=t1, ended_at=None, label="Curr")
session.commit()
new_start = _T0 + timedelta(days=12) # shift boundary 2 days later
update_meter(session, curr, started_at=new_start)
session.commit()
from app.services.meters import _as_utc
fetched_prev = session.get(Meter, prev.id)
fetched_curr = session.get(Meter, curr.id)
assert _as_utc(fetched_prev.ended_at) == _as_utc(fetched_curr.started_at), (
"Timeline must remain contiguous after retroactive started_at shift"
)
def test_update_started_at_rejects_at_or_past_ended_at(self, session: Session):
"""New started_at ≥ ended_at must raise MeterIntervalError (empty/inverted epoch)."""
t1 = _T0 + timedelta(days=10)
m = _make_meter(session, started_at=_T0, ended_at=t1)
with pytest.raises(MeterIntervalError):
update_meter(session, m, started_at=t1) # == ended_at → empty epoch
with pytest.raises(MeterIntervalError):
update_meter(session, m, started_at=t1 + timedelta(hours=1)) # > ended_at
def test_update_started_at_rejects_at_or_before_prev_started_at(self, session: Session):
"""New started_at ≤ prev.started_at must raise MeterIntervalError."""
t1 = _T0 + timedelta(days=10)
_make_meter(session, started_at=_T0, ended_at=t1, label="Prev")
curr = _make_meter(session, started_at=t1, ended_at=None, label="Curr")
session.commit()
with pytest.raises(MeterIntervalError):
# exactly at prev.started_at — would collapse prev epoch to zero
update_meter(session, curr, started_at=_T0)
with pytest.raises(MeterIntervalError):
# strictly before prev.started_at — inverts prev epoch
update_meter(session, curr, started_at=_T0 - timedelta(hours=1))
def test_noop_call_does_not_raise(self, session: Session):
"""Calling update_meter with all None args must succeed without error."""
m = _make_meter(session, started_at=_T0)
result = update_meter(session, m)
assert result is m
def test_update_started_at_no_previous_meter_shift_backward(self, session: Session):
"""For the first meter (no previous), shifting started_at backward must succeed."""
m = _make_meter(session, started_at=_T0, ended_at=None)
earlier = _T0 - timedelta(days=5)
update_meter(session, m, started_at=earlier)
session.commit()
from app.services.meters import _as_utc
fetched = session.get(Meter, m.id)
assert _as_utc(fetched.started_at) == _as_utc(earlier)
+51
View File
@@ -217,6 +217,57 @@ class TestReadBlocks:
mock_client.close.assert_called_once()
@patch("app.integrations.modbus.driver.ModbusTcpClient")
def test_default_function_code_uses_fc04_input_registers(
self, mock_client_cls: MagicMock
) -> None:
"""With no function_code given, read_blocks uses FC04 (read_input_registers)."""
mock_client = MagicMock()
mock_client_cls.return_value = mock_client
mock_client.connect.return_value = True
mock_client.read_input_registers.return_value = _make_ok_response([0x4366, 0x3334])
read_blocks("127.0.0.1", 502, 1, [{"start": 0x0000, "count": 2}])
mock_client.read_input_registers.assert_called_once_with(0x0000, count=2, device_id=1)
mock_client.read_holding_registers.assert_not_called()
@patch("app.integrations.modbus.driver.ModbusTcpClient")
def test_function_code_3_uses_fc03_holding_registers(
self, mock_client_cls: MagicMock
) -> None:
"""function_code=3 dispatches FC03 (read_holding_registers), e.g. DDSU666."""
mock_client = MagicMock()
mock_client_cls.return_value = mock_client
mock_client.connect.return_value = True
mock_client.read_holding_registers.return_value = _make_ok_response(
[0x4366, 0x3334, 0x3F80, 0x0000]
)
blocks = [{"start": 0x2000, "count": 4}]
result = read_blocks("127.0.0.1", 502, 1, blocks, function_code=3)
assert result == {0x2000: 0x4366, 0x2001: 0x3334, 0x2002: 0x3F80, 0x2003: 0x0000}
mock_client.read_holding_registers.assert_called_once_with(
0x2000, count=4, device_id=1
)
mock_client.read_input_registers.assert_not_called()
@patch("app.integrations.modbus.driver.ModbusTcpClient")
def test_invalid_function_code_raises_before_connecting(
self, mock_client_cls: MagicMock
) -> None:
"""An unsupported function code is rejected without opening a connection."""
mock_client = MagicMock()
mock_client_cls.return_value = mock_client
with pytest.raises(ModbusDriverError, match="Unsupported read function code"):
read_blocks("127.0.0.1", 502, 1, [{"start": 0, "count": 2}], function_code=16)
# No client should have been constructed or connected for a bad FC.
mock_client_cls.assert_not_called()
mock_client.connect.assert_not_called()
@patch("app.integrations.modbus.driver.ModbusTcpClient")
def test_unit_id_passed_as_device_id(self, mock_client_cls: MagicMock) -> None:
"""unit_id is forwarded as device_id= keyword argument (pymodbus 3.13.x)."""
+316
View File
@@ -0,0 +1,316 @@
"""Tests for M6-T06: MqttManager subscribe + on_message dispatch.
All paho interaction is mocked no real broker is required.
Covers:
- subscribe() registers topic handler in the internal registry.
- on_message dispatches the payload to the correct handler.
- A handler that raises does NOT propagate the exception out of on_message.
- Existing publish() behaviour is not regressed.
- When already connected, subscribe() calls client.subscribe() immediately.
- on_connect re-subscribes to all registered topics after (re-)connect.
- subscribe() before connect: topic is subscribed when connect fires.
"""
from __future__ import annotations
from unittest.mock import MagicMock, patch
from app.integrations.mqtt import MqttManager
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _make_settings(
*,
mqtt_enabled: bool = True,
mqtt_broker_host: str = "broker.test",
mqtt_broker_port: int = 1883,
mqtt_username: str = "",
mqtt_password: str = "",
mqtt_tls_enabled: bool = False,
):
s = MagicMock()
s.mqtt_enabled = mqtt_enabled
s.mqtt_broker_host = mqtt_broker_host
s.mqtt_broker_port = mqtt_broker_port
s.mqtt_username = mqtt_username
s.mqtt_password = mqtt_password
s.mqtt_tls_enabled = mqtt_tls_enabled
return s
def _make_mqtt_message(topic: str, payload: bytes) -> MagicMock:
"""Construct a minimal mock of a paho MQTTMessage."""
msg = MagicMock()
msg.topic = topic
msg.payload = payload
return msg
# ---------------------------------------------------------------------------
# subscribe() — registry
# ---------------------------------------------------------------------------
def test_subscribe_registers_handler() -> None:
"""subscribe() must store the handler in the internal _subscriptions dict."""
manager = MqttManager()
handler = MagicMock()
manager.subscribe("test/topic", handler)
assert "test/topic" in manager._subscriptions
assert manager._subscriptions["test/topic"] is handler
def test_subscribe_overwrites_previous_handler() -> None:
"""Subscribing to the same topic twice replaces the handler."""
manager = MqttManager()
h1 = MagicMock()
h2 = MagicMock()
manager.subscribe("test/topic", h1)
manager.subscribe("test/topic", h2)
assert manager._subscriptions["test/topic"] is h2
def test_subscribe_when_not_connected_does_not_call_paho_subscribe() -> None:
"""subscribe() before connect must not attempt to call client.subscribe()."""
real_manager = MqttManager()
handler = MagicMock()
mock_client = MagicMock()
real_manager._client = None
real_manager._connected = False
real_manager.subscribe("dsmr/json", handler)
# No client → subscribe should not have been called on paho
mock_client.subscribe.assert_not_called()
def test_subscribe_when_connected_calls_paho_subscribe_immediately() -> None:
"""subscribe() while connected must call client.subscribe(topic) right away."""
manager = MqttManager()
mock_client = MagicMock()
manager._client = mock_client
manager._connected = True
handler = MagicMock()
manager.subscribe("dsmr/json", handler)
mock_client.subscribe.assert_called_once_with("dsmr/json")
def test_unsubscribe_removes_handler_from_registry() -> None:
"""unsubscribe() must drop the handler so it is not re-subscribed on reconnect."""
manager = MqttManager()
manager.subscribe("dsmr/json", MagicMock())
manager.unsubscribe("dsmr/json")
assert "dsmr/json" not in manager._subscriptions
def test_unsubscribe_when_connected_calls_paho_unsubscribe() -> None:
"""unsubscribe() while connected must call client.unsubscribe(topic)."""
manager = MqttManager()
mock_client = MagicMock()
manager._client = mock_client
manager._connected = True
manager.subscribe("dsmr/json", MagicMock())
manager.unsubscribe("dsmr/json")
mock_client.unsubscribe.assert_called_once_with("dsmr/json")
def test_unsubscribe_unknown_topic_is_noop() -> None:
"""unsubscribe() on a topic that was never registered must not raise."""
manager = MqttManager()
manager.unsubscribe("never/registered") # must not raise
assert "never/registered" not in manager._subscriptions
# ---------------------------------------------------------------------------
# on_message dispatch
# ---------------------------------------------------------------------------
def _extract_on_message(manager: MqttManager, settings) -> object:
"""Connect with a mocked paho client and return the on_message callback that
was registered on the client mock."""
mock_client = MagicMock()
mock_client.connect.return_value = None
with patch("app.integrations.mqtt.mqtt.Client", return_value=mock_client):
manager.connect(settings)
# The callback was assigned as: client.on_message = _on_message
return mock_client.on_message
def test_on_message_dispatches_to_registered_handler() -> None:
"""on_message must invoke the handler registered for the message's topic."""
manager = MqttManager()
settings = _make_settings()
received_payloads: list[bytes] = []
def handler(payload: bytes) -> None:
received_payloads.append(payload)
manager.subscribe("dsmr/json", handler)
on_message = _extract_on_message(manager, settings)
msg = _make_mqtt_message("dsmr/json", b'{"id": 1}')
on_message(MagicMock(), None, msg) # simulate paho calling the callback
assert received_payloads == [b'{"id": 1}']
def test_on_message_ignores_unknown_topic() -> None:
"""on_message for an unregistered topic must not raise and must not call any handler."""
manager = MqttManager()
settings = _make_settings()
handler = MagicMock()
manager.subscribe("other/topic", handler)
on_message = _extract_on_message(manager, settings)
msg = _make_mqtt_message("unknown/topic", b"data")
# Must not raise
on_message(MagicMock(), None, msg)
handler.assert_not_called()
def test_on_message_swallows_handler_exception() -> None:
"""If the handler raises, on_message must NOT propagate the exception."""
manager = MqttManager()
settings = _make_settings()
def bad_handler(payload: bytes) -> None:
raise RuntimeError("handler exploded")
manager.subscribe("dsmr/json", bad_handler)
on_message = _extract_on_message(manager, settings)
msg = _make_mqtt_message("dsmr/json", b"{}")
# This must not raise
on_message(MagicMock(), None, msg)
def test_on_message_does_not_crash_on_handler_exception_multiple_calls() -> None:
"""After a handler exception the manager remains functional for subsequent messages."""
manager = MqttManager()
settings = _make_settings()
call_count = [0]
def flaky_handler(payload: bytes) -> None:
call_count[0] += 1
if call_count[0] == 1:
raise ValueError("first call fails")
manager.subscribe("dsmr/json", flaky_handler)
on_message = _extract_on_message(manager, settings)
msg1 = _make_mqtt_message("dsmr/json", b"first")
msg2 = _make_mqtt_message("dsmr/json", b"second")
on_message(MagicMock(), None, msg1) # should not raise despite exception in handler
on_message(MagicMock(), None, msg2) # handler called again
assert call_count[0] == 2
# ---------------------------------------------------------------------------
# on_connect re-subscribes registered topics
# ---------------------------------------------------------------------------
def test_on_connect_subscribes_all_registered_topics() -> None:
"""When connect fires successfully, _on_connect must subscribe every registered topic."""
manager = MqttManager()
settings = _make_settings()
manager.subscribe("topic/a", MagicMock())
manager.subscribe("topic/b", MagicMock())
mock_client = MagicMock()
mock_client.connect.return_value = None
# Capture the on_connect callback that was registered on the mock client
with patch("app.integrations.mqtt.mqtt.Client", return_value=mock_client):
manager.connect(settings)
on_connect_cb = mock_client.on_connect
# Simulate broker accepting the connection (reason_code.is_failure == False)
reason_code = MagicMock()
reason_code.is_failure = False
on_connect_cb(mock_client, None, MagicMock(), reason_code, None)
# client.subscribe should have been called once per topic during on_connect
subscribe_calls = [c[0][0] for c in mock_client.subscribe.call_args_list]
assert "topic/a" in subscribe_calls
assert "topic/b" in subscribe_calls
def test_on_connect_does_not_subscribe_on_failure() -> None:
"""When the broker rejects the connection, _on_connect must NOT subscribe."""
manager = MqttManager()
settings = _make_settings()
manager.subscribe("topic/a", MagicMock())
mock_client = MagicMock()
mock_client.connect.return_value = None
with patch("app.integrations.mqtt.mqtt.Client", return_value=mock_client):
manager.connect(settings)
on_connect_cb = mock_client.on_connect
reason_code = MagicMock()
reason_code.is_failure = True
mock_client.subscribe.reset_mock()
on_connect_cb(mock_client, None, MagicMock(), reason_code, None)
mock_client.subscribe.assert_not_called()
# ---------------------------------------------------------------------------
# Existing publish() is not regressed
# ---------------------------------------------------------------------------
def test_publish_still_works_after_subscribe_registered() -> None:
"""Adding a subscription must not break publish() behaviour."""
manager = MqttManager()
mock_client = MagicMock()
mock_client.connect.return_value = None
manager._client = mock_client
manager._connected = True
manager.subscribe("dsmr/json", MagicMock())
manager.publish("home/sensor", b"data", retain=True, qos=1)
mock_client.publish.assert_called_once_with(
"home/sensor", payload=b"data", qos=1, retain=True
)
def test_publish_skipped_when_not_connected_regression() -> None:
"""publish() must remain a no-op when not connected, even with subscriptions."""
manager = MqttManager()
manager.subscribe("dsmr/json", MagicMock())
# No client set up — publish should be silent
manager.publish("topic", b"payload") # must not raise
+415
View File
@@ -0,0 +1,415 @@
"""Tests for app/integrations/pricing/profiles.py.
Acceptance criteria covered
----------------------------
1. ``load_profile("manual")`` succeeds and returns a valid ``ManualProfile``.
2. ``load_profile("tibber")`` succeeds and returns a valid ``TibberProfile``.
3. Missing profile file raises ``ProfileNotFoundError``.
4. Malformed YAML (missing required fields) raises ``ProfileValidationError``.
5. Wrong kind in YAML raises ``ProfileValidationError``.
6. ``validate_values`` accepts conforming values for both kinds.
7. ``validate_values`` fills in default values (``ode``, ``sell_adjust``,
tibber ``management_fee``) when absent.
8. ``validate_values`` raises ``ProfileValidationError`` for missing required
fields (no default).
9. ``validate_values`` raises ``ProfileValidationError`` for wrong-typed fields.
10. ``list_profiles()`` returns both profiles in a list of dicts.
"""
from __future__ import annotations
import textwrap
from pathlib import Path
from unittest.mock import patch
import pytest
import yaml
from app.integrations.pricing.profiles import (
ManualProfile,
ProfileNotFoundError,
ProfileValidationError,
TibberProfile,
list_profiles,
load_profile,
validate_values,
)
# ---------------------------------------------------------------------------
# 1-2: load_profile happy path
# ---------------------------------------------------------------------------
class TestLoadProfileManual:
"""Validate that the shipped manual.yaml loads and validates correctly."""
def test_returns_manual_profile_instance(self) -> None:
profile = load_profile("manual")
assert isinstance(profile, ManualProfile)
def test_kind_is_manual(self) -> None:
profile = load_profile("manual")
assert profile.kind == "manual"
def test_has_label(self) -> None:
profile = load_profile("manual")
assert isinstance(profile.label, str) and profile.label
def test_dual_tariff_is_true(self) -> None:
profile = load_profile("manual")
assert profile.energy.dual_tariff is True
def test_energy_buy_has_normal_and_dal(self) -> None:
profile = load_profile("manual")
assert profile.energy.buy.normal.unit == "EUR/kWh"
assert profile.energy.buy.dal.unit == "EUR/kWh"
def test_energy_sell_has_normal_and_dal(self) -> None:
profile = load_profile("manual")
assert profile.energy.sell.normal.unit == "EUR/kWh"
assert profile.energy.sell.dal.unit == "EUR/kWh"
def test_energy_tax_unit(self) -> None:
profile = load_profile("manual")
assert profile.energy.energy_tax.unit == "EUR/kWh"
assert profile.energy.energy_tax.default is None # required field, no default
def test_ode_has_default_zero(self) -> None:
profile = load_profile("manual")
assert profile.energy.ode.default == 0
def test_standing_fields(self) -> None:
profile = load_profile("manual")
assert profile.standing.network_fee.unit == "EUR/month"
assert profile.standing.management_fee.unit == "EUR/month"
def test_credits_heffingskorting(self) -> None:
profile = load_profile("manual")
assert profile.credits.heffingskorting.unit == "EUR/year"
def test_no_concrete_values_in_profile(self) -> None:
"""Profile YAML must not contain any concrete price numbers."""
profile_path = (
Path(__file__).parent.parent
/ "app/integrations/pricing/profiles/manual.yaml"
)
with profile_path.open() as fh:
raw = yaml.safe_load(fh)
# Leaf nodes should only have 'unit' and optionally 'default: 0',
# not actual price values like 0.133 or 0.127.
energy = raw.get("energy", {})
buy = energy.get("buy", {})
# Leaf buy nodes: only 'unit' key, no numeric value key.
assert set(buy["normal"].keys()) == {"unit"}, (
"buy.normal leaf must only have 'unit'"
)
assert set(buy["dal"].keys()) == {"unit"}, (
"buy.dal leaf must only have 'unit'"
)
class TestLoadProfileTibber:
"""Validate that the shipped tibber.yaml loads and validates correctly."""
def test_returns_tibber_profile_instance(self) -> None:
profile = load_profile("tibber")
assert isinstance(profile, TibberProfile)
def test_kind_is_tibber(self) -> None:
profile = load_profile("tibber")
assert profile.kind == "tibber"
def test_has_label(self) -> None:
profile = load_profile("tibber")
assert isinstance(profile.label, str) and profile.label
def test_energy_source_is_tibber_api(self) -> None:
profile = load_profile("tibber")
assert profile.energy.source == "tibber_api"
def test_energy_tax_unit(self) -> None:
profile = load_profile("tibber")
assert profile.energy.energy_tax.unit == "EUR/kWh"
assert profile.energy.energy_tax.default is None # required
def test_sell_adjust_has_default_zero(self) -> None:
profile = load_profile("tibber")
assert profile.energy.sell_adjust.default == 0
def test_management_fee_has_default(self) -> None:
profile = load_profile("tibber")
assert profile.standing.management_fee.default is not None
assert isinstance(profile.standing.management_fee.default, float)
def test_network_fee_unit(self) -> None:
profile = load_profile("tibber")
assert profile.standing.network_fee.unit == "EUR/month"
def test_credits_heffingskorting(self) -> None:
profile = load_profile("tibber")
assert profile.credits.heffingskorting.unit == "EUR/year"
# ---------------------------------------------------------------------------
# 3-5: load_profile error cases
# ---------------------------------------------------------------------------
class TestLoadProfileErrors:
def test_missing_profile_raises_not_found(self) -> None:
with pytest.raises(ProfileNotFoundError, match="nonexistent"):
load_profile("nonexistent")
def test_missing_required_fields_raises_validation_error(
self, tmp_path: Path
) -> None:
"""A YAML missing required fields raises ProfileValidationError."""
bad_yaml = textwrap.dedent(
"""\
kind: manual
label: Bad manual profile
# missing energy / standing / credits sections entirely
"""
)
bad_path = tmp_path / "manual.yaml"
bad_path.write_text(bad_yaml)
with patch("app.integrations.pricing.profiles._PROFILES_DIR", tmp_path):
with pytest.raises(ProfileValidationError, match="manual"):
load_profile("manual")
def test_wrong_type_raises_validation_error(self, tmp_path: Path) -> None:
"""A YAML with a wrong type (string where dict is expected) raises ProfileValidationError."""
bad_yaml = textwrap.dedent(
"""\
kind: tibber
label: Wrong type profile
energy: "should be a mapping not a string"
standing:
management_fee: { unit: EUR/month, default: 5.99 }
network_fee: { unit: EUR/month }
credits:
heffingskorting: { unit: EUR/year }
"""
)
bad_path = tmp_path / "tibber.yaml"
bad_path.write_text(bad_yaml)
with patch("app.integrations.pricing.profiles._PROFILES_DIR", tmp_path):
with pytest.raises(ProfileValidationError, match="tibber"):
load_profile("tibber")
def test_unknown_kind_raises_validation_error(self, tmp_path: Path) -> None:
"""A YAML with an unknown kind raises ProfileValidationError."""
bad_yaml = textwrap.dedent(
"""\
kind: unknown_kind
label: Unknown kind profile
"""
)
bad_path = tmp_path / "unknown_kind.yaml"
bad_path.write_text(bad_yaml)
with patch("app.integrations.pricing.profiles._PROFILES_DIR", tmp_path):
with pytest.raises(ProfileValidationError, match="unknown_kind"):
load_profile("unknown_kind")
# ---------------------------------------------------------------------------
# 6-9: validate_values
# ---------------------------------------------------------------------------
# A complete, conforming set of manual contract values.
_VALID_MANUAL_VALUES = {
"energy": {
"buy": {"normal": 0.133, "dal": 0.127},
"sell": {"normal": 0.05, "dal": 0.05},
"energy_tax": 0.1108,
"ode": 0.0,
},
"standing": {
"network_fee": 9.87,
"management_fee": 9.87,
},
"credits": {
"heffingskorting": 600.0,
},
}
# A complete, conforming set of tibber contract values.
_VALID_TIBBER_VALUES = {
"energy": {
"energy_tax": 0.1108,
"sell_adjust": 0.0,
},
"standing": {
"management_fee": 5.99,
"network_fee": 9.87,
},
"credits": {
"heffingskorting": 600.0,
},
}
class TestValidateValuesManual:
def test_valid_values_accepted(self) -> None:
filled = validate_values("manual", dict(_VALID_MANUAL_VALUES))
# Should not raise and should return a dict.
assert isinstance(filled, dict)
def test_ode_default_applied_when_absent(self) -> None:
"""When 'ode' is not in values, the default (0) should be inserted."""
values = {
"energy": {
"buy": {"normal": 0.133, "dal": 0.127},
"sell": {"normal": 0.05, "dal": 0.05},
"energy_tax": 0.1108,
# ode is absent
},
"standing": {
"network_fee": 9.87,
"management_fee": 9.87,
},
"credits": {"heffingskorting": 600.0},
}
filled = validate_values("manual", values)
assert filled["energy"]["ode"] == 0
def test_missing_energy_tax_raises(self) -> None:
values = {
"energy": {
"buy": {"normal": 0.133, "dal": 0.127},
"sell": {"normal": 0.05, "dal": 0.05},
# energy_tax absent — no default
},
"standing": {"network_fee": 9.87, "management_fee": 9.87},
"credits": {"heffingskorting": 600.0},
}
with pytest.raises(ProfileValidationError, match="energy_tax"):
validate_values("manual", values)
def test_missing_buy_normal_raises(self) -> None:
values = {
"energy": {
"buy": {"dal": 0.127}, # normal absent
"sell": {"normal": 0.05, "dal": 0.05},
"energy_tax": 0.1108,
"ode": 0.0,
},
"standing": {"network_fee": 9.87, "management_fee": 9.87},
"credits": {"heffingskorting": 600.0},
}
with pytest.raises(ProfileValidationError, match="normal"):
validate_values("manual", values)
def test_wrong_type_for_energy_tax_raises(self) -> None:
values = {
"energy": {
"buy": {"normal": 0.133, "dal": 0.127},
"sell": {"normal": 0.05, "dal": 0.05},
"energy_tax": "not_a_number", # wrong type
"ode": 0.0,
},
"standing": {"network_fee": 9.87, "management_fee": 9.87},
"credits": {"heffingskorting": 600.0},
}
with pytest.raises(ProfileValidationError, match="energy_tax"):
validate_values("manual", values)
def test_missing_heffingskorting_raises(self) -> None:
values = {
"energy": {
"buy": {"normal": 0.133, "dal": 0.127},
"sell": {"normal": 0.05, "dal": 0.05},
"energy_tax": 0.1108,
"ode": 0.0,
},
"standing": {"network_fee": 9.87, "management_fee": 9.87},
"credits": {}, # heffingskorting absent — no default
}
with pytest.raises(ProfileValidationError, match="heffingskorting"):
validate_values("manual", values)
class TestValidateValuesTibber:
def test_valid_values_accepted(self) -> None:
filled = validate_values("tibber", dict(_VALID_TIBBER_VALUES))
assert isinstance(filled, dict)
def test_sell_adjust_default_applied_when_absent(self) -> None:
values = {
"energy": {
"energy_tax": 0.1108,
# sell_adjust absent — has default 0
},
"standing": {"management_fee": 5.99, "network_fee": 9.87},
"credits": {"heffingskorting": 600.0},
}
filled = validate_values("tibber", values)
assert filled["energy"]["sell_adjust"] == 0
def test_management_fee_default_applied_when_absent(self) -> None:
values = {
"energy": {"energy_tax": 0.1108, "sell_adjust": 0.0},
"standing": {
"network_fee": 9.87,
# management_fee absent — has a default
},
"credits": {"heffingskorting": 600.0},
}
filled = validate_values("tibber", values)
assert "management_fee" in filled["standing"]
assert isinstance(filled["standing"]["management_fee"], float)
def test_missing_energy_tax_raises(self) -> None:
values = {
"energy": {"sell_adjust": 0.0}, # energy_tax absent — no default
"standing": {"management_fee": 5.99, "network_fee": 9.87},
"credits": {"heffingskorting": 600.0},
}
with pytest.raises(ProfileValidationError, match="energy_tax"):
validate_values("tibber", values)
def test_missing_network_fee_raises(self) -> None:
values = {
"energy": {"energy_tax": 0.1108, "sell_adjust": 0.0},
"standing": {"management_fee": 5.99}, # network_fee absent — no default
"credits": {"heffingskorting": 600.0},
}
with pytest.raises(ProfileValidationError, match="network_fee"):
validate_values("tibber", values)
def test_wrong_type_for_sell_adjust_raises(self) -> None:
values = {
"energy": {"energy_tax": 0.1108, "sell_adjust": "zero"}, # wrong type
"standing": {"management_fee": 5.99, "network_fee": 9.87},
"credits": {"heffingskorting": 600.0},
}
with pytest.raises(ProfileValidationError, match="sell_adjust"):
validate_values("tibber", values)
# ---------------------------------------------------------------------------
# 10: list_profiles
# ---------------------------------------------------------------------------
class TestListProfiles:
def test_returns_list_of_dicts(self) -> None:
profiles = list_profiles()
assert isinstance(profiles, list)
for item in profiles:
assert isinstance(item, dict)
def test_contains_manual_and_tibber(self) -> None:
profiles = list_profiles()
kinds = {p["kind"] for p in profiles}
assert "manual" in kinds
assert "tibber" in kinds
def test_each_entry_has_label(self) -> None:
profiles = list_profiles()
for p in profiles:
assert "label" in p and isinstance(p["label"], str)
+553
View File
@@ -0,0 +1,553 @@
"""Tests for app/integrations/pricing/strategies.py.
Acceptance criteria covered
----------------------------
1. ``get_strategy("manual")`` / ``get_strategy("tibber")`` return registered callables.
2. Manual strategy: dual-tariff import/export/net calculated correctly (hand-verified).
3. Manual strategy: Decimal precision no float binary rounding errors.
4. Tibber strategy: queries the most recent TibberPrice with starts_at t0.
5. Tibber strategy: buy=total, sell=totalenergy_taxsell_adjust.
6. Tibber strategy: negative total negative export_revenue (not clamped).
7. Tibber strategy: raises TibberPriceNotFoundError when no matching row exists.
8. ``register_strategy`` / ``get_strategy`` round-trip works.
9. ``get_strategy`` raises KeyError for unknown kinds.
"""
from __future__ import annotations
from datetime import datetime, timezone
from decimal import Decimal
from pathlib import Path
import pytest
from alembic import command
from alembic.config import Config
from sqlalchemy import create_engine
from sqlalchemy.orm import Session
from app.integrations.pricing.strategies import (
PeriodDeltas,
TibberPriceNotFoundError,
get_strategy,
register_strategy,
)
from app.models.energy import TibberPrice
# ---------------------------------------------------------------------------
# Fixtures: in-memory / temp-file SQLite with energy tables
# ---------------------------------------------------------------------------
def _make_app_alembic_config(database_url: str) -> Config:
cfg = Config("alembic_app.ini")
cfg.set_main_option("sqlalchemy.url", database_url)
return cfg
@pytest.fixture()
def tibber_db(tmp_path: Path):
"""Temporary SQLite DB upgraded to head (has tibber_price table)."""
db_path = tmp_path / "tibber_strategy_test.db"
db_url = f"sqlite:///{db_path}"
alembic_cfg = _make_app_alembic_config(db_url)
command.upgrade(alembic_cfg, "head")
engine = create_engine(db_url, connect_args={"check_same_thread": False})
yield engine
engine.dispose()
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
_UTC = timezone.utc
def _ts(hour: int, minute: int = 0) -> datetime:
"""Return a UTC datetime on 2026-06-23 at the given hour:minute."""
return datetime(2026, 6, 23, hour, minute, 0, tzinfo=_UTC)
def _insert_tibber_price(
session: Session,
starts_at: datetime,
total: float,
energy: float = 0.08,
tax: float | None = None,
currency: str = "EUR",
) -> TibberPrice:
"""Insert and flush a TibberPrice row; return the ORM instance."""
if tax is None:
tax = round(total - energy, 6)
row = TibberPrice(
starts_at=starts_at,
resolution="QUARTER_HOURLY",
energy=energy,
tax=tax,
total=total,
level="NORMAL",
currency=currency,
fetched_at=datetime.now(_UTC),
)
session.add(row)
session.flush()
return row
# ---------------------------------------------------------------------------
# 1. Registry round-trip
# ---------------------------------------------------------------------------
class TestRegistry:
def test_manual_strategy_registered(self) -> None:
fn = get_strategy("manual")
assert callable(fn)
def test_tibber_strategy_registered(self) -> None:
fn = get_strategy("tibber")
assert callable(fn)
def test_unknown_kind_raises_key_error(self) -> None:
with pytest.raises(KeyError, match="no_such_kind"):
get_strategy("no_such_kind")
def test_register_and_retrieve_custom_strategy(self) -> None:
def _my_fn(deltas, t0, values, session):
return {}
register_strategy("_test_custom", _my_fn)
assert get_strategy("_test_custom") is _my_fn
# Cleanup to avoid polluting other tests.
from app.integrations.pricing.strategies import _REGISTRY
_REGISTRY.pop("_test_custom", None)
# ---------------------------------------------------------------------------
# 2-3. Manual strategy
# ---------------------------------------------------------------------------
# Dummy session (manual strategy does not use the session).
_NO_SESSION = None # type: ignore[assignment]
class TestManualStrategy:
"""Verify manual dual-tariff price calculations."""
# Contract values for all manual tests.
_VALUES = {
"energy": {
"buy": {"normal": 0.133, "dal": 0.127},
"sell": {"normal": 0.05, "dal": 0.05},
"energy_tax": 0.11,
"ode": 0.0,
},
"standing": {"network_fee": 9.87, "management_fee": 9.87},
"credits": {"heffingskorting": 600.0},
}
def _call(self, deltas: PeriodDeltas) -> dict:
fn = get_strategy("manual")
return fn(deltas, _ts(10), self._VALUES, _NO_SESSION)
# --- hand-calculated expected values ---
# buy_dal = 0.127 + 0.11 + 0.0 = 0.237
# buy_normal = 0.133 + 0.11 + 0.0 = 0.243
# sell_dal = 0.05
# sell_normal = 0.05
#
# With Δd1=2, Δd2=3, Δr1=1, Δr2=4:
# import_cost = 2×0.237 + 3×0.243 = 0.474 + 0.729 = 1.203
# export_revenue = 1×0.05 + 4×0.05 = 0.05 + 0.20 = 0.25
# net_cost = 1.203 0.25 = 0.953
def test_import_cost_dual_tariff(self) -> None:
deltas = PeriodDeltas(
d1=Decimal("2"), d2=Decimal("3"),
r1=Decimal("1"), r2=Decimal("4"),
)
result = self._call(deltas)
expected = Decimal("2") * Decimal("0.237") + Decimal("3") * Decimal("0.243")
assert result["import_cost"] == expected
def test_export_revenue_dual_tariff(self) -> None:
deltas = PeriodDeltas(
d1=Decimal("2"), d2=Decimal("3"),
r1=Decimal("1"), r2=Decimal("4"),
)
result = self._call(deltas)
expected = Decimal("1") * Decimal("0.05") + Decimal("4") * Decimal("0.05")
assert result["export_revenue"] == expected
def test_net_cost_equals_import_minus_export(self) -> None:
deltas = PeriodDeltas(
d1=Decimal("2"), d2=Decimal("3"),
r1=Decimal("1"), r2=Decimal("4"),
)
result = self._call(deltas)
assert result["net_cost"] == result["import_cost"] - result["export_revenue"]
def test_hand_calculated_values(self) -> None:
"""Verify exact hand-calculated result for the reference deltas."""
deltas = PeriodDeltas(
d1=Decimal("2"), d2=Decimal("3"),
r1=Decimal("1"), r2=Decimal("4"),
)
result = self._call(deltas)
assert result["import_cost"] == Decimal("1.203")
assert result["export_revenue"] == Decimal("0.25")
assert result["net_cost"] == Decimal("0.953")
def test_zero_deltas_yields_zero_costs(self) -> None:
deltas = PeriodDeltas(
d1=Decimal("0"), d2=Decimal("0"),
r1=Decimal("0"), r2=Decimal("0"),
)
result = self._call(deltas)
assert result["import_cost"] == Decimal("0")
assert result["export_revenue"] == Decimal("0")
assert result["net_cost"] == Decimal("0")
def test_ode_included_in_buy_price(self) -> None:
"""When ode > 0 it is added to the effective buy price."""
values = {
"energy": {
"buy": {"normal": 0.10, "dal": 0.10},
"sell": {"normal": 0.05, "dal": 0.05},
"energy_tax": 0.10,
"ode": 0.01, # non-zero ode
},
"standing": {"network_fee": 0.0, "management_fee": 0.0},
"credits": {"heffingskorting": 0.0},
}
deltas = PeriodDeltas(
d1=Decimal("1"), d2=Decimal("0"),
r1=Decimal("0"), r2=Decimal("0"),
)
fn = get_strategy("manual")
result = fn(deltas, _ts(10), values, _NO_SESSION)
# buy_dal = 0.10 + 0.10 + 0.01 = 0.21; import_cost = 1 × 0.21 = 0.21
assert result["import_cost"] == Decimal("0.21")
def test_import_export_kept_separate(self) -> None:
"""import_cost and export_revenue must not be netted before assignment."""
deltas = PeriodDeltas(
d1=Decimal("5"), d2=Decimal("5"),
r1=Decimal("3"), r2=Decimal("3"),
)
result = self._call(deltas)
# Both must be individually non-zero.
assert result["import_cost"] > 0
assert result["export_revenue"] > 0
def test_pricing_snapshot_present(self) -> None:
deltas = PeriodDeltas(
d1=Decimal("1"), d2=Decimal("1"),
r1=Decimal("0"), r2=Decimal("0"),
)
result = self._call(deltas)
snapshot = result["pricing"]
assert snapshot["kind"] == "manual"
assert "buy_dal" in snapshot
assert "buy_normal" in snapshot
assert "sell_dal" in snapshot
assert "sell_normal" in snapshot
# ---------------------------------------------------------------------------
# 3. Decimal precision — float-error exposure test
# ---------------------------------------------------------------------------
class TestManualDecimalPrecision:
"""Use values known to produce binary float errors if float arithmetic is used."""
def test_no_float_rounding_error(self) -> None:
"""0.1 + 0.2 in float gives 0.30000000000000004; Decimal must be exact."""
values = {
"energy": {
"buy": {"normal": 0.1, "dal": 0.2},
"sell": {"normal": 0.1, "dal": 0.1},
"energy_tax": 0.0,
"ode": 0.0,
},
"standing": {"network_fee": 0.0, "management_fee": 0.0},
"credits": {"heffingskorting": 0.0},
}
deltas = PeriodDeltas(
d1=Decimal("1"), d2=Decimal("1"),
r1=Decimal("0"), r2=Decimal("0"),
)
fn = get_strategy("manual")
result = fn(deltas, _ts(10), values, _NO_SESSION)
# import_cost = 1×0.2 + 1×0.1 = 0.3 exactly
assert result["import_cost"] == Decimal("0.3"), (
f"Expected Decimal('0.3'), got {result['import_cost']!r}"
"float arithmetic leaking in?"
)
def test_result_values_are_decimal_type(self) -> None:
values = {
"energy": {
"buy": {"normal": 0.133, "dal": 0.127},
"sell": {"normal": 0.05, "dal": 0.05},
"energy_tax": 0.11,
"ode": 0.0,
},
"standing": {"network_fee": 9.87, "management_fee": 9.87},
"credits": {"heffingskorting": 600.0},
}
deltas = PeriodDeltas(
d1=Decimal("1"), d2=Decimal("1"),
r1=Decimal("1"), r2=Decimal("1"),
)
fn = get_strategy("manual")
result = fn(deltas, _ts(10), values, _NO_SESSION)
assert isinstance(result["import_cost"], Decimal)
assert isinstance(result["export_revenue"], Decimal)
assert isinstance(result["net_cost"], Decimal)
# ---------------------------------------------------------------------------
# 4-7. Tibber strategy
# ---------------------------------------------------------------------------
class TestTibberStrategy:
"""Verify Tibber price-lookup and billing calculations."""
_VALUES = {
"energy": {
"energy_tax": 0.10,
"sell_adjust": 0.0,
},
"standing": {"management_fee": 5.99, "network_fee": 9.87},
"credits": {"heffingskorting": 600.0},
}
def _call(
self,
deltas: PeriodDeltas,
t0: datetime,
session: Session,
values: dict | None = None,
) -> dict:
fn = get_strategy("tibber")
return fn(deltas, t0, values or self._VALUES, session)
def test_buy_equals_total(self, tibber_db) -> None:
t0 = _ts(10, 0)
with Session(tibber_db) as session:
_insert_tibber_price(session, starts_at=_ts(9, 45), total=0.25)
session.commit()
with Session(tibber_db) as session:
deltas = PeriodDeltas(
d1=Decimal("1"), d2=Decimal("1"),
r1=Decimal("0"), r2=Decimal("0"),
)
result = self._call(deltas, t0, session)
# buy = total = 0.25; import_cost = 2 × 0.25 = 0.50
assert result["import_cost"] == Decimal("2") * Decimal("0.25")
def test_sell_equals_total_minus_energy_tax_minus_adjust(self, tibber_db) -> None:
t0 = _ts(10, 0)
with Session(tibber_db) as session:
_insert_tibber_price(session, starts_at=_ts(9, 45), total=0.25)
session.commit()
values = {
"energy": {"energy_tax": 0.10, "sell_adjust": 0.02},
"standing": {"management_fee": 5.99, "network_fee": 9.87},
"credits": {"heffingskorting": 600.0},
}
with Session(tibber_db) as session:
deltas = PeriodDeltas(
d1=Decimal("0"), d2=Decimal("0"),
r1=Decimal("1"), r2=Decimal("1"),
)
result = self._call(deltas, t0, session, values=values)
# sell = 0.25 - 0.10 - 0.02 = 0.13; export_revenue = 2 × 0.13 = 0.26
assert result["export_revenue"] == Decimal("2") * Decimal("0.13")
def test_uses_most_recent_price_before_t0(self, tibber_db) -> None:
"""Correct row: starts_at ≤ t0, most recent wins."""
t0 = _ts(10, 0)
with Session(tibber_db) as session:
# Older price (should NOT be used)
_insert_tibber_price(session, starts_at=_ts(9, 0), total=0.10)
# Closer price (starts_at ≤ t0, should be used)
_insert_tibber_price(session, starts_at=_ts(9, 45), total=0.30)
# Future price (starts_at > t0, must NOT be used)
_insert_tibber_price(session, starts_at=_ts(10, 15), total=0.99)
session.commit()
with Session(tibber_db) as session:
deltas = PeriodDeltas(
d1=Decimal("1"), d2=Decimal("0"),
r1=Decimal("0"), r2=Decimal("0"),
)
result = self._call(deltas, t0, session)
# Only the 09:45 price (total=0.30) should be used.
snapshot = result["pricing"]
assert Decimal(snapshot["total"]) == Decimal("0.30")
def test_tibber_sums_both_tariff_registers(self, tibber_db) -> None:
"""Tibber does not split dal/normal; import_cost = (d1+d2) × buy."""
t0 = _ts(10, 0)
with Session(tibber_db) as session:
_insert_tibber_price(session, starts_at=_ts(9, 45), total=0.20)
session.commit()
with Session(tibber_db) as session:
deltas = PeriodDeltas(
d1=Decimal("3"), d2=Decimal("2"),
r1=Decimal("0"), r2=Decimal("0"),
)
result = self._call(deltas, t0, session)
# import_cost = (3+2) × 0.20 = 1.00
assert result["import_cost"] == Decimal("1.00")
def test_negative_total_gives_negative_export_revenue(self, tibber_db) -> None:
"""When total is negative, selling electricity costs money (correct behaviour)."""
t0 = _ts(10, 0)
with Session(tibber_db) as session:
# total = -0.05; sell = -0.05 - 0.10 - 0.0 = -0.15
_insert_tibber_price(session, starts_at=_ts(9, 45), total=-0.05)
session.commit()
with Session(tibber_db) as session:
deltas = PeriodDeltas(
d1=Decimal("0"), d2=Decimal("0"),
r1=Decimal("2"), r2=Decimal("2"),
)
result = self._call(deltas, t0, session)
# sell = -0.05 - 0.10 - 0.0 = -0.15; export_revenue = 4 × (-0.15) = -0.60
assert result["export_revenue"] < 0
assert result["export_revenue"] == Decimal("4") * (
Decimal("-0.05") - Decimal("0.10") - Decimal("0.0")
)
def test_negative_total_exact_calculation(self, tibber_db) -> None:
"""Full hand-calculation for negative-total scenario."""
t0 = _ts(10, 0)
total = Decimal("-0.05")
energy_tax = Decimal("0.10")
sell_adjust = Decimal("0.0")
with Session(tibber_db) as session:
_insert_tibber_price(session, starts_at=_ts(9, 45), total=float(total))
session.commit()
with Session(tibber_db) as session:
deltas = PeriodDeltas(
d1=Decimal("1"), d2=Decimal("1"), # delivered = 2 kWh
r1=Decimal("1"), r2=Decimal("1"), # returned = 2 kWh
)
result = self._call(deltas, t0, session)
buy = total # -0.05
sell = total - energy_tax - sell_adjust # -0.15
expected_import = Decimal("2") * buy # -0.10
expected_export = Decimal("2") * sell # -0.30
expected_net = expected_import - expected_export # 0.20
assert result["import_cost"] == expected_import
assert result["export_revenue"] == expected_export
assert result["net_cost"] == expected_net
def test_no_price_before_t0_raises(self, tibber_db) -> None:
"""When no TibberPrice exists with starts_at ≤ t0, TibberPriceNotFoundError is raised."""
t0 = _ts(10, 0)
with Session(tibber_db) as session:
# Only a future price — starts_at > t0.
_insert_tibber_price(session, starts_at=_ts(10, 15), total=0.25)
session.commit()
fn = get_strategy("tibber")
with Session(tibber_db) as session:
deltas = PeriodDeltas(
d1=Decimal("1"), d2=Decimal("0"),
r1=Decimal("0"), r2=Decimal("0"),
)
with pytest.raises(TibberPriceNotFoundError):
fn(deltas, t0, self._VALUES, session)
def test_empty_tibber_table_raises(self, tibber_db) -> None:
"""Empty tibber_price table raises TibberPriceNotFoundError."""
fn = get_strategy("tibber")
with Session(tibber_db) as session:
deltas = PeriodDeltas(
d1=Decimal("1"), d2=Decimal("0"),
r1=Decimal("0"), r2=Decimal("0"),
)
with pytest.raises(TibberPriceNotFoundError):
fn(deltas, _ts(10), self._VALUES, session)
def test_pricing_snapshot_contains_expected_keys(self, tibber_db) -> None:
t0 = _ts(10, 0)
with Session(tibber_db) as session:
_insert_tibber_price(session, starts_at=_ts(9, 45), total=0.20)
session.commit()
with Session(tibber_db) as session:
deltas = PeriodDeltas(
d1=Decimal("1"), d2=Decimal("0"),
r1=Decimal("0"), r2=Decimal("0"),
)
result = self._call(deltas, t0, session)
snapshot = result["pricing"]
assert snapshot["kind"] == "tibber"
assert "tibber_price_starts_at" in snapshot
assert "buy" in snapshot
assert "sell" in snapshot
def test_result_values_are_decimal_type(self, tibber_db) -> None:
t0 = _ts(10, 0)
with Session(tibber_db) as session:
_insert_tibber_price(session, starts_at=_ts(9, 45), total=0.20)
session.commit()
with Session(tibber_db) as session:
deltas = PeriodDeltas(
d1=Decimal("1"), d2=Decimal("1"),
r1=Decimal("1"), r2=Decimal("1"),
)
result = self._call(deltas, t0, session)
assert isinstance(result["import_cost"], Decimal)
assert isinstance(result["export_revenue"], Decimal)
assert isinstance(result["net_cost"], Decimal)
# ---------------------------------------------------------------------------
# Tibber precision test
# ---------------------------------------------------------------------------
class TestTibberDecimalPrecision:
"""Ensure Tibber arithmetic is exact Decimal (no float leakage)."""
def test_no_float_rounding_for_tricky_values(self, tibber_db) -> None:
"""total=0.1, energy_tax=0.2 — float gives 0.1+0.2 error; Decimal must be exact."""
t0 = _ts(10, 0)
with Session(tibber_db) as session:
_insert_tibber_price(session, starts_at=_ts(9, 45), total=0.3)
session.commit()
values = {
"energy": {"energy_tax": 0.1, "sell_adjust": 0.2},
"standing": {"management_fee": 5.99, "network_fee": 9.87},
"credits": {"heffingskorting": 600.0},
}
fn = get_strategy("tibber")
with Session(tibber_db) as session:
deltas = PeriodDeltas(
d1=Decimal("0"), d2=Decimal("0"),
r1=Decimal("1"), r2=Decimal("0"),
)
result = fn(deltas, t0, values, session)
# sell = 0.3 - 0.1 - 0.2 = 0.0 exactly (float gives ~2.8e-17)
assert result["export_revenue"] == Decimal("0"), (
f"Expected Decimal('0'), got {result['export_revenue']!r}"
"float arithmetic leaking in?"
)
+439
View File
@@ -0,0 +1,439 @@
"""Tests for M6-T05: Tibber API client (app/integrations/tibber/client.py).
Covers:
1. ``fetch_price_range``: happy path parses nodes into PricePoint objects with
UTC-aware starts_at; does not assume a fixed number of nodes (tested with 3).
2. ``fetch_price_range``: HTTP 401 TibberAuthError.
3. ``fetch_price_range``: HTTP 403 TibberAuthError.
4. ``fetch_price_range``: httpx.TimeoutException TibberError.
5. ``fetch_price_range``: generic HTTP error TibberError.
6. Token does not appear in any raised exception's string representation.
7. ``fetch_price_range``: home_id selection (picks correct home).
8. ``fetch_current_price``: happy path returns a single PricePoint.
9. ``fetch_current_price``: HTTP 401 TibberAuthError.
10. GraphQL-level errors in response body TibberError.
All HTTP calls are intercepted via ``httpx.MockTransport`` (httpx built-in) so
no network access is required.
"""
from __future__ import annotations
import json
from datetime import UTC, datetime, timedelta
import httpx
import pytest
from app.integrations.tibber.client import (
PricePoint,
TibberAuthError,
TibberError,
fetch_current_price,
fetch_price_range,
)
# ---------------------------------------------------------------------------
# Helpers / fixture builders
# ---------------------------------------------------------------------------
_FAKE_TOKEN = "fake-secret-token-do-not-log"
# A minimal set of 3 price nodes to verify that the parser does not assume a
# fixed number of nodes (e.g. 96).
_THREE_NODES = [
{
"startsAt": "2026-06-23T00:00:00.000+02:00",
"total": 0.25,
"energy": 0.20,
"tax": 0.05,
"currency": "EUR",
"level": "NORMAL",
},
{
"startsAt": "2026-06-23T00:15:00.000+02:00",
"total": 0.22,
"energy": 0.18,
"tax": 0.04,
"currency": "EUR",
"level": "CHEAP",
},
{
"startsAt": "2026-06-23T00:30:00.000+02:00",
"total": 0.30,
"energy": 0.24,
"tax": 0.06,
"currency": "EUR",
"level": None,
},
]
_PRICE_RANGE_RESPONSE = {
"data": {
"viewer": {
"homes": [
{
"id": "home-id-1",
"currentSubscription": {
"priceInfoRange": {
"nodes": _THREE_NODES,
}
},
}
]
}
}
}
_CURRENT_PRICE_RESPONSE = {
"data": {
"viewer": {
"homes": [
{
"id": "home-id-1",
"currentSubscription": {
"priceInfo": {
"current": {
"startsAt": "2026-06-23T10:00:00.000+02:00",
"total": 0.28,
"energy": 0.22,
"tax": 0.06,
"currency": "EUR",
"level": "EXPENSIVE",
}
}
},
}
]
}
}
}
def _make_transport(status_code: int, body: dict | None = None, raise_timeout: bool = False):
"""Create an ``httpx.MockTransport`` that returns the given response.
If *raise_timeout* is True, the transport raises ``httpx.TimeoutException``
instead of returning a response.
"""
def _handler(request: httpx.Request) -> httpx.Response:
if raise_timeout:
raise httpx.TimeoutException("timed out", request=request)
payload = json.dumps(body or {}).encode()
return httpx.Response(
status_code=status_code,
content=payload,
headers={"Content-Type": "application/json"},
)
return httpx.MockTransport(_handler)
# ---------------------------------------------------------------------------
# fetch_price_range — happy path
# ---------------------------------------------------------------------------
def test_fetch_price_range_parses_nodes(monkeypatch):
"""Successful response with 3 nodes is parsed into 3 PricePoint objects."""
transport = _make_transport(200, _PRICE_RANGE_RESPONSE)
def _patched_post(url, *, json, headers, timeout): # noqa: A002
client = httpx.Client(transport=transport)
return client.post(url, json=json, headers=headers, timeout=timeout)
monkeypatch.setattr("app.integrations.tibber.client.httpx.post", _patched_post)
points = fetch_price_range(_FAKE_TOKEN)
assert len(points) == 3
# All starts_at must be UTC-aware.
for p in points:
assert isinstance(p, PricePoint)
assert p.starts_at.tzinfo is not None
assert p.starts_at.tzinfo == UTC or p.starts_at.utcoffset() == timedelta(0)
# First node: 2026-06-23T00:00:00+02:00 → 2026-06-22T22:00:00Z
first = points[0]
assert first.starts_at == datetime(2026, 6, 22, 22, 0, 0, tzinfo=UTC)
assert first.total == pytest.approx(0.25)
assert first.energy == pytest.approx(0.20)
assert first.tax == pytest.approx(0.05)
assert first.currency == "EUR"
assert first.level == "NORMAL"
assert first.resolution == "QUARTER_HOURLY"
# Third node has level=None in the raw API response.
third = points[2]
assert third.level is None
def test_fetch_price_range_does_not_assume_node_count(monkeypatch):
"""Parser handles an arbitrary number of nodes (not hardcoded to 96)."""
# Build a response with a single node only.
one_node_response = {
"data": {
"viewer": {
"homes": [
{
"id": "home-id-1",
"currentSubscription": {
"priceInfoRange": {
"nodes": [_THREE_NODES[0]],
}
},
}
]
}
}
}
transport = _make_transport(200, one_node_response)
def _patched_post(url, *, json, headers, timeout): # noqa: A002
client = httpx.Client(transport=transport)
return client.post(url, json=json, headers=headers, timeout=timeout)
monkeypatch.setattr("app.integrations.tibber.client.httpx.post", _patched_post)
points = fetch_price_range(_FAKE_TOKEN)
assert len(points) == 1
def test_fetch_price_range_home_id_selection(monkeypatch):
"""When home_id is specified, the matching home is selected."""
two_homes_response = {
"data": {
"viewer": {
"homes": [
{
"id": "home-id-first",
"currentSubscription": {
"priceInfoRange": {
"nodes": [_THREE_NODES[0]],
}
},
},
{
"id": "home-id-second",
"currentSubscription": {
"priceInfoRange": {
"nodes": [_THREE_NODES[1], _THREE_NODES[2]],
}
},
},
]
}
}
}
transport = _make_transport(200, two_homes_response)
def _patched_post(url, *, json, headers, timeout): # noqa: A002
client = httpx.Client(transport=transport)
return client.post(url, json=json, headers=headers, timeout=timeout)
monkeypatch.setattr("app.integrations.tibber.client.httpx.post", _patched_post)
# Selecting the second home should return 2 nodes.
points = fetch_price_range(_FAKE_TOKEN, home_id="home-id-second")
assert len(points) == 2
# Selecting the first home (default when home_id=None) returns 1 node.
points_default = fetch_price_range(_FAKE_TOKEN)
assert len(points_default) == 1
# ---------------------------------------------------------------------------
# fetch_price_range — authentication errors
# ---------------------------------------------------------------------------
@pytest.mark.parametrize("status_code", [401, 403])
def test_fetch_price_range_auth_error(monkeypatch, status_code):
"""HTTP 401 or 403 raises TibberAuthError (subclass of TibberError)."""
transport = _make_transport(status_code, {})
def _patched_post(url, *, json, headers, timeout): # noqa: A002
client = httpx.Client(transport=transport)
return client.post(url, json=json, headers=headers, timeout=timeout)
monkeypatch.setattr("app.integrations.tibber.client.httpx.post", _patched_post)
with pytest.raises(TibberAuthError):
fetch_price_range(_FAKE_TOKEN)
def test_fetch_price_range_auth_error_is_tibber_error(monkeypatch):
"""TibberAuthError is a subclass of TibberError for broad catches."""
transport = _make_transport(401, {})
def _patched_post(url, *, json, headers, timeout): # noqa: A002
client = httpx.Client(transport=transport)
return client.post(url, json=json, headers=headers, timeout=timeout)
monkeypatch.setattr("app.integrations.tibber.client.httpx.post", _patched_post)
with pytest.raises(TibberError):
fetch_price_range(_FAKE_TOKEN)
# ---------------------------------------------------------------------------
# fetch_price_range — network / timeout errors
# ---------------------------------------------------------------------------
def test_fetch_price_range_timeout_raises_tibber_error(monkeypatch):
"""httpx.TimeoutException is wrapped in TibberError."""
transport = _make_transport(200, raise_timeout=True)
def _patched_post(url, *, json, headers, timeout): # noqa: A002
client = httpx.Client(transport=transport)
return client.post(url, json=json, headers=headers, timeout=timeout)
monkeypatch.setattr("app.integrations.tibber.client.httpx.post", _patched_post)
with pytest.raises(TibberError):
fetch_price_range(_FAKE_TOKEN)
def test_fetch_price_range_timeout_not_auth_error(monkeypatch):
"""A timeout should raise TibberError but NOT TibberAuthError."""
transport = _make_transport(200, raise_timeout=True)
def _patched_post(url, *, json, headers, timeout): # noqa: A002
client = httpx.Client(transport=transport)
return client.post(url, json=json, headers=headers, timeout=timeout)
monkeypatch.setattr("app.integrations.tibber.client.httpx.post", _patched_post)
with pytest.raises(TibberError) as exc_info:
fetch_price_range(_FAKE_TOKEN)
assert not isinstance(exc_info.value, TibberAuthError)
def test_fetch_price_range_http_500_raises_tibber_error(monkeypatch):
"""An unexpected 5xx response raises TibberError."""
transport = _make_transport(500, {"error": "internal"})
def _patched_post(url, *, json, headers, timeout): # noqa: A002
client = httpx.Client(transport=transport)
return client.post(url, json=json, headers=headers, timeout=timeout)
monkeypatch.setattr("app.integrations.tibber.client.httpx.post", _patched_post)
with pytest.raises(TibberError):
fetch_price_range(_FAKE_TOKEN)
# ---------------------------------------------------------------------------
# Token safety — token must not appear in exception messages
# ---------------------------------------------------------------------------
def test_token_not_in_auth_error_message(monkeypatch):
"""The API token must not appear in TibberAuthError's string representation."""
transport = _make_transport(401, {})
def _patched_post(url, *, json, headers, timeout): # noqa: A002
client = httpx.Client(transport=transport)
return client.post(url, json=json, headers=headers, timeout=timeout)
monkeypatch.setattr("app.integrations.tibber.client.httpx.post", _patched_post)
with pytest.raises(TibberAuthError) as exc_info:
fetch_price_range(_FAKE_TOKEN)
# The token must not appear anywhere in the exception string.
assert _FAKE_TOKEN not in str(exc_info.value)
assert _FAKE_TOKEN not in repr(exc_info.value)
def test_token_not_in_timeout_error_message(monkeypatch):
"""The API token must not appear in TibberError's string representation on timeout."""
transport = _make_transport(200, raise_timeout=True)
def _patched_post(url, *, json, headers, timeout): # noqa: A002
client = httpx.Client(transport=transport)
return client.post(url, json=json, headers=headers, timeout=timeout)
monkeypatch.setattr("app.integrations.tibber.client.httpx.post", _patched_post)
with pytest.raises(TibberError) as exc_info:
fetch_price_range(_FAKE_TOKEN)
assert _FAKE_TOKEN not in str(exc_info.value)
assert _FAKE_TOKEN not in repr(exc_info.value)
# ---------------------------------------------------------------------------
# GraphQL errors in response body
# ---------------------------------------------------------------------------
def test_graphql_errors_raise_tibber_error(monkeypatch):
"""A 200 response that contains a GraphQL 'errors' field raises TibberError."""
graphql_error_body = {
"errors": [{"message": "Some GraphQL error"}],
"data": None,
}
transport = _make_transport(200, graphql_error_body)
def _patched_post(url, *, json, headers, timeout): # noqa: A002
client = httpx.Client(transport=transport)
return client.post(url, json=json, headers=headers, timeout=timeout)
monkeypatch.setattr("app.integrations.tibber.client.httpx.post", _patched_post)
with pytest.raises(TibberError):
fetch_price_range(_FAKE_TOKEN)
# ---------------------------------------------------------------------------
# fetch_current_price — happy path
# ---------------------------------------------------------------------------
def test_fetch_current_price_returns_single_price_point(monkeypatch):
"""Successful response returns a single UTC-aware PricePoint."""
transport = _make_transport(200, _CURRENT_PRICE_RESPONSE)
def _patched_post(url, *, json, headers, timeout): # noqa: A002
client = httpx.Client(transport=transport)
return client.post(url, json=json, headers=headers, timeout=timeout)
monkeypatch.setattr("app.integrations.tibber.client.httpx.post", _patched_post)
point = fetch_current_price(_FAKE_TOKEN)
assert isinstance(point, PricePoint)
# 2026-06-23T10:00:00+02:00 → 2026-06-23T08:00:00Z
assert point.starts_at == datetime(2026, 6, 23, 8, 0, 0, tzinfo=UTC)
assert point.total == pytest.approx(0.28)
assert point.energy == pytest.approx(0.22)
assert point.tax == pytest.approx(0.06)
assert point.currency == "EUR"
assert point.level == "EXPENSIVE"
assert point.resolution == "QUARTER_HOURLY"
# UTC-aware check
assert point.starts_at.tzinfo is not None
# ---------------------------------------------------------------------------
# fetch_current_price — authentication errors
# ---------------------------------------------------------------------------
def test_fetch_current_price_auth_error(monkeypatch):
"""HTTP 401 on current-price query raises TibberAuthError."""
transport = _make_transport(401, {})
def _patched_post(url, *, json, headers, timeout): # noqa: A002
client = httpx.Client(transport=transport)
return client.post(url, json=json, headers=headers, timeout=timeout)
monkeypatch.setattr("app.integrations.tibber.client.httpx.post", _patched_post)
with pytest.raises(TibberAuthError):
fetch_current_price(_FAKE_TOKEN)
+394
View File
@@ -0,0 +1,394 @@
"""Tests for M6-T05: Tibber price refresh service (app/services/tibber_prices.py).
Covers:
1. With active tibber contract + non-empty token: ``refresh_prices`` calls the
client, upserts returned price points, and returns the count.
2. Idempotency: running ``refresh_prices`` twice does not double-insert rows;
rows are updated in-place (``starts_at`` remains unique).
3. No active contract no-op (returns 0, no DB writes).
4. Active contract with kind="manual" (not tibber) no-op.
5. Token is empty string no-op.
6. Token is whitespace-only no-op.
7. Client errors propagate (not swallowed by the service).
All Tibber API calls are intercepted via monkeypatching ``fetch_price_range``
so no real network access is needed.
"""
from __future__ import annotations
from datetime import UTC, datetime, timedelta
from pathlib import Path
import pytest
from alembic import command
from alembic.config import Config
from sqlalchemy import create_engine, select
from sqlalchemy.orm import Session
from app.integrations.tibber.client import PricePoint, TibberError
from app.models.energy import EnergyContract, EnergyContractVersion, TibberPrice
from app.services.tibber_prices import refresh_prices
# ---------------------------------------------------------------------------
# Fixtures
# ---------------------------------------------------------------------------
def _make_app_alembic_config(database_url: str) -> Config:
cfg = Config("alembic_app.ini")
cfg.set_main_option("sqlalchemy.url", database_url)
return cfg
@pytest.fixture()
def energy_db(tmp_path: Path):
"""Temporary SQLite DB upgraded to head (has all energy tables)."""
db_path = tmp_path / "tibber_prices_test.db"
db_url = f"sqlite:///{db_path}"
alembic_cfg = _make_app_alembic_config(db_url)
command.upgrade(alembic_cfg, "head")
engine = create_engine(db_url, connect_args={"check_same_thread": False})
session = Session(engine)
yield session
session.close()
engine.dispose()
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
_UTC = UTC
def _ts(hour: int, minute: int = 0) -> datetime:
"""Return a UTC datetime on 2026-06-23 at the given hour:minute."""
return datetime(2026, 6, 23, hour, minute, 0, tzinfo=_UTC)
def _make_price_points(count: int = 3) -> list[PricePoint]:
"""Build a list of *count* fake PricePoint objects 15 minutes apart."""
base = _ts(10, 0)
points = []
for i in range(count):
starts_at = base + timedelta(minutes=15 * i)
points.append(
PricePoint(
starts_at=starts_at,
total=0.20 + i * 0.01,
energy=0.16 + i * 0.01,
tax=0.04,
currency="EUR",
level="NORMAL",
resolution="QUARTER_HOURLY",
)
)
return points
class _FakeSettings:
"""Minimal settings stand-in with Tibber fields."""
def __init__(self, token: str = "valid-token", home_id: str = ""):
self.tibber_api_token = token
self.tibber_home_id = home_id
def _create_active_tibber_contract(session: Session) -> EnergyContract:
"""Insert and commit an active tibber contract + one open version."""
now = datetime.now(UTC)
contract = EnergyContract(
name="Tibber Test",
kind="tibber",
active=True,
currency="EUR",
created_at=now,
updated_at=now,
)
session.add(contract)
session.flush()
version = EnergyContractVersion(
contract_id=contract.id,
effective_from=now,
effective_to=None,
values={"energy": {"energy_tax": 0.1108, "sell_adjust": 0.0}, "standing": {}, "credits": {}},
created_at=now,
)
session.add(version)
session.commit()
return contract
def _create_active_manual_contract(session: Session) -> EnergyContract:
"""Insert and commit an active manual contract."""
now = datetime.now(UTC)
contract = EnergyContract(
name="Manual Test",
kind="manual",
active=True,
currency="EUR",
created_at=now,
updated_at=now,
)
session.add(contract)
session.flush()
version = EnergyContractVersion(
contract_id=contract.id,
effective_from=now,
effective_to=None,
values={},
created_at=now,
)
session.add(version)
session.commit()
return contract
def _count_tibber_price_rows(session: Session) -> int:
"""Return the current number of rows in tibber_price."""
return len(session.execute(select(TibberPrice)).scalars().all())
# ---------------------------------------------------------------------------
# Test: happy path (active tibber contract + valid token)
# ---------------------------------------------------------------------------
def test_refresh_prices_upserts_price_points(energy_db, monkeypatch):
"""Active tibber contract + token → price points are upserted and count returned."""
session = energy_db
_create_active_tibber_contract(session)
points = _make_price_points(3)
monkeypatch.setattr(
"app.services.tibber_prices.fetch_price_range",
lambda token, home_id_or_none: points,
)
count = refresh_prices(session, _FakeSettings())
assert count == 3
rows = session.execute(select(TibberPrice).order_by(TibberPrice.starts_at)).scalars().all()
assert len(rows) == 3
for row, point in zip(rows, points):
# starts_at is stored UTC-aware; SQLite may return naive — normalise.
stored_starts_at = row.starts_at
if stored_starts_at.tzinfo is None:
stored_starts_at = stored_starts_at.replace(tzinfo=UTC)
assert stored_starts_at == point.starts_at
assert row.total == pytest.approx(point.total)
assert row.energy == pytest.approx(point.energy)
assert row.tax == pytest.approx(point.tax)
assert row.currency == point.currency
assert row.resolution == "QUARTER_HOURLY"
# ---------------------------------------------------------------------------
# Test: idempotency (running twice must not double-insert)
# ---------------------------------------------------------------------------
def test_refresh_prices_is_idempotent(energy_db, monkeypatch):
"""Running refresh_prices twice does not increase the row count."""
session = energy_db
_create_active_tibber_contract(session)
points = _make_price_points(4)
call_count = {"n": 0}
def _fake_fetch(token, home_id_or_none):
call_count["n"] += 1
return points
monkeypatch.setattr("app.services.tibber_prices.fetch_price_range", _fake_fetch)
count1 = refresh_prices(session, _FakeSettings())
count2 = refresh_prices(session, _FakeSettings())
assert call_count["n"] == 2 # fetch was called twice
assert count1 == 4
assert count2 == 4 # same number of "upserted" (all updated in-place)
# DB must still have exactly 4 rows (not 8).
assert _count_tibber_price_rows(session) == 4
def test_refresh_prices_idempotent_updates_fields(energy_db, monkeypatch):
"""On second run, existing rows are updated (not duplicated)."""
session = energy_db
_create_active_tibber_contract(session)
points = _make_price_points(2)
monkeypatch.setattr(
"app.services.tibber_prices.fetch_price_range",
lambda token, home_id_or_none: points,
)
refresh_prices(session, _FakeSettings())
# Modify the points to simulate Tibber updating a price.
updated_points = [
PricePoint(
starts_at=p.starts_at,
total=p.total + 0.10, # price changed
energy=p.energy + 0.08,
tax=p.tax + 0.02,
currency=p.currency,
level="EXPENSIVE",
resolution=p.resolution,
)
for p in points
]
monkeypatch.setattr(
"app.services.tibber_prices.fetch_price_range",
lambda token, home_id_or_none: updated_points,
)
refresh_prices(session, _FakeSettings())
rows = session.execute(select(TibberPrice).order_by(TibberPrice.starts_at)).scalars().all()
assert len(rows) == 2 # still 2, not 4
for row, up in zip(rows, updated_points):
assert row.total == pytest.approx(up.total)
assert row.level == "EXPENSIVE"
# ---------------------------------------------------------------------------
# Test: no-op conditions
# ---------------------------------------------------------------------------
def test_refresh_prices_noop_no_active_contract(energy_db, monkeypatch):
"""No active contract at all → no-op (returns 0, no DB writes)."""
session = energy_db
fetch_called = {"called": False}
def _fake_fetch(token, home_id_or_none):
fetch_called["called"] = True
return _make_price_points(3)
monkeypatch.setattr("app.services.tibber_prices.fetch_price_range", _fake_fetch)
count = refresh_prices(session, _FakeSettings())
assert count == 0
assert not fetch_called["called"]
assert _count_tibber_price_rows(session) == 0
def test_refresh_prices_noop_manual_contract(energy_db, monkeypatch):
"""Active manual contract (not tibber) → no-op."""
session = energy_db
_create_active_manual_contract(session)
fetch_called = {"called": False}
def _fake_fetch(token, home_id_or_none):
fetch_called["called"] = True
return _make_price_points(3)
monkeypatch.setattr("app.services.tibber_prices.fetch_price_range", _fake_fetch)
count = refresh_prices(session, _FakeSettings())
assert count == 0
assert not fetch_called["called"]
assert _count_tibber_price_rows(session) == 0
def test_refresh_prices_noop_empty_token(energy_db, monkeypatch):
"""Empty token → no-op even if tibber contract is active."""
session = energy_db
_create_active_tibber_contract(session)
fetch_called = {"called": False}
def _fake_fetch(token, home_id_or_none):
fetch_called["called"] = True
return _make_price_points(3)
monkeypatch.setattr("app.services.tibber_prices.fetch_price_range", _fake_fetch)
count = refresh_prices(session, _FakeSettings(token=""))
assert count == 0
assert not fetch_called["called"]
assert _count_tibber_price_rows(session) == 0
def test_refresh_prices_noop_whitespace_token(energy_db, monkeypatch):
"""Whitespace-only token → no-op (treated as not configured)."""
session = energy_db
_create_active_tibber_contract(session)
fetch_called = {"called": False}
def _fake_fetch(token, home_id_or_none):
fetch_called["called"] = True
return _make_price_points(2)
monkeypatch.setattr("app.services.tibber_prices.fetch_price_range", _fake_fetch)
count = refresh_prices(session, _FakeSettings(token=" "))
assert count == 0
assert not fetch_called["called"]
# ---------------------------------------------------------------------------
# Test: inactive tibber contract
# ---------------------------------------------------------------------------
def test_refresh_prices_noop_inactive_tibber_contract(energy_db, monkeypatch):
"""Inactive tibber contract → no-op (active flag is False)."""
session = energy_db
# Create a tibber contract but do NOT activate it.
now = datetime.now(UTC)
contract = EnergyContract(
name="Inactive Tibber",
kind="tibber",
active=False, # <-- inactive
currency="EUR",
created_at=now,
updated_at=now,
)
session.add(contract)
session.commit()
fetch_called = {"called": False}
def _fake_fetch(token, home_id_or_none):
fetch_called["called"] = True
return _make_price_points(2)
monkeypatch.setattr("app.services.tibber_prices.fetch_price_range", _fake_fetch)
count = refresh_prices(session, _FakeSettings())
assert count == 0
assert not fetch_called["called"]
# ---------------------------------------------------------------------------
# Test: client errors propagate
# ---------------------------------------------------------------------------
def test_refresh_prices_propagates_client_error(energy_db, monkeypatch):
"""TibberError from the client is NOT swallowed by the service."""
session = energy_db
_create_active_tibber_contract(session)
def _failing_fetch(token, home_id_or_none):
raise TibberError("simulated network failure")
monkeypatch.setattr("app.services.tibber_prices.fetch_price_range", _failing_fetch)
with pytest.raises(TibberError):
refresh_prices(session, _FakeSettings())
# No rows should have been written.
assert _count_tibber_price_rows(session) == 0
+187
View File
@@ -0,0 +1,187 @@
"""Tests for app/services/timezone.py (FU10-energy-tz-accrual).
All tests monkeypatch ``local_tz`` to a fixed zone so they are deterministic
on any CI host timezone.
"""
from __future__ import annotations
from datetime import UTC, date, datetime
from zoneinfo import ZoneInfo
import pytest
from app.services import timezone as tz_mod
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
@pytest.fixture()
def ams(monkeypatch):
"""Pin local_tz to Europe/Amsterdam for all tests in this module."""
monkeypatch.setattr(tz_mod, "local_tz", lambda: ZoneInfo("Europe/Amsterdam"))
@pytest.fixture()
def utc_tz(monkeypatch):
"""Pin local_tz to UTC for simple arithmetic tests."""
monkeypatch.setattr(tz_mod, "local_tz", lambda: ZoneInfo("UTC"))
# ---------------------------------------------------------------------------
# local_tz() — resolution and monkeypatch contract
# ---------------------------------------------------------------------------
def test_local_tz_returns_tzinfo(ams):
"""local_tz() must return a tzinfo object."""
tz = tz_mod.local_tz()
assert tz is not None
# Verify it behaves like a valid tzinfo.
d = datetime(2026, 6, 25, 12, 0, tzinfo=tz)
assert d.utcoffset() is not None
def test_local_tz_env_var_overrides(monkeypatch):
"""When TZ env var is set, local_tz() returns that zone."""
monkeypatch.setenv("TZ", "America/New_York")
tz = tz_mod.local_tz()
# ZoneInfo("America/New_York") should have UTC-5 or UTC-4 offset.
d = datetime(2026, 1, 1, 12, 0, tzinfo=tz) # January → UTC-5
offset_hours = d.utcoffset().total_seconds() / 3600
assert offset_hours == pytest.approx(-5.0)
monkeypatch.delenv("TZ")
# ---------------------------------------------------------------------------
# to_local() — conversion correctness
# ---------------------------------------------------------------------------
def test_to_local_aware_utc(ams):
"""Aware UTC datetime must convert to CEST (UTC+2) in summer."""
dt_utc = datetime(2026, 6, 25, 12, 0, tzinfo=UTC)
local = tz_mod.to_local(dt_utc)
assert local.tzinfo is not None
assert local.hour == 14 # CEST = UTC+2
def test_to_local_naive_treated_as_utc(ams):
"""Naive datetime must be treated as UTC and converted to local."""
naive = datetime(2026, 6, 25, 12, 0) # naive = UTC convention
local = tz_mod.to_local(naive)
assert local.hour == 14 # CEST = UTC+2
def test_to_local_winter_offset(monkeypatch):
"""CET (UTC+1) applies in January."""
monkeypatch.setattr(tz_mod, "local_tz", lambda: ZoneInfo("Europe/Amsterdam"))
dt_utc = datetime(2026, 1, 15, 12, 0, tzinfo=UTC)
local = tz_mod.to_local(dt_utc)
assert local.hour == 13 # CET = UTC+1
# ---------------------------------------------------------------------------
# local_date() — date extraction
# ---------------------------------------------------------------------------
def test_local_date_around_cest_midnight(ams):
"""UTC 22:00 on June 24 is local midnight June 25 in CEST."""
# CEST = UTC+2; June 24 22:00 UTC = June 25 00:00 CEST
dt = datetime(2026, 6, 24, 22, 0, tzinfo=UTC)
d = tz_mod.local_date(dt)
assert d == date(2026, 6, 25)
def test_local_date_just_before_cest_midnight(ams):
"""UTC 21:59 on June 24 is still June 24 in CEST (23:59 local)."""
dt = datetime(2026, 6, 24, 21, 59, tzinfo=UTC)
d = tz_mod.local_date(dt)
assert d == date(2026, 6, 24)
# ---------------------------------------------------------------------------
# local_midnight_utc() — round-trip correctness
# ---------------------------------------------------------------------------
def test_local_midnight_utc_cest_summer(ams):
"""Europe/Amsterdam local midnight June 25 → June 24 22:00 UTC (CEST=UTC+2)."""
utc_midnight = tz_mod.local_midnight_utc(date(2026, 6, 25))
expected = datetime(2026, 6, 24, 22, 0, tzinfo=UTC)
assert utc_midnight == expected
def test_local_midnight_utc_cet_winter(monkeypatch):
"""Europe/Amsterdam local midnight Jan 15 → Jan 14 23:00 UTC (CET=UTC+1)."""
monkeypatch.setattr(tz_mod, "local_tz", lambda: ZoneInfo("Europe/Amsterdam"))
utc_midnight = tz_mod.local_midnight_utc(date(2026, 1, 15))
expected = datetime(2026, 1, 14, 23, 0, tzinfo=UTC)
assert utc_midnight == expected
def test_local_midnight_utc_roundtrip(ams):
"""local_date(local_midnight_utc(D)) must return D for any date D."""
for day in range(1, 32):
d = date(2026, 6, day) if day <= 30 else date(2026, 7, day - 30)
if day > 30:
continue
midnight_utc = tz_mod.local_midnight_utc(d)
recovered = tz_mod.local_date(midnight_utc)
assert recovered == d, f"Round-trip failed for {d}"
# ---------------------------------------------------------------------------
# local_now() — sanity check
# ---------------------------------------------------------------------------
def test_local_now_is_aware(ams):
"""local_now() must return a timezone-aware datetime."""
now = tz_mod.local_now()
assert now.tzinfo is not None
assert now.utcoffset() is not None
# ---------------------------------------------------------------------------
# effective_from naive→local→UTC (Principle A — contract routes)
# ---------------------------------------------------------------------------
def test_localize_naive_effective_from(monkeypatch):
"""A naive '2026-06-25T00:00:00' must be interpreted as CEST local midnight.
Expected: CEST midnight June 25 = UTC June 24 22:00.
"""
monkeypatch.setattr(tz_mod, "local_tz", lambda: ZoneInfo("Europe/Amsterdam"))
# Simulate what the route helper does:
from app.api.routes.api.energy_contracts import _localize_effective_from
naive = datetime(2026, 6, 25, 0, 0, 0) # naive, no tzinfo
utc_result = _localize_effective_from(naive)
assert utc_result.tzinfo is not None
expected = datetime(2026, 6, 24, 22, 0, 0, tzinfo=UTC)
assert utc_result == expected, (
f"Naive local midnight should localize to {expected}, got {utc_result}"
)
def test_localize_aware_effective_from():
"""An aware datetime must be stored as UTC unchanged (no double conversion)."""
from app.api.routes.api.energy_contracts import _localize_effective_from
aware = datetime(2026, 6, 25, 0, 0, 0, tzinfo=UTC)
utc_result = _localize_effective_from(aware)
assert utc_result == aware
def test_localize_none_effective_from():
"""None effective_from must return current UTC time (as before)."""
from app.api.routes.api.energy_contracts import _localize_effective_from
before = datetime.now(UTC)
result = _localize_effective_from(None)
after = datetime.now(UTC)
assert result.tzinfo is not None
assert before <= result <= after