77 Commits
Author SHA1 Message Date
tliu93 d4acfc438a FUE-T07: use meter label directly as energy-cost HA device name (drop 'Energy Cost' prefix)
frontend / frontend (push) Successful in 2m14s
pytest / test (push) Successful in 10m49s
2026-06-25 21:28:13 +02:00
tliu93 c26160b10b FUE-T06: trigger HA discovery republish after meter declare/update
frontend / frontend (push) Successful in 2m11s
pytest / test (push) Successful in 10m36s
2026-06-25 20:54:42 +02:00
tliu93 efbe36d7c0 FUE-T05: anchor energy-cost HA device identity to active meter (uuid id, label name, empty when none) 2026-06-25 20:43:07 +02:00
tliu93 f663981cdb FUE-T04: add Meter.uuid (stable HA identity anchor) + backfill migration 2026-06-25 20:20:01 +02:00
tliu93 da05fd2f09 FUE-T03: correct meters POST docstring (no recomputed_periods field) and inaccurate test comments
frontend / frontend (push) Successful in 2m11s
pytest / test (push) Successful in 8m51s
2026-06-25 18:36:46 +02:00
tliu93 188168b16a FUE-T02: fix EditMeterForm timestamp parsing to use shared parseBackendTimestamp (handles offset form) 2026-06-25 18:33:42 +02:00
tliu93 f1e7ce3133 FUE-T01: count billing window start day in full for fixed-fee/credit accrual (symmetric with end day) 2026-06-25 18:33:42 +02:00
tliu93 e3d0d17cac M7-T07: finalize M7 docs (roadmap entry, design index, status done, meter-epochs module doc)
frontend / frontend (push) Successful in 2m10s
pytest / test (push) Successful in 8m34s
2026-06-25 17:01:54 +02:00
tliu93 33f5de0591 M7-T06: add meter management UI (timeline, declare swap, edit) to Energy view 2026-06-25 17:01:54 +02:00
tliu93 839faaf95b M7-T05: add meter CRUD API + retroactive recompute + OpenAPI regen 2026-06-25 17:01:54 +02:00
tliu93 82bb329500 M7-T04: anchor cumulative cost entities to active meter start (per-meter reset) 2026-06-25 17:01:54 +02:00
tliu93 17057ed41e M7-T03: make billing engine meter-aware (no cross-meter delta, delta sanity guard, period.meter_id) 2026-06-25 17:01:54 +02:00
tliu93 227b146c2c M7-T02: add meter service layer (declare/swap/close/edit, meter_at, mutual exclusion) 2026-06-25 17:01:54 +02:00
tliu93 32a20785ee M7-T01: add meter table + model + energy_cost_period.meter_id + backfill migration 2026-06-25 14:48:39 +02:00
tliu93 2544514f52 M7: add meter-epochs / meter-swap-archival milestone design doc
frontend / frontend (push) Successful in 2m6s
pytest / test (push) Successful in 7m51s
Design for explicit meter lifecycle (epochs) so the register-difference
billing engine never computes a delta across a meter swap (2G->4G or house
move). Meter-centric (no Home entity); per-meter cumulative reset; retroactive
declaration with recompute; delta sanity guard as backstop. 7 atomic task cards.
2026-06-25 14:23:25 +02:00
tliu93 96e85fb48e FU11: anchor cumulative cost at recording start; add daily-reset cost entities; prefill add-version form
pytest / test (push) Successful in 7m32s
frontend / frontend (push) Successful in 2m6s
- MQTT import_cost_total / export_revenue_total now anchor fixed-fee / tax-credit
  accrual at max(contract start, first recorded period), so standing charges for
  untracked pre-recording days are no longer folded into the running total.
- Add energy.import_cost_today / energy.export_revenue_today (monetary,
  total_increasing) that reset at server-local midnight, for per-day cost
  aggregation in Home Assistant alongside the running totals.
- ContractForm add-version mode prefills the contract's open version values, so
  only the fields that actually change need editing.
2026-06-25 12:45:24 +02:00
tliu93 b71009620a compose add tz 2026-06-25 11:48:45 +02:00
tliu93 8b91146c29 FU10: localize energy time math to server tz; accrue fixed/credit per elapsed local day
frontend / frontend (push) Successful in 2m5s
pytest / test (push) Successful in 7m34s
- Store UTC, compute day boundaries in server local timezone (new app/services/timezone.py).
- summarize: fixed fee / tax credit accrue only for elapsed whole local days,
  integrated across contract versions — no future-day inflation, no version-switch collapse.
- MQTT import_cost_total / export_revenue_total getters delegate to summarize (no inline
  apportionment), anchored at the active contract's earliest version effective_from so the
  total sensors stay monotonic and never jump backward.
- effective_from: naive datetimes interpreted as server-local wall-clock, then stored UTC.
- frontend ContractForm sends local-midnight naive datetime (was UTC midnight).
- get_costs_summary default window uses server-local "today".
2026-06-25 11:13:30 +02:00
tliu93 682e06d256 modbus: add explicit cascade delete (device + readings + toggles + HA cleanup)
frontend / frontend (push) Successful in 2m4s
pytest / test (push) Successful in 7m32s
A disabled device with historical readings still cannot be deleted by default
(409 guard stays, to prevent accidental data loss). A new explicit opt-in lets
the user remove it for real:

- DELETE /api/modbus/devices/{uuid}?cascade=true removes the device's readings
  and exposed_entity_toggle rows, then the device, in one transaction (FK order:
  readings before device). Best-effort clears the HA discovery configs first
  (MQTT not connected -> no-op, never blocks the delete). Default (no cascade)
  unchanged. Returns 200 with deletion counts.
- Frontend: a 'Force Delete (all data)' button appears only after the 409, with
  a red irreversible warning; cascade is sent only on that explicit action.
- Also corrected a stale comment: FK RESTRICT IS enforced at runtime now.
2026-06-24 17:38:52 +02:00
tliu93 d7f0731d1c ha_discovery: publish state/availability under home_automation prefix
Follow the standard MQTT Discovery split: discovery config stays under the HA
discovery prefix (homeassistant/.../config, required), but the actual state and
availability are published under our own prefix (default home_automation/). The
config payload's state_topic/availability fields point at the new prefix so HA
reads state from there.

- new config ha_state_topic_prefix (default home_automation).
- build_discovery_payload + all publish_* split discovery_prefix (config topic)
  from state_prefix (state/availability). energy-cost still omits availability;
  modbus availability structure unchanged. Tests updated.
2026-06-24 16:21:15 +02:00
tliu93 9bc46f8d2d expose: buy/sell_price_now follow the live dual-tariff slot
Subscribe the separate DSMR tariff topic (dsmr/meter-stats/electricity_tariff,
1=dal/off-peak, 2=normal/peak), keep the latest slot in memory, and make the
manual-contract buy/sell_price_now sensors show the matching tariff's price
(tariff 1 -> dal, 2/unknown -> normal). Tibber (single 15-min price) unchanged.
Billing is untouched (register-split already handles tariffs correctly).

- new config dsmr_tariff_topic; subscribe managed by apply_dsmr_subscription
  (restart-free on config save). In-memory tariff is lock-guarded.
- import_cost_total / export_revenue_total cumulative getters unchanged.
2026-06-24 15:39:21 +02:00
tliu93 6f8cb05eab expose: fold standing fee into import cost, tax credit into export revenue
So HA can show the full payable: net = import - export = Total Payable, with
both entities staying monotonic-positive (no negative-value/reset pitfalls).

- import_cost_total = SUM(import_cost) + days*(network_fee+management_fee)/30
- export_revenue_total = SUM(export_revenue) + days*heffingskorting/365
  (days = calendar days since the active version's effective_from, incl. today)
- state_class total_increasing -> total (monetary-compatible; values still only
  increase). device_class/key/names: key unchanged, names clarified.
- buy/sell_price_now and DeviceInfo untouched. Decimal money math. Tests added.
2026-06-24 14:26:32 +02:00
tliu93 effe434637 frontend: show Energy/DSMR/meter times in local timezone, 24h
Backend serializes SQLite-read (naive) UTC datetimes without a Z, so the browser
was parsing them as local time -> the UI effectively showed UTC values.

- New src/utils/datetime.ts: treats a tz-less timestamp as UTC, then formats in
  the browser's local timezone with hour12:false (no AM/PM).
- Applied to Energy tabs (contracts/prices/costs/DSMR), per-device readings &
  trends, and the meter readings table on the Records page.
- CostView Today/This month now use local day boundaries (queries stay UTC ISO).
- Untouched: location/poo (raw Zulu strings), the map, and query-window params.
- Tests are timezone-independent (verified under TZ=America/Los_Angeles).
2026-06-24 13:22:07 +02:00
tliu93 9d3e28a529 Billing fix: don't bill periods from stale/out-of-range DSMR readings
Root cause (found in the live DB): register_at() returned the latest reading
for any boundary beyond the DSMR data range (a future instant or a long gap),
so future periods got start==end -> delta 0 and were written as degraded=False
'ok' rows with cost 0. recompute_range() over a future end then filled the whole
month with such fake rows, which then occupied the slots so the per-minute tick
(immutability guard skips non-degraded rows) never recomputed the real data.

- register_at: ignore a reading older than one period (15min) before the
  boundary -> returns None -> period is marked degraded instead of a fake 0.
- recompute_range: only process closed periods (t1 <= now); never future ones.
- Tests cover staleness window, out-of-range -> degraded, recompute skips
  future, and a regression that normal closed periods are unaffected.
2026-06-24 12:44:05 +02:00
tliu93 7e61b0a938 HA Discovery fix: energy-cost sensors are always-available
The energy-cost device has only sensors (no online/offline heartbeat), but the
discovery config still declared an availability topic that nothing ever publishes
'online' to — so Home Assistant marked every energy-cost entity 'unavailable'
even though the state topic was being published (visible in MQTT Explorer).

- DeviceInfo gains provides_availability (default True, keeps Modbus behavior).
- _energy_cost_provider sets provides_availability=False.
- build_discovery_payload only emits availability/availability_mode for devices
  that publish a heartbeat; HA treats the rest as always-available.
- Test asserts energy-cost configs omit availability.
2026-06-24 11:54:49 +02:00
tliu93 e8521351d7 frontend: add DSMR data panel to Energy page
- New 'DSMR' tab showing the latest parsed telegram (GET /api/energy/dsmr/latest)
  as a key/value table — the source-of-truth view so you can see ingest is
  landing data without reading the DB.
- Auto-refresh toggle (~10s); loading/error/empty states (empty explains the
  likely cause); null phase values shown as a dash.
- useDsmrLatest extended with optional auto-refresh. Tests added.
2026-06-24 11:02:42 +02:00
tliu93 8d4f496ff4 DSMR hardening: restart-free config + decouple from telegram id
Config hot-reload (no restart):
- MqttManager.unsubscribe(); apply_dsmr_subscription(settings) re-applies the
  DSMR subscription (subscribe/unsubscribe/topic-change, fresh snapshot so the
  sample interval also takes effect).
- Called from lifespan AND PUT /api/config, so toggling DSMR ingest in the UI
  takes effect immediately without restarting the app.

Decouple ingest from the DSMR telegram id (overflows / resets to zero):
- Migration 20260624_12: drop UNIQUE(source_id), make recorded_at UNIQUE.
- Idempotency now keys on recorded_at (telegram timestamp); the table's own
  autoincrement PK is the stable identity. source_id kept only as a reference.
- Regression test: colliding telegram ids no longer drop new data.
2026-06-24 11:02:32 +02:00
tliu93 78f3cc4776 M6-T11: document M6 (energy pricing + DSMR + Tibber) across README/roadmap/arch
- README: M6 section + feature list + new tables/config items.
- roadmap: M6 row (graduated) + detail section; M6 in milestone index.
- architecture-overview: M6 models/services/integrations/routes/jobs/config.
- design/README: index m6, drop stale '三个里程碑' wording.
- OpenAPI already in sync (no diff).
2026-06-23 23:52:07 +02:00
tliu93 96e88861d4 M6-T10: frontend contract management + price/cost views + Tibber test
- EnergyPage: Mantine Tabs (Devices kept intact + Contracts/Prices/Costs).
- ContractManager/ContractForm: list/activate/add-version + version history;
  form fields rendered dynamically from /api/energy/profiles structure.
- TibberPrices + CostView: Recharts price curve, cost trend/detail/summary,
  recompute; window-bounded, currency/units from API, empty/error/loading states.
- ConfigPage: tri-state Tibber test button (token never shown).
- hooks.ts: typed energy hooks; schema.d.ts regenerated via codegen.
2026-06-23 23:32:39 +02:00
tliu93 57f2459f60 M6-T09: add Energy data API (prices/costs/summary/dsmr-latest/recompute/tibber-test)
- routes/api/energy.py + schemas/energy.py: GET prices (tibber points or manual
  tariff, buy/sell consistent with the pricing strategies), GET costs (time
  window + limit, ascending), GET costs/summary (summarize passthrough), GET
  dsmr/latest, POST costs/recompute (idempotent, <=366d guard, CSRF), POST
  tibber/test (three-state, token never echoed, CSRF).
- main.py registers router; OpenAPI re-exported; tests for all endpoints.
2026-06-23 23:05:45 +02:00
tliu93 c61fc2d4ba M6-T08: add energy_cost expose provider + post-billing state publish
- expose.py: _energy_cost_provider yields buy/sell_price_now + import_cost_total
  /export_revenue_total (total_increasing, monetary), value_getters read from
  energy_cost_period; 2-element identifiers so the existing HA Discovery builder
  (which indexes identifiers[1]) works. Other providers/mechanism untouched.
- main.py: energy-cost job calls publish_states after computing periods (no-op
  when MQTT/Discovery off). Tests added.
Note: energy-cost sensors lack an availability heartbeat (ha_discovery mechanism
limitation, out of scope here).
2026-06-23 22:44:41 +02:00
tliu93 d35ac7d752 M6-T07: add metering billing engine + period job + summary + recompute
- app/services/energy_cost.py: register_at (register values at a UTC boundary),
  compute_period (register deltas x active-version strategy, snapshot price +
  contract_version_id, immutable unless overwrite), compute_closed_periods
  (bounded lookback), recompute_range (explicit overwrite), summarize
  (sum net + standing fees month/30 - heffingskorting year/365).
- Missing tibber price -> skip; missing reading -> degraded (zeroed amounts,
  null version), consistently on both create and recompute paths.
- main.py: 1-minute energy-cost job (existing jobs/MQTT untouched). Tests added.
2026-06-23 22:26:56 +02:00
tliu93 ef48032adb M6-T06: extend MqttManager with subscribe + DSMR ingest
- mqtt.py: subscription registry, on_message dispatch (swallows handler
  errors), re-subscribe on (re)connect; publish/connect/disconnect/reconnect
  unchanged.
- app/services/dsmr_ingest.py: handle_message stores the full telegram frame
  (no allowlist; gas/phases/null preserved), 10s downsample by timestamp.second,
  source_id idempotency (select + IntegrityError rollback). Net-thread safe.
- main.py registers the DSMR subscription only when dsmr_ingest_enabled.
2026-06-23 21:52:57 +02:00
tliu93 7e266a21eb M6-T05: add Tibber GraphQL client + price refresh service + schedule
- app/integrations/tibber/client.py: fetch_price_range/fetch_current_price
  (priceInfoRange QUARTER_HOURLY, startsAt->UTC, no fixed node count assumption),
  TibberError/TibberAuthError; token never logged or put in exception text.
- app/services/tibber_prices.py: refresh_prices upserts by starts_at (idempotent),
  no-op unless an active kind=tibber contract + token exist.
- main.py: hourly tibber-refresh job (existing jobs/MQTT untouched). Tests added.
2026-06-23 21:35:29 +02:00
tliu93 4284bd40a7 M6-T04: add EnergyContract CRUD + versions + profile validation API
- app/services/contracts.py: create_contract, add_version (append-only,
  auto-closes prior version), activate_contract (mutex), and
  active_contract_version_at (half-open [from, to)) for the billing engine.
- app/schemas/energy_contract.py + routes/api/energy_contracts.py: 6 endpoints
  (GET/POST contracts, GET/PATCH contracts/{id}, POST .../versions, GET profiles).
  Values validated against pricing profiles (422 on mismatch); no DELETE.
- main.py registers router; OpenAPI re-exported; tests for CRUD/version/activate.
2026-06-23 21:19:25 +02:00
tliu93 bedea196c3 M6-T03: add pricing profile framework + strategy registry
- app/integrations/pricing: profiles.py (pydantic ManualProfile/TibberProfile,
  load_profile/list_profiles/validate_values), manual.yaml + tibber.yaml
  (structure-only, no price values), strategies.py (register/get_strategy,
  manual dual-tariff + tibber strategies, all Decimal).
- tibber strategy matches nearest tibber_price with starts_at <= t0.
- tests for profiles + strategies (hand-checked dual-tariff & tibber math).
2026-06-23 20:58:34 +02:00
tliu93 df54f5518b M6-T02: add DSMR + Tibber flat config fields
- Settings: dsmr_ingest_enabled, dsmr_mqtt_topic, dsmr_sample_interval_s,
  tibber_api_token (secret), tibber_home_id (all opt-in / off by default).
- CONFIG_FIELDS DSMR + Tibber sections; _settings_payload extended.
- .env.example examples; test_api_config coverage (sections, secret, 422).
2026-06-23 20:43:46 +02:00
tliu93 18fe18281b M6-T01: add 5 energy tables, models, and migration
- app/models/energy.py: DsmrReading, EnergyContract, EnergyContractVersion,
  TibberPrice, EnergyCostPeriod (per design §3.5; JSON blobs, RESTRICT FKs).
- alembic_app migration 20260623_11_energy_tables (head), env.py imports.
- app_db_adopt APP_BASELINE_REVISION bumped to new head.
- tests/test_energy_models.py + expose_catalog test fixups for new head.
2026-06-23 20:33:35 +02:00
tliu93 355c8eb8b8 fix: add paho-mqtt and pymodbus to dev-requirements lock (CI install was missing M5 deps)
pytest / test (push) Successful in 5m13s
frontend / frontend (push) Successful in 1m48s
2026-06-23 20:06:13 +02:00
tliu93 1ffdaebbbd M6: add Tibber API + NL energy pricing + DSMR reference doc
frontend / frontend (push) Successful in 1m54s
pytest / test (push) Failing after 32s
2026-06-23 19:56:02 +02:00
tliu93 c64693b8e5 M6: add design doc — generic pricing layer (manual/tibber) + DSMR cost engine 2026-06-23 19:53:26 +02:00
tliu93 c89d083942 M5: add read-only per-meter readings tabs to Records page; English-only UI strings 2026-06-22 22:16:55 +02:00
tliu93 6090b98ab0 M5: enable SQLite foreign_keys pragma for DB-level FK enforcement 2026-06-22 20:45:47 +02:00
tliu93 489cd7df3d M5: default Modbus polling to off (opt-in) to avoid unnecessary DB writes 2026-06-22 20:28:12 +02:00
tliu93 35cdc7f22a M5: use DB-merged runtime settings for MQTT/discovery; add MQTT test button; move Expose panel into config accordion 2026-06-22 20:11:54 +02:00
tliu93 75d685f04d M5: roll Energy chart window for live auto-refresh (English label); render boolean config fields as switches 2026-06-22 19:40:18 +02:00
tliu93 935e68846c M5: Energy view — per-metric decimal formatting and chart auto-refresh toggle 2026-06-22 19:18:29 +02:00
tliu93 4767a7af60 M5-T13: document Modbus/Energy/MQTT, mark M5 done, finalize OpenAPI 2026-06-22 16:10:11 +02:00
tliu93 45d87f36f7 M5-T12: add expose toggle API and Home Assistant Expose settings panel 2026-06-22 15:50:49 +02:00
tliu93 35b95f5c88 M5-T11: add HA discovery and state publishing wired to polling 2026-06-22 15:32:43 +02:00
tliu93 9d05f5ea62 M5-T10: add paho MQTT client with lifespan connect/reconnect and mqtt/test endpoint 2026-06-22 15:06:30 +02:00
tliu93 360cddc05b M5-T09: add exposed_entities table and ExposableEntity provider framework 2026-06-22 14:50:40 +02:00
tliu93 14b846549e M5-T08: add MQTT, HA Discovery, and Modbus polling config fields 2026-06-22 14:27:21 +02:00
tliu93 68165f0e01 M5-T07: add latest-reading cards and Recharts trend charts; readings return most-recent rows 2026-06-22 14:17:59 +02:00
tliu93 2dc469274a M5-T06: add Energy device management UI and sidebar entry 2026-06-22 13:52:33 +02:00
tliu93 702a1cec00 M5-T05: add Modbus JSON API (device CRUD, readings, metrics, test) 2026-06-22 13:37:48 +02:00
tliu93 49d15d8ffe M5-T04: add Modbus poll service and APScheduler polling job 2026-06-22 13:24:11 +02:00
tliu93 c89cea4953 M5-T03: add Modbus TCP driver, YAML profile framework, sdm120, modbus_cli 2026-06-22 13:03:50 +02:00
tliu93 d9c82e39fb M5-T02: add modbus_device and modbus_reading tables and models 2026-06-22 12:45:50 +02:00
tliu93 eac7860b51 M5-T01B: render ConfigPage sections as Mantine Accordion 2026-06-22 12:34:04 +02:00
tliu93 da7d0f9d62 M5-T01: refactor AppLayout from top bar to sidebar navigation 2026-06-22 12:26:52 +02:00
tliu93 e49a6ad963 M5(design): generic modbus_device + JSON readings; YAML profiles; Accordion config; manual CLI/MQTT testing
frontend / frontend (push) Successful in 1m26s
pytest / test (push) Successful in 3m52s
Rework the M5 design doc data model and add manual-testing surface:

- Two-layer model: rename energy_meters/energy_readings -> generic
  modbus_device + modbus_reading; readings become a JSON payload blob
  instead of a wide fixed-column table (aggregation via SQLite
  json_extract, generated-column index as a future lever).
- Protocol knowledge moves to read-only YAML profiles (data+functions,
  no OOP inheritance); deployment/config (friendly_name, unit_id, host)
  stays on the device row; multiple devices share one profile.
- UUID = stable internal identity, also anchors HA discovery unique_id
  (Z2M rename model); device metadata for HA derived from the profile.
- Naming scheme C: modbus_* / /api/modbus/devices at the data+API layer,
  'Energy' as the first domain view in the frontend.
- New task M5-T01B: ConfigPage Accordion sectioning (avoids a second
  in-page side nav conflicting with the T01 app sidebar).
- Manual testing: read-only modbus_cli 'probe' subcommand (FC03/04 only,
  no register writes); mqtt/test now publishes a test message viewable
  in MQTT Explorer; no MQTT CLI (manual verification via UI + HA).
2026-06-22 11:55:39 +02:00
tliu93 d70b985eb0 fix(ci): recompile dev-requirements.txt to include pyotp (pytest CI install)
pytest / test (push) Successful in 3m51s
frontend / frontend (push) Successful in 1m26s
2026-06-22 11:08:06 +02:00
tliu93 e3c47b2762 docs(CLAUDE): switch to direct-on-main workflow for this single-user repo
frontend / frontend (push) Successful in 1m35s
pytest / test (push) Failing after 31s
2026-06-22 10:37:34 +02:00
tliu93 dbf17f3592 docs(CLAUDE): pin default sub-agent models (Implementer/Fixer=Sonnet, Reviewer=Opus) with manual-override clause 2026-06-22 09:50:46 +02:00
tliu93 ae75e4582d M4-T09: document login hardening, finalize OpenAPI and roadmap 2026-06-21 23:01:50 +02:00
tliu93 ee1264b66b M4-T08: add two-step login and TOTP settings panel (qrcode.react) 2026-06-21 22:42:41 +02:00
tliu93 51da8c5716 M4-T07: add admin_cli disable-totp and reissue-totp escape commands 2026-06-21 22:27:07 +02:00
tliu93 81bd32f613 M4-T06: add TOTP second factor to login (totp_code + recovery codes) 2026-06-21 22:08:55 +02:00
tliu93 ee3031013e M4-T05: add TOTP service and setup/enable/disable/status API 2026-06-21 21:55:17 +02:00
tliu93 5026967cee M4-T04: add AuthUser TOTP columns and auth_recovery_code table 2026-06-21 21:38:22 +02:00
tliu93 5481fbc99c M4-T03: add admin_cli escape hatch (reset-password, unlock, list-admin) 2026-06-21 21:28:41 +02:00
tliu93 e480a84a44 M4-T02: add exponential login back-off service and wire into login endpoint 2026-06-21 21:20:28 +02:00
tliu93 64d3882c16 M4-T01: add auth_login_throttle table + LoginThrottle model 2026-06-21 21:03:20 +02:00
tliu93 6b67b8e0f8 modify compose dev port 2026-06-21 20:49:24 +02:00
tliu93 0cb94d85ec docs(design): plan M4 login-hardening (first) + M5 IoT/energy
- Add docs/references/SDM120-Modbus-Protocol.md extracted from the SDM120 PDF
- M4 login hardening: brute-force exponential backoff, CLI escape hatch
  (reset-password/unlock/disable-totp), optional TOTP 2FA
- M5 IoT/energy: Modbus-TCP SDM120 polling + energy tables, MQTT/HA
  Discovery expose framework, frontend sidebar + energy view
- Add manual acceptance walkthroughs (M4 lock/unlock; M5 energy_cli read)
- Index both milestones in docs/design/README.md
2026-06-21 20:46:12 +02:00
tliu93 9da88db221 docs(roadmap): graduate next-phase work out of Future Ideas
frontend / frontend (push) Successful in 1m18s
pytest / test (push) Successful in 1m32s
Promote four now-decided directions from Future Ideas into a new
"下一阶段:已确定要做" section (roadmap altitude, not yet broken into task cards):
TOTP second factor for the public dashboard, frontend optimization (scope TBD),
MQTT / IoT integration, and a settings-page long-lived API token (PAT-style,
related to but distinct from M3's mobile OAuth). Future Ideas is now an empty,
purpose-stated bucket for not-yet-decided ideas.
2026-06-13 17:38:50 +02:00
158 changed files with 55899 additions and 341 deletions
+35
View File
@@ -28,3 +28,38 @@ TICKTICK_CLIENT_ID=
TICKTICK_CLIENT_SECRET=
TICKTICK_TOKEN=
HOME_ASSISTANT_ACTION_TASK_PROJECT_ID=
# Optional: Modbus polling (global kill-switch; default false — opt-in).
# MODBUS_POLLING_ENABLED=false
# Optional: MQTT broker connection.
# Leave MQTT_ENABLED=false (or unset) when MQTT is not needed.
MQTT_ENABLED=false
MQTT_BROKER_HOST=
MQTT_BROKER_PORT=1883
MQTT_USERNAME=
MQTT_PASSWORD=
MQTT_TLS_ENABLED=false
# Optional: Home Assistant MQTT Discovery.
# Requires MQTT_ENABLED=true and a running MQTT broker.
# HA_DISCOVERY_PREFIX is the config topic prefix (must be "homeassistant" for HA auto-discovery).
# HA_STATE_TOPIC_PREFIX is the prefix for state/availability topics (separate from discovery).
HA_DISCOVERY_ENABLED=false
HA_DISCOVERY_PREFIX=homeassistant
HA_STATE_TOPIC_PREFIX=home_automation
# Optional: DSMR smart-meter ingest via MQTT (M6; default off — opt-in).
# Set DSMR_INGEST_ENABLED=true to subscribe to the DSMR MQTT topic and
# store 10-second downsampled telegram frames in dsmr_reading.
# DSMR_INGEST_ENABLED=false
# DSMR_MQTT_TOPIC=dsmr/json
# DSMR_SAMPLE_INTERVAL_S=10
# DSMR tariff topic: publishes "1" (dal/off-peak) or "2" (normal/peak).
# Empty string disables tariff-aware pricing (buy/sell_price_now always show normal rate).
# DSMR_TARIFF_TOPIC=dsmr/meter-stats/electricity_tariff
# Optional: Tibber dynamic pricing credentials (M6).
# Only used when an active energy contract with kind=tibber is configured.
# TIBBER_API_TOKEN=
# TIBBER_HOME_ID=
+25 -7
View File
@@ -36,14 +36,29 @@
- **默认逐步**:给一个 milestone 文档,按其中原子任务**一步一步**实现。
- **(a) 只实现一步**:用户说"只实现一步 / 这一个任务"时,**只做那一个任务卡**,跑完校验闸门后停下,等用户确认,不要顺手往下做。
- **(b) 完成整个 milestone**:仅当用户在提示词里**显式要求启用 sub-agent 并指定模型**时,才用指定模型起 implementer sub-agent,按任务依赖顺序跑完整条链。
- **Sub-agent 纪律**:只在用户显式要求时才 spawn sub-agent;单步/小改动在主线内联完成。起 sub-agent 时用用户**指定的模型**Agent 工具的 `model` 覆盖)
- **(b) 完成整个 milestone**:仅当用户在提示词里**显式要求启用 sub-agent**时,才起 implementer / reviewer / fixer sub-agent(模型用下方**『默认模型档位』**,用户人工指定则覆盖),按任务依赖顺序跑完整条链。
- **Sub-agent 纪律**:只在用户显式要求时才 spawn sub-agent;单步/小改动在主线内联完成。起 sub-agent 时按下方**『默认模型档位』**选择模型(用户人工指定则以人工指定为准),用 Agent 工具的 `model` 字段落实
### 角色(Orchestrator → Implementer → Reviewer
### 默认模型档位(实现模式 sub-agent;可被人工指定覆盖
起 implementer / reviewer / fixer sub-agent 时,**默认**用下列模型档位,无需用户每次人工指定:
| 角色 | 默认模型 | 推理档位 |
| --- | --- | --- |
| **Implementer** | **Sonnet** | high reasoning effort |
| **Fixer**(返工) | **Sonnet** | high reasoning effort |
| **Reviewer** | **Opus** | extra-high reasoning effort |
- **人工指定覆盖**:若用户在提示词里**显式指定了其他模型**(针对任一角色),则**以用户人工指定为准**,覆盖上述默认。
- 用 Agent 工具的 `model` 字段落实模型选择;该字段当前仅支持 `sonnet` / `opus` / `haiku` / `fable`
- **推理档位说明**:Agent 工具未暴露独立的 reasoning-effort 旋钮,"high / extra-high reasoning" 通过 spawn prompt 里的显式指令传达(要求 implementer/fixer 动手前充分推理边界条件;要求 reviewer 以对抗性外部审计心态最高强度复核)。若所在 harness 提供真正的 effort 设置,则一并按此档位设置。
### 角色(Orchestrator → Implementer → Reviewer → Fixer
- 我(主线)= **Orchestrator**:挑依赖已满足的下一个任务、派发、转述结果、维护任务 `Status`
- **Implementer**便宜模型,用户指定):一次一个任务,严格按任务卡,不扩范围。
- **Reviewer**强模型,用户指定):实现完成后起 Reviewer sub-agent,按任务卡 `Acceptance criteria` + `Reviewer checklist` 复核、**独立重跑校验闸门**,驱动 implementer 返工直到本轮 PASS。
- **Implementer**默认 **Sonnet**high reasoning;见上方『默认模型档位』):一次一个任务,严格按任务卡,不扩范围。
- **Reviewer**默认 **Opus**extra-high reasoning):实现完成后起 Reviewer sub-agent,按任务卡 `Acceptance criteria` + `Reviewer checklist` 复核、**独立重跑校验闸门**,驱动返工直到本轮 PASS。
- **Fixer**(默认 **Sonnet**high reasoning):按 reviewer 的编号返工清单返工;**每轮返工起一个干净的 Fixer**(与首次实现的 Implementer 分开冷启动),先读对应 `review-notes/<task>-review-<n>.md` 再改。
#### Reviewer 盲审纪律(M1 教训)
@@ -101,7 +116,9 @@ python scripts/export_openapi.py && git diff --exit-code openapi/ # 改了路
## Commit 规范(重点)
### 分支
- 每个 milestone/feature 一个分支(如 `feature/m1-db-consolidation`),**不在 `main` 上直接提交**。
- **本仓库是个人单用户项目:默认直接在 `main` 上开发**,不强制 feature 分支,**直接提交并 push 到 `main` 是允许的(无需开 PR**。
- 仍保持**每个任务一个干净 commit**message 前缀任务/里程碑 ID)。改动较大想隔离时可临时开分支,用完**快进合并**回 `main`(保持线性历史),非必需。
- 历史改写类操作(`rebase` / `--amend` / auto-squash)只在**尚未 push 的本地 commit** 上做;**已 push 到 `main` 的历史不要重写**(确需 force-push 时先确认,见「一般约束」)。
### 一轮实现完成(用户确认「实现完成」后)
- 准备好**这一轮的 commit message** 并提交,作为本轮的 **base commit**
@@ -130,7 +147,8 @@ python scripts/export_openapi.py && git diff --exit-code openapi/ # 改了路
- autosquash **改写历史**:仅在 push / 开 PR **之前**做。若该分支已 push,需要 force-push——属对外操作,**先取得用户确认再做**。
### 一般约束
- commit / push 只在用户要求时进行;push、force-push、开/改 PR 等对外操作先确认
- **个人单用户仓库:直接 commit push 到 `main` 已获授权**——在校验闸门全绿、一轮工作完成时即可提交 / 推送,无需逐次征求同意
- 仍需**先取得用户确认**的操作:**force-push / 改写已推送历史**,以及**打 tag**(会触发镜像 CI / 对外发布;且打 tag 前须按下方「发版前置走查」真跑一次 `docker build`)。
## 发版前置走查(打 tag 前必做)
+214 -5
View File
@@ -6,14 +6,21 @@
- FastAPI Web 应用(React SPA 前端 + JSON API
- SQLite + SQLAlchemy + Alembic 的单库结构
- username/password + server-side session 鉴权
- username/password + server-side session 鉴权(含登录加固,见下文)
- runtime config 页面与 app DB 持久化
- public IPv4 monitor、历史持久化与定时检查
- SMTP 配置、测试发信与 public IPv4 changed 邮件通知
- location recorder
- poo recorder
- Home Assistant inbound / outbound integration
- Home Assistant inbound / outbound integrationREST 通道)
- TickTick OAuth 与 action task 集成
- **Modbus 设备采集**:通过 YAML profile(首个:SDM120 电表)按设备周期轮询 Modbus-TCP 网关,解码工程量并落通用读数表(`modbus_device` + `modbus_reading`
- **MQTT + Home Assistant Discovery**:以可勾选方式把 Modbus 设备/工程量注册为 HA device/entity(含 binary_sensor online),state 周期发布;配置变更可重连重发
- **前端侧边栏 + Energy 视图**:侧边导航替换顶栏;Energy 页管理 Modbus 设备、展示最新读数与 Recharts 走势图;Config 页 Accordion 分区展开;Expose 设置勾选 HA 可暴露实体
- **DSMR 实时电表接入**:订阅 DSMR Reader 的 `dsmr/json`(每秒一帧)、整帧 JSON blob 按 10 秒降采样落库(`dsmr_reading`
- **通用电价合同层**YAML profile 定合同结构(manual 固定/双费率 / tibber 动态电价);`EnergyContract`+`EnergyContractVersion` 存 UI 可填的数值,改价加新版本旧版本保留;price strategy 按 kind 出价
- **实时买卖电费计算**:每 15 分钟按寄存器差值(`_1`=dal/低、`_2`=normal/高)× 买/卖价算计量电费,快照不可变;日/月/年汇总加固定费减 heffingskorting
- **反哺 Home Assistant Energy**:当前买/卖价 + 累计买电支出/卖电收入(`total_increasing`)发成 HA 实体,可直接挂 HA Energy 仪表盘
- pytest 测试与 OpenAPI 导出脚本
- Docker / Compose 部署入口
@@ -30,6 +37,13 @@
- public IPv4 当前状态与变化历史
- location 记录(`location` 表)
- poo 记录(`poo_records` 表)
- Modbus 设备定义(`modbus_device` 表)
- Modbus 通用读数(`modbus_reading` 表,JSON payload
- HA 实体暴露开关(`exposed_entity_toggle` 表)
- DSMR 电表实时读数(`dsmr_reading` 表,整帧 JSON blob10s 降采样)
- 电价合同(`energy_contract` 表)与版本(`energy_contract_version` 表,values JSON
- Tibber 15 分钟电价缓存(`tibber_price` 表,不可变)
- 每 15 分钟计量电费(`energy_cost_period` 表,快照价,不可变)
配置层只保留一个数据库环境变量:
@@ -41,7 +55,7 @@
python -m scripts.run_migrations
```
该命令会通过 Alembic 将 `app.db` 初始化或升级到最新 head(含 `location` / `poo_records`)。
该命令会通过 Alembic 将 `app.db` 初始化或升级到最新 head(含全部表,包括 M5 新增的 `modbus_device``modbus_reading``exposed_entity_toggle`,以及 M6 新增的 `dsmr_reading``energy_contract``energy_contract_version``tibber_price``energy_cost_period`)。
## 当前目录
@@ -49,7 +63,7 @@ python -m scripts.run_migrations
- `app/`: FastAPI 应用代码(包含 JSON API、业务服务、数据模型)
- `frontend/`: React SPA 前端(Vite + React + TypeScript + Mantine
- `alembic_app/`: App DB 的 Alembic migration 环境(同时管理 `location` / `poo_records`
- `alembic_app/`: App DB 的 Alembic migration 环境(管理所有表,含 M5 新增的 `modbus_device``modbus_reading``exposed_entity_toggle`,以及 M6 新增的 `dsmr_reading``energy_contract``energy_contract_version``tibber_price``energy_cost_period`
- `tests/`: pytest 测试
- `docs/`: 当前系统说明文档
- `scripts/`: 辅助脚本,例如 OpenAPI 导出
@@ -128,6 +142,7 @@ M2 用 React SPA 取代了原有 Jinja 服务端模板,由 FastAPI 同源托
- **Vite + React + TypeScript + Mantine**(组件库)
- **TanStack Query**(数据请求/缓存)
- **Leaflet / react-leaflet**(地图与热力图)
- **Recharts**Energy 视图走势图,M5 引入)
- **openapi-typescript + openapi-fetch**(类型化 API client,由 `openapi/openapi.json` 生成)
### 本地开发(前端)
@@ -177,7 +192,7 @@ npm run build # 构建,确认产出 dist
- App DB`sqlite:///./data/app.db`
- 数据目录:`./data/`
所有模型(auth / config / public_ip / location / poo)共用同一个 `Base`,均通过单一 Alembic 链管理:
所有模型(auth / config / public_ip / location / poo / modbus / expose)共用同一个 `Base`,均通过单一 Alembic 链管理:
- Alembic 环境:`alembic_app.ini` + `alembic_app/`
- 统一 migration job`python -m scripts.run_migrations`
@@ -228,6 +243,195 @@ React SPA 主要页面路由(客户端路由,均由 FastAPI fallback 到 `in
无论是本地 `host:port` 还是反向代理后的域名访问,登录成功后进入 SPA 首页(`/`)。
## M4 登录加固
M4 在基础鉴权之上叠加了三层防御,详细说明见 [`docs/auth.md`](./docs/auth.md)。
### 防爆破 / 指数退避
登录失败超过 3 次后进入指数退避(`wait = min(900s, 1s × 2^(failures-3))`),期间请求返回 `429 Too Many Requests`(含 `Retry-After` 响应头);成功登录后自动清零。退避按 **client IP****username** 双键取较大值,不会因此永久锁定账号(只是延迟,不是封号)。
- 全局开关:`AUTH_LOGIN_THROTTLE_ENABLED`CONFIG_FIELDS,默认 `true`
- 反代后需要 `AUTH_TRUST_FORWARDED_FOR=true` 才会读 `X-Forwarded-For``.env` 部署级配置,默认 `false`
### CLI 逃生通道
拿到服务器 CLI 权限时,可以在**不依赖任何已存凭据**(无需密码、恢复码)的情况下重置密码、解锁退避、关停 TOTP:
```bash
# 重置密码(不加 --password 则交互式输入,不回显)
python -m scripts.admin_cli reset-password admin
# 解锁退避(被 429 挡住时使用)
python -m scripts.admin_cli unlock --all # 清所有退避行
python -m scripts.admin_cli unlock --ip 1.2.3.4 # 按 IP 清
python -m scripts.admin_cli unlock --username admin # 按 username 清
# TOTP 相关(需要先启用,见下文)
python -m scripts.admin_cli disable-totp admin # 关停 TOTP(零凭据,逃生口)
python -m scripts.admin_cli reissue-totp admin # 重新发放 TOTP secret(打印新 URI
# 查看用户列表
python -m scripts.admin_cli list-admin
```
在 Docker 容器内执行时:
```bash
docker compose exec app python -m scripts.admin_cli <command>
```
### 可选 TOTP 二次验证
admin 可在 React SPA 设置页(`/config`)自选启用 RFC 6238 TOTP
1. 设置页点「启用 TOTP」→ 后端生成 `otpauth://` URI,前端渲染二维码(`qrcode.react`
2. 用 Authenticator App(如 Google Authenticator、Authy)扫码
3. 输入当前 6 位动态码确认 → TOTP 启用
4. 妥善保存一次性展示的 10 个恢复码(格式 `xxxx-xxxx`
启用后,登录需要两步:密码 → 6 位动态码(或恢复码,一次性)。不启用则维持纯密码登录,行为不变。
恢复码丢失时,可用 CLI 逃生:`python -m scripts.admin_cli disable-totp admin`,随后即可纯密码登录。
TOTP issuer 标签(显示在 Authenticator 里)通过 `AUTH_TOTP_ISSUER` 环境变量配置(`.env` 部署级),默认回退 `app_name`
## M5 Modbus 设备采集 / Energy / MQTT + HA Discovery
M5 给后端接入家庭 IoT 生态,新增通用 Modbus 采集链路(首个领域:能耗)、MQTT + Home Assistant Discovery 发布,以及前端侧边栏与 Energy 视图。
### 依赖
后端新增:
- `pymodbus`Modbus-TCP 客户端(轮询电表等 slave 设备)
- `paho-mqtt`MQTT 客户端(HA Discovery 与 state 发布)
- `pyyaml`YAML profile 加载(设备协议声明式描述)
前端新增:
- `recharts`Energy 视图走势图
### Modbus 设备采集
采集链路采用两层分离:**YAML profile**(协议知识,随代码走)+ **`modbus_device` 数据库行**(部署/可配置信息)+ **`modbus_reading` 通用读数表**JSON payload 遥测)。
- **profile**(如 `sdm120.yaml`)描述:读哪些寄存器(FC04 块读)、每个量的 key/unit/device_class/ha_component。纯协议知识,不含 unit_id / friendly_name 等部署项。
- **`modbus_device` 行**friendly_name、网关 host/port、Modbus slave `unit_id`(电表 Meter ID,设备面板可改故落 DB)、选用哪个 profile、采样周期、是否启用。
- **`modbus_reading` 行**device_id FK、recorded_at、payloadJSON,如 `{"voltage": 230.2, "current": 1.3, ...}`)。
- 多设备可共享同一 profile(如两块 SDM120 共用 `sdm120` profile,各自独立 unit_id 和 friendly_name)。
- APScheduler 后台 job 周期轮询所有 `enabled` 设备,更新 `last_poll_at` / `last_poll_ok`。全局开关 `MODBUS_POLLING_ENABLED`CONFIG_FIELDS)。
**手工命令行试读**(不依赖 DB,最快验证网关连通性):
```bash
# 按 profile 解码读一次(验证整套解码链路)
python -m scripts.modbus_cli read --host <网关IP> --port 502 --unit 1 --profile sdm120
# 手工指定请求内容(first-contact 验证,不依赖 profile
python -m scripts.modbus_cli probe --host <网关IP> --port 502 --unit 1 --fc 4 --address 0x0000 --count 2 --decode float32
```
CLI 工具为受控手工验证而设(设备需接市电),仅暴露读功能码(FC03/04),无写寄存器子命令。
### MQTT + Home Assistant Discovery
后端作为 MQTT 发布方,把 Modbus 设备/工程量以 HA Discovery 协议自动注册为 device/entity
- 每个 Modbus 设备 = 一个 HA device`unique_id` 锚定于设备 `uuid`,不随改名变)
- 各工程量 = sensor entitydevice_class/unit 取自 YAML profile);另有 binary_sensor `online`(取 `last_poll_ok`
- 每次轮询成功后推 state topic;连接成功或勾选变更时重发 retained discovery config
- `POST /api/config/mqtt/test`:试连 broker 并发布一条测试消息(可用 MQTT Explorer 验证链路)
**Config 页配置流程**
1.`/config` 的「MQTT」section 填写 broker host/port/username/password,启用 `MQTT_ENABLED`
2. 点「发送测试消息」确认 broker 链路通
3. 启用 `HA_DISCOVERY_ENABLED`
4. 在「Home Assistant Expose」面板勾选要暴露的实体,点「重新发布 discovery」
5. 在 Home Assistant 确认对应 device/entity 出现
### API 端点(M5 新增)
| 端点 | 用途 |
| --- | --- |
| `GET /api/modbus/devices` | 列出 Modbus 设备 |
| `POST /api/modbus/devices` | 新建设备 |
| `GET /api/modbus/devices/{uuid}` | 单个设备 |
| `PATCH /api/modbus/devices/{uuid}` | 修改设备(含 enable/disable|
| `DELETE /api/modbus/devices/{uuid}` | 删除设备;有读数时 409 |
| `GET /api/modbus/devices/{uuid}/metrics` | 该设备 profile 的量目录(key/unit/device_class|
| `GET /api/modbus/devices/{uuid}/latest` | 最新一条读数 payload |
| `GET /api/modbus/devices/{uuid}/readings` | 时间范围读数(start/end/limit),供走势图 |
| `POST /api/modbus/devices/{uuid}/test` | 即时试读(不落库) |
| `GET /api/modbus/profiles` | 列出可用 profile 名 + 描述 |
| `GET /api/expose` | 可暴露实体目录 + 勾选状态 + MQTT/Discovery 状态 |
| `PUT /api/expose` | 设置逐 key 暴露开关 |
| `POST /api/expose/republish` | 手动重发 discovery |
| `POST /api/config/mqtt/test` | 试连 broker 并发布测试消息 |
### 前端视图(M5 新增)
- **侧边栏**:把顶栏改为侧边导航(Home / Records / Energy / Config + 主题切换 + 注销),当前路由高亮,移动端可折叠。
- **`/energy`Energy 视图)**:设备 CRUD(新建/编辑/删除,删除有二次确认;有读数时引导改用禁用);最新读数卡片(字段标签/单位取自 profile metrics);时间序列走势图(Recharts,支持电压/电流/功率/电能,带时间范围选择)。
- **Config 页 Accordion**:各大 config section 可独立折叠/展开;「Home Assistant Expose」面板按设备分组勾选可暴露实体、显示 MQTT/Discovery 连接状态、「重新发布 discovery」按钮。
- SPA 路由新增 `/energy`
## M6 DSMR 接入 / 电价合同 / 实时电费计算 / HA Energy 反哺
M6 在 M5 IoT 基建之上接入 DSMR 实时智能电表数据,建立通用电价合同层,按每 15 分钟算出实际买卖电费并反哺 HA Energy。
### 依赖
M6 **不新增任何 Python 依赖**,复用 M5 已有的 `httpx`Tibber GraphQL)、`paho-mqtt`DSMR 订阅)、`pyyaml`pricing profile 加载)、`apscheduler`(抓价 job、计费 job)。
### DSMR 实时电表接入
订阅 DSMR Reader 的 `dsmr/json` topic(每秒一帧完整 telegram),整帧存为 JSON blob、按 `dsmr_sample_interval_s`(默认 10 秒)降采样落 `dsmr_reading``source_id` 幂等去重)。`dsmr_ingest_enabled`(默认 falseopt-in)。
### 电价合同层
- **YAML profile 定结构**(仓库内,不放数值):`manual.yaml`(固定/双费率:buy_normal/dal、sell_normal/dal、energy_tax、ode、固定费、heffingskorting);`tibber.yaml`(动态:source=tibber_apienergy_tax、sell_adjust
- **`EnergyContract` + `EnergyContractVersion`**(UI 填数值):改价 = 加新版本行(带 `effective_from`),旧版本保留(审计链);一次只有一个 active 合同
- **price strategy**`manual` 用双费率常数(`buy = energy_buy_档 + energy_tax``sell = sell_档`);`tibber``tibber_price.total` 作买价(已含税,demo 确认 `total=energy+tax`),`total energy_tax sell_adjust` 作卖价(卖价残差 `sell_adjust` 默认 0,待真实账单核定)
### 每 15 分钟计量电费(不可变)
APScheduler 1 分钟 tick,取每个闭合 15 分钟窗口的 DSMR 寄存器差值(`delivered_1/2``returned_1/2``_1`=dal/低,`_2`=normal/高,NL 惯例)× 当时合同版本的 strategy 出价,upsert `energy_cost_period`(快照当时价 + `contract_version_id`)。缺价/缺数据时标 `degraded``POST /api/energy/costs/recompute` 显式重算。
日/月/年汇总 = Σnet + 固定费(network_fee + management_fee 按月→天 × 天数)- heffingskorting(按年→天 × 天数),读时计算、不落表。能源税 `energy_tax` 参考值约 0.1108 EUR/kWh2026 第一档含 VAT,待真实账单核定;该值由 UI 填入合同版本,YAML profile 仅声明字段 unit,代码无写死默认数值)。
### Tibber 动态电价
`app/integrations/tibber/client.py` httpx POST GraphQL`priceInfoRange(QUARTER_HOURLY, first=96)`),解析 `startsAt`/`total`/`energy`/`tax`/`level`,按 `starts_at` upsert `tibber_price`(幂等)。启动 + 每小时抓取今明两天 15 分钟价、幂等 upserthourly trigger,确保每日刷新且可补重试);仅当 active 合同 kind=tibber 且 `tibber_api_token` 存在时运行。`POST /api/energy/tibber/test` 试连三态(success 带当前价 / config-error / failed)。
### 反哺 Home Assistant Energy
`_energy_cost_provider` 向 expose 框架注册 4 个实体:`buy_price_now``sell_price_now`(€/kWh sensor)、`import_cost_total``export_revenue_total``total_increasing` monetary,可直接挂 HA Energy 仪表盘)。默认未勾选,在 Expose 面板启用。
### API 端点(M6 新增)
| 端点 | 用途 |
| --- | --- |
| `GET /api/energy/contracts` | 列出合同 + active 标记 |
| `POST /api/energy/contracts` | 新建合同(kind + 首版本值,按 profile 校验)|
| `GET /api/energy/contracts/{id}` | 单个合同 + 版本历史 |
| `PATCH /api/energy/contracts/{id}` | 改名 / 激活 |
| `POST /api/energy/contracts/{id}/versions` | 加新版本(改价,带生效日期)|
| `GET /api/energy/profiles` | 列出 pricing profile 结构(前端按它渲染表单)|
| `GET /api/energy/prices` | 区间价格点(曲线)|
| `GET /api/energy/costs` | 区间 `energy_cost_period`(走势/明细)|
| `GET /api/energy/costs/summary` | 区间汇总(计量电费 + 固定费 − 抵扣)|
| `POST /api/energy/costs/recompute` | 幂等重算 |
| `GET /api/energy/dsmr/latest` | 最新 `dsmr_reading` |
| `POST /api/energy/tibber/test` | 试连 Tibber + 拉当前价,三态 |
DSMR/Tibber 标量配置复用现有 `GET/PUT /api/config`(新增 `dsmr_ingest_enabled``dsmr_mqtt_topic``dsmr_sample_interval_s``tibber_api_token`secret)、`tibber_home_id`)。
### 前端视图(M6 新增,并入 Energy 视图)
- **Contracts Tab**:合同列表 + 新建/编辑(表单按 `/api/energy/profiles` 结构渲染,不 hardcode 字段)+ 激活 + 改价加版本 + 版本历史只读。
- **Prices Tab**15 分钟价格曲线(tibber 动态或 manual 档位),复用 Recharts。
- **Costs Tab**:费用走势/明细 + 汇总卡片(含固定费/抵扣)。
- **Config 页 Tibber 测试**:三态(success/config-error/failed)。
## Config 持久化
当前 config 页面不会把修改写回 `.env`
@@ -253,6 +457,11 @@ React SPA 主要页面路由(客户端路由,均由 FastAPI fallback 到 `in
- SMTP 基础配置
- TickTick OAuth 配置
- Home Assistant 配置
- MQTT broker 配置(`MQTT_ENABLED``MQTT_BROKER_HOST/PORT/USERNAME/PASSWORD``MQTT_TLS_ENABLED`
- Home Assistant Discovery 配置(`HA_DISCOVERY_ENABLED``HA_DISCOVERY_PREFIX`
- Modbus 采集配置(`MODBUS_POLLING_ENABLED`
- DSMR 接入配置(`DSMR_INGEST_ENABLED``DSMR_MQTT_TOPIC``DSMR_SAMPLE_INTERVAL_S`
- Tibber 凭据(`TIBBER_API_TOKEN`secret)、`TIBBER_HOME_ID`
其中 SMTP password 与其他 secret 字段一致:
+11 -1
View File
@@ -6,10 +6,20 @@ from sqlalchemy import engine_from_config, pool
from app.config import get_settings
from app.db import Base
from app.models.config import AppConfigEntry # noqa: F401
from app.models.auth import AuthSession, AuthUser # noqa: F401
from app.models.auth import AuthSession, AuthUser, RecoveryCode # noqa: F401
from app.models.auth_throttle import LoginThrottle # noqa: F401
from app.models.public_ip import PublicIPHistory, PublicIPState # noqa: F401
from app.models.location import Location # noqa: F401
from app.models.poo import PooRecord # noqa: F401
from app.models.modbus import ModbusDevice, ModbusReading # noqa: F401
from app.models.expose import ExposedEntityToggle # noqa: F401
from app.models.energy import ( # noqa: F401
DsmrReading,
EnergyContract,
EnergyContractVersion,
TibberPrice,
EnergyCostPeriod,
)
config = context.config
@@ -0,0 +1,43 @@
"""add auth_login_throttle table for exponential back-off throttling
Revision ID: 20260621_07_auth_login_throttle
Revises: 20260611_06_merge_location_poo_tables
Create Date: 2026-06-21 00:00:00.000000
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
revision: str = "20260621_07_auth_login_throttle"
down_revision: Union[str, None] = "20260611_06_merge_location_poo_tables"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.create_table(
"auth_login_throttle",
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
sa.Column("key", sa.String(length=255), nullable=False),
sa.Column("scope", sa.String(length=16), nullable=False),
sa.Column("failures", sa.Integer(), nullable=False),
sa.Column("first_failed_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("last_failed_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("next_allowed_at", sa.DateTime(timezone=True), nullable=True),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("scope", "key", name="uq_auth_login_throttle_scope_key"),
)
op.create_index(
"ix_auth_login_throttle_scope_key",
"auth_login_throttle",
["scope", "key"],
unique=False,
)
def downgrade() -> None:
op.drop_index("ix_auth_login_throttle_scope_key", table_name="auth_login_throttle")
op.drop_table("auth_login_throttle")
+61
View File
@@ -0,0 +1,61 @@
"""add TOTP fields to auth_users and create auth_recovery_code table
Revision ID: 20260621_08_totp
Revises: 20260621_07_auth_login_throttle
Create Date: 2026-06-21 00:00:00.000000
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
revision: str = "20260621_08_totp"
down_revision: Union[str, None] = "20260621_07_auth_login_throttle"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# Add totp_secret (nullable) to auth_users — existing rows get NULL, which is correct.
op.add_column("auth_users", sa.Column("totp_secret", sa.String(length=64), nullable=True))
# Add totp_enabled (NOT NULL) with server_default="0" so existing rows default to false.
op.add_column(
"auth_users",
sa.Column(
"totp_enabled",
sa.Boolean(),
nullable=False,
server_default="0",
),
)
# Create auth_recovery_code table for one-time TOTP recovery codes.
op.create_table(
"auth_recovery_code",
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
sa.Column("user_id", sa.Integer(), nullable=False),
sa.Column("code_hash", sa.String(length=255), nullable=False),
sa.Column("used_at", sa.DateTime(timezone=True), nullable=True),
sa.ForeignKeyConstraint(["user_id"], ["auth_users.id"]),
sa.PrimaryKeyConstraint("id"),
)
op.create_index(
op.f("ix_auth_recovery_code_user_id"),
"auth_recovery_code",
["user_id"],
unique=False,
)
def downgrade() -> None:
# Drop auth_recovery_code table first (has FK to auth_users).
op.drop_index(op.f("ix_auth_recovery_code_user_id"), table_name="auth_recovery_code")
op.drop_table("auth_recovery_code")
# Drop the two TOTP columns from auth_users.
# Use batch_alter_table for SQLite compatibility (alembic's portable column-drop path).
with op.batch_alter_table("auth_users") as batch_op:
batch_op.drop_column("totp_enabled")
batch_op.drop_column("totp_secret")
@@ -0,0 +1,81 @@
"""add modbus_device and modbus_reading tables
Revision ID: 20260622_09_modbus_tables
Revises: 20260621_08_totp
Create Date: 2026-06-22 00:00:00.000000
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
revision: str = "20260622_09_modbus_tables"
down_revision: Union[str, None] = "20260621_08_totp"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# modbus_device — deployment/configurable metadata for each polled device.
op.create_table(
"modbus_device",
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
sa.Column("uuid", sa.String(length=36), nullable=False),
sa.Column("friendly_name", sa.String(length=255), nullable=False),
sa.Column("transport", sa.String(length=16), nullable=False),
sa.Column("host", sa.String(length=255), nullable=False),
sa.Column("port", sa.Integer(), nullable=False),
sa.Column("unit_id", sa.Integer(), nullable=False),
sa.Column("profile", sa.String(length=64), nullable=False),
sa.Column("poll_interval_s", sa.Integer(), nullable=False),
sa.Column("enabled", sa.Boolean(), nullable=False),
sa.Column("last_poll_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("last_poll_ok", sa.Boolean(), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("uuid", name="uq_modbus_device_uuid"),
)
# modbus_reading — generic telemetry, one row per device per poll cycle.
op.create_table(
"modbus_reading",
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
sa.Column("device_id", sa.Integer(), nullable=False),
sa.Column("recorded_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("payload", sa.JSON(), nullable=False),
sa.ForeignKeyConstraint(
["device_id"],
["modbus_device.id"],
ondelete="RESTRICT",
),
sa.PrimaryKeyConstraint("id"),
)
# Individual index on recorded_at (from the ORM-level index=True).
op.create_index(
"ix_modbus_reading_recorded_at",
"modbus_reading",
["recorded_at"],
unique=False,
)
# Composite index for efficient time-range queries per device.
op.create_index(
"ix_modbus_reading_device_recorded",
"modbus_reading",
["device_id", "recorded_at"],
unique=False,
)
def downgrade() -> None:
# Drop the reading table first (it has a FK referencing modbus_device).
op.drop_index("ix_modbus_reading_device_recorded", table_name="modbus_reading")
op.drop_index("ix_modbus_reading_recorded_at", table_name="modbus_reading")
op.drop_table("modbus_reading")
# Drop the device table.
op.drop_table("modbus_device")
@@ -0,0 +1,44 @@
"""add exposed_entity_toggle table
Revision ID: 20260622_10_exposed_entities
Revises: 20260622_09_modbus_tables
Create Date: 2026-06-22 00:00:00.000000
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
revision: str = "20260622_10_exposed_entities"
down_revision: Union[str, None] = "20260622_09_modbus_tables"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# exposed_entity_toggle — per-entity on/off switch for MQTT / HA Discovery.
op.create_table(
"exposed_entity_toggle",
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
sa.Column("key", sa.String(length=255), nullable=False),
sa.Column("enabled", sa.Boolean(), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("key", name="uq_exposed_entity_toggle_key"),
)
# Index on key for fast single-key lookups (toggle by key).
op.create_index(
"ix_exposed_entity_toggle_key",
"exposed_entity_toggle",
["key"],
unique=True,
)
def downgrade() -> None:
# Drop index then table — only removes what this revision created.
op.drop_index("ix_exposed_entity_toggle_key", table_name="exposed_entity_toggle")
op.drop_table("exposed_entity_toggle")
@@ -0,0 +1,132 @@
"""add energy pricing and DSMR metering tables
Revision ID: 20260623_11_energy_tables
Revises: 20260622_10_exposed_entities
Create Date: 2026-06-23 00:00:00.000000
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
revision: str = "20260623_11_energy_tables"
down_revision: Union[str, None] = "20260622_10_exposed_entities"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# dsmr_reading — raw DSMR telegram blobs (10-second down-sampled).
# No device table: single P1 smart meter; a ``source`` column can be added later
# if a second meter is introduced.
op.create_table(
"dsmr_reading",
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
sa.Column("recorded_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("source_id", sa.Integer(), nullable=True),
sa.Column("payload", sa.JSON(), nullable=False),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("source_id", name="uq_dsmr_reading_source_id"),
)
op.create_index(
"ix_dsmr_reading_recorded_at",
"dsmr_reading",
["recorded_at"],
unique=False,
)
# energy_contract — contract head (manual or tibber, one active at a time).
op.create_table(
"energy_contract",
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
sa.Column("name", sa.String(length=255), nullable=False),
sa.Column("kind", sa.String(length=32), nullable=False),
sa.Column("active", sa.Boolean(), nullable=False),
sa.Column("currency", sa.String(length=8), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False),
sa.PrimaryKeyConstraint("id"),
)
# energy_contract_version — versioned pricing values; append-only for auditability.
# Must be created after energy_contract because of the FK dependency.
op.create_table(
"energy_contract_version",
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
sa.Column("contract_id", sa.Integer(), nullable=False),
sa.Column("effective_from", sa.DateTime(timezone=True), nullable=False),
sa.Column("effective_to", sa.DateTime(timezone=True), nullable=True),
sa.Column("values", sa.JSON(), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(
["contract_id"],
["energy_contract.id"],
ondelete="RESTRICT",
),
sa.PrimaryKeyConstraint("id"),
)
# tibber_price — cached Tibber 15-minute spot prices (immutable once fetched).
op.create_table(
"tibber_price",
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
sa.Column("starts_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("resolution", sa.String(length=32), nullable=False),
sa.Column("energy", sa.Float(), nullable=False),
sa.Column("tax", sa.Float(), nullable=False),
sa.Column("total", sa.Float(), nullable=False),
sa.Column("level", sa.String(length=32), nullable=True),
sa.Column("currency", sa.String(length=8), nullable=False),
sa.Column("fetched_at", sa.DateTime(timezone=True), nullable=False),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("starts_at", name="uq_tibber_price_starts_at"),
)
# energy_cost_period — computed 15-minute billing periods (immutable snapshot).
# Must be created after energy_contract_version because of the FK dependency.
op.create_table(
"energy_cost_period",
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
sa.Column("period_start", sa.DateTime(timezone=True), nullable=False),
sa.Column("d1_kwh", sa.Float(), nullable=False),
sa.Column("d2_kwh", sa.Float(), nullable=False),
sa.Column("r1_kwh", sa.Float(), nullable=False),
sa.Column("r2_kwh", sa.Float(), nullable=False),
sa.Column("import_cost", sa.Float(), nullable=False),
sa.Column("export_revenue", sa.Float(), nullable=False),
sa.Column("net_cost", sa.Float(), nullable=False),
sa.Column("currency", sa.String(length=8), nullable=False),
sa.Column("pricing", sa.JSON(), nullable=False),
sa.Column("contract_version_id", sa.Integer(), nullable=True),
sa.Column("degraded", sa.Boolean(), nullable=False),
sa.Column("computed_at", sa.DateTime(timezone=True), nullable=False),
sa.ForeignKeyConstraint(
["contract_version_id"],
["energy_contract_version.id"],
ondelete="RESTRICT",
),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("period_start", name="uq_energy_cost_period_period_start"),
)
def downgrade() -> None:
# Drop tables in reverse dependency order: tables with FKs first.
# energy_cost_period has a FK to energy_contract_version — drop it first.
op.drop_table("energy_cost_period")
# tibber_price has no FK dependencies on our new tables.
op.drop_table("tibber_price")
# energy_contract_version has a FK to energy_contract — drop it before energy_contract.
op.drop_table("energy_contract_version")
# energy_contract has no FK dependencies on our new tables.
op.drop_table("energy_contract")
# dsmr_reading has no FK dependencies.
op.drop_index("ix_dsmr_reading_recorded_at", table_name="dsmr_reading")
op.drop_table("dsmr_reading")
@@ -0,0 +1,58 @@
"""decouple dsmr_reading from the telegram id
The DSMR Reader's own ``id`` field is auto-incrementing but is known to overflow
and require a manual reset to zero (a long-standing DSMR firmware quirk). If we
keep a UNIQUE constraint on ``source_id`` (the telegram id) and use it for
idempotency, an overflow/reset would make legitimately-new telegrams collide with
old ids and be silently dropped as "duplicates" — data loss.
This migration removes that coupling:
- Drops the UNIQUE constraint on ``source_id`` (the column is kept as a plain,
nullable reference value; it is no longer relied upon for uniqueness/dedup).
- Makes ``recorded_at`` (the telegram timestamp) UNIQUE instead — a single P1
meter emits one telegram per timestamp, so this is a robust, telegram-id-
independent idempotency key. The table's own autoincrement ``id`` PK remains
the stable internal identity.
Revision ID: 20260624_12_dsmr_decouple_telegram_id
Revises: 20260623_11_energy_tables
Create Date: 2026-06-24 00:00:00.000000
"""
from typing import Sequence, Union
from alembic import op
revision: str = "20260624_12_dsmr_decouple_telegram_id"
down_revision: Union[str, None] = "20260623_11_energy_tables"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# SQLite cannot ALTER away a constraint in place, so recreate the table via
# Alembic's batch mode. The table is recreated and rows are copied; existing
# data (if any) is preserved. recorded_at must be unique for this to succeed
# — a single meter never emits two telegrams at the exact same timestamp.
with op.batch_alter_table("dsmr_reading", schema=None) as batch_op:
# Old non-unique index on recorded_at is replaced by the unique constraint.
batch_op.drop_index("ix_dsmr_reading_recorded_at")
# Telegram id is no longer a uniqueness/idempotency key.
batch_op.drop_constraint("uq_dsmr_reading_source_id", type_="unique")
# Timestamp becomes the telegram-id-independent dedup key.
batch_op.create_unique_constraint(
"uq_dsmr_reading_recorded_at", ["recorded_at"]
)
def downgrade() -> None:
with op.batch_alter_table("dsmr_reading", schema=None) as batch_op:
batch_op.drop_constraint("uq_dsmr_reading_recorded_at", type_="unique")
batch_op.create_unique_constraint(
"uq_dsmr_reading_source_id", ["source_id"]
)
batch_op.create_index(
"ix_dsmr_reading_recorded_at", ["recorded_at"], unique=False
)
@@ -0,0 +1,238 @@
"""add meter table and energy_cost_period.meter_id
Introduces the ``meter`` table (one row per physical meter installation epoch)
and a nullable FK column ``energy_cost_period.meter_id`` that attributes each
billing period to a specific physical meter.
**Backfill logic (§3.7 of the M7 design doc)**:
If the database already contains any ``dsmr_reading`` or
``energy_cost_period`` rows, one initial ``meter`` row is created:
label = "Initial meter"
commodity = "electricity"
started_at = earliest dsmr_reading.recorded_at
(or, if none, earliest energy_cost_period.period_start,
or, if still none, the migration timestamp)
ended_at = NULL (still active)
reason = "initial"
All existing ``energy_cost_period`` rows are then back-filled with that
initial meter's id.
**Idempotency**: the backfill is guarded with a check for any existing
``meter`` row whose ``reason = 'initial'`` and ``commodity = 'electricity'``
and ``ended_at IS NULL``, so repeating the upgrade does not create duplicate
meters or overwrite already-filled meter_id values.
**Audit (on-non-degraded periods only)**: after backfilling, the number of
non-degraded ``energy_cost_period`` rows with ``meter_id IS NULL`` must be
zero; if it is not, the migration raises a ``RuntimeError`` and rolls back.
**Data safety**: this migration is additive only — no existing rows are
deleted or overwritten; it only creates a new table, adds a nullable column,
and back-fills that column.
Revision ID: 20260625_13_meter_table
Revises: 20260624_12_dsmr_decouple_telegram_id
Create Date: 2026-06-25 00:00:00.000000
"""
from datetime import datetime, timezone
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
revision: str = "20260625_13_meter_table"
down_revision: Union[str, None] = "20260624_12_dsmr_decouple_telegram_id"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# ------------------------------------------------------------------ #
# 1. Create the meter table. #
# ------------------------------------------------------------------ #
op.create_table(
"meter",
sa.Column("id", sa.Integer(), autoincrement=True, nullable=False),
sa.Column("label", sa.String(length=255), nullable=False),
sa.Column("commodity", sa.String(length=32), nullable=False),
sa.Column("started_at", sa.DateTime(timezone=True), nullable=False),
sa.Column("ended_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("reason", sa.String(length=64), nullable=False),
sa.Column("note", sa.String(length=1024), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), nullable=False),
sa.PrimaryKeyConstraint("id"),
)
# ------------------------------------------------------------------ #
# 2. Add meter_id column to energy_cost_period (nullable FK). #
# ------------------------------------------------------------------ #
with op.batch_alter_table("energy_cost_period", schema=None) as batch_op:
batch_op.add_column(
sa.Column("meter_id", sa.Integer(), nullable=True)
)
batch_op.create_foreign_key(
"fk_energy_cost_period_meter_id",
"meter",
["meter_id"],
["id"],
ondelete="RESTRICT",
)
# ------------------------------------------------------------------ #
# 3. Backfill initial meter (idempotent). #
# ------------------------------------------------------------------ #
conn = op.get_bind()
# Check if there is already an initial meter (idempotency guard).
existing_initial = conn.execute(
sa.text(
"SELECT id FROM meter "
"WHERE reason = 'initial' AND commodity = 'electricity' AND ended_at IS NULL "
"LIMIT 1"
)
).fetchone()
if existing_initial is not None:
# Already backfilled — nothing to do.
return
# Determine whether there is any historical data to create a meter for.
has_readings = conn.execute(
sa.text("SELECT 1 FROM dsmr_reading LIMIT 1")
).fetchone()
has_periods = conn.execute(
sa.text("SELECT 1 FROM energy_cost_period LIMIT 1")
).fetchone()
if not has_readings and not has_periods:
# Empty database: no historical data, so no initial meter is needed.
# meter_id will remain NULL on any future rows until T02 service layer
# starts populating it.
return
# Determine started_at: earliest dsmr_reading.recorded_at, falling back to
# earliest energy_cost_period.period_start, and finally to now().
earliest_reading_row = conn.execute(
sa.text("SELECT MIN(recorded_at) AS ts FROM dsmr_reading")
).fetchone()
earliest_period_row = conn.execute(
sa.text("SELECT MIN(period_start) AS ts FROM energy_cost_period")
).fetchone()
started_at_value: datetime | None = None
if earliest_reading_row and earliest_reading_row[0] is not None:
# SQLite returns ISO strings for datetime columns; parse to datetime.
raw = earliest_reading_row[0]
started_at_value = _parse_sqlite_datetime(raw)
if started_at_value is None and earliest_period_row and earliest_period_row[0] is not None:
raw = earliest_period_row[0]
started_at_value = _parse_sqlite_datetime(raw)
if started_at_value is None:
started_at_value = datetime.now(tz=timezone.utc)
now_utc = datetime.now(tz=timezone.utc)
# Insert the initial meter row.
conn.execute(
sa.text(
"INSERT INTO meter (label, commodity, started_at, ended_at, reason, note, created_at) "
"VALUES (:label, :commodity, :started_at, NULL, :reason, NULL, :created_at)"
),
{
"label": "Initial meter",
"commodity": "electricity",
"started_at": _iso(started_at_value),
"reason": "initial",
"created_at": _iso(now_utc),
},
)
# Retrieve the newly created meter id.
meter_row = conn.execute(
sa.text(
"SELECT id FROM meter "
"WHERE reason = 'initial' AND commodity = 'electricity' AND ended_at IS NULL "
"LIMIT 1"
)
).fetchone()
assert meter_row is not None, "Initial meter row not found after insert"
meter_id: int = meter_row[0]
# Back-fill all existing energy_cost_period rows that have meter_id IS NULL.
conn.execute(
sa.text(
"UPDATE energy_cost_period SET meter_id = :mid WHERE meter_id IS NULL"
),
{"mid": meter_id},
)
# ------------------------------------------------------------------ #
# 4. Audit: verify all non-degraded periods have a meter_id. #
# ------------------------------------------------------------------ #
unmatched_row = conn.execute(
sa.text(
"SELECT COUNT(*) FROM energy_cost_period "
"WHERE meter_id IS NULL AND degraded = 0"
)
).fetchone()
unmatched_count: int = unmatched_row[0] if unmatched_row else 0
if unmatched_count != 0:
raise RuntimeError(
f"Meter backfill audit failed: {unmatched_count} non-degraded "
"energy_cost_period row(s) still have meter_id IS NULL after backfill. "
"Migration aborted to protect data integrity."
)
def downgrade() -> None:
# Remove the FK column from energy_cost_period first (references meter).
with op.batch_alter_table("energy_cost_period", schema=None) as batch_op:
batch_op.drop_constraint("fk_energy_cost_period_meter_id", type_="foreignkey")
batch_op.drop_column("meter_id")
# Drop the meter table.
op.drop_table("meter")
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _parse_sqlite_datetime(value: str | datetime) -> datetime:
"""Parse a SQLite datetime value into an aware UTC datetime.
SQLite stores datetimes as ISO 8601 strings. SQLAlchemy may return them
as plain strings or as naive datetimes (no tzinfo) depending on the driver
and column declaration. This helper normalises both forms to an aware UTC
``datetime``.
"""
if isinstance(value, datetime):
if value.tzinfo is None:
return value.replace(tzinfo=timezone.utc)
return value
# String form — strip trailing Z or +00:00 variants, then attach UTC.
s = str(value).strip()
for suffix in ("+00:00", "Z", " UTC"):
if s.endswith(suffix):
s = s[: -len(suffix)]
# SQLite uses space as the T separator in some formats.
s = s.replace(" ", "T")
try:
dt = datetime.fromisoformat(s)
except ValueError:
# Fallback: strip subseconds if present to handle unusual formats.
dt = datetime.strptime(s[:19], "%Y-%m-%dT%H:%M:%S")
return dt.replace(tzinfo=timezone.utc)
def _iso(dt: datetime) -> str:
"""Serialise a datetime to an ISO 8601 string for SQLite storage."""
if dt.tzinfo is not None:
dt = dt.astimezone(timezone.utc).replace(tzinfo=None)
return dt.strftime("%Y-%m-%dT%H:%M:%S")
@@ -0,0 +1,104 @@
"""add uuid column to meter table
Adds a stable ``uuid`` (UUID v4 string) column to the ``meter`` table so that
each meter epoch has a durable identity anchor suitable for use as an HA
Discovery ``unique_id``.
**Migration strategy (SQLite-safe)**:
SQLite does not support adding a NOT NULL + UNIQUE column to a non-empty table
in a single ``ALTER TABLE ADD COLUMN`` statement (adding a NOT NULL column
without a default value is rejected if the table already has rows). The
safe approach used here is:
1. Add ``uuid`` as a **nullable** column (SQLite allows this).
2. **Back-fill** every existing ``meter`` row with a distinct ``str(uuid4())``
value. Each row gets its *own* random UUID — not a shared value — so the
subsequent UNIQUE constraint is satisfied.
3. Use ``batch_alter_table`` (which re-creates the table under the hood in
SQLite) to alter the column to ``NOT NULL`` and add a UNIQUE constraint.
**Idempotency**: only rows where ``uuid IS NULL`` are back-filled; rows that
already have a uuid (e.g. from a repeated upgrade after a partial failure) are
left untouched.
**Audit**: after back-fill, the count of rows with ``uuid IS NULL`` must be
exactly zero; if not, the migration raises ``RuntimeError`` and rolls back.
**Data safety**: this migration is additive only — no existing rows are deleted
or overwritten; it only adds a new column and fills it in.
Revision ID: 20260625_14_meter_uuid
Revises: 20260625_13_meter_table
Create Date: 2026-06-25 00:00:00.000000
"""
import uuid as _uuid
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
revision: str = "20260625_14_meter_uuid"
down_revision: Union[str, None] = "20260625_13_meter_table"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
conn = op.get_bind()
# ------------------------------------------------------------------ #
# 1. Add uuid as a nullable column. #
# ------------------------------------------------------------------ #
with op.batch_alter_table("meter", schema=None) as batch_op:
batch_op.add_column(
sa.Column("uuid", sa.String(length=36), nullable=True)
)
# ------------------------------------------------------------------ #
# 2. Back-fill: assign a distinct UUID to every row that has #
# uuid IS NULL. Each row gets its own random value so that the #
# subsequent UNIQUE constraint is satisfied. #
# ------------------------------------------------------------------ #
rows = conn.execute(sa.text("SELECT id FROM meter WHERE uuid IS NULL")).fetchall()
for (meter_id,) in rows:
new_uuid = str(_uuid.uuid4())
conn.execute(
sa.text("UPDATE meter SET uuid = :uuid WHERE id = :mid"),
{"uuid": new_uuid, "mid": meter_id},
)
# ------------------------------------------------------------------ #
# 3. Audit: verify no rows remain with uuid IS NULL. #
# ------------------------------------------------------------------ #
null_count_row = conn.execute(
sa.text("SELECT COUNT(*) FROM meter WHERE uuid IS NULL")
).fetchone()
null_count: int = null_count_row[0] if null_count_row else 0
if null_count != 0:
raise RuntimeError(
f"meter.uuid back-fill audit failed: {null_count} meter row(s) still have "
"uuid IS NULL after back-fill. Migration aborted to protect data integrity."
)
# ------------------------------------------------------------------ #
# 4. Alter column to NOT NULL + UNIQUE (requires batch on SQLite). #
# batch_alter_table re-creates the table, so the UNIQUE constraint #
# and NOT NULL are applied atomically. #
# ------------------------------------------------------------------ #
with op.batch_alter_table("meter", schema=None) as batch_op:
batch_op.alter_column(
"uuid",
existing_type=sa.String(length=36),
nullable=False,
)
batch_op.create_unique_constraint("uq_meter_uuid", ["uuid"])
def downgrade() -> None:
# Drop the UNIQUE constraint and the uuid column (batch on SQLite).
with op.batch_alter_table("meter", schema=None) as batch_op:
batch_op.drop_constraint("uq_meter_uuid", type_="unique")
batch_op.drop_column("uuid")
+195 -2
View File
@@ -9,12 +9,14 @@ from sqlalchemy.orm import Session
from app.api.routes.api.deps import require_csrf, require_session
from app.config import Settings, get_settings
from app.dependencies import get_app_settings, get_db
from app.integrations.mqtt import MQTT_SETTINGS_KEYS, mqtt_manager
from app.schemas.config import (
ConfigField,
ConfigResponse,
ConfigSection,
ConfigUpdateRequest,
ConfigUpdateResponse,
MqttTestResponse,
SmtpTestResponse,
)
from app.services.auth import AuthenticatedSession
@@ -58,7 +60,12 @@ def put_config(
- Blank secret value keeps the existing stored value (no change).
- Invalid values return 422 and nothing is written to the database.
- If MQTT-related settings changed, the MQTT client reconnects automatically.
"""
# Detect whether any MQTT-related key is being submitted (non-secret change
# or non-blank secret change) so we know to reconnect after saving.
mqtt_keys_submitted = any(k.lower() in MQTT_SETTINGS_KEYS for k in body.updates)
try:
save_config_updates(db, body.updates, settings)
except ConfigSaveError as exc:
@@ -68,8 +75,23 @@ def put_config(
detail="invalid config submission",
) from exc
# Re-read settings after save (save_config_updates clears the settings cache)
refreshed_settings = get_settings()
# Re-read settings after save (save_config_updates clears the settings cache).
# Use build_runtime_settings so the reconnect picks up DB-stored values, not
# just the bootstrap env (otherwise a broker configured via the UI is ignored).
from app.services.config_page import build_runtime_settings
refreshed_settings = build_runtime_settings(db, get_settings())
# Reconnect MQTT client if any MQTT setting was updated.
if mqtt_keys_submitted:
logger.info("MQTT settings changed — triggering reconnect.")
mqtt_manager.reconnect(refreshed_settings)
# Re-apply the DSMR subscription so enabling/disabling DSMR ingest (or changing
# its topic / sample interval) takes effect immediately, without an app restart.
# Done after any MQTT reconnect so it operates on the current client.
from app.services.dsmr_ingest import apply_dsmr_subscription
apply_dsmr_subscription(refreshed_settings)
sections_raw = build_config_sections(db, refreshed_settings)
return ConfigUpdateResponse(sections=_sections_from_raw(sections_raw))
@@ -117,3 +139,174 @@ def post_smtp_test(
status_code=status.HTTP_200_OK,
content={"result": "success", "message": "Test email sent successfully."},
)
# ---------------------------------------------------------------------------
# POST /api/config/mqtt/test — M5-T10
# ---------------------------------------------------------------------------
class _MqttConfigurationError(ValueError):
"""Raised when MQTT settings are incomplete or disabled."""
class _MqttConnectionError(RuntimeError):
"""Raised when MQTT broker connection or publish fails."""
@router.post(
"/config/mqtt/test",
responses={
200: {"model": MqttTestResponse},
400: {"model": MqttTestResponse},
502: {"model": MqttTestResponse},
},
)
def post_mqtt_test(
settings: Settings = Depends(get_app_settings),
_auth: AuthenticatedSession = Depends(require_session),
_csrf: None = Depends(require_csrf),
) -> JSONResponse:
"""
Test MQTT broker connectivity by attempting to connect and publishing a
test message to ``<ha_discovery_prefix>/home-automation/test``.
The message is visible in MQTT Explorer (or any subscriber) so users can
confirm the full broker publish path is working.
Three possible outcomes:
- 200 { "result": "success", "message": ... }
- 400 { "result": "config-error", "message": ... } (not configured)
- 502 { "result": "failed", "message": ... } (connection/publish error)
MQTT credentials are never echoed in the response.
"""
try:
_run_mqtt_test(settings)
except _MqttConfigurationError as exc:
logger.warning("MQTT test rejected due to configuration: %s", exc)
return JSONResponse(
status_code=status.HTTP_400_BAD_REQUEST,
content={"result": "config-error", "message": str(exc)},
)
except _MqttConnectionError as exc:
logger.warning("MQTT test connection/publish failed: %s", exc)
return JSONResponse(
status_code=status.HTTP_502_BAD_GATEWAY,
content={"result": "failed", "message": str(exc)},
)
return JSONResponse(
status_code=status.HTTP_200_OK,
content={
"result": "success",
"message": (
f"Test message published to "
f"{settings.ha_discovery_prefix}/home-automation/test."
),
},
)
def _run_mqtt_test(settings: Settings) -> None:
"""Attempt a transient MQTT connection and publish a test message.
Raises
------
_MqttConfigurationError
When MQTT is not enabled or broker host is not configured.
_MqttConnectionError
When the broker is unreachable or the publish fails.
"""
import socket
import paho.mqtt.client as mqtt
if not settings.mqtt_broker_host:
raise _MqttConfigurationError("MQTT broker host is not configured.")
test_topic = f"{settings.ha_discovery_prefix}/home-automation/test"
test_payload = '{"source": "home-automation", "event": "mqtt_test"}'
connected_event = __import__("threading").Event()
published_event = __import__("threading").Event()
connect_error: list[str] = []
client = mqtt.Client(
callback_api_version=mqtt.CallbackAPIVersion.VERSION2,
client_id="home-automation-test",
)
def _on_connect(
_client: mqtt.Client,
_userdata: object,
_flags: mqtt.ConnectFlags,
reason_code: mqtt.ReasonCode,
_properties: mqtt.Properties | None,
) -> None:
if reason_code.is_failure:
connect_error.append(f"Broker refused connection: {reason_code}")
connected_event.set()
def _on_publish(
_client: mqtt.Client,
_userdata: object,
_mid: int,
_reason_code: mqtt.ReasonCode,
_properties: mqtt.Properties | None,
) -> None:
published_event.set()
client.on_connect = _on_connect
client.on_publish = _on_publish
if settings.mqtt_tls_enabled:
try:
client.tls_set()
except Exception as exc:
raise _MqttConnectionError(f"TLS setup failed: {exc}") from exc
if settings.mqtt_username:
client.username_pw_set(
username=settings.mqtt_username,
password=settings.mqtt_password or None,
)
client.loop_start()
try:
try:
client.connect(
host=settings.mqtt_broker_host,
port=settings.mqtt_broker_port,
keepalive=10,
)
except (OSError, socket.error) as exc:
# Sanitise: ensure password never leaks into the error message.
msg = _sanitize_mqtt_error(str(exc), settings.mqtt_password)
raise _MqttConnectionError(f"Cannot reach broker: {msg}") from exc
# Wait up to 5 s for connection acknowledgement.
if not connected_event.wait(timeout=5):
raise _MqttConnectionError("Broker connection timed out (5 s).")
if connect_error:
raise _MqttConnectionError(connect_error[0])
# Publish test message.
client.publish(test_topic, payload=test_payload, qos=1, retain=False)
# Wait up to 5 s for the publish ACK (QoS 1).
if not published_event.wait(timeout=5):
raise _MqttConnectionError("Publish timed out (5 s) — broker reachable but no ACK.")
finally:
try:
client.disconnect()
except Exception:
pass
client.loop_stop()
def _sanitize_mqtt_error(message: str, password: str | None) -> str:
"""Replace *password* in *message* with ``[redacted]``."""
if password:
return message.replace(password, "[redacted]")
return message
+584
View File
@@ -0,0 +1,584 @@
"""Energy data API: prices, costs, summary, DSMR, recompute, Tibber test (M6-T09).
All endpoints are under /api/energy, require an authenticated session, and
write endpoints (POST) additionally require a non-empty X-CSRF-Token header.
Route prefix note
-----------------
This router shares the ``/api/energy`` prefix with ``energy_contracts.py``
(which handles contract CRUD at /contracts/* and /profiles). The sub-paths
used here (/prices, /costs, /costs/summary, /costs/recompute, /dsmr/latest,
/tibber/test) are disjoint from the contract router's paths, so there is no
conflict.
Tibber token security
---------------------
``POST /api/energy/tibber/test`` calls the Tibber API but **never** echoes the
token in the response body or in log messages. Three-state logic mirrors the
MQTT test endpoint (M5-T10, app/api/routes/api/config.py::post_mqtt_test):
200 { result: "success", message: ..., price: {...} }
400 { result: "config-error", message: ... }
502 { result: "failed", message: ... }
Recompute safety
----------------
``POST /api/energy/costs/recompute`` is idempotent: it calls
``energy_cost.recompute_range`` which upserts existing rows without deleting
anything. The endpoint enforces a maximum time-window of 366 days to avoid
unbounded recomputation triggered by erroneous client requests.
Prices endpoint behaviour
-------------------------
``GET /api/energy/prices`` queries the ``tibber_price`` table for tibber
contracts, or derives the effective fixed-tariff prices for manual contracts
using the same formula as the billing engine (_manual_strategy in strategies.py):
buy_dal = energy.buy.dal + energy.energy_tax + energy.ode
buy_normal = energy.buy.normal + energy.energy_tax + energy.ode
sell_dal = energy.sell.dal (no tax added to sell price)
sell_normal = energy.sell.normal
"""
from __future__ import annotations
import logging
from datetime import UTC, datetime, timedelta
from typing import Any
from fastapi import APIRouter, Depends, Query, status
from fastapi.responses import JSONResponse
from sqlalchemy import select
from sqlalchemy.orm import Session
from app.api.routes.api.deps import require_csrf, require_session
from app.dependencies import get_app_settings, get_db
from app.config import Settings
from app.integrations.tibber.client import (
TibberAuthError,
TibberError,
fetch_current_price,
)
from app.models.energy import DsmrReading, EnergyCostPeriod, TibberPrice
from app.schemas.energy import (
CostPeriodSchema,
CostsResponse,
DsmrLatestResponse,
ManualTariffSchema,
PricePointSchema,
PricesResponse,
RecomputeResponse,
SummaryResponse,
TibberTestPriceSchema,
TibberTestResponse,
)
from app.services.auth import AuthenticatedSession
from app.services.contracts import active_contract_version_at
from app.services.energy_cost import recompute_range, summarize
from app.services.timezone import local_midnight_utc, local_now
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/energy", tags=["api-energy"])
# Maximum number of cost periods returned per request (mirrors modbus readings cap).
_COSTS_LIMIT_MAX = 5000
# Maximum allowed time-window for recompute to prevent unbounded computation.
_RECOMPUTE_MAX_DAYS = 366
# ---------------------------------------------------------------------------
# Internal helpers
# ---------------------------------------------------------------------------
def _as_utc(dt: datetime) -> datetime:
"""Attach UTC tzinfo to a naive datetime (SQLite read-back workaround)."""
if dt.tzinfo is None:
return dt.replace(tzinfo=UTC)
return dt
def _manual_tariff_from_values(values: dict[str, Any]) -> ManualTariffSchema:
"""Derive the effective fixed tariff from a manual contract version's values dict.
Mirrors the _manual_strategy formula (strategies.py):
buy_dal = energy.buy.dal + energy.energy_tax + energy.ode
buy_normal = energy.buy.normal + energy.energy_tax + energy.ode
sell_dal = energy.sell.dal (no tax added)
sell_normal = energy.sell.normal
"""
from decimal import Decimal
def _d(v: Any) -> Decimal:
return Decimal(str(v or 0))
energy = values.get("energy", {})
buy = energy.get("buy", {})
sell = energy.get("sell", {})
energy_tax = _d(energy.get("energy_tax", 0))
ode = _d(energy.get("ode", 0))
buy_dal = _d(buy.get("dal", 0)) + energy_tax + ode
buy_normal = _d(buy.get("normal", 0)) + energy_tax + ode
sell_dal = _d(sell.get("dal", 0))
sell_normal = _d(sell.get("normal", 0))
return ManualTariffSchema(
buy_dal=float(buy_dal),
buy_normal=float(buy_normal),
sell_dal=float(sell_dal),
sell_normal=float(sell_normal),
)
# ---------------------------------------------------------------------------
# GET /api/energy/prices
# ---------------------------------------------------------------------------
@router.get("/prices", response_model=PricesResponse)
def get_prices(
start: datetime | None = Query(
default=None,
description="Inclusive start of the time window (ISO 8601). "
"Defaults to the start of today UTC when omitted.",
),
end: datetime | None = Query(
default=None,
description="Inclusive end of the time window (ISO 8601). "
"Defaults to the end of tomorrow UTC when omitted.",
),
limit: int = Query(
default=500,
ge=1,
le=_COSTS_LIMIT_MAX,
description="Maximum number of Tibber price points to return.",
),
db: Session = Depends(get_db),
_auth: AuthenticatedSession = Depends(require_session),
) -> PricesResponse:
"""Return the price curve for the active contract.
**Tibber contracts** (kind="tibber"):
Fetches ``tibber_price`` rows within ``[start, end]``, ordered ascending
by ``starts_at``. At most ``limit`` rows are returned (most recent first
within the window, then reversed to ascending order — identical to the
modbus readings pattern).
Response ``points`` carries per-slot:
- ``buy = total`` (Tibber all-inclusive price)
- ``sell = total energy_tax sell_adjust`` (from active version values)
- ``level`` (Tibber price level, may be null)
``tariff`` is null.
**Manual contracts** (kind="manual"):
``points`` is empty. ``tariff`` carries the four effective prices
derived using the billing engine formula:
- ``buy_dal = energy.buy.dal + energy_tax + ode``
- ``buy_normal = energy.buy.normal + energy_tax + ode``
- ``sell_dal = energy.sell.dal``
- ``sell_normal = energy.sell.normal``
**No active contract**: returns kind=null, currency="EUR", points=[], tariff=null (200).
"""
now = datetime.now(UTC)
# Default window: today + tomorrow.
if start is None:
start = now.replace(hour=0, minute=0, second=0, microsecond=0)
if end is None:
end = (start + timedelta(days=2)).replace(hour=0, minute=0, second=0, microsecond=0)
start_utc = _as_utc(start)
end_utc = _as_utc(end)
# Resolve the active contract version at the start of the window.
version = active_contract_version_at(db, start_utc)
if version is None:
return PricesResponse(
kind=None,
currency="EUR",
points=[],
tariff=None,
)
contract = version.contract
currency = contract.currency
if contract.kind == "tibber":
# Fetch tibber_price rows in the window.
stmt = (
select(TibberPrice)
.where(
TibberPrice.starts_at >= start_utc,
TibberPrice.starts_at <= end_utc,
)
.order_by(TibberPrice.starts_at.desc())
.limit(limit)
)
rows = list(reversed(db.execute(stmt).scalars().all()))
# Derive sell price per-point using version values (energy_tax + sell_adjust).
from decimal import Decimal
def _d(v: Any) -> Decimal:
return Decimal(str(v or 0))
energy = version.values.get("energy", {}) if version.values else {}
energy_tax = _d(energy.get("energy_tax", 0))
sell_adjust = _d(energy.get("sell_adjust", 0))
points = []
for row in rows:
total = _d(row.total)
sell = float(total - energy_tax - sell_adjust)
points.append(
PricePointSchema(
starts_at=_as_utc(row.starts_at),
buy=row.total,
sell=sell,
level=row.level,
)
)
return PricesResponse(
kind="tibber",
currency=currency,
points=points,
tariff=None,
)
elif contract.kind == "manual":
tariff = _manual_tariff_from_values(version.values or {})
return PricesResponse(
kind="manual",
currency=currency,
points=[],
tariff=tariff,
)
else:
# Unknown kind — return empty response gracefully.
return PricesResponse(
kind=contract.kind,
currency=currency,
points=[],
tariff=None,
)
# ---------------------------------------------------------------------------
# GET /api/energy/costs
# ---------------------------------------------------------------------------
@router.get("/costs", response_model=CostsResponse)
def get_costs(
start: datetime | None = Query(
default=None,
description="Inclusive lower bound for period_start (ISO 8601).",
),
end: datetime | None = Query(
default=None,
description="Inclusive upper bound for period_start (ISO 8601).",
),
limit: int = Query(
default=500,
ge=1,
le=_COSTS_LIMIT_MAX,
description=f"Maximum number of cost periods to return (default 500, max {_COSTS_LIMIT_MAX}).",
),
db: Session = Depends(get_db),
_auth: AuthenticatedSession = Depends(require_session),
) -> CostsResponse:
"""Return energy_cost_period rows within a time window.
Rows are ordered by ``period_start`` ascending. When the window contains
more rows than ``limit``, the **most recent** N rows are returned (DESC LIMIT),
then reversed to ascending order — identical to the modbus readings pattern.
Query parameters:
- ``start``: inclusive lower bound on ``period_start`` (ISO 8601 datetime).
- ``end``: inclusive upper bound on ``period_start`` (ISO 8601 datetime).
- ``limit``: max rows to return (default 500, max {_COSTS_LIMIT_MAX}).
"""
stmt = (
select(EnergyCostPeriod)
.order_by(EnergyCostPeriod.period_start.desc())
.limit(limit)
)
if start is not None:
stmt = stmt.where(EnergyCostPeriod.period_start >= _as_utc(start))
if end is not None:
stmt = stmt.where(EnergyCostPeriod.period_start <= _as_utc(end))
rows = list(reversed(db.execute(stmt).scalars().all()))
items = [CostPeriodSchema.model_validate(r) for r in rows]
return CostsResponse(items=items, total=len(items))
# ---------------------------------------------------------------------------
# GET /api/energy/costs/summary
# ---------------------------------------------------------------------------
@router.get("/costs/summary", response_model=SummaryResponse)
def get_costs_summary(
start: datetime | None = Query(
default=None,
description=(
"Inclusive start of the summary interval (ISO 8601). "
"Defaults to the start of the current UTC day."
),
),
end: datetime | None = Query(
default=None,
description=(
"Exclusive end of the summary interval (ISO 8601). "
"Defaults to the start of the next UTC day (i.e. today's full data)."
),
),
db: Session = Depends(get_db),
_auth: AuthenticatedSession = Depends(require_session),
) -> SummaryResponse:
"""Aggregate billing for a time interval.
Calls ``energy_cost.summarize(session, start, end)`` which computes:
total_payable = Σ(net_cost) + fixed_costs credits
where ``fixed_costs`` is (network_fee + management_fee) apportioned to the
interval length in days (÷30 per month), and ``credits`` is heffingskorting
apportioned similarly (÷365 per year).
Both ``fixed_costs`` and ``credits`` are derived from the **currently active
contract version at ``end``**. When no active contract exists they are 0.
"""
if start is None or end is None:
# Default to the server's local today: [local_midnight, next_local_midnight).
# This ensures "today" aligns with the local calendar day (NL time) rather
# than UTC midnight.
local_today = local_now().date()
if start is None:
start = local_midnight_utc(local_today)
if end is None:
end = local_midnight_utc(local_today + timedelta(days=1))
result = summarize(db, _as_utc(start), _as_utc(end))
return SummaryResponse(**result)
# ---------------------------------------------------------------------------
# GET /api/energy/dsmr/latest
# ---------------------------------------------------------------------------
@router.get("/dsmr/latest", response_model=DsmrLatestResponse)
def get_dsmr_latest(
db: Session = Depends(get_db),
_auth: AuthenticatedSession = Depends(require_session),
) -> DsmrLatestResponse:
"""Return the most recent dsmr_reading row.
Returns ``{"found": false, "recorded_at": null, "payload": null}`` (200, not
404) when no rows exist yet, so the front-end can distinguish "no data" from
a server error.
"""
row = db.execute(
select(DsmrReading)
.order_by(DsmrReading.recorded_at.desc())
.limit(1)
).scalar_one_or_none()
if row is None:
return DsmrLatestResponse(found=False)
return DsmrLatestResponse(
found=True,
recorded_at=_as_utc(row.recorded_at),
payload=row.payload,
)
# ---------------------------------------------------------------------------
# POST /api/energy/costs/recompute
# ---------------------------------------------------------------------------
@router.post(
"/costs/recompute",
responses={
200: {"model": RecomputeResponse},
422: {"description": "Validation error (missing window or range too large)"},
},
)
def post_recompute(
start: datetime = Query(
...,
description="Inclusive start of the recompute window (ISO 8601). Required.",
),
end: datetime = Query(
...,
description="Exclusive end of the recompute window (ISO 8601). Required.",
),
db: Session = Depends(get_db),
_auth: AuthenticatedSession = Depends(require_session),
_csrf: None = Depends(require_csrf),
) -> RecomputeResponse:
"""Idempotently recompute billing records in a time window.
Calls ``energy_cost.recompute_range(session, start, end)`` which overwrites
existing rows (including successful ones) for every UTC quarter-hour boundary
in ``[start, end)``.
**Idempotency**: repeated calls with the same window produce the same
outcome. No rows are deleted; only upserted.
**Window constraint**: the maximum allowed range is {_RECOMPUTE_MAX_DAYS} days.
Requests exceeding this return 422.
Returns the number of periods for which a billing record was written.
Periods skipped due to missing contract or missing Tibber price are not counted.
"""
start_utc = _as_utc(start)
end_utc = _as_utc(end)
if end_utc <= start_utc:
from fastapi import HTTPException
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
detail="'end' must be strictly after 'start'.",
)
span_days = (end_utc - start_utc).total_seconds() / 86400
if span_days > _RECOMPUTE_MAX_DAYS:
from fastapi import HTTPException
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
detail=(
f"Time window is {span_days:.1f} days which exceeds the maximum of "
f"{_RECOMPUTE_MAX_DAYS} days. Use a smaller window."
),
)
n = recompute_range(db, start_utc, end_utc)
logger.info(
"POST /api/energy/costs/recompute [%s, %s): wrote %d period(s).",
start_utc.isoformat(),
end_utc.isoformat(),
n,
)
return RecomputeResponse(recomputed=n)
# ---------------------------------------------------------------------------
# POST /api/energy/tibber/test
# ---------------------------------------------------------------------------
@router.post(
"/tibber/test",
responses={
200: {"model": TibberTestResponse},
400: {"model": TibberTestResponse},
502: {"model": TibberTestResponse},
},
)
def post_tibber_test(
settings: Settings = Depends(get_app_settings),
_auth: AuthenticatedSession = Depends(require_session),
_csrf: None = Depends(require_csrf),
) -> JSONResponse:
"""Test Tibber API connectivity by fetching the current price point.
Three possible outcomes:
- **200** ``{ result: "success", message: ..., price: {...} }``
The Tibber API responded with a valid current price. ``price`` contains
starts_at, total, energy, tax, currency, and level.
- **400** ``{ result: "config-error", message: ... }``
The Tibber API token is empty or not configured.
- **502** ``{ result: "failed", message: ... }``
The API call failed (authentication rejected, network error, timeout,
unexpected response, etc.).
The API token is **never** included in the response body or logged.
"""
token = settings.tibber_api_token
home_id = settings.tibber_home_id or None # treat empty string as None
if not token:
logger.info("POST /api/energy/tibber/test: no token configured.")
return JSONResponse(
status_code=status.HTTP_400_BAD_REQUEST,
content=TibberTestResponse(
result="config-error",
message=(
"Tibber API token is not configured. "
"Set TIBBER_API_TOKEN in the Config page."
),
price=None,
).model_dump(mode="json"),
)
try:
price_point = fetch_current_price(token, home_id)
except TibberAuthError:
logger.warning("POST /api/energy/tibber/test: authentication failed.")
return JSONResponse(
status_code=status.HTTP_502_BAD_GATEWAY,
content=TibberTestResponse(
result="failed",
message=(
"Tibber API authentication failed. "
"Check that your API token is correct."
),
price=None,
).model_dump(mode="json"),
)
except TibberError as exc:
logger.warning("POST /api/energy/tibber/test: API call failed — %s", exc)
return JSONResponse(
status_code=status.HTTP_502_BAD_GATEWAY,
content=TibberTestResponse(
result="failed",
message=f"Tibber API call failed: {exc}",
price=None,
).model_dump(mode="json"),
)
price_schema = TibberTestPriceSchema(
starts_at=price_point.starts_at,
total=price_point.total,
energy=price_point.energy,
tax=price_point.tax,
currency=price_point.currency,
level=price_point.level,
)
logger.info(
"POST /api/energy/tibber/test: success (starts_at=%s, total=%s %s).",
price_point.starts_at.isoformat(),
price_point.total,
price_point.currency,
)
return JSONResponse(
status_code=status.HTTP_200_OK,
content=TibberTestResponse(
result="success",
message=(
f"Tibber API connected. Current price: "
f"{price_point.total} {price_point.currency}/kWh "
f"(starts {price_point.starts_at.isoformat()})."
),
price=price_schema,
).model_dump(mode="json"),
)
+331
View File
@@ -0,0 +1,331 @@
"""EnergyContract CRUD, versioning, and pricing-profile API (M6-T04).
All endpoints are under /api/energy, require an authenticated session, and
write endpoints (POST/PATCH) additionally require a non-empty X-CSRF-Token
header.
There is deliberately no DELETE endpoint for contracts: the FK RESTRICT
constraint on energy_contract_version.contract_id and
energy_cost_period.contract_version_id prevents accidental deletion of
contracts that have billing history. Users should instead deactivate a
contract (PATCH active=false) to stop it from being used.
Route ordering note
-------------------
GET /api/energy/profiles and GET /api/energy/contracts/{id} are on separate
path prefixes (/energy/profiles vs /energy/contracts/{id}) so there is no
ambiguity even without extra ordering gymnastics.
"""
from __future__ import annotations
import logging
from datetime import UTC, datetime
from typing import Any
from fastapi import APIRouter, Depends, HTTPException, status
from sqlalchemy.orm import Session
from app.api.routes.api.deps import require_csrf, require_session
from app.dependencies import get_db
from app.integrations.pricing.profiles import (
ProfileNotFoundError,
ProfileValidationError,
list_profiles,
)
from app.schemas.energy_contract import (
ContractCreate,
ContractDetailResponse,
ContractListResponse,
ContractPatch,
ContractResponse,
ContractVersionResponse,
ProfilesResponse,
VersionCreate,
)
from app.services.auth import AuthenticatedSession
from app.services import timezone as _tz_mod
from app.services.contracts import (
ContractVersionError,
activate_contract,
add_version,
create_contract,
deactivate_contract,
get_contract_or_none,
list_contracts,
)
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/energy", tags=["api-energy-contracts"])
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _get_contract_or_404(db: Session, contract_id: int):
"""Return the contract with the given id or raise 404."""
contract = get_contract_or_none(db, contract_id)
if contract is None:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"Energy contract {contract_id!r} not found.",
)
return contract
def _contract_detail(db: Session, contract) -> ContractDetailResponse:
"""Build a ContractDetailResponse for *contract*, loading versions."""
# Eager-load versions ordered by effective_from for consistent display.
from sqlalchemy import select
from app.models.energy import EnergyContractVersion
versions = list(
db.execute(
select(EnergyContractVersion)
.where(EnergyContractVersion.contract_id == contract.id)
.order_by(EnergyContractVersion.effective_from)
)
.scalars()
.all()
)
version_schemas = [ContractVersionResponse.model_validate(v) for v in versions]
return ContractDetailResponse(
id=contract.id,
name=contract.name,
kind=contract.kind,
active=contract.active,
currency=contract.currency,
created_at=contract.created_at,
updated_at=contract.updated_at,
versions=version_schemas,
)
def _raise_422_for_profile_error(exc: Exception) -> Any:
"""Convert a ProfileValidationError or ProfileNotFoundError into HTTP 422."""
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
detail=str(exc),
)
def _localize_effective_from(dt: datetime | None) -> datetime:
"""Resolve *dt* to an aware UTC datetime for storage.
Rules (Principle A):
- If *dt* is None → use ``datetime.now(UTC)`` (unchanged from before).
- If *dt* is timezone-aware → convert to UTC as-is.
- If *dt* is timezone-naive → interpret as server local wall-clock time,
localize with ``local_tz()``, then convert to UTC.
This means a front-end that sends ``"2026-06-25T00:00:00"`` (no Z) has it
interpreted as local midnight (e.g. CEST = UTC+2 → stored as 2026-06-24T22:00:00Z),
not as UTC midnight.
"""
if dt is None:
return datetime.now(UTC)
if dt.tzinfo is not None:
# Already aware: convert to UTC.
return dt.astimezone(UTC)
# Naive: assume server local wall-clock.
tz = _tz_mod.local_tz()
local_dt = dt.replace(tzinfo=tz)
return local_dt.astimezone(UTC)
# ---------------------------------------------------------------------------
# GET /api/energy/profiles
# ---------------------------------------------------------------------------
@router.get("/profiles", response_model=ProfilesResponse)
def get_profiles(
_auth: AuthenticatedSession = Depends(require_session),
) -> ProfilesResponse:
"""List all available pricing profile structures.
Returns the full profile structure for each supported contract kind
(``manual`` and ``tibber``). The front-end uses this to dynamically
render the correct fields and labels for the contract creation/editing form.
"""
return ProfilesResponse(profiles=list_profiles())
# ---------------------------------------------------------------------------
# GET /api/energy/contracts
# ---------------------------------------------------------------------------
@router.get("/contracts", response_model=ContractListResponse)
def list_energy_contracts(
db: Session = Depends(get_db),
_auth: AuthenticatedSession = Depends(require_session),
) -> ContractListResponse:
"""List all energy contracts with their active status.
Returns a flat list (no embedded version history); use
GET /api/energy/contracts/{id} to fetch the full version history for a
specific contract.
"""
contracts = list_contracts(db)
items = [ContractResponse.model_validate(c) for c in contracts]
return ContractListResponse(items=items, total=len(items))
# ---------------------------------------------------------------------------
# POST /api/energy/contracts
# ---------------------------------------------------------------------------
@router.post(
"/contracts",
response_model=ContractDetailResponse,
status_code=status.HTTP_201_CREATED,
)
def create_energy_contract(
body: ContractCreate,
db: Session = Depends(get_db),
_auth: AuthenticatedSession = Depends(require_session),
_csrf: None = Depends(require_csrf),
) -> ContractDetailResponse:
"""Create a new energy contract with an initial pricing version.
The ``values`` dict is validated against the YAML profile for the given
``kind``; non-conforming values result in 422 Unprocessable Entity.
The new contract is created with ``active=False``; use
PATCH /api/energy/contracts/{id} with ``active=true`` to activate it.
"""
effective_from = _localize_effective_from(body.effective_from)
try:
contract = create_contract(
db,
name=body.name,
kind=body.kind,
currency=body.currency,
values=body.values,
effective_from=effective_from,
)
except (ProfileNotFoundError, ProfileValidationError) as exc:
_raise_422_for_profile_error(exc)
db.commit()
db.refresh(contract)
logger.info("Created energy contract id=%d name=%r kind=%s", contract.id, contract.name, contract.kind)
return _contract_detail(db, contract)
# ---------------------------------------------------------------------------
# GET /api/energy/contracts/{id}
# ---------------------------------------------------------------------------
@router.get("/contracts/{contract_id}", response_model=ContractDetailResponse)
def get_energy_contract(
contract_id: int,
db: Session = Depends(get_db),
_auth: AuthenticatedSession = Depends(require_session),
) -> ContractDetailResponse:
"""Return a single energy contract with its full version history.
Versions are ordered by ``effective_from`` ascending so the caller can
easily inspect the pricing timeline.
"""
contract = _get_contract_or_404(db, contract_id)
return _contract_detail(db, contract)
# ---------------------------------------------------------------------------
# PATCH /api/energy/contracts/{id}
# ---------------------------------------------------------------------------
@router.patch("/contracts/{contract_id}", response_model=ContractDetailResponse)
def patch_energy_contract(
contract_id: int,
body: ContractPatch,
db: Session = Depends(get_db),
_auth: AuthenticatedSession = Depends(require_session),
_csrf: None = Depends(require_csrf),
) -> ContractDetailResponse:
"""Partially update a contract: rename or change activation status.
- ``name``: updates the human-readable label.
- ``active=true``: activates this contract (all others are deactivated).
- ``active=false``: deactivates this contract (no effect on others).
At most one contract may be active at any time; the service layer enforces
mutual exclusion.
"""
contract = _get_contract_or_404(db, contract_id)
if body.name is not None:
contract.name = body.name
contract.updated_at = datetime.now(UTC)
if body.active is True:
activate_contract(db, contract)
elif body.active is False:
deactivate_contract(db, contract)
db.commit()
db.refresh(contract)
return _contract_detail(db, contract)
# ---------------------------------------------------------------------------
# POST /api/energy/contracts/{id}/versions
# ---------------------------------------------------------------------------
@router.post(
"/contracts/{contract_id}/versions",
response_model=ContractDetailResponse,
status_code=status.HTTP_201_CREATED,
)
def add_contract_version(
contract_id: int,
body: VersionCreate,
db: Session = Depends(get_db),
_auth: AuthenticatedSession = Depends(require_session),
_csrf: None = Depends(require_csrf),
) -> ContractDetailResponse:
"""Add a new pricing version to an existing contract.
This is how price changes are recorded: the current open version is
automatically closed (its ``effective_to`` is set to ``body.effective_from``)
and a new version is created starting at ``body.effective_from``.
The ``values`` dict must conform to the contract's pricing profile.
Non-conforming values return 422. If ``effective_from`` is not strictly
after the previous version's ``effective_from``, 422 is returned without
writing any rows.
Historical versions are never modified; this endpoint is append-only.
"""
contract = _get_contract_or_404(db, contract_id)
effective_from = _localize_effective_from(body.effective_from)
try:
add_version(
db,
contract,
effective_from=effective_from,
values=body.values,
)
except ContractVersionError as exc:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
detail=str(exc),
)
except (ProfileNotFoundError, ProfileValidationError) as exc:
_raise_422_for_profile_error(exc)
db.commit()
db.refresh(contract)
return _contract_detail(db, contract)
+221
View File
@@ -0,0 +1,221 @@
"""Expose API routes (M5-T12).
Three endpoints:
GET /api/expose — Return catalog of exposable entities, per-key
toggle state, and MQTT/Discovery connection status.
PUT /api/expose — Set per-key toggle state (map key → bool);
triggers a HA Discovery re-publish on success.
POST /api/expose/republish — Manually trigger a full HA Discovery re-publish.
Auth model:
- GET: session required (no CSRF — read-only).
- PUT, POST: session + CSRF required.
"""
from __future__ import annotations
import logging
from datetime import UTC, datetime
from fastapi import APIRouter, Depends, HTTPException, status
from sqlalchemy.orm import Session
from app.api.routes.api.deps import require_csrf, require_session
from app.config import get_settings
from app.dependencies import get_db
from app.services.config_page import build_runtime_settings
from app.integrations.expose import CatalogEntry, build_catalog
from app.integrations.mqtt import mqtt_manager
from app.models.expose import ExposedEntityToggle
from app.schemas.expose import (
CatalogEntrySchema,
DeviceInfoSchema,
ExposeResponse,
ExposeUpdateRequest,
ExposeUpdateResponse,
ExposableEntitySchema,
MqttStatusSchema,
RepublishResponse,
)
from app.services.auth import AuthenticatedSession
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api", tags=["api-expose"])
# ---------------------------------------------------------------------------
# Internal helpers
# ---------------------------------------------------------------------------
def _get_mqtt_status(db: Session) -> MqttStatusSchema:
"""Build MQTT/Discovery status from live runtime state (DB-merged settings)."""
settings = build_runtime_settings(db, get_settings())
return MqttStatusSchema(
mqtt_configured=mqtt_manager.is_configured(settings),
mqtt_connected=mqtt_manager.is_connected,
discovery_enabled=bool(settings.ha_discovery_enabled),
)
def _entry_to_schema(entry: CatalogEntry) -> CatalogEntrySchema:
"""Convert a CatalogEntry to its Pydantic schema form.
Excludes ``value_getter`` because it is a non-serialisable callable.
"""
entity = entry.entity
return CatalogEntrySchema(
entity=ExposableEntitySchema(
key=entity.key,
component=entity.component,
device=DeviceInfoSchema(
identifiers=list(entity.device.identifiers),
name=entity.device.name,
),
device_class=entity.device_class,
unit=entity.unit,
name=entity.name,
state_class=entity.state_class,
),
enabled=entry.enabled,
)
def _build_response_data(
session: Session,
) -> tuple[list[CatalogEntrySchema], MqttStatusSchema]:
"""Return (catalog_entries, mqtt_status) for building GET / PUT responses."""
catalog = build_catalog(session)
catalog_schema = [_entry_to_schema(e) for e in catalog]
mqtt_status = _get_mqtt_status(session)
return catalog_schema, mqtt_status
# ---------------------------------------------------------------------------
# GET /api/expose
# ---------------------------------------------------------------------------
@router.get("/expose", response_model=ExposeResponse)
def get_expose(
db: Session = Depends(get_db),
_auth: AuthenticatedSession = Depends(require_session),
) -> ExposeResponse:
"""Return the full exposable-entity catalog with toggle states and MQTT status.
The catalog is computed dynamically from registered providers (e.g. the
Modbus provider enumerates all enabled devices and their metric entities).
Toggle states come from the ``exposed_entity_toggle`` table; entities with
no row default to ``enabled=False``.
"""
catalog_schema, mqtt_status = _build_response_data(db)
return ExposeResponse(catalog=catalog_schema, mqtt_status=mqtt_status)
# ---------------------------------------------------------------------------
# PUT /api/expose
# ---------------------------------------------------------------------------
@router.put("/expose", response_model=ExposeUpdateResponse)
def put_expose(
body: ExposeUpdateRequest,
db: Session = Depends(get_db),
_auth: AuthenticatedSession = Depends(require_session),
_csrf: None = Depends(require_csrf),
) -> ExposeUpdateResponse:
"""Set per-entity toggle state.
Accepts a map of ``{key: bool}`` and upserts rows in the
``exposed_entity_toggle`` table. Only keys present in ``body.toggles``
are touched; other entities' toggles are left unchanged.
After writing the toggles, triggers a HA Discovery re-publish so any
changes (enabled ↔ disabled) are reflected in Home Assistant immediately.
"""
now = datetime.now(UTC)
for key, enabled in body.toggles.items():
existing = (
db.query(ExposedEntityToggle)
.filter(ExposedEntityToggle.key == key)
.first()
)
if existing is not None:
existing.enabled = enabled
existing.updated_at = now
else:
db.add(
ExposedEntityToggle(
key=key,
enabled=enabled,
updated_at=now,
)
)
db.commit()
# Trigger discovery re-publish after toggle change.
_trigger_republish(db)
catalog_schema, mqtt_status = _build_response_data(db)
return ExposeUpdateResponse(catalog=catalog_schema, mqtt_status=mqtt_status)
# ---------------------------------------------------------------------------
# POST /api/expose/republish
# ---------------------------------------------------------------------------
@router.post("/expose/republish", response_model=RepublishResponse)
def post_expose_republish(
db: Session = Depends(get_db),
_auth: AuthenticatedSession = Depends(require_session),
_csrf: None = Depends(require_csrf),
) -> RepublishResponse:
"""Manually trigger a full HA Discovery re-publish.
Calls ``publish_discovery(session)`` from the HA discovery service (M5-T11).
Returns a status indicating whether the publish was attempted (or skipped
because MQTT / discovery is not enabled / connected).
"""
settings = build_runtime_settings(db, get_settings())
if not (settings.mqtt_enabled and settings.ha_discovery_enabled and mqtt_manager.is_connected):
return RepublishResponse(
ok=False,
message="MQTT / HA Discovery not enabled or broker not connected.",
)
try:
_trigger_republish(db)
return RepublishResponse(
ok=True,
message="HA Discovery re-published successfully.",
)
except Exception as exc:
logger.exception("post_expose_republish: unexpected error during publish")
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail="Discovery re-publish failed.",
) from exc
# ---------------------------------------------------------------------------
# Internal: trigger discovery re-publish
# ---------------------------------------------------------------------------
def _trigger_republish(session: Session) -> None:
"""Call publish_discovery from the HA discovery service.
No-op if MQTT / discovery is not enabled or broker is not connected
(publish_discovery guards internally). All errors are swallowed to avoid
breaking the API response.
"""
try:
from app.services.ha_discovery import publish_discovery
publish_discovery(session)
except Exception:
logger.exception("_trigger_republish: publish_discovery raised an error")
+331
View File
@@ -0,0 +1,331 @@
"""Meter CRUD, swap declaration, and retroactive recompute API (M7-T05).
All endpoints are under /api/energy/meters, require an authenticated session,
and write endpoints (POST/PATCH) additionally require a non-empty X-CSRF-Token
header.
Route semantics
---------------
GET /api/energy/meters — list all meter epochs (ascending started_at)
POST /api/energy/meters — declare a meter swap / initial meter epoch
PATCH /api/energy/meters/{id} — edit label / note, or correct started_at (retroactive)
Retroactive recompute
---------------------
Whenever a write operation changes a meter's ``started_at`` (new declaration
or PATCH correction), the affected billing window is re-judged via
``recompute_range``:
- **POST** (new meter, possibly retroactive):
window = [new_meter.started_at, now)
Rationale: the new meter's ``started_at`` closes the previous meter at that
point; all periods from that boundary forward may have a different meter
attribution. Using ``now`` as the upper bound is safe because
``recompute_range`` only processes closed quarters and the operation is
idempotent.
- **PATCH started_at** (retroactive correction):
window = [min(old_started_at, new_started_at), now)
Rationale: shifting the boundary in either direction affects all periods
between the old and new boundary (and potentially beyond if re-attribution
cascades). Using the minimum of the two timestamps guarantees the entire
affected range is covered; using ``now`` as the upper bound is safe and
idempotent.
``started_at`` localisation (Principle A, FU10 convention)
----------------------------------------------------------
If the client sends a timezone-naive ``started_at`` value, it is interpreted as
the **server's local wall-clock time** and converted to UTC before storage.
Timezone-aware values are converted to UTC as-is. This is identical to the
``_localize_effective_from`` convention used in ``energy_contracts.py``.
"""
from __future__ import annotations
import logging
from datetime import UTC, datetime
from typing import Optional
from fastapi import APIRouter, Depends, HTTPException, status
from sqlalchemy.orm import Session
from app.api.routes.api.deps import require_csrf, require_session
from app.dependencies import get_db
from app.models.energy import Meter
from app.schemas.meter import (
MeterDeclareRequest,
MeterListResponse,
MeterPatchRequest,
MeterResponse,
)
from app.services import timezone as _tz_mod
from app.services.auth import AuthenticatedSession
from app.services.energy_cost import recompute_range
from app.services.meters import (
MeterIntervalError,
MeterOverlapError,
declare_meter,
list_meters,
update_meter,
)
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/energy", tags=["api-energy-meters"])
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _trigger_discovery_republish(session: Session) -> None:
"""Call publish_discovery after a meter write operation (best-effort).
No-op if MQTT / discovery is not enabled or the broker is not connected
(publish_discovery guards internally). All errors are swallowed so that a
discovery failure never breaks the API response.
Must be called **after** db.commit() so that publish_discovery sees the
final committed state of the meter table when it rebuilds the catalog.
"""
try:
from app.services.ha_discovery import publish_discovery
publish_discovery(session)
except Exception:
logger.exception("_trigger_discovery_republish: publish_discovery raised an error")
def _get_meter_or_404(db: Session, meter_id: int) -> Meter:
"""Return the meter with the given id or raise 404."""
meter: Optional[Meter] = db.get(Meter, meter_id)
if meter is None:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"Meter {meter_id!r} not found.",
)
return meter
def _localize_started_at(dt: datetime) -> datetime:
"""Resolve *dt* to an aware UTC datetime for storage.
Follows the same Principle-A convention as ``_localize_effective_from``
in ``energy_contracts.py`` (FU10):
- Timezone-aware → convert to UTC as-is.
- Timezone-naive → interpret as server local wall-clock time, localize
with ``local_tz()``, then convert to UTC.
A front-end sending ``"2026-06-25T00:00:00"`` (no Z) has it interpreted
as local midnight (e.g. CEST = UTC+2 → stored as 2026-06-24T22:00:00Z),
not as UTC midnight.
"""
if dt.tzinfo is not None:
return dt.astimezone(UTC)
tz = _tz_mod.local_tz()
local_dt = dt.replace(tzinfo=tz)
return local_dt.astimezone(UTC)
def _trigger_recompute(db: Session, start: datetime, label: str) -> int:
"""Trigger recompute_range from *start* to now (UTC).
This is the standard "retroactive window" call: everything from the
affected boundary up to the current moment needs re-attribution.
Using ``now`` as the upper bound is safe because ``recompute_range``
only touches closed quarter-hour periods and the operation is idempotent.
"""
end = datetime.now(UTC)
if start >= end:
# started_at is in the future — nothing to recompute.
logger.info("%s: started_at (%s) is in the future, skipping recompute.", label, start)
return 0
n = recompute_range(db, start, end)
logger.info(
"%s: recomputed %d period(s) in window [%s, %s).",
label,
n,
start.isoformat(),
end.isoformat(),
)
return n
# ---------------------------------------------------------------------------
# GET /api/energy/meters
# ---------------------------------------------------------------------------
@router.get("/meters", response_model=MeterListResponse)
def list_energy_meters(
db: Session = Depends(get_db),
_auth: AuthenticatedSession = Depends(require_session),
) -> MeterListResponse:
"""List all meter epochs in ascending ``started_at`` order.
Returns the full historical sequence of meter installations across all
commodities. The active meter (``ended_at=null``) appears last because it
has the latest ``started_at``.
"""
meters = list_meters(db)
items = [MeterResponse.model_validate(m) for m in meters]
return MeterListResponse(items=items, total=len(items))
# ---------------------------------------------------------------------------
# POST /api/energy/meters
# ---------------------------------------------------------------------------
@router.post(
"/meters",
response_model=MeterResponse,
status_code=status.HTTP_201_CREATED,
)
def declare_energy_meter(
body: MeterDeclareRequest,
db: Session = Depends(get_db),
_auth: AuthenticatedSession = Depends(require_session),
_csrf: None = Depends(require_csrf),
) -> MeterResponse:
"""Declare a new meter epoch (swap, home move, or initial declaration).
Closes the current active meter for the given commodity at ``started_at``
and opens a new active meter. If no active meter exists, the new meter is
simply created without closing anything.
**Validation**: ``started_at`` must be **≥** the current active meter's
own ``started_at`` (no chronological backdate below the active epoch's
start). Equal timestamps are allowed (replaces the current meter at the
same logical moment). Violation → 422.
**Retroactive recompute**: if ``started_at`` is in the past, billing
records from that point forward are re-judged via ``recompute_range`` to
reflect the new meter attribution. The recompute is transparent — the
response body is the created meter (``MeterResponse``) only and does **not**
include a recompute count; callers should re-fetch costs if they need the
updated totals.
"""
started_at_utc = _localize_started_at(body.started_at)
try:
new_meter = declare_meter(
db,
label=body.label,
started_at=started_at_utc,
reason=body.reason.value,
commodity=body.commodity,
note=body.note,
)
except MeterOverlapError as exc:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
detail=str(exc),
)
db.flush() # assign PK before recompute (recompute uses session, needs meter in DB)
# Retroactive recompute: re-judge attribution from the new boundary onward.
now = datetime.now(UTC)
if started_at_utc < now:
_trigger_recompute(db, started_at_utc, "POST /api/energy/meters")
db.commit()
db.refresh(new_meter)
# Trigger HA discovery re-publish so the new active meter's energy-cost
# device/sensor configuration is pushed to Home Assistant. Best-effort:
# failures are logged and swallowed; the API response is not affected.
_trigger_discovery_republish(db)
logger.info(
"POST /api/energy/meters: declared %r meter id=%d label=%r started_at=%s",
body.commodity,
new_meter.id,
new_meter.label,
started_at_utc.isoformat(),
)
return MeterResponse.model_validate(new_meter)
# ---------------------------------------------------------------------------
# PATCH /api/energy/meters/{id}
# ---------------------------------------------------------------------------
@router.patch("/meters/{meter_id}", response_model=MeterResponse)
def patch_energy_meter(
meter_id: int,
body: MeterPatchRequest,
db: Session = Depends(get_db),
_auth: AuthenticatedSession = Depends(require_session),
_csrf: None = Depends(require_csrf),
) -> MeterResponse:
"""Partially update a meter epoch: rename, edit note, or correct started_at.
- ``label``: updates the human-readable label.
- ``note``: updates the free-form note.
- ``started_at``: **retroactive correction** — shifts this meter's start
boundary. The service layer maintains timeline continuity by also
updating the preceding meter's ``ended_at``. Validation:
* Must be strictly after the previous meter's own ``started_at``.
* Must be strictly before this meter's ``ended_at`` (if closed).
Violation → 422.
**Retroactive recompute when ``started_at`` changes**: billing records in
the window ``[min(old, new), now)`` are re-judged to reflect the corrected
meter attribution.
Not found → 404.
"""
meter = _get_meter_or_404(db, meter_id)
# Capture old started_at before mutation (needed for recompute window).
old_started_at: Optional[datetime] = meter.started_at
# Localise started_at if provided.
new_started_at_utc: Optional[datetime] = None
if body.started_at is not None:
new_started_at_utc = _localize_started_at(body.started_at)
try:
update_meter(
db,
meter,
label=body.label,
note=body.note,
started_at=new_started_at_utc,
)
except MeterIntervalError as exc:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
detail=str(exc),
)
# Retroactive recompute if started_at was changed.
if new_started_at_utc is not None and old_started_at is not None:
# Normalise old_started_at to UTC-aware for comparison.
if old_started_at.tzinfo is None:
old_started_at = old_started_at.replace(tzinfo=UTC)
# Window = [min(old, new), now) — covers all periods whose attribution
# may have changed due to the boundary shift in either direction.
window_start = min(old_started_at, new_started_at_utc)
_trigger_recompute(db, window_start, f"PATCH /api/energy/meters/{meter_id}")
db.commit()
db.refresh(meter)
# Trigger HA discovery re-publish so label renames on the active meter
# propagate to the HA device name. Best-effort: failures are logged and
# swallowed; the API response is not affected.
_trigger_discovery_republish(db)
logger.info(
"PATCH /api/energy/meters/%d: updated meter label=%r started_at=%s",
meter_id,
meter.label,
meter.started_at,
)
return MeterResponse.model_validate(meter)
+510
View File
@@ -0,0 +1,510 @@
"""Modbus device CRUD, readings, metrics, and test-read API (M5-T05).
All endpoints are under /api/modbus, require an authenticated session, and
write endpoints (POST/PATCH/DELETE + test-read) additionally require a
non-empty X-CSRF-Token header.
Deletion safety:
DELETE /api/modbus/devices/{uuid} queries the application layer for
existing modbus_reading rows before deleting. Without ``cascade`` it
returns 409 and suggests disabling the device instead — a friendly guard
rather than letting the DB raise. With ``cascade=true`` it removes the
readings (and expose toggles) first, then the device. SQLite FK RESTRICT
*is* enforced at runtime here (the app sets PRAGMA foreign_keys=ON; see
app/db.py), so the cascade delete order matters.
"""
from __future__ import annotations
import logging
from datetime import UTC, datetime
from typing import Any
from fastapi import APIRouter, Depends, HTTPException, Query, status
from sqlalchemy import delete as sa_delete, func, select
from sqlalchemy.orm import Session
from app.api.routes.api.deps import require_csrf, require_session
from app.dependencies import get_db
from app.integrations.modbus import driver as modbus_driver
from app.integrations.modbus.driver import ModbusDriverError
from app.integrations.modbus.profiles import (
ProfileNotFoundError,
decode as decode_profile,
list_profiles,
load_profile,
)
from app.models.expose import ExposedEntityToggle
from app.models.modbus import ModbusDevice, ModbusReading
from app.schemas.modbus import (
MetricInfo,
ModbusDeleteResponse,
ModbusDeviceCreate,
ModbusDeviceListResponse,
ModbusDeviceResponse,
ModbusDeviceUpdate,
ModbusLatestResponse,
ModbusMetricsResponse,
ModbusProfilesResponse,
ModbusReadingResponse,
ModbusReadingsResponse,
ModbusTestReadResponse,
ProfileSummary,
)
from app.services.auth import AuthenticatedSession
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/modbus", tags=["api-modbus"])
# Maximum number of readings that can be returned per request.
_READINGS_LIMIT_MAX = 5000
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _get_device_or_404(db: Session, uuid: str) -> ModbusDevice:
"""Return the device with the given UUID or raise 404."""
device = db.execute(
select(ModbusDevice).where(ModbusDevice.uuid == uuid)
).scalar_one_or_none()
if device is None:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"Device '{uuid}' not found.",
)
return device
def _label_from_key(key: str) -> str:
"""Derive a human-readable label from a metric key.
Example: ``"active_power"`` → ``"Active Power"``
"""
return key.replace("_", " ").title()
# ---------------------------------------------------------------------------
# GET /api/modbus/profiles
# ---------------------------------------------------------------------------
@router.get("/profiles", response_model=ModbusProfilesResponse)
def get_profiles(
_auth: AuthenticatedSession = Depends(require_session),
) -> ModbusProfilesResponse:
"""List all available Modbus YAML profiles (name + description).
Intended for the front-end's device-creation profile drop-down.
"""
pairs = list_profiles()
return ModbusProfilesResponse(
profiles=[ProfileSummary(name=name, description=desc) for name, desc in pairs]
)
# ---------------------------------------------------------------------------
# Device CRUD
# ---------------------------------------------------------------------------
@router.get("/devices", response_model=ModbusDeviceListResponse)
def list_devices(
db: Session = Depends(get_db),
_auth: AuthenticatedSession = Depends(require_session),
) -> ModbusDeviceListResponse:
"""Return all Modbus devices (no pagination — device counts stay small)."""
devices = db.execute(select(ModbusDevice)).scalars().all()
items = [ModbusDeviceResponse.model_validate(d) for d in devices]
return ModbusDeviceListResponse(items=items, total=len(items))
@router.post(
"/devices",
response_model=ModbusDeviceResponse,
status_code=status.HTTP_201_CREATED,
)
def create_device(
body: ModbusDeviceCreate,
db: Session = Depends(get_db),
_auth: AuthenticatedSession = Depends(require_session),
_csrf: None = Depends(require_csrf),
) -> ModbusDeviceResponse:
"""Create a new Modbus device.
- Validates that the referenced ``profile`` exists; returns 422 if not.
- Returns 201 with the created device on success.
"""
# Validate profile exists (422 if not)
try:
load_profile(body.profile)
except ProfileNotFoundError:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
detail=f"Profile '{body.profile}' does not exist.",
)
now = datetime.now(UTC)
device = ModbusDevice(
friendly_name=body.friendly_name,
transport=body.transport,
host=body.host,
port=body.port,
unit_id=body.unit_id,
profile=body.profile,
poll_interval_s=body.poll_interval_s,
enabled=body.enabled,
created_at=now,
updated_at=now,
)
db.add(device)
db.commit()
db.refresh(device)
logger.info("Created Modbus device %r (uuid=%s)", device.friendly_name, device.uuid)
return ModbusDeviceResponse.model_validate(device)
@router.get("/devices/{uuid}", response_model=ModbusDeviceResponse)
def get_device(
uuid: str,
db: Session = Depends(get_db),
_auth: AuthenticatedSession = Depends(require_session),
) -> ModbusDeviceResponse:
"""Return a single Modbus device by UUID."""
device = _get_device_or_404(db, uuid)
return ModbusDeviceResponse.model_validate(device)
@router.patch("/devices/{uuid}", response_model=ModbusDeviceResponse)
def patch_device(
uuid: str,
body: ModbusDeviceUpdate,
db: Session = Depends(get_db),
_auth: AuthenticatedSession = Depends(require_session),
_csrf: None = Depends(require_csrf),
) -> ModbusDeviceResponse:
"""Partially update a Modbus device (including enable/disable).
Only fields explicitly provided in the request body are updated.
Providing ``profile`` triggers a profile-existence check (422 if unknown).
"""
device = _get_device_or_404(db, uuid)
update_data = body.model_dump(exclude_none=True)
# If profile is being changed, validate it exists.
if "profile" in update_data:
try:
load_profile(update_data["profile"])
except ProfileNotFoundError:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
detail=f"Profile '{update_data['profile']}' does not exist.",
)
for field, value in update_data.items():
setattr(device, field, value)
device.updated_at = datetime.now(UTC)
db.commit()
db.refresh(device)
logger.info("Updated Modbus device %r (uuid=%s)", device.friendly_name, device.uuid)
return ModbusDeviceResponse.model_validate(device)
@router.delete(
"/devices/{uuid}",
status_code=status.HTTP_204_NO_CONTENT,
response_model=None,
)
def delete_device(
uuid: str,
cascade: bool = Query(default=False),
db: Session = Depends(get_db),
_auth: AuthenticatedSession = Depends(require_session),
_csrf: None = Depends(require_csrf),
) -> None | ModbusDeleteResponse:
"""Delete a Modbus device.
**Default behaviour (cascade=false)**:
Returns 409 Conflict if the device has any associated readings; use
``enabled=false`` to disable it instead.
**Cascade delete (cascade=true)**:
Permanently deletes the device together with all its readings and any
``ExposedEntityToggle`` rows whose key matches ``modbus.<uuid>.*``.
Also makes a best-effort attempt to clear the device's HA Discovery
config topics from MQTT (empty retained payload) before the DB rows
are removed. MQTT failures are swallowed — the DB deletion proceeds
regardless.
Returns HTTP 200 with a ``ModbusDeleteResponse`` JSON body on success.
**Application-layer safety**: the 409 guard uses an explicit SELECT COUNT
query to return a friendly message. FK RESTRICT is enforced at runtime
(the app sets ``PRAGMA foreign_keys=ON``), so the cascade path deletes
readings before the device.
"""
from app.services.ha_discovery import clear_device_discovery
device = _get_device_or_404(db, uuid)
# Application-layer guard: refuse deletion if any readings exist.
reading_count: int = db.execute(
select(func.count(ModbusReading.id)).where(ModbusReading.device_id == device.id)
).scalar_one()
if reading_count > 0 and not cascade:
raise HTTPException(
status_code=status.HTTP_409_CONFLICT,
detail=(
f"Device '{uuid}' has {reading_count} associated reading(s) and cannot be "
"deleted. Disable the device (set enabled=false) instead to stop polling "
"without losing historical data."
),
)
if cascade:
# --- Cascade deletion path ---
logger.info(
"Cascade-deleting Modbus device %r (uuid=%s): %d reading(s)",
device.friendly_name,
uuid,
reading_count,
)
# 1. Best-effort HA Discovery cleanup (before DB rows are gone).
try:
clear_device_discovery(db, uuid)
except Exception: # noqa: BLE001
logger.exception(
"delete_device(cascade): HA discovery cleanup raised for uuid=%s; continuing",
uuid,
)
# 2. Delete all readings for this device (must happen before device row).
deleted_readings_result = db.execute(
sa_delete(ModbusReading).where(ModbusReading.device_id == device.id)
)
readings_deleted: int = deleted_readings_result.rowcount
# 3. Delete all ExposedEntityToggle rows for this device.
# Keys follow the pattern "modbus.<uuid>.<metric>".
toggle_prefix = f"modbus.{uuid}.%"
deleted_toggles_result = db.execute(
sa_delete(ExposedEntityToggle).where(ExposedEntityToggle.key.like(toggle_prefix))
)
toggles_deleted: int = deleted_toggles_result.rowcount
# 4. Delete the device itself.
db.delete(device)
db.commit()
logger.info(
"Cascade-delete complete for device uuid=%s: %d reading(s), %d toggle(s) removed",
uuid,
readings_deleted,
toggles_deleted,
)
from fastapi.responses import JSONResponse
return JSONResponse(
status_code=status.HTTP_200_OK,
content={
"deleted": True,
"readings_deleted": readings_deleted,
"toggles_deleted": toggles_deleted,
},
)
# --- Non-cascade path (no readings exist at this point) ---
logger.info("Deleting Modbus device %r (uuid=%s)", device.friendly_name, uuid)
db.delete(device)
db.commit()
# ---------------------------------------------------------------------------
# Readings endpoints
# ---------------------------------------------------------------------------
@router.get("/devices/{uuid}/latest", response_model=ModbusLatestResponse)
def get_latest_reading(
uuid: str,
db: Session = Depends(get_db),
_auth: AuthenticatedSession = Depends(require_session),
) -> ModbusLatestResponse:
"""Return the most recent reading for a device.
If no readings exist yet, returns ``{"found": false, "recorded_at": null,
"payload": null}`` (200, not 404) so the front-end can distinguish
"device exists but has no data" from "device not found".
"""
device = _get_device_or_404(db, uuid)
row = db.execute(
select(ModbusReading)
.where(ModbusReading.device_id == device.id)
.order_by(ModbusReading.recorded_at.desc())
.limit(1)
).scalar_one_or_none()
if row is None:
return ModbusLatestResponse(found=False, recorded_at=None, payload=None)
return ModbusLatestResponse(
found=True,
recorded_at=row.recorded_at,
payload=row.payload,
)
@router.get("/devices/{uuid}/readings", response_model=ModbusReadingsResponse)
def get_readings(
uuid: str,
start: datetime | None = Query(default=None),
end: datetime | None = Query(default=None),
limit: int = Query(default=500, ge=1, le=_READINGS_LIMIT_MAX),
db: Session = Depends(get_db),
_auth: AuthenticatedSession = Depends(require_session),
) -> ModbusReadingsResponse:
"""Return time-range readings for a device.
When the window contains more rows than ``limit``, the **most recent** N rows
are returned (``ORDER BY recorded_at DESC LIMIT n``), then reversed to
ascending order before being sent to the client. This ensures that for long
time-range requests (e.g. 6 h / 24 h) the caller always sees the latest data
rather than the oldest segment of the window.
When the window has fewer rows than ``limit`` the full window is returned in
ascending order — behaviour is identical to a plain ascending query.
The response schema is unchanged: items are always ``recorded_at`` ascending.
The query uses the ``(device_id, recorded_at)`` composite index for
efficient time-window scans.
Query parameters:
- ``start``: inclusive lower bound (ISO8601 datetime)
- ``end``: inclusive upper bound (ISO8601 datetime)
- ``limit``: max rows to return (default 500, max {_READINGS_LIMIT_MAX})
"""
device = _get_device_or_404(db, uuid)
stmt = (
select(ModbusReading)
.where(ModbusReading.device_id == device.id)
.order_by(ModbusReading.recorded_at.desc())
.limit(limit)
)
if start is not None:
stmt = stmt.where(ModbusReading.recorded_at >= start)
if end is not None:
stmt = stmt.where(ModbusReading.recorded_at <= end)
# Fetch most-recent N rows, then reverse to restore ascending order for the response.
rows = list(reversed(db.execute(stmt).scalars().all()))
items = [ModbusReadingResponse(recorded_at=r.recorded_at, payload=r.payload) for r in rows]
return ModbusReadingsResponse(items=items)
# ---------------------------------------------------------------------------
# Metrics endpoint
# ---------------------------------------------------------------------------
@router.get("/devices/{uuid}/metrics", response_model=ModbusMetricsResponse)
def get_metrics(
uuid: str,
db: Session = Depends(get_db),
_auth: AuthenticatedSession = Depends(require_session),
) -> ModbusMetricsResponse:
"""Return the metric catalogue for a device's profile.
Each entry has ``key``, ``label`` (derived from key if the profile has
none — underscores → spaces, title-cased), ``unit``, and ``device_class``.
This is the authoritative metadata source for front-end card labels and
chart axis labels.
"""
device = _get_device_or_404(db, uuid)
try:
profile = load_profile(device.profile)
except ProfileNotFoundError:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
detail=f"Device profile '{device.profile}' could not be loaded.",
)
metrics = [
MetricInfo(
key=m.key,
label=_label_from_key(m.key),
unit=m.unit,
device_class=m.device_class,
)
for m in profile.metrics
]
return ModbusMetricsResponse(profile=profile.name, metrics=metrics)
# ---------------------------------------------------------------------------
# Test-read endpoint
# ---------------------------------------------------------------------------
@router.post("/devices/{uuid}/test", response_model=ModbusTestReadResponse)
def test_read(
uuid: str,
db: Session = Depends(get_db),
_auth: AuthenticatedSession = Depends(require_session),
_csrf: None = Depends(require_csrf),
) -> ModbusTestReadResponse:
"""Immediately read and decode the device once without persisting any data.
Useful for validating gateway connectivity and Modbus address settings
before relying on the background polling job.
This is a write-class endpoint (it initiates network I/O on demand) and
therefore requires a CSRF token. The response payload is never stored.
"""
device = _get_device_or_404(db, uuid)
try:
profile = load_profile(device.profile)
except ProfileNotFoundError:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
detail=f"Device profile '{device.profile}' could not be loaded.",
)
try:
registers = modbus_driver.read_blocks(
device.host,
device.port,
device.unit_id,
[{"start": b.start, "count": b.count} for b in profile.blocks],
)
payload: dict[str, Any] = decode_profile(profile, registers)
return ModbusTestReadResponse(ok=True, payload=payload)
except ModbusDriverError as exc:
logger.warning(
"Test read failed for device %r (uuid=%s): %s",
device.friendly_name,
uuid,
exc,
)
return ModbusTestReadResponse(ok=False, error=str(exc))
except Exception as exc: # noqa: BLE001
logger.warning(
"Unexpected error during test read for device %r (uuid=%s): %s",
device.friendly_name,
uuid,
exc,
)
return ModbusTestReadResponse(ok=False, error=f"Unexpected error: {exc}")
+184 -1
View File
@@ -2,7 +2,7 @@ from __future__ import annotations
import logging
from fastapi import APIRouter, Depends, HTTPException, Response, status
from fastapi import APIRouter, Depends, HTTPException, Request, Response, status
from sqlalchemy.orm import Session
from app.api.routes.api.deps import require_csrf, require_session
@@ -14,6 +14,12 @@ from app.schemas.session import (
SessionResponse,
SessionUser,
)
from app.schemas.totp import (
TotpDisableRequest,
TotpEnableRequest,
TotpSetupResponse,
TotpStatusResponse,
)
from app.services.auth import (
AuthPasswordChangeError,
AuthenticatedSession,
@@ -22,6 +28,8 @@ from app.services.auth import (
create_session,
revoke_session,
)
from app.services import login_throttle
from app.services import totp as totp_service
logger = logging.getLogger(__name__)
@@ -46,9 +54,26 @@ def get_session(
return _build_session_response(auth)
def _get_client_ip(request: Request, *, trust_forwarded_for: bool) -> str:
"""Extract the client IP address from the request.
When ``trust_forwarded_for`` is True (reverse-proxy deployments) the
left-most value from the ``X-Forwarded-For`` header is used. Otherwise the
direct socket IP (``request.client.host``) is used.
"""
if trust_forwarded_for:
xff = request.headers.get("X-Forwarded-For", "")
if xff:
return xff.split(",")[0].strip()
if request.client is not None:
return request.client.host
return "unknown"
@router.post("/auth/login", response_model=SessionResponse)
def post_login(
body: LoginRequest,
request: Request,
response: Response,
db: Session = Depends(get_db),
settings: Settings = Depends(get_app_settings),
@@ -58,15 +83,67 @@ def post_login(
On success, sets an HttpOnly session cookie and returns the session user + CSRF token.
On failure, returns 401 with no cookie set.
Repeated failures trigger exponential back-off (429 + Retry-After).
No X-CSRF-Token required (unauthenticated endpoint).
"""
client_ip = _get_client_ip(request, trust_forwarded_for=settings.auth_trust_forwarded_for)
# --- Throttle check (before any password verification) ---
if settings.auth_login_throttle_enabled:
wait_seconds = login_throttle.check_and_get_wait(
db, ip=client_ip, username=body.username
)
if wait_seconds > 0:
raise HTTPException(
status_code=status.HTTP_429_TOO_MANY_REQUESTS,
detail="too many failed login attempts; please try again later",
headers={"Retry-After": str(wait_seconds)},
)
# --- Password verification ---
user = authenticate_user(db, username=body.username, password=body.password)
if user is None:
if settings.auth_login_throttle_enabled:
login_throttle.register_failure(db, ip=client_ip, username=body.username)
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="invalid username or password",
)
# --- TOTP second-factor check (only when TOTP is enabled for the user) ---
if user.totp_enabled:
if not body.totp_code:
# Password correct but no TOTP code supplied: signal the front-end to
# prompt for the second factor. Do NOT issue a session.
# Deliberate: this is a normal two-step protocol step from a legitimate
# user — we do NOT register a throttle failure here. The attacker must
# already have the correct password to reach this branch, and counting
# each normal first-step as a failure would risk locking out the real
# admin on every login attempt.
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail={"totp_required": True},
)
# TOTP code supplied — verify against time-based code or a recovery code.
totp_ok = totp_service.verify_totp_code(user, body.totp_code)
if not totp_ok:
totp_ok = totp_service.verify_recovery_code(db, user=user, code=body.totp_code)
if not totp_ok:
# Second-factor failure is an active attack signal — register failure
# so that repeated wrong TOTP/recovery-code guesses trigger back-off.
if settings.auth_login_throttle_enabled:
login_throttle.register_failure(db, ip=client_ip, username=body.username)
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="invalid username or password",
)
# --- Success: clear back-off state and issue session ---
if settings.auth_login_throttle_enabled:
login_throttle.clear(db, ip=client_ip, username=body.username)
auth_session, raw_token = create_session(db, user=user, settings=settings)
logger.info("Created API authenticated session for user '%s'", user.username)
@@ -139,3 +216,109 @@ def post_change_password(
logger.info("Password updated for user '%s'", auth.user.username)
return Response(status_code=status.HTTP_204_NO_CONTENT)
# ---------------------------------------------------------------------------
# TOTP endpoints (M4-T05)
# ---------------------------------------------------------------------------
@router.post("/auth/totp/setup", response_model=TotpSetupResponse)
def post_totp_setup(
db: Session = Depends(get_db),
settings: Settings = Depends(get_app_settings),
auth: AuthenticatedSession = Depends(require_session),
_csrf: None = Depends(require_csrf),
) -> TotpSetupResponse:
"""
Generate a new pending TOTP secret, otpauth URI, and one-time recovery codes.
The secret is stored in the DB but TOTP is NOT yet enabled (totp_enabled stays
False until the user confirms with POST /api/auth/totp/enable).
Recovery codes are returned here as plaintext exactly once; their Argon2 hashes
are persisted immediately so enable only needs to flip the enabled flag.
Repeating this call replaces any prior pending secret and regenerates codes.
Requires: session cookie + X-CSRF-Token.
"""
secret, otpauth_uri, recovery_codes = totp_service.setup(
db,
user=auth.user,
issuer=settings.effective_totp_issuer,
)
return TotpSetupResponse(
secret=secret,
otpauth_uri=otpauth_uri,
recovery_codes=recovery_codes,
)
@router.post("/auth/totp/enable", status_code=status.HTTP_204_NO_CONTENT)
def post_totp_enable(
body: TotpEnableRequest,
db: Session = Depends(get_db),
auth: AuthenticatedSession = Depends(require_session),
_csrf: None = Depends(require_csrf),
) -> Response:
"""
Enable TOTP by confirming with the current 6-digit code from the authenticator app.
Requires a prior call to POST /api/auth/totp/setup (so that a pending secret
exists). On success, totp_enabled becomes True.
Returns 400 if the code is wrong or there is no pending secret.
Requires: session cookie + X-CSRF-Token.
"""
ok = totp_service.enable(db, user=auth.user, code=body.code)
if not ok:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="invalid TOTP code or no pending setup",
)
return Response(status_code=status.HTTP_204_NO_CONTENT)
@router.post("/auth/totp/disable", status_code=status.HTTP_204_NO_CONTENT)
def post_totp_disable(
body: TotpDisableRequest,
db: Session = Depends(get_db),
auth: AuthenticatedSession = Depends(require_session),
_csrf: None = Depends(require_csrf),
) -> Response:
"""
Disable TOTP. The caller must provide exactly one of:
- ``password``: the user's current login password, OR
- ``code``: the current 6-digit TOTP code.
On success: totp_enabled=False, totp_secret cleared, all recovery codes deleted.
Returns 400 if neither credential matches or neither is provided.
Requires: session cookie + X-CSRF-Token.
"""
ok = totp_service.disable(
db,
user=auth.user,
password=body.password,
code=body.code,
)
if not ok:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="invalid credential; provide a valid password or TOTP code",
)
return Response(status_code=status.HTTP_204_NO_CONTENT)
@router.get("/auth/totp", response_model=TotpStatusResponse)
def get_totp_status(
auth: AuthenticatedSession = Depends(require_session),
) -> TotpStatusResponse:
"""
Return the current TOTP status for the authenticated user.
Response contains only ``{"enabled": bool}``.
Secret and recovery codes are NEVER returned here.
Requires: session cookie only (no CSRF — read-only).
"""
return TotpStatusResponse(enabled=auth.user.totp_enabled)
+42
View File
@@ -37,6 +37,42 @@ class Settings(BaseSettings):
auth_session_cookie_name: str = "home_automation_session"
auth_session_ttl_hours: int = 12
auth_cookie_secure_override: bool | None = True
auth_login_throttle_enabled: bool = True
auth_trust_forwarded_for: bool = False
auth_totp_issuer: str = "" # defaults to app_name when empty
# Modbus polling — global kill-switch (CONFIG_FIELDS registered in T08).
# Off by default: polling is opt-in so a fresh deploy writes no modbus_reading
# rows until the admin explicitly enables it (matches mqtt/discovery defaults).
modbus_polling_enabled: bool = False
# MQTT broker connection (T08 wires into CONFIG_FIELDS/UI; T10 builds the client).
mqtt_enabled: bool = False
mqtt_broker_host: str = ""
mqtt_broker_port: int = 1883
mqtt_username: str = ""
mqtt_password: str = ""
mqtt_tls_enabled: bool = False
# Home Assistant MQTT Discovery (T08 wires into CONFIG_FIELDS/UI; T11 does publishing).
ha_discovery_enabled: bool = False
ha_discovery_prefix: str = "homeassistant"
# State/availability topics use a separate prefix so they live outside the HA discovery
# namespace. HA still receives state via the state_topic declared in the discovery config.
ha_state_topic_prefix: str = "home_automation"
# DSMR smart-meter ingest via MQTT (M6; default off — opt-in).
# Subscribe to dsmr_mqtt_topic when dsmr_ingest_enabled=True.
dsmr_ingest_enabled: bool = False
dsmr_mqtt_topic: str = "dsmr/json"
dsmr_sample_interval_s: int = 10
# DSMR dual-tariff topic: publishes "1" (dal/off-peak) or "2" (normal/peak).
# Empty string = do not subscribe (tariff-aware pricing disabled).
dsmr_tariff_topic: str = "dsmr/meter-stats/electricity_tariff"
# Tibber dynamic pricing credentials (M6; only used when active contract kind=tibber).
tibber_api_token: str = ""
tibber_home_id: str = ""
model_config = SettingsConfigDict(
env_file=".env",
@@ -86,6 +122,12 @@ class Settings(BaseSettings):
return self.auth_cookie_secure_override
return not self.is_development
@computed_field
@property
def effective_totp_issuer(self) -> str:
"""The issuer label shown in Authenticator apps. Falls back to app_name."""
return self.auth_totp_issuer.strip() or self.app_name
@lru_cache
def get_settings() -> Settings:
+1
View File
@@ -28,6 +28,7 @@ def _get_engine(database_url: str) -> Engine:
def _enable_sqlite_wal(dbapi_connection, _connection_record):
cursor = dbapi_connection.cursor()
cursor.execute("PRAGMA journal_mode=WAL")
cursor.execute("PRAGMA foreign_keys=ON")
cursor.close()
return engine
+869
View File
@@ -0,0 +1,869 @@
"""Expose framework: ExposableEntity, provider protocol, and build_catalog.
This module defines:
- ``ExposableEntity`` — a dataclass describing one publishable MQTT/HA entity.
- Provider protocol — a simple callable ``(session) -> list[ExposableEntity]``.
- A provider registry and registration helper.
- ``build_catalog(session)`` — merges all registered providers' entities with
per-key toggle state from the ``exposed_entity_toggle`` table.
Design notes
------------
- **No over-engineering**: providers are plain callables, not abstract base
classes. This project is personal / single-user; keep it flat.
- **Key stability**: entity keys MUST be derived from device uuid + metric key
(e.g. ``"modbus.<uuid>.voltage"``), never from auto-increment IDs. Stable
keys mean the toggle table survives device removal/re-addition without drift.
- **Metadata from profile**: the modbus provider reads ``device_class``,
``unit``, and ``component`` directly from the YAML profile's ``metrics[]``
rather than maintaining a separate mapping.
- **value_getter**: a callable ``() -> object | None`` or ``None`` if not yet
implemented (T11 will wire real getters). Presence of the field lets T11
call it without changing the dataclass interface.
"""
from __future__ import annotations
import logging
from dataclasses import dataclass, field
from typing import Any, Callable, Optional, Protocol
from sqlalchemy.orm import Session
logger = logging.getLogger(__name__)
# ---------------------------------------------------------------------------
# ExposableEntity
# ---------------------------------------------------------------------------
@dataclass
class DeviceInfo:
"""Grouping metadata that maps an entity to a logical HA device.
``identifiers`` corresponds to ``device.identifiers`` in HA Discovery
config — a stable tuple used to group entities under one device card.
``name`` is the human-readable device name (friendly_name).
"""
identifiers: tuple[str, ...]
"""Stable identifiers for HA device grouping (e.g. ``("modbus", "<uuid>")``).
These must not change over the device lifetime; they anchor the HA device
card even when friendly_name changes.
"""
name: str
"""Human-readable device name (may change; triggers HA discovery re-publish)."""
provides_availability: bool = True
"""Whether this device publishes an availability ("online"/"offline") heartbeat.
When True (the default — e.g. Modbus devices, which have an ``online``
binary_sensor driving availability), the HA Discovery config for each entity
declares an ``availability`` topic and HA marks the entity unavailable until
"online" is published.
When False (e.g. the energy-cost device, which has only sensors and no
heartbeat source), the config omits ``availability`` so HA treats the entity
as *always available* and shows its state as soon as one arrives. Without
this, such entities would stay perpetually ``unavailable`` in HA even though
their state is being published.
"""
@dataclass
class ExposableEntity:
"""One publishable entity in the MQTT / HA Discovery expose framework.
Fields
------
key
Stable unique identifier for this entity. Used as the toggle table
key and as part of the HA Discovery ``unique_id`` derivation.
Convention: ``"<provider>.<device_uuid>.<metric_key>"``,
e.g. ``"modbus.abc123.voltage"`` or ``"modbus.abc123.online"``.
component
HA MQTT component type: ``"sensor"``, ``"binary_sensor"``, ``"switch"``, etc.
device
Grouping info (DeviceInfo) that determines which HA device card this
entity belongs to.
device_class
HA ``device_class`` string (e.g. ``"voltage"``, ``"power"``, ``"None"``).
Empty string or ``None`` means no device_class (e.g. for the online sensor).
unit
Physical unit string (e.g. ``"V"``, ``"A"``, ``"kWh"``).
Empty string for dimensionless.
name
Human-readable entity label (friendly display name, may include device
context, e.g. ``"SDM120 Voltage"``).
value_getter
Optional callable ``(session) -> object | None`` that returns the current
entity value given an open SQLAlchemy session. ``None`` means "not yet
wired". T11 calls this to obtain the current state before publishing.
Signature: ``Callable[[Session], Any]``.
state_class
HA ``state_class`` string (e.g. ``"measurement"``, ``"total_increasing"``).
``None`` means not set.
"""
key: str
component: str
device: DeviceInfo
device_class: Optional[str]
unit: str
name: str
value_getter: Optional[Callable[["Session"], Any]] = field(default=None, repr=False)
state_class: Optional[str] = None
# ---------------------------------------------------------------------------
# Provider protocol
# ---------------------------------------------------------------------------
class EntityProvider(Protocol):
"""Protocol for entity provider callables.
A provider is any callable that accepts a ``Session`` and returns a list
of ``ExposableEntity`` objects. Using a Protocol (not ABC) keeps the
implementation lightweight — any matching callable qualifies.
"""
def __call__(self, session: Session) -> list[ExposableEntity]: ...
# ---------------------------------------------------------------------------
# Provider registry
# ---------------------------------------------------------------------------
_REGISTRY: list[EntityProvider] = []
def register_provider(provider: EntityProvider) -> EntityProvider:
"""Register an entity provider.
Can be used as a decorator::
@register_provider
def my_provider(session: Session) -> list[ExposableEntity]:
...
Or called directly::
register_provider(my_provider)
Returns the provider unchanged (for decorator compatibility).
"""
_REGISTRY.append(provider)
return provider
def get_providers() -> list[EntityProvider]:
"""Return a snapshot of the currently registered providers."""
return list(_REGISTRY)
# ---------------------------------------------------------------------------
# Catalog builder
# ---------------------------------------------------------------------------
@dataclass
class CatalogEntry:
"""An entity from the catalog, enriched with its current toggle state."""
entity: ExposableEntity
enabled: bool
"""True if this entity is currently enabled in the toggle table."""
def build_catalog(session: Session) -> list[CatalogEntry]:
"""Enumerate all entities from registered providers and attach toggle state.
For each entity key, the toggle state is looked up in ``exposed_entity_toggle``.
Entities with no toggle row default to ``enabled=False``.
Parameters
----------
session:
Active SQLAlchemy session for DB access.
Returns
-------
list[CatalogEntry]
All entities from all registered providers, each paired with its
current ``enabled`` state.
"""
from app.models.expose import ExposedEntityToggle # local import to avoid circular
# Collect all entities from all providers.
all_entities: list[ExposableEntity] = []
for provider in _REGISTRY:
try:
entities = provider(session)
all_entities.extend(entities)
except Exception:
logger.exception("Provider %r raised an error; skipping its entities", provider)
if not all_entities:
return []
# Load all toggle rows in one query for efficiency.
keys = [e.key for e in all_entities]
toggle_rows = session.query(ExposedEntityToggle).filter(
ExposedEntityToggle.key.in_(keys)
).all()
toggle_map: dict[str, bool] = {row.key: row.enabled for row in toggle_rows}
return [
CatalogEntry(entity=entity, enabled=toggle_map.get(entity.key, False))
for entity in all_entities
]
# ---------------------------------------------------------------------------
# Modbus provider
# ---------------------------------------------------------------------------
def _modbus_provider(session: Session) -> list[ExposableEntity]:
"""Enumerate ExposableEntity objects for all enabled Modbus devices.
For each enabled ``ModbusDevice``:
- Loads its YAML profile.
- Produces one ``sensor`` entity per metric in ``profile.metrics``
(device_class / unit / component taken from the metric spec).
- Produces one ``binary_sensor`` entity named "online" (derived from
``device.last_poll_ok``).
Entity keys follow the pattern ``"modbus.<uuid>.<metric_key>"``, which is
stable across device renames and DB rebuilds.
"""
from app.models.modbus import ModbusDevice # local import
from app.integrations.modbus.profiles import (
load_profile,
ProfileNotFoundError,
ProfileValidationError,
)
devices: list[ModbusDevice] = session.query(ModbusDevice).filter(
ModbusDevice.enabled.is_(True)
).all()
entities: list[ExposableEntity] = []
for device in devices:
device_info = DeviceInfo(
identifiers=("modbus", device.uuid),
name=device.friendly_name,
)
# Load the profile to get metric metadata.
try:
profile = load_profile(device.profile)
except (ProfileNotFoundError, ProfileValidationError) as exc:
logger.warning(
"Skipping device %r (uuid=%s): cannot load profile %r: %s",
device.friendly_name,
device.uuid,
device.profile,
exc,
)
continue
# One sensor entity per profile metric.
for metric in profile.metrics:
entity_key = f"modbus.{device.uuid}.{metric.key}"
# Capture device id and metric key for the value_getter closure.
# The getter queries the most recent ModbusReading for this device
# and extracts the metric value from its payload.
_device_id = device.id
_metric_key = metric.key
def _make_value_getter(
dev_id: int, m_key: str
) -> Callable[["Session"], Any]:
"""Return a getter that fetches the latest reading value from DB."""
def _getter(sess: "Session") -> Any:
from app.models.modbus import ModbusReading
from sqlalchemy import desc
reading = (
sess.query(ModbusReading)
.filter(ModbusReading.device_id == dev_id)
.order_by(desc(ModbusReading.recorded_at))
.first()
)
if reading is None:
return None
return reading.payload.get(m_key)
return _getter
entities.append(
ExposableEntity(
key=entity_key,
component=metric.ha_component,
device=device_info,
device_class=metric.device_class or None,
unit=metric.unit,
name=f"{device.friendly_name} {metric.key.replace('_', ' ').title()}",
value_getter=_make_value_getter(_device_id, _metric_key),
state_class=metric.state_class,
)
)
# One binary_sensor entity for device "online" status.
online_key = f"modbus.{device.uuid}.online"
_dev_id_online = device.id
def _make_online_getter(dev_id: int) -> Callable[["Session"], Any]:
"""Return a getter that reports the device online status from DB."""
def _getter(sess: "Session") -> Any:
from app.models.modbus import ModbusDevice
dev = sess.query(ModbusDevice).filter(
ModbusDevice.id == dev_id
).first()
if dev is None:
return None
# Map last_poll_ok to HA binary_sensor payload strings.
if dev.last_poll_ok is True:
return "ON"
return "OFF"
return _getter
entities.append(
ExposableEntity(
key=online_key,
component="binary_sensor",
device=device_info,
device_class="connectivity",
unit="",
name=f"{device.friendly_name} Online",
value_getter=_make_online_getter(_dev_id_online),
state_class=None,
)
)
return entities
# Register the modbus provider at module load time.
register_provider(_modbus_provider)
# ---------------------------------------------------------------------------
# Energy Cost provider
# ---------------------------------------------------------------------------
def _energy_cost_provider(session: Session) -> list[ExposableEntity]:
"""Enumerate ExposableEntity objects for the energy cost subsystem.
Produces 6 sensor entities grouped under a single HA device whose identity
is anchored to the **current active electricity meter**:
- ``buy_price_now`` — current effective buy price (EUR/kWh or local currency).
- ``sell_price_now`` — current effective sell price (EUR/kWh or local currency).
- ``import_cost_total`` — cumulative import cost (total, monetary).
- ``export_revenue_total`` — cumulative export revenue (total, monetary).
- ``import_cost_today`` — today's import cost (total_increasing, monetary).
- ``export_revenue_today`` — today's export revenue (total_increasing, monetary).
Active meter requirement
------------------------
**If no active electricity meter exists, the provider returns ``[]``.**
No energy-cost entities are exposed to HA until a meter has been declared.
This prevents spurious sensor creation with an undefined device identity.
HA device identity (换表 → 新 sensor)
--------------------------------------
``identifiers[1]`` is set to the active meter's **uuid** (not the fixed
string ``"energy-cost"``). ``ha_discovery.py`` uses ``identifiers[1]`` as
the MQTT node_id and as part of the ``unique_id`` for every entity.
Declaring a new active electricity meter produces a new uuid → new node_id /
unique_id → HA creates a brand-new sensor, cleanly isolating post-swap data.
Entity key stability
--------------------
Entity keys remain the fixed stable strings (``"energy.buy_price_now"`` etc.),
**not** derived from the meter uuid. The ``exposed_entity_toggle`` table uses
keys as its primary handle; keeping them stable means toggled-on entities stay
enabled after a meter swap without requiring the user to re-tick them.
Current-price algorithm (source-agnostic, with fallback)
---------------------------------------------------------
Strategy A: read the ``pricing`` snapshot from the most recent non-degraded
``energy_cost_period`` row.
- For ``kind="tibber"``: the snapshot contains ``"buy"`` and ``"sell"`` keys
(per-unit prices in the contract currency).
- For ``kind="manual"``: ``"buy_normal"`` and ``"sell_normal"`` are used as
representative effective per-unit prices. (Both tariff-slot prices differ
only by the base rate; energy_tax and ODE are the same for both, so
buy_normal is the higher/conservative single representative value.)
When no non-degraded period exists or the pricing snapshot lacks the
expected keys, ``value_getter`` returns ``None`` (``publish_states``
skips None values automatically).
Cumulative totals
-----------------
``SUM(import_cost)`` and ``SUM(export_revenue)`` over **all non-degraded**
``energy_cost_period`` rows within the current meter's window. Degraded rows
carry 0 costs and are excluded to avoid double-counting when they are later
overwritten by real values.
Currency
--------
Taken from the most recent non-degraded period's ``currency`` column.
Falls back to ``"EUR"`` when no such row exists.
Key convention
--------------
Fixed stable string keys (not derived from any mutable field or DB id):
- ``"energy.buy_price_now"``
- ``"energy.sell_price_now"``
- ``"energy.import_cost_total"``
- ``"energy.export_revenue_total"``
- ``"energy.import_cost_today"``
- ``"energy.export_revenue_today"``
DeviceInfo identifiers
----------------------
**Two-element tuple** ``("energy-cost", meter.uuid)`` so that
``ha_discovery.py``'s ``entity.device.identifiers[1]`` resolves to the
meter uuid (used as the MQTT node_id and unique_id seed throughout).
"""
from app.models.energy import EnergyCostPeriod, Meter # local import to avoid circular
from sqlalchemy import select
# --- Require an active electricity meter; return [] if none exists ---
# Using an inline query (ended_at IS NULL) rather than a service-layer helper
# to avoid a new public dependency and remain consistent with the value_getter
# implementations below (which use the same inline pattern).
active_meter: Meter | None = session.execute(
select(Meter)
.where(
Meter.commodity == "electricity",
Meter.ended_at.is_(None),
)
.limit(1)
).scalar_one_or_none()
if active_meter is None:
# No active electricity meter → do not expose any energy-cost entities.
# HA will not see these sensors until a meter is declared.
return []
# --- Determine currency from the latest non-degraded row ---
latest_period: EnergyCostPeriod | None = (
session.query(EnergyCostPeriod)
.filter(EnergyCostPeriod.degraded.is_(False))
.order_by(EnergyCostPeriod.period_start.desc())
.first()
)
currency: str = "EUR"
if latest_period is not None and latest_period.currency:
currency = latest_period.currency
# --- Shared DeviceInfo anchored to the active meter's uuid ---
# identifiers[1] = meter.uuid drives the MQTT node_id and unique_id in
# ha_discovery.py. Swapping the meter produces a new uuid → new HA sensor.
# provides_availability=False: the energy-cost device has only sensors and no
# online/offline heartbeat, so its entities must be "always available" in HA.
# (Otherwise HA shows them unavailable despite state being published.)
device_info = DeviceInfo(
identifiers=("energy-cost", active_meter.uuid),
name=active_meter.label,
provides_availability=False,
)
# --- value_getter: current buy price ---
def _make_buy_price_getter() -> Callable[["Session"], Any]:
"""Return a getter for the current buy price per kWh.
For ``kind="manual"`` contracts the getter selects the price slot that
matches the current DSMR electricity tariff:
- tariff == 1 (dal / off-peak) → ``buy_dal``
- tariff == 2 (normal / peak) → ``buy_normal``
- tariff is None / unknown → ``buy_normal`` (safe fallback = current status quo)
For ``kind="tibber"`` (hourly dynamic pricing) the tariff slot is not
applicable; the getter always uses the ``buy`` key from the snapshot.
"""
def _getter(sess: "Session") -> Any:
from app.models.energy import EnergyCostPeriod as _ECP
from app.services.dsmr_ingest import get_current_tariff
period = (
sess.query(_ECP)
.filter(_ECP.degraded.is_(False))
.order_by(_ECP.period_start.desc())
.first()
)
if period is None or not period.pricing:
return None
snap = period.pricing
kind = snap.get("kind")
if kind == "tibber":
raw = snap.get("buy")
elif kind == "manual":
tariff = get_current_tariff()
if tariff == 1:
raw = snap.get("buy_dal")
else:
# tariff == 2, None, or any unexpected value → normal (peak) rate.
raw = snap.get("buy_normal")
else:
# Unknown kind — attempt common keys gracefully.
raw = snap.get("buy") or snap.get("buy_normal")
if raw is None:
return None
try:
return float(raw)
except (TypeError, ValueError):
return None
return _getter
# --- value_getter: current sell price ---
def _make_sell_price_getter() -> Callable[["Session"], Any]:
"""Return a getter for the current sell price per kWh.
For ``kind="manual"`` contracts the getter selects the price slot that
matches the current DSMR electricity tariff:
- tariff == 1 (dal / off-peak) → ``sell_dal``
- tariff == 2 (normal / peak) → ``sell_normal``
- tariff is None / unknown → ``sell_normal`` (safe fallback = current status quo)
For ``kind="tibber"`` the tariff slot is not applicable; always uses ``sell``.
"""
def _getter(sess: "Session") -> Any:
from app.models.energy import EnergyCostPeriod as _ECP
from app.services.dsmr_ingest import get_current_tariff
period = (
sess.query(_ECP)
.filter(_ECP.degraded.is_(False))
.order_by(_ECP.period_start.desc())
.first()
)
if period is None or not period.pricing:
return None
snap = period.pricing
kind = snap.get("kind")
if kind == "tibber":
raw = snap.get("sell")
elif kind == "manual":
tariff = get_current_tariff()
if tariff == 1:
raw = snap.get("sell_dal")
else:
# tariff == 2, None, or any unexpected value → normal (peak) rate.
raw = snap.get("sell_normal")
else:
raw = snap.get("sell") or snap.get("sell_normal")
if raw is None:
return None
try:
return float(raw)
except (TypeError, ValueError):
return None
return _getter
# --- value_getter: cumulative import cost (incl. standing charges) ---
def _make_import_cost_getter() -> Callable[["Session"], Any]:
"""Return a getter for the cumulative import cost plus prorated standing charges.
D2 (M7): anchor = current active electricity meter's ``started_at``.
After a meter swap the cumulative resets to zero for the new meter —
old-meter periods have ``period_start < new_meter.started_at`` and fall
outside the [anchor, now) window, so they are naturally excluded.
Cross-meter boundary periods are already degraded and also excluded.
No active electricity meter → returns None (cannot anchor; safer than
returning a stale or wrong value). The check is done via an inline
query (``ended_at IS NULL``) rather than ``meter_at(now)`` to be robust
against the edge case where the active meter's ``started_at`` is in the
future (``meter_at(now)`` would return None in that scenario).
No non-degraded periods at all → returns None (has_data guard).
No active contract → ``summarize`` still runs but fixed_costs = 0;
the return value is the pure metered sum within the current meter window.
This is consistent with D2: the anchor is the meter, not the contract.
value = summarize(anchor → now).metered_import + summarize(anchor → now).fixed_costs
"""
def _getter(sess: "Session") -> Any:
from datetime import UTC, datetime as _dt
from app.models.energy import EnergyCostPeriod as _ECP, Meter as _Meter
from app.services.energy_cost import summarize as _summarize
from sqlalchemy import func as _func, select as _select
# Quick check: any non-degraded period at all?
has_data = sess.query(_func.sum(_ECP.import_cost)).filter(
_ECP.degraded.is_(False)
).scalar()
if has_data is None:
return None
# D2: anchor = active electricity meter's started_at.
# Inline query (ended_at IS NULL) is more robust than meter_at(now)
# because it avoids the edge case where started_at is in the future.
active_meter = sess.execute(
_select(_Meter)
.where(
_Meter.commodity == "electricity",
_Meter.ended_at.is_(None),
)
.limit(1)
).scalar_one_or_none()
if active_meter is None:
# No active electricity meter — cannot anchor; return None.
return None
from app.services.contracts import _as_utc as _cu
anchor_utc = _cu(active_meter.started_at)
now_utc = _dt.now(UTC)
result = _summarize(sess, anchor_utc, now_utc)
return result["metered_import"] + result["fixed_costs"]
return _getter
# --- value_getter: cumulative export revenue (incl. tax credit) ---
def _make_export_revenue_getter() -> Callable[["Session"], Any]:
"""Return a getter for the cumulative export revenue plus prorated tax credit.
D2 (M7): anchor = current active electricity meter's ``started_at``.
Same reasoning as the import cost getter — see its docstring.
value = summarize(anchor → now).metered_export + summarize(anchor → now).credits
Returns None when no non-degraded period exists or no active electricity meter.
"""
def _getter(sess: "Session") -> Any:
from datetime import UTC, datetime as _dt
from app.models.energy import EnergyCostPeriod as _ECP, Meter as _Meter
from app.services.energy_cost import summarize as _summarize
from sqlalchemy import func as _func, select as _select
# Quick check: any non-degraded period at all?
has_data = sess.query(_func.sum(_ECP.export_revenue)).filter(
_ECP.degraded.is_(False)
).scalar()
if has_data is None:
return None
# D2: anchor = active electricity meter's started_at.
active_meter = sess.execute(
_select(_Meter)
.where(
_Meter.commodity == "electricity",
_Meter.ended_at.is_(None),
)
.limit(1)
).scalar_one_or_none()
if active_meter is None:
# No active electricity meter — cannot anchor; return None.
return None
from app.services.contracts import _as_utc as _cu
anchor_utc = _cu(active_meter.started_at)
now_utc = _dt.now(UTC)
result = _summarize(sess, anchor_utc, now_utc)
return result["metered_export"] + result["credits"]
return _getter
# --- value_getter: today's import cost (local-day window, resets at local midnight) ---
def _make_import_cost_today_getter() -> Callable[["Session"], Any]:
"""Return a getter for today's import cost (metered + standing for today).
Window: [local today 00:00, local tomorrow 00:00) in UTC.
After local midnight the window rolls to the new day → value resets to that
day's running cost, implementing "resets at local midnight" semantics for HA.
Uses timezone module via module-attribute access to preserve monkeypatch safety.
Returns None when no non-degraded period exists or no active contract.
"""
def _getter(sess: "Session") -> Any:
from app.services.contracts import active_contract_versions
from app.models.energy import EnergyCostPeriod as _ECP
from app.services.energy_cost import summarize as _summarize
from app.services import timezone as _tz_mod
from sqlalchemy import func as _func
from datetime import timedelta as _td
# Quick check: any non-degraded period at all?
has_data = sess.query(_func.sum(_ECP.import_cost)).filter(
_ECP.degraded.is_(False)
).scalar()
if has_data is None:
return None
versions = active_contract_versions(sess)
if not versions:
return None
# Today's window in UTC, using monkeypatch-safe module attribute calls.
today_local = _tz_mod.local_now().date()
tomorrow_local = today_local + _td(days=1)
today_start_utc = _tz_mod.local_midnight_utc(today_local)
tomorrow_start_utc = _tz_mod.local_midnight_utc(tomorrow_local)
result = _summarize(sess, today_start_utc, tomorrow_start_utc)
return result["metered_import"] + result["fixed_costs"]
return _getter
# --- value_getter: today's export revenue (local-day window, resets at local midnight) ---
def _make_export_revenue_today_getter() -> Callable[["Session"], Any]:
"""Return a getter for today's export revenue (metered + tax credit for today).
Same window semantics as import_cost_today.
Returns None when no non-degraded period exists or no active contract.
"""
def _getter(sess: "Session") -> Any:
from app.services.contracts import active_contract_versions
from app.models.energy import EnergyCostPeriod as _ECP
from app.services.energy_cost import summarize as _summarize
from app.services import timezone as _tz_mod
from sqlalchemy import func as _func
from datetime import timedelta as _td
# Quick check: any non-degraded period at all?
has_data = sess.query(_func.sum(_ECP.export_revenue)).filter(
_ECP.degraded.is_(False)
).scalar()
if has_data is None:
return None
versions = active_contract_versions(sess)
if not versions:
return None
today_local = _tz_mod.local_now().date()
tomorrow_local = today_local + _td(days=1)
today_start_utc = _tz_mod.local_midnight_utc(today_local)
tomorrow_start_utc = _tz_mod.local_midnight_utc(tomorrow_local)
result = _summarize(sess, today_start_utc, tomorrow_start_utc)
return result["metered_export"] + result["credits"]
return _getter
# Price unit string: "<currency>/kWh"
price_unit = f"{currency}/kWh"
entities: list[ExposableEntity] = [
ExposableEntity(
key="energy.buy_price_now",
component="sensor",
device=device_info,
device_class=None,
unit=price_unit,
name="Energy Buy Price Now",
value_getter=_make_buy_price_getter(),
state_class="measurement",
),
ExposableEntity(
key="energy.sell_price_now",
component="sensor",
device=device_info,
device_class=None,
unit=price_unit,
name="Energy Sell Price Now",
value_getter=_make_sell_price_getter(),
state_class="measurement",
),
ExposableEntity(
key="energy.import_cost_total",
component="sensor",
device=device_info,
device_class="monetary",
unit=currency,
name="Energy Import Cost (incl. standing)",
value_getter=_make_import_cost_getter(),
state_class="total",
),
ExposableEntity(
key="energy.export_revenue_total",
component="sensor",
device=device_info,
device_class="monetary",
unit=currency,
name="Energy Export Revenue (incl. tax credit)",
value_getter=_make_export_revenue_getter(),
state_class="total",
),
# Daily entities: reset at local midnight, state_class=total_increasing.
#
# device_class=monetary is correct here. The HA developer docs (long-term
# statistics section) prohibit MONETARY from being combined with
# state_class=measurement — they do NOT prohibit it with total or
# total_increasing. monetary+total_increasing is valid (the cumulative
# entities above use monetary+total with no issue).
#
# total_increasing is appropriate because these daily quantities are
# non-negative (import/export metered cost ≥ 0, fixed standing fee/tax
# credit ≥ 0) and monotonically non-decreasing within any given local day.
# When the local midnight rolls over the value drops back to ~1 day's fixed
# fee; HA interprets that decrease as a new cycle, giving correct daily
# aggregation without any explicit last_reset pipeline.
ExposableEntity(
key="energy.import_cost_today",
component="sensor",
device=device_info,
device_class="monetary",
unit=currency,
name="Energy Import Cost Today (incl. standing)",
value_getter=_make_import_cost_today_getter(),
state_class="total_increasing",
),
ExposableEntity(
key="energy.export_revenue_today",
component="sensor",
device=device_info,
device_class="monetary",
unit=currency,
name="Energy Export Revenue Today (incl. tax credit)",
value_getter=_make_export_revenue_today_getter(),
state_class="total_increasing",
),
]
return entities
# Register the energy cost provider at module load time.
register_provider(_energy_cost_provider)
+1
View File
@@ -0,0 +1 @@
"""Modbus integration package — driver, profile loader, and decoder."""
+203
View File
@@ -0,0 +1,203 @@
"""Modbus TCP driver — thin wrapper around pymodbus.
This module provides a single public function ``read_blocks`` that performs
one or more FC04 (Read Input Registers) block reads against a Modbus TCP
gateway and returns a flat ``dict[register_address -> 16-bit_value]`` map.
Design decisions
----------------
- **Only read function codes** (FC03/FC04) are exposed. There is no write path.
- float32 decoding uses ``struct.unpack('>f', ...)`` directly rather than the
pymodbus ``BinaryPayloadDecoder`` helper, which has had API churn across 3.x
releases. Big-endian word order + big-endian byte order means the 4 raw bytes
are already in standard network (big-endian) order.
- ``ModbusTcpClient`` is used in synchronous mode (``connect()`` / ``close()``).
The client is created fresh per call; this keeps the driver stateless and
avoids threading concerns at the cost of one TCP handshake per read cycle.
APScheduler jobs call this from a thread pool, so stateless is safer.
- pymodbus 3.13.x uses ``device_id=`` as the slave-address keyword argument
(renamed from ``slave=`` in earlier 3.x releases).
Exceptions
----------
``ModbusDriverError``
Base exception for all driver-level errors.
``ModbusConnectionError``
Raised when the TCP connection to the gateway cannot be established or is
lost during the read.
``ModbusResponseError``
Raised when the gateway responds with a Modbus exception frame or when the
returned register count does not match the requested count.
"""
from __future__ import annotations
import logging
import struct
from typing import TYPE_CHECKING
from pymodbus.client import ModbusTcpClient
from pymodbus.exceptions import ConnectionException, ModbusException
if TYPE_CHECKING:
from collections.abc import Sequence
logger = logging.getLogger(__name__)
# ---------------------------------------------------------------------------
# Custom exceptions
# ---------------------------------------------------------------------------
class ModbusDriverError(RuntimeError):
"""Base class for all Modbus driver errors."""
class ModbusConnectionError(ModbusDriverError):
"""Cannot connect to (or lost connection with) the Modbus TCP gateway."""
class ModbusResponseError(ModbusDriverError):
"""Modbus gateway returned an exception frame or an unexpected response."""
# ---------------------------------------------------------------------------
# Block definition type alias
# ---------------------------------------------------------------------------
#: A single read block: ``{"start": int, "count": int}``.
Block = dict[str, int]
# ---------------------------------------------------------------------------
# Public API
# ---------------------------------------------------------------------------
def read_blocks(
host: str,
port: int,
unit_id: int,
blocks: Sequence[Block],
*,
timeout: float = 3.0,
) -> dict[int, int]:
"""Read one or more contiguous register blocks via FC04 (input registers).
Parameters
----------
host:
Hostname or IP address of the Modbus TCP gateway.
port:
TCP port (typically 502).
unit_id:
Modbus slave / unit address (the device's Meter ID for SDM120).
blocks:
Sequence of ``{"start": int, "count": int}`` dicts describing the
contiguous register ranges to read. ``count`` is the number of
16-bit registers (not bytes).
timeout:
TCP connect/read timeout in seconds (default 3 s).
Returns
-------
dict[int, int]
Flat mapping of ``{register_address: 16-bit_register_value}`` for
every register returned across all blocks.
Raises
------
ModbusConnectionError
If the TCP connection to the gateway fails.
ModbusResponseError
If the gateway returns a Modbus exception frame or an unexpected
number of registers.
"""
client = ModbusTcpClient(host, port=port, timeout=timeout)
try:
connected = client.connect()
if not connected:
raise ModbusConnectionError(
f"Could not connect to Modbus gateway at {host}:{port}"
)
registers: dict[int, int] = {}
for block in blocks:
start: int = block["start"]
count: int = block["count"]
_read_block(client, unit_id, start, count, registers)
return registers
except ConnectionException as exc:
raise ModbusConnectionError(
f"Connection to Modbus gateway {host}:{port} failed: {exc}"
) from exc
except ModbusException as exc:
raise ModbusResponseError(f"Modbus protocol error: {exc}") from exc
finally:
client.close()
def _read_block(
client: ModbusTcpClient,
unit_id: int,
start: int,
count: int,
result: dict[int, int],
) -> None:
"""Read one block and merge into *result*. Raises on any error."""
try:
response = client.read_input_registers(start, count=count, device_id=unit_id)
except ConnectionException as exc:
raise ModbusConnectionError(
f"Lost connection while reading registers 0x{start:04X}+{count}: {exc}"
) from exc
if response.isError():
raise ModbusResponseError(
f"Modbus exception reading registers 0x{start:04X}+{count}: {response}"
)
regs = response.registers
if len(regs) != count:
raise ModbusResponseError(
f"Expected {count} registers from 0x{start:04X}, got {len(regs)}"
)
for offset, value in enumerate(regs):
result[start + offset] = value
# ---------------------------------------------------------------------------
# Decoding helpers
# ---------------------------------------------------------------------------
def registers_to_float(hi_reg: int, lo_reg: int) -> float:
"""Decode two 16-bit registers to a big-endian IEEE-754 float32.
The SDM120 (and most Modbus float devices) use big-endian word order:
the high 16-bit register comes first, then the low register. Combined
with big-endian byte order within each register, the raw 4-byte sequence
is standard network-byte-order (big-endian) float32.
Parameters
----------
hi_reg:
The first (higher-address-range, most-significant) 16-bit register.
lo_reg:
The second (lower-address-range, least-significant) 16-bit register.
Returns
-------
float
The decoded IEEE-754 single-precision floating-point value.
Example
-------
>>> registers_to_float(0x4366, 0x3334)
230.20001220703125
"""
raw = struct.pack(">HH", hi_reg, lo_reg)
return struct.unpack(">f", raw)[0]
+231
View File
@@ -0,0 +1,231 @@
"""Modbus profile loader, validator, and decoder.
A *profile* is a YAML file that describes the protocol-level knowledge for
a particular Modbus device model:
- Which registers to read (``blocks`` for efficient bulk reads).
- How to decode each quantity (``metrics``: register address, type, unit, …).
- Home Assistant metadata per metric (``device_class``, ``ha_component``, …).
Profiles live in ``app/integrations/modbus/profiles/<name>.yaml`` and are
located at runtime relative to *this file* (not CWD), so they work whether
the package is installed as a wheel, run in-process, or invoked via
``python -m scripts.modbus_cli``.
Design notes
------------
- **Purely data + functions** — no abstract base classes or inheritance.
- ``ModbusProfile`` and ``MetricSpec`` are Pydantic models: they validate on
construction and raise ``pydantic.ValidationError`` for malformed YAML.
- ``decode`` uses ``driver.registers_to_float`` for float32 quantities.
Unsupported types are silently skipped with a warning (future-proofing for
int16/uint16 etc.).
"""
from __future__ import annotations
import logging
from pathlib import Path
from typing import Optional
import yaml
from pydantic import BaseModel, ValidationError
from app.integrations.modbus.driver import registers_to_float
logger = logging.getLogger(__name__)
# Directory containing the YAML profiles, located relative to *this* file.
_PROFILES_DIR = Path(__file__).parent / "profiles"
# ---------------------------------------------------------------------------
# Pydantic models for profile validation
# ---------------------------------------------------------------------------
class MetricSpec(BaseModel):
"""Specification for a single measurable quantity within a profile."""
key: str
"""Stable identifier used as the key in decoded payloads (e.g. ``"voltage"``)."""
address: int
"""Starting Modbus register address (0-based, e.g. ``0x0000`` for voltage)."""
type: str
"""Encoding type — currently only ``"float32"`` is supported."""
unit: str
"""Physical unit string (e.g. ``"V"``, ``"A"``, ``"kWh"``). Empty string for dimensionless."""
device_class: str
"""Home Assistant device class (e.g. ``"voltage"``, ``"power"``, ``"energy"``)."""
ha_component: str
"""Home Assistant component type (e.g. ``"sensor"``)."""
state_class: Optional[str] = None
"""Home Assistant state class, if applicable (e.g. ``"total_increasing"``)."""
class BlockSpec(BaseModel):
"""A contiguous register block for bulk reading."""
start: int
"""First register address in the block."""
count: int
"""Number of 16-bit registers to read."""
class ModbusProfile(BaseModel):
"""Complete description of a Modbus device's protocol-level knowledge."""
name: str
"""Short identifier matching the YAML file name (e.g. ``"sdm120"``)."""
description: str
"""Human-readable description of the device."""
function_code: int
"""Modbus function code for reading measurements (4 = FC04 input registers)."""
word_order: str
"""Word (register) order: ``"big"`` means high register first."""
byte_order: str
"""Byte order within each register: ``"big"`` means standard network order."""
blocks: list[BlockSpec]
"""Register blocks to read in bulk, reducing Modbus transaction count."""
metrics: list[MetricSpec]
"""List of individual measurable quantities and their decoding rules."""
# ---------------------------------------------------------------------------
# Public API
# ---------------------------------------------------------------------------
class ProfileNotFoundError(FileNotFoundError):
"""Raised when the requested profile YAML file does not exist."""
class ProfileValidationError(ValueError):
"""Raised when a profile YAML file fails Pydantic validation."""
def load_profile(name: str) -> ModbusProfile:
"""Load and validate a Modbus device profile by name.
The profile file is expected at
``app/integrations/modbus/profiles/<name>.yaml``.
Parameters
----------
name:
Profile name without extension (e.g. ``"sdm120"``).
Returns
-------
ModbusProfile
Validated profile model.
Raises
------
ProfileNotFoundError
If ``profiles/<name>.yaml`` does not exist.
ProfileValidationError
If the YAML is syntactically valid but the content fails schema
validation (missing required fields, wrong types, etc.).
"""
path = _PROFILES_DIR / f"{name}.yaml"
if not path.exists():
raise ProfileNotFoundError(
f"Modbus profile '{name}' not found (looked for {path})"
)
with path.open("r", encoding="utf-8") as fh:
raw = yaml.safe_load(fh)
try:
profile = ModbusProfile.model_validate(raw)
except ValidationError as exc:
raise ProfileValidationError(
f"Profile '{name}' failed validation: {exc}"
) from exc
return profile
def list_profiles() -> list[tuple[str, str]]:
"""Return ``(name, description)`` pairs for all available profiles.
Scans ``app/integrations/modbus/profiles/*.yaml``, loads each, and
returns a sorted list of ``(name, description)`` tuples. Profiles that
fail to load are skipped with a warning.
"""
results: list[tuple[str, str]] = []
if not _PROFILES_DIR.exists():
return results
for path in sorted(_PROFILES_DIR.glob("*.yaml")):
name = path.stem
try:
profile = load_profile(name)
results.append((profile.name, profile.description))
except (ProfileNotFoundError, ProfileValidationError, Exception) as exc:
logger.warning("Skipping malformed profile '%s': %s", name, exc)
return results
def decode(profile: ModbusProfile, registers: dict[int, int]) -> dict[str, float]:
"""Decode raw register values into a mapping of ``{key: float_value}``.
For each metric in *profile*, the function looks up the two registers at
``address`` and ``address + 1`` in *registers* and decodes them as a
big-endian float32 (high register first).
Metrics whose registers are absent from *registers* are skipped (not
an error — this can happen if a block was not requested or a device does
not populate that register). Metrics with unsupported ``type`` values
are also skipped.
Parameters
----------
profile:
Validated ``ModbusProfile`` describing the device.
registers:
Mapping of ``{register_address: 16-bit_value}`` as returned by
``driver.read_blocks``.
Returns
-------
dict[str, float]
Decoded engineering values keyed by each metric's ``key`` field.
"""
result: dict[str, float] = {}
for metric in profile.metrics:
if metric.type == "float32":
hi_addr = metric.address
lo_addr = metric.address + 1
if hi_addr not in registers or lo_addr not in registers:
logger.debug(
"Skipping metric '%s': registers 0x%04X/0x%04X not available",
metric.key,
hi_addr,
lo_addr,
)
continue
value = registers_to_float(registers[hi_addr], registers[lo_addr])
result[metric.key] = value
else:
logger.warning(
"Metric '%s' has unsupported type '%s', skipping",
metric.key,
metric.type,
)
return result
@@ -0,0 +1,74 @@
name: sdm120
description: Eastron SDM120 single-phase energy meter
function_code: 4 # input registers (FC04)
word_order: big # high register first (big-endian word order)
byte_order: big # high byte first within each register
blocks:
# Registers 3000130095 (0x00000x005F): voltage through max export demand
- { start: 0x0000, count: 0x0060 }
# Registers 3034330346 (0x01560x0159): total active/reactive energy
- { start: 0x0156, count: 0x0004 }
metrics:
# Core measurements — addresses from SDM120 Modbus Protocol §4 (FC04 input registers)
# Each float32 occupies two consecutive 16-bit registers; address = Hex start in §4.
- key: voltage
address: 0x0000 # register 30001 — Voltage (V)
type: float32
unit: "V"
device_class: voltage
ha_component: sensor
- key: current
address: 0x0006 # register 30007 — Current (A)
type: float32
unit: "A"
device_class: current
ha_component: sensor
- key: active_power
address: 0x000C # register 30013 — Active power (W)
type: float32
unit: "W"
device_class: power
ha_component: sensor
- key: power_factor
address: 0x001E # register 30031 — Power factor (dimensionless)
type: float32
unit: ""
device_class: power_factor
ha_component: sensor
- key: frequency
address: 0x0046 # register 30071 — Frequency (Hz)
type: float32
unit: "Hz"
device_class: frequency
ha_component: sensor
- key: import_energy
address: 0x0048 # register 30073 — Import active energy (kWh)
type: float32
unit: "kWh"
device_class: energy
state_class: total_increasing
ha_component: sensor
- key: export_energy
address: 0x004A # register 30075 — Export active energy (kWh)
type: float32
unit: "kWh"
device_class: energy
state_class: total_increasing
ha_component: sensor
- key: total_energy
address: 0x0156 # register 30343 — Total active energy (kWh)
type: float32
unit: "kWh"
device_class: energy
state_class: total_increasing
ha_component: sensor
+341
View File
@@ -0,0 +1,341 @@
"""MQTT client integration (paho-mqtt 2.x).
Provides :class:`MqttManager`: a long-lived MQTT client that runs paho's
loop in a background thread. Designed to be started in FastAPI's lifespan
and shared across the application as a module-level singleton (``mqtt_manager``).
Key design decisions
--------------------
- **paho 2.x API**: ``Client`` requires ``CallbackAPIVersion.VERSION2`` as the
first positional argument. VERSION2 on_connect callback receives
``(client, userdata, connect_flags, reason_code, properties)`` — not the
older RC int.
- **Background thread**: ``loop_start()`` spawns a daemon thread; ``loop_stop()``
joins it cleanly on shutdown.
- **Never crash the process**: all paho operations are wrapped in try/except;
connection failure is logged but does not propagate to the caller.
- **Password safety**: the MQTT password is never passed to ``logger`` calls.
- **Reconnect**: ``reconnect(settings)`` tears down the old client and
establishes a fresh connection with the new settings. Callers (e.g. PUT
/api/config) call this after saving MQTT-related config values.
- **Subscribe**: ``subscribe(topic, handler)`` registers a topic → handler
mapping. Subscriptions are re-established automatically on reconnect.
The ``on_message`` callback dispatches incoming payloads to the registered
handler; handler exceptions are caught and logged so they never crash the
paho loop thread or the network connection.
"""
from __future__ import annotations
import logging
import threading
from collections.abc import Callable
from typing import TYPE_CHECKING
import paho.mqtt.client as mqtt
if TYPE_CHECKING:
from app.config import Settings
logger = logging.getLogger(__name__)
# MQTT settings keys that, when changed, should trigger a reconnect.
MQTT_SETTINGS_KEYS = {
"mqtt_enabled",
"mqtt_broker_host",
"mqtt_broker_port",
"mqtt_username",
"mqtt_password",
"mqtt_tls_enabled",
}
def _is_configured(settings: Settings) -> bool:
"""Return True if MQTT is enabled *and* the broker host is set."""
return bool(settings.mqtt_enabled and settings.mqtt_broker_host)
class MqttManager:
"""Long-lived MQTT client wrapper.
Lifecycle
---------
1. ``connect(settings)`` — start background loop + connect to broker.
2. ``publish(...)`` — publish messages while connected.
3. ``disconnect()`` — graceful teardown (called on app shutdown).
4. ``reconnect(settings)`` — disconnect then re-connect with new settings
(called after saving updated MQTT configuration).
When MQTT is not enabled or not configured, all methods are no-ops.
Connection failures are caught and logged; they do not raise.
"""
def __init__(self) -> None:
self._client: mqtt.Client | None = None
self._lock = threading.Lock()
self._connected = False
# topic → handler registry; persists across reconnects so subscriptions
# are automatically re-established when the client reconnects.
self._subscriptions: dict[str, Callable[[bytes], None]] = {}
# ------------------------------------------------------------------
# Public properties
# ------------------------------------------------------------------
@property
def is_connected(self) -> bool:
"""True if the underlying paho client is currently connected."""
return self._connected
# ------------------------------------------------------------------
# Public interface
# ------------------------------------------------------------------
def is_configured(self, settings: Settings) -> bool:
"""Return True if MQTT is enabled and broker host is configured."""
return _is_configured(settings)
def connect(self, settings: Settings) -> None:
"""Connect to the MQTT broker and start the background loop thread.
No-op if MQTT is not enabled or broker host is not set.
Connection errors are logged but do not raise.
"""
if not _is_configured(settings):
logger.debug("MQTT not configured or not enabled — skipping connect.")
return
with self._lock:
self._start_client(settings)
def disconnect(self) -> None:
"""Disconnect from the broker and stop the background loop thread.
No-op if no client is active.
"""
with self._lock:
self._stop_client()
def reconnect(self, settings: Settings) -> None:
"""Disconnect the current client (if any) and reconnect with *settings*.
Call this after saving updated MQTT configuration values.
No-op if MQTT is not enabled or broker host is not set.
"""
with self._lock:
self._stop_client()
self.connect(settings)
def publish(
self,
topic: str,
payload: str | bytes | None,
*,
retain: bool = False,
qos: int = 0,
) -> None:
"""Publish a message to *topic*.
If the client is not connected the call is silently skipped.
Errors are logged but do not raise.
"""
with self._lock:
client = self._client
if client is None or not self._connected:
logger.debug("MQTT publish skipped — not connected (topic=%s).", topic)
return
try:
client.publish(topic, payload=payload, qos=qos, retain=retain)
except Exception:
logger.exception("MQTT publish error (topic=%s).", topic)
def subscribe(self, topic: str, handler: Callable[[bytes], None]) -> None:
"""Register *handler* to be called when a message arrives on *topic*.
If the client is already connected, the subscription is sent to the
broker immediately. Otherwise it is queued and will be established
the next time ``_on_connect`` fires (including after a reconnect).
Handler exceptions are swallowed by the ``on_message`` dispatcher so
that a buggy handler can never crash the paho loop thread.
"""
self._subscriptions[topic] = handler
# Grab the client reference outside the lock to avoid holding the lock
# while calling back into paho (which may itself acquire internal locks).
with self._lock:
client = self._client
connected = self._connected
if client is not None and connected:
try:
client.subscribe(topic)
logger.debug("MQTT subscribed to topic=%s (immediate).", topic)
except Exception:
logger.exception("MQTT subscribe error (topic=%s).", topic)
def unsubscribe(self, topic: str) -> None:
"""Remove the handler for *topic* and unsubscribe from the broker.
Idempotent: unknown topics are ignored. Used to apply config changes
without a restart (e.g. when DSMR ingest is turned off or its topic
changes). If the client is connected, the broker unsubscribe is sent
immediately; either way the handler is removed from the registry so it
will not be re-subscribed on the next ``_on_connect``.
"""
self._subscriptions.pop(topic, None)
with self._lock:
client = self._client
connected = self._connected
if client is not None and connected:
try:
client.unsubscribe(topic)
logger.debug("MQTT unsubscribed from topic=%s.", topic)
except Exception:
logger.exception("MQTT unsubscribe error (topic=%s).", topic)
# ------------------------------------------------------------------
# Internal helpers — must be called with self._lock held
# ------------------------------------------------------------------
def _start_client(self, settings: Settings) -> None:
"""Build a fresh paho Client, configure it, and call loop_start + connect."""
client = mqtt.Client(
callback_api_version=mqtt.CallbackAPIVersion.VERSION2,
client_id="home-automation",
)
# Callbacks — VERSION2 on_connect signature:
# (client, userdata, connect_flags, reason_code, properties)
def _on_connect(
_client: mqtt.Client,
_userdata: object,
_flags: mqtt.ConnectFlags,
reason_code: mqtt.ReasonCode,
_properties: mqtt.Properties | None,
) -> None:
if reason_code.is_failure:
logger.warning("MQTT connection refused: %s", reason_code)
self._connected = False
else:
logger.info("MQTT connected (reason_code=%s).", reason_code)
self._connected = True
# Re-establish all registered subscriptions after (re-)connect.
for sub_topic in self._subscriptions:
try:
_client.subscribe(sub_topic)
logger.debug("MQTT re-subscribed to topic=%s.", sub_topic)
except Exception:
logger.exception(
"MQTT re-subscribe failed (topic=%s).", sub_topic
)
# VERSION2 on_disconnect signature:
# (client, userdata, disconnect_flags, reason_code, properties)
def _on_disconnect(
_client: mqtt.Client,
_userdata: object,
_disconnect_flags: mqtt.DisconnectFlags,
reason_code: mqtt.ReasonCode,
_properties: mqtt.Properties | None,
) -> None:
self._connected = False
if reason_code.is_failure:
logger.warning("MQTT disconnected unexpectedly (reason_code=%s).", reason_code)
else:
logger.info("MQTT disconnected cleanly.")
# VERSION2 on_message signature: (client, userdata, message)
def _on_message(
_client: mqtt.Client,
_userdata: object,
message: mqtt.MQTTMessage,
) -> None:
handler = self._subscriptions.get(message.topic)
if handler is None:
logger.debug(
"MQTT on_message: no handler for topic=%s.", message.topic
)
return
try:
handler(message.payload)
except Exception:
logger.exception(
"MQTT message handler raised for topic=%s (swallowed).",
message.topic,
)
client.on_connect = _on_connect
client.on_disconnect = _on_disconnect
client.on_message = _on_message
# TLS
if settings.mqtt_tls_enabled:
try:
client.tls_set()
except Exception:
logger.exception("MQTT TLS setup failed.")
return
# Credentials — never log the password
if settings.mqtt_username:
client.username_pw_set(
username=settings.mqtt_username,
password=settings.mqtt_password or None,
)
# Start background loop thread *before* connect so paho can handle
# the connection handshake asynchronously.
client.loop_start()
try:
client.connect(
host=settings.mqtt_broker_host,
port=settings.mqtt_broker_port,
keepalive=60,
)
except Exception:
# Log without leaking the password
logger.exception(
"MQTT connect failed (host=%s, port=%s). Stopping loop.",
settings.mqtt_broker_host,
settings.mqtt_broker_port,
)
try:
client.loop_stop()
except Exception:
pass
return
self._client = client
logger.info(
"MQTT client started (host=%s, port=%s).",
settings.mqtt_broker_host,
settings.mqtt_broker_port,
)
def _stop_client(self) -> None:
"""Disconnect and stop the background loop. Idempotent."""
client = self._client
if client is None:
return
self._client = None
self._connected = False
try:
client.disconnect()
except Exception:
logger.debug("MQTT disconnect raised (ignoring).", exc_info=True)
try:
client.loop_stop()
except Exception:
logger.debug("MQTT loop_stop raised (ignoring).", exc_info=True)
logger.info("MQTT client stopped.")
# ---------------------------------------------------------------------------
# Module-level singleton — shared across lifespan and route handlers
# ---------------------------------------------------------------------------
mqtt_manager = MqttManager()
+11
View File
@@ -0,0 +1,11 @@
"""Pricing profile framework and strategy registry.
This package provides:
- ``profiles``: Pydantic models describing the structure of pricing profiles
(``ManualProfile`` / ``TibberProfile``), YAML loaders, and contract-values
validation (``load_profile`` / ``list_profiles`` / ``validate_values``).
- ``strategies``: A lightweight registry of price-calculation strategies
(``register_strategy`` / ``get_strategy``), with built-in implementations for
the ``manual`` (fixed dual-tariff) and ``tibber`` (dynamic API) kinds.
"""
+393
View File
@@ -0,0 +1,393 @@
"""Pricing profile loader, validator, and contract-values checker.
A *pricing profile* is a YAML file that describes the **structure** of an energy
contract — which fields exist, their units, and which have defaults. Actual
pricing values (the numbers the user fills in via the UI) are stored in the DB
as ``EnergyContractVersion.values`` (a JSON blob) and must conform to the
corresponding profile's structure.
Profiles live in ``app/integrations/pricing/profiles/<kind>.yaml`` and are
located at runtime relative to *this file* (not CWD), matching the pattern
established by the Modbus profile loader.
Design notes
------------
- **Purely data + functions** — no abstract base classes or inheritance.
- Two Pydantic models — ``ManualProfile`` and ``TibberProfile`` — capture the
different structures of the two supported kinds. A thin union dispatcher in
``load_profile`` picks the right one.
- ``validate_values(kind, values)`` fills in fields that carry a ``default`` and
raises ``ProfileValidationError`` for missing required fields or wrong types.
- All errors are subclasses of built-ins so callers need not import this module
just to catch them.
"""
from __future__ import annotations
import logging
from pathlib import Path
from typing import Any, Optional
import yaml
from pydantic import BaseModel, ValidationError, model_validator
logger = logging.getLogger(__name__)
# Directory containing the YAML profiles, located relative to *this* file.
_PROFILES_DIR = Path(__file__).parent / "profiles"
# ---------------------------------------------------------------------------
# Custom exceptions
# ---------------------------------------------------------------------------
class ProfileNotFoundError(FileNotFoundError):
"""Raised when the requested profile YAML file does not exist."""
class ProfileValidationError(ValueError):
"""Raised when a profile YAML fails Pydantic validation or when contract
values do not conform to the profile structure."""
# ---------------------------------------------------------------------------
# Leaf-node models (a single pricing field: unit + optional default)
# ---------------------------------------------------------------------------
class FieldSpec(BaseModel):
"""Specification for a single numeric pricing field."""
unit: str
"""Physical / monetary unit string (e.g. ``"EUR/kWh"``, ``"EUR/month"``)."""
default: Optional[float] = None
"""Default value used when the field is absent from contract values.
``None`` means the field is required (no default)."""
# ---------------------------------------------------------------------------
# ManualProfile — fixed / variable dual-tariff contract structure
# ---------------------------------------------------------------------------
class ManualBuySpec(BaseModel):
normal: FieldSpec # high-tariff buy price (delivered_2 registers)
dal: FieldSpec # low-tariff buy price (delivered_1 registers)
class ManualSellSpec(BaseModel):
normal: FieldSpec # high-tariff sell / return price
dal: FieldSpec # low-tariff sell / return price
class ManualEnergySpec(BaseModel):
dual_tariff: bool # always True for manual profiles
buy: ManualBuySpec
sell: ManualSellSpec
energy_tax: FieldSpec # added to buy price; includes VAT
ode: FieldSpec # currently merged into energy_tax; default 0
class ManualStandingSpec(BaseModel):
network_fee: FieldSpec # EUR/month
management_fee: FieldSpec # EUR/month
class ManualCreditsSpec(BaseModel):
heffingskorting: FieldSpec # EUR/year — energy-tax credit deducted at summary
class ManualProfile(BaseModel):
"""Complete structure description for a ``manual`` pricing contract."""
kind: str
label: str
energy: ManualEnergySpec
standing: ManualStandingSpec
credits: ManualCreditsSpec
@model_validator(mode="after")
def _check_kind(self) -> "ManualProfile":
if self.kind != "manual":
raise ValueError(f"ManualProfile requires kind='manual', got {self.kind!r}")
return self
# ---------------------------------------------------------------------------
# TibberProfile — dynamic Tibber API contract structure
# ---------------------------------------------------------------------------
class TibberEnergySpec(BaseModel):
source: str # must be "tibber_api"
energy_tax: FieldSpec # subtracted from total to derive sell price
sell_adjust: FieldSpec # additional sell-price adjustment; default 0
class TibberStandingSpec(BaseModel):
management_fee: FieldSpec # EUR/month; has a default
network_fee: FieldSpec # EUR/month
class TibberCreditsSpec(BaseModel):
heffingskorting: FieldSpec # EUR/year
class TibberProfile(BaseModel):
"""Complete structure description for a ``tibber`` pricing contract."""
kind: str
label: str
energy: TibberEnergySpec
standing: TibberStandingSpec
credits: TibberCreditsSpec
@model_validator(mode="after")
def _check_kind(self) -> "TibberProfile":
if self.kind != "tibber":
raise ValueError(f"TibberProfile requires kind='tibber', got {self.kind!r}")
return self
# A union type for type hints where either profile is acceptable.
AnyProfile = ManualProfile | TibberProfile
# Map kind → Pydantic model class used for validation.
_PROFILE_MODELS: dict[str, type[ManualProfile] | type[TibberProfile]] = {
"manual": ManualProfile,
"tibber": TibberProfile,
}
# ---------------------------------------------------------------------------
# Public API
# ---------------------------------------------------------------------------
def load_profile(kind: str) -> AnyProfile:
"""Load and validate a pricing profile by kind name.
The profile file is expected at
``app/integrations/pricing/profiles/<kind>.yaml``.
Parameters
----------
kind:
Profile kind without extension (``"manual"`` or ``"tibber"``).
Returns
-------
ManualProfile | TibberProfile
Validated profile model.
Raises
------
ProfileNotFoundError
If ``profiles/<kind>.yaml`` does not exist.
ProfileValidationError
If the YAML is syntactically valid but fails schema validation.
"""
path = _PROFILES_DIR / f"{kind}.yaml"
if not path.exists():
raise ProfileNotFoundError(
f"Pricing profile '{kind}' not found (looked for {path})"
)
with path.open("r", encoding="utf-8") as fh:
raw = yaml.safe_load(fh)
if not isinstance(raw, dict):
raise ProfileValidationError(
f"Profile '{kind}': expected a YAML mapping, got {type(raw).__name__}"
)
# Choose the right Pydantic model based on the ``kind`` field in the YAML.
yaml_kind = raw.get("kind", kind)
model_cls = _PROFILE_MODELS.get(yaml_kind)
if model_cls is None:
raise ProfileValidationError(
f"Profile '{kind}' has unknown kind={yaml_kind!r}; "
f"supported: {list(_PROFILE_MODELS)}"
)
try:
return model_cls.model_validate(raw)
except ValidationError as exc:
raise ProfileValidationError(
f"Profile '{kind}' failed validation: {exc}"
) from exc
def list_profiles() -> list[dict[str, Any]]:
"""Return structural data for all available pricing profiles.
Scans ``app/integrations/pricing/profiles/*.yaml``, loads each, and returns
a list of dicts (Pydantic model dumps). Profiles that fail to load are
skipped with a warning.
Returns
-------
list[dict[str, Any]]
One entry per valid profile, suitable for JSON serialisation and
front-end form rendering.
"""
results: list[dict[str, Any]] = []
if not _PROFILES_DIR.exists():
return results
for path in sorted(_PROFILES_DIR.glob("*.yaml")):
kind = path.stem
try:
profile = load_profile(kind)
results.append(profile.model_dump())
except (ProfileNotFoundError, ProfileValidationError, Exception) as exc:
logger.warning("Skipping malformed pricing profile '%s': %s", kind, exc)
return results
# ---------------------------------------------------------------------------
# Contract-values validation
# ---------------------------------------------------------------------------
def _fill_defaults_manual(values: dict[str, Any], profile: ManualProfile) -> dict[str, Any]:
"""Return a copy of *values* with any ``ode`` default applied if missing."""
filled = dict(values)
energy = dict(filled.get("energy", {}))
# Apply default for ode (default=0) if absent.
if "ode" not in energy and profile.energy.ode.default is not None:
energy["ode"] = profile.energy.ode.default
filled["energy"] = energy
return filled
def _fill_defaults_tibber(values: dict[str, Any], profile: TibberProfile) -> dict[str, Any]:
"""Return a copy of *values* with sell_adjust and management_fee defaults applied."""
filled = dict(values)
energy = dict(filled.get("energy", {}))
# Apply default for sell_adjust (default=0) if absent.
if "sell_adjust" not in energy and profile.energy.sell_adjust.default is not None:
energy["sell_adjust"] = profile.energy.sell_adjust.default
filled["energy"] = energy
standing = dict(filled.get("standing", {}))
# Apply default for management_fee if absent.
if (
"management_fee" not in standing
and profile.standing.management_fee.default is not None
):
standing["management_fee"] = profile.standing.management_fee.default
filled["standing"] = standing
return filled
def _require_numeric(section: str, key: str, container: dict[str, Any]) -> None:
"""Assert that *container[key]* exists and is a number; raise ProfileValidationError."""
if key not in container:
raise ProfileValidationError(
f"Contract values missing required field '{section}.{key}'"
)
val = container[key]
if not isinstance(val, (int, float)):
raise ProfileValidationError(
f"Contract values field '{section}.{key}' must be a number, "
f"got {type(val).__name__!r}"
)
def _validate_manual_values(values: dict[str, Any], profile: ManualProfile) -> dict[str, Any]:
"""Validate and fill-defaults for manual contract values.
Returns the filled values dict on success. Raises ProfileValidationError
on missing required fields or wrong types.
"""
filled = _fill_defaults_manual(values, profile)
energy = filled.get("energy", {})
buy = energy.get("buy", {})
sell = energy.get("sell", {})
standing = filled.get("standing", {})
credits = filled.get("credits", {})
# Required energy.buy fields.
_require_numeric("energy.buy", "normal", buy)
_require_numeric("energy.buy", "dal", buy)
# Required energy.sell fields.
_require_numeric("energy.sell", "normal", sell)
_require_numeric("energy.sell", "dal", sell)
# Required energy fields.
_require_numeric("energy", "energy_tax", energy)
_require_numeric("energy", "ode", energy)
# Required standing fields.
_require_numeric("standing", "network_fee", standing)
_require_numeric("standing", "management_fee", standing)
# Required credits fields.
_require_numeric("credits", "heffingskorting", credits)
return filled
def _validate_tibber_values(values: dict[str, Any], profile: TibberProfile) -> dict[str, Any]:
"""Validate and fill-defaults for tibber contract values.
Returns the filled values dict on success. Raises ProfileValidationError
on missing required fields or wrong types.
"""
filled = _fill_defaults_tibber(values, profile)
energy = filled.get("energy", {})
standing = filled.get("standing", {})
credits = filled.get("credits", {})
# Required energy fields.
_require_numeric("energy", "energy_tax", energy)
_require_numeric("energy", "sell_adjust", energy)
# Required standing fields.
_require_numeric("standing", "management_fee", standing)
_require_numeric("standing", "network_fee", standing)
# Required credits fields.
_require_numeric("credits", "heffingskorting", credits)
return filled
def validate_values(kind: str, values: dict[str, Any]) -> dict[str, Any]:
"""Validate a contract-values dict against the named profile structure.
Fields that carry a ``default`` in the profile (e.g. ``ode``,
``sell_adjust``, tibber ``management_fee``) are silently filled in when
absent from *values*. Fields with no default that are absent, or fields
whose value is not a number, cause a ``ProfileValidationError``.
Parameters
----------
kind:
Profile kind (``"manual"`` or ``"tibber"``).
values:
Contract values dict as stored in ``EnergyContractVersion.values``.
Returns
-------
dict[str, Any]
A (possibly mutated) copy of *values* with defaults applied.
Raises
------
ProfileNotFoundError
If the profile YAML for *kind* does not exist.
ProfileValidationError
If required fields are missing or have wrong types.
"""
profile = load_profile(kind)
if isinstance(profile, ManualProfile):
return _validate_manual_values(values, profile)
if isinstance(profile, TibberProfile):
return _validate_tibber_values(values, profile)
# Unreachable with current kinds, but guard for future extensions.
raise ProfileValidationError(f"No validator implemented for kind={kind!r}")
@@ -0,0 +1,20 @@
kind: manual
label: 固定 / 可变费率(NL,双费率)
energy:
dual_tariff: true # use delivered_1/2, returned_1/2 for low/high tariffs
buy:
normal: { unit: EUR/kWh } # high-tariff buy price (delivered_2 register)
dal: { unit: EUR/kWh } # low-tariff buy price (delivered_1 register)
sell:
normal: { unit: EUR/kWh } # high-tariff sell / return price
dal: { unit: EUR/kWh } # low-tariff sell / return price (currently equal to normal)
energy_tax: { unit: EUR/kWh } # energy tax added to buy price (incl. VAT)
ode: { unit: EUR/kWh, default: 0 } # currently merged into energy_tax
standing: # fixed charges; UI fills per month, engine prorates to days
network_fee: { unit: EUR/month }
management_fee: { unit: EUR/month }
credits:
heffingskorting: { unit: EUR/year } # energy-tax credit; deducted at summary layer
@@ -0,0 +1,14 @@
kind: tibber
label: Tibber 动态电价(15 分钟)
energy:
source: tibber_api # buy = total (from API); sell = total energy_tax sell_adjust
energy_tax: { unit: EUR/kWh } # subtracted from total to derive sell price (incl. VAT)
sell_adjust: { unit: EUR/kWh, default: 0 } # additional sell-price adjustment (residual spread)
standing: # fixed charges; UI fills per month, engine prorates to days
management_fee: { unit: EUR/month, default: 5.99 }
network_fee: { unit: EUR/month }
credits:
heffingskorting: { unit: EUR/year } # energy-tax credit; deducted at summary layer
+288
View File
@@ -0,0 +1,288 @@
"""Price-calculation strategy registry and built-in implementations.
A *strategy* is a plain function registered under a ``kind`` string. Given
per-register kWh deltas, the period start time, the active contract-version
values, and a DB session, it returns a result dict with:
{
"import_cost": Decimal, # total cost of electricity drawn from grid
"export_revenue": Decimal, # total revenue from electricity fed to grid
"net_cost": Decimal, # import_cost export_revenue
"pricing": dict, # snapshot of the price inputs used (for auditing)
}
**Import and export are kept separate throughout** — net_cost is only derived
at the end. This supports the Dutch no-netting rule (saldering afgebouwd).
**All monetary arithmetic uses Decimal** converted via ``Decimal(str(x))`` to
avoid float binary rounding errors.
Design notes
------------
- **Purely data + functions**: no ABC, no class hierarchy. A ``dict``-based
registry is the simplest structure that supports the two current kinds and
leaves the door open for future additions (e.g. ``octopus``, ``frank``).
- ``deltas`` is a plain dataclass ``PeriodDeltas`` — typed, but no ORM.
- Tibber strategy queries ``TibberPrice`` directly from the session; it does not
call the Tibber API (that is T05's job).
"""
from __future__ import annotations
import logging
from dataclasses import dataclass
from datetime import datetime
from decimal import Decimal
from typing import Any, Callable
from sqlalchemy.orm import Session
logger = logging.getLogger(__name__)
# ---------------------------------------------------------------------------
# Shared data types
# ---------------------------------------------------------------------------
@dataclass
class PeriodDeltas:
"""Per-register kWh deltas for one 15-minute billing period.
All values are ``Decimal`` and represent ``end_reading start_reading``
for the corresponding cumulative energy register.
Attributes
----------
d1: Decimal
Delivered (consumed) kWh on the low/dal tariff register (delivered_1).
d2: Decimal
Delivered (consumed) kWh on the normal/high tariff register (delivered_2).
r1: Decimal
Returned (fed-to-grid) kWh on the low/dal tariff register (returned_1).
r2: Decimal
Returned (fed-to-grid) kWh on the normal/high tariff register (returned_2).
"""
d1: Decimal # delivered low-tariff
d2: Decimal # delivered high-tariff
r1: Decimal # returned low-tariff
r2: Decimal # returned high-tariff
# Strategy callable signature.
StrategyFn = Callable[
[PeriodDeltas, datetime, dict[str, Any], Session],
dict[str, Any],
]
# ---------------------------------------------------------------------------
# Registry
# ---------------------------------------------------------------------------
_REGISTRY: dict[str, StrategyFn] = {}
def register_strategy(kind: str, fn: StrategyFn) -> None:
"""Register *fn* as the price-calculation strategy for *kind*.
Overwrites any previously registered strategy for the same kind (allows
monkey-patching in tests).
"""
_REGISTRY[kind] = fn
def get_strategy(kind: str) -> StrategyFn:
"""Return the registered strategy for *kind*.
Raises
------
KeyError
If no strategy is registered for *kind*.
"""
if kind not in _REGISTRY:
raise KeyError(
f"No price strategy registered for kind={kind!r}. "
f"Available: {sorted(_REGISTRY)}"
)
return _REGISTRY[kind]
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _to_decimal(value: Any) -> Decimal:
"""Convert *value* to Decimal via str() to avoid float binary rounding."""
return Decimal(str(value))
# ---------------------------------------------------------------------------
# Manual strategy — fixed dual-tariff
# ---------------------------------------------------------------------------
def _manual_strategy(
deltas: PeriodDeltas,
t0: datetime,
values: dict[str, Any],
session: Session,
) -> dict[str, Any]:
"""Calculate billing costs for one period using fixed dual-tariff rates.
Formula (§3.4):
- ``buy_dal = energy.buy.dal + energy.energy_tax + energy.ode``
- ``buy_normal = energy.buy.normal + energy.energy_tax + energy.ode``
- ``import_cost = Δd1 × buy_dal + Δd2 × buy_normal``
- ``sell_dal = energy.sell.dal`` (no tax on return price)
- ``sell_normal = energy.sell.normal``
- ``export_revenue = Δr1 × sell_dal + Δr2 × sell_normal``
- ``net_cost = import_cost export_revenue``
The ``pricing`` snapshot contains all per-unit prices used so that the
result is fully auditable without re-querying the contract version.
"""
energy = values.get("energy", {})
buy = energy.get("buy", {})
sell = energy.get("sell", {})
energy_tax = _to_decimal(energy.get("energy_tax", 0))
ode = _to_decimal(energy.get("ode", 0))
buy_dal_base = _to_decimal(buy.get("dal", 0))
buy_normal_base = _to_decimal(buy.get("normal", 0))
sell_dal = _to_decimal(sell.get("dal", 0))
sell_normal = _to_decimal(sell.get("normal", 0))
# Effective buy prices including all taxes.
buy_dal = buy_dal_base + energy_tax + ode
buy_normal = buy_normal_base + energy_tax + ode
import_cost = deltas.d1 * buy_dal + deltas.d2 * buy_normal
export_revenue = deltas.r1 * sell_dal + deltas.r2 * sell_normal
net_cost = import_cost - export_revenue
pricing_snapshot = {
"kind": "manual",
"buy_dal": str(buy_dal),
"buy_normal": str(buy_normal),
"sell_dal": str(sell_dal),
"sell_normal": str(sell_normal),
"energy_tax": str(energy_tax),
"ode": str(ode),
}
return {
"import_cost": import_cost,
"export_revenue": export_revenue,
"net_cost": net_cost,
"pricing": pricing_snapshot,
}
# ---------------------------------------------------------------------------
# Tibber strategy — dynamic 15-minute spot price
# ---------------------------------------------------------------------------
class TibberPriceNotFoundError(LookupError):
"""Raised when no TibberPrice row covers the requested period start.
The billing engine (T07) catches this to mark the period as ``degraded``
or skip it until a price becomes available.
"""
def _tibber_strategy(
deltas: PeriodDeltas,
t0: datetime,
values: dict[str, Any],
session: Session,
) -> dict[str, Any]:
"""Calculate billing costs for one period using Tibber dynamic pricing.
Price lookup:
Takes the most recent ``TibberPrice`` row where ``starts_at ≤ t0``
(i.e. the slot that was in effect at *t0*). This is a DESC LIMIT 1
query on ``starts_at``.
Formula (§3.4):
- ``buy = total`` (Tibber's all-inclusive price, already includes tax)
- ``sell = total energy_tax sell_adjust``
- ``import_cost = (Δd1 + Δd2) × buy``
- ``export_revenue = (Δr1 + Δr2) × sell``
- ``net_cost = import_cost export_revenue``
Tibber does not differentiate tariff slots (dal vs normal) — the 15-minute
API price applies to the full delivered/returned volume.
Negative ``total`` (extreme negative spot prices):
When ``total`` is negative the ``sell`` price will also be negative,
meaning ``export_revenue`` becomes negative (feeding to grid *costs*
money). This is the mathematically correct outcome and is left as-is.
Raises
------
TibberPriceNotFoundError
If no ``TibberPrice`` row exists with ``starts_at ≤ t0``. The caller
(T07 billing engine) should catch this and mark the period as
``degraded`` or skip it for later recomputation.
"""
from app.models.energy import TibberPrice # local import to avoid circular
from sqlalchemy import desc
price_row: TibberPrice | None = (
session.query(TibberPrice)
.filter(TibberPrice.starts_at <= t0)
.order_by(desc(TibberPrice.starts_at))
.first()
)
if price_row is None:
raise TibberPriceNotFoundError(
f"No TibberPrice found covering t0={t0.isoformat()!r}; "
"period cannot be billed until price data is available."
)
energy = values.get("energy", {})
energy_tax = _to_decimal(energy.get("energy_tax", 0))
sell_adjust = _to_decimal(energy.get("sell_adjust", 0))
total = _to_decimal(price_row.total)
buy = total
sell = total - energy_tax - sell_adjust
total_delivered = deltas.d1 + deltas.d2
total_returned = deltas.r1 + deltas.r2
import_cost = total_delivered * buy
export_revenue = total_returned * sell
net_cost = import_cost - export_revenue
pricing_snapshot = {
"kind": "tibber",
"tibber_price_starts_at": price_row.starts_at.isoformat(),
"tibber_price_id": price_row.id,
"total": str(total),
"buy": str(buy),
"sell": str(sell),
"energy_tax": str(energy_tax),
"sell_adjust": str(sell_adjust),
}
return {
"import_cost": import_cost,
"export_revenue": export_revenue,
"net_cost": net_cost,
"pricing": pricing_snapshot,
}
# ---------------------------------------------------------------------------
# Register built-in strategies at module import time
# ---------------------------------------------------------------------------
register_strategy("manual", _manual_strategy)
register_strategy("tibber", _tibber_strategy)
+10
View File
@@ -0,0 +1,10 @@
"""Tibber dynamic electricity pricing integration.
This package provides:
- ``client``: HTTP client for the Tibber GraphQL API (price fetching).
- Custom exceptions: ``TibberError`` (base) and ``TibberAuthError`` (auth failure).
Usage::
from app.integrations.tibber.client import fetch_price_range, TibberAuthError
"""
+329
View File
@@ -0,0 +1,329 @@
"""Tibber GraphQL API client for fetching electricity price data.
Design decisions
----------------
- Uses ``httpx`` (already a project dependency) for all HTTP requests.
- Token is **never** written to log messages or exception strings to prevent
credential leakage into log aggregators.
- ``fetch_price_range`` returns a list of ``PricePoint`` objects with UTC-aware
``starts_at`` datetimes; the number of nodes is not assumed — all returned
nodes are parsed regardless of count.
- ``fetch_current_price`` uses a separate, simpler query that asks for the
*current* price point only; used by the connection-test endpoint (T09) to
produce a fast three-state result (success / auth-error / network-error).
- Authentication failures (HTTP 401/403) are raised as ``TibberAuthError`` so
that callers (the test endpoint) can distinguish them from generic network or
API errors.
"""
from __future__ import annotations
import logging
from dataclasses import dataclass
from datetime import UTC, datetime
from typing import Any
import httpx
logger = logging.getLogger(__name__)
_TIBBER_API_URL = "https://api.tibber.com/v1-beta/gql"
_DEFAULT_TIMEOUT = 15.0
# GraphQL query to fetch a range of 15-minute price nodes.
# ``priceInfoRange(resolution: QUARTER_HOURLY, first: 96)`` fetches up to
# 96 quarter-hourly slots which covers today + tomorrow (2 × 24 × 4 = 192 max,
# but the Tibber API typically starts from the current slot and returns at
# most the remaining hours of today plus tomorrow, so 96 is a good cap for
# "today + tomorrow").
_PRICE_RANGE_QUERY = """
{
viewer {
homes {
id
currentSubscription {
priceInfoRange(resolution: QUARTER_HOURLY, first: 96) {
nodes {
startsAt
total
energy
tax
currency
level
}
}
}
}
}
}
"""
# GraphQL query to fetch the single *current* price point.
_CURRENT_PRICE_QUERY = """
{
viewer {
homes {
id
currentSubscription {
priceInfo {
current {
startsAt
total
energy
tax
currency
level
}
}
}
}
}
}
"""
# ---------------------------------------------------------------------------
# Custom exceptions
# ---------------------------------------------------------------------------
class TibberError(Exception):
"""Base class for all Tibber client errors.
Raised for network failures, timeouts, unexpected HTTP status codes, and
malformed API responses. The exception message will **never** contain the
API token.
"""
class TibberAuthError(TibberError):
"""Raised when the Tibber API rejects the provided token (HTTP 401/403).
Callers that implement a three-state connection test should catch this
exception separately from ``TibberError`` to distinguish auth problems
from network / API problems.
"""
# ---------------------------------------------------------------------------
# Data model
# ---------------------------------------------------------------------------
@dataclass(slots=True)
class PricePoint:
"""One 15-minute price slot returned by the Tibber API.
All monetary values are in ``currency`` and include VAT (user-facing).
``starts_at`` is always a timezone-aware UTC datetime regardless of the
timezone offset that Tibber returns in ``startsAt``.
"""
starts_at: datetime # UTC-aware
total: float # full all-in price (energy + tax)
energy: float # spot energy component
tax: float # tax component
currency: str # ISO 4217 (typically "EUR")
level: str | None # e.g. "CHEAP", "NORMAL", "EXPENSIVE"; None if absent
resolution: str # label for the slot resolution (e.g. "QUARTER_HOURLY")
# ---------------------------------------------------------------------------
# Internal helpers
# ---------------------------------------------------------------------------
def _parse_starts_at(raw: str) -> datetime:
"""Parse an ISO 8601 timestamp with timezone offset and convert to UTC.
Tibber returns timestamps like ``2026-06-23T00:00:00.000+02:00``.
``datetime.fromisoformat`` handles this format in Python 3.11+; the result
is then converted to UTC via ``.astimezone(UTC)``.
"""
return datetime.fromisoformat(raw).astimezone(UTC)
def _parse_node(node: dict[str, Any], resolution: str) -> PricePoint:
"""Parse a single price node dict into a ``PricePoint``."""
return PricePoint(
starts_at=_parse_starts_at(node["startsAt"]),
total=float(node["total"]),
energy=float(node["energy"]),
tax=float(node["tax"]),
currency=str(node["currency"]),
level=node.get("level") or None,
resolution=resolution,
)
def _pick_home(homes: list[dict[str, Any]], home_id: str | None) -> dict[str, Any]:
"""Return the home dict matching *home_id*, or the first home if None."""
if not homes:
raise TibberError("Tibber API returned no homes")
if home_id is not None:
for h in homes:
if h.get("id") == home_id:
return h
raise TibberError("Tibber home id not found in API response")
return homes[0]
def _post_graphql(token: str, query: str, timeout: float) -> dict[str, Any]:
"""POST the GraphQL *query* and return the parsed ``data`` dict.
Raises
------
TibberAuthError
On HTTP 401 or 403.
TibberError
On timeouts, network errors, non-2xx responses, or unexpected body shape.
"""
headers = {
"Authorization": f"Bearer {token}",
"Content-Type": "application/json",
}
try:
response = httpx.post(
_TIBBER_API_URL,
json={"query": query},
headers=headers,
timeout=timeout,
)
except httpx.TimeoutException as exc:
raise TibberError("Tibber API request timed out") from exc
except httpx.HTTPError as exc:
raise TibberError("Tibber API request failed") from exc
if response.status_code in (401, 403):
raise TibberAuthError("Tibber API authentication failed")
try:
response.raise_for_status()
except httpx.HTTPStatusError as exc:
raise TibberError(f"Tibber API returned unexpected status {response.status_code}") from exc
try:
body = response.json()
except Exception as exc:
raise TibberError("Tibber API returned non-JSON response") from exc
if "errors" in body:
# GraphQL errors are not HTTP errors; surface them as TibberError.
# Do not include token in the message.
raise TibberError("Tibber GraphQL returned errors")
data = body.get("data")
if data is None:
raise TibberError("Tibber API response missing 'data' field")
return data
# ---------------------------------------------------------------------------
# Public API
# ---------------------------------------------------------------------------
def fetch_price_range(
token: str,
home_id: str | None = None,
*,
timeout: float = _DEFAULT_TIMEOUT,
) -> list[PricePoint]:
"""Fetch a range of 15-minute price nodes from the Tibber API.
Sends the ``priceInfoRange(resolution: QUARTER_HOURLY, first: 96)`` query
and parses every returned node into a ``PricePoint``. The number of nodes
is not assumed — all returned nodes are parsed regardless of count.
Parameters
----------
token:
Tibber API token. **Never** logged or included in exception messages.
home_id:
If given, the home with this Tibber home ID is selected. If ``None``,
the first home in the account is used.
timeout:
HTTP request timeout in seconds (default 15 s).
Returns
-------
list[PricePoint]
List of price points with UTC-aware ``starts_at`` values.
Raises
------
TibberAuthError
If the token is rejected (HTTP 401/403).
TibberError
For network failures, timeouts, or unexpected API responses.
"""
data = _post_graphql(token, _PRICE_RANGE_QUERY, timeout)
try:
homes = data["viewer"]["homes"]
except (KeyError, TypeError) as exc:
raise TibberError("Tibber API response has unexpected shape") from exc
home = _pick_home(homes, home_id)
try:
nodes = home["currentSubscription"]["priceInfoRange"]["nodes"]
except (KeyError, TypeError) as exc:
raise TibberError("Tibber API response missing priceInfoRange nodes") from exc
return [_parse_node(node, "QUARTER_HOURLY") for node in nodes]
def fetch_current_price(
token: str,
home_id: str | None = None,
*,
timeout: float = _DEFAULT_TIMEOUT,
) -> PricePoint:
"""Fetch the *current* price point from the Tibber API.
Uses the lighter ``priceInfo { current { ... } }`` query rather than the
full range query, making it suitable for a fast connection test.
Parameters
----------
token:
Tibber API token. **Never** logged or included in exception messages.
home_id:
If given, the home with this Tibber home ID is selected. If ``None``,
the first home in the account is used.
timeout:
HTTP request timeout in seconds (default 15 s).
Returns
-------
PricePoint
The current price point with a UTC-aware ``starts_at``.
Raises
------
TibberAuthError
If the token is rejected (HTTP 401/403).
TibberError
For network failures, timeouts, missing current price, or unexpected
API responses.
"""
data = _post_graphql(token, _CURRENT_PRICE_QUERY, timeout)
try:
homes = data["viewer"]["homes"]
except (KeyError, TypeError) as exc:
raise TibberError("Tibber API response has unexpected shape") from exc
home = _pick_home(homes, home_id)
try:
current = home["currentSubscription"]["priceInfo"]["current"]
except (KeyError, TypeError) as exc:
raise TibberError("Tibber API response missing current price") from exc
if current is None:
raise TibberError("Tibber API returned null for current price")
return _parse_node(current, "QUARTER_HOURLY")
+178 -1
View File
@@ -13,6 +13,11 @@ from sqlalchemy.orm import Session
from app import models # noqa: F401
from app.api.routes.api.config import router as api_config_router
from app.api.routes.api.data import router as api_data_router
from app.api.routes.api.energy import router as api_energy_router
from app.api.routes.api.energy_contracts import router as api_energy_contracts_router
from app.api.routes.api.expose import router as api_expose_router
from app.api.routes.api.meters import router as api_meters_router
from app.api.routes.api.modbus import router as api_modbus_router
from app.api.routes.api.session import router as api_session_router
from app.api.routes import status
from app.db import get_session_local
@@ -22,9 +27,15 @@ from app.api.routes.poo import router as poo_router
from app.api.routes.public_ip import router as public_ip_router
from app.api.routes.ticktick import router as ticktick_router
from app.config import get_settings
from app.integrations.mqtt import mqtt_manager
from app.services.auth import AuthBootstrapError, initialize_auth_schema
from app.services.config_page import seed_missing_config_from_bootstrap, sync_app_hostname_from_bootstrap
from app.services.config_page import build_runtime_settings, seed_missing_config_from_bootstrap, sync_app_hostname_from_bootstrap
from app.services.dsmr_ingest import apply_dsmr_subscription
from app.services.public_ip import check_public_ipv4_and_notify
from app.services.modbus_poll import poll_all_enabled_devices, BASE_POLL_TICK_SECONDS
from app.services.ha_discovery import publish_discovery, publish_states
from app.services.tibber_prices import refresh_prices
from app.services.energy_cost import compute_closed_periods
from scripts.app_db_adopt import AppDatabaseAdoptionError, validate_app_runtime_db
logger = logging.getLogger(__name__)
@@ -48,6 +59,106 @@ def _run_scheduled_public_ip_check() -> None:
session.close()
def _run_scheduled_modbus_poll() -> None:
session_local = get_session_local()
session: Session = session_local()
try:
poll_all_enabled_devices(session, bootstrap_settings=get_settings())
finally:
session.close()
def _run_scheduled_tibber_refresh() -> None:
"""Scheduled job: fetch Tibber 15-minute prices and upsert into tibber_price.
Runs every hour so that:
- Today's prices are available from startup.
- Tomorrow's prices (published by Tibber around 13:00 CET / 11:00 UTC) are
picked up within an hour of publication without requiring a server restart.
The job is a no-op when:
- No active energy contract with kind="tibber" exists.
- The Tibber API token is empty in the runtime settings.
Any client exceptions (auth failures, network errors) are caught and logged
so that a single failed fetch does not crash the scheduler or affect the
other background jobs.
"""
session_local = get_session_local()
session: Session = session_local()
try:
runtime_settings = build_runtime_settings(session, get_settings())
refresh_prices(session, runtime_settings)
except Exception:
logger.exception("_run_scheduled_tibber_refresh: unexpected error")
finally:
session.close()
def _run_scheduled_energy_cost() -> None:
"""Scheduled job: compute billing records for all uncalculated closed 15-minute periods.
Runs every minute so that a new period is picked up within 1 minute of
closing. The job is a no-op when:
- No active energy contract with a version covering the period exists.
- DSMR data has not yet arrived for the period boundaries.
- The period's billing record already exists and is not degraded.
Any unexpected exceptions are caught and logged so that a single failure
does not crash the scheduler or affect the other background jobs.
"""
session_local = get_session_local()
session: Session = session_local()
try:
compute_closed_periods(session)
# After billing periods are computed, push fresh energy-cost state values
# to MQTT/HA. publish_states is internally guarded by _should_publish
# (MQTT disabled / not connected → no-op), so this never raises due to
# unconfigured MQTT and does not block the billing job.
try:
from app.services.ha_discovery import publish_states
publish_states(session)
except Exception:
logger.exception("_run_scheduled_energy_cost: publish_states failed (non-fatal)")
except Exception:
logger.exception("_run_scheduled_energy_cost: unexpected error")
finally:
session.close()
def _run_scheduled_ha_state_publish() -> None:
"""Periodic job: publish discovery configs + state + availability for all enabled exposed entities.
Runs every 60 seconds. When the MQTT broker is connected:
- Publishes (or re-publishes) all HA Discovery configs (retained, idempotent).
- Publishes the current state / availability for all enabled entities.
This also serves as the reliable "publish discovery after connect" mechanism:
because paho connects asynchronously, a synchronous call immediately after
``mqtt_manager.connect()`` would fire before the TCP handshake completes and
be a no-op. Instead, this periodic job picks it up within 60 seconds of the
broker becoming available — retained payloads make repeated publishes harmless.
Additionally, if MQTT is configured in DB but the manager is not yet connected
(e.g. MQTT was enabled via UI after startup), this job attempts to connect so
the user does not need to restart the server.
"""
session_local = get_session_local()
session: Session = session_local()
try:
runtime_settings = build_runtime_settings(session, get_settings())
# Reconnect if MQTT is configured (in DB) but not yet connected.
if mqtt_manager.is_configured(runtime_settings) and not mqtt_manager.is_connected:
logger.info("_run_scheduled_ha_state_publish: MQTT configured but not connected — attempting connect.")
mqtt_manager.connect(runtime_settings)
publish_discovery(session)
publish_states(session)
except Exception:
logger.exception("_run_scheduled_ha_state_publish: unexpected error")
finally:
session.close()
def ensure_auth_db_ready() -> None:
session_local = get_session_local()
session: Session = session_local()
@@ -83,8 +194,69 @@ async def lifespan(_: FastAPI):
max_instances=1,
coalesce=True,
)
scheduler.add_job(
_run_scheduled_modbus_poll,
trigger=IntervalTrigger(seconds=BASE_POLL_TICK_SECONDS),
id="modbus-poll",
replace_existing=True,
max_instances=1,
coalesce=True,
)
# Periodic HA state / availability publish (60-second fallback sweep).
scheduler.add_job(
_run_scheduled_ha_state_publish,
trigger=IntervalTrigger(seconds=60),
id="ha-state-publish",
replace_existing=True,
max_instances=1,
coalesce=True,
)
# Tibber price refresh: fetch today + tomorrow every hour.
# The job is a no-op when no active tibber contract or token is configured,
# so it is safe to register unconditionally.
scheduler.add_job(
_run_scheduled_tibber_refresh,
trigger=IntervalTrigger(hours=1),
id="tibber-refresh",
replace_existing=True,
max_instances=1,
coalesce=True,
)
# Energy cost billing: compute uncalculated closed 15-minute periods every minute.
# The job is a no-op when no active contract or DSMR data is present, so it is
# safe to register unconditionally.
scheduler.add_job(
_run_scheduled_energy_cost,
trigger=IntervalTrigger(minutes=1),
id="energy-cost",
replace_existing=True,
max_instances=1,
coalesce=True,
)
scheduler.start()
# MQTT: connect using DB-merged runtime settings so broker configured via UI
# is picked up on restart (not just from env/bootstrap settings).
# Discovery will be published by the first run of _run_scheduled_ha_state_publish
# (within 60 s of startup), after the async paho handshake completes.
_startup_session_local = get_session_local()
_startup_session: Session = _startup_session_local()
try:
_startup_runtime_settings = build_runtime_settings(_startup_session, get_settings())
finally:
_startup_session.close()
mqtt_manager.connect(_startup_runtime_settings)
# DSMR ingest: subscribe to the configured MQTT topic when enabled. The same
# applier is called from PUT /api/config, so toggling DSMR via the UI takes
# effect without an app restart.
apply_dsmr_subscription(_startup_runtime_settings)
yield
# MQTT: clean shutdown before the process exits.
mqtt_manager.disconnect()
scheduler.shutdown(wait=False)
@@ -107,6 +279,11 @@ def create_app() -> FastAPI:
app.include_router(status.router)
app.include_router(api_config_router)
app.include_router(api_data_router)
app.include_router(api_energy_router)
app.include_router(api_energy_contracts_router)
app.include_router(api_meters_router)
app.include_router(api_expose_router)
app.include_router(api_modbus_router)
app.include_router(api_session_router)
app.include_router(homeassistant_router)
app.include_router(location_router)
+22
View File
@@ -15,8 +15,12 @@ class AuthUser(Base):
is_active: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True)
force_password_change: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
# TOTP fields (Phase B, M4-T04) — nullable/false by default so existing users are unaffected
totp_secret: Mapped[str | None] = mapped_column(String(64), nullable=True)
totp_enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
sessions: Mapped[list["AuthSession"]] = relationship(back_populates="user")
recovery_codes: Mapped[list["RecoveryCode"]] = relationship(back_populates="user")
class AuthSession(Base):
@@ -31,3 +35,21 @@ class AuthSession(Base):
revoked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
user: Mapped[AuthUser] = relationship(back_populates="sessions")
class RecoveryCode(Base):
"""One-time TOTP recovery codes for AuthUser.
``code_hash`` stores the Argon2 hash of the plaintext code (plaintext is
returned only at setup time and never stored). ``used_at`` is NULL while
the code is still valid; set to the consumption timestamp when consumed.
"""
__tablename__ = "auth_recovery_code"
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
user_id: Mapped[int] = mapped_column(ForeignKey("auth_users.id"), nullable=False, index=True)
code_hash: Mapped[str] = mapped_column(String(255), nullable=False)
used_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
user: Mapped[AuthUser] = relationship(back_populates="recovery_codes")
+31
View File
@@ -0,0 +1,31 @@
from datetime import datetime
from sqlalchemy import DateTime, Index, Integer, String, UniqueConstraint
from sqlalchemy.orm import Mapped, mapped_column
from app.db import Base
class LoginThrottle(Base):
"""Tracks per-key login failure state for exponential back-off throttling.
``scope`` is either ``'ip'`` or ``'user'``; ``key`` is the IP address or
username string respectively. The pair ``(scope, key)`` is unique — one
row per tracked entity. A row is deleted on successful login (clear).
"""
__tablename__ = "auth_login_throttle"
__table_args__ = (
UniqueConstraint("scope", "key", name="uq_auth_login_throttle_scope_key"),
Index("ix_auth_login_throttle_scope_key", "scope", "key"),
)
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
key: Mapped[str] = mapped_column(String(255), nullable=False)
scope: Mapped[str] = mapped_column(String(16), nullable=False) # 'ip' | 'user'
failures: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
first_failed_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
last_failed_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
next_allowed_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True), nullable=True
)
+318
View File
@@ -0,0 +1,318 @@
"""SQLAlchemy models for the energy pricing and DSMR metering subsystem.
Six tables:
- meter: physical electricity meter lifecycle epoch.
- dsmr_reading: raw DSMR telegram blobs (10-second down-sampled).
- energy_contract: contract head (manual or tibber, one active at a time).
- energy_contract_version: versioned pricing values; append-only for auditability.
- tibber_price: cached Tibber 15-minute spot prices (immutable).
- energy_cost_period: computed 15-minute billing periods (immutable snapshot).
"""
from __future__ import annotations
import uuid as _uuid
from datetime import datetime
from sqlalchemy import Boolean, DateTime, Float, ForeignKey, Integer, String
from sqlalchemy.orm import Mapped, mapped_column, relationship
from sqlalchemy.types import JSON
from app.db import Base
def _uuid4_str() -> str:
return str(_uuid.uuid4())
class Meter(Base):
"""One physical electricity meter's installation epoch.
A ``meter`` record represents the period ``[started_at, ended_at)`` during
which a particular physical meter was installed and active. Replacing a meter
(swap, home move, etc.) is modelled by closing the current record
(``ended_at = swap_timestamp``) and opening a new one
(``started_at = swap_timestamp``).
**Invariant**: for each ``commodity`` there is at most one active meter
(``ended_at IS NULL``) at any point in time. The service layer enforces
this — no DB-level constraint is added to keep the migration simple and to
allow the application to return a meaningful error message.
``commodity`` defaults to ``"electricity"``; the field is a free-form string
(no CHECK constraint) so future commodities (``gas``, ``heating``) can be
added without a schema change.
``reason`` captures why this epoch started — one of ``initial``,
``meter_swap``, ``home_move``, or ``other`` — stored as a plain string so
the application layer controls the allowed set.
"""
__tablename__ = "meter"
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
# Stable internal identity — used as HA Discovery unique_id anchor.
uuid: Mapped[str] = mapped_column(
String(36), unique=True, nullable=False, default=_uuid4_str
)
# Human-readable label for this physical meter (e.g. address, serial, tariff zone).
label: Mapped[str] = mapped_column(String(255), nullable=False)
# Energy commodity this meter measures. Defaults to "electricity".
commodity: Mapped[str] = mapped_column(String(32), nullable=False, default="electricity")
# UTC timestamp when this meter epoch starts (inclusive). May be in the past
# (retroactive declaration); effective billing start = max(started_at, data start).
started_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
# UTC timestamp when this meter epoch ends (exclusive). NULL = currently active.
ended_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
# Why this epoch was created. Application-layer validation enforces the
# allowed set; no CHECK constraint to keep migrations simple.
reason: Mapped[str] = mapped_column(String(64), nullable=False)
# Free-form note (e.g. location, physical meter id, reason details).
note: Mapped[str | None] = mapped_column(String(1024), nullable=True)
# UTC timestamp of when this row was created.
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
# Relationship to cost periods attributed to this meter epoch (not loaded eagerly).
cost_periods: Mapped[list["EnergyCostPeriod"]] = relationship(
back_populates="meter", cascade="save-update, merge"
)
class DsmrReading(Base):
"""One down-sampled DSMR telegram stored as a full JSON blob.
Identity & idempotency are **independent of the DSMR Reader's telegram id**
(that field overflows and must be manually reset to zero — a known DSMR
quirk — so relying on it for uniqueness risks silently dropping new data).
The table's own autoincrement ``id`` PK is the stable internal identity, and
``recorded_at`` (the telegram timestamp) is the UNIQUE de-duplication key: a
single P1 meter emits exactly one telegram per timestamp.
``recorded_at`` is a real column (not inside the payload) so time-range
queries are efficient. The entire telegram frame is stored verbatim in
``payload``; no field allow-list is applied so future commodities (gas,
heating, three-phase) are accommodated without a schema change.
"""
__tablename__ = "dsmr_reading"
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
# UTC timestamp of the sample — real column, UNIQUE (telegram-id-independent
# idempotency key). The unique index also serves time-range queries.
recorded_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), nullable=False, unique=True
)
# Telegram's own id (DSMR Reader assigns it). Stored only as a reference /
# debugging aid — NOT used for uniqueness or idempotency (it overflows and
# gets reset to zero). Nullable because some DSMR sources may not emit one.
source_id: Mapped[int | None] = mapped_column(Integer, nullable=True)
# Full telegram frame as a JSON object; values are typically JSON strings
# (e.g. "20915.154") — callers must cast to Decimal before arithmetic.
payload: Mapped[dict] = mapped_column(JSON, nullable=False)
class EnergyContract(Base):
"""Contract head: a named energy contract with a chosen pricing strategy.
``kind`` determines which price strategy is used (``manual`` for fixed
dual-tariff rates entered by the user, ``tibber`` for dynamic API prices).
Only one contract may be ``active`` at a time; the service layer enforces
mutual exclusion. Specific pricing values live in ``EnergyContractVersion``
so that price changes can be tracked without modifying historical records.
"""
__tablename__ = "energy_contract"
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
# Human-readable label; freely editable by the user.
name: Mapped[str] = mapped_column(String(255), nullable=False)
# Strategy selector: "manual" or "tibber". Application-layer validation
# enforces the allowed set; no DB CHECK constraint is added to keep the
# migration simple and the strategy registry extensible.
kind: Mapped[str] = mapped_column(String(32), nullable=False)
# Whether this is the currently active contract (at most one should be True).
active: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
# ISO 4217 currency code for all monetary values in this contract.
currency: Mapped[str] = mapped_column(String(8), nullable=False, default="EUR")
# Audit timestamps.
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
# Relationship to versions (back-reference; not loaded eagerly).
versions: Mapped[list["EnergyContractVersion"]] = relationship(
back_populates="contract", cascade="save-update, merge"
)
class EnergyContractVersion(Base):
"""One time-bounded version of an energy contract's pricing values.
Pricing changes are modelled as new versions (append-only); existing versions
are never modified so that historical ``EnergyCostPeriod`` records remain
fully auditable. ``effective_to`` is ``NULL`` for the currently open version.
"""
__tablename__ = "energy_contract_version"
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
# FK to the parent contract. RESTRICT prevents deletion of a contract that
# still has versioned pricing rows attached to it.
contract_id: Mapped[int] = mapped_column(
ForeignKey("energy_contract.id", ondelete="RESTRICT"), nullable=False
)
# Start of this version's validity window (inclusive, UTC).
effective_from: Mapped[datetime] = mapped_column(
DateTime(timezone=True), nullable=False
)
# End of this version's validity window (exclusive, UTC). NULL means open-ended
# (i.e. this is the most recent / current version).
effective_to: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True), nullable=True
)
# Pricing values as a JSON object conforming to the profile structure for
# ``contract.kind`` (validated by the application layer against the YAML profile).
values: Mapped[dict] = mapped_column(JSON, nullable=False)
# Creation timestamp.
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
# Relationship back to the parent contract.
contract: Mapped["EnergyContract"] = relationship(back_populates="versions")
# Relationship to cost periods that reference this version.
cost_periods: Mapped[list["EnergyCostPeriod"]] = relationship(
back_populates="contract_version", cascade="save-update, merge"
)
class TibberPrice(Base):
"""Cached Tibber 15-minute spot price point (immutable once fetched).
``starts_at`` is unique so that upserts are idempotent. Past prices are
never overwritten; the fetch job only adds rows for future time slots.
"""
__tablename__ = "tibber_price"
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
# UTC start of the 15-minute slot; unique so upsert is idempotent.
starts_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), nullable=False, unique=True
)
# Resolution label as returned by the Tibber API (e.g. "QUARTER_HOURLY").
resolution: Mapped[str] = mapped_column(String(32), nullable=False)
# Price components in the contract currency (all include VAT, user-facing).
energy: Mapped[float] = mapped_column(Float, nullable=False)
tax: Mapped[float] = mapped_column(Float, nullable=False)
total: Mapped[float] = mapped_column(Float, nullable=False)
# Tibber price level (e.g. "NORMAL", "CHEAP", "EXPENSIVE"); may be absent.
level: Mapped[str | None] = mapped_column(String(32), nullable=True)
# ISO 4217 currency code as returned by the API.
currency: Mapped[str] = mapped_column(String(8), nullable=False)
# UTC timestamp of when this row was fetched/inserted.
fetched_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
class EnergyCostPeriod(Base):
"""Computed billing record for one 15-minute metering period (immutable snapshot).
Each row captures the per-register kWh deltas, the resulting import cost and
export revenue, and a full snapshot of the pricing values used so that the
calculation is fully auditable and reproducible without re-querying the
contract version. Rows are written once and never modified; explicit
recomputation via the API is the only way to overwrite a period.
"""
__tablename__ = "energy_cost_period"
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
# UTC start of the 15-minute period; unique so upsert is idempotent.
period_start: Mapped[datetime] = mapped_column(
DateTime(timezone=True), nullable=False, unique=True
)
# Per-register kWh deltas for the period (end minus start of cumulative registers).
# _1 = dal/low-tariff, _2 = normal/high-tariff (NL convention).
d1_kwh: Mapped[float] = mapped_column(Float, nullable=False) # delivered low
d2_kwh: Mapped[float] = mapped_column(Float, nullable=False) # delivered high
r1_kwh: Mapped[float] = mapped_column(Float, nullable=False) # returned low
r2_kwh: Mapped[float] = mapped_column(Float, nullable=False) # returned high
# Computed monetary amounts for the period (in ``currency``).
import_cost: Mapped[float] = mapped_column(Float, nullable=False)
export_revenue: Mapped[float] = mapped_column(Float, nullable=False)
net_cost: Mapped[float] = mapped_column(Float, nullable=False)
# ISO 4217 currency code matching the contract.
currency: Mapped[str] = mapped_column(String(8), nullable=False)
# Full snapshot of the pricing inputs used during computation. This makes
# each row self-contained and auditable even if the contract is later changed.
pricing: Mapped[dict] = mapped_column(JSON, nullable=False)
# FK to the exact contract version whose values were used. RESTRICT prevents
# deletion of a version that has cost records attached. Nullable to support
# periods computed in ``degraded`` mode (missing price data).
contract_version_id: Mapped[int | None] = mapped_column(
ForeignKey("energy_contract_version.id", ondelete="RESTRICT"), nullable=True
)
# FK to the meter epoch this period belongs to. RESTRICT prevents deletion of
# a meter that still has attributed cost periods. Nullable for backwards
# compatibility (pre-M7 rows) and degraded periods where the meter was not
# determinable.
meter_id: Mapped[int | None] = mapped_column(
ForeignKey("meter.id", ondelete="RESTRICT"), nullable=True
)
# True when the period was computed with incomplete data (missing readings or
# missing price); serves as a flag for later recomputation.
degraded: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
# UTC timestamp of when this row was computed/inserted.
computed_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
# Relationship back to the contract version.
contract_version: Mapped["EnergyContractVersion | None"] = relationship(
back_populates="cost_periods"
)
# Relationship back to the meter epoch.
meter: Mapped["Meter | None"] = relationship(back_populates="cost_periods")
# Index on recorded_at for efficient time-range queries on DSMR readings.
# (The ORM-level index=True on recorded_at already creates ix_dsmr_reading_recorded_at;
# no composite index is needed for single-meter deployments.)
# Index on period_start is covered by the unique constraint (SQLite creates an
# implicit index for UNIQUE columns), so no additional index is required.
# Index on starts_at for TibberPrice is covered by the unique constraint similarly.
+49
View File
@@ -0,0 +1,49 @@
"""SQLAlchemy model for the exposed-entity toggle table.
``ExposedEntityToggle`` stores the per-entity enable/disable state for the
MQTT / HA Discovery expose framework. The *catalog* of what *can* be
exposed is computed dynamically by provider functions (see
``app/integrations/expose.py``); this table only records which entries the
user has explicitly enabled.
Key design decisions
--------------------
- ``key`` is a stable string identifier derived from device uuid + metric key
(e.g. ``"modbus.<uuid>.voltage"``), so it does not drift if rows are
deleted and re-inserted.
- Default state for an entity not yet in this table is **disabled** (false).
``build_catalog`` in expose.py treats a missing row as ``enabled=False``.
- Only one row per entity key (``key`` is UNIQUE).
"""
from __future__ import annotations
from datetime import datetime
from sqlalchemy import Boolean, DateTime, Integer, String
from sqlalchemy.orm import Mapped, mapped_column
from app.db import Base
class ExposedEntityToggle(Base):
"""Per-entity on/off switch for MQTT / HA Discovery publishing.
Rows are created on demand (first toggle); entities with no row are
treated as disabled.
"""
__tablename__ = "exposed_entity_toggle"
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
# Stable entity key, e.g. "modbus.<uuid>.voltage" or "modbus.<uuid>.online".
key: Mapped[str] = mapped_column(String(255), unique=True, nullable=False)
# Whether this entity should be published via MQTT / HA Discovery.
enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
# Last time this row was created or modified.
updated_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), nullable=False
)
+114
View File
@@ -0,0 +1,114 @@
"""SQLAlchemy models for Modbus device management and telemetry.
Two tables:
- modbus_device: deployment/configurable metadata for each polled device.
- modbus_reading: generic telemetry rows (one per device per poll cycle).
"""
from __future__ import annotations
import uuid as _uuid
from datetime import datetime
from sqlalchemy import Boolean, DateTime, ForeignKey, Index, Integer, String
from sqlalchemy.orm import Mapped, mapped_column, relationship
from sqlalchemy.types import JSON
from app.db import Base
def _uuid4_str() -> str:
return str(_uuid.uuid4())
class ModbusDevice(Base):
"""Deployment-layer record for a Modbus slave device reachable via a TCP gateway.
Protocol knowledge (register map, decoding rules) lives in the YAML profile
referenced by ``profile``. Per-device variables (network address, slave ID,
display name, poll rate) live here.
"""
__tablename__ = "modbus_device"
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
# Stable internal identity — used as the API path key and HA Discovery unique_id anchor.
uuid: Mapped[str] = mapped_column(
String(36), unique=True, nullable=False, default=_uuid4_str
)
# Human-readable label (may be changed; changing it triggers re-publish of HA Discovery).
friendly_name: Mapped[str] = mapped_column(String(255), nullable=False)
# Transport protocol — only "tcp" is supported for now.
transport: Mapped[str] = mapped_column(String(16), nullable=False, default="tcp")
# TCP gateway address.
host: Mapped[str] = mapped_column(String(255), nullable=False)
port: Mapped[int] = mapped_column(Integer, nullable=False, default=502)
# Modbus slave address (set on the device panel; different meters on the same gateway
# must have distinct unit_ids).
unit_id: Mapped[int] = mapped_column(Integer, nullable=False, default=1)
# Which YAML profile to use for register mapping and decoding (e.g. "sdm120").
profile: Mapped[str] = mapped_column(String(64), nullable=False)
# Polling interval in seconds.
poll_interval_s: Mapped[int] = mapped_column(Integer, nullable=False, default=5)
# Whether this device is included in the periodic poll sweep.
enabled: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True)
# Poll-status fields (updated by the poll service after each attempt).
last_poll_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True), nullable=True
)
last_poll_ok: Mapped[bool | None] = mapped_column(Boolean, nullable=True)
# Audit timestamps.
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), nullable=False
)
updated_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), nullable=False
)
# Relationship to readings (back-reference; not loaded eagerly).
readings: Mapped[list["ModbusReading"]] = relationship(
back_populates="device", cascade="save-update, merge"
)
class ModbusReading(Base):
"""Generic telemetry row: one device, one poll instant, all decoded metrics as JSON.
``payload`` contains the full dict of engineering values produced by the YAML profile
decoder, e.g. ``{"voltage": 230.2, "current": 1.3, ...}``. The profile is the key
to interpreting which keys exist and what units they carry.
"""
__tablename__ = "modbus_reading"
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
# FK to the device that produced this reading.
# ON DELETE RESTRICT: prevents accidental deletion of a device that has historical data.
device_id: Mapped[int] = mapped_column(
ForeignKey("modbus_device.id", ondelete="RESTRICT"), nullable=False
)
# The UTC timestamp of when the sample was taken — real indexed column, NOT embedded in payload.
recorded_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), nullable=False, index=True
)
# Profile-decoded engineering values as a JSON object.
payload: Mapped[dict] = mapped_column(JSON, nullable=False)
device: Mapped["ModbusDevice"] = relationship(back_populates="readings")
# Composite index for efficient time-range queries scoped to a single device.
Index("ix_modbus_reading_device_recorded", ModbusReading.device_id, ModbusReading.recorded_at)
+7
View File
@@ -38,3 +38,10 @@ class SmtpTestResponse(BaseModel):
result: Literal["success", "config-error", "failed"]
message: str
class MqttTestResponse(BaseModel):
"""Response from POST /api/config/mqtt/test."""
result: Literal["success", "config-error", "failed"]
message: str
+216
View File
@@ -0,0 +1,216 @@
"""Pydantic schemas for the Energy data API (M6-T09).
Covers six endpoint groups under /api/energy:
- GET /prices — price curve (tibber 15min points or manual tariff)
- GET /costs — energy_cost_period rows (time-range, paginated)
- GET /costs/summary — aggregated metered + standing charges credits
- GET /dsmr/latest — most recent dsmr_reading blob
- POST /costs/recompute — explicit idempotent recompute (returns count)
- POST /tibber/test — three-state Tibber connection test
"""
from __future__ import annotations
from datetime import datetime
from typing import Any, Literal
from pydantic import BaseModel, Field
# ---------------------------------------------------------------------------
# GET /api/energy/prices
# ---------------------------------------------------------------------------
class PricePointSchema(BaseModel):
"""A single 15-minute price point (tibber) or placeholder entry."""
starts_at: datetime
buy: float = Field(description="All-in buy price in EUR/kWh (including taxes).")
sell: float = Field(description="Net sell price in EUR/kWh.")
level: str | None = Field(
default=None,
description="Tibber price level (CHEAP / NORMAL / EXPENSIVE); null for manual.",
)
class ManualTariffSchema(BaseModel):
"""Fixed-tariff breakdown for manual contracts.
Prices are the *effective* buy prices as used by the billing engine
(energy_buy_x + energy_tax + ode) and the raw sell prices.
"""
buy_dal: float = Field(description="Effective buy price, low-tariff / dal (EUR/kWh).")
buy_normal: float = Field(description="Effective buy price, normal / high-tariff (EUR/kWh).")
sell_dal: float = Field(description="Sell price, low-tariff / dal (EUR/kWh).")
sell_normal: float = Field(description="Sell price, normal / high-tariff (EUR/kWh).")
class PricesResponse(BaseModel):
"""Response for GET /api/energy/prices.
``kind`` mirrors the active contract kind:
- ``"tibber"`` → ``points`` has actual 15-min price entries; ``tariff`` is null.
- ``"manual"`` → ``points`` is empty; ``tariff`` carries the fixed-rate table.
- ``None`` → no active contract; both ``points`` and ``tariff`` are empty/null.
``currency`` comes from the active contract (or "EUR" fallback).
``points`` is always ascending by ``starts_at``.
"""
kind: str | None = Field(
description="Active contract kind ('tibber' or 'manual'), or null if no active contract."
)
currency: str = Field(description="ISO 4217 currency code.")
points: list[PricePointSchema] = Field(
description=(
"15-minute price points for tibber contracts (ascending by starts_at). "
"Empty for manual contracts or when no active contract exists."
)
)
tariff: ManualTariffSchema | None = Field(
default=None,
description=(
"Fixed tariff table for manual contracts. "
"Null for tibber contracts and when no active contract exists."
),
)
# ---------------------------------------------------------------------------
# GET /api/energy/costs
# ---------------------------------------------------------------------------
class CostPeriodSchema(BaseModel):
"""One 15-minute billing record from the energy_cost_period table."""
period_start: datetime
d1_kwh: float = Field(description="Delivered low-tariff kWh for this period.")
d2_kwh: float = Field(description="Delivered normal-tariff kWh for this period.")
r1_kwh: float = Field(description="Returned low-tariff kWh for this period.")
r2_kwh: float = Field(description="Returned normal-tariff kWh for this period.")
import_cost: float = Field(description="Cost of electricity drawn from grid (EUR).")
export_revenue: float = Field(description="Revenue from electricity fed to grid (EUR).")
net_cost: float = Field(description="import_cost export_revenue (EUR).")
currency: str = Field(description="ISO 4217 currency code.")
degraded: bool = Field(
description="True when the period was computed with incomplete data."
)
contract_version_id: int | None = Field(
default=None,
description="FK to the contract version used for this billing period (null when degraded).",
)
model_config = {"from_attributes": True}
class CostsResponse(BaseModel):
"""Response for GET /api/energy/costs."""
items: list[CostPeriodSchema]
total: int = Field(description="Number of items returned.")
# ---------------------------------------------------------------------------
# GET /api/energy/costs/summary
# ---------------------------------------------------------------------------
class SummaryResponse(BaseModel):
"""Response for GET /api/energy/costs/summary.
All monetary values are in ``currency``.
``total_payable = metered_net + fixed_costs credits``
"""
currency: str
metered_import: float = Field(description="Σ import_cost for non-degraded periods.")
metered_export: float = Field(description="Σ export_revenue for non-degraded periods.")
metered_net: float = Field(description="Σ net_cost for non-degraded periods.")
fixed_costs: float = Field(
description="Standing charges (network_fee + management_fee) apportioned over the interval."
)
credits: float = Field(
description="Energy-tax credit (heffingskorting) apportioned over the interval."
)
total_payable: float = Field(
description="metered_net + fixed_costs credits (actual amount owed)."
)
period_count: int = Field(description="Number of non-degraded billing periods in range.")
degraded_count: int = Field(description="Number of degraded billing periods in range.")
days: float = Field(description="Interval length in days.")
# ---------------------------------------------------------------------------
# GET /api/energy/dsmr/latest
# ---------------------------------------------------------------------------
class DsmrLatestResponse(BaseModel):
"""Response for GET /api/energy/dsmr/latest.
``found`` is False when no dsmr_reading rows exist yet. The front-end
should check ``found`` before reading ``recorded_at`` or ``payload``.
"""
found: bool
recorded_at: datetime | None = None
payload: dict[str, Any] | None = None
# ---------------------------------------------------------------------------
# POST /api/energy/costs/recompute
# ---------------------------------------------------------------------------
class RecomputeResponse(BaseModel):
"""Response for POST /api/energy/costs/recompute."""
recomputed: int = Field(
description=(
"Number of 15-minute periods for which a billing record was written "
"(inserted or updated). Periods skipped due to missing contract or "
"missing Tibber price are not counted."
)
)
# ---------------------------------------------------------------------------
# POST /api/energy/tibber/test
# ---------------------------------------------------------------------------
class TibberTestPriceSchema(BaseModel):
"""Current Tibber price point returned on a successful test.
Carries enough fields for the front-end to confirm the API is working and
display the live price. The API token is **never** included.
"""
starts_at: datetime
total: float
energy: float
tax: float
currency: str
level: str | None = None
class TibberTestResponse(BaseModel):
"""Three-state response for POST /api/energy/tibber/test.
Possible ``result`` values:
- ``"success"`` — Tibber API responded with a valid price.
- ``"config-error"`` — Token is missing or not configured.
- ``"failed"`` — API call failed (auth rejected, network error, timeout, etc.).
``price`` is populated only on ``"success"``; it is null otherwise.
``message`` always contains a human-readable explanation.
"""
result: Literal["success", "config-error", "failed"]
message: str
price: TibberTestPriceSchema | None = None
+148
View File
@@ -0,0 +1,148 @@
"""Pydantic schemas for the EnergyContract CRUD + versioning API (M6-T04).
Schema hierarchy
----------------
ContractVersionResponse — single version row (id, dates, values, created_at)
ContractResponse — contract head (id, name, kind, active, currency, timestamps)
ContractDetailResponse — contract head + embedded versions list (for GET{id})
ContractListResponse — paginated list of ContractResponse items
ContractCreate — POST /api/energy/contracts body
ContractPatch — PATCH /api/energy/contracts/{id} body (all fields optional)
VersionCreate — POST /api/energy/contracts/{id}/versions body
ProfilesResponse — GET /api/energy/profiles response
"""
from __future__ import annotations
from datetime import datetime
from typing import Any
from pydantic import BaseModel, Field
# ---------------------------------------------------------------------------
# Version schemas
# ---------------------------------------------------------------------------
class ContractVersionResponse(BaseModel):
"""Response schema for a single EnergyContractVersion row."""
id: int
effective_from: datetime
effective_to: datetime | None
values: dict[str, Any]
created_at: datetime
model_config = {"from_attributes": True}
# ---------------------------------------------------------------------------
# Contract schemas
# ---------------------------------------------------------------------------
class ContractResponse(BaseModel):
"""Response schema for a single EnergyContract (without embedded versions).
Used for list responses where embedding all versions would be expensive.
"""
id: int
name: str
kind: str
active: bool
currency: str
created_at: datetime
updated_at: datetime
model_config = {"from_attributes": True}
class ContractDetailResponse(BaseModel):
"""Response schema for a single EnergyContract with full version history.
Returned by GET /api/energy/contracts/{id} and by successful POST / PATCH
operations where the caller needs to see all version data.
"""
id: int
name: str
kind: str
active: bool
currency: str
created_at: datetime
updated_at: datetime
versions: list[ContractVersionResponse]
model_config = {"from_attributes": True}
class ContractListResponse(BaseModel):
"""Response schema for GET /api/energy/contracts."""
items: list[ContractResponse]
total: int
# ---------------------------------------------------------------------------
# Request schemas
# ---------------------------------------------------------------------------
class ContractCreate(BaseModel):
"""Request body for POST /api/energy/contracts.
``effective_from`` defaults to the current UTC time if not provided,
giving the first version an open-ended start from "now".
``kind`` is validated at the application layer against the profile registry;
clients should send ``"manual"`` or ``"tibber"``.
"""
name: str = Field(..., min_length=1, max_length=255)
kind: str = Field(..., min_length=1, max_length=32)
currency: str = Field(default="EUR", min_length=1, max_length=8)
values: dict[str, Any]
effective_from: datetime | None = Field(
default=None,
description=(
"UTC datetime from which the first pricing version is effective. "
"Defaults to the current UTC time when omitted."
),
)
class ContractPatch(BaseModel):
"""Request body for PATCH /api/energy/contracts/{id}.
All fields are optional. Sending ``active=true`` activates this contract
(deactivating all others); ``active=false`` deactivates it without affecting
other contracts.
"""
name: str | None = Field(default=None, min_length=1, max_length=255)
active: bool | None = None
class VersionCreate(BaseModel):
"""Request body for POST /api/energy/contracts/{id}/versions."""
effective_from: datetime
values: dict[str, Any]
# ---------------------------------------------------------------------------
# Profile response schemas
# ---------------------------------------------------------------------------
class ProfilesResponse(BaseModel):
"""Response schema for GET /api/energy/profiles.
``profiles`` is a list of raw profile dicts as produced by
``list_profiles()`` (Pydantic model dumps). Each entry contains at minimum
``kind`` and ``label``; the full nested structure allows the front-end to
render a type-appropriate form for each pricing profile.
"""
profiles: list[dict[str, Any]]
+95
View File
@@ -0,0 +1,95 @@
"""Pydantic schemas for the Expose API (M5-T12).
Three endpoints:
GET /api/expose — catalog + toggle state + MQTT/Discovery status
PUT /api/expose — set toggles (map key → bool)
POST /api/expose/republish — trigger discovery re-publish
"""
from __future__ import annotations
from pydantic import BaseModel
# ---------------------------------------------------------------------------
# Nested schemas
# ---------------------------------------------------------------------------
class DeviceInfoSchema(BaseModel):
"""HA device grouping info for an exposable entity."""
identifiers: list[str]
name: str
class ExposableEntitySchema(BaseModel):
"""One exposable entity in the catalog.
``value_getter`` is intentionally excluded — it is a non-serialisable
callable and is only used internally by the HA Discovery service.
"""
key: str
component: str
device: DeviceInfoSchema
device_class: str | None
unit: str
name: str
state_class: str | None = None
class CatalogEntrySchema(BaseModel):
"""An entity from the catalog with its current toggle state."""
entity: ExposableEntitySchema
enabled: bool
class MqttStatusSchema(BaseModel):
"""Connection status for MQTT and HA Discovery."""
mqtt_configured: bool
mqtt_connected: bool
discovery_enabled: bool
# ---------------------------------------------------------------------------
# Response schemas
# ---------------------------------------------------------------------------
class ExposeResponse(BaseModel):
"""Response for GET /api/expose."""
catalog: list[CatalogEntrySchema]
mqtt_status: MqttStatusSchema
class ExposeUpdateResponse(BaseModel):
"""Response for PUT /api/expose (returns updated catalog + status)."""
catalog: list[CatalogEntrySchema]
mqtt_status: MqttStatusSchema
class RepublishResponse(BaseModel):
"""Response for POST /api/expose/republish."""
ok: bool
message: str
# ---------------------------------------------------------------------------
# Request schemas
# ---------------------------------------------------------------------------
class ExposeUpdateRequest(BaseModel):
"""Request body for PUT /api/expose.
``toggles`` is a map from entity key to desired enabled state (bool).
Only keys present in the map are updated; absent keys are untouched.
"""
toggles: dict[str, bool]
+131
View File
@@ -0,0 +1,131 @@
"""Pydantic schemas for the Meter CRUD + swap declaration API (M7-T05).
Schema hierarchy
----------------
MeterResponse — single meter row (id/label/commodity/started_at/ended_at/reason/note/created_at)
MeterListResponse — ordered list of MeterResponse items
MeterDeclareRequest — POST /api/energy/meters body (declare a swap or initial meter)
MeterPatchRequest — PATCH /api/energy/meters/{id} body (all fields optional)
"""
from __future__ import annotations
from datetime import datetime
from enum import Enum
from pydantic import BaseModel, Field
# ---------------------------------------------------------------------------
# Enums
# ---------------------------------------------------------------------------
class MeterReason(str, Enum):
"""Allowed values for the meter epoch creation reason."""
initial = "initial"
meter_swap = "meter_swap"
home_move = "home_move"
other = "other"
# ---------------------------------------------------------------------------
# Response schemas
# ---------------------------------------------------------------------------
class MeterResponse(BaseModel):
"""Response schema for a single Meter epoch row.
``ended_at`` is ``null`` for the currently active meter.
"""
id: int
label: str
commodity: str
started_at: datetime
ended_at: datetime | None
reason: str
note: str | None
created_at: datetime
model_config = {"from_attributes": True}
class MeterListResponse(BaseModel):
"""Response schema for GET /api/energy/meters.
Meters are returned in ascending ``started_at`` order so the caller sees
the historical installation sequence.
"""
items: list[MeterResponse]
total: int
# ---------------------------------------------------------------------------
# Request schemas
# ---------------------------------------------------------------------------
_VALID_REASONS = ", ".join(r.value for r in MeterReason)
class MeterDeclareRequest(BaseModel):
"""Request body for POST /api/energy/meters.
Declares a new meter epoch (swap, home move, or initial declaration). The
service layer closes the current active meter for the given commodity at
``started_at`` and opens a new one.
``started_at`` follows the Principle-A localisation convention: a
timezone-naive value is interpreted as the **server's local wall-clock time**
(e.g. CEST midnight → stored as UTC the night before); a timezone-aware
value is converted to UTC as-is. Omitting ``started_at`` is not allowed —
every meter declaration must carry an explicit start timestamp.
``commodity`` defaults to ``"electricity"``; the field is available for
future use with ``gas`` or ``heating``.
"""
label: str = Field(..., min_length=1, max_length=255)
started_at: datetime = Field(
...,
description=(
"UTC (or server-local naive) datetime from which this meter epoch starts. "
"May be in the past (retroactive declaration)."
),
)
reason: MeterReason = Field(
...,
description=f"Why this epoch was created. One of: {_VALID_REASONS}.",
)
note: str | None = Field(default=None, max_length=1024)
commodity: str = Field(
default="electricity",
min_length=1,
max_length=32,
description="Energy commodity this meter measures. Defaults to 'electricity'.",
)
class MeterPatchRequest(BaseModel):
"""Request body for PATCH /api/energy/meters/{id}.
All fields are optional. Only non-``None`` values are applied.
Updating ``started_at`` is a **retroactive correction**: the service layer
maintains timeline continuity (adjusting the preceding meter's ``ended_at``)
and the API layer triggers ``recompute_range`` over the affected window so
that billing attribution is re-judged.
"""
label: str | None = Field(default=None, min_length=1, max_length=255)
note: str | None = Field(default=None, max_length=1024)
started_at: datetime | None = Field(
default=None,
description=(
"Retroactive correction of the meter epoch start timestamp. "
"Triggers billing recompute over the affected window."
),
)
+171
View File
@@ -0,0 +1,171 @@
"""Pydantic schemas for the Modbus device CRUD + readings + metrics API (M5-T05)."""
from __future__ import annotations
from datetime import datetime
from typing import Any
from pydantic import BaseModel, Field
# ---------------------------------------------------------------------------
# Device schemas
# ---------------------------------------------------------------------------
class ModbusDeviceCreate(BaseModel):
"""Request body for POST /api/modbus/devices."""
friendly_name: str = Field(..., min_length=1, max_length=255)
transport: str = Field(default="tcp", max_length=16)
host: str = Field(..., min_length=1, max_length=255)
port: int = Field(default=502, ge=1, le=65535)
unit_id: int = Field(default=1, ge=0, le=247)
profile: str = Field(..., min_length=1, max_length=64)
poll_interval_s: int = Field(default=5, ge=1, le=3600)
enabled: bool = Field(default=True)
class ModbusDeviceUpdate(BaseModel):
"""Request body for PATCH /api/modbus/devices/{uuid} — all fields optional."""
friendly_name: str | None = Field(default=None, min_length=1, max_length=255)
transport: str | None = Field(default=None, max_length=16)
host: str | None = Field(default=None, min_length=1, max_length=255)
port: int | None = Field(default=None, ge=1, le=65535)
unit_id: int | None = Field(default=None, ge=0, le=247)
profile: str | None = Field(default=None, min_length=1, max_length=64)
poll_interval_s: int | None = Field(default=None, ge=1, le=3600)
enabled: bool | None = None
class ModbusDeviceResponse(BaseModel):
"""Response schema for a single Modbus device."""
uuid: str
friendly_name: str
transport: str
host: str
port: int
unit_id: int
profile: str
poll_interval_s: int
enabled: bool
last_poll_at: datetime | None
last_poll_ok: bool | None
created_at: datetime
updated_at: datetime
model_config = {"from_attributes": True}
class ModbusDeviceListResponse(BaseModel):
"""Response schema for listing Modbus devices."""
items: list[ModbusDeviceResponse]
total: int
# ---------------------------------------------------------------------------
# Reading schemas
# ---------------------------------------------------------------------------
class ModbusReadingResponse(BaseModel):
"""A single reading row: timestamp + decoded payload."""
recorded_at: datetime
payload: dict[str, Any]
model_config = {"from_attributes": True}
class ModbusReadingsResponse(BaseModel):
"""Response schema for the readings time-range endpoint."""
items: list[ModbusReadingResponse]
class ModbusLatestResponse(BaseModel):
"""Response for the /latest endpoint.
``found`` is False and ``recorded_at``/``payload`` are None when the device
has no readings yet. Callers should check ``found`` before using the values.
"""
found: bool
recorded_at: datetime | None
payload: dict[str, Any] | None
# ---------------------------------------------------------------------------
# Metrics / profile schemas
# ---------------------------------------------------------------------------
class MetricInfo(BaseModel):
"""Metadata for a single measurable quantity in a device's profile."""
key: str
label: str
unit: str
device_class: str
class ModbusMetricsResponse(BaseModel):
"""Response schema for GET /api/modbus/devices/{uuid}/metrics."""
profile: str
metrics: list[MetricInfo]
# ---------------------------------------------------------------------------
# Profile list schema
# ---------------------------------------------------------------------------
class ProfileSummary(BaseModel):
"""One entry in the GET /api/modbus/profiles response."""
name: str
description: str
class ModbusProfilesResponse(BaseModel):
"""Response schema for GET /api/modbus/profiles."""
profiles: list[ProfileSummary]
# ---------------------------------------------------------------------------
# Test-read schema
# ---------------------------------------------------------------------------
class ModbusTestReadResponse(BaseModel):
"""Response for POST /api/modbus/devices/{uuid}/test.
On success ``ok=True`` and ``payload`` contains the decoded values.
On failure ``ok=False`` and ``error`` describes the problem.
"""
ok: bool
payload: dict[str, Any] | None = None
error: str | None = None
# ---------------------------------------------------------------------------
# Cascade-delete schema
# ---------------------------------------------------------------------------
class ModbusDeleteResponse(BaseModel):
"""Response for DELETE /api/modbus/devices/{uuid}?cascade=true.
Returned only when cascade deletion succeeds (HTTP 200). Non-cascade
successful deletes continue to return HTTP 204 (no body).
"""
deleted: bool
readings_deleted: int
toggles_deleted: int
+1
View File
@@ -16,6 +16,7 @@ class SessionResponse(BaseModel):
class LoginRequest(BaseModel):
username: str
password: str
totp_code: str | None = None
class PasswordChangeRequest(BaseModel):
+59
View File
@@ -0,0 +1,59 @@
"""Pydantic schemas for TOTP setup / enable / disable / status endpoints (M4-T05)."""
from __future__ import annotations
from pydantic import BaseModel
# ---------------------------------------------------------------------------
# Setup (POST /api/auth/totp/setup) — response
# ---------------------------------------------------------------------------
class TotpSetupResponse(BaseModel):
"""Returned once after a setup call.
``secret`` and ``recovery_codes`` are **one-time plaintext values**.
They are never returned again by any subsequent API call.
The frontend must display and instruct the user to save them before confirming.
"""
secret: str
otpauth_uri: str
recovery_codes: list[str]
# ---------------------------------------------------------------------------
# Enable (POST /api/auth/totp/enable) — request
# ---------------------------------------------------------------------------
class TotpEnableRequest(BaseModel):
"""The user confirms setup by providing the 6-digit TOTP code."""
code: str
# ---------------------------------------------------------------------------
# Disable (POST /api/auth/totp/disable) — request
# ---------------------------------------------------------------------------
class TotpDisableRequest(BaseModel):
"""Disable TOTP by proving identity.
Exactly one of ``password`` or ``code`` must be provided.
"""
password: str | None = None
code: str | None = None
# ---------------------------------------------------------------------------
# Status (GET /api/auth/totp) — response
# ---------------------------------------------------------------------------
class TotpStatusResponse(BaseModel):
"""Minimal status response — never exposes secret or recovery codes."""
enabled: bool
+83 -3
View File
@@ -25,9 +25,9 @@ class ConfigField:
CONFIG_FIELDS: tuple[ConfigField, ...] = (
ConfigField("System", "APP_NAME", "app_name", "App Name"),
ConfigField("System", "APP_ENV", "app_env", "App Env"),
ConfigField("System", "APP_DEBUG", "app_debug", "App Debug"),
ConfigField("System", "APP_DEBUG", "app_debug", "App Debug", input_type="checkbox"),
ConfigField("System", "APP_HOSTNAME", "app_hostname", "App Hostname"),
ConfigField("SMTP", "SMTP_ENABLED", "smtp_enabled", "SMTP Enabled"),
ConfigField("SMTP", "SMTP_ENABLED", "smtp_enabled", "SMTP Enabled", input_type="checkbox"),
ConfigField("SMTP", "SMTP_HOST", "smtp_host", "SMTP Host"),
ConfigField("SMTP", "SMTP_PORT", "smtp_port", "SMTP Port"),
ConfigField("SMTP", "SMTP_USERNAME", "smtp_username", "SMTP Username"),
@@ -35,7 +35,7 @@ CONFIG_FIELDS: tuple[ConfigField, ...] = (
ConfigField("SMTP", "SMTP_FROM_NAME", "smtp_from_name", "SMTP From Name"),
ConfigField("SMTP", "SMTP_FROM_ADDRESS", "smtp_from_address", "SMTP From Address"),
ConfigField("SMTP", "SMTP_TO_ADDRESS", "smtp_to_address", "SMTP To Address"),
ConfigField("SMTP", "SMTP_USE_STARTTLS", "smtp_use_starttls", "SMTP Use STARTTLS"),
ConfigField("SMTP", "SMTP_USE_STARTTLS", "smtp_use_starttls", "SMTP Use STARTTLS", input_type="checkbox"),
ConfigField(
"Authentication",
"AUTH_SESSION_COOKIE_NAME",
@@ -49,6 +49,13 @@ CONFIG_FIELDS: tuple[ConfigField, ...] = (
"auth_cookie_secure_override",
"Cookie Secure Override",
),
ConfigField(
"Authentication",
"AUTH_LOGIN_THROTTLE_ENABLED",
"auth_login_throttle_enabled",
"Login Throttle Enabled",
input_type="checkbox",
),
ConfigField("Poo", "POO_WEBHOOK_ID", "poo_webhook_id", "Poo Webhook ID", secret=True),
ConfigField(
"Poo",
@@ -96,6 +103,61 @@ CONFIG_FIELDS: tuple[ConfigField, ...] = (
"home_assistant_action_task_project_id",
"Home Assistant Action Task Project ID",
),
ConfigField("MQTT", "MQTT_ENABLED", "mqtt_enabled", "MQTT Enabled", input_type="checkbox"),
ConfigField("MQTT", "MQTT_BROKER_HOST", "mqtt_broker_host", "MQTT Broker Host"),
ConfigField("MQTT", "MQTT_BROKER_PORT", "mqtt_broker_port", "MQTT Broker Port", input_type="number"),
ConfigField("MQTT", "MQTT_USERNAME", "mqtt_username", "MQTT Username"),
ConfigField("MQTT", "MQTT_PASSWORD", "mqtt_password", "MQTT Password", secret=True),
ConfigField("MQTT", "MQTT_TLS_ENABLED", "mqtt_tls_enabled", "MQTT TLS Enabled", input_type="checkbox"),
ConfigField(
"Home Assistant Discovery",
"HA_DISCOVERY_ENABLED",
"ha_discovery_enabled",
"HA Discovery Enabled",
input_type="checkbox",
),
ConfigField(
"Home Assistant Discovery",
"HA_DISCOVERY_PREFIX",
"ha_discovery_prefix",
"HA Discovery Prefix",
),
ConfigField(
"Home Assistant Discovery",
"HA_STATE_TOPIC_PREFIX",
"ha_state_topic_prefix",
"HA State Topic Prefix",
),
ConfigField("Modbus", "MODBUS_POLLING_ENABLED", "modbus_polling_enabled", "Modbus Polling Enabled", input_type="checkbox"),
ConfigField(
"DSMR",
"DSMR_INGEST_ENABLED",
"dsmr_ingest_enabled",
"DSMR Ingest Enabled",
input_type="checkbox",
),
ConfigField("DSMR", "DSMR_MQTT_TOPIC", "dsmr_mqtt_topic", "DSMR MQTT Topic"),
ConfigField(
"DSMR",
"DSMR_SAMPLE_INTERVAL_S",
"dsmr_sample_interval_s",
"DSMR Sample Interval (s)",
input_type="number",
),
ConfigField(
"DSMR",
"DSMR_TARIFF_TOPIC",
"dsmr_tariff_topic",
"DSMR Tariff Topic",
),
ConfigField(
"Tibber",
"TIBBER_API_TOKEN",
"tibber_api_token",
"Tibber API Token",
secret=True,
),
ConfigField("Tibber", "TIBBER_HOME_ID", "tibber_home_id", "Tibber Home ID"),
)
@@ -284,4 +346,22 @@ def _settings_payload(settings: Settings) -> dict[str, Any]:
"auth_session_cookie_name": settings.auth_session_cookie_name,
"auth_session_ttl_hours": settings.auth_session_ttl_hours,
"auth_cookie_secure_override": settings.auth_cookie_secure_override,
"auth_login_throttle_enabled": settings.auth_login_throttle_enabled,
"auth_trust_forwarded_for": settings.auth_trust_forwarded_for,
"modbus_polling_enabled": settings.modbus_polling_enabled,
"mqtt_enabled": settings.mqtt_enabled,
"mqtt_broker_host": settings.mqtt_broker_host,
"mqtt_broker_port": settings.mqtt_broker_port,
"mqtt_username": settings.mqtt_username,
"mqtt_password": settings.mqtt_password,
"mqtt_tls_enabled": settings.mqtt_tls_enabled,
"ha_discovery_enabled": settings.ha_discovery_enabled,
"ha_discovery_prefix": settings.ha_discovery_prefix,
"ha_state_topic_prefix": settings.ha_state_topic_prefix,
"dsmr_ingest_enabled": settings.dsmr_ingest_enabled,
"dsmr_mqtt_topic": settings.dsmr_mqtt_topic,
"dsmr_sample_interval_s": settings.dsmr_sample_interval_s,
"dsmr_tariff_topic": settings.dsmr_tariff_topic,
"tibber_api_token": settings.tibber_api_token,
"tibber_home_id": settings.tibber_home_id,
}
+364
View File
@@ -0,0 +1,364 @@
"""Service layer for EnergyContract CRUD, versioning, and activation.
All functions accept an explicit SQLAlchemy Session; callers are responsible
for committing or rolling back the transaction.
Design decisions
----------------
- ``create_contract``: validates values against the pricing profile *before*
writing any rows; raises ``ProfileValidationError`` on non-compliance.
- ``add_version``: append-only; closes the previous open version by
setting its ``effective_to`` to the new version's ``effective_from``; raises
``ContractVersionError`` if the new date is strictly earlier than the previous
version's ``effective_from``.
- ``activate_contract``: mutual-exclusion; sets all other contracts' ``active``
to False, then sets the given contract's ``active`` to True.
- ``active_contract_version_at``: returns the single version of the currently
active contract that covers *ts* (``effective_from ≤ ts < effective_to``,
or open-ended when ``effective_to`` is None).
SQLite timezone note
--------------------
SQLite stores ``DateTime(timezone=True)`` columns as naive UTC strings; on
read-back they come out as **timezone-naive** datetimes. Wherever this code
compares timestamps from the DB against timezone-aware values (e.g. from
Pydantic or ``datetime.now(UTC)``), it calls ``_as_utc()`` to make both sides
comparable without tripping on "offset-naive vs offset-aware" TypeErrors.
"""
from __future__ import annotations
import logging
from datetime import UTC, datetime
from typing import Any
from sqlalchemy import select
from sqlalchemy.orm import Session
from app.integrations.pricing.profiles import validate_values
from app.models.energy import EnergyContract, EnergyContractVersion
logger = logging.getLogger(__name__)
# ---------------------------------------------------------------------------
# Internal helpers
# ---------------------------------------------------------------------------
def _as_utc(dt: datetime) -> datetime:
"""Return *dt* as a timezone-aware UTC datetime.
SQLite's DateTime(timezone=True) column type stores datetimes as naive UTC
strings and gives them back as naive datetimes on read. This helper
re-attaches the UTC timezone info when it is missing, making cross-origin
comparisons safe.
"""
if dt.tzinfo is None:
return dt.replace(tzinfo=UTC)
return dt
# ---------------------------------------------------------------------------
# Custom exception
# ---------------------------------------------------------------------------
class ContractVersionError(ValueError):
"""Raised when a new contract version has an invalid effective date.
Specifically: the new version's ``effective_from`` must be greater than or
equal to the previous open version's ``effective_from``. Allowing equal
timestamps would cause ambiguous overlap; the service therefore also rejects
strictly-equal values (same second) to avoid silent data loss.
"""
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def get_contract_or_none(session: Session, contract_id: int) -> EnergyContract | None:
"""Return the contract with the given id, or None if not found."""
return session.execute(
select(EnergyContract).where(EnergyContract.id == contract_id)
).scalar_one_or_none()
def list_contracts(session: Session) -> list[EnergyContract]:
"""Return all contracts ordered by id (ascending)."""
return list(
session.execute(select(EnergyContract).order_by(EnergyContract.id)).scalars().all()
)
def _open_version(session: Session, contract: EnergyContract) -> EnergyContractVersion | None:
"""Return the current open version (effective_to IS NULL) for *contract*, or None."""
return session.execute(
select(EnergyContractVersion)
.where(
EnergyContractVersion.contract_id == contract.id,
EnergyContractVersion.effective_to.is_(None),
)
.order_by(EnergyContractVersion.effective_from.desc())
.limit(1)
).scalar_one_or_none()
# ---------------------------------------------------------------------------
# Core service functions
# ---------------------------------------------------------------------------
def create_contract(
session: Session,
*,
name: str,
kind: str,
currency: str = "EUR",
values: dict[str, Any],
effective_from: datetime,
) -> EnergyContract:
"""Create a new energy contract with its first pricing version.
Parameters
----------
session:
Active SQLAlchemy session. Caller must commit after this returns.
name:
Human-readable label for the contract.
kind:
Pricing strategy identifier (``"manual"`` or ``"tibber"``).
currency:
ISO 4217 currency code (default ``"EUR"``).
values:
Pricing values dict conforming to the named profile's structure.
Validated via ``validate_values(kind, values)`` before any writes.
effective_from:
UTC datetime at which the first pricing version takes effect.
Returns
-------
EnergyContract
The newly created contract (not yet committed).
Raises
------
ProfileNotFoundError
If no YAML profile exists for *kind*.
ProfileValidationError
If *values* does not conform to the profile structure.
"""
# Validate (and fill defaults) before any DB write.
filled_values = validate_values(kind, values)
now = datetime.now(UTC)
contract = EnergyContract(
name=name,
kind=kind,
currency=currency,
active=False, # New contracts are inactive; caller must explicitly activate.
created_at=now,
updated_at=now,
)
session.add(contract)
session.flush() # Assign contract.id so we can reference it in the version FK.
version = EnergyContractVersion(
contract_id=contract.id,
effective_from=effective_from,
effective_to=None,
values=filled_values,
created_at=now,
)
session.add(version)
logger.info("Created energy contract %r (kind=%s, id=%d)", name, kind, contract.id)
return contract
def add_version(
session: Session,
contract: EnergyContract,
*,
effective_from: datetime,
values: dict[str, Any],
) -> EnergyContractVersion:
"""Add a new pricing version to an existing contract (append-only).
The previous open version's ``effective_to`` is automatically set to the
new version's ``effective_from`` (version closure), ensuring there is never
a gap or overlap between consecutive versions.
The new ``effective_from`` **must be strictly greater than** the previous
open version's ``effective_from``. Equal timestamps are rejected because
they would produce two versions starting at the same instant, making it
impossible to determine which is current. If this constraint is not met,
``ContractVersionError`` is raised and **no rows are written**.
Parameters
----------
session:
Active SQLAlchemy session. Caller must commit after this returns.
contract:
The parent ``EnergyContract`` to append a version to.
effective_from:
UTC datetime at which this version's pricing takes effect.
values:
New pricing values dict conforming to the contract's profile.
Returns
-------
EnergyContractVersion
The newly created version (not yet committed).
Raises
------
ContractVersionError
If *effective_from* is not strictly after the previous open version's
``effective_from``.
ProfileNotFoundError
If no YAML profile exists for the contract's kind.
ProfileValidationError
If *values* does not conform to the profile structure.
"""
# Validate (and fill defaults) before any DB write.
filled_values = validate_values(contract.kind, values)
prev = _open_version(session, contract)
if prev is not None:
# Enforce strictly-after constraint to prevent ambiguous overlap.
# Use _as_utc() on both sides: the incoming value may be tz-aware
# while the DB-read value is tz-naive (SQLite limitation).
if _as_utc(effective_from) <= _as_utc(prev.effective_from):
raise ContractVersionError(
f"New version effective_from ({effective_from.isoformat()}) must be strictly "
f"after the previous open version's effective_from "
f"({prev.effective_from.isoformat()})."
)
# Close the previous open version.
prev.effective_to = effective_from
now = datetime.now(UTC)
new_version = EnergyContractVersion(
contract_id=contract.id,
effective_from=effective_from,
effective_to=None,
values=filled_values,
created_at=now,
)
session.add(new_version)
logger.info(
"Added version to contract id=%d (kind=%s, effective_from=%s)",
contract.id,
contract.kind,
effective_from.isoformat(),
)
return new_version
def activate_contract(session: Session, contract: EnergyContract) -> None:
"""Activate a contract with mutual exclusion.
Sets every other contract's ``active`` flag to False, then sets the given
contract's ``active`` to True. This guarantees at most one active contract
at any time.
Caller must commit after this returns.
"""
# Deactivate all contracts (including the target; we re-activate below).
for other in session.execute(select(EnergyContract)).scalars().all():
other.active = False
contract.active = True
contract.updated_at = datetime.now(UTC)
logger.info("Activated contract %r (id=%d)", contract.name, contract.id)
def deactivate_contract(session: Session, contract: EnergyContract) -> None:
"""Deactivate a contract without touching other contracts.
Caller must commit after this returns.
"""
contract.active = False
contract.updated_at = datetime.now(UTC)
logger.info("Deactivated contract %r (id=%d)", contract.name, contract.id)
def active_contract_versions(session: Session) -> list[EnergyContractVersion]:
"""Return all versions of the currently active contract, ordered by effective_from ascending.
Returns an empty list when there is no active contract. The list spans the
full history of the active contract (all closed + the current open version)
and is used to iterate over pricing-rate segments for cross-version fixed-
cost / credit accumulation (Principle C).
"""
active = session.execute(
select(EnergyContract).where(EnergyContract.active.is_(True)).limit(1)
).scalar_one_or_none()
if active is None:
return []
return list(
session.execute(
select(EnergyContractVersion)
.where(EnergyContractVersion.contract_id == active.id)
.order_by(EnergyContractVersion.effective_from)
)
.scalars()
.all()
)
def active_contract_version_at(
session: Session, ts: datetime
) -> EnergyContractVersion | None:
"""Return the active contract's version that covers *ts*.
A version covers *ts* when:
``effective_from ≤ ts`` AND (``effective_to IS NULL`` OR ``ts < effective_to``)
Returns None when:
- There is no active contract.
- The active contract has no version covering *ts*.
Parameters
----------
session:
Active SQLAlchemy session (read-only usage).
ts:
UTC datetime to look up.
Returns
-------
EnergyContractVersion | None
"""
active = session.execute(
select(EnergyContract).where(EnergyContract.active.is_(True)).limit(1)
).scalar_one_or_none()
if active is None:
return None
# Build a query for all versions of the active contract covering ts.
stmt = (
select(EnergyContractVersion)
.where(
EnergyContractVersion.contract_id == active.id,
EnergyContractVersion.effective_from <= ts,
)
.order_by(EnergyContractVersion.effective_from.desc())
.limit(1)
)
version = session.execute(stmt).scalar_one_or_none()
if version is None:
return None
# Exclude versions whose effective window has already closed before ts.
if version.effective_to is not None and _as_utc(ts) >= _as_utc(version.effective_to):
return None
return version
+297
View File
@@ -0,0 +1,297 @@
"""DSMR telegram ingest service.
Subscribes to the DSMR Reader MQTT topic (``dsmr/json``) and persists
down-sampled DSMR telegram frames to the ``dsmr_reading`` table.
Design decisions
----------------
- **Whole-frame storage**: the entire parsed telegram dict is stored as a JSON
blob in ``DsmrReading.payload``; no field allow-list is applied. This lets
future commodities (gas, heating, three-phase) be accommodated without a
table-schema change.
- **10-second down-sampling** (configurable via ``dsmr_sample_interval_s``):
only telegrams whose ``timestamp`` second falls on an exact multiple of the
interval are persisted. This reduces write volume from ~60 rows/min to ~6
rows/min while guaranteeing that every 15-minute boundary (second=00) is
captured.
- **Idempotency**: the telegram's own ``id`` field is stored as ``source_id``
with a UNIQUE constraint. A second delivery of the same telegram (e.g. after
a broker reconnect) is silently skipped.
- **Network-thread safety**: ``handle_message`` is called from paho's background
loop thread. It opens and closes its own short-lived SQLAlchemy session and
swallows all exceptions so that a buggy payload or transient DB error never
crashes the paho loop or drops the MQTT connection.
- **Numeric values kept as strings**: the DSMR Reader emits all numeric readings
as JSON strings (e.g. ``"20915.154"``). They are stored verbatim; conversion
to ``Decimal`` is deferred to the billing engine (T07) where precision matters.
- **Null phases**: some telegrams omit certain phase readings (``null`` in JSON);
these are stored as-is without special handling.
"""
from __future__ import annotations
import json
import logging
import threading
from datetime import datetime, timezone
from typing import TYPE_CHECKING
import sqlalchemy.exc
from sqlalchemy import select
from app.db import get_session_local
from app.models.energy import DsmrReading
if TYPE_CHECKING:
from app.config import Settings
logger = logging.getLogger(__name__)
# Tracks the DSMR topic currently subscribed via the MQTT manager, so a config
# change can unsubscribe the old topic before subscribing the new one.
_current_dsmr_topic: str | None = None
# Tracks the DSMR tariff topic currently subscribed via the MQTT manager.
_current_tariff_topic: str | None = None
# Current electricity tariff: 1 = dal/off-peak, 2 = normal/peak.
# Written by the paho network thread, read by the publish job — guarded by a lock.
_current_tariff: int | None = None
_tariff_lock = threading.Lock()
def get_current_tariff() -> int | None:
"""Return the most recently received electricity tariff (1 or 2), or None."""
with _tariff_lock:
return _current_tariff
def set_current_tariff(value: int | None) -> None:
"""Set the current electricity tariff (1 or 2), or clear it with None."""
with _tariff_lock:
global _current_tariff
_current_tariff = value
def handle_tariff_message(payload_bytes: bytes) -> None:
"""Parse one DSMR tariff MQTT payload and update the in-memory tariff state.
Called from the paho network thread; *must* swallow all exceptions so that
a bad payload never crashes the loop or drops the broker connection.
Accepts payload as bytes or str (paho can deliver either). Ignores
whitespace. Only accepts integer values 1 or 2; anything else is discarded
and the previous known tariff is preserved.
Parameters
----------
payload_bytes:
Raw bytes from the MQTT message (may also be a str in some paho versions).
"""
try:
# Decode bytes → str if needed; strip surrounding whitespace.
if isinstance(payload_bytes, (bytes, bytearray)):
raw = payload_bytes.decode("utf-8", errors="replace").strip()
else:
raw = str(payload_bytes).strip()
value = int(raw)
if value not in (1, 2):
logger.debug(
"dsmr_ingest.handle_tariff_message: unexpected tariff value %d (expected 1 or 2, ignored).",
value,
)
return
set_current_tariff(value)
logger.debug("dsmr_ingest.handle_tariff_message: tariff updated to %d.", value)
except Exception:
# Malformed payload (e.g. non-numeric); swallow silently to protect network thread.
logger.debug(
"dsmr_ingest.handle_tariff_message: could not parse payload %r (ignored).",
payload_bytes,
)
def apply_dsmr_subscription(settings: "Settings") -> None:
"""(Re)apply the DSMR MQTT subscriptions to match *settings* — restart-free.
Call this at startup and after every config save. It makes the live MQTT
subscriptions reflect the current ``dsmr_ingest_enabled`` / ``dsmr_mqtt_topic``
/ ``dsmr_sample_interval_s`` / ``dsmr_tariff_topic`` settings without an app
restart:
- **Disabled** → unsubscribe any active DSMR and tariff subscriptions.
- **Enabled** → (re)subscribe to ``dsmr_mqtt_topic`` with a handler bound to
a *fresh* settings snapshot, so a changed sample interval also takes effect.
Also subscribe to ``dsmr_tariff_topic`` when non-empty.
- **Topic changed** → unsubscribe the old topic before subscribing the new one.
Idempotent and safe to call when MQTT is not connected (the subscription is
queued in the manager and established on the next connect).
"""
# Imported here (not at module top) to avoid a circular import at app start.
from app.integrations.mqtt import mqtt_manager
global _current_dsmr_topic, _current_tariff_topic
if not settings.dsmr_ingest_enabled:
if _current_dsmr_topic is not None:
mqtt_manager.unsubscribe(_current_dsmr_topic)
logger.info("DSMR ingest disabled — unsubscribed from topic=%s.", _current_dsmr_topic)
_current_dsmr_topic = None
if _current_tariff_topic is not None:
mqtt_manager.unsubscribe(_current_tariff_topic)
logger.info(
"DSMR ingest disabled — unsubscribed from tariff topic=%s.",
_current_tariff_topic,
)
_current_tariff_topic = None
return
# --- Main DSMR telegram topic ---
topic = settings.dsmr_mqtt_topic
if _current_dsmr_topic is not None and _current_dsmr_topic != topic:
mqtt_manager.unsubscribe(_current_dsmr_topic)
# Re-subscribe (overwrites any existing handler for this topic) with a fresh
# settings snapshot so dsmr_sample_interval_s changes take effect too.
snapshot = settings
mqtt_manager.subscribe(topic, lambda payload: handle_message(payload, snapshot))
_current_dsmr_topic = topic
logger.info("DSMR ingest enabled — subscribed to topic=%s.", topic)
# --- DSMR tariff topic (dual-tariff slot indicator) ---
tariff_topic = settings.dsmr_tariff_topic if settings.dsmr_tariff_topic else ""
if tariff_topic:
if _current_tariff_topic is not None and _current_tariff_topic != tariff_topic:
mqtt_manager.unsubscribe(_current_tariff_topic)
mqtt_manager.subscribe(tariff_topic, lambda payload: handle_tariff_message(payload))
_current_tariff_topic = tariff_topic
logger.info("DSMR tariff topic — subscribed to topic=%s.", tariff_topic)
else:
# tariff_topic is empty → unsubscribe any existing tariff subscription.
if _current_tariff_topic is not None:
mqtt_manager.unsubscribe(_current_tariff_topic)
logger.info(
"DSMR tariff topic cleared — unsubscribed from topic=%s.",
_current_tariff_topic,
)
_current_tariff_topic = None
def handle_message(payload_bytes: bytes, settings: "Settings") -> None:
"""Parse one DSMR MQTT payload and persist it if it passes the sample filter.
Called from the paho network thread; *must* swallow all exceptions so that
a bad payload or transient error does not crash the loop or drop the broker
connection.
Parameters
----------
payload_bytes:
Raw bytes from the MQTT message.
settings:
Runtime settings snapshot (captured at subscription time). Used for
``dsmr_sample_interval_s``.
"""
try:
_handle_message_inner(payload_bytes, settings)
except Exception:
logger.exception("dsmr_ingest.handle_message: unexpected error (swallowed).")
def _handle_message_inner(payload_bytes: bytes, settings: "Settings") -> None:
"""Inner implementation — may raise; caller wraps in try/except."""
# --- 1. Parse JSON ---
try:
data: dict = json.loads(payload_bytes)
except (json.JSONDecodeError, ValueError):
logger.debug("dsmr_ingest: invalid JSON payload (skipped).")
return
if not isinstance(data, dict):
logger.debug("dsmr_ingest: payload is not a JSON object (skipped).")
return
# --- 2. Parse timestamp ---
raw_ts = data.get("timestamp")
if raw_ts is None:
logger.debug("dsmr_ingest: missing 'timestamp' field (skipped).")
return
try:
# Python 3.11+ accepts the trailing 'Z' directly; for 3.10 compat we
# replace 'Z' with '+00:00' before parsing.
ts_str = raw_ts if not isinstance(raw_ts, str) else raw_ts.replace("Z", "+00:00")
ts_utc: datetime = datetime.fromisoformat(ts_str)
# Ensure it is timezone-aware UTC.
if ts_utc.tzinfo is None:
ts_utc = ts_utc.replace(tzinfo=timezone.utc)
except (ValueError, TypeError, AttributeError):
logger.debug(
"dsmr_ingest: cannot parse 'timestamp' value %r (skipped).", raw_ts
)
return
# --- 3. Down-sample: only persist if second falls on interval boundary ---
interval = settings.dsmr_sample_interval_s
if interval > 0 and (ts_utc.second % interval) != 0:
# This telegram is between sample points; discard silently.
return
# --- 4. Extract source_id (telegram's own id) — stored only as a reference,
# NOT used for uniqueness/idempotency (it overflows and gets reset). ---
source_id: int | None = data.get("id")
if source_id is not None and not isinstance(source_id, int):
# Unexpected type — treat as missing rather than raising.
logger.debug(
"dsmr_ingest: 'id' field has unexpected type %s (ignoring).",
type(source_id).__name__,
)
source_id = None
# --- 5. Persist to database ---
# Idempotency is keyed on recorded_at (the telegram timestamp), which is
# telegram-id-independent: a single P1 meter emits one telegram per second,
# and down-sampling keeps at most one per interval-aligned second. The
# UNIQUE(recorded_at) constraint is the backstop for the IntegrityError race.
session_local = get_session_local()
session = session_local()
try:
existing = session.scalar(
select(DsmrReading).where(DsmrReading.recorded_at == ts_utc)
)
if existing is not None:
logger.debug(
"dsmr_ingest: recorded_at=%s already in DB, skipping.",
ts_utc.isoformat(),
)
return
reading = DsmrReading(
recorded_at=ts_utc,
source_id=source_id,
payload=data, # full frame, verbatim
)
session.add(reading)
session.commit()
logger.debug(
"dsmr_ingest: persisted reading recorded_at=%s source_id=%s.",
ts_utc.isoformat(),
source_id,
)
except sqlalchemy.exc.IntegrityError:
# Race / duplicate: another insert beat us to the same recorded_at.
session.rollback()
logger.debug(
"dsmr_ingest: IntegrityError for recorded_at=%s (duplicate, skipped).",
ts_utc.isoformat(),
)
except Exception:
session.rollback()
logger.exception("dsmr_ingest: DB error (swallowed).")
finally:
session.close()
+881
View File
@@ -0,0 +1,881 @@
"""Billing engine for DSMR 15-minute energy metering periods.
This module implements the two-layer billing model described in §3.4 of the
M6 design document, extended in M7-T03 to be meter-aware:
**Layer 1 — per-period metering cost (immutable, price-snapshot)**
``compute_period(session, t0)`` computes the import cost, export revenue, and
net cost for the 15-minute period ``[t0, t0+15min)``. The result is written
to ``energy_cost_period`` with a full pricing snapshot so each row is
self-contained and auditable. Existing *successful* rows are never overwritten
by the normal tick path; only an explicit ``recompute_range`` call passes
``overwrite=True``.
**Layer 2 — summary (computed at read time, not stored)**
``summarize(session, start, end)`` aggregates all non-degraded
``energy_cost_period`` rows in ``[start, end)``, then adds the daily
standing charges (network_fee + management_fee, apportioned at EUR/month
÷ 30 per day) and subtracts the energy-tax credit (heffingskorting,
apportioned at EUR/year ÷ 365 per day).
Design notes
------------
- **Decimal arithmetic throughout**: all monetary computations use
``decimal.Decimal`` to avoid float binary rounding errors. Only when
writing to ``EnergyCostPeriod`` columns (Float) are values converted to
float. ``summarize`` converts back to Decimal for summation.
- **UTC quarter-hour grid**: period boundaries are aligned to UTC 00/15/30/45
minutes (``floor_to_quarter``). NL local time (CET/CEST) is always a whole
number of hours from UTC, so the quarter-hour grid is the same in both
timezone representations.
- **Register keys**: DSMR payload uses JSON strings like ``"20915.154"``
for cumulative kWh registers. ``register_at`` converts them to Decimal.
- **Degraded vs skip semantics**:
- *No meter coverage* (``meter_at`` returns None for t0): write a
``degraded=True`` row with ``meter_id=None``.
- *Cross-meter boundary* (m0.id != m1.id for t0/t1): write a ``degraded=True``
row with ``meter_id=m0.id``; losing this one period at the swap boundary is
acceptable (D5 decision).
- *Missing readings* (``register_at`` returns None for start or end
boundary within the meter window): write a ``degraded=True`` row with
``meter_id=m0.id`` so the period is tracked and can be retried by
``compute_closed_periods``.
- *Negative or excessively large delta* (delta sanity guard D6): write a
``degraded=True`` row with ``meter_id=m0.id``; prevents negative costs and
grossly inflated costs from meter resets, DSMR rollover, or data spikes.
- *Missing Tibber price* (``TibberPriceNotFoundError``): skip entirely (do
not write a row); the period will be retried once prices arrive.
- *Missing active contract version*: skip (no contract to compute against).
- **Meter-aware register lookup**: ``register_at`` now accepts a ``meter``
parameter and restricts the DSMR reading query to readings within
``[meter.started_at, meter.ended_at)`` (half-open), preventing old-meter
readings from leaking into a new-meter epoch.
- **Lookback window in ``compute_closed_periods``**: to avoid scanning all
historical DSMR data on every tick, the function looks back at most 7 days
from the current time. This covers typical short outages (no data / no
contract) while staying bounded. Periods older than 7 days must be
recovered via an explicit ``recompute_range`` call.
Meter-aware compute_period ordering rationale (M7-T03)
-------------------------------------------------------
The order of checks inside ``compute_period`` is:
1. **Immutability guard** (existing non-degraded row, overwrite=False) → return False.
2. **Meter determination** (m0 = meter_at(t0), m1 = meter_at(t1)):
- No meter (m0 is None) → write degraded, meter_id=None.
- Cross-meter boundary (m0.id != m1.id) → write degraded, meter_id=m0.id.
3. **Active contract version check** → skip (no write) if absent.
4. **Boundary register readings** within m0's window → write degraded if missing.
5. **Delta sanity guard** → write degraded if any delta < 0 or > _MAX_DELTA_KWH.
6. **Price strategy** → skip (no write) if Tibber price missing.
7. **Upsert billing record** with meter_id=m0.id.
Why meter before contract? The meter is a *structural* prerequisite: without a
known meter epoch we cannot trust the delta at all, so we commit a degraded row
immediately. The contract skip, by contrast, is transient (the period can be
re-computed once a contract is configured), so it produces no row.
"""
from __future__ import annotations
import logging
from datetime import UTC, datetime, timedelta
from decimal import Decimal
from typing import Any
from sqlalchemy import select
from sqlalchemy.orm import Session
from app.integrations.pricing.strategies import (
PeriodDeltas,
TibberPriceNotFoundError,
get_strategy,
)
from app.models.energy import DsmrReading, EnergyCostPeriod, Meter
from app.services.contracts import active_contract_version_at, active_contract_versions
from app.services.meters import meter_at
from app.services.timezone import local_date, local_now
logger = logging.getLogger(__name__)
# ---------------------------------------------------------------------------
# Constants
# ---------------------------------------------------------------------------
_PERIOD_MINUTES = 15
_LOOKBACK_DAYS = 7 # maximum lookback window for compute_closed_periods
# Maximum age a DSMR reading may have relative to the boundary being queried.
# Under normal operation DSMR readings arrive every ~10 seconds, so a reading
# more than one period (15 minutes) old at the boundary indicates either a data
# gap or — critically — a *future* boundary being resolved against the last
# historical reading. In both cases the reading is considered stale and
# ``register_at`` returns None, letting the period be marked degraded instead of
# producing a spurious zero-delta "successful" row.
_READING_MAX_STALENESS = timedelta(minutes=_PERIOD_MINUTES)
# Maximum plausible kWh delta for a single 15-minute period (D6 sanity guard).
# A typical Dutch household uses well under 5 kWh per quarter hour even under
# heavy load. 100 kWh per 15 minutes corresponds to ~400 kW — far beyond any
# residential consumption — but is lenient enough to never fire on legitimate
# data. Any delta at or above this threshold indicates a meter reset, DSMR
# rollover, sign error, or other data anomaly, and the period is marked
# degraded to prevent negative costs or grossly inflated charges.
_MAX_DELTA_KWH = Decimal("100")
# DSMR payload register keys (cumulative kWh, JSON string values).
_KEY_D1 = "electricity_delivered_1" # delivered low-tariff (dal / _1)
_KEY_D2 = "electricity_delivered_2" # delivered high-tariff (normal / _2)
_KEY_R1 = "electricity_returned_1" # returned low-tariff
_KEY_R2 = "electricity_returned_2" # returned high-tariff
# ---------------------------------------------------------------------------
# Internal helpers
# ---------------------------------------------------------------------------
def floor_to_quarter(dt: datetime) -> datetime:
"""Return *dt* floored to the nearest UTC quarter-hour boundary.
The result always has seconds=0 and microseconds=0, and minutes in
{0, 15, 30, 45}. Timezone info is preserved if present.
"""
floored_minute = (dt.minute // _PERIOD_MINUTES) * _PERIOD_MINUTES
return dt.replace(minute=floored_minute, second=0, microsecond=0)
def _to_decimal(value: Any) -> Decimal:
"""Convert *value* to Decimal via str() to avoid float binary rounding."""
return Decimal(str(value))
def _as_utc(dt: datetime) -> datetime:
"""Attach UTC tzinfo to a naive datetime (SQLite read-back workaround)."""
if dt.tzinfo is None:
return dt.replace(tzinfo=UTC)
return dt
def _existing_period(session: Session, t0: datetime) -> EnergyCostPeriod | None:
"""Return the EnergyCostPeriod row for period_start=t0, or None."""
return session.execute(
select(EnergyCostPeriod).where(EnergyCostPeriod.period_start == t0)
).scalar_one_or_none()
# ---------------------------------------------------------------------------
# register_at — boundary reading lookup (meter-aware)
# ---------------------------------------------------------------------------
def register_at(
session: Session,
boundary: datetime,
meter: Meter,
) -> dict[str, Decimal] | None:
"""Return the four cumulative kWh register values at *boundary*, within *meter*'s window.
Queries the most recent ``DsmrReading`` with:
``recorded_at ≤ boundary``
AND ``recorded_at ≥ meter.started_at``
AND (``meter.ended_at IS NULL`` OR ``recorded_at < meter.ended_at``)
The meter window constraint (half-open ``[started_at, ended_at)``) ensures
that readings from a previous meter epoch are never used to anchor a new
meter's computation. Without this guard, the final reading of the old meter
would be visible at the start of the new meter's epoch and produce a
cross-meter delta, defeating the isolation guarantee.
Extracts the four energy registers from ``payload``:
d1 — electricity_delivered_1 (delivered low-tariff / dal)
d2 — electricity_delivered_2 (delivered high-tariff / normal)
r1 — electricity_returned_1 (returned low-tariff)
r2 — electricity_returned_2 (returned high-tariff)
Returns
-------
dict[str, Decimal] with keys ``d1``, ``d2``, ``r1``, ``r2``, or ``None``
when:
- No ``DsmrReading`` row exists with ``recorded_at ≤ boundary`` within
*meter*'s epoch window.
- The most recent such reading is older than ``_READING_MAX_STALENESS``
relative to *boundary* (freshness guard).
- Any of the four register keys is absent from the payload.
- Any of the four register values is ``None`` (null in JSON).
SQLite naive datetime note
--------------------------
``recorded_at`` is stored as a naive UTC datetime in SQLite. Comparisons
against *boundary* (always tz-aware UTC) use ``_as_utc()`` for the
freshness check. The SQL ``WHERE`` clause comparisons work correctly
because SQLAlchemy's SQLite dialect strips tzinfo when binding parameters
(leaving the wall-clock UTC value unchanged), consistent with the storage
format.
"""
# Build the meter-window constraints: [started_at, ended_at).
meter_lower = meter.started_at # DsmrReading.recorded_at >= meter.started_at
meter_upper = meter.ended_at # DsmrReading.recorded_at < meter.ended_at (if set)
stmt = (
select(DsmrReading)
.where(
DsmrReading.recorded_at <= boundary,
DsmrReading.recorded_at >= meter_lower,
)
.order_by(DsmrReading.recorded_at.desc())
.limit(1)
)
# Apply the upper bound only when the meter is closed (ended_at is not None).
if meter_upper is not None:
stmt = stmt.where(DsmrReading.recorded_at < meter_upper)
row: DsmrReading | None = session.execute(stmt).scalar_one_or_none()
if row is None:
return None
# Freshness guard: reject readings that are too old relative to *boundary*.
# ``recorded_at`` is stored as a naive UTC datetime in SQLite; attach UTC
# tzinfo before comparing with *boundary* (which is always tz-aware UTC) to
# avoid an "offset-naive vs offset-aware" TypeError.
if _as_utc(row.recorded_at) < _as_utc(boundary) - _READING_MAX_STALENESS:
return None
payload = row.payload or {}
try:
d1_raw = payload[_KEY_D1]
d2_raw = payload[_KEY_D2]
r1_raw = payload[_KEY_R1]
r2_raw = payload[_KEY_R2]
except KeyError:
return None
if any(v is None for v in (d1_raw, d2_raw, r1_raw, r2_raw)):
return None
return {
"d1": _to_decimal(d1_raw),
"d2": _to_decimal(d2_raw),
"r1": _to_decimal(r1_raw),
"r2": _to_decimal(r2_raw),
}
# ---------------------------------------------------------------------------
# compute_period — single 15-minute period
# ---------------------------------------------------------------------------
def compute_period(session: Session, t0: datetime, *, overwrite: bool = False) -> bool:
"""Compute and upsert the billing record for the period ``[t0, t0+15min)``.
Parameters
----------
session:
Active SQLAlchemy session. Caller is responsible for committing.
t0:
UTC start of the 15-minute period. **Must** lie on a quarter-hour
grid boundary (minutes ∈ {0, 15, 30, 45}, seconds=0, microseconds=0).
overwrite:
If ``True``, overwrite an existing *successful* row (i.e. re-compute
even when a non-degraded record already exists). The normal tick path
always passes ``False``; only ``recompute_range`` passes ``True``.
Returns
-------
bool
``True`` if a record was written (inserted or updated), ``False`` if
the period was skipped (missing contract or missing Tibber price).
Side-effects
------------
- Inserts or updates an ``EnergyCostPeriod`` row keyed on ``period_start=t0``.
- If no meter covers t0 (``meter_at`` returns None for t0): inserts/updates
a degraded row with ``meter_id=None``.
- If the period spans a meter boundary (``meter_at(t0).id != meter_at(t1).id``):
inserts/updates a degraded row with ``meter_id=m0.id`` (D5 decision).
- If readings are missing at either boundary within the meter window:
inserts/updates a degraded row with ``meter_id=m0.id``.
- If any delta is negative or exceeds ``_MAX_DELTA_KWH`` (D6 sanity guard):
inserts/updates a degraded row with ``meter_id=m0.id``.
- If the active contract version is missing: **skips** (returns False, no write).
- If the Tibber price is missing (TibberPriceNotFoundError): **skips**
(returns False, no write).
"""
t1 = t0 + timedelta(minutes=_PERIOD_MINUTES)
now = datetime.now(UTC)
# Immutability guard: skip if a successful record already exists and we
# are not in overwrite mode.
existing = _existing_period(session, t0)
if existing is not None and not existing.degraded and not overwrite:
return False
# --- Meter determination (structural prerequisite, checked before contract) ---
#
# A missing or cross-boundary meter is a structural problem: we cannot trust
# the delta at all, so we write a degraded row immediately. This is different
# from the contract skip (transient, no write): the degraded row ensures the
# period appears in the history and can be revisited once the meter timeline
# is corrected and a recompute_range is triggered.
#
# Ordering rationale:
# 1. No meter (m0 is None) → degraded(meter_id=None): no epoch for t0.
# 2. Cross-meter boundary (m0.id != m1.id) → degraded(meter_id=m0.id): D5.
# 3. (Single meter, proceed) → contract check → readings → delta guard → price.
#
# We place meter before contract so that "cross-table period" is always
# marked degraded regardless of contract state. If we checked contract
# first, a missing-contract skip would silently discard the cross-table
# evidence; once a contract is added and recompute runs, the engine would
# incorrectly use cross-table reads.
m0 = meter_at(session, t0)
m1 = meter_at(session, t1)
if m0 is None:
# No meter epoch covers t0 — degraded with no meter attribution.
logger.debug(
"compute_period(%s): no active meter at t0 — writing degraded (meter_id=None).",
t0.isoformat(),
)
_upsert_degraded(session, t0, now, existing, meter_id=None)
return True
if m1 is None or m0.id != m1.id:
# Period spans a meter boundary or t1 has no meter. Degrade with m0's id
# (t0's meter attribution): the period's start belongs to m0's epoch.
logger.debug(
"compute_period(%s): period crosses meter boundary "
"(m0.id=%s, m1.id=%s) — writing degraded.",
t0.isoformat(),
m0.id,
m1.id if m1 is not None else None,
)
_upsert_degraded(session, t0, now, existing, meter_id=m0.id)
return True
# --- Active contract version at t0 ---
# If there is no active contract covering t0, skip the period entirely.
# We do not write a degraded row — there is no meaningful state to recover
# without a contract (we would not know which strategy to apply once data
# arrives). The period can be recovered via an explicit recompute_range once
# a contract is configured and activated.
version = active_contract_version_at(session, t0)
if version is None:
logger.debug("compute_period(%s): no active contract version — skipping.", t0.isoformat())
return False
# --- Boundary readings within m0's meter window ---
start_regs = register_at(session, t0, m0)
end_regs = register_at(session, t1, m0)
if start_regs is None or end_regs is None:
# Missing readings within the meter window → degraded with m0 attribution.
_upsert_degraded(session, t0, now, existing, meter_id=m0.id)
return True # a record was written (degraded)
# --- Compute deltas (end start) ---
deltas = PeriodDeltas(
d1=end_regs["d1"] - start_regs["d1"],
d2=end_regs["d2"] - start_regs["d2"],
r1=end_regs["r1"] - start_regs["r1"],
r2=end_regs["r2"] - start_regs["r2"],
)
# --- Delta sanity guard (D6) ---
# Any negative delta indicates a meter reset, DSMR rollover, or data error.
# Any delta exceeding _MAX_DELTA_KWH (100 kWh per 15 min = 400 kW average)
# is implausible for residential use and indicates an anomaly.
# Both cases produce a degraded row so no negative or grossly inflated cost
# is ever written to the billing record.
all_deltas = (deltas.d1, deltas.d2, deltas.r1, deltas.r2)
if any(d < Decimal("0") for d in all_deltas) or any(d > _MAX_DELTA_KWH for d in all_deltas):
logger.debug(
"compute_period(%s): delta sanity guard triggered "
"(d1=%s, d2=%s, r1=%s, r2=%s) — writing degraded.",
t0.isoformat(),
deltas.d1,
deltas.d2,
deltas.r1,
deltas.r2,
)
_upsert_degraded(session, t0, now, existing, meter_id=m0.id)
return True
# --- Price strategy ---
strategy = get_strategy(version.contract.kind)
try:
result = strategy(deltas, t0, version.values, session)
except TibberPriceNotFoundError:
# Missing Tibber price → skip the period; it will be retried once the
# price arrives (e.g. after the next Tibber refresh job runs).
logger.debug(
"compute_period(%s): no Tibber price found — skipping.", t0.isoformat()
)
return False
# --- Upsert the billing record ---
import_cost: Decimal = result["import_cost"]
export_revenue: Decimal = result["export_revenue"]
net_cost: Decimal = result["net_cost"]
pricing: dict = result["pricing"]
if existing is not None:
# Update in-place (overwrite=True or previous record was degraded).
existing.d1_kwh = float(deltas.d1)
existing.d2_kwh = float(deltas.d2)
existing.r1_kwh = float(deltas.r1)
existing.r2_kwh = float(deltas.r2)
existing.import_cost = float(import_cost)
existing.export_revenue = float(export_revenue)
existing.net_cost = float(net_cost)
existing.currency = version.contract.currency
existing.pricing = pricing
existing.contract_version_id = version.id
existing.meter_id = m0.id
existing.degraded = False
existing.computed_at = now
else:
period = EnergyCostPeriod(
period_start=t0,
d1_kwh=float(deltas.d1),
d2_kwh=float(deltas.d2),
r1_kwh=float(deltas.r1),
r2_kwh=float(deltas.r2),
import_cost=float(import_cost),
export_revenue=float(export_revenue),
net_cost=float(net_cost),
currency=version.contract.currency,
pricing=pricing,
contract_version_id=version.id,
meter_id=m0.id,
degraded=False,
computed_at=now,
)
session.add(period)
return True
def _upsert_degraded(
session: Session,
t0: datetime,
now: datetime,
existing: EnergyCostPeriod | None,
*,
meter_id: int | None,
) -> None:
"""Insert or update a degraded placeholder for period *t0*.
Parameters
----------
session:
Active SQLAlchemy session.
t0:
UTC start of the 15-minute period.
now:
Current UTC timestamp for the ``computed_at`` field.
existing:
The existing ``EnergyCostPeriod`` row for this period, or ``None``.
meter_id:
The meter ID to attribute this degraded period to, or ``None`` when
no meter epoch covers the period (no-meter degraded case).
When *existing* is not None (row was previously written — either degraded
or successful), the row is explicitly reset to the standard degraded state.
This is required for the ``recompute_range`` (overwrite=True) path: if the
row was previously a *successful* computation and the boundary readings have
since disappeared, the stale non-zero costs must be cleared so the row
accurately reflects the current "missing readings" state rather than
masquerading as a valid result.
The ``meter_id`` is always updated to reflect the current meter attribution
judgment (the result of ``meter_at`` at the time of recompute). This
ensures that a retroactive ``started_at`` change + ``recompute_range`` will
re-attribute historical degraded periods to the correct meter epoch.
"""
if existing is not None:
# Explicitly reset to degraded state — identical field values to the
# new-row path below. This covers the recompute-over-successful-row
# case where old non-zero costs must not survive the downgrade.
existing.d1_kwh = 0.0
existing.d2_kwh = 0.0
existing.r1_kwh = 0.0
existing.r2_kwh = 0.0
existing.import_cost = 0.0
existing.export_revenue = 0.0
existing.net_cost = 0.0
existing.pricing = {}
existing.contract_version_id = None
existing.meter_id = meter_id
existing.degraded = True
existing.computed_at = now
else:
period = EnergyCostPeriod(
period_start=t0,
d1_kwh=0.0,
d2_kwh=0.0,
r1_kwh=0.0,
r2_kwh=0.0,
import_cost=0.0,
export_revenue=0.0,
net_cost=0.0,
currency="EUR", # placeholder; real currency known after contract lookup
pricing={},
contract_version_id=None,
meter_id=meter_id,
degraded=True,
computed_at=now,
)
session.add(period)
# ---------------------------------------------------------------------------
# compute_closed_periods — periodic tick
# ---------------------------------------------------------------------------
def compute_closed_periods(session: Session) -> int:
"""Find and compute all uncalculated closed 15-minute periods.
A period ``[t0, t1)`` is *closed* when ``t1 ≤ now``. This function:
1. Determines the lookback window: from ``now LOOKBACK_DAYS`` to ``now``,
floored to the nearest quarter-hour. This avoids an unbounded full
historical scan on every tick while still covering the typical recovery
window (short outages, missing contract, etc.). Periods older than
``LOOKBACK_DAYS`` must be recovered via an explicit ``recompute_range``.
2. Iterates over all quarter-hour boundaries in that window where
``t1 ≤ now`` (i.e. the period has already closed).
3. For each boundary, calls ``compute_period(overwrite=False)``, which:
- Skips periods that already have a *successful* (non-degraded) record.
- Retries periods that have a *degraded* record.
- Writes degraded rows for periods with no meter or cross-meter boundaries.
- Skips periods for which no active contract version exists or the
Tibber price is unavailable (without writing a degraded row).
Returns
-------
int
Number of periods for which a record was written (inserted or updated).
Does not count skipped periods.
"""
now = datetime.now(UTC)
# Current period boundary (the one whose t1 has not yet passed).
current_t0 = floor_to_quarter(now)
# Earliest boundary to consider.
earliest_t0 = floor_to_quarter(now - timedelta(days=_LOOKBACK_DAYS))
written = 0
t0 = earliest_t0
while t0 < current_t0:
t1 = t0 + timedelta(minutes=_PERIOD_MINUTES)
if t1 <= now:
try:
did_write = compute_period(session, t0, overwrite=False)
if did_write:
written += 1
except Exception:
logger.exception(
"compute_closed_periods: unexpected error for t0=%s — continuing.",
t0.isoformat(),
)
t0 += timedelta(minutes=_PERIOD_MINUTES)
if written:
session.commit()
logger.info("compute_closed_periods: wrote %d period(s).", written)
return written
# ---------------------------------------------------------------------------
# recompute_range — explicit full recompute
# ---------------------------------------------------------------------------
def recompute_range(session: Session, start: datetime, end: datetime) -> int:
"""Recompute (overwrite) all 15-minute periods in ``[start, end)``.
This is the *explicit opt-in* path for recovering from:
- Periods where readings or prices arrived late.
- Price corrections (new contract version retroactively applied).
- Retroactive meter changes (``update_meter`` with new ``started_at``) —
re-running this function will re-judge meter attribution and re-compute
costs using the corrected epoch boundaries.
- Any other reason to override the immutability guard.
The function iterates over every UTC quarter-hour boundary in
``[floor(start), end)`` and calls ``compute_period(overwrite=True)``.
Existing rows (including successful ones) are overwritten; their
``meter_id`` fields will reflect the *current* ``meter_at`` judgment for
each period's start timestamp, naturally re-attributing periods when meter
``started_at`` values have been retroactively corrected.
Parameters
----------
session:
Active SQLAlchemy session. The function commits after all periods
have been processed.
start:
Inclusive start datetime (floored to the nearest quarter-hour internally).
end:
Exclusive end datetime.
Returns
-------
int
Number of periods for which a record was written (inserted or updated).
Periods skipped due to missing contract or missing Tibber price are
*not* counted.
"""
t0 = floor_to_quarter(_as_utc(start))
end_utc = _as_utc(end)
now = datetime.now(UTC)
written = 0
while t0 < end_utc:
t1 = t0 + timedelta(minutes=_PERIOD_MINUTES)
# Only recompute periods that have already closed (t1 ≤ now). Even
# when the caller passes a future *end*, we must not write speculative
# "future" rows: register_at would resolve to the latest historical
# reading for both boundaries, producing a zero-delta fake-success row
# that blocks the real computation once the period actually closes.
if t1 <= now:
try:
did_write = compute_period(session, t0, overwrite=True)
if did_write:
written += 1
except Exception:
logger.exception(
"recompute_range: unexpected error for t0=%s — continuing.",
t0.isoformat(),
)
t0 += timedelta(minutes=_PERIOD_MINUTES)
session.commit()
logger.info(
"recompute_range(%s, %s): wrote %d period(s).",
start.isoformat(),
end.isoformat(),
written,
)
return written
# ---------------------------------------------------------------------------
# summarize — layer-2 aggregation (read-time, not stored)
# ---------------------------------------------------------------------------
def summarize(session: Session, start: datetime, end: datetime) -> dict[str, Any]:
"""Aggregate billing for the interval ``[start, end)``.
Computes the total payable as:
total_payable = Σ(net_cost) -- metered electricity
+ fixed_costs -- per-day standing charges, cross-version
- credits -- per-day heffingskorting, cross-version
**Fixed-cost / credit counting — Principle C (symmetric begin/end)**:
Both the local calendar day in which *start* falls and the local calendar
day in which *end* falls are counted as full days. Fixed charges
(network_fee, management_fee) and the energy-tax credit (heffingskorting)
are assessed on a "service-is-active" basis — if the meter was online on a
given calendar day, the full day's charge/credit applies, regardless of
whether the window starts at midnight or mid-morning.
For each local calendar date D in the range
``[local_date(start), min(local_date(end), today_local)]``:
- D ≤ today_local (only elapsed / today days count as "whole days").
- The contract version whose effective_from local-date ≤ D is used.
- This is cross-version: if V1 is from June 1 and V2 from June 25,
querying June 130 uses V1 for days 1-24 and V2 for day 25.
The end-day (last_counted) is included when the end's local midnight falls
strictly before end_utc; combined with the always-counted start day this
makes the begin/end handling symmetric. A short same-day window therefore
counts its single local day. A window contributes 0 days only when the
counted range is empty (first_counted > last_counted) — e.g. a window lying
entirely in the future, since last_counted is capped at today_local.
Daily getters (``*_today``) use windows exactly aligned to local midnight,
so their ``first_counted`` is always today — unaffected by this fix.
Days that have not yet started in local time (D > today_local) are
never counted. Switching versions never resets the counter.
**Timezone note**: the ``days`` field in the returned dict still represents
the window length in calendar days (total_seconds / 86400), for backward
compatibility with existing API consumers. The fixed/credit calculation
independently counts whole local days as described above.
All arithmetic uses Decimal; the returned dict contains Python floats for
JSON-serialisation convenience.
Parameters
----------
session:
Active read-only SQLAlchemy session.
start:
Inclusive start of the summary interval (UTC or naive-UTC).
end:
Exclusive end of the summary interval (UTC or naive-UTC).
Returns
-------
dict with keys:
currency str ISO 4217 currency (from contract, or "EUR" fallback)
metered_import float Σ import_cost from non-degraded periods
metered_export float Σ export_revenue from non-degraded periods
metered_net float Σ net_cost from non-degraded periods
fixed_costs float standing charges for elapsed whole local days
credits float energy-tax credit for elapsed whole local days
total_payable float metered_net + fixed_costs credits
period_count int number of non-degraded periods in range
degraded_count int number of degraded periods in range
days float interval length in days (total_seconds / 86400)
"""
from datetime import timedelta as _td, date as _date
start_utc = _as_utc(start)
end_utc = _as_utc(end)
# --- Fetch all EnergyCostPeriod rows in [start, end) ---
rows = session.execute(
select(EnergyCostPeriod).where(
EnergyCostPeriod.period_start >= start_utc,
EnergyCostPeriod.period_start < end_utc,
)
).scalars().all()
good_rows = [r for r in rows if not r.degraded]
degraded_rows = [r for r in rows if r.degraded]
# Σ monetary amounts (Decimal arithmetic).
sum_import = sum((_to_decimal(r.import_cost) for r in good_rows), Decimal("0"))
sum_export = sum((_to_decimal(r.export_revenue) for r in good_rows), Decimal("0"))
sum_net = sum((_to_decimal(r.net_cost) for r in good_rows), Decimal("0"))
# --- Interval length in days (window, not elapsed — kept for API compat) ---
total_seconds = (end_utc - start_utc).total_seconds()
days = _to_decimal(str(total_seconds)) / _to_decimal("86400")
# --- Fixed costs and credits: Principle C cross-version whole-day counting ---
today_local: _date = local_now().date()
# --- Compute [first_counted, last_counted] local date range (inclusive) ---
#
# Both the window-start day and the window-end day are counted as complete
# local calendar days, regardless of whether the window starts/ends at midnight.
#
# Principle C (symmetric begin/end):
# • first_counted = local calendar date of start_utc (start day always counted)
# • last_counted = local calendar date of end_utc (end day counted if its
# local midnight is strictly before end_utc)
# • Both are then capped at today_local (only elapsed / today days count).
#
# Why symmetric? Fixed charges (network_fee, management_fee) and energy-tax credits
# (heffingskorting) are assessed on a "service-is-active" basis, not on how many
# hours the service was actually running within that calendar day. If the meter
# anchor (started_at) falls at 09:18 on June 24, the full June 24 standing charge
# and credit still apply because the service was online for that day.
#
# Previous asymmetric behaviour: a start_utc that was *later* than the local
# midnight of local_start_date caused first_counted to be bumped to the *next*
# day, silently dropping the anchor day's charges/credits. This was incorrect
# for cumulative entities (import_cost_total / export_revenue_total) whose anchor
# is often an above-midnight started_at. The end-side had always been symmetric
# (counted if local midnight < end_utc), creating an inconsistency.
#
# Daily getters (window = [local today 00:00, local tomorrow 00:00)) are
# unaffected: local_date(local_midnight_utc(today)) == today, so first_counted
# is still today regardless of the fix.
#
# A short same-day window now counts its single local day (the start day is
# always counted, symmetric with the end side). A window contributes 0 days
# only when the counted range is empty (first_counted > last_counted) — e.g. a
# window lying entirely in the future, where last_counted is capped at
# today_local while first_counted is later.
from app.services.timezone import local_midnight_utc as _lmu
# Start side: always count the local calendar day in which start_utc falls.
first_counted: _date = local_date(start_utc)
local_end_date: _date = local_date(end_utc)
# Is the midnight of local_end_date < end_utc? If yes, that day's midnight is in range.
if _lmu(local_end_date) < end_utc:
last_counted: _date = local_end_date
else:
last_counted = local_end_date - _td(days=1)
# Cap: only elapsed or today's days.
last_counted = min(last_counted, today_local)
# If the range is empty (first_counted > last_counted), no days are counted.
# Fetch all versions of the active contract once (single DB call).
versions = active_contract_versions(session)
fixed_dec = Decimal("0")
credits_dec = Decimal("0")
currency = "EUR"
if versions:
# Currency comes from the contract regardless of day count.
currency = versions[0].contract.currency
if versions and first_counted <= last_counted:
# For each version segment, find its overlap with [first_counted, last_counted]
# and accumulate whole days.
version_segments: list[tuple[_date, _date | None, dict]] = []
for v in versions:
v_start_local = local_date(_as_utc(v.effective_from))
v_end_local = local_date(_as_utc(v.effective_to)) if v.effective_to is not None else None
version_segments.append((v_start_local, v_end_local, v.values or {}))
for v_start, v_end_excl, v_values in version_segments:
# The version covers [v_start, v_end_excl) in local dates,
# where v_end_excl=None means open-ended (no upper bound).
# Intersection with [first_counted, last_counted]:
seg_start = max(first_counted, v_start)
if v_end_excl is not None:
seg_end_excl = min(last_counted + _td(days=1), v_end_excl)
else:
seg_end_excl = last_counted + _td(days=1)
if seg_start >= seg_end_excl:
continue # no overlap
n_days = (seg_end_excl - seg_start).days
if n_days <= 0:
continue
standing: dict = v_values.get("standing", {})
creds: dict = v_values.get("credits", {})
network_fee = _to_decimal(standing.get("network_fee", 0))
management_fee = _to_decimal(standing.get("management_fee", 0))
heffingskorting = _to_decimal(creds.get("heffingskorting", 0))
# Standing charges: EUR/month → EUR/day (÷ 30) × n_days.
fixed_dec += (network_fee + management_fee) / Decimal("30") * Decimal(str(n_days))
# Energy-tax credit: EUR/year → EUR/day (÷ 365) × n_days.
credits_dec += heffingskorting / Decimal("365") * Decimal(str(n_days))
total_payable = sum_net + fixed_dec - credits_dec
return {
"currency": currency,
"metered_import": float(sum_import),
"metered_export": float(sum_export),
"metered_net": float(sum_net),
"fixed_costs": float(fixed_dec),
"credits": float(credits_dec),
"total_payable": float(total_payable),
"period_count": len(good_rows),
"degraded_count": len(degraded_rows),
"days": float(days),
}
+499
View File
@@ -0,0 +1,499 @@
"""HA MQTT Discovery service — build and publish discovery configs + state.
This module handles:
1. Building HA MQTT Discovery payloads for each ``ExposableEntity``.
2. Publishing discovery configs (retained) for enabled entities.
3. Clearing (empty-payload) discovery configs for disabled entities.
4. Publishing current entity state values.
5. A per-device helper for use in the Modbus poll loop.
Design
------
- **No-op when MQTT / discovery is not configured**: every public function
checks ``ha_discovery_enabled`` and ``mqtt_manager.is_connected`` before
doing any work. Callers never need to guard these conditions.
- **Stable unique_id**: derived from device ``uuid`` + metric ``key``, never
from mutable fields like ``friendly_name`` or auto-increment DB ids.
- **Discovery topic format**: ``<discovery_prefix>/<component>/<node_id>/<object_id>/config``
where ``node_id`` is the device uuid (slugified to be safe) and
``object_id`` is a uuid-prefixed stable string. Uses ``ha_discovery_prefix``
(default ``"homeassistant"``), which must stay under the HA discovery namespace.
- **State topic format**: ``<state_prefix>/<component>/<node_id>/<object_id>/state``
Uses ``ha_state_topic_prefix`` (default ``"home_automation"``), separate from
the discovery namespace so HA discovery and state/availability topics live under
different prefixes.
- **Availability topic**: ``<state_prefix>/modbus/<node_id>/availability`` (shared
across all entities of the same device; publishes "online"/"offline"). Also uses
the state prefix, not the discovery prefix.
- **best-effort**: functions catch all exceptions internally so that callers
(e.g. the Modbus poll loop) never crash due to MQTT publish errors.
"""
from __future__ import annotations
import json
import logging
from typing import Any
from sqlalchemy.orm import Session
from app.integrations.expose import ExposableEntity, build_catalog
from app.integrations.mqtt import mqtt_manager
from app.services.config_page import build_runtime_settings
logger = logging.getLogger(__name__)
# ---------------------------------------------------------------------------
# Internal helpers
# ---------------------------------------------------------------------------
def _should_publish(settings: Any) -> bool:
"""Return True only if MQTT and HA Discovery are both enabled and connected."""
return bool(
settings.mqtt_enabled
and settings.ha_discovery_enabled
and mqtt_manager.is_connected
)
def _node_id(device_uuid: str) -> str:
"""Stable MQTT node_id derived from device uuid (replace hyphens for safety)."""
return device_uuid.replace("-", "_")
def _object_id(entity: ExposableEntity) -> str:
"""Stable MQTT object_id derived from entity key (dots + hyphens → underscores)."""
return entity.key.replace(".", "_").replace("-", "_")
def _discovery_topic(entity: ExposableEntity, prefix: str) -> str:
"""Build the HA Discovery config topic for *entity*.
Format: ``<prefix>/<component>/<node_id>/<object_id>/config``
"""
node = _node_id(entity.device.identifiers[1]) # device uuid
obj = _object_id(entity)
return f"{prefix}/{entity.component}/{node}/{obj}/config"
def _state_topic(entity: ExposableEntity, prefix: str) -> str:
"""Build the state publish topic for *entity*.
Format: ``<prefix>/<component>/<node_id>/<object_id>/state``
"""
node = _node_id(entity.device.identifiers[1])
obj = _object_id(entity)
return f"{prefix}/{entity.component}/{node}/{obj}/state"
def _availability_topic(device_uuid: str, prefix: str) -> str:
"""Shared availability topic for all entities of a device.
Format: ``<prefix>/modbus/<node_id>/availability``
"""
node = _node_id(device_uuid)
return f"{prefix}/modbus/{node}/availability"
def _unique_id(entity: ExposableEntity) -> str:
"""Stable unique_id — device uuid + metric key (never from mutable fields)."""
device_uuid = entity.device.identifiers[1]
# entity.key is already "modbus.<uuid>.<metric_key>" — use it as the seed
return f"{device_uuid}_{entity.key.replace('.', '_')}"
# ---------------------------------------------------------------------------
# Public: build discovery payload
# ---------------------------------------------------------------------------
def build_discovery_payload(
entity: ExposableEntity,
discovery_prefix: str,
state_prefix: str | None = None,
) -> tuple[str, dict[str, Any]]:
"""Build the HA MQTT Discovery config topic and payload dict for *entity*.
Parameters
----------
entity:
An ``ExposableEntity`` (from ``build_catalog``).
discovery_prefix:
The HA Discovery config topic prefix (e.g. ``"homeassistant"``).
This determines where HA looks for the discovery config topic.
state_prefix:
The prefix used for state and availability topics (e.g.
``"home_automation"``). When omitted or ``None``, falls back to
``discovery_prefix`` for backward compatibility (e.g. in tests that
pass only one prefix).
Returns
-------
tuple[str, dict]
``(topic, config_dict)`` — the config topic and the HA Discovery
payload (not yet JSON-serialised).
"""
if state_prefix is None:
state_prefix = discovery_prefix
device_uuid = entity.device.identifiers[1]
avail_topic = _availability_topic(device_uuid, state_prefix)
state_t = _state_topic(entity, state_prefix)
topic = _discovery_topic(entity, discovery_prefix)
config: dict[str, Any] = {
"unique_id": _unique_id(entity),
"name": entity.name,
"state_topic": state_t,
"device": {
"identifiers": list(entity.device.identifiers),
"name": entity.device.name,
},
}
# Only declare an availability topic for devices that actually publish an
# online/offline heartbeat (e.g. Modbus, via its "online" binary_sensor).
# Devices without a heartbeat (e.g. energy-cost) omit availability so HA
# treats their entities as always-available — otherwise HA would mark them
# ``unavailable`` forever even though their state is being published.
if entity.device.provides_availability:
config["availability"] = [{"topic": avail_topic}]
config["availability_mode"] = "all"
# Component-specific fields
if entity.component == "binary_sensor":
# "online" binary sensor: payload ON/OFF
config["payload_on"] = "ON"
config["payload_off"] = "OFF"
if entity.device_class:
config["device_class"] = entity.device_class
else:
# sensor (and others)
if entity.device_class:
config["device_class"] = entity.device_class
if entity.unit:
config["unit_of_measurement"] = entity.unit
if entity.state_class:
config["state_class"] = entity.state_class
return topic, config
# ---------------------------------------------------------------------------
# Public: publish discovery
# ---------------------------------------------------------------------------
def publish_discovery(session: Session) -> None:
"""Publish HA Discovery configs for all entities in the catalog.
- Enabled entities: publish retained JSON config payload.
- Disabled entities: publish empty (b"") payload to clear any previously
retained config from HA.
No-op if MQTT / discovery is not enabled or the client is not connected.
All MQTT errors are caught internally — this function never raises.
"""
from app.config import get_settings
settings = build_runtime_settings(session, get_settings())
if not _should_publish(settings):
logger.debug("publish_discovery: skipped (MQTT/Discovery not enabled or not connected)")
return
discovery_prefix = settings.ha_discovery_prefix
state_prefix = settings.ha_state_topic_prefix
try:
catalog = build_catalog(session)
except Exception:
logger.exception("publish_discovery: failed to build catalog; aborting")
return
for entry in catalog:
entity = entry.entity
try:
topic, config = build_discovery_payload(entity, discovery_prefix, state_prefix)
if entry.enabled:
payload = json.dumps(config)
mqtt_manager.publish(topic, payload, retain=True)
logger.debug(
"publish_discovery: published config for %r%s", entity.key, topic
)
else:
# Clear retained config for disabled entities.
mqtt_manager.publish(topic, b"", retain=True)
logger.debug(
"publish_discovery: cleared config for disabled entity %r%s",
entity.key,
topic,
)
except Exception:
logger.exception(
"publish_discovery: error processing entity %r; continuing", entity.key
)
# ---------------------------------------------------------------------------
# Public: publish states
# ---------------------------------------------------------------------------
def publish_states(session: Session) -> None:
"""Publish current state values for all enabled entities.
Calls each entity's ``value_getter`` to obtain the current value and
publishes it to the entity's state topic. Also publishes availability
(online/offline) for each device.
No-op if MQTT / discovery is not enabled or the client is not connected.
All errors are caught internally — this function never raises.
"""
from app.config import get_settings
settings = build_runtime_settings(session, get_settings())
if not _should_publish(settings):
logger.debug("publish_states: skipped (MQTT/Discovery not enabled or not connected)")
return
state_prefix = settings.ha_state_topic_prefix
try:
catalog = build_catalog(session)
except Exception:
logger.exception("publish_states: failed to build catalog; aborting")
return
# Collect enabled entries and publish
for entry in catalog:
if not entry.enabled:
continue
entity = entry.entity
try:
_publish_entity_state(entity, state_prefix, session)
except Exception:
logger.exception(
"publish_states: error publishing state for %r; continuing", entity.key
)
def _publish_entity_state(
entity: ExposableEntity,
prefix: str,
session: Session,
) -> None:
"""Publish one entity's current state value to its state topic.
Also publishes the availability topic for ``binary_sensor`` "online" entities.
"""
state_t = _state_topic(entity, prefix)
device_uuid = entity.device.identifiers[1]
if entity.component == "binary_sensor" and "online" in entity.key:
# The online sensor represents device availability.
# Ask the value_getter for the current ON/OFF string, then also publish
# the shared availability topic (online/offline).
raw_value = None
if entity.value_getter is not None:
try:
raw_value = entity.value_getter(session)
except Exception:
logger.exception("value_getter raised for online entity %r", entity.key)
# Default to offline when no reading is available.
online = (raw_value == "ON")
avail_payload = "online" if online else "offline"
avail_topic = _availability_topic(device_uuid, prefix)
mqtt_manager.publish(avail_topic, avail_payload, retain=False)
# The state of the binary_sensor itself
state_payload = "ON" if online else "OFF"
mqtt_manager.publish(state_t, state_payload, retain=False)
else:
# Regular sensor: call value_getter(session) if available.
value = None
if entity.value_getter is not None:
try:
value = entity.value_getter(session)
except Exception:
logger.exception("value_getter raised for entity %r", entity.key)
if value is None:
logger.debug("publish_states: no value for %r — skipping state publish", entity.key)
return
mqtt_manager.publish(state_t, str(value), retain=False)
# ---------------------------------------------------------------------------
# Public: per-device state push (called by modbus_poll after each poll)
# ---------------------------------------------------------------------------
def publish_device_state(session: Session, device: Any) -> None:
"""Publish state + availability for all enabled entities of *device*.
Called by ``modbus_poll.poll_device`` after a successful (or failed) poll.
Publishes:
- The current availability topic ("online" / "offline") for the device.
- The state topic for each **enabled** entity of the device.
No-op if MQTT / discovery is not enabled or the client is not connected.
All errors are caught internally — never raises.
"""
from app.config import get_settings
settings = build_runtime_settings(session, get_settings())
if not _should_publish(settings):
return
state_prefix = settings.ha_state_topic_prefix
device_uuid = device.uuid
online = bool(device.last_poll_ok)
# Publish availability topic first.
try:
avail_topic = _availability_topic(device_uuid, state_prefix)
avail_payload = "online" if online else "offline"
mqtt_manager.publish(avail_topic, avail_payload, retain=False)
except Exception:
logger.exception(
"publish_device_state: failed to publish availability for device uuid=%s", device_uuid
)
if not online:
# Device is offline — no point pushing stale state values.
return
# Publish state for each enabled entity of this device.
try:
catalog = build_catalog(session)
except Exception:
logger.exception(
"publish_device_state: failed to build catalog for device uuid=%s", device_uuid
)
return
for entry in catalog:
if not entry.enabled:
continue
entity = entry.entity
# Only process entities belonging to this device.
if entity.device.identifiers[1] != device_uuid:
continue
# Skip the online binary_sensor itself (availability handled above).
if entity.component == "binary_sensor" and "online" in entity.key:
# Publish the binary_sensor state too.
try:
state_t = _state_topic(entity, state_prefix)
mqtt_manager.publish(state_t, "ON", retain=False)
except Exception:
logger.exception(
"publish_device_state: error publishing online sensor state for %r",
entity.key,
)
continue
# Regular sensor — call value_getter(session).
try:
value = None
if entity.value_getter is not None:
value = entity.value_getter(session)
if value is None:
continue
state_t = _state_topic(entity, state_prefix)
mqtt_manager.publish(state_t, str(value), retain=False)
except Exception:
logger.exception(
"publish_device_state: error publishing state for entity %r", entity.key
)
def clear_device_discovery(session: Session, device_uuid: str) -> None:
"""Best-effort: send empty retained payloads to all HA Discovery config topics
for the given device, effectively removing the device's entities from HA.
This must be called **before** the device rows are deleted from the DB, so
that ``build_catalog`` can still enumerate the device's entities.
Behaviour
---------
- No-op if MQTT / HA Discovery is not enabled or the MQTT client is not
connected.
- All exceptions are caught internally; this function never raises.
The caller proceeds with the DB deletion regardless of MQTT outcome.
Parameters
----------
session:
Active SQLAlchemy session (device must still exist in DB at call time).
device_uuid:
UUID string of the device being deleted.
"""
from app.config import get_settings
settings = build_runtime_settings(session, get_settings())
if not _should_publish(settings):
logger.debug(
"clear_device_discovery: skipped (MQTT/Discovery not enabled or not connected)"
)
return
discovery_prefix = settings.ha_discovery_prefix
state_prefix = settings.ha_state_topic_prefix
try:
catalog = build_catalog(session)
except Exception:
logger.exception(
"clear_device_discovery: failed to build catalog for device uuid=%s; skipping HA cleanup",
device_uuid,
)
return
cleared = 0
for entry in catalog:
entity = entry.entity
# Only clear entities belonging to this device.
if entity.device.identifiers[1] != device_uuid:
continue
try:
topic, _config = build_discovery_payload(entity, discovery_prefix, state_prefix)
mqtt_manager.publish(topic, b"", retain=True)
cleared += 1
logger.debug(
"clear_device_discovery: cleared config topic for entity %r%s",
entity.key,
topic,
)
except Exception:
logger.exception(
"clear_device_discovery: error clearing entity %r; continuing", entity.key
)
logger.info(
"clear_device_discovery: cleared %d HA discovery topic(s) for device uuid=%s",
cleared,
device_uuid,
)
def publish_device_offline(session: Session, device_uuid: str) -> None:
"""Publish an "offline" availability payload for *device_uuid*.
Called by ``modbus_poll.poll_device`` when a poll fails, to immediately
reflect the device going offline in HA (before the next full state sweep).
No-op if MQTT / discovery is not enabled or the client is not connected.
Never raises.
"""
from app.config import get_settings
settings = build_runtime_settings(session, get_settings())
if not _should_publish(settings):
return
try:
state_prefix = settings.ha_state_topic_prefix
avail_topic = _availability_topic(device_uuid, state_prefix)
mqtt_manager.publish(avail_topic, "offline", retain=False)
except Exception:
logger.exception(
"publish_device_offline: failed for device uuid=%s", device_uuid
)
+179
View File
@@ -0,0 +1,179 @@
"""Exponential back-off throttle service for the login endpoint.
Design
------
Failures are tracked per (scope, key) in the ``auth_login_throttle`` table:
* scope ``'ip'`` → key is the client IP address
* scope ``'user'`` → key is the username
On each login attempt the endpoint checks **both** keys and takes the **maximum**
remaining wait (in seconds). A wait > 0 means the caller is still inside a
back-off window and should receive 429.
Exponential formula
-------------------
The first ``N_FREE`` failures are free (no delay). After that::
wait = min(BACKOFF_CAP_S, BACKOFF_BASE_S * 2 ** (failures - N_FREE))
Constants (module-level so they can be tuned without touching the algorithm):
* ``N_FREE`` = 3 — failures before any delay starts
* ``BACKOFF_BASE_S`` = 1 — base wait in seconds (1st delayed attempt → 2 s)
* ``BACKOFF_CAP_S`` = 900 — hard cap (15 min)
Example delay schedule:
failures 1, 2, 3 → 0 s (free)
failures 4 → 2 s
failures 5 → 4 s
failures 6 → 8 s
failures 7 → 16 s
failures 13+ → 900 s (capped)
Timezone notes
--------------
The model uses ``DateTime(timezone=True)``. SQLite stores timestamps without
timezone info, so when rows are read back the column value may be timezone-naive.
We normalise every stored value to UTC-aware via ``_as_utc`` before comparing
against ``datetime.now(UTC)``.
"""
from __future__ import annotations
import math
from datetime import UTC, datetime, timedelta
from sqlalchemy import delete, select
from sqlalchemy.orm import Session
from app.models.auth_throttle import LoginThrottle
# ---------------------------------------------------------------------------
# Tunable constants
# ---------------------------------------------------------------------------
N_FREE: int = 3 # failures before any backoff starts
BACKOFF_BASE_S: int = 1 # seconds — wait after the first delayed failure
BACKOFF_CAP_S: int = 900 # seconds — maximum wait (15 min)
# Internal scope labels (keep in sync with model constraint)
_SCOPE_IP = "ip"
_SCOPE_USER = "user"
# ---------------------------------------------------------------------------
# Public API
# ---------------------------------------------------------------------------
def check_and_get_wait(session: Session, *, ip: str, username: str) -> int:
"""Return the number of seconds the caller must still wait, or 0 if allowed.
Checks both the IP row and the username row; returns the *larger* remaining
wait so that both keys must have their back-off window expire before the
caller is permitted to try again.
"""
now = datetime.now(UTC)
wait_ip = _remaining_wait(session, scope=_SCOPE_IP, key=ip, now=now)
wait_user = _remaining_wait(session, scope=_SCOPE_USER, key=username, now=now)
return max(wait_ip, wait_user)
def register_failure(session: Session, *, ip: str, username: str) -> None:
"""Record one login failure for both the IP and the username keys.
Updates (or creates) the throttle row for each key, increments the failure
counter, and recomputes ``next_allowed_at`` using the exponential formula.
"""
now = datetime.now(UTC)
_upsert_failure(session, scope=_SCOPE_IP, key=ip, now=now)
_upsert_failure(session, scope=_SCOPE_USER, key=username, now=now)
session.commit()
def clear(session: Session, *, ip: str, username: str) -> None:
"""Delete the throttle rows for the given IP and username (successful login).
It is safe to call this even if no rows exist for the given keys.
"""
session.execute(
delete(LoginThrottle).where(
(LoginThrottle.scope == _SCOPE_IP) & (LoginThrottle.key == ip)
| (LoginThrottle.scope == _SCOPE_USER) & (LoginThrottle.key == username)
)
)
session.commit()
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _remaining_wait(session: Session, *, scope: str, key: str, now: datetime) -> int:
"""Return the number of seconds that ``key`` in ``scope`` must still wait.
Returns 0 if the key has no throttle row or if the back-off window has
already expired.
"""
row = session.scalar(
select(LoginThrottle)
.where(LoginThrottle.scope == scope, LoginThrottle.key == key)
.limit(1)
)
if row is None or row.next_allowed_at is None:
return 0
next_allowed = _as_utc(row.next_allowed_at)
if next_allowed <= now:
return 0
remaining = (next_allowed - now).total_seconds()
# Round up so callers are never told "0 seconds" while still blocked.
return math.ceil(remaining)
def _compute_next_allowed_at(failures: int, now: datetime) -> datetime | None:
"""Compute the earliest time the key should be allowed to try again.
Returns ``None`` for the first ``N_FREE`` failures (no delay).
"""
if failures <= N_FREE:
return None
wait_s = min(BACKOFF_CAP_S, BACKOFF_BASE_S * (2 ** (failures - N_FREE)))
return now + timedelta(seconds=wait_s)
def _upsert_failure(session: Session, *, scope: str, key: str, now: datetime) -> None:
"""Create or update the throttle row for (scope, key) with one more failure."""
row = session.scalar(
select(LoginThrottle)
.where(LoginThrottle.scope == scope, LoginThrottle.key == key)
.limit(1)
)
if row is None:
new_failures = 1
row = LoginThrottle(
scope=scope,
key=key,
failures=new_failures,
first_failed_at=now,
last_failed_at=now,
next_allowed_at=_compute_next_allowed_at(new_failures, now),
)
session.add(row)
else:
row.failures += 1
row.last_failed_at = now
row.next_allowed_at = _compute_next_allowed_at(row.failures, now)
def _as_utc(value: datetime | None) -> datetime | None:
"""Normalise a possibly-naive datetime to UTC-aware."""
if value is None:
return None
if value.tzinfo is None:
return value.replace(tzinfo=UTC)
return value.astimezone(UTC)
+480
View File
@@ -0,0 +1,480 @@
"""Service layer for Meter epoch CRUD, swap/close, and time-range lookup.
All functions accept an explicit SQLAlchemy Session; callers are responsible
for committing or rolling back the transaction.
Design decisions
----------------
- ``meter_at``: half-open interval ``[started_at, ended_at)`` lookup;
returns the meter whose epoch covers *ts* for the given commodity.
SQLite naive datetime is normalised via ``_as_utc()`` before comparison.
- ``declare_meter``: validates that the new ``started_at`` is **not earlier
than** the current active meter's ``started_at`` (rejects back-dating below
the active meter's own start). Equal timestamps are allowed because the
typical "swap now" use-case sets ``started_at`` to the current moment, which
coincides with the active meter's ``started_at`` only in degenerate test
scenarios — but blocking equal values would make that workflow impossible.
The old active meter is closed (``ended_at = started_at``) and a new active
meter is opened in the same operation, guaranteeing continuity: the old
meter's ``ended_at`` equals the new meter's ``started_at`` (contiguous,
no gap, no overlap).
- ``update_meter``: when ``started_at`` is modified, the service keeps the
timeline contiguous by also updating the **previous** meter's ``ended_at``
(the one whose ``ended_at`` matched the old ``started_at``) to the new
``started_at``. Validation ensures the new ``started_at``:
* is strictly after the previous meter's own ``started_at``
(cannot push the boundary before the previous meter even started);
* is strictly before the current meter's ``ended_at``, if set
(cannot push the boundary past where the current meter was already
closed).
- Mutual exclusion (at most one active meter per commodity) is enforced by the
service layer; no DB-level unique partial index is added to keep migrations
simple and to allow the application to return a meaningful error message.
SQLite timezone note
--------------------
SQLite stores ``DateTime(timezone=True)`` columns as naive UTC strings; on
read-back they come out as **timezone-naive** datetimes. Wherever this code
compares timestamps from the DB against timezone-aware values, it calls
``_as_utc()`` to make both sides comparable without tripping on
"offset-naive vs offset-aware" TypeErrors.
"""
from __future__ import annotations
import logging
from datetime import UTC, datetime
from typing import Optional
from sqlalchemy import select
from sqlalchemy.orm import Session
from app.models.energy import Meter
logger = logging.getLogger(__name__)
# ---------------------------------------------------------------------------
# Internal helpers
# ---------------------------------------------------------------------------
def _as_utc(dt: datetime) -> datetime:
"""Return *dt* as a timezone-aware UTC datetime.
SQLite's ``DateTime(timezone=True)`` column type stores datetimes as naive
UTC strings and gives them back as naive datetimes on read. This helper
re-attaches the UTC timezone info when it is missing, making cross-origin
comparisons safe.
"""
if dt.tzinfo is None:
return dt.replace(tzinfo=UTC)
return dt
# ---------------------------------------------------------------------------
# Custom exceptions
# ---------------------------------------------------------------------------
class MeterError(ValueError):
"""Base class for meter service validation errors."""
class MeterOverlapError(MeterError):
"""Raised when a new meter's ``started_at`` would create an overlap or backdate.
Specifically: the new meter's ``started_at`` must be greater than or equal
to the current active meter's ``started_at``. Allowing a value strictly
earlier than the active meter's start would mean the new epoch begins before
the current one, which is chronologically inconsistent.
"""
class MeterIntervalError(MeterError):
"""Raised when an ``update_meter`` call would produce an inconsistent interval.
Examples of inconsistent intervals:
- New ``started_at`` ≥ this meter's ``ended_at`` (epoch would be empty/inverted).
- New ``started_at`` ≤ the previous meter's own ``started_at`` (previous meter
would become empty/inverted after its ``ended_at`` is updated).
"""
# ---------------------------------------------------------------------------
# Internal query helpers
# ---------------------------------------------------------------------------
def _active_meter(session: Session, commodity: str) -> Optional[Meter]:
"""Return the current active (``ended_at IS NULL``) meter for *commodity*, or None."""
return session.execute(
select(Meter)
.where(
Meter.commodity == commodity,
Meter.ended_at.is_(None),
)
.limit(1)
).scalar_one_or_none()
def _meter_before(session: Session, meter: Meter) -> Optional[Meter]:
"""Return the meter whose ``ended_at`` equals *meter*'s ``started_at``.
This is the meter that was closed when *meter* was opened; its ``ended_at``
needs to stay equal to *meter*'s ``started_at`` to maintain timeline
continuity. Returns None if *meter* is the first epoch for its commodity.
The lookup compares naive/aware datetimes via string to avoid SQLite timezone
quirks: both are formatted as ISO 8601 UTC strings for the WHERE clause.
We rely on the fact that the service layer always stores the same timestamp
object as both ``prev.ended_at`` and ``new.started_at``, so their string
representations are identical.
"""
# Normalise the target to a tz-aware UTC datetime for comparison.
# We scan in Python (rather than SQL) to avoid SQLite naive-vs-aware
# mismatch issues when comparing DateTime columns against tz-aware values.
target = _as_utc(meter.started_at)
# Fetch all closed meters of the same commodity and find the one whose
# ended_at equals this meter's started_at (the standard contiguous handoff).
candidates = session.execute(
select(Meter)
.where(
Meter.commodity == meter.commodity,
Meter.ended_at.is_not(None),
)
).scalars().all()
for candidate in candidates:
if candidate.id == meter.id:
continue
candidate_ended = _as_utc(candidate.ended_at)
if candidate_ended == target:
return candidate
return None
# ---------------------------------------------------------------------------
# Core service functions
# ---------------------------------------------------------------------------
def meter_at(
session: Session,
ts: datetime,
commodity: str = "electricity",
) -> Optional[Meter]:
"""Return the meter epoch that covers *ts* for *commodity*.
A meter covers *ts* when:
``started_at ≤ ts`` AND (``ended_at IS NULL`` OR ``ts < ended_at``)
This is the standard half-open interval ``[started_at, ended_at)`` lookup,
consistent with ``active_contract_version_at`` in ``contracts.py``.
Returns ``None`` when no meter covers *ts* (e.g. before the first epoch
was declared, or after a gap).
Parameters
----------
session:
Active SQLAlchemy session (read-only usage).
ts:
UTC datetime to look up.
commodity:
Energy commodity (default ``"electricity"``).
Returns
-------
Meter | None
Implementation note — why the upper bound is pushed into SQL
------------------------------------------------------------
When ``declare_meter`` is called with ``started_at == active.started_at``
(the "equal-timestamp swap" allowed by §3.5), the old meter is closed as a
zero-width epoch ``[T0, T0)`` and the new active meter also has
``started_at == T0``. Two rows share the same ``started_at``; SQLite's
row-ordering for ``ORDER BY started_at DESC LIMIT 1`` is then determined by
rowid (i.e. insertion order), which returns the older zero-width row first.
If the upper bound were checked in Python *after* fetching that one row, the
condition ``ts < ended_at`` would be False for **any** ``ts >= T0`` (because
``ended_at == T0``), causing the function to return ``None`` and making the
new active meter permanently invisible.
Pushing both bounds into the SQL ``WHERE`` clause eliminates the ambiguity:
the zero-width row is excluded by ``ts < ended_at`` before ``LIMIT 1`` is
applied, so only the genuinely covering row survives.
SQLite naive-vs-aware datetime note: SQLAlchemy's SQLite dialect strips the
``tzinfo`` from aware datetimes when binding parameters (it does *not*
convert to UTC first). Since all datetimes in this codebase are UTC (either
naive-UTC from the DB or aware-UTC from ``datetime.now(UTC)``), stripping
the tzinfo leaves the wall-clock value unchanged and comparisons remain
correct. This is consistent with how ``contracts.py`` handles the same
situation (see OBS 2 in the M7-T02 review notes).
"""
# Push both the lower and upper bounds into the SQL WHERE clause so that
# zero-width epochs (ended_at == started_at) are excluded *before* LIMIT 1
# is applied. This prevents an equal-timestamp swap from making the new
# active meter invisible (see implementation note above).
stmt = (
select(Meter)
.where(
Meter.commodity == commodity,
Meter.started_at <= ts,
(Meter.ended_at.is_(None)) | (Meter.ended_at > ts),
)
.order_by(Meter.started_at.desc())
.limit(1)
)
return session.execute(stmt).scalar_one_or_none()
def declare_meter(
session: Session,
*,
label: str,
started_at: datetime,
reason: str,
commodity: str = "electricity",
note: Optional[str] = None,
) -> Meter:
"""Declare a meter swap or initial meter epoch.
Closes the current active meter for *commodity* (if one exists) by setting
its ``ended_at`` to *started_at*, then opens a new active meter. The
resulting timeline is **contiguous**: old meter's ``ended_at`` equals new
meter's ``started_at``.
If there is no current active meter (first-ever declaration for this
commodity), the new meter is simply opened without closing anything.
Validation
----------
- If a current active meter exists, *started_at* must be **≥** that meter's
own ``started_at``. A value strictly earlier would place the new epoch
entirely before the current active meter, which is a chronological
contradiction ("back-dating before the active epoch's start"). Raises
``MeterOverlapError`` when this constraint is violated.
Note: equal timestamps (``started_at == active.started_at``) are allowed
because that scenario effectively replaces the current meter at the same
logical moment (e.g. correcting a mis-entry), which is a valid use-case.
The old meter is then closed with ``ended_at == started_at`` (a zero-width
epoch), which is intentional and auditable.
Parameters
----------
session:
Active SQLAlchemy session. Caller must commit after this returns.
label:
Human-readable label for the new meter.
started_at:
UTC datetime at which this meter epoch starts. May be in the past.
reason:
Why this epoch was created (``"initial"``, ``"meter_swap"``,
``"home_move"``, or ``"other"``).
commodity:
Energy commodity (default ``"electricity"``).
note:
Optional free-form note.
Returns
-------
Meter
The newly created, not-yet-committed active meter.
Raises
------
MeterOverlapError
If *started_at* is strictly earlier than the current active meter's
``started_at`` (chronological backdate below the active epoch's start).
"""
active = _active_meter(session, commodity)
if active is not None:
# Reject back-dating: new started_at must be ≥ current active's started_at.
if _as_utc(started_at) < _as_utc(active.started_at):
raise MeterOverlapError(
f"New meter started_at ({started_at.isoformat()}) must be ≥ the current "
f"active {commodity!r} meter's started_at "
f"({active.started_at.isoformat()}). "
"Declare a started_at on or after the active meter's start to avoid "
"a chronologically inconsistent epoch ordering."
)
# Close the current active meter at the swap point (contiguous handoff).
active.ended_at = started_at
logger.info(
"Closed active %r meter id=%d (ended_at=%s)",
commodity,
active.id,
started_at.isoformat(),
)
now = datetime.now(UTC)
new_meter = Meter(
label=label,
commodity=commodity,
started_at=started_at,
ended_at=None,
reason=reason,
note=note,
created_at=now,
)
session.add(new_meter)
logger.info(
"Declared new %r meter %r (started_at=%s, reason=%s)",
commodity,
label,
started_at.isoformat(),
reason,
)
return new_meter
def list_meters(
session: Session,
commodity: Optional[str] = None,
) -> list[Meter]:
"""List all meter epochs, ordered by started_at ascending.
Active meters (``ended_at IS NULL``) sort naturally to the end of the
timeline since they have the latest ``started_at``. Within a single
commodity, the ascending ``started_at`` order reflects the historical
sequence of installed meters.
Parameters
----------
session:
Active SQLAlchemy session (read-only usage).
commodity:
If provided, filter to this commodity only. If ``None``, return all
meters across all commodities.
Returns
-------
list[Meter]
Meters ordered by (started_at ASC).
"""
stmt = select(Meter).order_by(Meter.started_at.asc())
if commodity is not None:
stmt = stmt.where(Meter.commodity == commodity)
return list(session.execute(stmt).scalars().all())
def update_meter(
session: Session,
meter: Meter,
*,
label: Optional[str] = None,
note: Optional[str] = None,
started_at: Optional[datetime] = None,
) -> Meter:
"""Update a meter's mutable fields (label, note, started_at).
Passing ``None`` for a field leaves it unchanged. At least one keyword
argument must be non-``None``; calling with all-``None`` is a no-op but
is not an error.
Updating ``started_at`` (retroactive correction)
------------------------------------------------
When ``started_at`` is provided the service maintains **timeline
continuity** across the adjacent meter boundaries:
1. **Previous meter's ``ended_at``** — if the meter immediately before
this one has ``ended_at == meter.started_at`` (the standard contiguous
handoff), its ``ended_at`` is updated to the new ``started_at`` so the
boundary between the two epochs stays seamless.
2. **Validation** — the new ``started_at`` is checked for consistency:
a. It must be **strictly after** the previous meter's own ``started_at``
(otherwise the previous meter's epoch would collapse to zero or
invert).
b. It must be **strictly before** this meter's ``ended_at`` (if set),
so this meter's epoch remains non-empty.
Note: triggering a billing recompute (``recompute_range``) after a
retroactive ``started_at`` change is **out of scope** for this service
layer; that is the API layer's responsibility (M7-T05).
Parameters
----------
session:
Active SQLAlchemy session. Caller must commit after this returns.
meter:
The ``Meter`` ORM instance to update (already loaded from the session).
label:
New human-readable label; ``None`` = keep existing.
note:
New free-form note; ``None`` = keep existing.
started_at:
New start timestamp for this meter epoch; ``None`` = keep existing.
Returns
-------
Meter
The updated ``Meter`` instance (not yet committed).
Raises
------
MeterIntervalError
If the new ``started_at`` would produce an invalid (empty or inverted)
epoch for this meter or the immediately preceding one.
"""
if label is not None:
meter.label = label
logger.info("Updated meter id=%d label=%r", meter.id, label)
if note is not None:
meter.note = note
logger.info("Updated meter id=%d note=%r", meter.id, note)
if started_at is not None:
old_started_at = meter.started_at
# --- Validate upper bound: new started_at must be < this meter's ended_at (if set).
if meter.ended_at is not None:
if _as_utc(started_at) >= _as_utc(meter.ended_at):
raise MeterIntervalError(
f"New started_at ({started_at.isoformat()}) must be strictly before "
f"this meter's ended_at ({meter.ended_at.isoformat()}). "
"The meter epoch would become empty or inverted."
)
# --- Find the immediately preceding meter (its ended_at == meter's old started_at).
prev = _meter_before(session, meter)
# --- Validate lower bound: new started_at must be strictly after prev's started_at.
if prev is not None:
if _as_utc(started_at) <= _as_utc(prev.started_at):
raise MeterIntervalError(
f"New started_at ({started_at.isoformat()}) must be strictly after "
f"the previous meter's started_at ({prev.started_at.isoformat()}). "
"Moving the boundary that far back would collapse the previous meter's epoch."
)
# Maintain continuity: update the previous meter's ended_at to match the new start.
prev.ended_at = started_at
logger.info(
"Updated previous meter id=%d ended_at=%s (boundary shift from %s)",
prev.id,
started_at.isoformat(),
old_started_at.isoformat(),
)
meter.started_at = started_at
logger.info(
"Updated meter id=%d started_at=%s (was %s)",
meter.id,
started_at.isoformat(),
old_started_at.isoformat(),
)
return meter
+209
View File
@@ -0,0 +1,209 @@
"""Modbus polling service — periodic read-and-store for enabled Modbus devices.
Design notes
------------
- ``poll_device`` loads the device's YAML profile, calls the driver to bulk-read
all register blocks in a single TCP connection, decodes the raw registers into
an engineering-value dict via the profile, and persists one ``ModbusReading``
row. It also stamps ``last_poll_at`` / ``last_poll_ok`` on the device.
- All exceptions are caught and logged inside ``poll_device``; the function
never re-raises. This prevents a single broken device from aborting the
sweep for all other devices.
- ``poll_all_enabled_devices`` respects per-device ``poll_interval_s``: it skips
a device whose ``last_poll_at`` is recent enough. This lets the APScheduler
job run on a short base tick (e.g. 5 s) while each device self-regulates its
own cadence.
- The global ``modbus_polling_enabled`` flag from ``Settings`` (merged with any
DB overrides via ``build_runtime_settings``) gates the entire sweep as a
no-op. T08 will expose this flag in CONFIG_FIELDS/UI; for now it is
controlled by the ``MODBUS_POLLING_ENABLED`` env var (or the default True).
"""
from __future__ import annotations
import logging
from datetime import UTC, datetime
from sqlalchemy import select
from sqlalchemy.orm import Session
from app.config import Settings
from app.integrations.modbus import driver, profiles
from app.models.modbus import ModbusDevice, ModbusReading
from app.services.config_page import build_runtime_settings
from app.services.ha_discovery import publish_device_offline, publish_device_state
logger = logging.getLogger(__name__)
# Base polling tick in seconds (used by APScheduler; each device checks its own interval).
BASE_POLL_TICK_SECONDS = 5
def poll_device(session: Session, device: ModbusDevice) -> ModbusReading | None:
"""Read one enabled Modbus device, decode, and persist a reading row.
Parameters
----------
session:
Open SQLAlchemy session. The caller is responsible for session
lifecycle (open / commit / close).
device:
``ModbusDevice`` ORM instance to poll.
Returns
-------
ModbusReading | None
The newly-persisted reading on success, or ``None`` on any failure.
Failure is logged and the device's ``last_poll_ok`` is set to ``False``.
"""
now = datetime.now(UTC)
try:
profile = profiles.load_profile(device.profile)
registers = driver.read_blocks(
device.host,
device.port,
device.unit_id,
[{"start": b.start, "count": b.count} for b in profile.blocks],
)
payload = profiles.decode(profile, registers)
reading = ModbusReading(
device_id=device.id,
recorded_at=now,
payload=payload,
)
session.add(reading)
device.last_poll_at = now
device.last_poll_ok = True
session.commit()
logger.debug(
"Polled device %r (id=%d): %d metrics recorded at %s",
device.friendly_name,
device.id,
len(payload),
now.isoformat(),
)
# Best-effort: push MQTT state after successful poll.
# Must never let MQTT errors crash the poll loop.
try:
publish_device_state(session, device)
except Exception: # noqa: BLE001
logger.exception(
"poll_device: MQTT state publish failed for device %r (id=%d); "
"continuing (poll itself succeeded)",
device.friendly_name,
device.id,
)
return reading
except Exception: # noqa: BLE001
logger.exception(
"Failed to poll device %r (id=%d, host=%s:%d, unit_id=%d)",
device.friendly_name,
device.id,
device.host,
device.port,
device.unit_id,
)
# Rollback first — if session.add(reading) already ran in the success
# branch before commit() failed, that pending ModbusReading must be
# discarded. Without rollback it would remain staged and could be
# flushed/committed by a subsequent operation (even for a different
# device), producing a spurious reading row for a poll that failed.
# Rollback also expires all ORM objects; device fields are re-set below.
try:
session.rollback()
except Exception: # noqa: BLE001
logger.exception(
"session.rollback() failed for device id=%d; session may be unusable", device.id
)
try:
device.last_poll_at = now
device.last_poll_ok = False
session.commit()
except Exception: # noqa: BLE001
logger.exception(
"Also failed to persist last_poll_ok=False for device id=%d", device.id
)
# Best-effort: publish "offline" availability after failed poll.
try:
publish_device_offline(session, device.uuid)
except Exception: # noqa: BLE001
logger.exception(
"poll_device: MQTT offline publish failed for device %r (id=%d)",
device.friendly_name,
device.id,
)
return None
def poll_all_enabled_devices(
session: Session,
*,
bootstrap_settings: Settings,
) -> None:
"""Sweep all enabled Modbus devices and poll each one whose interval has elapsed.
Global kill-switch: if ``build_runtime_settings(...).modbus_polling_enabled``
is ``False``, this function returns immediately without touching any device.
Per-device interval: a device is skipped if its ``last_poll_at`` is not yet
``poll_interval_s`` seconds in the past. This allows the APScheduler job to
run on a short fixed tick while each device self-regulates its own cadence.
Exceptions from individual devices are swallowed inside ``poll_device``;
this function itself will not raise.
Parameters
----------
session:
Open SQLAlchemy session.
bootstrap_settings:
The bootstrap ``Settings`` instance (from ``get_settings()``), used as
the base for ``build_runtime_settings`` to pick up any DB overrides.
"""
try:
runtime_settings = build_runtime_settings(session, bootstrap_settings)
except Exception: # noqa: BLE001
logger.exception("Failed to build runtime settings for Modbus poll; aborting sweep")
return
if not runtime_settings.modbus_polling_enabled:
logger.debug("Modbus polling is disabled (modbus_polling_enabled=False); skipping sweep")
return
try:
devices = session.execute(
select(ModbusDevice).where(ModbusDevice.enabled == True) # noqa: E712
).scalars().all()
except Exception: # noqa: BLE001
logger.exception("Failed to query enabled Modbus devices; aborting sweep")
return
now = datetime.now(UTC)
for device in devices:
# Respect per-device poll interval: skip if last poll was too recent.
if device.last_poll_at is not None:
# SQLite may return a naive datetime despite DateTime(timezone=True) —
# treat naive values as UTC so the comparison is safe.
last_poll = device.last_poll_at
if last_poll.tzinfo is None:
last_poll = last_poll.replace(tzinfo=UTC)
elapsed = (now - last_poll).total_seconds()
if elapsed < device.poll_interval_s:
logger.debug(
"Skipping device %r (id=%d): %.1fs elapsed < %ds interval",
device.friendly_name,
device.id,
elapsed,
device.poll_interval_s,
)
continue
poll_device(session, device)
+139
View File
@@ -0,0 +1,139 @@
"""Service layer for fetching and persisting Tibber 15-minute electricity prices.
``refresh_prices`` is the main entry point. It is designed to be called from a
scheduled background job (see ``app/main.py``) and from tests.
Design decisions
----------------
- **Guard clause**: if no active contract with ``kind="tibber"`` exists, or if
the Tibber API token is empty in the runtime settings, the function is a
complete no-op (returns 0) and does not raise. This means the scheduler can
call ``refresh_prices`` unconditionally; the service itself decides whether to
do anything based on current configuration.
- **Upsert idempotency**: rows are matched by ``starts_at`` (the unique
constraint on ``tibber_price``). If a row already exists, its price fields
and ``fetched_at`` are updated in-place; if it does not exist, a new row is
inserted. Running ``refresh_prices`` twice in a row must not double-insert.
- **No destructive operations**: this service never deletes ``tibber_price``
rows. Only INSERT or UPDATE.
- **Exception propagation**: exceptions from the Tibber client are *not*
swallowed here. The scheduled job wrapper in ``main.py`` is responsible for
catching and logging errors so that a single fetch failure does not crash the
scheduler.
- **SQLite timezone note**: ``DateTime(timezone=True)`` columns come back as
timezone-naive UTC datetimes on read. We store UTC-aware datetimes on write
(``starts_at`` from ``PricePoint`` is already UTC-aware; ``fetched_at`` is
``datetime.now(UTC)``). Reads elsewhere that compare against these values
must normalise accordingly.
"""
from __future__ import annotations
import logging
from datetime import UTC, datetime
from sqlalchemy import select
from sqlalchemy.orm import Session
from app.integrations.tibber.client import fetch_price_range
from app.models.energy import EnergyContract, TibberPrice
logger = logging.getLogger(__name__)
def _active_tibber_contract_exists(session: Session) -> bool:
"""Return True if there is an active contract with kind='tibber'."""
row = session.execute(
select(EnergyContract).where(
EnergyContract.active.is_(True),
EnergyContract.kind == "tibber",
).limit(1)
).scalar_one_or_none()
return row is not None
def refresh_prices(session: Session, settings: object) -> int:
"""Fetch today-and-tomorrow Tibber prices and upsert them into ``tibber_price``.
Parameters
----------
session:
An active SQLAlchemy session. The caller is responsible for closing it;
this function commits each upserted row individually to keep transactions
short.
settings:
A runtime settings object that exposes ``tibber_api_token`` and
``tibber_home_id`` attributes (typically a ``Settings`` or merged
runtime-settings instance from ``build_runtime_settings``).
Returns
-------
int
Number of rows upserted (inserted or updated). Returns 0 for a no-op.
Notes
-----
- No-op (returns 0) when:
* No active contract exists with ``kind="tibber"``, OR
* ``settings.tibber_api_token`` is empty or whitespace.
- Exceptions from the Tibber client are *not* caught here; they propagate to
the caller (the scheduled job wrapper swallows them and logs).
"""
token: str = getattr(settings, "tibber_api_token", "") or ""
if not token.strip():
logger.debug("refresh_prices: tibber_api_token is empty — no-op")
return 0
if not _active_tibber_contract_exists(session):
logger.debug("refresh_prices: no active tibber contract — no-op")
return 0
home_id: str = getattr(settings, "tibber_home_id", "") or ""
home_id_or_none: str | None = home_id.strip() or None
logger.info("refresh_prices: fetching Tibber price range (home_id=%r)", home_id_or_none)
# May raise TibberError or TibberAuthError — let them propagate.
price_points = fetch_price_range(token, home_id_or_none)
if not price_points:
logger.info("refresh_prices: API returned zero price points — nothing to upsert")
return 0
fetched_at = datetime.now(UTC)
upserted = 0
for point in price_points:
existing = session.execute(
select(TibberPrice).where(TibberPrice.starts_at == point.starts_at)
).scalar_one_or_none()
if existing is not None:
# Update in-place — price fields may change (e.g. Tibber corrects a
# forecast), but starts_at and resolution stay the same.
existing.total = point.total
existing.energy = point.energy
existing.tax = point.tax
existing.level = point.level
existing.currency = point.currency
existing.fetched_at = fetched_at
else:
session.add(
TibberPrice(
starts_at=point.starts_at,
resolution=point.resolution,
total=point.total,
energy=point.energy,
tax=point.tax,
level=point.level,
currency=point.currency,
fetched_at=fetched_at,
)
)
upserted += 1
session.commit()
logger.info("refresh_prices: upserted %d price points", upserted)
return upserted
+100
View File
@@ -0,0 +1,100 @@
"""Server local-timezone helpers.
All time is stored as UTC (no change). This module provides a single,
monkeypatch-friendly entry point for converting UTC moments to the server's
local timezone used for business-day boundary calculations (standing charges,
effective-from date interpretation, etc.).
Design goals
------------
- Zero third-party dependencies (no tzlocal, no pytz).
- Testable without depending on CI host timezone:
monkeypatch ``local_tz`` to a fixed ``ZoneInfo`` in unit tests.
- DST-correct: conversions use ``astimezone(local_tz())`` per instant, so each
moment is independently correct even during DST transitions.
- Consistent with the existing ``.astimezone()`` pattern already used in
``homeassistant_inbound.py`` and ``poo.py``.
Priority for resolving the local timezone
-----------------------------------------
1. ``TZ`` environment variable ``ZoneInfo(os.environ["TZ"])``.
Set ``TZ=Europe/Amsterdam`` in the deployment env for correct NL handling.
2. System local timezone fallback: ``datetime.now().astimezone().tzinfo``.
This matches the behaviour callers already relied on implicitly.
"""
from __future__ import annotations
import os
from datetime import date, datetime, timezone
from typing import TYPE_CHECKING
from zoneinfo import ZoneInfo
if TYPE_CHECKING:
from datetime import tzinfo
def local_tz() -> "tzinfo":
"""Return the server's local timezone.
Resolution order:
1. ``TZ`` environment variable (``ZoneInfo(TZ)``). Set
``TZ=Europe/Amsterdam`` in production for correct NL/DST handling.
2. System local timezone via ``datetime.now().astimezone().tzinfo``.
**Monkeypatch this function in tests** to get deterministic timezone
behaviour regardless of CI host configuration::
monkeypatch.setattr("app.services.timezone.local_tz",
lambda: ZoneInfo("Europe/Amsterdam"))
"""
tz_env = os.environ.get("TZ", "").strip()
if tz_env:
return ZoneInfo(tz_env)
# System fallback — identical to the .astimezone() pattern already used
# in homeassistant_inbound.py and poo.py.
return datetime.now().astimezone().tzinfo # type: ignore[return-value]
def to_local(dt: datetime) -> datetime:
"""Convert *dt* to the server local timezone.
Handles three input cases:
- timezone-aware (any zone): converted via ``astimezone``.
- timezone-naive: assumed UTC (SQLite read-back convention), then converted.
Returns a timezone-aware datetime in ``local_tz()``.
"""
if dt.tzinfo is None:
dt = dt.replace(tzinfo=timezone.utc)
return dt.astimezone(local_tz())
def local_now() -> datetime:
"""Return the current moment as a timezone-aware local datetime."""
return datetime.now(local_tz())
def local_date(dt: datetime) -> date:
"""Return the local calendar date for *dt*.
Equivalent to ``to_local(dt).date()``. A distinct helper to make
call sites readable.
"""
return to_local(dt).date()
def local_midnight_utc(d: date) -> datetime:
"""Return the UTC instant that corresponds to local midnight on *d*.
Example (Europe/Amsterdam, CEST=UTC+2):
local_midnight_utc(date(2026, 6, 25))
datetime(2026, 6, 24, 22, 0, 0, tzinfo=UTC)
This is used to convert a local calendar date back to a UTC boundary, e.g.
to compute the start/end of a local day for DB queries.
"""
tz = local_tz()
# Build a naive local datetime at midnight, then localize and convert to UTC.
local_midnight = datetime(d.year, d.month, d.day, 0, 0, 0, tzinfo=tz)
return local_midnight.astimezone(timezone.utc)
+234
View File
@@ -0,0 +1,234 @@
"""TOTP service: secret management, recovery-code lifecycle, setup/enable/disable (M4-T05).
State-machine summary
---------------------
DISABLED (default)
totp_secret=None, totp_enabled=False, no recovery codes
PENDING (after setup, before enable)
totp_secret=<base32>, totp_enabled=False, recovery codes stored as hashes
- Re-calling setup replaces the pending secret and regenerates recovery codes.
ENABLED (after enable)
totp_secret=<base32>, totp_enabled=True, recovery codes stored as hashes
After disable:
totp_secret=None, totp_enabled=False, recovery codes deleted back to DISABLED
Recovery-code timing
--------------------
1. setup generate 10 plaintext codes, persist their Argon2 hashes immediately,
return plaintext to caller (ONLY time).
2. enable verify 6-digit TOTP code; if ok, flip totp_enabled=True.
Recovery codes are already in the DB from step 1.
3. disable clear secret, clear enabled flag, delete all recovery codes.
Security note: ``totp_secret`` is stored as plaintext (same posture as other
secrets in this project: protected by file-system permissions on the SQLite
database). Recovery codes are stored as Argon2 hashes only.
"""
from __future__ import annotations
import logging
import secrets
import string
import pyotp
from sqlalchemy import delete, select
from sqlalchemy.orm import Session
from app.models.auth import AuthUser, RecoveryCode
from app.services.auth import hash_password, verify_password
logger = logging.getLogger(__name__)
# Number of recovery codes to generate per setup
_RECOVERY_CODE_COUNT = 10
# Alphabet for each 4-character segment: lowercase letters + digits, no ambiguous chars
_CODE_ALPHABET = string.ascii_lowercase + string.digits
# ---------------------------------------------------------------------------
# Internal helpers
# ---------------------------------------------------------------------------
def _generate_recovery_code() -> str:
"""Return a single recovery code in ``xxxx-xxxx`` format."""
segment = lambda: "".join(secrets.choice(_CODE_ALPHABET) for _ in range(4)) # noqa: E731
return f"{segment()}-{segment()}"
def _verify_totp_code(secret: str, code: str) -> bool:
"""Verify a 6-digit TOTP code against the given base-32 secret (±1 window)."""
return pyotp.TOTP(secret).verify(code, valid_window=1)
def _delete_recovery_codes(db: Session, *, user_id: int) -> None:
"""Delete ALL recovery codes for a user (called on setup re-run and disable)."""
db.execute(delete(RecoveryCode).where(RecoveryCode.user_id == user_id))
# ---------------------------------------------------------------------------
# Public API
# ---------------------------------------------------------------------------
def setup(
db: Session,
*,
user: AuthUser,
issuer: str,
) -> tuple[str, str, list[str]]:
"""Generate a new pending TOTP secret and recovery codes.
The secret is stored in ``user.totp_secret`` (``totp_enabled`` stays
``False``). Any pre-existing pending secret and recovery codes are
replaced atomically.
Returns
-------
(secret, otpauth_uri, plaintext_recovery_codes)
``plaintext_recovery_codes`` are returned **once** here and MUST NOT be
stored or returned elsewhere.
"""
# Generate new TOTP secret
new_secret = pyotp.random_base32()
otpauth_uri = pyotp.TOTP(new_secret).provisioning_uri(
name=user.username,
issuer_name=issuer,
)
# Generate plaintext recovery codes
plaintext_codes = [_generate_recovery_code() for _ in range(_RECOVERY_CODE_COUNT)]
# --- Persist atomically ---
# 1. Delete any old pending recovery codes (idempotent on re-setup)
assert user.id is not None # mypy guard; always set after DB insertion
_delete_recovery_codes(db, user_id=user.id)
# 2. Update the secret (pending — totp_enabled stays False)
user.totp_secret = new_secret
# 3. Persist new recovery codes as Argon2 hashes
for plaintext in plaintext_codes:
db.add(RecoveryCode(user_id=user.id, code_hash=hash_password(plaintext)))
db.commit()
db.refresh(user)
logger.info("TOTP setup (pending) for user '%s'; %d recovery codes generated.", user.username, _RECOVERY_CODE_COUNT)
return new_secret, otpauth_uri, plaintext_codes
def enable(db: Session, *, user: AuthUser, code: str) -> bool:
"""Enable TOTP after the user proves they can generate the correct code.
Parameters
----------
code:
The current 6-digit TOTP code from the user's authenticator app.
Returns
-------
True on success; False if the code is invalid or no secret is pending.
"""
if not user.totp_secret:
logger.info("TOTP enable rejected for '%s': no pending secret.", user.username)
return False
if not _verify_totp_code(user.totp_secret, code):
logger.info("TOTP enable rejected for '%s': wrong code.", user.username)
return False
user.totp_enabled = True
db.commit()
db.refresh(user)
logger.info("TOTP enabled for user '%s'.", user.username)
return True
def disable(
db: Session,
*,
user: AuthUser,
password: str | None = None,
code: str | None = None,
) -> bool:
"""Disable TOTP. Caller must supply exactly one of ``password`` or ``code``.
On success: ``totp_enabled=False``, ``totp_secret=None``, all recovery
codes deleted.
Returns
-------
True on success; False if neither credential is valid.
"""
if not password and not code:
logger.info("TOTP disable rejected for '%s': no credential provided.", user.username)
return False
if password:
if not verify_password(password, user.password_hash):
logger.info("TOTP disable rejected for '%s': wrong password.", user.username)
return False
elif code:
if not user.totp_secret or not _verify_totp_code(user.totp_secret, code):
logger.info("TOTP disable rejected for '%s': wrong TOTP code.", user.username)
return False
# Clear TOTP state
assert user.id is not None
_delete_recovery_codes(db, user_id=user.id)
user.totp_enabled = False
user.totp_secret = None
db.commit()
db.refresh(user)
logger.info("TOTP disabled for user '%s'.", user.username)
return True
def verify_totp_code(user: AuthUser, code: str) -> bool:
"""Verify a 6-digit TOTP code for a user.
Returns True if the code is valid (±1 time window), False otherwise.
The user must have a ``totp_secret`` set; returns False if not.
This is the public entry-point for T06 login two-factor verification.
"""
if not user.totp_secret:
return False
return _verify_totp_code(user.totp_secret, code)
def verify_recovery_code(db: Session, *, user: AuthUser, code: str) -> bool:
"""Verify and consume a one-time recovery code.
Finds the first unused recovery code whose hash matches ``code``, marks it
as consumed (sets ``used_at``), and returns ``True``. Returns ``False`` if
no matching unused code exists.
This function is provided for T06 (login two-factor) but lives here so the
TOTP service owns all recovery-code logic.
"""
from datetime import UTC, datetime
unused = db.execute(
select(RecoveryCode).where(
RecoveryCode.user_id == user.id,
RecoveryCode.used_at.is_(None),
)
).scalars().all()
for rc in unused:
if verify_password(code, rc.code_hash):
rc.used_at = datetime.now(UTC)
db.commit()
logger.info(
"Recovery code consumed for user '%s' (id=%d).", user.username, rc.id
)
return True
return False
+6
View File
@@ -62,6 +62,8 @@ packaging==26.1
# build
# pytest
# wheel
paho-mqtt==2.1.0
# via -r requirements.in
pip-tools==7.5.3
# via -r dev-requirements.in
pluggy==1.6.0
@@ -78,6 +80,10 @@ pydantic-settings==2.13.1
# via -r requirements.in
pygments==2.20.0
# via pytest
pymodbus==3.13.1
# via -r requirements.in
pyotp==2.10.0
# via -r requirements.in
pyproject-hooks==1.2.0
# via
# build
+2 -2
View File
@@ -20,9 +20,9 @@ services:
build: .
image: home-automation:dev
container_name: home-automation-app-dev
# Publish on 8001 for dev. `!override` REPLACES the base ports list instead of
# Publish on 8002 for dev. `!override` REPLACES the base ports list instead of
# appending to it, so the dev stack does NOT also bind the production 8881.
ports: !override
- "127.0.0.1:8001:8000"
- "127.0.0.1:8002:8000"
environment:
APP_DATABASE_URL: "sqlite:////app/data/app.db"
+2
View File
@@ -9,6 +9,7 @@ services:
volumes:
- ./data:/app/data
- ./.env:/app/.env:ro
- /etc/localtime:/etc/localtime:ro
app:
container_name: home-automation-app
@@ -24,4 +25,5 @@ services:
volumes:
- ./data:/app/data
- ./.env:/app/.env:ro
- /etc/localtime:/etc/localtime:ro
+94 -7
View File
@@ -19,31 +19,42 @@
- `main.py`
- FastAPI app factory
- lifespan
- lifespanAPScheduler 启停、MQTT 客户端起停、连接后触发 HA Discovery 发布;M6 新增 `tibber-refresh` 抓价 job + `energy-cost` 1 分钟计费 tick jobM6 启用时注册 DSMR MQTT 订阅)
- 基础路由注册
- `config.py`
- 环境变量驱动的 settings
- 环境变量驱动的 settings(含 M5 新增的 MQTT/HA Discovery/Modbus 配置项;M6 新增 `dsmr_ingest_enabled``dsmr_mqtt_topic``dsmr_sample_interval_s``tibber_api_token`secret)、`tibber_home_id`
- `db.py`
- 统一数据层:一个 `Base`、一个绑定 `app_database_url` 的 cached engineSQLite WAL)、`get_engine` / `get_session_local` / `reset_db_caches` / `get_db_session`
- `dependencies.py`
- 通用依赖注入
- `api/`
- HTTP routes
- `api/routes/api/`JSON API`/api/*` 前缀),供 React SPA 调用:会话/鉴权、配置读写、数据查询、记录 CRUD
- `api/routes/api/`JSON API`/api/*` 前缀),供 React SPA 调用:会话/鉴权、配置读写、数据查询、记录 CRUD、Modbus 设备 CRUD + readings + metrics + test`/api/modbus/*`)、Expose 勾选 + 重发 discovery`/api/expose`)、MQTT 测试连接(`/api/config/mqtt/test`)、M6 新增合同 CRUD + 版本(`/api/energy/contracts*`)、pricing profile 列表(`/api/energy/profiles`)、价格/费用/汇总/DSMR 最新/重算/Tibber 测试(`/api/energy/prices``/api/energy/costs``/api/energy/costs/summary``/api/energy/dsmr/latest``/api/energy/costs/recompute``/api/energy/tibber/test`
- 裸 ingestion 端点:`GET /public-ip/check``POST /homeassistant/publish``POST /poo/record``GET /poo/latest`、TickTick OAuth 等
- `models/`
- SQLAlchemy models
- 所有模型(auth / config / public_ip / location / poo)共用同一个 `Base`,均落在单一 `app.db`
- 所有模型(auth / config / public_ip / location / poo / modbus / expose / energy)共用同一个 `Base`,均落在单一 `app.db`
- M5 新增:`ModbusDevice`(设备部署层)、`ModbusReading`(通用遥测,JSON payload)、`ExposedEntityToggle`HA 实体暴露开关)
- M6 新增:`DsmrReading`(整帧 DSMR telegram10s 降采样)、`EnergyContract`(合同头,含 active 标记)、`EnergyContractVersion`(版本/时段,values JSON,只增不改)、`TibberPrice`15 分钟价缓存,不可变)、`EnergyCostPeriod`(每 15 分钟计量电费,快照价,不可变)
- `schemas/`
- Pydantic schemas
- Pydantic schemasM5 新增 `modbus.py``expose.py`M6 新增 `energy_contract.py``energy.py`
- `services/`
- 业务服务层
- 当前已迁入 config page 的 DB 持久化逻辑
- 当前已迁入 public IPv4 检查、状态持久化与变化通知逻辑
- 当前已迁入 SMTP 发信与测试发信逻辑
- M5 新增:`modbus_poll.py`(采集 service,逐设备 poll + 落库 + 推 MQTT state)、`ha_discovery.py`(构建 HA Discovery payload、发布 retained config、发布 state
- M6 新增:`tibber_prices.py`httpx GraphQL 抓 15 分钟价,upsert `tibber_price`,幂等;仅 active=tibber 且 token 存在时运行)、`dsmr_ingest.py`MQTT handler,整帧 JSON blob + 10s 降采样落库,`source_id` 幂等)、`energy_cost.py`(计费引擎:每 15 分钟寄存器差 × strategy 出价 → `energy_cost_period` 不可变快照;汇总 Σnet + 固定费 heffingskorting;重算显式 opt-in
- `integrations/`
- 外部系统适配层
- 当前已迁入 Home Assistant outbound adapter
- Home Assistant outbound adapterREST 通道,原有)
- M5 新增:`modbus/`pymodbus 薄封装:`driver.py` 块读 + float32 解码;`profiles.py` YAML profile 加载/校验/解码;`profiles/sdm120.yaml` SDM120 协议声明)
- M5 新增:`mqtt.py`paho-mqtt 长连接 `MqttManager`:lifespan 起/停、配置变更重连、`publish(topic, payload, retain)`
- M5 新增:`expose.py`(通用 expose 框架:`ExposableEntity`、provider 注册表、`build_catalog`Modbus provider 从 YAML profile 派生 sensor/binary_sensor 实体目录)
- M6 新增:`pricing/`(通用电价层:`profiles.py` pydantic 加载/校验 YAML profile + `validate_values``strategies.py` manual/tibber 出价策略注册表;`profiles/manual.yaml` 固定/双费率结构;`profiles/tibber.yaml` 动态电价结构)
- M6 新增:`tibber/``client.py`httpx GraphQL 客户端,`priceInfoRange(QUARTER_HOURLY)` 查询,按 `starts_at` 解析,不假设固定节点数)
- M6 扩展:`mqtt.py` 新增订阅端(`subscribe(topic, handler)``on_connect` 里 subscribe`on_message` 按 topic 分发,handler 异常吞掉不崩连接)
- M6 扩展:`expose.py` 新增 `_energy_cost_provider``buy_price_now``sell_price_now``import_cost_total`/`export_revenue_total``total_increasing`,反哺 HA Energy
- `static/`
- 极简静态资源
@@ -69,18 +80,94 @@ React SPA 前端(M2 引入)。Vite + React + TypeScript + Mantine,由 Fast
### `scripts/`
辅助脚本目录。当前包含 OpenAPI 导出脚本(`export_openapi.py`)与数据层辅助脚本。
辅助脚本目录。当前包含
- `export_openapi.py`:导出 OpenAPI schema 静态产物
- `run_migrations.py`:运行 Alembic migration
- `app_db_adopt.py`App DB 接管 / 初始化
- `migrate_legacy_data.py`:一次性历史数据搬迁脚本
- `admin_cli.py`Admin CLI 逃生通道(M4),见下方"登录加固"说明
- `modbus_cli.py`M5):Modbus 手工试读 CLI`read`/`probe` 两个只读子命令),不依赖 DB,供受控手工验证链路连通性
### `openapi/`
OpenAPI schema 静态产物(`openapi.json` / `openapi.yaml`),由 `python scripts/export_openapi.py` 生成,纳入版本控制。前端 codegen 以此为契约源。
## 登录加固(M4
M4 在基础 Argon2 + server-side session 鉴权之上叠加了三层防御:
**防爆破 / 指数退避**`app/services/login_throttle.py` 按 client IP 与 username 双键记失败计数,失败超过 3 次后指数增长等待时间(最长 15 分钟),`POST /api/auth/login` 在退避窗口内直接返回 `429 + Retry-After`,不执行 Argon2 验证。成功登录后清零。退避是延迟而非永久封号;全局开关 `AUTH_LOGIN_THROTTLE_ENABLED`CONFIG_FIELDS,默认开);反代后需设 `AUTH_TRUST_FORWARDED_FOR=true``.env` 部署级,默认 false)。
**CLI 逃生通道**`scripts/admin_cli.py`(入口 `python -m scripts.admin_cli`)直连本地 DB,**无需 HTTP 服务运行、无需任何已存凭据**,支持:重置密码(`reset-password`)、解锁退避(`unlock`)、关停 TOTP`disable-totp`,零凭据最终逃生)、重新发放 TOTP secret`reissue-totp`)、查看用户列表(`list-admin`)。CLI 只动 auth 行,不触碰用户数据表。
**可选 TOTP 二次验证**:admin 可在设置页自选启用 RFC 6238 TOTP。启用后登录为两步(密码 → 6 位动态码或一次性恢复码);不启用维持纯密码。TOTP secret 明文存库(与其他 secret 一致,靠文件权限保护);恢复码以 Argon2 哈希存储,使用后消费(一次性)。后端使用 `pyotp`,二维码由前端 `qrcode.react` 渲染。issuer 标签由 `AUTH_TOTP_ISSUER``.env` 部署级)配置,默认回退 `app_name`
详细说明:[`docs/auth.md`](./auth.md)
## M5 — 通用 Modbus 采集链路与 MQTT 通道
### Modbus 采集链路
```
YAML profile(协议知识) modbus_device 行(部署信息,DB
- 寄存器块/地址/类型 - host / port(网关 IP
- 每量:key/unit/device_class - unit_idModbus slave 地址)
- ha_component - friendly_name / profile / poll_interval_s / enabled
│ │
└────────────┬───────────────────────┘
│ APScheduler job(轮询所有 enabled 设备)
│ driver.py: ModbusTcpClient → FC04 块读 → 大端 float32 解码
│ profiles.py: decode(profile, registers) → dict[key → value]
modbus_reading 行
device_id · recorded_at · payload(JSON)
{"voltage": 230.2, "current": 1.3, "active_power": 295.0, ...}
```
**关键设计决策**
- 命名分层:存储/采集/API 全部通用 `modbus_*``/api/modbus/devices`);面向用户的领域视图叫 **Energy**(第一个)。接入新设备型号只需新增 YAML profile,不改表/不改 API。
- 读数为 JSON `payload`(无固定列),SQLite `json_extract` 在 DB 端做 AVG/GROUP BY(被 `(device_id, recorded_at)` 索引圈住)。
- FK `ON DELETE RESTRICT`:有读数的设备拒删,引导改为 `enabled=false`
- 设备 `uuid`(uuid4 内部生成)是**稳定身份锚点**API 路径键、HA Discovery `unique_id` 来源,不随 friendly_name 改变。
### 第二条 MQTT 通道(HA Discovery 发布)
与已有 REST 通道(`app/integrations/homeassistant.py``POST /homeassistant/publish`**并行、不冲突**
```
MQTT 通道(M5 新增):
paho-mqtt MqttManagerlifespan 长连接,配置变更可重连)
├─► Discovery configretained
│ topic: <prefix>/<component>/<node>/<object>/config
│ 内容:device 块(identifiers=uuid)、state_topic、unique_iduuid+key)、
│ namefriendly_name)、device_class、unit_of_measurement、availability
│ 时机:连接成功时 / 目录或勾选变更时(全量重发);
│ 取消勾选时发空 payload(清除 entity
└─► State / Availability(非 retained
时机:每次轮询成功后推该设备所有 enabled entity 的最新值;
周期兜底 job 重推所有 enabled entity + online topic
expose 框架:
provider 动态产出 ExposableEntity 目录(元数据从 YAML profile 派生)
ExposedEntityToggle 表:只存逐 key 开关(default=disabled
build_catalog(session) → 目录 + 勾选状态(合并所有 provider)
```
**HA entity 身份模型(Z2M 语义)**
- `unique_id` = `f"{device.uuid}_{metric.key}"`(稳定,改名不变)
- `name` = `friendly_name`(改名重发 discovery,HA 显示名跟着变、历史不丢)
- 每设备除各 sensor entity 外,另有 `binary_sensor` `online`(取 `last_poll_ok`)——这是"不止 sensor"的体现
## 当前约束
- 当前数据库继续使用 SQLite
- ~~当前不引入前后端分离~~ **已退役(M2**:现为 React SPA + JSON `/api` 层,由 FastAPI 同源托管
- 当前不设计 Notion 模块
- 当前通知能力仍保持极小范围,不引入独立通知中心或多渠道抽象
- Modbus 当前**仅 TCP**Waveshare RTU↔TCP 网关),**只读**FC03/04),不写设备寄存器
## 关于 Notion
+191 -82
View File
@@ -1,120 +1,229 @@
# 基础鉴权说明
# 鉴权说明
本文档说明当前 Python 重构项目里已经落地的第一版鉴权基座
这一轮只解决:
- 登录页
- 登录 / 登出流程
- server-side session
- 一个最小受保护页面
这一轮明确不解决:
- 完整 config persistence
- 完整 config CRUD
- 多用户权限系统
- OAuth / SSO / RBAC
本文档说明当前已落地的鉴权基座(基础 session 鉴权 + M4 登录加固)
## 当前 auth 模型
- 认证方式:`username/password`
- 认证方式:`username/password`(可选启用 TOTP 二次验证)
- 会话方式:server-side session
- 客户端凭据:session cookie
- 页面形态:Jinja server-side template
## 当前持久化
## 持久化
当前新增一个共享 App DB
所有 auth 相关数据存放在单一 App DB(`APP_DATABASE_URL`,默认 `sqlite:///./data/app.db`)中
- `APP_DATABASE_URL`
- 默认值:`sqlite:///./data/app.db`
当前 auth 相关数据存放在这个 DB 中:
- `auth_users`
- `auth_sessions`
- `app_config`
当前没有把 auth 数据和 `location` / `poo` DB 混放。
当前这部分现在也走 Alembic 管理:
- Alembic 环境:`alembic_app.ini` + `alembic_app/`
- 初始化脚本:`python scripts/app_db_adopt.py`
当前没有 legacy app DB,所以这一版脚本只负责初始化新库,不负责 legacy adoption。
`app_config` 现在承接运行时配置持久化。
其中:
- `.env` 负责 bootstrap / fallback
- `app_config` 表负责运行时配置覆盖
- 登录密码仍然属于认证数据,使用 Argon2 哈希,不存进 `app_config`
- `auth_users`:用户表(含 TOTP 字段 `totp_secret` / `totp_enabled`
- `auth_sessions`session token 哈希与过期时间
- `auth_login_throttle`:登录失败退避状态(按 IP / username 双键)
- `auth_recovery_code`TOTP 恢复码哈希(一次性)
- `app_config`runtime 配置持久化
## 首次启动与 bootstrap
如果 auth DB 中还没有任何用户,应用启动时会要求
如果 auth DB 中还没有任何用户,应用启动时会使用
- `AUTH_BOOTSTRAP_USERNAME`
- `AUTH_BOOTSTRAP_PASSWORD`
创建首个 admin 用户。
当前默认 bootstrap 值就是:
创建首个 admin 用户。当前默认 bootstrap 值为:
- username: `admin`
- password: `admin`
首次登录后,系统会强制要求修改密码。
如果你希望在首次启动前就覆盖默认值,可以直接设置环境变量:
## 基础安全设计
- `AUTH_BOOTSTRAP_USERNAME`
- `AUTH_BOOTSTRAP_PASSWORD`
建议流程是:
1. 配好 `.env`
2. 运行 `python scripts/app_db_adopt.py`
3. 启动应用
4. 用 `admin / admin` 首次登录
5. 立即修改密码
## 安全设计
当前这版已经落实的基础安全点:
当前这版已经落实的安全点:
- 密码不明文存储,使用 Argon2 哈希
- session cookie 为 `HttpOnly`
- cookie 使用 `SameSite=Lax`
- `Secure` cookie 在非 `development` 环境默认开启
- 登录表单与登出表单都有基础 CSRF 校验
- 写请求(POST/PUT/PATCH/DELETE)需携带 `X-CSRF-Token` headerSameSite=Lax + 自定义 header 纵深防御,无需 per-session token 值比对)
- session token 为随机生成,服务端只持久化 token hash
- session 有过期时间与显式失效机制
- session 有过期时间(默认 12 小时)与显式失效机制
## 当前受保护范围
---
当前这轮只保护了页面入口:
## M4 登录加固
- `GET /config`
- `POST /config`
- `POST /config/change-password`
- `POST /logout`
M4 在基础鉴权之上叠加了三层防御:防爆破/指数退避、CLI 逃生通道、可选 TOTP 二次验证。
相关流程:
### 1. 防爆破 / 指数退避
- `GET /login`
- `POST /login`
#### 机制
登录访问 `/config` 时会被重定向到 `/login`
登录失败按指数增长延迟,目标是拖垮暴力枚举,同时不因此永久锁定账号
## 下一步不在本轮内
**退避是延迟(429 + Retry-After),不是永久封号**——单 admin 场景下永久锁会被攻击者反向用来故意打锁,所以退避只增加等待时间,CLI 是最终逃生口。
后续可以在这个基座上继续做:
#### 双键计算
- 配置页面接入
- config persistence
- 更细的受保护路由范围
- 用户初始化 / 密码轮换的更正式 runbook
每次登录请求同时按 **client IP****username** 各记一套失败计数,本次需等待时间 = 两者退避的**较大值**。
- 按 IP:堵单点暴力(攻击 IP 越敲越慢,合法用户换 IP 不受影响)
- 按 username:全局兜底(即便攻击者分布式换 IP 也有一层保护)
#### 指数公式
```
N_FREE = 3 — 前 3 次失败不延迟(免费次数)
BASE = 1 秒
CAP = 900 秒(15 分钟)
wait = min(CAP, BASE × 2^(failures - N_FREE)) failures > N_FREE 时生效)
```
延迟示意:
| 累计失败次数 | 等待时间 |
| --- | --- |
| 13 | 0 秒(免费) |
| 4 | 2 秒 |
| 5 | 4 秒 |
| 6 | 8 秒 |
| 7 | 16 秒 |
| … | … |
| 13+ | 900 秒(封顶) |
成功登录后,该 IP 和 username 的退避状态**立即清零**。
#### 登录端点行为(`POST /api/auth/login`
1. 先查退避(在窗口内直接 `429`**不验密码**,节省 Argon2 计算并防止枚举)
2. 验密码失败 → 记一次失败(IP + username 各记),返回 `401`
3. 密码正确但 TOTP 启用且缺少 `totp_code` → 返回 `401 {totp_required: true}`,**不记失败**(这是正常两步流程的第一步,不是攻击信号)
4. 密码正确但 TOTP 验证失败 → 记一次失败,返回 `401`
5. 全部通过 → 清零退避状态,发 session cookie
#### 配置项
| 配置项 | 类型 | 默认 | 说明 |
| --- | --- | --- | --- |
| `AUTH_LOGIN_THROTTLE_ENABLED` | boolCONFIG_FIELDS | `true` | 全局开关,关闭后整个退避机制 no-op |
| `AUTH_TRUST_FORWARDED_FOR` | bool`.env` 部署级) | `false` | `true` 时取 `X-Forwarded-For` 最左作为 client IP;**反代后必须显式开启**,否则反代 IP 全视为同一 IP,按 IP 退避失效 |
> **反代部署注意**`AUTH_TRUST_FORWARDED_FOR` 默认 `false`(直接用 socket IP)。部署在 nginx 等反代后面时,应设 `AUTH_TRUST_FORWARDED_FOR=true`,并确保反代正确设置 `X-Forwarded-For`
---
### 2. CLI 逃生通道
#### 设计原则
CLI 通过 **直连本地 DB**`get_session_local()`)工作,**无需 HTTP 服务器运行、无需任何已存凭据**(密码、恢复码均不需要)。拿到服务器 CLI 权限本身就意味着对系统有完全控制,因此这是可接受且必须存在的最终逃生口。
CLI 只动 auth 相关行(`auth_users` 的密码/TOTP 字段、`auth_login_throttle``auth_recovery_code`),**绝不**触碰用户数据表(`location``poo_records``public_ip_state` 等)。
入口:`python -m scripts.admin_cli <command>`
#### 子命令
| 命令 | 用途 | 逃生场景 |
| --- | --- | --- |
| `reset-password <username> [--password <pwd>]` | 重置密码(不带 `--password` 则交互式输入,不回显) | 忘记密码 |
| `unlock [--all \| --ip <ip> \| --username <u>]` | 清 `auth_login_throttle` 行 | 被退避锁住(429)时解锁 |
| `disable-totp <username>` | 关停 TOTP(清 secret + 删全部恢复码),**零凭据可执行** | 恢复码全丢也能进 |
| `reissue-totp <username>` | 生成新 TOTP secret 并打印 `otpauth://` URI | 设备丢失、需重新注册 Authenticator |
| `list-admin` | 列出所有用户与状态列 | 排障用 |
#### 使用示例
```bash
# 重置密码(不带 --password 时交互式 prompt,不回显)
python -m scripts.admin_cli reset-password admin
# 重置密码(非交互,脚本里用)
python -m scripts.admin_cli reset-password admin --password "newpassword"
# 解锁所有退避
python -m scripts.admin_cli unlock --all
# 解锁特定 IP
python -m scripts.admin_cli unlock --ip 1.2.3.4
# 解锁特定 username
python -m scripts.admin_cli unlock --username admin
# 关停 TOTP(零凭据,最终逃生口)
python -m scripts.admin_cli disable-totp admin
# 重新发放 TOTP secret(打印新 otpauth:// URI,扫码重新注册)
python -m scripts.admin_cli reissue-totp admin
# 查看用户列表
python -m scripts.admin_cli list-admin
```
在 Docker 容器内执行:
```bash
docker compose exec app python -m scripts.admin_cli <command>
```
#### `reissue-totp` 语义说明
- 对**已启用** TOTP 的用户:新 secret **立即在登录时生效**,旧 Authenticator 生成的码立即失效;**无需**再走 web `enable` 步骤。
- 现有恢复码**不被删除**——恢复码是独立的随机哈希值,与 TOTP secret 无密码学绑定,reissue 后恢复码依然有效(仍可用于登录)。
- 如果需要完整清理(secret + 所有恢复码),使用 `disable-totp`
---
### 3. 可选 TOTP 二次验证
#### 设计
- TOTP 遵循 RFC 6238,使用 `pyotp` 库。
- 二维码在**前端**由 `qrcode.react` 渲染(后端只返回 `otpauth://` URI,不引图像依赖)。
- `totp_secret` 明文存库(与项目其他 secret 处理一致,靠数据库文件权限保护)。
- 恢复码以 Argon2 哈希存库,一次性(使用后标记 `used_at`)。
#### 启用流程
1. **setup**`POST /api/auth/totp/setup`):生成 pending secret,返回 secret、`otpauth://` URI、10 个明文恢复码(**仅此一次**);此时 `totp_enabled` 仍为 `false`
2. **扫码**:用 Authenticator App 扫前端渲染的二维码(或手动输入 secret)。
3. **enable**`POST /api/auth/totp/enable`):输入当前 6 位码确认 → `totp_enabled` 变为 `true`
4. 妥善保存恢复码(不会再次展示)。
#### 停用流程
**web 停用**`POST /api/auth/totp/disable`):需提供当前密码或当前 6 位 TOTP 码。成功后清 secret、删全部恢复码,恢复纯密码登录。
**CLI 停用**(逃生口,恢复码全丢时):`python -m scripts.admin_cli disable-totp admin`,零凭据,立即生效。
#### 登录二步流程
1. 提交 `POST /api/auth/login {username, password}` → 密码正确但 TOTP 已启用 → `401 {totp_required: true}`(不发 session
2. 前端切到第二屏,提交 `POST /api/auth/login {username, password, totp_code}` → 通过 → 发 session cookie
`totp_code` 可以是 6 位 TOTP 动态码,也可以是 `xxxx-xxxx` 格式恢复码(命中即消费,不可复用)。
#### API 端点
| 端点 | 用途 |
| --- | --- |
| `POST /api/auth/totp/setup` | 生成 pending secret + URI + 恢复码(一次性明文返回) |
| `POST /api/auth/totp/enable` | 带当前 6 位码确认启用 |
| `POST /api/auth/totp/disable` | 带密码或当前码停用 |
| `GET /api/auth/totp` | 返回当前 TOTP 状态(`{enabled: bool}`),不返回 secret/恢复码 |
全部端点需要 session cookie`GET /api/auth/totp` 不需 CSRF;其余写端点需 `X-CSRF-Token`)。
#### 配置项
| 配置项 | 类型 | 默认 | 说明 |
| --- | --- | --- | --- |
| `AUTH_TOTP_ISSUER` | str`.env` 部署级) | 空(回退 `app_name` | 显示在 Authenticator App 里的 issuer 标签 |
---
## 受保护范围
当前 JSON API 端点(`/api/*`)需要 session cookie;写端点需额外携带 `X-CSRF-Token` header。
裸 ingestion 端点(`/location/record``/poo/record` 等设备调用端点)暂未收口到 session 保护(M3 计划引入 token 鉴权)。
## 下一步(不在当前范围)
- M3token 鉴权(供脚本 / 设备 / 移动端调用 API),ingestion 端点收口。
+6 -2
View File
@@ -1,12 +1,16 @@
# 设计文档与多模型协作约定
本目录把 `docs/roadmap.md` 里的个里程碑展开成**可被 coding agent 流水线执行**的详细设计文档。
本目录把 `docs/roadmap.md` 里的个里程碑展开成**可被 coding agent 流水线执行**的详细设计文档。
- [`m1-db-consolidation.md`](./m1-db-consolidation.md) — 单库化地基
- [`m2-frontend-v2.md`](./m2-frontend-v2.md) — React SPA 前端 v2
- [`m3-token-mobile.md`](./m3-token-mobile.md) — token 鉴权与移动端(远期)
- [`m4-login-hardening.md`](./m4-login-hardening.md) — 登录加固(防爆破/指数退避 + CLI 逃生 + 可选 TOTP**先做**
- [`m5-iot-energy.md`](./m5-iot-energy.md) — IoT 集成与能耗采集(Modbus/Energy + MQTT/HA Discovery + 前端侧边栏)
- [`m6-tibber-dynamic-energy.md`](./m6-tibber-dynamic-energy.md) — 通用电价层 + DSMR 实时电表接入 + 实时买卖电费计算 + HA Energy 反哺
- [`m7-meter-epochs-archival.md`](./m7-meter-epochs-archival.md) — 电表生命周期 / 换表归档(Meter epochs
本文件定义**所有任务共用的格式与协作规则**,个里程碑文档不再重复这些约定。
本文件定义**所有任务共用的格式与协作规则**,个里程碑文档不再重复这些约定。
---
+309
View File
@@ -0,0 +1,309 @@
# M4 — 登录加固(Login Hardening
> 阅读前提:先读 [`README.md`](./README.md)(协作模型、任务卡格式、校验闸门、数据安全红线)。
> **排期:本里程碑排在 [M5](./m5-iot-energy.md) 之前。** 系统现为公网 + 仅密码 + 无限重试,是现成风险;M4 与 M5 零文件重叠,先堵这个洞再做 IoT。
## 1. 目标
给暴露在公网的单 admin 登录做纵深防御:
1. **防爆破 / 指数退避**:失败登录按指数增长延迟,拖垮暴力枚举;成功即清零。
2. **CLI 逃生通道**:纯命令行重置密码 / 解锁 / 关停 TOTP,**不依赖任何已存恢复凭据**——拿到服务器 CLI 权限的人本来就无所不能,所以这是可接受、且必须存在的最终逃生口(消除"锁死就彻底进不去"的死角)。
3. **TOTP 二次验证(可选配)**admin 可**自选启用** RFC 6238 TOTP;启用后登录需密码 + 6 位动态码;提供一次性恢复码;CLI 能在恢复码全丢时关掉 TOTP。**非强制**,不启用则维持纯密码登录。
> 三段顺序:A(防爆破,紧急、可独立先发版)→ B(TOTP,可选配)→ 收尾。Phase A 自成可用增量,Phase B 失败不影响 A。
## 2. 现状(实现者可据此工作,不必通读全仓库)
**单 admin + Argon2**
- `app/models/auth.py``AuthUser``id / username unique / password_hash / is_active / force_password_change / created_at`)、`AuthSession``token_hash / csrf_token / expires_at / revoked_at`)。单库 `Base`
- `app/services/auth.py`
- `authenticate_user(session, *, username, password) -> AuthUser | None`:查用户→`verify_password`Argon2);失败只 `logger.info`**无任何节流**。
- `create_session(session, *, user, settings) -> (AuthSession, raw_token)``secrets.token_urlsafe`SHA256 存 `token_hash`TTL `auth_session_ttl_hours`(默认 12h)。
- `change_password(...)``get_authenticated_session(...)``initialize_auth_schema(session, settings)`**仅当无任何用户时**用 `AUTH_BOOTSTRAP_USERNAME/PASSWORD` 建初始 admin)。
- `app/api/routes/api/session.py`
- `POST /api/auth/login`body `{username, password}`):`authenticate_user` → None 则 401;否则 `create_session` + `set_cookie`(HttpOnly/SameSite=Lax/`auth_cookie_secure`) → 返回 `{user, csrf_token}`
- `GET /api/session`401 或 user+csrf)、`POST /api/auth/logout`CSRF)、`POST /api/auth/password`CSRF + 校验当前密码)。
**配置**
- `app/config.py``auth_bootstrap_username/password``auth_session_cookie_name``auth_session_ttl_hours``auth_cookie_secure_override`computed `auth_cookie_secure`)。
- 配置系统:扁平 KV`CONFIG_FIELDS` 注册表 + `Settings` + `_settings_payload`),新增标量配置项前端自动渲染(见 M5 文档 §2)。
**无现成 CLI**
- `scripts/``run_migrations.py` / `app_db_adopt.py` / `migrate_legacy_data.py`(argparse) / `export_openapi.py`。**没有**任何 admin/密码/auth 的 CLI。
- `pyproject.toml`**无 `console_scripts`**。CLI 入口沿用 `python -m scripts.<mod>` 风格(与 `python -m scripts.run_migrations` 一致)。
**前端登录(单步)**
- `frontend/src/pages/LoginPage.tsx``POST /api/auth/login {username,password}` → 401 显示通用错误;成功存 csrf、跳转。
- `frontend/src/auth/SessionProvider.tsx``GET /api/session` 引导。
- 无"锁定/稍后再试"提示、无二步流程。
**测试**
- `tests/test_api_session.py`(凭据/cookie/登出/改密/强制改密);`tests/conftest.py` 用 env 设 bootstrap 凭据。
## 3. 目标架构
### 3.1 防爆破 / 指数退避
- **状态表 `auth_login_throttle`**:按 key 记失败窗口(`key / scope / failures / first_failed_at / last_failed_at / next_allowed_at`),成功即删该 key 的行。
- **双键**:同时按 **client IP****username** 计;本次请求需等待 = 两者退避的**较大值**。
- 按 IP:堵单点暴力(attacker IP 越敲越慢,合法用户换 IP 不受影响)。
- 按 username:单 admin 的全局兜底(即便分布式换 IP 也有一层)。
- **DoS 权衡**:单 admin 下纯按 username 硬锁会被攻击者故意失败把你锁死,所以退避是**延迟(429 + Retry-After)而非永久锁定**,且 CLI 是最终逃生口。
- **client IP 来源**:默认用 socket IP`AUTH_TRUST_FORWARDED_FOR=true` 时取 `X-Forwarded-For` 最左(部署在反代后才开)。**这点必须显式配置**,否则反代后所有请求同一 IP,按 IP 退避失效。
- **指数公式(默认,常量可后调)**:前 `N_free=3` 次不延迟;之后 `wait = min(cap, base * 2^(failures - N_free))``base=1s``cap=900s`
- **接入点**`POST /api/auth/login` 开头先查退避 → 在窗口内直接 429(带 `Retry-After`),**不验密码**(省 Argon2、防枚举);验密码失败 → 记一次失败;成功 → 清该 IP + username 的退避行。
- **全局开关** `AUTH_LOGIN_THROTTLE_ENABLED`CONFIG_FIELDS,默认 true)。
### 3.2 CLI 逃生通道
`scripts/admin_cli.py``python -m scripts.admin_cli <cmd>`,argparse 子命令;直接连本地 DB,无网络,复用 `get_session_local()` + 模型 + Argon2 hasher):
| 命令 | 作用 |
| --- | --- |
| `reset-password <username> [--password ...]` | 重置密码(不给则交互式 prompt,不回显);逃生:忘密码 |
| `unlock [--all | --ip <ip> | --username <u>]` | 清 `auth_login_throttle` 行;逃生:被退避锁住 |
| `disable-totp <username>` | 关 TOTP(清 secret + 恢复码);逃生:**恢复码全丢也能进**(Phase B 才有意义)|
| `reissue-totp <username>` | 重新发放 TOTP secret(可选)|
| `list-admin` | 列用户/状态(排障)|
- **数据安全**:CLI 只动 auth 状态行(人工执行的 admin 动作),**绝不**触碰用户数据表(location/poo/energy 等),不 drop 有数据的表。
### 3.3 TOTP 二次验证(可选配)
- **库**`pyotp`(Python 侧);二维码在**前端**用 JS 库(`qrcode.react`)从 `otpauth://` URI 渲染——后端不引图像依赖。
- **存储**
- `AuthUser` 增列 `totp_secret`(nullable)、`totp_enabled`(bool, default false)。secret 落库明文(与现有 secret 处理一致,靠文件权限保护;文档注明)。
- `auth_recovery_code` 表(`user_id / code_hash / used_at`):恢复码**哈希**存(复用 Argon2 hasher),一次性。
- **启用流程(opt-in**
1. `POST /api/auth/totp/setup`(已登录)→ 生成 pending secret + `otpauth://` URI + 一组恢复码(**仅此一次明文返回**),尚未启用。
2. `POST /api/auth/totp/enable`(带一个当前 6 位码确认)→ `totp_enabled=true`,持久化恢复码哈希。
3. `POST /api/auth/totp/disable`(带密码或当前码)→ 关闭、清 secret/恢复码。
- **登录二因子(单端点、无独立 challenge token,保持无状态)**
- `POST /api/auth/login` body 增可选 `totp_code`
- 密码通过且 `totp_enabled`:无 `totp_code` → 返回 401 + `{totp_required: true}`**不发 session**);有 `totp_code` → 校验 TOTP **或**恢复码(命中恢复码则消费它)→ 通过才发 session。
- 未启用 TOTP:维持现状单步。
- 退避(§3.1)对两步都生效。
- **issuer 标签**`AUTH_TOTP_ISSUER`(默认 `app_name`),显示在 Authenticator app 里。
### 3.4 前端
- **登录页两步**`LoginPage` 在收到 `401 + totp_required` 时切到第二屏(6 位码输入,也接受恢复码),再次 `POST /api/auth/login``totp_code`。被退避(429)显示"稍后再试"(用 `Retry-After`)。
- **设置页 TOTP 区**:未启用→「启用」走 setup(展示二维码 + 恢复码一次性、要求输入码确认);已启用→「停用」。恢复码只在生成时展示一次,提示妥善保存。
## 4. API 契约(M4 要落地的端点)
> 全部 `/api`、写端点 session + CSRF 保护、JSON 进出;schema 经 `export_openapi.py` 固化。
| 分组 | 端点 | 用途 |
| --- | --- | --- |
| 登录 | `POST /api/auth/login` | 加退避(429+Retry-After);body 增可选 `totp_code`;启用 TOTP 且缺码→401 `{totp_required:true}` |
| TOTP | `POST /api/auth/totp/setup` | 生成 pending secret + otpauth URI + 恢复码(一次性)|
| TOTP | `POST /api/auth/totp/enable` | 带当前码确认启用 |
| TOTP | `POST /api/auth/totp/disable` | 关闭 TOTP |
| TOTP | `GET /api/auth/totp` | 返回当前 TOTP 状态(enabled 与否)|
> CLI 不是 HTTP 端点。throttle 开关与 issuer 走现有 `GET/PUT /api/config`(仅加 CONFIG_FIELDS)。
## 5. 已锁定决策(讨论后拍板)
1. **M4 独立里程碑、排在 M5 之前**
2. **范围**:防爆破 + 指数退避 + CLI 重置(密码/解锁/关 TOTP)+ TOTP(**可选配,非强制**)。
3. **退避双键(IP + username)、延迟非永久锁定、成功清零、CLI 兜底**;反代后需开 `AUTH_TRUST_FORWARDED_FOR`
4. **CLI 逃生通道不依赖任何已存恢复凭据**;只动 auth 行,不碰用户数据。
5. **TOTP 单端点二因子**login 加可选 `totp_code`,无独立 challenge token);恢复码哈希存、一次性;secret 明文存(与现有一致)。
6. **二维码前端渲染**`qrcode.react`),后端只给 `otpauth://` URI + secretPython 仅加 `pyotp`
7. **CLI 入口 = `python -m scripts.admin_cli`**(无 console_scripts)。
> 项目定位:个人自用、单 admin——可按单用户简化,不为多租户/找回邮件等过度设计。
## 6. 任务依赖图
```
Phase A(防爆破,紧急,自成可发版增量)
M4-T01 [schema] auth_login_throttle 表 + 模型
└─► M4-T02 退避 service + 接入 login429/Retry-After/成功清零)
└─► M4-T03 CLI 骨架 + reset-password + unlock
Phase BTOTP,可选配)
M4-T04 [schema] AuthUser TOTP 字段 + auth_recovery_code 表
├─► M4-T05 TOTP service + setup/enable/disable/status API
├─► M4-T06 登录二因子(login 加 totp_code;依赖 T02 已改过 login
└─► M4-T07 CLI disable-totp / reissue-totp
M4-T08 前端:两步登录 + 设置页 TOTP(依赖 T05+T06;引入 qrcode.react
收尾
M4-T09 文档 + OpenAPI + roadmap(依赖全部)
```
`T01` 可先开。Phase A(T01–T03)跑完即可发一版只含防爆破 + CLI 的安全增量。
---
## 7. 原子任务(任务卡)
> 后端沿用校验闸门(`pytest`/`ruff`/改路由或 schema 则 `export_openapi` 重导出入库)。前端闸门见 §8。新增依赖(`pyotp``qrcode.react`)须同步 `requirements.in/.txt``frontend/package.json` 重新锁定。
### M4-T01 — `auth_login_throttle` 表 + 模型 `[schema]`
- **Status**: `done` · **Depends**: none
- **Context**: 存按 IP / username 的失败退避状态。仅建 schema。
- **Files**: `create app/models/auth_throttle.py`(或并入 `app/models/auth.py`);`create alembic_app/versions/<date>_NN_auth_login_throttle.py``modify alembic_app/env.py``scripts/app_db_adopt.py`baseline 常量);`create tests/test_auth_throttle_model.py`
- **Steps**: 模型 `LoginThrottle``id / key str / scope str('ip'|'user') / failures int / first_failed_at / last_failed_at / next_allowed_at``(scope,key)` unique 索引);新 revision 建表 + 索引;更新 baseline 常量。
- **Out of scope / 不要碰**: 不写退避逻辑(T02);不动登录端点。
- **Acceptance criteria**:
- [ ] 临时库 upgrade 到 head 含该表与唯一索引;`downgrade -1` 干净。
- [ ] `APP_BASELINE_REVISION` 更新;`env.py` 已 import。
- [ ] 校验闸门全绿。
- **Reviewer checklist**: `(scope,key)` 唯一;链上单 head;env.py 注册模型。
### M4-T02 — 退避 service + 接入登录
- **Status**: `done` · **Depends**: M4-T01
- **Context**: 指数退避核心 + 接 `POST /api/auth/login`
- **Files**: `create app/services/login_throttle.py``modify app/api/routes/api/session.py``app/services/config_page.py`(+`CONFIG_FIELDS` `AUTH_LOGIN_THROTTLE_ENABLED`)、`app/config.py`(+`auth_login_throttle_enabled``auth_trust_forwarded_for`)`modify tests/test_api_session.py``create tests/test_login_throttle.py`
- **Steps**:
1. `check_and_get_wait(session, *, ip, username) -> int`(秒;>0 表示仍在窗口);`register_failure(...)`(更新双键 failures/next_allowed_at,按 §3.1 公式);`clear(session, *, ip, username)`(成功清零)。
2. 登录端点:取 client IP(按 `auth_trust_forwarded_for` 决定是否信 XFF 最左);**先**查退避,>0 → 429 + `Retry-After`,不验密码;验密码失败 → `register_failure` 后 401;成功 → `clear` 再发 session。
3. 开关关闭时整段 no-op。
- **Out of scope / 不要碰**: 不做 TOTPPhase B);不改 logout/password 端点。
- **Acceptance criteria**:
- [ ] 单测:连续失败后 `wait` 按指数增长且封顶;窗口内请求 429 带 `Retry-After` 且不验密码。
- [ ] 单测:成功登录清零;下次无延迟。
- [ ] 401 仍为通用文案(不泄露用户是否存在);现有登录测试仍绿。
- [ ] 开关 false 时无节流。
- [ ] 校验闸门全绿(OpenAPI 若变重导出)。
- **Reviewer checklist**: 退避在验密码**之前**生效;双键较大值;XFF 仅在配置开启时信任;无把密码写进日志。
### M4-T03 — CLI 骨架 + reset-password + unlock
- **Status**: `done` · **Depends**: M4-T01
- **Context**: 逃生通道第一批命令。
- **Files**: `create scripts/admin_cli.py``create tests/test_admin_cli.py`
- **Steps**: argparse 子命令;`reset-password <username> [--password|prompt(getpass)]` 用 Argon2 hasher 重置 `password_hash``unlock [--all|--ip|--username]``auth_login_throttle` 行;`list-admin`;复用 `get_session_local()` + 模型;无网络。
- **Out of scope / 不要碰**: 不动 HTTP 端点;不碰用户数据表;TOTP 命令在 T07。
- **Acceptance criteria**:
- [ ] 单测:`reset-password` 后新密码可 `verify_password` 通过、旧密码失败。
- [ ] 单测:`unlock` 清掉指定/全部退避行。
- [ ] 不存在的 user 友好报错、非零退出。
- [ ] `grep` 确认 CLI 不含对用户数据表的删除/drop。
- [ ] 校验闸门全绿。
- **Reviewer checklist**: 密码可交互输入且不回显;只动 auth 行;幂等可重跑。
### M4-T04 — AuthUser TOTP 字段 + `auth_recovery_code``[schema]`
- **Status**: `done` · **Depends**: none(与 Phase A 并行的 schema,但 T06 依赖 T02
- **Files**: `modify app/models/auth.py``AuthUser.totp_secret`/`totp_enabled`);`create` 模型 `RecoveryCode``create alembic_app/versions/<date>_NN_totp.py``modify alembic_app/env.py``scripts/app_db_adopt.py``create tests/test_totp_models.py`
- **Steps**: `totp_secret` nullable、`totp_enabled` bool default false`auth_recovery_code(user_id FK, code_hash, used_at nullable)`revision 建列 + 表;更新 baseline。
- **Out of scope / 不要碰**: 不写 TOTP 逻辑(T05)。
- **Acceptance criteria**:
- [ ] upgrade/downgrade 干净;默认 `totp_enabled=false``totp_secret=null`
- [ ] baseline 常量更新;env.py import。
- [ ] 校验闸门全绿。
- **Reviewer checklist**: 现有用户迁移后默认未启用 TOTP(不破坏现有登录)。
### M4-T05 — TOTP service + setup/enable/disable/status API
- **Status**: `done` · **Depends**: M4-T04
- **Files**: `create app/services/totp.py``app/schemas/totp.py``modify app/api/routes/api/session.py`(或新 `auth_totp.py` 路由)、`app/config.py`(+`auth_totp_issuer`)`modify requirements.in/.txt`(+`pyotp`)`create tests/test_api_totp.py`
- **Steps**: pyotp 生成 secret + `otpauth://` URIissuer=`auth_totp_issuer`);恢复码生成(N=10、`xxxx-xxxx`Argon2 哈希存、明文仅 setup 返回一次);`setup`pending,不启用)/`enable`(带当前码确认)/`disable`(带密码或当前码)/`GET status`;全部 session+CSRF。
- **Out of scope / 不要碰**: 不改 login 校验(T06)。
- **Acceptance criteria**:
- [ ] `setup` 返回 secret+URI+恢复码(一次性);未确认前 `totp_enabled` 仍 false。
- [ ] `enable` 用错码失败、对码成功;恢复码以哈希持久化。
- [ ] `disable` 关闭并清 secret/恢复码。
- [ ] schema 经 OpenAPI 固化;secret/恢复码不回显于 `GET status`
- [ ] 校验闸门全绿。
- **Reviewer checklist**: 恢复码只存哈希、明文仅一次;secret 不进 OpenAPI 示例/日志;`requirements.txt` 锁定 `pyotp`
### M4-T06 — 登录二因子(login 加 `totp_code`
- **Status**: `done` · **Depends**: M4-T05, M4-T02
- **Files**: `modify app/api/routes/api/session.py``app/schemas/session.py``LoginRequest.totp_code` 可选)、`app/services/totp.py`(校验 + 消费恢复码);`modify tests/test_api_session.py``tests/test_api_totp.py`
- **Steps**: 密码通过后:若 `totp_enabled` 且无 `totp_code` → 401 `{totp_required:true}`(不发 session、退避照算);有 `totp_code` → 校验 TOTP 或恢复码(命中恢复码置 `used_at`)→ 通过发 session;未启用 → 现状。
- **Out of scope / 不要碰**: 不改 setup/enableT05);不动退避公式(T02)。
- **Acceptance criteria**:
- [ ] 未启用 TOTP 的用户登录与现状完全一致。
- [ ] 启用后:缺码→401 totp_required;错码→401;对码→发 session;恢复码可用且一次性。
- [ ] 退避对二步同样生效。
- [ ] OpenAPI 重导出入库;校验闸门全绿。
- **Reviewer checklist**: `totp_required` 时确未发 cookie;恢复码消费后不可复用;通用错误不泄露细节。
### M4-T07 — CLI disable-totp / reissue-totp
- **Status**: `done` · **Depends**: M4-T04(逻辑上配合 T03 的 CLI 骨架)
- **Files**: `modify scripts/admin_cli.py``modify tests/test_admin_cli.py`
- **Steps**: `disable-totp <username>``totp_enabled=false` + 清 secret + 删恢复码(**不需任何恢复码即可执行**);`reissue-totp <username>`:生成新 secret(可选打印新 URI)。
- **Out of scope / 不要碰**: 不碰用户数据表。
- **Acceptance criteria**:
- [ ] 单测:启用 TOTP 的用户经 `disable-totp` 后可纯密码登录(结合 T06 行为)。
- [ ] 不依赖任何恢复码即可关停。
- [ ] 校验闸门全绿。
- **Reviewer checklist**: 逃生不依赖已存凭据;只动 auth 行。
### M4-T08 — 前端:两步登录 + 设置页 TOTP
- **Status**: `done` · **Depends**: M4-T05, M4-T06
- **Files**: `modify frontend/src/pages/LoginPage.tsx``create frontend/src/pages/.../TotpSettings.tsx`(或并入 ConfigPage)、`frontend/src/auth/totp.ts``modify frontend/package.json`(+`qrcode.react`lock 同步)`create` 对应测试
- **Steps**: 登录页:401+`totp_required`→第二屏 6 位码(也接受恢复码);429→"稍后再试"(读 `Retry-After`)。设置页:未启用→启用流程(二维码由 `otpauth` URI 渲染 + 恢复码一次性展示 + 输码确认);已启用→停用。
- **Out of scope / 不要碰**: 不改后端;不碰防爆破逻辑。
- **Acceptance criteria**:
- [ ] 未启用 TOTP 的登录体验不变;启用后两步可走通;恢复码可登录。
- [ ] 二维码可被 Authenticator 扫描;恢复码仅展示一次。
- [ ] 429 有"稍后再试"提示。
- [ ] 前端闸门全绿。
- **Reviewer checklist**: 全走类型化 clientsecret/恢复码不落 localStorage/日志;空/错/加载态有处理。
### M4-T09 — 文档 + OpenAPI + roadmap 收尾
- **Status**: `done` · **Depends**: 全部
- **Files**: `modify README.md``docs/auth.md`(防爆破 + CLI + TOTP 说明、CLI 用法)、`docs/architecture-overview.md``modify docs/roadmap.md`(把"TOTP 二次验证"与新"登录防爆破"从「下一阶段」毕业、加 M4 行、注明 M4 先于 M5);`modify docs/design/README.md`(已含 m4);`run python scripts/export_openapi.py` 提交 `openapi/`
- **Acceptance criteria**:
- [ ] 文档含 CLI 逃生用法与 TOTP 启停说明;`git diff --exit-code openapi/` 无差异。
- [ ] 校验闸门全绿。
- **Reviewer checklist**: roadmap 反映 M4 已落地且先于 M5;无残留旧描述。
---
## 8. 前端校验闸门(前端任务每次结束都要全绿)
`frontend/` 下:`npm ci && npm run lint && npm run typecheck && npm run test && npm run build`。新增依赖 `qrcode.react` 须提交 `package.json` + `package-lock.json`。后端同任务改路由/schema 仍需根目录 `export_openapi.py` 并提交 `openapi/`
## 9. 构建上下文 / 依赖完整性
- 新增 Python 依赖 `pyotp`、前端 `qrcode.react`:同步重新锁定 `requirements.txt` / `package-lock.json`,否则镜像缺包。
- 新文件均在既有 `app/``scripts/``frontend/` 的 COPY 范围内;`scripts/admin_cli.py` 须在镜像里可 `python -m scripts.admin_cli` 调用(确认 `scripts/` 已进镜像)。
- `tests/test_deployment.py::test_dockerfile_copy_sources_exist` 仍应通过。
## 10. 人工验收 walkthrough(实现完成后)
> 重点:连续输错 → 系统自动退避锁住 → CLI 解锁后恢复登录。可用 `docker compose` 起服务,CLI 在容器内或容器外跑均可。
**准备**:起服务(任选其一)
- Docker`docker compose up -d`(应用容器名见 `docker-compose.yml`,下文记为 `<app>`)。
- 或本地:`source .venv/bin/activate && uvicorn app.main:app --port 8000`
**1) 触发退避锁定** —— 用**错误密码**连续登录同一账号:
```bash
for i in $(seq 1 8); do \
code=$(curl -s -o /dev/null -w "%{http_code}" -X POST http://localhost:8000/api/auth/login \
-H 'Content-Type: application/json' -d '{"username":"admin","password":"wrong"}'); \
echo "尝试 $i -> HTTP $code"; done
```
- 预期:前几次 `401`,超过免费次数后变 `429`(被锁/退避)。单独跑一次带 `-i` 可看到 `Retry-After` 头逐次增大(指数退避)。
- 在退避窗口内,**即使输正确密码**也应被 `429` 挡住(验证锁定确实生效)。
**2) CLI 解锁**
- 容器内:`docker compose exec <app> python -m scripts.admin_cli unlock --all`
- 或容器外:`python -m scripts.admin_cli unlock --all`
- 预期:打印清除的退避条目数。
**3) 验证恢复** —— 立即用**正确密码**登录:
```bash
curl -i -X POST http://localhost:8000/api/auth/login \
-H 'Content-Type: application/json' -d '{"username":"admin","password":"<正确密码>"}'
```
- 预期:`200``Set-Cookie` 下发 session → 解锁成功。
**4)(可选)其它逃生命令**
- 重置密码:`python -m scripts.admin_cli reset-password admin`(交互输入新密码、不回显)。
- 关停 TOTP(若已启用):`python -m scripts.admin_cli disable-totp admin`,随后纯密码即可登录。
## 11. 里程碑完成定义(DoD
- 登录失败按指数退避(429 + Retry-After),成功清零;可经 `AUTH_LOGIN_THROTTLE_ENABLED` 开关。
- `python -m scripts.admin_cli` 能在**无任何 Web 访问、无恢复码**的情况下重置密码 / 解锁 / 关停 TOTP。
- TOTP 可由 admin 自选启用:启用后两步登录、恢复码可用且一次性;不启用维持纯密码。
- 后端 `pytest`/`ruff`/`export_openapi` + 前端 `lint/typecheck/test/build` 全绿且 `openapi/` 入库。
- README/auth/architecture/roadmap 反映 M4 现实与"先于 M5"的排期。
+533
View File
@@ -0,0 +1,533 @@
# M5 — IoT 集成与能耗采集(Modbus 设备采集 + MQTT/HA Discovery + 前端侧边栏)
> 阅读前提:先读 [`README.md`](./README.md)(协作模型、任务卡格式、校验闸门、数据安全红线)。本里程碑建立在 M1 单库 + M2 React SPA 之上。
> 配套参考:电表协议见 [`../references/SDM120-Modbus-Protocol.md`](../references/SDM120-Modbus-Protocol.md)。
## 1. 目标
给后端接入家庭 IoT 生态,并新增一条通用的 Modbus 设备采集链路(首个落地的领域是能耗):
1. **Modbus 设备采集(通用管道)**:通过 Modbus-TCP 网关周期读取挂在网关后面的 Modbus slave 设备(首个 profile 为 SDM120 单相电表),按设备的 **YAML profile** 解码为工程量,存入单库的**通用读数表**。后台静默轮询,支持多设备、多 profile。
2. **MQTT + Home Assistant Discovery**:后端作为 MQTT 发布方,按"**可勾选暴露**"的方式把数据以 HA Discovery 自动注册成 device/entity(不止 sensor)。
3. **前端侧边栏 + Energy 视图**:把现有顶栏改成侧边导航,承载首个领域视图 **Energy**(设备管理 + 最新读数 + 走势图)。
> **命名分层(本里程碑的核心决策)**:**存储 / 采集 / API 是通用的 `modbus_*`**(不锁死在"电表"上,以后接别的 Modbus 设备无需改表);**面向用户的呈现是领域特定的**——本里程碑只落第一个领域视图 **Energy**,消费通用 device 数据、用电表视角展示。
> 三段有依赖关系,按 §6 的 `Depends` 顺序推进:A(侧边栏,独立)→ B(Modbus 采集后端 + Energy 前端)→ CMQTT/Discovery,消费 B 的数据)。
## 2. 现状(实现者可据此工作,不必通读全仓库)
**单库数据层**M1 完成态)
- `app/db.py``class Base(DeclarativeBase)`,绑 `settings.app_database_url` 的 cached engineWAL 已开),`get_engine` / `get_session_local` / `reset_db_caches` / `get_db_session`
- 模型都继承同一 `Base``app/models/{auth,config,public_ip,location,poo}.py`
- 单 Alembic 链 `alembic_app/`head = `20260611_06_merge_location_poo_tables`(见 `alembic_app/versions/`);`alembic_app/env.py` 逐个 import 所有模型。
- 迁移命名惯例:`YYYYMMDD_NN_<desc>.py``revision` / `down_revision` 串链。
- `scripts/app_db_adopt.py` 常量 `APP_BASELINE_REVISION` 指向当前 head`scripts/run_migrations.py` 负责把 app 库升到 head。
**配置系统(扁平 KV,自动渲染)**
- `app/config.py``class Settings(BaseSettings)`,每个配置项一个带类型的字段 + 默认值。
- `app/services/config_page.py``CONFIG_FIELDS: tuple[ConfigField, ...]` 是**注册表**`section / env_name / setting_attr / label / secret / input_type`);`build_config_sections`(读,secret 回空串)、`save_config_updates`(写,空 secret 保留旧值)、`build_runtime_settings`DB override 合并进 Settings)、`_settings_payload`(把 Settings 摊平成 dict,新字段要在此补一行)。
- `app/api/routes/api/config.py``GET/PUT /api/config`session + CSRF 保护;非法值 422 且不写库。
- 前端 `frontend/src/pages/ConfigPage.tsx`**通用渲染**——按 section 分组、按 `input_type`/`secret` 渲染输入框。**新增标量配置项零前端改动**(追加 `CONFIG_FIELDS` + `Settings` 字段 + `_settings_payload` 一行即可)。
- ⚠️ 扁平 KV **装不下"设备列表"和"逐实体勾选"**——这两者走专用表 + 专用 API + 自定义 UI(见 §3.2 / §3.4)。
**后台调度(APScheduler**
- `app/main.py` lifespan`BackgroundScheduler(timezone="UTC")``scheduler.add_job(_run_scheduled_public_ip_check, IntervalTrigger(hours=4), id=..., max_instances=1, coalesce=True)``scheduler.start()``yield``scheduler.shutdown(wait=False)`
- 周期任务惯例:一个**同步** wrapper 自己开/关 session`session = get_session_local()(); try: service(session, ...) finally: session.close()`service 内部不抛崩溃。
**Home Assistant 现状(REST,不碰 MQTT**
- `app/integrations/homeassistant.py``HomeAssistantClient.publish_sensor()``POST /api/states/{entity}`)、`trigger_webhook()`
- 入站 webhook `app/api/routes/homeassistant.py``POST /homeassistant/publish`envelope `target/action/content`)。
- 新 MQTT Discovery 与此**并行、不冲突**,是第二条独立通道。
**前端(M2**
- React + react-router v6 + Mantine + TanStack Query + `openapi-fetch` 生成的类型化 client`frontend/src/api/client.ts` + `schema.d.ts`)。
- `frontend/src/App.tsx``AppLayout`(当前是**顶栏**),包住所有受保护页;路由 `/`(HomePage 地图)、`/config``/records``/login``/change-password` 不带 layout。
- 数据请求惯例:`useQuery`/`useMutation` + `apiClient.GET/POST/...`(见 `frontend/src/records/hooks.ts`)。
- **无图表库**(只有 Leaflet 地图);走势图需新引入 **Recharts**
## 3. 目标架构
### 3.0 两层数据模型(本里程碑的地基决策)
把"设备是什么、怎么读"与"读到了什么"彻底分成两层,**协议知识与部署信息分离**:
```
┌─────────────────────────────────────────────────────────────┐
│ 协议层(固定,随代码走) 部署层(可配置,落 DB) │
│ ── YAML profile ────────── ── modbus_device 行 ────── │
│ • 读哪些寄存器(FC/地址/块) • friendly_name(可改) │
│ • 每个量:key / 类型 / 解码 • host / port(网关) │
│ • 每个量:unit / device_class • unit_id=电表 Meter ID
│ / ha_component 设备上可设,故落 DB) │
│ • 唯一真相源,喂给"读/存/HA注册" • profile 名(选哪个 YAML
│ • uuid / poll_interval / enabled│
└─────────────────────────────────────────────────────────────┘
▲ 多个 device 可共享同一 profile │
└─────────────────────────────────────────┘
│ 采集
── modbus_reading 行(通用遥测)──
device_id · recorded_at · payload(JSON blob)
```
- **profile = 仓库内只读 YAML**(如 `app/integrations/modbus/profiles/sdm120.yaml`),随镜像打包、纳入版本控制,**只描述固定协议知识**:读哪些寄存器、怎么解码、每个量的 `key`/`unit`/`device_class`/`ha_component`。**绝不**放 friendly_name / unit_id 这类每设备各异的可配置项。
- **可配置项落 `modbus_device` 行**,由前端 UI 设置:friendly_name、host、port、**unit_idModbus 从机地址,设备面板上可改)**、选用的 profile 名、poll_interval、enabled。
- **多设备共享一个 profile**:例如两块 SDM120 都 `profile="sdm120"`,一块 friendly_name="SDM120 空调"、unit_id=1,另一块 friendly_name="SDM120 服务器"、unit_id=2——共用同一份解码规则,部署参数各自独立。
- **读数存通用 JSON `payload`**,不再用固定列。profile 是解释器:知道 payload 里有哪些 key、各自单位与 device_class。
- 聚合不必搬到后端硬算:**SQLite 用 `json_extract` 在 SQL 端做 `AVG`/`MAX`/`GROUP BY`**(走势图都是带时间窗的查询,被 `(device_id, recorded_at)` 索引圈住,只扫窗内行)。
- 真有某个量需要高频聚合 → 后续给该量加 **generated column + 表达式索引**(非破坏性,要哪个补哪个),无需一开始把表锁成固定列。
### 3.1 Modbus 采集
- **传输:仅 Modbus-TCP**(用户的 Waveshare RTU↔TCP 网关,RJ45 以太网;服务器无串口)。用 **pymodbus**`ModbusTcpClient`,由它处理封帧 / CRC / 超时重试 / float 解码。
- 角色厘清:**我们的后端 = Modbus client/master****网关 = TCP server**(在 "Modbus TCP" 模式下终结 TCP 再转 RTU);**电表 = 网关后面的 RTU slave**,由 `unit_id` 寻址。`modbus_device` 一行同时装下网关地址(host:port)与 slave 地址(unit_id)。
- 网关若开"Modbus TCP"协议转换 → pymodbus 默认 framer 直连。
- 网关若是透传(RTU-over-TCP,裸 RTU 帧含 CRC)→ pymodbus 用 RTU framer over TCP。
- **二者实现时对一次即可确定**(连上读 Voltage 寄存器验证),不影响表结构与上层。
- **协议知识在 YAML profile,部署信息在 DB**(见 §3.0):
- `app/integrations/modbus/driver.py`:薄封装 pymodbus 的连接 + 块读 + 大端 float32 解码(word/byte 都大端,高寄存器在前)。
- `app/integrations/modbus/profiles.py`YAML profile 的**加载 + 校验 + 解码 + 实体枚举**——纯"数据 + 函数"**不做 OOP 抽象基类/继承**。`load_profile(name) -> ModbusProfile`(pydantic 模型,启动期校验)、`decode(profile, registers) -> dict[key -> value]``enumerate_entities(device, profile) -> list[ExposableEntity]`(供 §3.3)。
- `app/integrations/modbus/profiles/*.yaml`:每个设备型号一个 YAML。首个 = `sdm120.yaml`(寄存器地址见参考文档 §4)。
- profile 输出统一的 `dict[key -> value]`,由 service 整个塞进 `modbus_reading.payload`JSON);profile 没列出的量根本不读。
- **只读**:不写电表配置寄存器(改 Meter ID/波特率有通信中断风险)。
- **轮询**:每个设备一个 `poll_interval_s`(默认 5s),APScheduler 一个 job 扫所有 `enabled` 设备,逐设备块读→解码→落库。9600 总线上多设备串行,5s 余量充足。
- 全局开关 `MODBUS_POLLING_ENABLED`CONFIG_FIELDS)可一键停采。
- 采集成功/失败写入设备的 `last_poll_at` / `last_poll_ok`(供 §3.3 的 online binary_sensor)。
- **两级周期 / 降采样 / 保留是后续杠杆**(见 §10),本里程碑用单周期读全。
**`sdm120.yaml` 示例(只含固定协议知识)**
```yaml
name: sdm120
description: Eastron SDM120 single-phase energy meter
function_code: 4 # input registers (FC04)
word_order: big # 高寄存器在前
byte_order: big
blocks: # 块读,减少 Modbus 事务
- { start: 0x0000, count: 0x0060 } # 30001..30095 连续段
- { start: 0x0156, count: 0x0004 } # total active/reactive energy
metrics:
- { key: voltage, address: 0x0000, type: float32, unit: V, device_class: voltage, ha_component: sensor }
- { key: current, address: 0x0006, type: float32, unit: A, device_class: current, ha_component: sensor }
- { key: active_power, address: 0x000C, type: float32, unit: W, device_class: power, ha_component: sensor }
- { key: power_factor, address: 0x001E, type: float32, unit: "", device_class: power_factor, ha_component: sensor }
- { key: frequency, address: 0x0046, type: float32, unit: Hz, device_class: frequency, ha_component: sensor }
- { key: import_energy, address: 0x0048, type: float32, unit: kWh, device_class: energy, state_class: total_increasing, ha_component: sensor }
- { key: export_energy, address: 0x004A, type: float32, unit: kWh, device_class: energy, state_class: total_increasing, ha_component: sensor }
- { key: total_energy, address: 0x0156, type: float32, unit: kWh, device_class: energy, state_class: total_increasing, ha_component: sensor }
# 冷门量(demands / maxima / 无功电能…)可后续按需补进 metrics;未列出的就不读。
```
> 注意 YAML 里**没有** unit_id / friendly_name——那些是 `modbus_device` 行各自带的。
### 3.2 数据模型(新增两张通用表,单库 app 链)
**`modbus_device`**(设备定义 = 部署/可配置项,CRUD 管理)
| 列 | 类型 | 说明 |
| --- | --- | --- |
| id | int PK | 内部代理主键(FK join 用)|
| uuid | str unique | uuid4 生成的**稳定内部身份**;也作 API 路径键与 HA `unique_id` 的锚 |
| friendly_name | str | 显示名(可改;改名重发 discovery,HA 显示名跟着变)|
| transport | str | 现仅 `'tcp'` |
| host | str | 网关 IP |
| port | int | 网关端口(默认 502|
| unit_id | int | Modbus 从机地址 = 电表 Meter ID(设备面板可改,默认 1)|
| profile | str | 用哪个 YAML profile(首个 `'sdm120'`|
| poll_interval_s | int | 采样周期(默认 5|
| enabled | bool | 是否轮询 |
| last_poll_at | datetime null | 最近一次轮询时刻(online 判定)|
| last_poll_ok | bool null | 最近一次轮询成败(online 判定)|
| created_at / updated_at | datetime | |
**`modbus_reading`**(通用遥测表,一行 = 一个设备的一次采样)
| 列 | 类型 | 说明 |
| --- | --- | --- |
| id | int PK | |
| device_id | int FK→modbus_device.id | **ON DELETE RESTRICT**(见 §5 删除语义)|
| recorded_at | datetime (UTC) | 采样时刻,**真实列、带索引**(所有查询按它走时间窗)|
| payload | JSON | profile 解码出的全部工程量 `{key: value}`;无固定列 |
- 索引:`(device_id, recorded_at)`
- **SDM120 单相 payload 示例**`{"voltage": 230.2, "current": 1.3, "active_power": 295.0, "power_factor": 0.98, "frequency": 50.0, "import_energy": 123.4, "export_energy": 0.0, "total_energy": 123.4}`——key 由 `sdm120.yaml``metrics[].key` 决定。
- **接入新设备型号无需改表**:换 profilepayload 里的 key 集合随之变;表结构不动。
- **三相电表**以后用三相 profilepayload 里多几个相位 key(如 `voltage_l1/l2/l3`),仍是同一张表。
### 3.3 MQTT + HA Discovery(通用 expose 框架,元数据由 profile 派生)
HA MQTT Discovery 模型 = **device → entities**:往 `<prefix>/<component>/<node>/<object>/config` 发 retained 消息定义一个 entityconfig 内 `device.identifiers` 相同的 entity 归到同一个 HA device 卡片下;之后往 `state_topic` 推值。
- **可暴露实体目录由 provider 动态产出**
- `app/integrations/expose.py`:定义 `ExposableEntity``key`(稳定)、`component`sensor/binary_sensor/switch…)、`device`(归属,决定 HA device 分组)、`device_class``unit`、取值来源)+ 一个 provider 注册表。
- **Modbus/energy provider**:每个 `enabled` 设备 = 一个 HA **device**`identifiers` 用设备 `uuid`),其各工程量 = 一组 sensor entity——**`device_class`/`unit`/`component` 直接取自该设备 profile 的 `metrics[]`**(不再单独维护一份映射),外加一个 `binary_sensor`「online」(取 `last_poll_ok`)——这就是"**不止 sensor**"的体现。
- 其它 provider(如 public-ip、poo)可后续挂入,本里程碑只接 Modbus/energy provider。
- **HA 实体身份锚定(Z2M 模型)**discovery config 的 **`unique_id` 用设备 `uuid` + 量的 `key` 派生**(稳定,不随改名变);可见的 **`name` 用 friendly_name**。改 friendly_name → 重发 discovery → HA 显示名跟着变、但 `unique_id` 不变故历史不丢。topic 的 object_id 也用 uuid 派生(稳定、丑无所谓)。
- **`exposed_entities` 表**:只存"逐 key 的开关"`key` unique + `enabled` + `updated_at`)。目录本身由 provider 计算,表只记被勾选的状态(默认未勾 = 不暴露)。
- **MQTT 客户端**`app/integrations/mqtt.py`**paho-mqtt**`loop_start()` 后台线程;在 lifespan 起/停;支持配置变更后**重连 + 重发 discovery**。
- **发布时机**
- discovery configretained):连接成功时、目录/勾选变更时全量发;取消勾选时发空 payload 清除该 entity。
- state:采集在每次轮询后推最新值;另有一个周期 job 兜底重发所有 enabled 实体的 state + availability(在线)topic。
- **配置**(走现有扁平 CONFIG_FIELDS):`MQTT_ENABLED``MQTT_BROKER_HOST/PORT/USERNAME/PASSWORD(secret)``MQTT_TLS_ENABLED``HA_DISCOVERY_ENABLED``HA_DISCOVERY_PREFIX`(默认 `homeassistant`)。
### 3.4 前端
- **侧边栏**:把 `AppLayout` 从顶栏重构为侧边导航(Mantine `AppShell` 或 flex sidebar),导航项:Home / Records / Energy / Config + 主题切换 + 注销;当前路由高亮;移动端可折叠。仅改 `App.tsx`+ 可抽 `AppSidebar`/`NavItem` 组件),各页面主体不动。
- **Energy 视图**(新页 `/energy`,首个领域视图,消费通用 `/api/modbus` 数据):
- 设备管理:列表 + 新建/编辑/删除(删除有二次确认;后端对有读数的设备拒删,引导改用"禁用")。新建/编辑表单里设 friendly_name、host、port、unit_id、profile(下拉选 `sdm120` 等)、poll_interval、enabled。
- 最新读数卡片(每设备当前各工程量;字段标签/单位取自 profile 的 metrics 元数据,见 `/metrics` 端点)。
- 走势图:用 **Recharts** 画时间序列(电压/电流/功率/电能),时间范围选择,取数走 readings API(窗口 + 上限),从 `payload` 里按 key 取序列。
- **Expose 设置**:设置页内一块「Home Assistant Expose」——列出可暴露实体目录、逐项勾选、显示 MQTT/Discovery 连接状态、一个"重新发布 discovery"按钮。
## 4. API 契约(M5 要落地的端点)
> 全部 `/api` 前缀、session + CSRF(写)保护、JSON 进出。schema 经 `export_openapi.py` 固化入库。路径键用设备 `uuid`(稳定、非自增)。
| 分组 | 端点 | 用途 |
| --- | --- | --- |
| Modbus | `GET /api/modbus/devices` | 列出设备 |
| Modbus | `POST /api/modbus/devices` | 新建设备 |
| Modbus | `GET /api/modbus/devices/{uuid}` | 单个设备 |
| Modbus | `PATCH /api/modbus/devices/{uuid}` | 修改设备(含 enable/disable|
| Modbus | `DELETE /api/modbus/devices/{uuid}` | 删除设备;**有读数时 409**,引导改 disable |
| Modbus | `GET /api/modbus/devices/{uuid}/metrics` | 该设备 profile 的量目录(key/label/unit/device_class),供前端渲染卡片与图表标签 |
| Modbus | `GET /api/modbus/devices/{uuid}/latest` | 该设备最新一条读数(payload)|
| Modbus | `GET /api/modbus/devices/{uuid}/readings` | 时间范围读数(`start/end/limit`limit 有上限),返回 `recorded_at + payload`,供走势图 |
| Modbus | `POST /api/modbus/devices/{uuid}/test` | 即时试读一次(验证网关连通/地址),返回解码 payload,不落库 |
| Modbus | `GET /api/modbus/profiles` | 列出可用 profile 名 + 描述(前端建设备时的下拉选项)|
| Expose | `GET /api/expose` | 返回可暴露实体目录 + 勾选状态 + MQTT/Discovery 状态 |
| Expose | `PUT /api/expose` | 设置逐 key 勾选(map key→bool|
| Expose | `POST /api/expose/republish` | 手动重发 discovery |
| 配置 | `POST /api/config/mqtt/test` | 试连 broker **并发布一条测试消息**MQTT Explorer 可见),仿 SMTP 测试三态 |
> MQTT broker / discovery 的**标量配置**复用现有 `GET/PUT /api/config`(只新增 CONFIG_FIELDS,不新增端点)。
## 5. 已锁定决策(讨论后拍板)
1. **里程碑编排**:一个 M5 文档分三段,`Depends` 串顺序(A 侧边栏 → B Modbus/Energy → C MQTT)。
2. **两层数据模型,协议与部署分离**`modbus_device`(部署/可配置项:friendly_name、host、port、unit_id、profile 名、poll、enabled+ `modbus_reading`(通用遥测:device_id、recorded_at、`payload` JSON)。取代原"宽表 + 固定列"。
3. **读数 = JSON `payload`,无固定列**;聚合走 SQLite `json_extract`DB 端做 AVG/MAX/GROUP BY),热点量后补 generated column + 表达式索引。两级周期/降采样为后续杠杆。
4. **协议知识 = 仓库内只读 YAML profile**(声明式"数据 + 函数",无 OOP 继承,pydantic 启动期校验);YAML 携带寄存器 + 每个量的 key/unit/device_class/ha_component,是**唯一真相源**,同时驱动"读 / 存 / HA 注册"。多设备可共享一个 profile。首个 profile `sdm120`
5. **Modbus 仅 TCP**pymodbusframer 配网关模式,**只读**采集。设备是网关后面的 slave,`unit_id` 寻址(设备面板可改,故落 DB)。
6. **命名分层(方案 C**:存储/采集/API 一律通用 `modbus_*``/api/modbus/devices`;前端首个领域视图叫 **Energy**(消费通用 device 数据)。
7. **UUID = 内部生成(uuid4)的稳定身份**:既做内部索引/ API 路径键,也做 HA discovery `unique_id` 的锚;friendly_name 可改,改名重发 discovery、HA 显示名跟着变(Z2M 模型,历史不丢)。
8. **MQTT = 通用 expose 框架**:provider 动态产出可暴露实体目录,**实体元数据(device_class/unit/component)从 profile 派生**`exposed_entities` 只存逐 key 开关;支持 sensor/binary_sensor/switch 等多 component;设备自动注册(每设备一 device、各量为 entity + 一个 online binary_sensor)。
9. **MQTT 库 = paho-mqtt**,lifespan 长连接,配置变更后重连 + 重发 discovery。
10. **MQTT broker/discovery 标量配置走现有扁平 CONFIG_FIELDS**(自动渲染);设备清单与 expose 勾选走专用表 + 自定义 UI。
11. **图表库 = Recharts**(封在自包含组件后,仿 M2 对 Leaflet 的隔离)。
12. **删除设备安全**FK `ON DELETE RESTRICT`,有读数拒删(避免一键删表丢历史);"停用"用 `enabled=false`
13. **Config 页用 Accordion 分区**(进页见大类、逐类展开),**不在 config 页内再放第二个 side nav**——避免与主侧栏(T01)的"双抽屉"冲突;纯前端、独立任务 M5-T01B。
14. **CLI 手工测试工具为"受控手工链路验证"而设**(设备接市电、非随时在线,不进自动化):`scripts/modbus_cli` 提供 `read`profile 解码)与 `probe`(手工指定请求内容、看原始回复)两个子命令,**一律只读**(仅 FC03/04),不暴露写寄存器。
15. **MQTT/HA 发布链路的手工验证走 UI + 外部工具,不另做 CLI**Config 页「发送测试消息」(`mqtt/test` 发一条到测试 topic)→ 在 **MQTT Explorer** 查看;Expose 勾选实体 + 开 `HA_DISCOVERY_ENABLED` + 「重新发布 discovery」(`/api/expose/republish`)→ 到 **Home Assistant** 查看。不通则迭代配置再重发。
> 项目定位:个人自用、家庭特化、不开源——可按单用户场景简化,不过度抽象。
## 6. 任务依赖图
```
Phase A(独立,可最先做,纯前端)
M5-T01 [structural] 侧边栏布局重构
M5-T01B Config 页分区折叠(Accordion) ← 与 T01 互不依赖,可并行/先后任意
Phase BModbus 采集 + Energy 前端)
M5-T02 [schema] modbus_device + modbus_reading 表 + 模型
├─► M5-T03 Modbus 驱动 + YAML profile 框架(pymodbus + sdm120.yaml,纯模块)
│ └─► M5-T04 采集 service + APScheduler 轮询(接 lifespan,落 payload
└─► M5-T05 Modbus JSON APIdevice CRUD + readings + metrics + test
└─► M5-T06 前端:设备管理 UI(依赖 T01 侧栏 + T05 API
└─► M5-T07 前端:读数展示 + Recharts 走势图(依赖 T05;引入 recharts
Phase CMQTT / Discovery,依赖 B 的设备数据与 provider 接口)
M5-T08 MQTT/Discovery 配置项(CONFIG_FIELDS
M5-T09 [schema] exposed_entities 表 + ExposableEntity/provider 框架(modbus provider 从 profile 派生)
├─► M5-T10 MQTT 客户端(paholifespan 连接 + 重连 + config/mqtt/test
│ └─► M5-T11 Discovery 发布 + state 发布(连 T04 轮询推 state
└─► M5-T12 前端:Expose 勾选 UI + /api/expose API
收尾
M5-T13 文档 + OpenAPI + roadmap 收尾(依赖全部)
```
`T01``T01B``T02``T08` 无前置可先开。
---
## 7. 原子任务(任务卡)
> 后端任务沿用校验闸门(`pytest` / `ruff` / 改路由或 schema 则 `export_openapi` 重导出入库)。前端任务闸门见 §8。新增依赖(`pymodbus``PyYAML``paho-mqtt``recharts`)须在对应任务里同步 `requirements.in/.txt``frontend/package.json` 并重新锁定。
### M5-T01 — 侧边栏布局重构 `[structural]`
- **Status**: `done` · **Depends**: none
- **Context**: 把 `AppLayout` 从顶栏改为侧边导航,给后续 Energy 等视图腾入口。纯前端,不碰各页主体。
- **Files**: `modify frontend/src/App.tsx``create frontend/src/components/AppSidebar.tsx``frontend/src/components/NavItem.tsx`(可选);`modify` 受影响的 `frontend/src/pages/*.test.tsx`(导航断言)
- **Steps**:
1. 用 Mantine `AppShell`(或 flex sidebar)重构 `AppLayout`:左侧竖直导航(Home/Records/Config + 主题切换 + 注销),`<Outlet/>` 在右。
2. 当前路由高亮(`useLocation` 比对 `pathname`);移动端可折叠(burger)。
3. 导航项图标沿用 `react-feather`;样式走 Mantine(暗色模式自动适配)。
4. 不在本任务加 Energy 项(页面还不存在,T06 加),保持导航无死链。
- **Out of scope / 不要碰**: 不改各页面主体;不动鉴权(SessionProvider/ProtectedRoute);不引入图表库。
- **Acceptance criteria**:
- [ ] 受保护页都在侧边栏布局内;`/login``/change-password` 不带布局(与现状一致)。
- [ ] 当前路由在侧栏高亮;移动端宽度下可折叠/展开。
- [ ] 前端闸门全绿(`lint`/`typecheck`/`test`/`build`)。
- **Reviewer checklist**: 布局只在 `App.tsx`/新组件内变动,未误改页面或鉴权;无死链导航项。
### M5-T01B — Config 页分区折叠(Accordion
- **Status**: `done` · **Depends**: none
- **Context**: config 内容会越来越多(M5 还要加 MQTT / HA Discovery / Modbus 配置 + Expose 面板)。把 ConfigPage 从一长串 section 改为 Mantine `Accordion`:进页只见各大类标题,逐类展开编辑。**纯前端、单列内容**——它不是"第二个 app 级侧栏",与主侧栏(T01)无冲突,故 `Depends: none`、可独立先做。
- **Files**: `modify frontend/src/pages/ConfigPage.tsx``modify frontend/src/pages/ConfigPage.test.tsx`(折叠/展开断言)
- **Steps**:
1. 用 Mantine `Accordion` 包住现有"按 section 分组"的渲染:每个 config section = 一个 `Accordion.Item`(标题 = section 名,面板 = 该 section 的字段表单)。
2. 默认折叠(或首个展开);**保留现有保存逻辑与 config API 不变**M2 的整页/按 section 保存语义照旧,本任务不碰后端)。
3. 预留:将来 T12 的「Home Assistant Expose」自定义面板也作为一个 `Accordion.Item` 接入,保持一致。
4. 移动端单列堆叠即可,**不引入第二个侧栏/抽屉**。
- **Out of scope / 不要碰**: 不改后端 config API/字段(`CONFIG_FIELDS` / `_settings_payload`);不动主侧栏(T01);不改保存语义;不在 config 页内放任何 app 级 side nav。
- **Acceptance criteria**:
- [ ] ConfigPage 以 accordion 呈现,每大类可独立展开/折叠;字段渲染与保存行为与现状一致。
- [ ] 与主侧栏无视觉/交互冲突(单列内容;移动端不出现双抽屉)。
- [ ] 前端闸门全绿(`lint`/`typecheck`/`test`/`build`)。
- **Reviewer checklist**: 仅改 ConfigPage+其测试),未碰 config 后端或主 layout;保存逻辑无回归;页面内**无第二个 app 级 sidebar**accordion 是内容、非 chrome)。
### M5-T02 — `modbus_device` + `modbus_reading` 表与模型 `[schema]`
- **Status**: `done` · **Depends**: none
- **Context**: 单库 app 链新增两张**通用**表,建出 §3.2 结构(设备 = 部署层,读数 = JSON payload 通用遥测层)。本任务只建 schema + 模型,不写采集/接口。
- **Files**: `create app/models/modbus.py``ModbusDevice``ModbusReading`,继承 `app.db.Base`);`create alembic_app/versions/<date>_07_modbus_tables.py``modify alembic_app/env.py`import 新模型);`modify scripts/app_db_adopt.py``APP_BASELINE_REVISION` → 新 head);`create tests/test_modbus_models.py`
- **Steps**:
1. 模型按 §3.2 列定义(`Mapped[...]` 2.0 风格):`ModbusDevice``uuid`(unique)、`friendly_name``transport``host``port``unit_id``profile``poll_interval_s``enabled``last_poll_at`/`last_poll_ok`(nullable)、时间戳;`uuid``default` 生成 uuid4 字符串。`ModbusReading``device_id` FK→`modbus_device.id`**`ondelete="RESTRICT"`**)、`recorded_at``payload`JSON 列,非 null)。
2. 新 revision`down_revision` = 当前 head`upgrade()``op.create_table` 建两表 + 索引 `(device_id, recorded_at)``downgrade()` 反向 drop。
3. 更新 `APP_BASELINE_REVISION`
- **Out of scope / 不要碰**: 不写 pymodbus/采集(T03/T04);不加路由(T05);不动其它模型。
- **Acceptance criteria**:
- [ ] 全新临时 app 库 upgrade 到 head 后含 `modbus_device``modbus_reading` 及索引;`downgrade -1` 干净回滚。
- [ ] `Base.metadata.tables` 含两新表;FK 为 RESTRICT`uuid` 唯一且自动生成。
- [ ] `APP_BASELINE_REVISION` == 新 head。
- [ ] 校验闸门全绿。
- **Reviewer checklist**: 列/约束与 §3.2 一致;`payload` 为 JSON 列;`recorded_at` 为真实索引列(非塞进 payload);链上单 head`env.py` 已 import 新模型(否则 autogenerate/建表漏表)。
### M5-T03 — Modbus 驱动 + YAML profile 框架(`sdm120`
- **Status**: `done` · **Depends**: M5-T02
- **Context**: 薄封装 pymodbus 的连接/块读/大端 float 解码,加 YAML profile 加载/校验/解码框架与 SDM120 profile。纯模块,mock client 单测。
- **Files**: `create app/integrations/modbus/__init__.py``app/integrations/modbus/driver.py``app/integrations/modbus/profiles.py``app/integrations/modbus/profiles/sdm120.yaml``scripts/modbus_cli.py``modify requirements.in`/`requirements.txt`(加 `pymodbus``PyYAML`,重新锁定);`create tests/test_modbus_driver.py``tests/test_modbus_profiles.py``tests/test_modbus_cli.py`
- **Steps**:
1. `driver.py``read_blocks(host, port, unit_id, blocks) -> dict[int,int]`(按 profile 的 blocks 块读 input registersFC04),用 `ModbusTcpClient`;超时/连接失败抛明确异常;大端 float32 解码 helper`registers_to_float`,高寄存器在前)。framer 选择留可配置/可探测。
2. `profiles.py`:定义 pydantic `ModbusProfile` / `MetricSpec``key`/`address`/`type`/`unit`/`device_class`/`state_class`?/`ha_component`);`load_profile(name) -> ModbusProfile`(读 `profiles/<name>.yaml`,校验失败抛错);`decode(profile, registers) -> dict[key -> value]`(按各 metric 的 address+type 从寄存器对解码);`list_profiles() -> list[(name, description)]`。**不做抽象基类/继承**,全是数据 + 模块函数。
3. `profiles/sdm120.yaml`:按 §3.1 示例写全核心量(参考文档 §4 地址)。
4. `scripts/modbus_cli.py`:纯命令行、**不依赖 DB / 不需先配设备**,供**受控手工测试**(设备接市电、非随时在线,不进自动化)。两个**只读**子命令:
- `read --host H --port P --unit U --profile sdm120`:按 profile 读一次、解码后**把各工程量打印成可读结果**(表格/JSON)——验证整套解码链路。
- `probe --host H --port P --unit U --fc 4 --address 0x0000 --count 2 [--decode float32]`:**手工指定要发送的请求内容**(功能码 FC03/04 + 起始地址 + 数量),打印**原始寄存器(hex)+ 可选大端 float 解码**——first-contact 验证网关连通、framer 模式与 unit 地址,可单读一个寄存器,不依赖 profile。
连接失败给清晰报错 + 非零退出。**只读**:CLI 仅暴露读功能码(FC03/04),**不提供任何写寄存器子命令**(数据红线 + 防改坏电表通信参数)。
- **Out of scope / 不要碰**: 不连真实硬件(单测用 mock/fake 返回已知寄存器字节);不写调度(T04);不写电表配置寄存器;不在 profile 里放 unit_id/friendly_name。
- **Acceptance criteria**:
- [ ] 单测:给定 `0x4366,0x3334` 解码为 `230.2`(参考文档实例);字序/字节序正确。
- [ ] 单测:`load_profile("sdm120")` 校验通过;`decode(profile, ...)` 把已知寄存器映射到正确 key 与值。
- [ ] 单测:profile YAML 缺字段/类型错时 `load_profile` 抛可识别校验错。
- [ ] 连接失败/超时抛可识别异常,不静默返回错值。
- [ ] `python -m scripts.modbus_cli read ...` 能(对 mock/真实网关)打印解码后的各工程量;连接失败非零退出。
- [ ] `python -m scripts.modbus_cli probe --fc 4 --address 0x0000 --count 2 ...` 能打印原始寄存器与可选解码值;CLI **无任何写寄存器子命令**
- [ ] 校验闸门全绿。
- **Reviewer checklist**: 解码确为大端、高寄存器在前;地址与参考文档一致;**CLI 与 driver 都无任何写寄存器路径(仅 FC03/04)**;profile 纯协议知识、无部署项;`requirements.txt` 已同步锁定 `pymodbus``PyYAML`
### M5-T04 — 采集 service + APScheduler 轮询
- **Status**: `done` · **Depends**: M5-T03
- **Context**: 周期扫所有 enabled 设备,调用 driver 读+解码,落 `modbus_reading.payload`。仿 public-ip 的同步 job 模式。
- **Files**: `create app/services/modbus_poll.py``modify app/main.py`lifespan 注册 job);`create tests/test_modbus_poll.py`
- **Steps**:
1. `modbus_poll.py``poll_device(session, device) -> ModbusReading | None``load_profile` → driver 读 → `decode` → 把 dict 存进 `payload` 插一行;更新 `last_poll_at`/`last_poll_ok`);`poll_all_enabled_devices(session)` 遍历 enabled 设备;service 内吞异常并日志,不让 job 崩。
2. `main.py`:加同步 wrapper `_run_scheduled_modbus_poll`(自管 session),`add_job(IntervalTrigger(seconds=...), id="modbus-poll", max_instances=1, coalesce=True)`。周期取**最小 per-device interval 或一个基础 tick**(实现可用单一基础 tick + 各设备按自身 interval 取模决定本 tick 是否读,保持 job 简单);受全局 `MODBUS_POLLING_ENABLED` 控制。
3. 失败的设备记 `last_poll_ok=false`(供 T11 暴露),不影响其它设备。
- **Out of scope / 不要碰**: 不发 MQTTT11);不加 HTTP 路由(T05);不引入两级周期(后续杠杆)。
- **Acceptance criteria**:
- [ ] 单测:mock driver 返回已知 dict`poll_all_enabled_devices``modbus_reading` 精确 +N 行、`payload` 内容正确。
- [ ] 单测:某设备读失败时其它设备仍正常落库,job 不抛;失败设备 `last_poll_ok=false`
- [ ] `MODBUS_POLLING_ENABLED=false` 时不轮询。
- [ ] 校验闸门全绿。
- **Reviewer checklist**: session 在 wrapper 内开关、try/finally 关闭;job `max_instances=1` 防叠加;无 N+1/每行单独 connect 的明显低效;异常不外泄崩 job;payload 为解码后的 dict(非裸寄存器)。
### M5-T05 — Modbus JSON APIdevice CRUD + readings + metrics + test
- **Status**: `done` · **Depends**: M5-T02
- **Context**: 给前端提供设备 CRUD、最新读数、时间范围读数、量目录、即时试读。
- **Files**: `create app/api/routes/api/modbus.py``app/schemas/modbus.py``modify app/main.py`(注册路由);`create tests/test_api_modbus.py`
- **Steps**:
1. devices`GET`(list)/`POST`/`GET{uuid}`/`PATCH{uuid}`/`DELETE{uuid}`session+CSRF`POST` 校验 profile 名存在;`DELETE` 有读数 → 409。
2. readings`GET {uuid}/latest`(最新一行 payload)、`GET {uuid}/readings``start/end/limit`limit 有上限防全表导出,按 `recorded_at` 升序,返回 `recorded_at + payload`)。
3. `GET {uuid}/metrics`:返回该设备 profile 的量目录(key/label/unit/device_class),供前端渲染。
4. `GET /api/modbus/profiles``list_profiles()` 的名+描述。
5. `POST {uuid}/test`:用 driver 即时读一次返回解码 payload(或错误),**不落库**。
- **Out of scope / 不要碰**: 不在此处发 MQTT;不写采集逻辑(复用 T03/T04 的 driver/service)。
- **Acceptance criteria**:
- [ ] CRUD 行为正确:创建/改/删行数精确;删有读数的设备返回 409;未登录 401、缺 CSRF 403;建设备引用不存在 profile → 422。
- [ ] readings 时间范围 + limit 上限生效;latest 返回最新一条 payloadmetrics 返回 profile 量目录。
- [ ] schema 经 OpenAPI 固化入库。
- [ ] 校验闸门全绿(含 `openapi/` 重导出)。
- **Reviewer checklist**: 删除受 RESTRICT 保护、无批量删/清表路径;查询走 `(device_id, recorded_at)` 索引;test 端点确不落库;路径键用 `uuid`
### M5-T06 — 前端:设备管理 UIEnergy 视图)
- **Status**: `done` · **Depends**: M5-T01, M5-T05
- **Context**: 在侧栏加 Energy 入口与 `/energy` 路由;设备增删改 + 试读。
- **Files**: `create frontend/src/pages/EnergyPage.tsx``frontend/src/energy/DeviceForm.tsx``frontend/src/energy/hooks.ts``modify frontend/src/App.tsx`(路由)、`frontend/src/components/AppSidebar.tsx`Energy 项);`create` 对应 `*.test.tsx`
- **Steps**: `useQuery`/`useMutation``/api/modbus/devices` API;列表 + 新建/编辑表单(friendly_name/host/port/unit_id/profile 下拉/poll/enabled+ 删除二次确认(删失败 409 提示改用禁用);"试读"按钮调 `POST {uuid}/test` 显示结果。
- **Out of scope / 不要碰**: 走势图在 T07;不碰其它页面。
- **Acceptance criteria**:
- [ ] 能增/改/删设备并即时刷新;删除有二次确认;409 有友好提示;profile 走 `/api/modbus/profiles` 下拉。
- [ ] 侧栏出现 Energy 入口、`/energy` 可达。
- [ ] 前端闸门全绿。
- **Reviewer checklist**: 全部走生成的类型化 client;删除走确认;无与契约不符的手写请求。
### M5-T07 — 前端:读数展示 + Recharts 走势图
- **Status**: `done` · **Depends**: M5-T05(数据), M5-T06(页面壳)
- **Context**: 在 Energy 页展示每设备最新读数 + 时间序列走势。
- **Files**: `modify frontend/src/pages/EnergyPage.tsx``create frontend/src/energy/EnergyCharts.tsx`(封装 Recharts);`modify frontend/package.json`(加 `recharts``package-lock.json` 同步);`create` 对应测试
- **Steps**: 最新读数卡片(接 `latest`,字段标签/单位取自 `/metrics`);时间范围选择 + 折线图(电压/电流/功率/电能,从 `payload` 按 key 取序列),接 `readings`(窗口 + limit);图表封在 `EnergyCharts` 内(仿 Leaflet 隔离,便于将来换库)。
- **Out of scope / 不要碰**: 不做服务端降采样(后续);不改后端。
- **Acceptance criteria**:
- [ ] 最新读数与走势图渲染正确;时间范围只取窗口数据(不拉全量);标签/单位来自 metrics。
- [ ] Recharts 封装自包含、仅此处 import。
- [ ] 前端闸门全绿(`build` 通过,注意 chunk 体积提示)。
- **Reviewer checklist**: 图表组件隔离;查询有窗口/上限;空数据/加载/错误态有处理;从 payload 取 key 的逻辑容忍缺 key。
### M5-T08 — MQTT / Discovery 配置项(CONFIG_FIELDS
- **Status**: `done` · **Depends**: none
- **Context**: 把 MQTT broker 与 discovery 的标量配置接入扁平配置系统(前端自动渲染)。
- **Files**: `modify app/config.py`(新增 Settings 字段)、`app/services/config_page.py`(追加 CONFIG_FIELDS + `_settings_payload`);`modify .env.example``modify tests/test_api_config.py`
- **Steps**: 加字段 `mqtt_enabled``mqtt_broker_host/port/username/password`(secret)、`mqtt_tls_enabled``ha_discovery_enabled``ha_discovery_prefix`(默认 `homeassistant`)、`modbus_polling_enabled`CONFIG_FIELDS 归入「MQTT」「Home Assistant Discovery」「Modbus」section`_settings_payload` 补齐对应行。
- **Out of scope / 不要碰**: 不建 MQTT 客户端(T10);不动 expose 表(T09)。
- **Acceptance criteria**:
- [ ] 新配置项在 `GET /api/config` 出现且分 sectionpassword 为 secret(回空、留空保留);port 为 number。
- [ ] 非法值(端口非数字)422 不写库。
- [ ] 校验闸门全绿(OpenAPI 若变化则重导出)。
- **Reviewer checklist**: secret 不回显/不入 OpenAPI 示例;`_settings_payload` 未漏字段(否则运行期 override 丢失)。
### M5-T09 — `exposed_entities` 表 + ExposableEntity/provider 框架 `[schema]`
- **Status**: `done` · **Depends**: M5-T02
- **Context**: 建"可暴露实体目录"的抽象与开关存储;modbus provider 把设备映射成 device/entities**实体元数据从 profile 派生**。
- **Files**: `create app/integrations/expose.py``ExposableEntity`、provider 协议、注册表、modbus provider);`create app/models/expose.py``ExposedEntityToggle``key` unique + `enabled` + `updated_at`);`create alembic_app/versions/<date>_08_exposed_entities.py``modify alembic_app/env.py``scripts/app_db_adopt.py``create tests/test_expose_catalog.py`
- **Steps**:
1. `ExposableEntity``key/component/device/device_class/unit/value_getter`+ provider 接口 `enumerate(session) -> list[ExposableEntity]`
2. modbus provider:每个 enabled 设备 → 一个 device`identifiers` 用设备 `uuid`),其各量 → sensor entity**device_class/unit/component 取自 profile 的 `metrics[]`**),加一个 `binary_sensor` online(取 `last_poll_ok`)。
3. `build_catalog(session)` 合并所有 provider 的实体 + 各自 `enabled`(来自 toggle 表,缺省 false)。
4. migration 建 toggle 表;更新 baseline 常量。
- **Out of scope / 不要碰**: 不发 MQTTT11);不加 HTTPT12)。
- **Acceptance criteria**:
- [ ] 单测:建若干设备后 `build_catalog` 产出每设备对应 entity(含 online binary_sensor+ 正确 device 分组、device_class、unit(与 profile 一致)。
- [ ] toggle 表 migration 可升/降;缺省 enabled=false。
- [ ] 至少含一个非 sensor componentonline binary_sensor)。
- [ ] 校验闸门全绿。
- **Reviewer checklist**: `key` 稳定(用设备 `uuid` + 量 key,不用自增 id,避免重建漂移);component 支持多类型;目录由 provider 计算、元数据源自 profile 而非写死。
### M5-T10 — MQTT 客户端(paholifespan 连接 + 重连)
- **Status**: `done` · **Depends**: M5-T08
- **Context**: 长连接 MQTT 客户端,配置变更可重连;含连接测试端点。
- **Files**: `create app/integrations/mqtt.py``modify app/main.py`lifespan 起/停)、`app/api/routes/api/config.py``POST /api/config/mqtt/test`);`modify requirements.in`/`requirements.txt`(加 `paho-mqtt` 重新锁定);`create tests/test_mqtt_client.py`
- **Steps**:
1. `MqttManager``is_configured()``connect()/disconnect()/reconnect(settings)``publish(topic, payload, retain)`paho `loop_start()` 后台线程;未配置/未启用则 no-op。
2. lifespan:启用则 connectshutdown disconnect。
3. 配置保存后若 MQTT 设置变化 → 触发 manager 重连(在 config 保存路径加 hook 或保存后比对)。
4. `POST /api/config/mqtt/test`:用提交/现存配置试连**并发布一条测试消息**到一个测试 topic(如 `<discovery_prefix>/home-automation/test`),返回三态(success/config-error/failed)。仿 SMTP 测试"发一封测试邮件"的语义——用户随后在 **MQTT Explorer** 里就能看到这条消息,确认 broker 发布链路通(这是 MQTT 端的手工验证手段,**不另做 CLI**)。
- **Out of scope / 不要碰**: 不构建 discovery/state 消息(T11)。
- **Acceptance criteria**:
- [ ] 单测(fake broker/paho mock):configured 时 connect 调用正确;未配置 no-oppublish 透传 topic/payload/retain。
- [ ] `POST /api/config/mqtt/test` 试连**并发布一条测试消息**(可在 MQTT Explorer 看到);三态有明确返回;session+CSRF 保护。
- [ ] 校验闸门全绿。
- **Reviewer checklist**: 断网/连接失败不崩主进程;线程在 shutdown 正确停止;`requirements.txt` 同步锁定 `paho-mqtt`;密码不进日志。
### M5-T11 — Discovery 发布 + state 发布
- **Status**: `done` · **Depends**: M5-T09, M5-T10
- **Context**: 把 enabled 实体发成 HA discovery configretained)并周期推 state;采集轮询后推最新值。
- **Files**: `create app/services/ha_discovery.py``modify app/services/modbus_poll.py`(轮询后推 state)、`app/main.py`state 周期 job + 连接后/勾选变更后发 discovery)、`app/api/routes/api/...``/api/expose/republish` 在 T12 接,本任务提供 service);`create tests/test_ha_discovery.py`
- **Steps**:
1. `build_discovery_payload(entity)` → HA 规范 config`<prefix>/<component>/<node>/<object>/config`,含 `device` 块、`state_topic``device_class``unit_of_measurement``availability`**`unique_id` 用设备 uuid + 量 key 派生,`name` 用 friendly_name**)。
2. `publish_discovery(session)`:对 enabled 实体发 retained config;对取消勾选的发空 payload 清除。
3. `publish_states(session)`:取各实体当前值发 state;采集在 `poll_device` 成功后顺带推该设备实体 state + online。
4. lifespan:连接成功 / 目录或勾选变更后 `publish_discovery`;周期 job 兜底 `publish_states` + availability。
- **Out of scope / 不要碰**: 不做前端(T12);不改采集解码逻辑。
- **Acceptance criteria**:
- [ ] 单测:discovery payload 符合 HA 结构(device 分组正确、topic/device_class/unit 正确、`unique_id` 取自 uuid);取消勾选发空 payload。
- [ ] 单测:采集轮询成功后推对应 state topic;失败推 online=false。
- [ ] discovery 用 retained。
- [ ] 校验闸门全绿。
- **Reviewer checklist**: 仅发 enabled 实体;entity `unique_id` 稳定(源自 uuid,不随改名变);MQTT 未启用时整链 no-op;不阻塞轮询。
### M5-T12 — 前端:Expose 勾选 UI + `/api/expose`
- **Status**: `done` · **Depends**: M5-T09, M5-T11
- **Context**: 后端 expose 读写端点 + 设置页勾选界面。
- **Files**: `create app/api/routes/api/expose.py``app/schemas/expose.py``modify app/main.py``create tests/test_api_expose.py`;前端 `create frontend/src/pages/.../ExposeSettings.tsx`(或并入 ConfigPage)、`modify` 路由/设置入口;`create` 前端测试
- **Steps**: `GET /api/expose`(目录 + 勾选 + MQTT/Discovery 状态)、`PUT /api/expose`key→bool)、`POST /api/expose/republish`(调 T11 service);前端列出目录、按 device 分组、逐项开关、显示连接状态、"重新发布"按钮。
- **Out of scope / 不要碰**: 不改采集/发布逻辑(T11)。
- **Acceptance criteria**:
- [ ] `GET/PUT /api/expose` 正确读写勾选;session+CSRFOpenAPI 固化。
- [ ] 勾选变更后(或点重新发布)触发 discovery 重发。
- [ ] 前端能勾选并显示状态;前后端闸门全绿。
- **Reviewer checklist**: PUT 只改 toggle 不误碰其它配置;republish 真触发 T11;类型化 client。
### M5-T13 — 文档 + OpenAPI + roadmap 收尾
- **Status**: `done` · **Depends**: 全部
- **Files**: `modify README.md`Modbus/Energy/MQTT 段、新依赖)、`docs/roadmap.md`M5 行 + 把"MQTT/IoT"从"下一阶段"毕业、新增 Modbus 采集方向)、`docs/architecture-overview.md`(新增 MQTT 通道与 Modbus 采集);`modify docs/design/README.md`(列入 m5);`run python scripts/export_openapi.py` 并提交 `openapi/`
- **Acceptance criteria**:
- [ ] 文档反映新链路;`git diff --exit-code openapi/` 无未提交差异。
- [ ] 校验闸门全绿。
- **Reviewer checklist**: 无残留旧描述(含旧 `energy_meters`/`/api/energy` 字样);OpenAPI 已入库。
---
## 8. 前端校验闸门(前端任务每次结束都要全绿)
`frontend/` 下:
```bash
npm ci
npm run lint
npm run typecheck
npm run test
npm run build # 必须产出 dist;留意 chunk 体积告警
```
- 后端若同任务改了路由/schema,仍需根目录 `python scripts/export_openapi.py` 并提交 `openapi/`
- 新增前端依赖(recharts)须提交 `package.json` + `package-lock.json`
## 9. 构建上下文完整性(M1 教训)
- 本里程碑**不删/移文件**,但新增了 Python 依赖(`pymodbus``PyYAML``paho-mqtt`)与前端依赖(`recharts`):必须同步 `requirements.in/.txt` 的重新锁定与 `package-lock.json`,否则镜像构建会缺包。
- 新增源文件都在 `app/``scripts/``frontend/` 既有 COPY 范围内,无需改 `Dockerfile``COPY`**但 `app/integrations/modbus/profiles/*.yaml` 是非 .py 资源**——确认 `COPY app ...` 把整个目录(含 YAML)带进镜像,且运行期能按相对路径定位 YAML(建议用 `importlib.resources` 或基于 `__file__` 的路径,别用 CWD 相对路径)。`scripts/modbus_cli.py` 须在镜像里可 `python -m scripts.modbus_cli` 调用;`tests/test_deployment.py::test_dockerfile_copy_sources_exist` 仍应通过。
- 发版前置走查(见 CLAUDE.md):真起 app 跑一次轮询、真连一次 broker、前端 Energy 视图人工瞄一眼渲染,再打 tag。
## 10. 后续杠杆(本里程碑不做,文档留痕)
- **热点量的 generated column + 索引**JSON `payload` 默认无法对单个量建索引;某个量若需高频聚合,给 `modbus_reading` 加一列 `GENERATED ALWAYS AS (json_extract(payload,'$.<key>'))` 并建索引——非破坏性、要哪个补哪个。
- **两级采样周期**:瞬时量(V/I/P/PF/Hz)快、累计电能慢——9600 总线吃紧或要把功率压到 5s 以下时再开(device 表加 `slow_interval_s`,读数表是通用 payload、无需改结构)。
- **保留 / 降采样**:5s 采样长期行数大(≈630 万行/年/设备),加定期降采样或保留窗口任务(独立于本里程碑),`GROUP BY` + `AVG(json_extract(...))` 在 SQL 端做。
- **更多 device profile**:三相电表(如 SDM630)新增一个 YAML profilepayload 里多几个相位 key),不改表、不改采集主链。
- **更多 expose provider**public-ip / poo 等挂入 expose 框架。
- **写 Modbus 寄存器**:当前只读;如需经 MQTT/UI 控制设备(switch 类),再单独评估安全边界。
## 11. 人工验收 walkthrough(实现完成后)
> 重点:用命令行**手工**读到设备数据并展示结果。这些是**受控手工测试**——设备接市电、并非随时在线,故**不纳入自动化测试**(自动化只用 mock);CLI 工具(`modbus_cli read/probe`)就是为这种"我想测的时候手动测一次"而设。可用 `docker compose` 起环境,命令在容器内或容器外跑均可。
**前提**:网关(Waveshare RTU↔TCP)已上电接入网络,电表 Meter ID 已知(默认 1)。
**1) 命令行直接试读(不依赖 DB,最快验证)**
- 容器内:`docker compose exec <app> python -m scripts.modbus_cli read --host <网关IP> --port 502 --unit 1 --profile sdm120`
- 或容器外:`source .venv/bin/activate && python -m scripts.modbus_cli read --host <网关IP> --port 502 --unit 1 --profile sdm120`
- 预期:打印解码后的工程量(电压/电流/有功功率/功率因数/频率/导入导出电能等),数值合理;连不上则清晰报错。
- **first-contact / 原始验证**`python -m scripts.modbus_cli probe --host <网关IP> --port 502 --unit 1 --fc 4 --address 0x0000 --count 2 --decode float32` —— 手工指定请求内容、看原始回复(电压寄存器应解出约 230V),用于确认网关 framer 模式与 unit 地址;建 profile / 设备前就能跑。
**2)(可选)经 API 试读已配置的设备**
- 先在前端 Energy 页或 `POST /api/modbus/devices` 建一个设备;
- `POST /api/modbus/devices/{uuid}/test` 即时试读,返回解码 payload(不落库)。
**3)(可选)验证后台轮询落库**
- 确认 `MODBUS_POLLING_ENABLED=true` 且设备 `enabled`;等一个采样周期;
- 看前端 Energy 视图的最新读数/走势图,或查 `GET /api/modbus/devices/{uuid}/readings` 有新行。
**4)(可选)手工验证 MQTT / HA Discovery 发布链路**(受控手工步骤,不进自动化)
- **broker 发布链路**:配好 MQTT broker 后,在 Config 页点「发送测试消息」(`POST /api/config/mqtt/test`)——它试连并发一条测试消息;打开 **MQTT Explorer** 确认能收到,即链路通。
- **HA Discovery**:在 Expose 设置勾选若干实体、开 `HA_DISCOVERY_ENABLED`,点「重新发布 discovery」(`POST /api/expose/republish`);到 **Home Assistant** 确认对应 device/entity 出现、值正确。
- **改名验证**:改某设备 friendly_name 后重发,确认 HA 显示名跟着变、历史不丢(`unique_id` 稳定)。
- 不通则按需调整配置/勾选再重发即可。
## 12. 里程碑完成定义(DoD
- 后端能按 per-device 周期静默轮询 Modbus-TCP 设备、按 YAML profile 解码落 `modbus_reading.payload`,支持多设备 CRUD。
- MQTT 启用时,勾选的实体以 HA Discovery 注册成 device/entities(含非 sensor),state 周期发布;配置变更可重连重发;改 friendly_name 重发后 HA 显示名跟着变、`unique_id` 不变。
- 前端侧边栏可切换功能;Energy 视图能管理设备、看最新读数与走势图;设置页可勾选 expose。
- 后端 `pytest`/`ruff`/`export_openapi` + 前端 `lint/typecheck/test/build` 全绿且 `openapi/` 已入库。
- README / architecture / roadmap / design 索引反映 M5 现实。
+483
View File
@@ -0,0 +1,483 @@
# M6 — 通用电价层 + DSMR 实时数据 → 实时买卖电费计算(含 Tibber 动态 / 固定合同两种 profile)
> 阅读前提:先读 [`README.md`](./README.md)(协作模型、任务卡格式、校验闸门、数据安全红线)。本里程碑建立在 **M5MQTT + HA Discovery + expose 框架 + Energy 视图 + Recharts** 之上,复用其大部分基建,并把 M5 的"两层模型"YAML profile 定结构 + DB 行存部署值)从设备采集复制到**电价合同**上。
> **状态**:架构与原子任务已拆到可开工。少数价格常数(卖价残差、能源税年值、双费率寄存器映射)等真实 Tibber token / 账单确认,已落成 §13 的可调值 + TODO,不阻塞实现。
## 1. 目标
把"电价合同"与"DSMR 实时电表数据"接起来,按 **15 分钟**周期算出**实际买电支出 / 卖电收入**,自己落库留底(不可变、可审计),并通过 MQTT + HA Discovery 反哺 Home Assistant 的 Energy 仪表盘:
1. **通用电价层(两层模型)**:仓库内**只读 YAML profile 定义合同结构**`manual` 固定/可变费率、`tibber` 动态电价各一个结构);DB 的 **`EnergyContract`(+ 版本)存可改的数值**UI 填);**price strategy** 按 `kind` 出价。一次激活一个合同。
2. **DSMR 实时数据接入**:订阅 DSMR Reader 的 `dsmr/json`(每秒一条完整 telegram),**整帧存为 JSON blob、按 10 秒降采样**落库(电、气、各相全存,未来加供暖不改表)。
3. **实时买卖电费计算(两层)**:每 15 分钟用**电表累计寄存器差值** × 当时合同的买/卖价算计量电费(不可变、快照价);日/月/年**汇总**再加固定费、减能源税抵扣。**不做净计量**(saldering 2027 取消)。
4. **反哺 Home Assistant**:复用 M5 expose/Discovery,把当前买/卖价、累计买电支出、累计卖电收入发成 HA 实体(累计用 `total_increasing`)。
5. **前端**:Energy 视图下新增合同管理(按 profile 结构生成表单)+ 价格曲线 + 费用走势/明细 + Tibber 测试。
> **命名分层(延续 M5**:存储/采集/计算层中性命名(`dsmr_*` / `tibber_price` / `energy_cost_period` / `energy_contract`);面向用户并入既有 **Energy** 视图。
## 2. 现状(实现者可据此工作,不必通读全仓库)
**单库 + AlembicM5 完成态)**
- `app/db.py``class Base(DeclarativeBase)`cached engine(已开 `journal_mode=WAL` + `foreign_keys=ON`)。
- 单 Alembic 链 `alembic_app/`**当前 head = `20260622_10_exposed_entities`**`scripts/app_db_adopt.py``APP_BASELINE_REVISION` 同。迁移命名 `YYYYMMDD_NN_<desc>.py`,串 `down_revision``alembic_app/env.py` 逐个 import 模型。
- `modbus_reading` 已确立**通用 JSON payload 遥测**范式(`recorded_at` 真实索引列 + `payload` JSON)——`dsmr_reading` 直接沿用。
- M5 两层模型(`app/integrations/modbus/profiles/*.yaml` 定结构 + `modbus_device` 存部署值 + pydantic 启动期校验 + `decode`)是本里程碑电价层的范本。
**配置系统(扁平 KV,自动渲染)**
- `app/config.py` `Settings(BaseSettings)``app/services/config_page.py``CONFIG_FIELDS` 注册表 + `build_runtime_settings(session, bootstrap)`DB override 合并,**消费方都用它**+ `_settings_payload`(新字段补一行)。前端 `ConfigPage.tsx` 按 section/`input_type`/`secret` 通用渲染,bool 渲染成开关。
- ⚠️ 扁平 KV **装不下"合同清单 + 逐字段数值 + 版本"**——这些走专用表 + 专用 API + 自定义 UI(仿 M5 `modbus_device`)。
**MQTT(M5,关键:当前只发不收)**
- `app/integrations/mqtt.py` `MqttManager``is_configured/is_connected/connect/disconnect/reconnect/publish`paho `loop_start()` 后台线程,VERSION2 回调。**无 `subscribe`/`on_message`**——M6-T06 在此扩订阅端。
- `app/integrations/expose.py``ExposableEntity`dataclass`value_getter: Callable[[Session],Any]`)、`EntityProvider = Protocol(session)->list[ExposableEntity]``register_provider` + `build_catalog`。已有 `_modbus_provider`。**M6 加 `_energy_cost_provider` 复用整条 Discovery 链**。
- `app/services/ha_discovery.py``build_discovery_payload`/`publish_discovery`/`publish_states`/`publish_device_state``app/models/expose.py` `ExposedEntityToggle`(默认未勾不暴露)。
**后台调度(APSchedulerUTC**
- `app/main.py` lifespan`BackgroundScheduler(timezone="UTC")``add_job(..., max_instances=1, coalesce=True)`;同步 wrapper 自管 session、吞异常不崩 job。已有 public-ip / modbus-poll / discovery-state 等 job——**新增 job/订阅不得破坏它们**。
**依赖**`httpx`Tibber GraphQL)、`paho-mqtt``pyyaml``apscheduler` 均已在 `requirements.in/.txt`。**M6 不新增 Python 依赖**。前端已有 Recharts / TanStack Query / `openapi-fetch` 类型化 client / Mantine。
## 3. 目标架构
### 3.0 数据流总览
```
EnergyContract(active, 版本带生效日期) DSMR Reader
│ kind=manual → 常数双费率 │ MQTT dsmr/json (1s)
│ kind=tibber → API 15min ▼
│ 订阅 + 10s 降采样 → dsmr_reading(整帧 JSON blob)
│ (kind=tibber 时) │
▼ │ 累计寄存器=真值
Tibber GraphQL → tibber_price(15min, 不可变) │
│ ▼
└──────────────► price strategy ◄────── 计费引擎(每 15min 边界)
(manual/tibber 出价) │ 寄存器差 × 价,快照
energy_cost_period(每15min 计量电费, 不可变)
┌────────────────────────┼───────────────────────────┐
▼ ▼ ▼
日/月/年汇总(+固定费 −抵扣) expose provider + HA Discovery /api/energy/* + 前端
```
三条核心原则:
1. **算钱用寄存器差值,不用功率积分**:每 15 分钟能量 = 累计寄存器@末 @初,电表真值、与采样频率无关。瞬时功率只用于显示(且 HA 里已有,后端不专门用它算钱)。
2. **结构固定(YAML)、数值可改(UI/DB)、历史不可变(快照 + 版本)**profile YAML 定合同长什么样;UI 填数值、改价存新版本;算过的周期快照当时的价,永不被回改。
3. **两层费用**:每 15 分钟只算**计量电费**;按天/年的**固定费、能源税抵扣**在**汇总层**加,不污染每周期引擎。三相 / 双费率 / 燃气都靠"通用 blob + profile 结构"容纳,不改表。
### 3.1 两层电价模型(本里程碑地基,仿 M5)
```
┌──────────────────────────────────────────────────────────────┐
│ 知识层(固定,随代码走) 部署层(可改,落 DB,UI 填) │
│ ── pricing profile YAML ── ── EnergyContract 行 ── │
│ • kind=manual / tibber • nameUI 自由填) │
│ • 有哪些字段、单位、计费语义 • kind(UI 选) │
│ • 哪些分档 / 是否动态源 • active(一次一个) │
│ • UI 不可编辑 • currency │
│ ── EnergyContractVersion ── │
│ • effective_from / to │
│ • values(JSON,符合 profile) │
│ • 改价=加新版本,旧版本保留 │
└──────────────────────────────────────────────────────────────┘
│ kind 决定 strategy
price strategy(代码): manual=常数双费率出价; tibber=读 tibber_price 出价
```
- **profile YAML(仓库内只读,定结构)**`app/integrations/pricing/profiles/manual.yaml``tibber.yaml`,pydantic 启动期校验。声明该 kind **有哪些字段、单位、计费语义、哪些分档**——**不放具体数值**。
- **`EnergyContract` 行(部署,UI 填值)**`name`(自由填,不 hardcode "fixed")、`kind`UI 选)、`active``currency`
- **`EnergyContractVersion`(版本/时段,可审计)**`effective_from`/`effective_to`(null=开口) + `values`(JSON,符合 profile 结构)。**改价 = 加一个新版本行,旧版本保留**;"2026 上半年一个价、下半年另一个价" = 同合同两个版本。引擎算某周期时挑**覆盖该时刻的版本**。
- **price strategy(代码,按 kind**`manual` 用版本里的双费率常数出价;`tibber``tibber_price` + 版本里的 energy_tax/sell_adjust 出价。计费引擎对两者一视同仁。
**`manual.yaml` 结构(块状、可读;值在 UI 填)**
```yaml
kind: manual
label: 固定 / 可变费率(NL,双费率)
energy:
dual_tariff: true # 用 delivered_1/2、returned_1/2 分高低谷
buy:
normal: { unit: EUR/kWh } # 高价档(_2
dal: { unit: EUR/kWh } # 低价档(_1
sell:
normal: { unit: EUR/kWh } # 回送高价
dal: { unit: EUR/kWh } # 回送低价(现两档同价,仍分开填)
energy_tax: { unit: EUR/kWh } # 能源税,加到买价上(含 VAT)
ode: { unit: EUR/kWh, default: 0 } # 现并入能源税
standing: # 固定费:UI 按月填,内部摊到每天
network_fee: { unit: EUR/month }
management_fee: { unit: EUR/month }
credits:
heffingskorting: { unit: EUR/year } # 能源税抵扣,年度,汇总时扣
```
**`tibber.yaml` 结构**
```yaml
kind: tibber
label: Tibber 动态电价(15 分钟)
energy:
source: tibber_api # buy = total; sell = total energy_tax sell_adjust
energy_tax: { unit: EUR/kWh }
sell_adjust: { unit: EUR/kWh, default: 0 }
standing:
management_fee: { unit: EUR/month, default: 5.99 }
network_fee: { unit: EUR/month }
credits:
heffingskorting: { unit: EUR/year }
```
> 所有金额含 VAT(用户口径)。**Tibber token** 不在合同里、走 secret config(凭据归 config)。
### 3.2 DSMR 接入
- **数据源**DSMR Reader 的 **Telegram JSON**(单 topic `dsmr/json`,一条 = 一帧完整 telegram,已是解析后的表寄存器值)。**每秒一条**。
- **整帧存 JSON blob**(沿用 `modbus_reading` payload 哲学):**不做字段 allowlist**,整帧存下来(电、气 `extra_device_*`、各相全要)。理由:现有燃气、未来供暖、三相——blob 全装,**不改表**;读取时按 key 取。
- **实测 JSON 的坑(已确认)**:数值是 **JSON 字符串**`"20915.154"`)→ 计费读取时转 **Decimal**;缺测相位是 **`null`** → 视为缺测;`id`telegram 自增)做幂等去重。
- **降采样**:仅当 `timestamp` 秒数落在 `dsmr_sample_interval_s`(默认 10)整数倍时落盘,其余丢弃(量从 60 行/分降到 ~6 行/分;秒=00 在网格上,每个 15 分钟边界都抓得到)。
- **订阅**:扩 `MqttManager` 订阅端(`on_connect` 里 subscribe、`topic→handler``on_message` 分发,handler 在网络线程内只做"解析+降采样+一次短事务"、吞异常不崩连接)。`dsmr_ingest_enabled`(默认 falseopt-in)。
- **进口/出口总量** = `delivered_1+delivered_2` / `returned_1+returned_2`**双费率分档** = `_1`(dal/低) vs `_2`(normal/高)(NL 惯例,§13 确认别接反)。
- **三相 / 燃气 / 供暖**:blob 天然容纳,单相→三相只是多几个 key,不改表(详 §10)。
### 3.3 Tibber 价格接入(仅 `kind=tibber` 时)
- **客户端**`app/integrations/tibber/client.py`httpx POST `https://api.tibber.com/v1-beta/gql``Bearer <token>`
- **15 分钟查询**introspection + demo 实证):
```graphql
{ viewer { homes { id currentSubscription {
priceInfoRange(resolution: QUARTER_HOURLY, first: 96) {
nodes { startsAt total energy tax currency level }
} } } } }
```
- `Price` 字段:`total`=energy+tax**全包价**demo 已证 `total==energy+tax`)、`energy`(纯 spot ex-VAT)、`tax``startsAt`(含时区偏移)、`currency``level`
- 枚举 **`PriceInfoRangeResolution`** 值 `DAILY/HOURLY/QUARTER_HOURLY`(不是 `PriceResolution`);老的 `priceInfo.range` 已废弃。
- **已验证(demo key**:返回 96 节点、间隔 15 分钟、`total==energy+tax`;老数据无真 15min 价时 API 把小时价**重复成 4 个刻钟**,当前 NL 数据则是真 15 分钟。**解析不假设固定间隔/节点数**,按 `startsAt` 逐点存。
- **抓取调度**:启动 + 每日 job 拉「今天+明天」,按 `starts_at` **upsert** `tibber_price`(幂等)。次日价约 13:00 CET 发布 → 每日 14:00 CET12:00 UTC)抓明天,缺失每小时重试。**仅当 active 合同 kind=tibber 且 token 存在时跑**。
- **连接测试** `POST /api/energy/tibber/test`:拉一次 current price,三态(success 带当前价 / config-error / failed),仿 M5 `mqtt/test`
### 3.4 计费引擎(两层)
**第一层 — 每 15 分钟计量电费(`energy_cost_period`,不可变、快照价)**
- 触发:APScheduler 1 分钟 tick,找"已闭合未算"的 15 分钟周期算(也支持按区间手动重算)。
- 单周期 `[t0,t1)``t1` 已过):
1. 取各寄存器在 `t0`/`t1` 的值(`recorded_at ≤ 边界` 的最后一行,Decimal),算 **per-register 差**`Δd1,Δd2,Δr1,Δr2`
2. 取 active 合同**在 t0 生效的版本** + 其 strategy
- `manual``import_cost = Δd1×(buy_dal) + Δd2×(buy_normal)``buy_x = energy_buy_x + energy_tax + ode`);`export_revenue = Δr1×sell_dal + Δr2×sell_normal`
- `tibber`:取覆盖 t0 的 `tibber_price``starts_at ≤ t0` 最近一条);`buy = total``sell = total energy_tax sell_adjust``import_cost = (Δd1+Δd2)×buy``export_revenue = (Δr1+Δr2)×sell`
3. `net_cost = import_cost export_revenue`**upsert** `energy_cost_period`**快照**当时用的价 + `contract_version_id`
- 缺价/缺数据:跳过或标 `degraded`,留待重算。**不做净计量**(进出口分开累加)。
**第二层 — 汇总(日/月/年,读时计算,不存表)**
- `Σ 周期 net_cost`(区间内)** 固定费**`network_fee + management_fee` 按月→按天 × 天数)**− 能源税抵扣**(`heffingskorting` 按年→按天 × 天数)= **实际应付**
- 固定费/抵扣取 active 合同版本的值;UI 按月/按年填,引擎内部摊到每天。
### 3.5 数据模型(新增 5 张表,单库 app 链)
> 单一 P1 智能电表,`dsmr_reading` 不设 device 表;将来第二块表加 `source` 列即可。
| 表 | 关键列 | 说明 |
| --- | --- | --- |
| **`dsmr_reading`** | `recorded_at`(idx)、`source_id`(unique=telegram id)、`payload`(JSON) | 整帧 telegram blob10s 降采样 |
| **`energy_contract`** | `name``kind`(manual/tibber)、`active``currency`、时间戳 | 合同头;一次一个 active |
| **`energy_contract_version`** | `contract_id`(FK)、`effective_from``effective_to`(null)、`values`(JSON)、`created_at` | 版本/时段;改价加新版本、旧版本保留(审计)|
| **`tibber_price`** | `starts_at`(unique)、`resolution``energy``tax``total``level``currency``fetched_at` | 15min 价缓存,**不可变**;仅 tibber kind |
| **`energy_cost_period`** | `period_start`(unique)、`d1/d2/r1/r2_kwh`(周期差)、`import_cost``export_revenue``net_cost``currency``pricing`(JSON 快照)、`contract_version_id`(FK)、`degraded``computed_at` | 每15min 计量电费,**不可变**(快照价+来源版本)|
- `energy_cost_period` 存 per-register 差 + 快照价 → 完全可审计、source-agnostic、可重算覆盖。
- FK`energy_contract_version.contract_id → energy_contract.id`ON DELETE RESTRICT);`energy_cost_period.contract_version_id → energy_contract_version.id`RESTRICT,保审计链)。
### 3.6 不可变 / 审计(贯穿)
1. **算账快照**`energy_cost_period` 存死当时实际用的价 + `contract_version_id`;改合同 / 改价**不回改**已算周期——"以前的价就是以前的价"。重算是**显式 opt-in**`POST /recompute`),不自动覆盖历史。
2. **合同版本只增不改**:改价 = 新 `energy_contract_version`(带生效日期),旧版本保留,全程审计链。
3. **Tibber 价不可变**:过去某 15 分钟的价不会变。
4. 三者叠加 → manual / tibber 都满足审计与不可变。
### 3.7 MQTT / HA expose(复用 M5
`app/integrations/expose.py::_energy_cost_provider``register_provider`),归一个 HA device`identifiers="energy-cost"`):
| key | component | 取值 | 用途 |
| --- | --- | --- | --- |
| `buy_price_now` | sensor | 当前周期/当前档买价(€/kWhmonetary | HA 当前单价 |
| `sell_price_now` | sensor | 当前卖价(€/kWh | |
| `import_cost_total` | sensor | 累计 import_cost`total_increasing`monetarycurrency | HA Energy "总成本实体" |
| `export_revenue_total` | sensor | 累计 export_revenue`total_increasing` | HA Energy 回送补偿 |
- `value_getter(session)``energy_cost_period`(累计/当前)取值。沿用 M5 勾选(默认未勾)、`publish_discovery``publish_states`(计费 job 后顺带推 + 周期兜底)。
### 3.8 前端(并入 Energy 视图)
- **合同管理**:列表 + 新建/编辑(**表单按选中 profile 结构渲染字段**:manual 双费率/能源税/固定费/抵扣;tibber token 提示走 Config+ 激活(一次一个 active)+ 改价=新版本(带生效日期)+ 版本历史只读展示。
- **价格曲线**:今天+明天 15min 买/卖价(tibber)或固定档位(manual)折线(复用 `EnergyCharts`),取 `/api/energy/prices`
- **费用走势 + 明细 + 汇总**:当日/月 import_cost/export_revenue/net 走势 + 每 15min 明细 + 汇总卡片(含固定费/抵扣),取 `/api/energy/costs` + `/costs/summary`
- **Tibber 测试**Config 页一块「Tibber」,三态(仿 MQTT 测试)。
- 全走类型化 client;空/加载/错误态齐全;缺 key 容错。
## 4. API 契约(M6 要落地的端点)
> 全部 `/api` 前缀、session + CSRF(写)保护、JSON 进出;schema 经 `scripts/export_openapi.py` 固化入库。
| 分组 | 端点 | 用途 |
| --- | --- | --- |
| 合同 | `GET /api/energy/contracts` | 列出合同 + active 标记 |
| 合同 | `POST /api/energy/contracts` | 新建合同(kind + 首版本值,按 profile 校验)|
| 合同 | `GET /api/energy/contracts/{id}` | 单个合同 + 版本历史 |
| 合同 | `PATCH /api/energy/contracts/{id}` | 改名 / 激活(一次一个 active)|
| 合同 | `POST /api/energy/contracts/{id}/versions` | 加新版本(改价,带生效日期)|
| 合同 | `GET /api/energy/profiles` | 列出 pricing profile 结构(前端按它渲染表单)|
| 价格 | `GET /api/energy/prices?start&end` | 区间价格点(曲线)|
| 费用 | `GET /api/energy/costs?start&end` | 区间 `energy_cost_period`(走势/明细)|
| 费用 | `GET /api/energy/costs/summary?start&end` | 区间汇总(计量电费 + 固定费 − 抵扣)|
| 费用 | `POST /api/energy/costs/recompute?start&end` | 幂等重算(补数据/改价后,显式)|
| DSMR | `GET /api/energy/dsmr/latest` | 最新 `dsmr_reading` |
| 测试 | `POST /api/energy/tibber/test` | 试连 Tibber + 拉当前价,三态 |
> DSMR/Tibber 的**标量配置 + token** 复用现有 `GET/PUT /api/config`(只加 CONFIG_FIELDS)。**价格数值全在合同里**,不进 flat config。
## 5. 已锁定决策(讨论后拍板)
1. **两层电价模型**profile YAML 定结构(仓库、固定、UI 不可编辑)+ `EnergyContract`(+版本) 存数值(UI 填、版本化)+ strategy 按 kind 出价。仿 M5。
2. **kind 不叫 "fixed"**`manual`(人工填、可双费率、可带时段)/ `tibber`API 动态);合同 `name` UI 自由填。
3. **买价**tibber = API `total`(全包,已证 total=energy+tax);manual = `energy_buy_档 + energy_tax`。**卖价**tibber = `total energy_tax sell_adjust`manual = `sell_档`(回送价,无能源税)。均含 VAT。
4. **双费率**manual 用 `delivered_1/2``returned_1/2` 分 dal/normal 计价(`_1`=dal/低、`_2`=normal/高);tibber 求和、15min 价不分档。
5. **两层费用**:每 15min `energy_cost_period` 只算计量电费(不可变、快照价);日/月/年汇总再加固定费(按月→天)− heffingskorting(按年→天)。
6. **回送阶梯罚金(terugleverkosten)不做**:按自然年累计、用户住不到年底算不准——不算、不记、不加功能(留痕见 §10)。
7. **DSMR 整帧存 JSON blob**(不做字段 allowlist),10s 降采样;数值转 Decimal、null 容错、`id` 幂等。扩 `MqttManager` 订阅端。
8. **5 张表**`dsmr_reading``energy_contract``energy_contract_version``tibber_price``energy_cost_period`。单电表不设 device 表。
9. **不可变 / 审计**:算账快照价 + 合同版本只增不改 + tibber 价天然不可变;重算显式 opt-in。
10. **Tibber 用 httpx + `priceInfoRange(QUARTER_HOURLY)`**,仅 active=tibber 时抓;token 走 secret config。
11. **复用 M5 expose/Discovery**:加 `_energy_cost_provider`;累计成本/收入用 `total_increasing` 喂 HA Energy。
12. **前端并入 Energy 视图**:合同管理表单按 profile 结构渲染;Recharts 画价/费。
13. **opt-in 开关**`dsmr_ingest_enabled` 默认 falseTibber 抓取/计费由 active 合同存在性驱动。
14. **周期边界用 UTC 刻钟**(NL 偏移整小时,本地/UTC 重合)。**不新增依赖**。
> 项目定位:个人自用、家庭特化、不开源——按单用户场景简化,不过度抽象。
## 6. 任务依赖图
```
Phase A(地基)
M6-T01 [schema] 5 张表 + 模型 + 迁移
M6-T02 flat 配置(dsmr_* + tibber token/home_id ← 与 T01 并行
M6-T03 pricing profile 框架(manual.yaml/tibber.yaml + pydantic + strategy 注册表) ← 纯模块,可早做
Phase B(合同 + 数据接入)
M6-T04 EnergyContract CRUD + 版本 + 按 profile 校验(API (Depends T01,T03)
M6-T05 Tibber 客户端 + 抓价 service + 调度 + tibber/test (Depends T01,T02,T03)
M6-T06 MqttManager 扩订阅 + DSMR ingestblob/10s (Depends T01,T02)
Phase C(计算 + 反哺)
M6-T07 计费引擎(每15min 计量 + 汇总)+ 周期 job + 重算 (Depends T01,T03,T04,T05,T06)
└─► M6-T08 energy_cost expose provider + state 发布 (Depends T07)
Phase DAPI + 前端)
M6-T09 Energy 数据 APIprices/costs/summary/dsmr-latest/recompute (Depends T05,T07)
└─► M6-T10 前端:合同管理 + 价格/费用视图 + Tibber 测试 (Depends T04,T09)
收尾
M6-T11 文档 + OpenAPI + roadmapDepends 全部)
```
`T01``T02``T03` 无前置可先开。
---
## 7. 原子任务(任务卡)
> 后端任务沿用校验闸门(`pytest` / `ruff` / 改路由或 schema 则 `export_openapi` 重导出入库);前端闸门见 §8。**不新增依赖**。纯新增、不删/移文件;改 `app/main.py` lifespan 时不得破坏既有 job/连接。
### M6-T01 — 5 张表与模型 `[schema]`
- **Status**: `done` · **Depends**: none
- **Context**: 单库 app 链新增 5 张表(§3.5)。只建 schema + 模型,不写采集/计算/接口。
- **Files**: `create app/models/energy.py``DsmrReading`/`EnergyContract`/`EnergyContractVersion`/`TibberPrice`/`EnergyCostPeriod`);`create alembic_app/versions/20260623_11_energy_tables.py``modify alembic_app/env.py``scripts/app_db_adopt.py``APP_BASELINE_REVISION`→新 head);`create tests/test_energy_models.py`
- **Steps**:
1. 按 §3.5 定义模型(`Mapped[...]`):`DsmrReading``recorded_at` idx、`source_id` unique nullable、`payload` JSON);`EnergyContract``name`/`kind`/`active`/`currency`/时间戳);`EnergyContractVersion``contract_id` FK RESTRICT、`effective_from`/`effective_to` nullable、`values` JSON、`created_at`);`TibberPrice``starts_at` unique、`resolution``energy/tax/total` float、`level` nullable、`currency``fetched_at`);`EnergyCostPeriod``period_start` unique、`d1/d2/r1/r2_kwh` float、`import_cost/export_revenue/net_cost` float、`currency``pricing` JSON、`contract_version_id` FK RESTRICT nullable、`degraded` bool、`computed_at`)。
2. 新 revision`down_revision="20260622_10_exposed_entities"``upgrade()` 建 5 表 + 索引/唯一/FK`downgrade()` 反向 drop。更新 `APP_BASELINE_REVISION`
- **Out of scope**: 不写客户端/采集/计费/接口;不动其它模型。
- **Acceptance criteria**:
- [ ] 全新临时库 upgrade 到 head 含 5 表及约束;`downgrade -1` 干净回滚;链上单 head。
- [ ] FK 为 RESTRICT;唯一/索引正确;`payload`/`values`/`pricing` 为 JSON 列。
- [ ] `APP_BASELINE_REVISION` == 新 head。
- [ ] 校验闸门全绿。
- **Reviewer checklist**: 列/约束/FK 与 §3.5 一致;`recorded_at`/`period_start`/`starts_at` 真实索引或唯一列;`env.py` 已 import 新模型;迁移 `down_revision` 指当前真实 head;无破坏性操作。
### M6-T02 — flat 配置(DSMR + Tibber 凭据)
- **Status**: `done` · **Depends**: none
- **Context**: DSMR 订阅设置 + Tibber 凭据接入扁平配置(前端自动渲染)。**价格数值不在这里**(在合同里)。
- **Files**: `modify app/config.py``app/services/config_page.py`CONFIG_FIELDS + `_settings_payload`);`modify .env.example``tests/test_api_config.py`
- **Steps**: `Settings``dsmr_ingest_enabled: bool=False``dsmr_mqtt_topic: str="dsmr/json"``dsmr_sample_interval_s: int=10``tibber_api_token: str=""``tibber_home_id: str=""`CONFIG_FIELDS 归「DSMR」「Tibber」section`tibber_api_token` 为 secretbool=checkbox、数值=number`_settings_payload` 补齐;`.env.example` 加注释样例(默认 off)。
- **Out of scope**: 不建客户端/采集;不动既有字段;不放任何价格常数。
- **Acceptance criteria**:
- [ ] 新项在 `GET /api/config` 分 sectiontoken 为 secret(回空/留空保留);非法值 422 不写库。
- [ ] 校验闸门全绿(OpenAPI 变化则重导出)。
- **Reviewer checklist**: secret 不回显/不进 OpenAPI 示例;`_settings_payload` 无遗漏;默认 off。
### M6-T03 — pricing profile 框架 + strategy 注册表
- **Status**: `done` · **Depends**: none
- **Context**: 仓库内 `manual.yaml`/`tibber.yaml` 定结构(pydantic 校验),加 price strategy 注册表(manual/tibber 出价)。纯模块,单测。
- **Files**: `create app/integrations/pricing/__init__.py``pricing/profiles.py`pydantic 模型 + `load_profile`/`list_profiles`/`validate_values`)、`pricing/profiles/manual.yaml``pricing/profiles/tibber.yaml``pricing/strategies.py``register_strategy`/`get_strategy``manual`/`tibber` 实现);`create tests/test_pricing_profiles.py``tests/test_pricing_strategies.py`
- **Steps**:
1. `profiles.py`pydantic 模型描述 §3.1 结构;`load_profile(kind)`(读 YAML,校验失败抛错);`validate_values(kind, values)`(合同数值是否符合结构)。
2. `profiles/*.yaml`:按 §3.1 写 manual / tibber 结构。
3. `strategies.py`:策略接口 `price_period(deltas, t0, values, session) -> {import_cost, export_revenue, net, pricing_snapshot}`Decimal):
- `manual`:双费率公式(§3.4),`buy_x = energy_buy_x + energy_tax + ode``sell_x` 直接用。
- `tibber`:取覆盖 t0 的 `tibber_price``buy=total``sell=totalenergy_taxsell_adjust`,进出口求和。
- **Out of scope**: 不连真实 APItibber 策略只读 `tibber_price` 表,由 T05 填);不写采集/调度/接口。
- **Acceptance criteria**:
- [ ] 单测:`load_profile("manual"/"tibber")` 校验通过;缺字段/类型错抛可识别错;`validate_values` 正确判合规。
- [ ] 单测:manual 策略双费率出价正确(`import = Δd1×buy_dal + Δd2×buy_normal` 等);tibber 策略 `buy=total``sell=totalenergy_taxsell_adjust`,负 total 出口得负值。
- [ ] Decimal 计算无 float 误差。
- [ ] 校验闸门全绿。
- **Reviewer checklist**: profile 纯结构、无具体数值;策略用 Decimal;进出口分开(不相抵);tibber 价匹配用 `starts_at ≤ t0` 最近一条;无 OOP 过度抽象(数据+函数,仿 M5 profiles.py)。
### M6-T04 — EnergyContract CRUD + 版本 + 校验(API
- **Status**: `done` · **Depends**: M6-T01, M6-T03
- **Context**: 合同增删改、版本(改价加新版本、生效日期)、激活(一次一个)、按 profile 校验数值。
- **Files**: `create app/api/routes/api/energy_contracts.py``app/schemas/energy_contract.py``app/services/contracts.py``modify app/main.py`(注册路由);`create tests/test_api_energy_contracts.py`
- **Steps**:
1. `contracts.py` service:建合同(含首版本,`validate_values` 校验)、加版本(带 `effective_from`,自动闭合上一版本 `effective_to`)、激活(把其它 active 置 false)、`active_contract_version_at(session, ts)`(取 active 合同在 ts 生效的版本)。
2. 路由:`GET/POST/GET{id}/PATCH{id}``POST {id}/versions``GET /profiles`(返回结构供前端渲染表单);session+CSRF;数值不合规 422。
- **Out of scope**: 不算费用(T07);不发 MQTT;不写采集。
- **Acceptance criteria**:
- [ ] CRUD + 版本 + 激活行为正确:新建按 profile 校验(不合规 422);加版本自动闭合上一段;激活互斥(同时只一个 active);`GET /profiles` 返回结构。
- [ ] 未登录 401、缺 CSRF 403schema 经 OpenAPI 固化入库。
- [ ] 校验闸门全绿。
- **Reviewer checklist**: 版本只增不改(不覆盖旧版本);激活互斥;删除受 FK RESTRICT(有版本/有费用记录不可裸删);数值校验走 T03 `validate_values`;路径键用合同 id。
### M6-T05 — Tibber 客户端 + 抓价 service + 调度 + tibber/test
- **Status**: `done` · **Depends**: M6-T01, M6-T02, M6-T03
- **Context**: httpx 拉 15min 价 upsert `tibber_price`;启动+每日抓(仅 active=tibber);连接测试。
- **Files**: `create app/integrations/tibber/__init__.py``tibber/client.py``app/services/tibber_prices.py``modify app/main.py`(抓价 job);`create tests/test_tibber_client.py``tests/test_tibber_prices.py`
- **Steps**:
1. `client.py``fetch_price_range(token, home_id|None) -> list[PricePoint]`(§3.3 query,httpx 超时/认证失败抛错,不假设固定节点数);`fetch_current_price(...)`(供 test)。
2. `tibber_prices.py``refresh_prices(session, settings)`(拉今天+明天 upsert,幂等);仅当 active 合同 kind=tibber 且 token 存在时执行。
3. `main.py`:同步 wrapper + `IntervalTrigger``max_instances=1, coalesce=True`,吞异常不崩。
- **Out of scope**: 不算费用(T07);不发 MQTT;test 路由在 T09(本任务备 service)。
- **Acceptance criteria**:
- [ ] 单测(httpx mock):解析返回为 PricePoint;认证失败/超时抛可识别异常。
- [ ] 单测:`refresh_prices` 幂等(按 `starts_at` upsert,不重复插);非 tibber active 时 no-op。
- [ ] 校验闸门全绿。
- **Reviewer checklist**: token 不进日志;httpx 超时;upsert 幂等;时间存 UTC;无对 `tibber_price` 的破坏性批删。
### M6-T06 — MqttManager 扩订阅 + DSMR ingest
- **Status**: `done` · **Depends**: M6-T01, M6-T02
- **Context**: 给只发不收的 `MqttManager` 扩订阅;DSMR ingest 整帧 blob + 10s 降采样落库。
- **Files**: `modify app/integrations/mqtt.py`(订阅注册 + on_message 分发);`create app/services/dsmr_ingest.py``modify app/main.py`(启用时注册订阅);`create tests/test_mqtt_subscribe.py``tests/test_dsmr_ingest.py`
- **Steps**:
1. `mqtt.py``subscribe(topic, handler)``on_connect` 里对已注册 topic subscribe`on_message` 按 topic 分发,handler 异常吞掉不崩连接);保持既有 publish 不回归。
2. `dsmr_ingest.py``handle_message(payload_bytes, settings)`parse JSON → 取 `timestamp`(UTC) → 降采样(`second % dsmr_sample_interval_s != 0` 丢弃)→ **整帧存 payload**(值原样/Decimal 友好)→ 短 session 写 `dsmr_reading``source_id`=`id`,存在则跳过)。
3. `main.py``dsmr_ingest_enabled` 时把 handler 注册到 `dsmr_mqtt_topic`
- **Out of scope**: 不算费用(T07);不改既有 publish/discovery 语义;不做字段 allowlist(整帧存)。
- **Acceptance criteria**:
- [ ] 单测:`subscribe` 注册后 `on_message` 分发到 handler;既有 publish 无回归。
- [ ] 单测:喂样本 JSON(字符串数值、null 相位),秒=00 整帧落盘、秒≠10 整数倍丢弃;同 `id` 不重插。
- [ ] `dsmr_ingest_enabled=false` 不订阅/不落盘。
- [ ] 校验闸门全绿。
- **Reviewer checklist**: on_message 网络线程内只做短事务、吞异常不崩连接;整帧存(含 gas/各相);null 容错;`source_id` 幂等;10s 降采样正确。
### M6-T07 — 计费引擎 + 周期 job + 汇总 + 重算
- **Status**: `done` · **Depends**: M6-T01, M6-T03, M6-T04, M6-T05, M6-T06
- **Context**: 每 15min 用寄存器差 × active 合同 strategy 算计量电费(快照、不可变);汇总加固定费 − 抵扣;周期 job + 重算。
- **Files**: `create app/services/energy_cost.py``modify app/main.py`(周期 job);`create tests/test_energy_cost.py`
- **Steps**:
1. `energy_cost.py``register_at(session, boundary)`(取 ≤boundary 最后一行的 d1/d2/r1/r2Decimal);`compute_period(session, t0)`(取 active 版本 + strategyT03/T04)→ 算 → upsert + 快照 `pricing` + `contract_version_id`,缺价/缺数据标 `degraded` 或跳过);`compute_closed_periods(session)``recompute_range(session, start, end)`(幂等覆盖);`summarize(session, start, end)`(Σnet + 固定费按天 heffingskorting 按天,取 active 版本值)。
2. `main.py`1 分钟 tick job`max_instances=1, coalesce=True`),有 active 合同 + DSMR 数据时算;吞异常不崩。
- **Out of scope**: 不发 MQTTT08);不加 HTTPT09,备 service);不改采集/价格抓取。
- **Acceptance criteria**:
- [ ] 单测:构造 `dsmr_reading`+合同版本(manual 双费率 / tibber + `tibber_price`),`compute_period` 算出正确 cost/revenue/net,快照价 + `contract_version_id` 落库;upsert 幂等。
- [ ] 单测:跨价格版本时按 t0 选对版本;缺价跳过、缺读数标 `degraded`
- [ ] 单测:`summarize` = Σnet + 固定费(月→天×天数)− heffingskorting(年→天×天数)。
- [ ] 校验闸门全绿。
- **Reviewer checklist**: Decimal 算钱;寄存器差为"末−初";进出口分开;周期边界 UTC 刻钟;快照不可变(重算才覆盖,且显式);版本选择按 `effective_from≤t0<effective_to`job 不崩。
### M6-T08 — energy_cost expose provider + state 发布
- **Status**: `done` · **Depends**: M6-T07
- **Context**: 复用 M5 expose/Discovery,发当前买/卖价 + 累计成本/收入。
- **Files**: `modify app/integrations/expose.py``_energy_cost_provider` + 注册);`modify app/services/energy_cost.py``app/main.py`(算完顺带 `publish_states`);`create tests/test_energy_expose.py`
- **Steps**: provider 产出 §3.7 实体(`buy_price_now`/`sell_price_now`/`import_cost_total`/`export_revenue_total`),累计项 `total_increasing`+`monetary`+currency`value_getter``energy_cost_period` 取;计费 job 后 `publish_states`
- **Out of scope**: 不改 Discovery 机制本身;不改采集/计费。
- **Acceptance criteria**:
- [ ] 单测:`build_catalog` 含 energy_cost 实体;累计项 `total_increasing`;勾选默认未勾。
- [ ] 单测:`value_getter` 取到正确当前价/累计值;MQTT 未启用整链 no-op。
- [ ] 校验闸门全绿。
- **Reviewer checklist**: `key` 稳定(固定字符串);复用既有 provider 协议、未改其它 provider;累计 `total_increasing` 语义正确;不阻塞计费 job。
### M6-T09 — Energy 数据 API
- **Status**: `done` · **Depends**: M6-T05, M6-T07
- **Context**: 价格/费用/汇总/最新 DSMR/重算/Tibber 测试端点(合同 CRUD 在 T04)。
- **Files**: `create app/api/routes/api/energy.py``app/schemas/energy.py``modify app/main.py`(注册);`create tests/test_api_energy.py`
- **Steps**: `GET /prices``GET /costs``GET /costs/summary``GET /dsmr/latest``POST /costs/recompute`(调 T07)、`POST /tibber/test`(调 T05 client,三态);session+CSRF;查询走索引 + 窗口/上限。
- **Out of scope**: 不写采集/计费/发布逻辑(复用 service)。
- **Acceptance criteria**:
- [ ] 各端点行为正确:窗口+limit 上限生效;recompute 幂等;tibber/test 三态;401/403 正确;schema 入库(`git diff --exit-code openapi/` 无差异)。
- [ ] 校验闸门全绿。
- **Reviewer checklist**: 查询有时间窗+上限;recompute 复用 T07、无破坏删;test 不泄 token。
### M6-T10 — 前端:合同管理 + 价格/费用视图 + Tibber 测试
- **Status**: `done` · **Depends**: M6-T04, M6-T09
- **Files**: `modify frontend/src/pages/EnergyPage.tsx``create frontend/src/energy/ContractManager.tsx``ContractForm.tsx``TibberPrices.tsx``CostView.tsx`、扩 `hooks.ts``modify` Config 页 Tibber 测试入口;`create` 对应 `*.test.tsx`
- **Steps**: 合同列表/新建/编辑/激活/加版本(**表单字段按 `/profiles` 结构渲染**,manual 双费率/税/固定费/抵扣)+ 版本历史只读;价格曲线 + 费用走势/明细 + 汇总卡片;Tibber 测试三态;全走类型化 client;空/加载/错误态 + 缺 key 容错。
- **Out of scope**: 不做服务端降采样;不改后端。
- **Acceptance criteria**:
- [ ] 合同 CRUD/激活/版本可用,表单按 profile 渲染;价格/费用/汇总渲染正确;区间只取窗口;币种/单位来自 API。
- [ ] Tibber 测试三态可用。
- [ ] 前端闸门全绿(`lint`/`typecheck`/`test`/`build`)。
- **Reviewer checklist**: 全走类型化 client;图表组件隔离复用;表单按结构渲染、不 hardcode 字段;窗口/上限;空/错/加载/缺 key 容错。
### M6-T11 — 文档 + OpenAPI + roadmap 收尾
- **Status**: `done` · **Depends**: 全部
- **Files**: `modify README.md``docs/roadmap.md`M6 行 + 毕业)、`docs/architecture-overview.md``docs/design/README.md`(列入 m6);`run python scripts/export_openapi.py` 并提交 `openapi/`
- **Acceptance criteria**:
- [ ] 文档反映新链路;`git diff --exit-code openapi/` 无未提交差异;校验闸门全绿。
- **Reviewer checklist**: 无残留过时描述;OpenAPI 入库;§13 已敲定项同步回文档。
---
## 8. 前端校验闸门
`frontend/` 下:`npm ci && npm run lint && npm run typecheck && npm run test && npm run build`(留意 chunk 体积告警)。后端同任务改路由/schema 仍需根目录 `export_openapi.py` 并提交 `openapi/`。**不新增前端依赖**(Recharts 已在)。
## 9. 构建上下文完整性(M1 教训)
- **纯新增、不删/移文件****不新增依赖**httpx/paho/pyyaml 已在)。新增源文件都在 `app/``scripts/``frontend/` 既有 `COPY` 范围内,无需改 `Dockerfile`**`app/integrations/pricing/profiles/*.yaml` 是非 .py 资源**——确认随 `COPY app` 进镜像、运行期按 `__file__`/`importlib.resources` 定位(别用 CWD 相对路径)。`tests/test_deployment.py::test_dockerfile_copy_sources_exist` 仍应通过。
- 改 `app/main.py` lifespan 注册新 job/订阅时**不得破坏**既有 public-ip / modbus-poll / discovery job 与 MQTT 连接。
- 发版前置走查(CLAUDE.md):真起 app → 订到 `dsmr/json`、(合同生效后)拉一次 Tibber 价、看费用落库、前端合同/价格/费用视图人工瞄一眼,再打 tag。
## 10. 后续杠杆(本里程碑不做,文档留痕)
- **回送阶梯罚金(terugleverkosten**:按自然年累计、阶梯式;本期不做。将来做就把阶梯表存进合同结构,在**年度汇总**层按 YTD 累计回送量套阶梯(不进每周期引擎)。
- **燃气 / 区域供暖计费**:数据已在 blob 里(gas `extra_device_*`);将来在合同结构加"商品价"+ `energy_cost_period``commodity` 维度即可,不改主链。
- **能源税分档**:年用电跨 10000 kWh 档时税率变;现按单档配置。
- **降采样 / 保留**`dsmr_reading` 10s 长期行数大,加定期降采样/保留窗口任务(SQL 端 `AVG(json_extract(...))`)。
- **价格 level 配色 / 便宜时段提示**:用 Tibber `level` 驱动前端配色与用电建议。
- **HA 让 HA 算 vs 后端算**:当前后端算累计;也可只发买/卖价 €/kWh 让 HA Energy 自乘(实体都已具备)。
## 11. 人工验收 walkthrough(实现完成后)
> 受控手工验证(依赖家里 DSMR Reader + brokerTibber 部分待合同生效),不进自动化。
1. **合同(先用 manual,立即可用)**Energy 页新建一个 `manual` 合同,按表单填双费率/能源税/固定费/抵扣,激活。
2. **DSMR 落库**:开 `dsmr_ingest_enabled`、确认 topicMQTT Explorer 看 `dsmr/json` 有消息;等若干 10 秒 → `GET /api/energy/dsmr/latest` 有最新整帧;`dsmr_reading` 约每 10 秒一行(整帧 blob)。
3. **费用计算**:跨过一个整刻钟边界 → `GET /api/energy/costs` 出现该周期 import/export/cost/revenue/netmanual 双费率分档对得上手算);`GET /costs/summary` 含固定费/抵扣;改价加新版本后旧周期不变、`POST /recompute` 才覆盖。
4. **Tibber(合同生效后)**:建 `tibber` 合同、Config 填 token、点「Tibber 测试」三态成功;激活后抓价 → `GET /prices` 有 15min 价、费用切到动态。
5. **反哺 HA**Expose 勾选 energy_cost 实体、开 Discovery、重发 → HA 出现当前买/卖价 + 累计成本/收入;HA Energy 挂累计实体。
## 12. 里程碑完成定义(DoD
- 后端能:订 `dsmr/json` 按 10s 整帧落库;按 active 合同(manual 双费率 / tibber 动态)每 15min 用寄存器差 × 价算计量电费(不可变、快照);日/月/年汇总加固定费 − 抵扣;不做净计量;改价走版本、历史不回改。
- 复用 M5 Discovery 把当前买/卖价 + 累计成本/收入(`total_increasing`)发成 HA 实体。
- 前端 Energy 视图能管理合同(表单按 profile 渲染、激活、版本)、看价格曲线与费用走势/明细/汇总、测 Tibber。
- 后端 `pytest`/`ruff`/`export_openapi` + 前端 `lint/typecheck/test/build` 全绿且 `openapi/` 入库。
- README / architecture / roadmap / design 索引反映 M6。
## 13. 待确认 / TODO(拿到真实 token + 账单后钉死,均已落成配置/默认值,不阻塞实现)
1. **买价**:✅ tibber = API `total`demo 已证 total=energy+tax);manual = energy_buy_档 + energy_tax。无待办。
2. **卖价残差(tibber**`sell = total energy_tax sell_adjust``sell_adjust` 默认 0(买卖费相等抵消)。真实账单确认后若有残差再调。
3. **双费率寄存器映射**`_1`=dal/低、`_2`=normal/高(NL 惯例)——接价前用真实数据确认别接反(差价小但要对)。
4. **能源税年值**manual/tibber 的 `energy_tax` 默认 ~0.11082026 第一档含 VAT),按当年实际值核。
5. **Tibber 15min + 币种**:✅ 查询/分辨率已 demo 证实;仍需合同生效后用**真实 token** 确认 NL 返回真 15 分钟价 + 币种 EUR。
6. **manual 现行数值**(你的固定合同):normal 0.133 / dal 0.127 / 回送两档(现同价)/ 管理费 ≈€0.329/天(≈€9.87/月)/ 电网费 + heffingskorting 待填。
</content>
+322
View File
@@ -0,0 +1,322 @@
# M7 — 电表生命周期 / 换表归档(Meter epochs
> 协作格式、任务卡结构、校验闸门、数据安全红线见 [`README.md`](./README.md),本文不再重复。
## 1. 目标
让计费系统正确处理**电表更换**这一**必然事件**:
- 荷兰 2G 智能电表退网后,电网公司**一定**会把表换成 4G 表(同址换表);搬家继承新表也是同类。
- 计费是**寄存器差值(delta)模型**:每个 15 分钟周期成本 =(周期末读数 − 周期初读数)× 单价。换表后新表寄存器基数与旧表**无关**(可能更高也可能更低),若跨表算 delta 会产出**负成本或巨额假成本**。
- 目标:引入显式的 **Meter(电表)** 概念,标记"某时刻起属于哪一块物理表",使引擎**永不跨表算 delta**,并让累计量按表归零、历史可追溯可查。
非目标(本里程碑不做,§9 留痕):gas / 区域供暖的**计费**"家庭(home)"分组实体;多合同时间线积分;自动识别换表(DSMR 帧无电表序列号,无法自动识别,靠用户显式声明)。
## 2. 现状(实现者可据此工作,不必通读全仓库)
- 计费数据模型(M6):
- `dsmr_reading`:整帧 JSON`recorded_at`(UNIQUE) 去重;含 4 个累计寄存器 `electricity_delivered_1/2``electricity_returned_1/2`。**帧内无电表序列号字段**。
- `energy_cost_period`:每 15 分钟一行,存 `d1/d2/r1/r2_kwh`(delta)、`import_cost/export_revenue/net_cost``pricing` 快照、`contract_version_id``degraded``computed_at``period_start`(unique)。
- 计费引擎 `app/services/energy_cost.py`
- `register_at(session, boundary)`:取 `recorded_at ≤ boundary` 的最近一条;有 15 分钟 staleness 护栏(超过则返回 None → 周期降级)。
- `compute_period(session, t0)`:取 t0/t1 两端寄存器,算 delta,按 `contract` 的 strategy 计价;缺读数→降级行(成本 0)。
- `compute_closed_periods` / `recompute_range`:遍历刻钟边界批量算。
- `summarize(session, start, end)`:聚合非降级周期电费 + 固定费/抵扣(FU10 已按本地日跨版本积分)。
- 计价策略 `app/integrations/pricing/strategies.py``import_cost = Δd1×buy_dal + Δd2×buy_normal` 等,**对 delta 无任何 clamp/sanity 检查**(负 delta → 负成本,超大 delta → 巨额成本)。
- 暴露 `app/integrations/expose.py`:累计 `import_cost_total`/`export_revenue_total` 锚点(FU11= `max(合同最早 effective_from, 最早非降级 period_start)``*_today` 日归零实体(FU11)。
- 合同 `app/services/contracts.py``active` 互斥 + 版本时间线 + `active_contract_version_at(ts)`(半开 `[from,to)`)。合同**不引用电表**。
- 配置/迁移:单库 appAlembic 链 `alembic_app/``PRAGMA foreign_keys=ON` 已开(`app/db.py`)。
- 时区:FU10 的 `app/services/timezone.py``local_*`,可 monkeypatch)。
## 3. 目标架构
### 3.1 核心概念:Meter= epoch
一条 `meter` 记录 = **一块物理电表的一段安装期** `[started_at, ended_at)`。换表 = 关闭当前 active 表(`ended_at=T`+ 新建一条 `started_at=T` 的表。计费**永不跨 meter 算 delta**:跨表的那个周期按设计降级。
- **每种 `commodity` 至多一个 active 表**`ended_at IS NULL`)。本里程碑只 `electricity` 参与计费,`commodity` 字段为未来 gas/heating 预留。
- `meter_at(session, ts, commodity="electricity")`:返回 `started_at ≤ ts AND (ended_at IS NULL OR ts < ended_at)` 的那块表(半开区间,仿 `active_contract_version_at`)。
- 计费基准:某表窗口内**最早的一条 `dsmr_reading`**`recorded_at ≥ started_at`)即基准;用户**只需填 `started_at` 日期,无需知道当前读数**。追溯(`started_at` 在过去)时,有效起点自然 = `max(started_at, 数据起点)`
- **隔离的前提 = 显式声明**:系统无法自动识别换表(DSMR 帧无序列号),搬家后新旧读数同落 `dsmr_reading`,靠用户**显式新建那块表**才隔离。万一忘了声明:搬家必有的数据空档(staleness)+ delta 护栏(§3.3)保证**不会产出垃圾成本**,只是新数据暂混在旧表 epoch、累计未归零;**事后补声明 + 追溯 `recompute_range`** 即可纠正。
### 3.2 数据模型(新增 1 张表 + 1 列)
`meter` 表(`app/models/energy.py`,单库 app 链):
| 列 | 类型 | 说明 |
| --- | --- | --- |
| `id` | int PK | 内置自增 ID |
| `label` | str | 人读标签,编址/标识(如 "旧 2G 表 @ Dorpsstraat 1" |
| `commodity` | str | 默认 `"electricity"`;预留 `gas`/`heating` |
| `started_at` | datetime(UTC) | 安装/继承/搬家起点(可在过去) |
| `ended_at` | datetime(UTC) \| null | null = 当前 active |
| `reason` | str | `initial` / `meter_swap` / `home_move` / `other` |
| `note` | str \| null | 备注 |
| `created_at` | datetime(UTC) | |
`energy_cost_period` 新增 `meter_id`nullable FK → `meter.id`):记录该周期归属哪块表,便于审计/按表查询/归档。
### 3.3 计费引擎改造(`energy_cost.py` / `strategies.py` 调用处)
1. `register_at(session, boundary, meter)`:**仅在该 meter 窗口内**取读数(`started_at ≤ recorded_at < ended_at``≤ boundary` 且过 staleness 护栏)。绝不把旧表读数拉进新表周期。
2. `compute_period(session, t0)`
- 查 `m0 = meter_at(t0)``m1 = meter_at(t1)`
- `m0 is None`(无表覆盖)→ **降级**(不计费)。
- `m0.id != m1.id`(周期跨表边界)→ **降级**D5:丢这一个周期可接受)。
- 否则在 `m0` 窗口内 `register_at` 两端、算 delta、计价;新增写 `period.meter_id = m0.id`
3. **delta sanity 护栏(D6**:算出 delta 后,若 `d1/d2/r1/r2` 任一 `< 0`,或任一 `> _MAX_DELTA_KWH`(常量,给一个远超住宅的宽松上限,如每 15 分钟 100 kWh)→ **降级**(兜底表重置/DSMR 回绕/数据毛刺,与换表无关也防住)。
4. `compute_closed_periods` / `recompute_range`:逐周期按 `meter_at` 判定,无需额外结构。
### 3.4 累计语义(per-meter 归零,D2
- `expose.py` 累计 `import_cost_total`/`export_revenue_total` 锚点改为**当前 active electricity 表的起点**`anchor = active electricity meter.started_at`(窗口内首条读数自然 ≥ 它)。换表后累计**从零起一段新序列**,不维护偏移、不跨表累加。
- `*_today` 日归零实体不变(其本地今天窗口必落在当前表内)。
- (可选,非核心)暴露"当前电表 label"作为一个文本 sensor,便于 HA 侧识别当前表。
- **已知行为(可接受)**:累计 `_total``state_class: total`,换表归零时其值会下台阶,HA 长期统计可能在换表那一刻记一次性负 blip。HA 侧自己存旧值做 down-sample——已与用户确认**先这样、观察后按需处理**(`last_reset` 信号见 §9 杠杆,本里程碑不做)。
### 3.5 API`app/api/routes/api/` + schemas
- `GET /api/energy/meters` — 列出全部表(时间线,active 在前/或按 started_at)。
- `POST /api/energy/meters` — 声明换表/继承:`{label, started_at, reason, note?}`;服务层关闭当前 active 同 commodity 表(`ended_at=started_at`+ 建新表。`started_at` 必须 `≥` 当前 active 表的 `started_at`(拒绝倒挂,仿合同版本"strictly after")。
- `PATCH /api/energy/meters/{id}` — 改 `label`/`note`,或修正 `started_at`(追溯)。
- `POST /api/energy/meters/{id}/recompute`(或复用现有 recompute)— 追溯变更后对受影响窗口 `recompute_range`,重判跨表周期与归属。
- 改了路由/schema → 重导出 OpenAPI 并提交。
### 3.6 前端(并入 Energy 视图)
- 一个"电表(Meters"管理区:表的时间线(label / 区间 / active 标记 / reason),"我换了表 / 继承了新表"表单(只填 label + 日期 + reason),编辑 label/日期。
- 追溯保存后给出"已重算受影响周期"的反馈(或提供 recompute 按钮)。
- 前端闸门见 §8 引用。
### 3.7 迁移与回填(数据安全 runbook)
- Alembic`alembic_app/`):建 `meter` 表 + 给 `energy_cost_period``meter_id` 列(nullable)。
- **回填(只增不删、幂等、对账)**
1. 若库内已有任何 `dsmr_reading`/`energy_cost_period`,创建一条**初始表** `meter(label="Initial meter", commodity="electricity", started_at = 最早 dsmr_reading.recorded_at(无则最早 period_start,再无则迁移时刻), ended_at=NULL, reason="initial")`
2. 把现有 `energy_cost_period.meter_id` 全部回填为这条初始表的 id。
3. **对账**:回填后 `meter_id IS NULL` 的非降级周期数必须为 0;对不上立即中止、非零退出。
- **红线**:迁移**不删除/不覆盖**任何 `dsmr_reading`/`energy_cost_period`/旧 `.db`/备份;纯新增表+列+回填。先在备份副本演练再对真实库执行。
## 4. 已锁定决策(讨论后拍板)
- **D1 粒度**:Meter 为一等公民,**不绑定 home**;`label` 编址 + `commodity` 区分品类(每 commodity 一个 active)。
- **D2 累计**:换表后累计量**归零**(锚当前表起点),直接吃 DSMR 累计 delta,不维护偏移。
- **D3 合同耦合**:**不引入 home 实体**。合同与电表是两条独立时间线,合同不引用电表 → 同址换表时合同自动照常套用,无需上层抽象。搬家若换合同:用"给现有合同加 version"保持历史连续(多合同时间线积分留作后续杠杆)。
- **D4 追溯**`started_at` 可在过去;有效计费起点 = `max(started_at, 数据起点)`;追溯变更触发 `recompute_range`
- **D5 边界周期**:跨表周期判 **degraded**(可接受,换表常伴随长时间无数据)。
- **D6 delta 护栏**:加(负/异常大 delta → degraded),通用兜底。
- **D7 扩展性**gas/heating **计费**不在 scope`commodity` 字段 + 每品类 active 表使未来加表低成本。
## 5. 任务依赖图
```
M7-T01 (meter 表+模型+列+迁移回填)
├──► M7-T02 (meter 服务层: swap/close/edit, meter_at, 互斥/校验)
│ ├──► M7-T03 (计费引擎 meter-aware + delta 护栏 + period.meter_id)
│ │ └──► M7-T04 (累计 per-meter 归零, expose 锚点)
│ └──► M7-T05 (meter API + 追溯 recompute + OpenAPI)
│ └──► M7-T06 (前端电表管理 UI)
└──────────────────────────────► M7-T07 (文档/roadmap/收尾)
```
## 6. 原子任务(任务卡)
### M7-T01 — `meter` 表 + 模型 + `energy_cost_period.meter_id` + 迁移回填 `[schema]`
- **Status**: `done`
- **Depends**: none
- **Context**: 引入 Meter 实体的数据地基;为现有数据回填初始表。
**Files**
- `modify app/models/energy.py`(新增 `Meter``EnergyCostPeriod``meter_id`
- `create alembic_app/versions/<rev>_meter_table.py`
- `modify tests/test_energy_models.py`
**Steps**
1. 加 `Meter` 模型(字段见 §3.2);`EnergyCostPeriod` 加 nullable `meter_id` FK。
2. 写迁移:建表 + 加列;按 §3.7 回填初始表并回填 `meter_id`;带对账(不一致则 raise)。
3. 测试:模型字段、active-per-commodity 约束的服务层留给 T02;这里测建表/列/回填幂等。
**Out of scope / 不要碰**:计费引擎、expose、API(后续任务)。
**Acceptance criteria**
- [ ] `meter` 表与 `energy_cost_period.meter_id` 建出;迁移可升降级。
- [ ] 有历史数据时回填出恰好一条 `reason="initial"` 的 active 表,且非降级周期 `meter_id` 全部非空。
- [ ] 迁移**不删除/覆盖**任何既有数据;回填幂等。
- [ ] 校验闸门全绿。
**Reviewer checklist**:回填对账逻辑;空库/有数据两种路径;数据安全红线(无删/覆盖)。
### M7-T02 — Meter 服务层(swap/close/edit + `meter_at` + 互斥/校验)
- **Status**: `done`
- **Depends**: M7-T01
- **Context**: 换表/继承的业务逻辑与按时刻查表。
**Files**
- `create app/services/meters.py`
- `create tests/test_meters.py`
**Steps**
1. `meter_at(session, ts, commodity="electricity")`(半开区间)。
2. `declare_meter(session, *, label, started_at, reason, commodity="electricity", note=None)`:校验 `started_at ≥ 当前 active 同 commodity 表的 started_at`(拒绝倒挂);关闭旧 active`ended_at=started_at`+ 建新 active。
3. `list_meters` / `update_meter`label/note/started_at);改 started_at 时维持区间自洽。
4. 测试:互斥(每 commodity 单 active)、倒挂拒绝、追溯、`meter_at` 边界。
**Out of scope**API、引擎、前端。
**Acceptance criteria**
- [ ] 每 commodity 至多一个 activeswap 正确续接区间。
- [ ] `meter_at` 半开区间正确(含边界)。
- [ ] 校验闸门全绿。
**Reviewer checklist**:互斥与续接的事务正确性;追溯改 started_at 的区间一致性。
### M7-T03 — 计费引擎 meter-aware + delta 护栏 + `period.meter_id`
- **Status**: `done`
- **Depends**: M7-T02
- **Context**: 让计费永不跨表算 delta,并兜底异常 delta。
**Files**
- `modify app/services/energy_cost.py`
- `modify tests/test_energy_cost.py`
**Steps**
1. `register_at` 限定在传入 meter 的窗口内取读数。
2. `compute_period`:按 §3.3 判定 `m0/m1`;无表/跨表 → 降级;否则算 delta、写 `meter_id`
3. 加 `_MAX_DELTA_KWH` 常量与护栏:任一寄存器 delta `<0``>上限` → 降级。
4. `compute_closed_periods`/`recompute_range` 逐周期 meter 判定。
5. 测试:同表内正确;跨表周期降级;负/超大 delta 降级;无 active 表降级。
**Out of scope**expose 锚点(T04)、APIT05)。
**Acceptance criteria**
- [ ] 跨 meter 边界周期 = degraded;无表覆盖 = degraded。
- [ ] 负 delta / 超 `_MAX_DELTA_KWH` = degraded,绝不产负/巨额成本。
- [ ] 同表内 delta 计费与归属 `meter_id` 正确。
- [ ] 校验闸门全绿。
**Reviewer checklist**staleness × meter 窗口的交互;护栏阈值是否远超住宅、不会误伤真实用量;recompute 重判归属。
### M7-T04 — 累计 per-meter 归零(expose 锚点)
- **Status**: `done`
- **Depends**: M7-T03
- **Context**: 换表后 MQTT 累计量从零起算。
**Files**
- `modify app/integrations/expose.py`
- `modify tests/test_energy_expose.py`
**Steps**
1. 累计 `import_cost_total`/`export_revenue_total` 锚点改为**当前 active electricity 表的 `started_at`**(替换 FU11 的 `max(合同, 首周期)`)。
2. `*_today` 不变;确认无非降级周期/无 active 表时仍 `None` 保护。
3.(可选)暴露当前电表 label 文本实体。
4. 测试:换表后累计仅含当前表;anchor 取当前表起点。
**Out of scope**API/前端。
**Acceptance criteria**
- [ ] 累计量锚 = 当前 active electricity meter 起点;换表归零。
- [ ] None 保护分支保留;daily 实体不受影响。
- [ ] 校验闸门全绿。
**Reviewer checklist**:与 FU11 行为差异是否符合 D2;无表/空数据分支。
### M7-T05 — Meter API + 追溯 recompute + OpenAPI
- **Status**: `done`
- **Depends**: M7-T02
- **Context**: 暴露电表 CRUD 与换表声明;追溯后重算。
**Files**
- `create app/api/routes/api/meters.py`
- `create app/schemas/meter.py`
- `modify app/api/routes/...`(注册路由)
- `modify openapi/openapi.json`, `openapi/openapi.yaml`
- `create tests/test_api_meters.py`
**Steps**
1. `GET/POST/PATCH /api/energy/meters`+ 追溯触发 `recompute_range`)。
2. 鉴权沿用 `require_session`
3. `python scripts/export_openapi.py` 并提交 `openapi/`
4. 测试:列出/声明换表/编辑/倒挂拒绝/追溯重算。
**Out of scope**:前端。
**Acceptance criteria**
- [ ] 端点行为符合 §3.5;追溯变更触发受影响窗口重算。
- [ ] `git diff --exit-code openapi/` 干净(已重导出提交)。
- [ ] 校验闸门全绿。
**Reviewer checklist**:错误码(倒挂/不存在);recompute 范围是否覆盖受影响周期。
### M7-T06 — 前端电表管理 UI
- **Status**: `done`
- **Depends**: M7-T05
- **Context**: 让用户在 Energy 页声明换表、查看电表时间线。
**Files**
- `create frontend/src/energy/MeterManager.tsx`
- `modify frontend/src/energy/EnergyPage.tsx`, `frontend/src/energy/hooks.ts`
- `modify frontend/src/energy/*.test.tsx`
- `modify frontend/src/api/schema.d.ts``npm run codegen`
**Steps**
1. `npm run codegen` 拉新端点类型。
2. 电表时间线 + "换表/继承"表单(label + 日期 + reason+ 编辑。
3. 追溯保存后反馈"已重算"。
4. 前端闸门(§8)。
**Out of scope**:后端。
**Acceptance criteria**
- [ ] 能列出/声明换表/编辑;UI 合理。
- [ ] 前端 lint/typecheck/test/build 全绿;`schema.d.ts` 已 codegen。
**Reviewer checklist**:日期→后端时区语义(沿用 FU10 本地午夜 naive 约定);空状态/加载/错误。
### M7-T07 — 文档 / OpenAPI / roadmap 收尾
- **Status**: `done`
- **Depends**: M7-T01..T06
- **Context**: 把 M7 落档、更新 roadmap 与索引。
**Files**
- `modify docs/roadmap.md`(新增 M7 条目,标完成)
- `modify docs/design/README.md`(索引加 m7
- `modify docs/design/m7-meter-epochs-archival.md`Status 收尾)
- `modify docs/*`(如新增 meter 模块说明,按需)
**Acceptance criteria**
- [ ] roadmap/README/本文 Status 一致;OpenAPI 已是最新并提交。
- [ ] 校验闸门全绿。
## 7. 构建上下文完整性(M1 教训)
- 本里程碑只**新增** `app/services/meters.py``app/api/routes/api/meters.py``app/schemas/meter.py``alembic_app/versions/<rev>_*.py` 与前端文件;均落在现有 `Dockerfile``COPY app ./app` / `COPY alembic_app ./alembic_app` / 前端构建阶段覆盖范围内,**无新增顶层目录**。
- `tests/test_deployment.py::test_dockerfile_copy_sources_exist` 应仍绿;删/移文件时按 README 规则 grep 构建清单(本里程碑预期无删除)。
## 8. 前端校验闸门
`frontend/` 下:`npm run lint && npm run typecheck && npm run test && npm run build`;改了端点/schema 还需 `npm run codegen` 并提交 `schema.d.ts`
## 9. 后续杠杆(本里程碑不做,留痕)
- **Home/Site 分组**:若将来要"一个地址下多品类表合并出账 + 整屋归档",再加薄分组层(可仅 `site_label` 字段,不必新实体)。
- **多合同时间线积分**:让 `summarize` 跨多个非重叠合同积分固定费/抵扣,免去搬家"加 version"的折中。
- **换表时给累计实体发 `last_reset` 信号**:消除 HA 长期统计在归零时刻的一次性负 blip(需给 expose/discovery 加 `last_reset` 管线,FU11 当时为省事未做)。已与用户确认本里程碑**先不做**,先观察、需要时再加。
- **Gas / 区域供暖计费**`commodity!="electricity"` 的 strategy 与寄存器映射。
- **当前电表 label 暴露给 HA**、按表/按地址的成本报表分段。
## 10. 里程碑完成定义(DoD
**已完成**M7-T01..T07 全部 done
- `meter` 数据地基 + 回填上线;计费引擎永不跨表算 delta,跨表/无表/异常 delta 一律降级。
- 累计量按当前表归零;追溯换表可重算。
- Meter CRUD API + 前端管理 UI 可用;OpenAPI/schema 已同步。
- 全程数据安全红线无违反;pytest/ruff/前端闸门全绿。
- 人工 walkthrough:声明一次(含追溯)换表 → 旧表周期保留、跨表周期降级、新表从零累计、HA 累计实体不出现负/巨额跳变。
+114
View File
@@ -0,0 +1,114 @@
# Meter Epochs(电表生命周期 / 换表归档)
本文档说明 **Meter epoch** 概念、换表声明流程、计费隔离行为、累计量归零、追溯重算,以及与 HA 累计 blip 的已知行为。
## 为什么需要 Meter epoch
计费引擎采用**寄存器差值(delta)模型**:每 15 分钟成本 =(周期末读数 − 周期初读数)× 单价。荷兰 2G 智能电表退网后,电网公司会把表换成 4G 表(同址换表);搬家继承新表也是同类场景。
问题在于:**新表的寄存器基数与旧表无关**,若跨表算 delta,会产出负成本或巨额假成本。Meter epoch 让引擎知道"某时刻起属于哪一块物理表",从而永不跨表算 delta。
## 核心概念
一条 `meter` 记录 = **一块物理电表的一段安装期** `[started_at, ended_at)`
- `started_at`:表安装/继承/搬家的起点(可在过去,追溯声明)。
- `ended_at`null = 当前 active;非 null = 已退役表的结束时刻。
- `label`:人读标签,便于识别(如 "旧 2G 表 @ Dorpsstraat 1")。
- `reason``initial`(初始建档)/ `meter_swap`(换表)/ `home_move`(搬家)/ `other`
- `commodity`:默认 `electricity`;为未来 gas/heating 预留,每 commodity 至多一个 active 表。
**换表** = 关闭旧 active 表(`ended_at = T`+ 新建 `started_at = T` 的表。系统**无法自动识别换表**(DSMR 帧无电表序列号),靠用户**显式声明**才能隔离。
## 计费隔离行为
每个 15 分钟计费周期,引擎先查该周期两端(t0/t1)分别属于哪块表:
| 情况 | 结果 |
| --- | --- |
| 同一块表(正常)| 在表窗口内取读数、算 delta、正常计费 |
| 无表覆盖(未声明)| **降级**(成本置 0`degraded=true` |
| 跨表边界(t0 ≠ t1 的表)| **降级**(丢这一个周期,可接受;换表常伴随长时间无数据) |
| delta < 0 或 > 100 kWh/15min | **降级**(兜底异常,如 DSMR 回绕/毛刺,与换表无关也防住) |
**忘了声明换表怎么办**:搬家必有的数据空档(staleness)+ delta 护栏保证不会产出垃圾成本,只是新数据暂混在旧表 epoch、累计未归零。事后补声明 + 追溯重算即可纠正(见下方"追溯换表")。
## 换表声明流程
### 通过前端(Energy 页 → Meters tab
1. 在 Energy 页切到 **Meters** tab,查看当前电表时间线。
2. 点击"声明换表 / 继承新表",填写:
- **Label**:新表的人读标签(如 "4G 新表 2026"
- **安装日期**:新表的 `started_at`(可选择历史日期追溯)
- **Reason**`meter_swap` / `home_move` / `other`
3. 保存后,旧表自动关闭(`ended_at = 填写日期`),新表成为 active。
4. 如填写的是过去日期,系统自动触发受影响窗口的重算(`recompute_range`),跨表周期变为降级,新表内周期重新归属。
### 通过 API
```http
POST /api/energy/meters
Content-Type: application/json
{
"label": "4G 新表 2026",
"started_at": "2026-07-01T00:00:00",
"reason": "meter_swap",
"note": "荷兰电网公司换装"
}
```
`started_at` 必须 `≥` 当前 active 表的 `started_at`(拒绝倒挂)。
查询所有表(时间线):
```http
GET /api/energy/meters
```
编辑 label / 备注 / 修正日期:
```http
PATCH /api/energy/meters/{id}
```
## 追溯换表(started_at 在过去)
若换表日期在过去但当时忘了声明:
1. 声明新表,填写过去的安装日期。
2. 系统自动对 `[started_at, now]` 范围内的周期触发 `recompute_range`
- 跨表边界周期 → 降级。
- 归属新表窗口内的周期 → `meter_id` 更新为新表。
3. MQTT 累计量锚点切换至新表起点,下一次 expose 推送时累计量从新表起点重算。
## 累计量归零(per-meter
MQTT 上报的累计成本实体(`import_cost_total` / `export_revenue_total`)锚点 = **当前 active electricity 表的 `started_at`**。换表后:
- 累计量从新表安装时刻起重新累加,不跨表维护历史偏移。
- 日归零实体(`*_today`)不受影响(其窗口必落在当前表内)。
### 已知行为:HA 长期统计的一次性负 blip
累计实体 `state_class: total`,换表归零时其值会**下台阶**。Home Assistant 长期统计(energy dashboard / statistics)可能在换表那一刻记录一次性负 blip。
这是**已知、可接受的行为**:HA 自身存旧值做 down-sample,短期 blip 不影响日常电费读数。已与用户确认先这样观察。如需消除 blip,可在未来通过 `last_reset` 信号通知 HA(见设计文档 §9 后续杠杆,本里程碑不做)。
## 回填(迁移时的初始表)
首次升级到含 M7 迁移的版本时:
- 若库内已有任何 `dsmr_reading` / `energy_cost_period`,迁移自动创建一条 `reason="initial"` 的初始表(`started_at = 最早 dsmr_reading.recorded_at`),并回填现有 `energy_cost_period.meter_id`
- 若是全新空库,初始表不创建(无历史数据)。
- 回填幂等:重复跑迁移不会创建多条初始表;回填后对账(非降级周期 `meter_id IS NULL` 数必须为 0)。
## 非目标(本里程碑不做)
- Gas / 区域供暖的计费(`commodity != "electricity"` 的 strategy)。
- "家庭(home)"分组实体;多合同时间线积分。
- 自动识别换表(DSMR 帧无电表序列号,无法自动识别,靠用户显式声明)。
- `last_reset` 信号(消除 HA 长期统计 blip)。
> 详细设计与任务卡:[`docs/design/m7-meter-epochs-archival.md`](./design/m7-meter-epochs-archival.md)
Binary file not shown.
+134
View File
@@ -0,0 +1,134 @@
# SDM120 Modbus 协议(从官方 PDF 提取)
> 来源:`docs/references/SDM120-MODBUS_Protocol.pdf`
> Eastron SDM120 Modbus Smart Meter Modbus Protocol Implementation **V2.4**
> 本文件是 PDF 的可读化提取,供本项目的 Modbus 采集驱动设计参考。**以官方 PDF 为准**,本文件如有出入以 PDF 为准。
## 0. 本项目的接入方式(重要)
SDM120 物理层是 **Modbus RTURS-485 串口)**。本项目通过一个 **Modbus-TCP 网关**接入:
- 后端用 **Modbus TCP**`IP:port`)连到网关,网关在串口侧转成 RTU 与电表通信。
- TCP 帧用 MBAP header、**无 CRC**CRC/Error Check 由网关在 RTU 侧处理)。本文档里 RTU 帧的 `Error Check (Lo/Hi)` 字段在 TCP 模式下不需要我们关心。
- **Slave Address / Unit ID = 电表的 Meter ID**(默认 `1`,范围 1–247),在 TCP 请求里作为 unit id 传入。
- 若以后直连串口(RTU),才需要管波特率 / 校验位 / CRC(见 §5 holding 寄存器)。
## 1. 协议帧格式
MODBUS 定义 master 查询 / slave 响应的格式。Eastron 电表用 16-bit 寄存器在主从间传值,**但实际数据是 32-bit IEEE-754 浮点**,因此每个测量参数占**两个相邻的 16-bit 寄存器**。
### Querymaster → slaveRTU 帧)
| 字段 | 说明 |
| --- | --- |
| Slave Address | 8-bit,目标从机地址 12470=广播,Eastron 不支持广播) |
| Function Code | 8-bit,功能码(Eastron 支持 **03 / 04 / 08 / 16(=0x10)** |
| Start Address (Hi/Lo) | 16-bit 起始寄存器地址;**寄存器成对使用、从 0 开始,所以起始地址必须是偶数** |
| Number of Points (Hi/Lo) | 16-bit 请求的寄存器数量;**也必须是偶数**(成对读浮点) |
| Error Check (Lo/Hi) | 16-bit CRCRTU 模式;TCP 网关模式无此字段) |
### Responseslave → master
| 字段 | 说明 |
| --- | --- |
| Slave Address | 响应从机地址 |
| Function Code | 与查询相同的功能码(表示识别并已响应) |
| Byte Count | 8-bit,本次返回的数据字节数 |
| Data (寄存器对) | 每个寄存器 Hi byte / Lo byte,按"高寄存器在前"排列 |
| Error Check (Lo/Hi) | 16-bit CRCRTU 模式) |
### Exception Response(异常响应)
- 异常响应的 Function Code = 查询功能码 **OR 0x80**(最高位置 1)。
- 数据是单字节 Error Code(异常码)。
- PDF 正文写了"见后文 Table Of Exception Codes",但提取的 8 页里**没有附上该表**。标准 Modbus 异常码(供参考,非本 PDF 内容):`01` 非法功能、`02` 非法数据地址、`03` 非法数据值、`04` 从机设备故障。
## 2. 功能码
| 功能码 | 作用 | 寄存器区 |
| --- | --- | --- |
| **04** | Read Input Registers(读输入寄存器,3X)—— **所有测量值都在这里** | 30001+ |
| **03** | Read Holding Registers(读保持寄存器,4X)—— 配置项 | 40001+ |
| **16 / 0x10** | Write Holding Registers(写保持寄存器,4X)—— 改配置 | 40001+ |
| 08 | Diagnostics(诊断) | — |
> ⚠️ **测量值用 FC 04(输入寄存器),不是 FC 03。** 这是最常见的踩坑点。
## 3. 浮点数据编码
- 每个参数 = **32-bit IEEE-754 float**,占两个相邻 16-bit 寄存器。
- **字序(word order= 大端:高寄存器在前。**
- **字节序(byte order)= 大端:寄存器内高字节在前。**
- 即整体就是标准大端 float`>f`),4 字节顺序 = `[Reg1 Hi][Reg1 Lo][Reg2 Hi][Reg2 Lo]`
**实例(来自 PDF**
| 含义 | 原始 4 字节 (hex) | 解码值 |
| --- | --- | --- |
| Volts 1 | `43 66 33 34` | `230.2` V |
| Demand Time | `3F 80 00 00` | `1.0` |
| Network Node | `42 70 00 00` | `60.0` |
> Python 解码:`struct.unpack('>f', bytes([0x43,0x66,0x33,0x34]))[0]``230.2`
> pymodbus 用 `BinaryPayloadDecoder.fromRegisters(regs, byteorder=Endian.BIG, wordorder=Endian.BIG)`
## 4. 输入寄存器表(FC 04 读测量值)
全部为 `Float`,长度 4 字节,每项占 2 个寄存器。"Hex 起始"是 Modbus 协议起始地址(即 Start Address Hi/Lo)。
| 寄存器 | 参数 | 单位 | Hex 起始 |
| --- | --- | --- | --- |
| 30001 | Voltage(电压) | Volts | `0000` |
| 30007 | Current(电流) | Amps | `0006` |
| 30013 | Active power(有功功率) | Watts | `000C` |
| 30019 | Apparent power(视在功率) | VA | `0012` |
| 30025 | Reactive power(无功功率) | VAr | `0018` |
| 30031 | Power factor(功率因数) | — | `001E` |
| 30071 | Frequency(频率) | Hz | `0046` |
| 30073 | Import active energy(导入有功电能) | kWh | `0048` |
| 30075 | Export active energy(导出有功电能) | kWh | `004A` |
| 30077 | Import reactive energy(导入无功电能) | kvarh | `004C` |
| 30079 | Export reactive energy(导出无功电能) | kvarh | `004E` |
| 30085 | Total system power demand | W | `0054` |
| 30087 | Maximum total system power demand | W | `0056` |
| 30089 | Import system power demand | W | `0058` |
| 30091 | Maximum import system power demand | W | `005A` |
| 30093 | Export system power demand | W | `005C` |
| 30095 | Maximum export system power demand | W | `005E` |
| 30259 | Current demand | Amps | `0102` |
| 30265 | Maximum current demand | Amps | `0108` |
| 30343 | Total active energy(总有功电能) | kWh | `0156` |
| 30345 | Total reactive energy(总无功电能) | kvarh | `0158` |
**读取分块建议**`0x00000x005E`(30001–30095)地址连续,可一次块读;`0x0102/0x0108``0x0156/0x0158` 各为独立小块。整表用 2–3 次块读即可覆盖,减少 Modbus 事务数。
### 常用核心子集(日常监控够用)
电压 `0000`、电流 `0006`、有功功率 `000C`、功率因数 `001E`、频率 `0046`、导入有功电能 `0048`、导出有功电能 `004A`、总有功电能 `0156`
## 5. 保持寄存器表(FC 03 读 / FC 16 写配置)
| 寄存器 | 参数 | Hex 起始 | 格式 | 说明 |
| --- | --- | --- | --- | --- |
| 40013 | Relay Pulse Width | `000C` | Float | 继电器脉宽 60/100/200 ms,默认 100ms |
| 40019 | Network Parity Stop | `0012` | Float | 0=1停止位无校验(默认),1=1停止位偶校验,2=1停止位奇校验,3=2停止位无校验;**改后需重启生效** |
| 40021 | Meter ID | `0014` | Float | 从机地址 1247,默认 1 |
| 40029 | Baud rate | `001C` | Float | 0=2400(默认)1=48002=96005=1200 |
| 40087 | Pulse 1 output mode | `0056` | Float | 0001 导入有功,0002 导入+导出有功,0004 导出有功(默认),0005 导入无功,0006 导入+导出无功,0008 导出无功 |
| 463745 | Time of scroll display | `F900` | HEX 2字节 | 滚动显示时间 0–30s,默认 0(不滚动) |
| 463761 | Pulse 1 output | `F910` | HEX 2字节 | 0000:0.001kWh/imp(默认)0001:0.010002:0.10003:1 kWh/imp |
| 463777 | Measurement mode | `F920` | HEX 2字节 | 1:total=import2:total=import+export(默认)3:total=import-export |
| 464513 | Serial number | `FC00` | uint32 4字节 | 序列号,**只读** |
| 464515 | Meter code | `FC02` | Hex 2字节 | 设备码=`0020`**只读** |
| 464516 | Software version | `FC03` | Hex 2字节 | 软件版本,**只读** |
> ⚠️ 写保持寄存器(改 Meter ID / 波特率 / 校验位等)会改变电表通信参数,配错可能导致**通信中断**。本项目默认**只读采集**,不建议在自动化链路里写电表配置。
## 6. 给本项目采集驱动的要点小结
1. 走 **Modbus TCP 网关**`ModbusTcpClient(host, port)``slave=<Meter ID>`
2. 测量值用 **FC 04 / 输入寄存器**,按 §4 地址读。
3. 解码 **大端 float32**word & byte 都大端,高寄存器在前)。
4. 起始地址与数量**都用偶数**(成对读)。
5. 默认**只读**,不写电表配置寄存器。
6. 不同型号电表 → 不同"寄存器 profile"。本表是 `sdm120` 这一个 profile 的定义。
@@ -0,0 +1,249 @@
# Tibber API + 荷兰电价 + DSMR Reader 参考
> 本文件汇总 M6(动态/固定电价 → 实时买卖电费计算)所需的全部外部事实,供 `docs/design/m6-tibber-dynamic-energy.md` 与实现者直接 refer。
> 来源:Tibber GraphQL introspection(实证)、Tibber Explorer demo 数据、Tibber NL support 文档、用户实际合同与 DSMR Reader 配置(2026-06 整理)。
> 凡标「✅ 实证」的为代码/数据验证过的事实;金额类以用户真实账单 / 合同生效后真实 token 为最终准(见末尾「待真实数据核对」)。
---
## 1. Tibber GraphQL API
- **Endpoint**`POST https://api.tibber.com/v1-beta/gql`
- **认证**HTTP header `Authorization: Bearer <token>`token 在 [developer.tibber.com](https://developer.tibber.com) 登录后生成(每个 Tibber 账户一个)。
- **在线 Explorer**[developer.tibber.com/explorer](https://developer.tibber.com/explorer),自带一个公共 **demo token**
- ⚠️ **demo token 现在只能做 schema introspection 与历史 demo 数据查询**;对 `viewer` 的真实数据查询返回 `UNAUTHENTICATED`。要拿自己家的实时价,必须用**自己账户的 token**。
### 1.1 `Price` 类型(✅ introspection 实证)
`Price` 对象恰好 6 个字段:
| 字段 | 类型 | 含义 |
| --- | --- | --- |
| `total` | Float | **全包价 = energy + tax**(即 App 里的 All-in Price,含 spot+能源税+VAT 等)|
| `energy` | Float | 纯能源/现货分量(ex-VAT 市场价)|
| `tax` | Float | 税费分量(能源税 + VAT 合并)|
| `startsAt` | String | 该价格段起点(ISO8601,**含时区偏移**,如 `+02:00`|
| `currency` | String! | 币种(唯一 non-null 字段;NL 为 EURdemo 为 SEK|
| `level` | PriceLevel | 价格档枚举(`VERY_CHEAP/CHEAP/NORMAL/EXPENSIVE/VERY_EXPENSIVE`|
**✅ 实证**`total == energy + tax` 精确成立(见 §1.4 样本)。
### 1.2 15 分钟价格:`Subscription.priceInfoRange`(✅ introspection 实证)
- **字段**`currentSubscription.priceInfoRange`
- **参数**`resolution: PriceInfoRangeResolution!`(必填)、`first: Int``last: Int``before: String``after: String`(游标分页)
- **resolution 枚举 `PriceInfoRangeResolution`**:值 `DAILY` / `HOURLY` / `QUARTER_HOURLY`
- ⚠️ 注意别用错枚举:另有 `PriceResolution`(只有 HOURLY/DAILY**无 QUARTER_HOURLY**)和 `PriceInfoResolution`——`priceInfoRange` 用的是 **`PriceInfoRangeResolution`**。
- **返回**`SubscriptionPriceConnection { nodes: [Price]!, edges: [...], pageInfo }`
- **废弃说明**:老的 `priceInfo.range``deprecationReason: "use Subscription.priceInfoRange instead"`**不支持 QUARTER_HOURLY**,必须改用 `Subscription.priceInfoRange`
- **✅ 实证(demo key2018 历史日)**`QUARTER_HOURLY` 返回 **96 个节点、间隔精确 15 分钟**;无真 15 分钟价的历史日,API 把**小时价重复成 4 个相同刻钟**(00/15/30/45 同值,到下一整点才变);NL 当前数据则是真 15 分钟。
- **解析注意**:**不要假设固定 96 个节点 / 固定 15 分钟间隔**,按 `startsAt` 逐点存最稳。
**15 分钟查询(拉今天 96 个刻钟)**:
```graphql
{ viewer { homes { id currentSubscription {
priceInfoRange(resolution: QUARTER_HOURLY, first: 96) {
nodes { startsAt total energy tax currency level }
} } } } }
```
### 1.3 小时价 / 当前价 / 探测:`priceInfo`
`currentSubscription.priceInfo``current`(当前价)、`today[]``tomorrow[]`(次日价约 13:00 CET 发布)。探测账户能力 + 当前价:
```graphql
{ viewer { homes {
id appNickname
features { realTimeConsumptionEnabled }
currentSubscription { priceInfo {
current { total energy tax startsAt level currency }
today { total energy tax startsAt level }
tomorrow { total energy tax startsAt level }
} } } } }
```
### 1.4 现成 curl + 样本响应
```bash
# 15 分钟价(换成自己账户 tokendemo token 拿不到 viewer 真实数据)
curl -s -X POST https://api.tibber.com/v1-beta/gql \
-H "Authorization: Bearer <YOUR_TOKEN>" -H "Content-Type: application/json" \
-d '{"query":"{ viewer { homes { id currentSubscription { priceInfoRange(resolution: QUARTER_HOURLY, first: 96) { nodes { startsAt total energy tax currency } } } } } }"}'
```
**样本:`priceInfo`demo,瑞典家庭,SEK**
```json
{"current": {"total": 0.8239, "energy": 0.5663, "tax": 0.2576,
"startsAt": "2026-06-23T14:00:00.000+02:00", "level": "NORMAL", "currency": "SEK"},
"today": [{"total": 1.276, "energy": 0.928, "tax": 0.348,
"startsAt": "2026-06-23T00:00:00.000+02:00", "level": "VERY_EXPENSIVE"}, … 24 项],
"tomorrow":[… 24 项]}
```
**样本:`priceInfoRange` QUARTER_HOURLYdemo2018-11-0296 节点)**
```json
{"nodes": [
{"startsAt":"2018-11-02T00:00:00.000+01:00","total":0.63, "energy":0.435,"tax":0.195,"currency":"SEK"},
{"startsAt":"2018-11-02T00:15:00.000+01:00","total":0.63, "energy":0.435,"tax":0.195,"currency":"SEK"},
{"startsAt":"2018-11-02T00:30:00.000+01:00","total":0.63, "energy":0.435,"tax":0.195,"currency":"SEK"},
{"startsAt":"2018-11-02T00:45:00.000+01:00","total":0.63, "energy":0.435,"tax":0.195,"currency":"SEK"},
{"startsAt":"2018-11-02T01:00:00.000+01:00","total":0.6141,"energy":0.4223,"tax":0.1918,"currency":"SEK"},
… 共 96 个,末节点 23:45 total 0.6342]}
```
(注意 00:00–00:45 四个刻钟同值 = 小时价被重复;`total==energy+tax`。)
---
## 2. 荷兰电价构成(Tibber NL + 用户合同)
> 所有金额**含 VATincl. btw**,除非另注。VAT/BTW 标准税率 **21%**
| 分量 | 荷兰语 | 单位 | 值 / 说明 |
| --- | --- | --- | --- |
| 现货/市场价 | marktprijs / dynamische kwartierprijs | €/kWh | 每 15 分钟跟随交易所;= API `energy` |
| 买侧采购费 | **inkoopvergoeding** | €/kWh | **€0.0248**(覆盖 onbalans + 绿证;见下「相等」事实)|
| 卖侧上网费 | **verkoopvergoeding** | €/kWh | **€0.0248**2026-01-01 起)|
| 能源税 | energiebelasting | €/kWh(含 VAT| 2026 第一档(010000 kWh)≈ **€0.1108**2025 ≈ €0.1228;高档更低 |
| 增值税 | BTW | % | **21%** |
| 固定月费 | vast bedrag / leveringskosten | €/月/合同 | Tibber **€5.99/月**(电、气各一次)|
| 电网维护费 | netbeheerkosten / systeembeheerkosten | €/天或/月 | 电网公司定、供应商代收;按地区,2026 约 +3.38% |
| 能源税抵扣 | heffingskorting / vermindering energiebelasting | €/年/连接 | 政府年度减免,从总费用扣 |
| ODE | Opslag Duurzame Energie | €/kWh | **现为 0**(已并入 energiebelasting|
**✅ 关键事实(Tibber NL 文档原文)**
> "De verkoopvergoeding van 2,48 cent is gelijk aan de inkoopvergoeding die je bij je afgenomen stroom betaalt."
> (卖侧 verkoopvergoeding 2.48 分 = 买侧 inkoopvergoeding。)
**买卖服务费相等(均 €0.0248/kWh)**,在买卖里一进一出**相互抵消**。
---
## 3. 净计量(saldering)、回送(teruglevering)、负电价、2027
### 3.1 回送价(净计量期内,文档原文)
> "Op het moment dat je teruglevert geven we je per kWh de beursprijs die op dat moment geldt …, inclusief energiebelasting en inkoopvergoeding plus de btw minus de verkoopvergoeding."
即净计量期内回送价 = `beursprijs + energiebelasting + inkoopvergoeding + btw verkoopvergoeding`。因 inkoopvergoeding = verkoopvergoeding 抵消 → **= 全额零售价**spot+能源税+VAT),正是 saldering "回送 1 度 = 用 1 度"的本质。
### 3.2 年末盈余 / 取消净计量后(文档原文,Scenario 2)
> "Voor de overproductie van 500 kWh heb je recht op de beursprijs en de inkoopvergoeding, maar heb je geen recht op de energiebelasting. … ontvang je nog een factuur van ons voor de te veel uitgekeerde belastingen …"
即**超额回送(或 2027 取消净计量后)**:盈余按 `beursprijs + inkoopvergoeding verkoopvergoeding`(**无能源税**)计价 → 因两费抵消 → **= 纯 spot**。
### 3.3 净计量机制
- 法定**按自然年**结算;可抵扣到「用电量」为止(回送抵到用电为止)。
- Tibber 不用预付,**净计量周期在第 12 张账单后结束**。
- **2027 起荷兰取消净计量(saldering stopt in 2027**——用户据此设计:**不再考虑净计量,直接按实时买卖算**。
### 3.4 负电价
- 负价时**用电**:理论上你拿钱(但仍计能源税,净计量期内税会退回)。
- 负价时**回送**:你要为回送的电付那个负价(= 倒贴)。
- 因 Tibber 给的是 all-in `total`(已含能源税),**只有 spot 负到比能源税还多,total 才转负**;中等负价时 total 仍正(照付)。
- **结论:负电价不需要特殊处理**——`buy=total``sell=total−能源税` 符号自洽。
---
## 4. 本项目采用的买/卖价公式(M6)
> spot 取 API `energy``total = energy + tax`(全包)。**买价直接用 `total`**,卖价从 `total` 扣掉卖电不交的能源税。
- **Tibber 动态合同**post-2027 口径):
- 买价 `buy = price.total`
- 卖价 `sell = price.total energy_tax_per_kwh sell_adjust``sell_adjust` 默认 0;含 VAT 归己;买卖费抵消已隐含在 total 里)
- **固定合同(manual,双费率)**
- 买价 `buy_档 = energy_buy_档 + energy_tax`(档 ∈ {normal, dal}
- 卖价 `sell_档 = sell_档`(回送价,**无能源税**
- **固定费/抵扣不进每度**`network_fee``management_fee`(月→天)、`heffingskorting`(年→天)在**日/月/年汇总**层加减。
- 详见 `docs/design/m6-tibber-dynamic-energy.md` §3.4。
---
## 5. 用户当前固定合同(manual 实例,2026-06
> 双费率(NL`_1`=dal/低谷、`_2`=normal/高峰)。以下为当前值,会随合同阶段/年度变(系统按"加新版本+生效日期"留底)。
| 项 | 当前值 | 备注 |
| --- | --- | --- |
| 能源价 normal(高) | €0.133/kWh | 买侧,含 VAT,不含能源税 |
| 能源价 dal(低) | €0.127/kWh | 买侧 |
| 回送价 normal | (现与 dal 同)| 卖侧,分档但现同价;系统仍分开填 |
| 回送价 dal | — | 卖侧 |
| 能源税 energiebelasting | 待填(≈€0.1108| 含 VAT,加到买价 |
| ODE | 0 | 已并入能源税 |
| 电网维护费 | 待填 | 按天(合同按天收)|
| 供应商管理费 | ≈€0.329/天(≈€9.87/月)| 对应 Tibber 的 €5.99/月 |
| heffingskorting | 待填 | 年度减免,汇总时扣 |
| 回送阶梯罚金 terugleverkosten | **不做** | 按自然年累计、阶梯式;用户住不到年底算不准,M6 不实现 |
---
## 6. DSMR Reader → MQTT
### 6.1 取数方式
- 用 **Telegram JSON**(单 topic **`dsmr/json`**,一条 = 一帧完整 telegram 的 JSON),**不用** split-topic(每字段一 topic、要按 id 拼)、**不用** raw(未解析 OBIS)。
- DSMR Reader **每秒一条**。本项目按 **10 秒降采样**(仅秒数整 10 落盘)、**整帧存 JSON blob**(不做字段 allowlist)。
### 6.2 telegram JSON 字段(DSMR Reader 配置的 JSON mapping
```
id, timestamp,
electricity_delivered_1, electricity_delivered_2, # 进口累计 kWh_1=dal 低谷, _2=normal 高峰)
electricity_returned_1, electricity_returned_2, # 出口/回送累计 kWh
electricity_currently_delivered, electricity_currently_returned, # 瞬时进/出口功率 kW
phase_currently_delivered_l1/l2/l3, phase_currently_returned_l1/l2/l3, # 各相瞬时功率 kW
phase_voltage_l1/l2/l3, # 各相电压 V
phase_power_current_l1/l2/l3, # 各相电流 A
extra_device_timestamp, extra_device_delivered # 燃气表(m³,每 5 分钟更新)
```
### 6.3 实测样本(单相,`dsmr/json`
```json
{
"id": 200086230,
"timestamp": "2026-06-23T12:16:48Z",
"electricity_delivered_1": "20915.154",
"electricity_returned_1": "2979.905",
"electricity_delivered_2": "15212.090",
"electricity_returned_2": "6786.406",
"electricity_currently_delivered": "0.000",
"electricity_currently_returned": "2.704",
"phase_currently_delivered_l1": "0.000",
"phase_currently_delivered_l2": null,
"phase_currently_delivered_l3": null,
"extra_device_timestamp": "2026-06-23T12:15:00Z",
"extra_device_delivered": "6208.234",
"phase_currently_returned_l1": "2.704",
"phase_currently_returned_l2": null,
"phase_currently_returned_l3": null,
"phase_voltage_l1": "237.0",
"phase_voltage_l2": null,
"phase_voltage_l3": null
}
```
### 6.4 解析要点(实测确认)
- **数值都是 JSON 字符串**`"20915.154"``"0.000"`)→ 计费读取转 **Decimal**(累计寄存器算钱要精度)。
- **缺测相位 = `null`**(不是缺 key)→ 视为缺测;单相只有 `*_l1`,三相后 `_l2/_l3` 由 null 变数字。
- **进口总量** = `electricity_delivered_1 + _2`**出口总量** = `electricity_returned_1 + _2`(双费率档对动态合同无意义,求和;对固定合同分档计价)。
- **`timestamp` 为 UTCZ**`id` 自增,做幂等去重。
- 历史样本里电压 key 曾被错配成带前缀 `dsmr/reading/phase_voltage_l1`DSMR Reader 的 JSON mapping 配置笔误,用户已改对)——解析仍按"键名容错"。
- **整表寄存器与相数无关**`delivered/returned_1/2` 是整表总量,三相只多了各相瞬时通道 → **计费逻辑相数无关**
- 燃气 `extra_device_delivered`(m³)一并存入 blob;燃气计费 M6 不做(数据先留,未来按 commodity 扩展)。
---
## 7. 来源 URL
- Tibber GraphQL reference / explorer`https://developer.tibber.com/docs/reference#rootsubscription``https://developer.tibber.com/explorer`
- Tibber NL 费用构成:`https://support.tibber.com/nl/articles/5605892-de-kosten-bij-tibber`
- Tibber NL 净计量与回送:`https://support.tibber.com/nl/articles/4669873-salderen-en-terugleveren-bij-tibber`
- DSMR ReaderHA 集成):`https://www.home-assistant.io/integrations/dsmr_reader/`
---
## 8. 待真实数据核对(合同生效后用真实 token / 账单)
1. **真实 token 复核**:跑 §1.4 的 15 分钟 curl,确认 NL 返回**真** 15 分钟价(非重复小时价)+ 币种 EUR。
2. **卖价残差**:确认 `total` 里 purchase fee 是否被卖侧 sales fee 完全抵掉、回送 VAT 口径 → 调 `sell_adjust`(默认 0)。
3. **双费率寄存器映射**:确认 `_1`=dal/`_2`=normal 没接反(差价小但要对)。
4. **能源税年值**:按当年实际值与年用电档位核 `energy_tax`
5. **固定合同数值**:回送两档价、电网费、heffingskorting 待用户从账单填。
</content>
+143 -12
View File
@@ -2,7 +2,7 @@
本文档记录 `home-automation``v1.0.3` 之后的下一阶段规划。这一阶段不是小修补,而是几次较大的结构性改动:单库化、前端重写、以及远期的移动端试水。
> 每个里程碑的**可执行原子任务**展开在 [`docs/design/`](./design/README.md)M1 [`m1-db-consolidation.md`](./design/m1-db-consolidation.md)、M2 [`m2-frontend-v2.md`](./design/m2-frontend-v2.md)、M3 [`m3-token-mobile.md`](./design/m3-token-mobile.md)。这些文档为 Orchestrator→Implementer→Reviewer 的多模型流水线设计。
> 每个里程碑的**可执行原子任务**展开在 [`docs/design/`](./design/README.md)M1 [`m1-db-consolidation.md`](./design/m1-db-consolidation.md)、M2 [`m2-frontend-v2.md`](./design/m2-frontend-v2.md)、M3 [`m3-token-mobile.md`](./design/m3-token-mobile.md)、M4 [`m4-login-hardening.md`](./design/m4-login-hardening.md)、M5 [`m5-iot-energy.md`](./design/m5-iot-energy.md)、M6 [`m6-tibber-dynamic-energy.md`](./design/m6-tibber-dynamic-energy.md)、M7 [`m7-meter-epochs-archival.md`](./design/m7-meter-epochs-archival.md)。这些文档为 Orchestrator→Implementer→Reviewer 的多模型流水线设计。
## 当前基线(v1.0.3
@@ -36,9 +36,13 @@
| --- | --- | --- |
| **M1** ✅ | 单库化地基 | 把三库合并成单一 `app.db`,清理散落数据层,删掉 Grafana |
| **M2** ✅ | 前端 v2 | React SPA 取代 Jinja,承载 config + 可视化 + 记录增删改 |
| **M4** ✅ | 登录加固 | 防爆破/指数退避 + CLI 逃生通道 + 可选 TOTP 二次验证(**先于 M5** |
| **M5** ✅ | IoT / 能耗采集 | 通用 Modbus 采集(YAML profile + JSON readings+ MQTT/HA Discovery + 前端侧边栏 + Energy 视图 |
| **M6** ✅ | 通用电价层 + DSMR 接入 + 实时电费计算 | 通用电价层(manual/tibber profile + 合同版本)+ DSMR 实时电表接入 + 每 15min 寄存器差×价计量电费(不可变快照)+ 日/月/年汇总 + 反哺 HA Energy + 前端合同/价格/费用视图 |
| **M7** ✅ | 电表生命周期 / 换表归档 | 引入 Meter epoch,计费永不跨表算 delta,跨表/无表/异常 delta 一律降级,累计按当前表归零,追溯换表可重算,Meter CRUD API + 前端管理 UI |
| **M3** | 开放与移动端(远期试水) | token 鉴权 + React Native 移动端 |
排序原则:**先清地基,再在干净结构上盖楼。** M2 的新 API 和 React 必须建立在合并后的单库之上,否则就是在准备推倒的旧数据层上盖新楼、之后回头返工
排序原则:**先清地基,再在干净结构上盖楼。** M2 的新 API 和 React 必须建立在合并后的单库之上;M4 是公网安全加固,在 M5 IoT 集成之前先堵住裸密码这个洞;M5 在安全基座就绪后再做 IoT 接入
---
@@ -132,6 +136,127 @@
---
## M4 — 登录加固(✅ 已完成,排在 M5 之前)
### 目标
给暴露在公网的单 admin 登录做纵深防御,先于 M5 IoT 集成关闭暴力枚举和单因子风险。
### 范围
- **防爆破 / 指数退避**:失败登录按双键(IP + username)指数增长延迟,成功即清零。退避是延迟(429 + Retry-After),不是永久封号。全局开关 `AUTH_LOGIN_THROTTLE_ENABLED`CONFIG_FIELDS);反代后需开 `AUTH_TRUST_FORWARDED_FOR``.env` 部署级)。
- **CLI 逃生通道**`python -m scripts.admin_cli` 直连本地 DB,无需 HTTP 服务、无需任何已存凭据;支持重置密码、解锁退避、关停/重发 TOTP、查看用户。
- **可选 TOTP 二次验证**:admin 可自选启用;启用后两步登录(密码 + 6 位动态码或一次性恢复码);不启用维持纯密码。CLI `disable-totp` 是连恢复码都丢了时的最终逃生口。
> 详细设计与任务卡:[`docs/design/m4-login-hardening.md`](./design/m4-login-hardening.md)
---
## M5 — IoT 集成与能耗采集(✅ 已完成)
### 目标
给后端接入家庭 IoT 生态,建立通用 Modbus 设备采集链路(首个领域:能耗),接入 MQTT + Home Assistant Discovery,并重构前端为侧边导航并新增 Energy 视图。
### 范围
- **两层数据模型(协议与部署分离)**YAML profile(随代码走,声明协议知识:寄存器/解码/key/unit/ha_component+ `modbus_device`(部署层 DB 行:friendly_name/host/port/unit_id/profile/poll_interval/enabled+ `modbus_reading`(通用遥测:device_id + recorded_at + JSON payload)。多设备可共享同一 profile。
- **通用 Modbus-TCP 采集**pymodbusAPScheduler 后台 job 轮询所有 enabled 设备;per-device `last_poll_at`/`last_poll_ok`;全局开关 `MODBUS_POLLING_ENABLED`CONFIG_FIELDS)。首个 profileSDM120 单相电表。
- **手工 CLI 试读**`scripts/modbus_cli``read`/`probe` 两个只读子命令),供受控手工验证,不进自动化。
- **MQTT + HA Discovery**paho-mqtt 长连接;通用 expose 框架(provider 动态产出可暴露实体目录,元数据从 YAML profile 派生);`exposed_entity_toggle` 表存逐 key 开关(默认不暴露);每设备 = 一个 HA device,各量 = sensor entity + online binary_sensor`unique_id` 锚定于设备 `uuid`(稳定,不随改名变);配置变更可重连重发 discovery。
- **前端侧边栏**`AppShell` 侧边导航(Home / Records / Energy / Config + 主题切换 + 注销),移动端可折叠,当前路由高亮。
- **Energy 视图**`/energy`):设备 CRUD;最新读数卡片(标签/单位取自 profile metrics 端点);Recharts 走势图(时间范围 + limit)。
- **Config 页 Accordion**:各大 section 折叠/展开;「Home Assistant Expose」面板勾选可暴露实体 + 连接状态 + 重新发布按钮。
- **API**`/api/modbus/*` + `/api/expose` + `/api/config/mqtt/test`):完整 CRUD + readings + metrics + test + expose 勾选 + 重发 discovery。
- `pytest`/`ruff`/`export_openapi` + 前端 `lint/typecheck/test/build` 全绿,`openapi/` 已入库。
### 命名决策(已锁定)
存储/采集/API 全部使用通用 `modbus_*``/api/modbus/devices`),**不锁死"电表"**;面向用户的领域呈现叫 **Energy**。接入新 Modbus 设备型号只需新增 YAML profile,无需改表/改 API。
> 详细设计与任务卡:[`docs/design/m5-iot-energy.md`](./design/m5-iot-energy.md)
---
## M6 — 通用电价层 + DSMR 实时数据 + 实时买卖电费计算(✅ 已完成)
### 目标
在 M5 IoT 基建之上,把"电价合同"与"DSMR 实时电表数据"接起来,按 **15 分钟**周期算出**实际买电支出 / 卖电收入**,自己落库留底(不可变、可审计),并通过 MQTT + HA Discovery 反哺 Home Assistant 的 Energy 仪表盘。
### 关键能力
- **通用电价层(两层模型)**:仓库内 YAML profile 定结构(`manual` 固定/双费率、`tibber` 动态电价);`EnergyContract`(+ 版本)存 UI 可改的数值;price strategy 按 `kind` 出价。一次激活一个合同。改价 = 加新版本,旧版本保留(审计链)。
- **DSMR 实时数据接入**:扩 `MqttManager` 订阅 `dsmr/json`(每秒一帧),整帧 JSON blob 按 10 秒降采样落库(电、气、各相全存;字段 allowlist 不做,blob 天然容纳未来字段)。
- **实时买卖电费计算(两层)**:每 15 分钟用电表累计寄存器差值(`delivered_1/2``returned_1/2`)× 当时合同的买/卖价算计量电费(不可变、快照价)。买价:tibber = API `total`(含税全包),manual = `energy_buy_档 + energy_tax`;卖价:tibber = `total energy_tax sell_adjust`manual = `sell_档`(无能源税)。双费率:`_1`=dal/低、`_2`=normal/高(NL 惯例)。日/月/年汇总再加固定费(按月→天)减 heffingskorting(按年→天)。不做净计量。卖价残差(sell_adjust)及能源税等当前数值在真实账单确认后钉死。
- **反哺 Home Assistant**`_energy_cost_provider``buy_price_now``sell_price_now``import_cost_total``total_increasing`)、`export_revenue_total``total_increasing`)—— 直接挂 HA Energy 仪表盘。
- **Tibber 动态电价**httpx GraphQL 客户端抓 `QUARTER_HOURLY` 15 分钟价(已 demo 验证 `total=energy+tax`);启动 + 每小时抓取今明两天价格、幂等 upsert `tibber_price`hourly trigger,确保每日刷新且可补重试);仅当 active 合同 kind=tibber 且 token 存在时运行。
- **前端**:Energy 视图新增三个 Tab——Contracts(合同管理,表单按 profile 结构渲染)、Prices15 分钟价格曲线)、Costs(费用走势/明细/汇总卡片);Config 页 Tibber 测试三态入口。
### 命名分层(延续 M5
存储/采集/计算层中性命名(`dsmr_*` / `tibber_price` / `energy_cost_period` / `energy_contract`);面向用户并入既有 **Energy** 视图。
### 新增 5 张表(单库 app 链,migration `20260623_11_energy_tables`
| 表 | 关键列 | 说明 |
| --- | --- | --- |
| `dsmr_reading` | `recorded_at`(idx)、`source_id`(unique)、`payload`(JSON) | 整帧 DSMR telegram blob10 秒降采样 |
| `energy_contract` | `name``kind``active``currency`、时间戳 | 合同头;一次一个 active |
| `energy_contract_version` | `contract_id`(FK)、`effective_from``effective_to`(null)、`values`(JSON)、`created_at` | 版本/时段;改价加新版本、旧版本保留 |
| `tibber_price` | `starts_at`(unique)、`resolution``energy/tax/total``level``currency``fetched_at` | 15 分钟价缓存,不可变 |
| `energy_cost_period` | `period_start`(unique)、`d1/d2/r1/r2_kwh``import_cost/export_revenue/net_cost``currency``pricing`(JSON 快照)、`contract_version_id`(FK)、`degraded``computed_at` | 每 15 分钟计量电费,不可变(快照价+版本) |
### 不新增依赖
httpx / paho-mqtt / pyyaml / apscheduler 均为 M5 已有依赖,M6 复用,不新增任何 Python 包。
> 详细设计与任务卡:[`docs/design/m6-tibber-dynamic-energy.md`](./design/m6-tibber-dynamic-energy.md)
---
## M7 — 电表生命周期 / 换表归档(✅ 已完成)
### 目标
让计费系统正确处理**电表更换**这一必然事件:荷兰 2G 智能电表退网后,电网公司会把表换成 4G 表(同址换表);搬家继承新表也是同类。引入显式的 **Meter(电表)epoch** 概念,标记"某时刻起属于哪一块物理表",使引擎**永不跨表算 delta**,并让累计量按表归零、历史可追溯可查。
### 关键能力
- **Meter epoch 数据模型**:一条 `meter` 记录 = 一块物理电表的一段安装期 `[started_at, ended_at)`。换表 = 关闭旧 active 表(`ended_at=T`+ 新建 `started_at=T` 的表。每 `commodity` 至多一个 active 表;`commodity` 字段预留 `gas`/`heating`
- **计费永不跨表**`compute_period` 先查 t0/t1 两端的 `meter_at`;无表覆盖或跨表边界 → **降级**(成本置 0),绝不产负成本或巨额假成本。
- **delta 护栏**:任一寄存器 delta `< 0``> _MAX_DELTA_KWH`100 kWh/15min,远超住宅用量)→ 降级。兜底表重置 / DSMR 回绕 / 数据毛刺,与换表无关的异常也一并防住。
- **累计按当前表归零(D2**`expose.py` 累计 `import_cost_total`/`export_revenue_total` 锚点 = 当前 active electricity meter 的 `started_at`;换表后累计从零起新序列,不维护跨表偏移。
- **追溯换表可重算**`started_at` 可在过去;PATCH 修正日期后,触发受影响窗口 `recompute_range` 重判跨表周期归属。
- **Meter CRUD API + 前端管理 UI**`GET/POST/PATCH /api/energy/meters`+ 追溯 recompute);Energy 页新增 Meters tab,展示电表时间线,支持"换表/继承"表单与编辑。
### 新增表与列(单库 app 链,migration `20260625_13_meter_table`
| 表/列 | 关键设计 |
| --- | --- |
| `meter` | `id``label``commodity`(默认 `electricity`)、`started_at``ended_at`null=active)、`reason``initial`/`meter_swap`/`home_move`/`other`)、`note``created_at` |
| `energy_cost_period.meter_id` | nullable FK → `meter.id`;记录每周期归属,便于审计/按表查询/归档 |
迁移含回填:有历史数据时自动创建一条 `reason="initial"` 的初始表,并回填现有 `energy_cost_period.meter_id`;幂等 + 对账(非降级周期回填后 `meter_id IS NULL` 数必须为 0)。
### 已锁定决策摘要
- **D1**Meter 不绑定 home`label` 编址 + `commodity` 区分品类。
- **D2**:换表后累计量归零(锚当前表起点),不维护跨表偏移。
- **D3**:合同与电表是独立时间线,合同不引用电表;同址换表时合同自动沿用。
- **D4**`started_at` 可在过去;有效计费起点 = `max(started_at, 数据起点)`
- **D5**:跨表边界周期判 degraded(换表常伴随长时间无数据,可接受)。
- **D6**:负/异常大 delta → degraded,通用兜底。
- **D7**gas/heating 计费不在本里程碑;`commodity` 字段为未来扩展预留。
**已知行为(可接受)**:累计 `_total``state_class: total`,换表归零时 HA 长期统计可能在换表那一刻记一次性负 blip。已与用户确认先这样、观察后按需处理(`last_reset` 信号见设计文档 §9 留痕,本里程碑不做)。
> 详细设计与任务卡:[`docs/design/m7-meter-epochs-archival.md`](./design/m7-meter-epochs-archival.md)
>
> 模块概念说明:[`docs/meter-epochs.md`](./meter-epochs.md)
---
## M3 — 开放与移动端(远期试水)
### 目标
@@ -149,22 +274,28 @@
- 移动端是这一阶段最远期、最不确定的部分。
- token 主要是移动端的前置条件;Web 端 React 用现有 session cookie 即可,不需要为它提前引入 token。
## Future Ideas(暂不排期,想到先记下
## 下一阶段:已确定要做(尚未拆解为任务卡
> 这里收集**还没排进里程碑**的想法。不是承诺、也没有先后顺序;想做时再从这里捞出来细化成 `docs/design/` 的任务卡。**明确不开 M2.5**——下列条目一律先躺在 Future Ideas,之后再说
> 这些是 M5 之后**已经定下来要做**的方向——区别于下面的 Future Ideas(仅备忘、未必做)。这里只记到 roadmap 粒度:确定**做什么、为什么**;具体排期、依赖与原子任务,等动手时再展开成 `docs/design/` 的任务卡。**先后顺序未定**,具体排期等动手时再定
### TOTP 二次验证(Dashboard 加固)
### 1. 前端优化
**动机**M2 之后多了一个 Web Dashboard。它虽有单 admin 密码保护,但**大概率会暴露在公网**上,只靠密码这一层不够。给登录再叠一层 **TOTP(基于时间的一次性密码,RFC 6238)** 作为第二因子,做纵深防御
**动机**M2 的 React SPA 先把功能跑通,性能 / 体验层面的打磨还没做。这一项**确定要做,但具体优化什么还没定**
**范围(待定)**:方向先留空,想清楚再细化。可能的候选(仅占位、非承诺):打包体积与代码分割(M2/M5 构建已提示存在 > 500 kB 的单 chunk)、首屏加载、热力图 / 地图的渲染性能、移动端适配、可访问性等。等确定具体目标后再拆任务卡。
### 2. 设置页生成 Long-lived Token(供 API 调用)
**动机**:浏览器端走 session cookie 即可,但**脚本 / 设备 / 外部程序调用 API** 需要一种长期有效、可随身携带的凭据。在设置页加一组功能,由 admin **手动签发 long-lived token**,之后用它来调 API。
**范围(粗略,待细化)**
- 在现有单 adminArgon2 + server-side session)登录之上,叠加 TOTP 第二步:密码校验通过后再验 6 位动态码,通过才发 session cookie
- 首次启用时生成 TOTP secret,给出可导入 Authenticator 的二维码 / 可手输密钥;同时生成一组一次性**恢复码(recovery codes**
- 设置页新增「API Token」区:生成 / 命名 / 吊销 long-lived token;明文只在**生成时展示一次**,此后只存哈希
- 后端支持用该 token 鉴权访问 API(与现有 session cookie 并存,互不影响)
- 与 [M3](#m3--开放与移动端远期试水) 的 token 主题相关,但**这条是 Web 设置页手动签发的 PAT 风格**,不依赖移动端 OAuth 流程;两者实现时可复用同一套 token 存储 / 校验。
**运维 / 命令行要求(关键,实现时必须满足)**:
## Future Ideas(暂不排期,想到先记下)
1. **忘记密码**:不需要任何 Web 端“找回密码”流程——直接在命令行里重置 admin 密码即可(沿用现有 CLI 思路)
2. **TOTP 重置 / 恢复**:必须提供**命令行重置入口**。要覆盖最坏情况——**连恢复码(restore key)都丢了**,也能纯靠 CLI 把 TOTP 关掉 / 重新发放新的 secret,从而恢复登录。即:**CLI 是不依赖任何已存恢复凭据的最终逃生通道**,不能出现“密钥丢了就彻底锁死”的死角。
> 这里收集**还没排进里程碑、也还没决定要不要做**的想法。不是承诺、也没有先后顺序;想做时再从这里捞出来——先升进上面的「下一阶段」,再细化成 `docs/design/` 的任务卡
**先不做**:本条仅记入 Future Ideas,不进 M2.5、不排期;之后再细化为 design 任务卡。
_(暂无条目。)_
+396 -3
View File
@@ -17,11 +17,13 @@
"leaflet.heat": "^0.2.0",
"leaflet.markercluster": "^1.5.3",
"openapi-fetch": "^0.17.0",
"qrcode.react": "^4.2.0",
"react": "^18.3.1",
"react-dom": "^18.3.1",
"react-feather": "^2.0.10",
"react-leaflet": "^4.2.1",
"react-router-dom": "^6.30.4"
"react-router-dom": "^6.30.4",
"recharts": "^3.8.1"
},
"devDependencies": {
"@eslint/js": "^9.39.4",
@@ -1445,6 +1447,42 @@
"node": ">=10"
}
},
"node_modules/@reduxjs/toolkit": {
"version": "2.12.0",
"resolved": "https://registry.npmjs.org/@reduxjs/toolkit/-/toolkit-2.12.0.tgz",
"integrity": "sha512-KiT+RzZbp6mQET+Mg+h2c97+9j1sNflUxQkIHI7Yuzf6Peu+OYpmkn6nbHWmLLWj+1ZODUJFwGZ7gx3L9R9EOw==",
"license": "MIT",
"dependencies": {
"@standard-schema/spec": "^1.0.0",
"@standard-schema/utils": "^0.3.0",
"immer": "^11.0.0",
"redux": "^5.0.1",
"redux-thunk": "^3.1.0",
"reselect": "^5.1.0"
},
"peerDependencies": {
"react": "^16.9.0 || ^17.0.0 || ^18 || ^19",
"react-redux": "^7.2.1 || ^8.1.3 || ^9.0.0"
},
"peerDependenciesMeta": {
"react": {
"optional": true
},
"react-redux": {
"optional": true
}
}
},
"node_modules/@reduxjs/toolkit/node_modules/immer": {
"version": "11.1.8",
"resolved": "https://registry.npmjs.org/immer/-/immer-11.1.8.tgz",
"integrity": "sha512-/tbkHMW7y10Lx6i1crLjD4/OhNkRG+Fo7byZHtah0547nIeXYcpIXaUh0IAQY6gO5459qpGGYapcEOHtFXkIuA==",
"license": "MIT",
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/immer"
}
},
"node_modules/@remix-run/router": {
"version": "1.23.3",
"resolved": "https://registry.npmjs.org/@remix-run/router/-/router-1.23.3.tgz",
@@ -1854,7 +1892,12 @@
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/@standard-schema/spec/-/spec-1.1.0.tgz",
"integrity": "sha512-l2aFy5jALhniG5HgqrD6jXLi/rUWrKvqN/qJx6yoJsgKhblVd+iqqU4RCXavm/jPityDo5TCvKMnpjKnOriy0w==",
"dev": true,
"license": "MIT"
},
"node_modules/@standard-schema/utils": {
"version": "0.3.0",
"resolved": "https://registry.npmjs.org/@standard-schema/utils/-/utils-0.3.0.tgz",
"integrity": "sha512-e7Mew686owMaPJVNNLs55PUvgz371nKgwsc4vxE49zsODpJEnxgxRo2y/OKrqueavXgZNMDVj3DdHFlaSAeU8g==",
"license": "MIT"
},
"node_modules/@tanstack/query-core": {
@@ -2057,6 +2100,69 @@
"assertion-error": "^2.0.1"
}
},
"node_modules/@types/d3-array": {
"version": "3.2.2",
"resolved": "https://registry.npmjs.org/@types/d3-array/-/d3-array-3.2.2.tgz",
"integrity": "sha512-hOLWVbm7uRza0BYXpIIW5pxfrKe0W+D5lrFiAEYR+pb6w3N2SwSMaJbXdUfSEv+dT4MfHBLtn5js0LAWaO6otw==",
"license": "MIT"
},
"node_modules/@types/d3-color": {
"version": "3.1.3",
"resolved": "https://registry.npmjs.org/@types/d3-color/-/d3-color-3.1.3.tgz",
"integrity": "sha512-iO90scth9WAbmgv7ogoq57O9YpKmFBbmoEoCHDB2xMBY0+/KVrqAaCDyCE16dUspeOvIxFFRI+0sEtqDqy2b4A==",
"license": "MIT"
},
"node_modules/@types/d3-ease": {
"version": "3.0.2",
"resolved": "https://registry.npmjs.org/@types/d3-ease/-/d3-ease-3.0.2.tgz",
"integrity": "sha512-NcV1JjO5oDzoK26oMzbILE6HW7uVXOHLQvHshBUW4UMdZGfiY6v5BeQwh9a9tCzv+CeefZQHJt5SRgK154RtiA==",
"license": "MIT"
},
"node_modules/@types/d3-interpolate": {
"version": "3.0.4",
"resolved": "https://registry.npmjs.org/@types/d3-interpolate/-/d3-interpolate-3.0.4.tgz",
"integrity": "sha512-mgLPETlrpVV1YRJIglr4Ez47g7Yxjl1lj7YKsiMCb27VJH9W8NVM6Bb9d8kkpG/uAQS5AmbA48q2IAolKKo1MA==",
"license": "MIT",
"dependencies": {
"@types/d3-color": "*"
}
},
"node_modules/@types/d3-path": {
"version": "3.1.1",
"resolved": "https://registry.npmjs.org/@types/d3-path/-/d3-path-3.1.1.tgz",
"integrity": "sha512-VMZBYyQvbGmWyWVea0EHs/BwLgxc+MKi1zLDCONksozI4YJMcTt8ZEuIR4Sb1MMTE8MMW49v0IwI5+b7RmfWlg==",
"license": "MIT"
},
"node_modules/@types/d3-scale": {
"version": "4.0.9",
"resolved": "https://registry.npmjs.org/@types/d3-scale/-/d3-scale-4.0.9.tgz",
"integrity": "sha512-dLmtwB8zkAeO/juAMfnV+sItKjlsw2lKdZVVy6LRr0cBmegxSABiLEpGVmSJJ8O08i4+sGR6qQtb6WtuwJdvVw==",
"license": "MIT",
"dependencies": {
"@types/d3-time": "*"
}
},
"node_modules/@types/d3-shape": {
"version": "3.1.8",
"resolved": "https://registry.npmjs.org/@types/d3-shape/-/d3-shape-3.1.8.tgz",
"integrity": "sha512-lae0iWfcDeR7qt7rA88BNiqdvPS5pFVPpo5OfjElwNaT2yyekbM0C9vK+yqBqEmHr6lDkRnYNoTBYlAgJa7a4w==",
"license": "MIT",
"dependencies": {
"@types/d3-path": "*"
}
},
"node_modules/@types/d3-time": {
"version": "3.0.4",
"resolved": "https://registry.npmjs.org/@types/d3-time/-/d3-time-3.0.4.tgz",
"integrity": "sha512-yuzZug1nkAAaBlBBikKZTgzCeA+k1uy4ZFwWANOfKw5z5LRhV0gNA7gNkKm7HoK+HRN0wX3EkxGk0fpbWhmB7g==",
"license": "MIT"
},
"node_modules/@types/d3-timer": {
"version": "3.0.2",
"resolved": "https://registry.npmjs.org/@types/d3-timer/-/d3-timer-3.0.2.tgz",
"integrity": "sha512-Ps3T8E8dZDam6fUyNiMkekK3XUsaUEik+idO9/YjPtfj2qruF8tFBXS7XhtE4iIXBLxhmLjP3SXpLhVf21I9Lw==",
"license": "MIT"
},
"node_modules/@types/deep-eql": {
"version": "4.0.2",
"resolved": "https://registry.npmjs.org/@types/deep-eql/-/deep-eql-4.0.2.tgz",
@@ -2130,6 +2236,12 @@
"@types/react": "^18.0.0"
}
},
"node_modules/@types/use-sync-external-store": {
"version": "0.0.6",
"resolved": "https://registry.npmjs.org/@types/use-sync-external-store/-/use-sync-external-store-0.0.6.tgz",
"integrity": "sha512-zFDAD+tlpf2r4asuHEj0XH6pY6i0g5NeAHPn+15wk3BV6JA69eERFXC1gyGThDkVa1zCyKr5jox1+2LbV/AMLg==",
"license": "MIT"
},
"node_modules/@typescript-eslint/eslint-plugin": {
"version": "8.61.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/eslint-plugin/-/eslint-plugin-8.61.0.tgz",
@@ -3119,6 +3231,127 @@
"devOptional": true,
"license": "MIT"
},
"node_modules/d3-array": {
"version": "3.2.4",
"resolved": "https://registry.npmjs.org/d3-array/-/d3-array-3.2.4.tgz",
"integrity": "sha512-tdQAmyA18i4J7wprpYq8ClcxZy3SC31QMeByyCFyRt7BVHdREQZ5lpzoe5mFEYZUWe+oq8HBvk9JjpibyEV4Jg==",
"license": "ISC",
"dependencies": {
"internmap": "1 - 2"
},
"engines": {
"node": ">=12"
}
},
"node_modules/d3-color": {
"version": "3.1.0",
"resolved": "https://registry.npmjs.org/d3-color/-/d3-color-3.1.0.tgz",
"integrity": "sha512-zg/chbXyeBtMQ1LbD/WSoW2DpC3I0mpmPdW+ynRTj/x2DAWYrIY7qeZIHidozwV24m4iavr15lNwIwLxRmOxhA==",
"license": "ISC",
"engines": {
"node": ">=12"
}
},
"node_modules/d3-ease": {
"version": "3.0.1",
"resolved": "https://registry.npmjs.org/d3-ease/-/d3-ease-3.0.1.tgz",
"integrity": "sha512-wR/XK3D3XcLIZwpbvQwQ5fK+8Ykds1ip7A2Txe0yxncXSdq1L9skcG7blcedkOX+ZcgxGAmLX1FrRGbADwzi0w==",
"license": "BSD-3-Clause",
"engines": {
"node": ">=12"
}
},
"node_modules/d3-format": {
"version": "3.1.2",
"resolved": "https://registry.npmjs.org/d3-format/-/d3-format-3.1.2.tgz",
"integrity": "sha512-AJDdYOdnyRDV5b6ArilzCPPwc1ejkHcoyFarqlPqT7zRYjhavcT3uSrqcMvsgh2CgoPbK3RCwyHaVyxYcP2Arg==",
"license": "ISC",
"engines": {
"node": ">=12"
}
},
"node_modules/d3-interpolate": {
"version": "3.0.1",
"resolved": "https://registry.npmjs.org/d3-interpolate/-/d3-interpolate-3.0.1.tgz",
"integrity": "sha512-3bYs1rOD33uo8aqJfKP3JWPAibgw8Zm2+L9vBKEHJ2Rg+viTR7o5Mmv5mZcieN+FRYaAOWX5SJATX6k1PWz72g==",
"license": "ISC",
"dependencies": {
"d3-color": "1 - 3"
},
"engines": {
"node": ">=12"
}
},
"node_modules/d3-path": {
"version": "3.1.0",
"resolved": "https://registry.npmjs.org/d3-path/-/d3-path-3.1.0.tgz",
"integrity": "sha512-p3KP5HCf/bvjBSSKuXid6Zqijx7wIfNW+J/maPs+iwR35at5JCbLUT0LzF1cnjbCHWhqzQTIN2Jpe8pRebIEFQ==",
"license": "ISC",
"engines": {
"node": ">=12"
}
},
"node_modules/d3-scale": {
"version": "4.0.2",
"resolved": "https://registry.npmjs.org/d3-scale/-/d3-scale-4.0.2.tgz",
"integrity": "sha512-GZW464g1SH7ag3Y7hXjf8RoUuAFIqklOAq3MRl4OaWabTFJY9PN/E1YklhXLh+OQ3fM9yS2nOkCoS+WLZ6kvxQ==",
"license": "ISC",
"dependencies": {
"d3-array": "2.10.0 - 3",
"d3-format": "1 - 3",
"d3-interpolate": "1.2.0 - 3",
"d3-time": "2.1.1 - 3",
"d3-time-format": "2 - 4"
},
"engines": {
"node": ">=12"
}
},
"node_modules/d3-shape": {
"version": "3.2.0",
"resolved": "https://registry.npmjs.org/d3-shape/-/d3-shape-3.2.0.tgz",
"integrity": "sha512-SaLBuwGm3MOViRq2ABk3eLoxwZELpH6zhl3FbAoJ7Vm1gofKx6El1Ib5z23NUEhF9AsGl7y+dzLe5Cw2AArGTA==",
"license": "ISC",
"dependencies": {
"d3-path": "^3.1.0"
},
"engines": {
"node": ">=12"
}
},
"node_modules/d3-time": {
"version": "3.1.0",
"resolved": "https://registry.npmjs.org/d3-time/-/d3-time-3.1.0.tgz",
"integrity": "sha512-VqKjzBLejbSMT4IgbmVgDjpkYrNWUYJnbCGo874u7MMKIWsILRX+OpX/gTk8MqjpT1A/c6HY2dCA77ZN0lkQ2Q==",
"license": "ISC",
"dependencies": {
"d3-array": "2 - 3"
},
"engines": {
"node": ">=12"
}
},
"node_modules/d3-time-format": {
"version": "4.1.0",
"resolved": "https://registry.npmjs.org/d3-time-format/-/d3-time-format-4.1.0.tgz",
"integrity": "sha512-dJxPBlzC7NugB2PDLwo9Q8JiTR3M3e4/XANkreKSUxF8vvXKqm1Yfq4Q5dl8budlunRVlUUaDUgFt7eA8D6NLg==",
"license": "ISC",
"dependencies": {
"d3-time": "1 - 3"
},
"engines": {
"node": ">=12"
}
},
"node_modules/d3-timer": {
"version": "3.0.1",
"resolved": "https://registry.npmjs.org/d3-timer/-/d3-timer-3.0.1.tgz",
"integrity": "sha512-ndfJ/JxxMd3nw31uyKoY2naivF+r29V+Lc0svZxe1JvvIRmi8hUsrMvdOwgS1o6uBHmiz91geQ0ylPP0aj1VUA==",
"license": "ISC",
"engines": {
"node": ">=12"
}
},
"node_modules/data-urls": {
"version": "7.0.0",
"resolved": "https://registry.npmjs.org/data-urls/-/data-urls-7.0.0.tgz",
@@ -3212,6 +3445,12 @@
"dev": true,
"license": "MIT"
},
"node_modules/decimal.js-light": {
"version": "2.5.1",
"resolved": "https://registry.npmjs.org/decimal.js-light/-/decimal.js-light-2.5.1.tgz",
"integrity": "sha512-qIMFpTMZmny+MMIitAB6D7iVPEorVw6YQRWkvarTkT4tBeSLLiHzcwj6q0MmYSFCiVpiqPJTJEYIrpcPzVEIvg==",
"license": "MIT"
},
"node_modules/deep-equal": {
"version": "2.2.3",
"resolved": "https://registry.npmjs.org/deep-equal/-/deep-equal-2.2.3.tgz",
@@ -3564,6 +3803,16 @@
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/es-toolkit": {
"version": "1.48.1",
"resolved": "https://registry.npmjs.org/es-toolkit/-/es-toolkit-1.48.1.tgz",
"integrity": "sha512-wfnXlwd5I75eXRtdD2vuEs50xHHESECDsGD7yiQnfFVNoa5522NwXEbmgo98LfiukSQHs+mBM7/YG3qKJB9/mQ==",
"license": "MIT",
"workspaces": [
"docs",
"benchmarks"
]
},
"node_modules/esbuild": {
"version": "0.25.12",
"resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.25.12.tgz",
@@ -3846,6 +4095,12 @@
"node": ">=0.10.0"
}
},
"node_modules/eventemitter3": {
"version": "5.0.4",
"resolved": "https://registry.npmjs.org/eventemitter3/-/eventemitter3-5.0.4.tgz",
"integrity": "sha512-mlsTRyGaPBjPedk6Bvw+aqbsXDtoAyAzm5MO7JgU+yVRyMQ5O8bD4Kcci7BS85f93veegeCPkL8R4GLClnjLFw==",
"license": "MIT"
},
"node_modules/expect-type": {
"version": "1.3.0",
"resolved": "https://registry.npmjs.org/expect-type/-/expect-type-1.3.0.tgz",
@@ -4311,6 +4566,16 @@
"node": ">= 4"
}
},
"node_modules/immer": {
"version": "10.2.0",
"resolved": "https://registry.npmjs.org/immer/-/immer-10.2.0.tgz",
"integrity": "sha512-d/+XTN3zfODyjr89gM3mPq1WNX2B8pYsu7eORitdwyA2sBubnTl3laYlBk4sXY5FUa5qTZGBDPJICVbvqzjlbw==",
"license": "MIT",
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/immer"
}
},
"node_modules/import-fresh": {
"version": "3.3.1",
"resolved": "https://registry.npmjs.org/import-fresh/-/import-fresh-3.3.1.tgz",
@@ -4376,6 +4641,15 @@
"node": ">= 0.4"
}
},
"node_modules/internmap": {
"version": "2.0.3",
"resolved": "https://registry.npmjs.org/internmap/-/internmap-2.0.3.tgz",
"integrity": "sha512-5Hh7Y1wQbvY5ooGgPbDaL5iYLAPzMTUrjMulskHLH6wnv/A+1q5rgEaiuqEjB+oxGXIVZs1FF+R/KPN3ZSQYYg==",
"license": "ISC",
"engines": {
"node": ">=12"
}
},
"node_modules/is-arguments": {
"version": "1.2.0",
"resolved": "https://registry.npmjs.org/is-arguments/-/is-arguments-1.2.0.tgz",
@@ -5661,6 +5935,15 @@
"node": ">=6"
}
},
"node_modules/qrcode.react": {
"version": "4.2.0",
"resolved": "https://registry.npmjs.org/qrcode.react/-/qrcode.react-4.2.0.tgz",
"integrity": "sha512-QpgqWi8rD9DsS9EP3z7BT+5lY5SFhsqGjpgW5DY/i3mK4M9DTBNz3ErMi8BWYEfI3L0d8GIbGmcdFAS1uIRGjA==",
"license": "ISC",
"peerDependencies": {
"react": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0"
}
},
"node_modules/react": {
"version": "18.3.1",
"resolved": "https://registry.npmjs.org/react/-/react-18.3.1.tgz",
@@ -5702,7 +5985,6 @@
"version": "17.0.2",
"resolved": "https://registry.npmjs.org/react-is/-/react-is-17.0.2.tgz",
"integrity": "sha512-w2GsyukL62IJnlaff/nRegPQR94C/XXamvMWmSHRJ4y7Ts/4ocGRmTHvOs8PSE6pB3dWOrD/nueuU5sduBsQ4w==",
"dev": true,
"license": "MIT"
},
"node_modules/react-leaflet": {
@@ -5729,6 +6011,29 @@
"react-dom": "^0.14 || ^15.0.0 || ^16.0.0 || ^17.0.0 || ^18.0.0 || ^19.0.0"
}
},
"node_modules/react-redux": {
"version": "9.3.0",
"resolved": "https://registry.npmjs.org/react-redux/-/react-redux-9.3.0.tgz",
"integrity": "sha512-KQopgqFo/p/fgmAs5qz6p5RWaNAzq40WAu7fJIXnQpYxFPbJYtsJPWvGeF2rOBaY/kEuV77AVsX8TsQzKm+A/g==",
"license": "MIT",
"dependencies": {
"@types/use-sync-external-store": "^0.0.6",
"use-sync-external-store": "^1.4.0"
},
"peerDependencies": {
"@types/react": "^18.2.25 || ^19",
"react": "^18.0 || ^19",
"redux": "^5.0.0"
},
"peerDependenciesMeta": {
"@types/react": {
"optional": true
},
"redux": {
"optional": true
}
}
},
"node_modules/react-refresh": {
"version": "0.17.0",
"resolved": "https://registry.npmjs.org/react-refresh/-/react-refresh-0.17.0.tgz",
@@ -5857,6 +6162,36 @@
"react": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0"
}
},
"node_modules/recharts": {
"version": "3.8.1",
"resolved": "https://registry.npmjs.org/recharts/-/recharts-3.8.1.tgz",
"integrity": "sha512-mwzmO1s9sFL0TduUpwndxCUNoXsBw3u3E/0+A+cLcrSfQitSG62L32N69GhqUrrT5qKcAE3pCGVINC6pqkBBQg==",
"license": "MIT",
"workspaces": [
"www"
],
"dependencies": {
"@reduxjs/toolkit": "^1.9.0 || 2.x.x",
"clsx": "^2.1.1",
"decimal.js-light": "^2.5.1",
"es-toolkit": "^1.39.3",
"eventemitter3": "^5.0.1",
"immer": "^10.1.1",
"react-redux": "8.x.x || 9.x.x",
"reselect": "5.1.1",
"tiny-invariant": "^1.3.3",
"use-sync-external-store": "^1.2.2",
"victory-vendor": "^37.0.2"
},
"engines": {
"node": ">=18"
},
"peerDependencies": {
"react": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0",
"react-dom": "^16.0.0 || ^17.0.0 || ^18.0.0 || ^19.0.0",
"react-is": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0"
}
},
"node_modules/redent": {
"version": "3.0.0",
"resolved": "https://registry.npmjs.org/redent/-/redent-3.0.0.tgz",
@@ -5871,6 +6206,21 @@
"node": ">=8"
}
},
"node_modules/redux": {
"version": "5.0.1",
"resolved": "https://registry.npmjs.org/redux/-/redux-5.0.1.tgz",
"integrity": "sha512-M9/ELqF6fy8FwmkpnF0S3YKOqMyoWJ4+CS5Efg2ct3oY9daQvd/Pc71FpGZsVsbl3Cpb+IIcjBDUnnyBdQbq4w==",
"license": "MIT"
},
"node_modules/redux-thunk": {
"version": "3.1.0",
"resolved": "https://registry.npmjs.org/redux-thunk/-/redux-thunk-3.1.0.tgz",
"integrity": "sha512-NW2r5T6ksUKXCabzhL9z+h206HQw/NJkcLm1GPImRQ8IzfXwRGqjVhKJGauHirT0DAuyy6hjdnMZaRoAcy0Klw==",
"license": "MIT",
"peerDependencies": {
"redux": "^5.0.0"
}
},
"node_modules/reflect.getprototypeof": {
"version": "1.0.10",
"resolved": "https://registry.npmjs.org/reflect.getprototypeof/-/reflect.getprototypeof-1.0.10.tgz",
@@ -5925,6 +6275,12 @@
"node": ">=0.10.0"
}
},
"node_modules/reselect": {
"version": "5.1.1",
"resolved": "https://registry.npmjs.org/reselect/-/reselect-5.1.1.tgz",
"integrity": "sha512-K/BG6eIky/SBpzfHZv/dd+9JBFiS4SWV7FIujVyJRux6e45+73RaUHXLmIR1f7WOMaQ0U1km6qwklRQxpJJY0w==",
"license": "MIT"
},
"node_modules/resolve": {
"version": "2.0.0-next.7",
"resolved": "https://registry.npmjs.org/resolve/-/resolve-2.0.0-next.7.tgz",
@@ -6448,6 +6804,12 @@
"integrity": "sha512-05PUHKSNE8ou2dwIxTngl4EzcnsCDZGJ/iCLtDflR/SHB/ny14rXc+qU5P4mG9JkusiV7EivzY9Mhm55AzAvCg==",
"license": "MIT"
},
"node_modules/tiny-invariant": {
"version": "1.3.3",
"resolved": "https://registry.npmjs.org/tiny-invariant/-/tiny-invariant-1.3.3.tgz",
"integrity": "sha512-+FbBPE1o9QAYvviau/qC5SE3caw21q3xkvWKBtja5vgqOWIHHJ3ioaq1VPfn/Szqctz2bU/oYeKd9/z5BL+PVg==",
"license": "MIT"
},
"node_modules/tinybench": {
"version": "2.9.0",
"resolved": "https://registry.npmjs.org/tinybench/-/tinybench-2.9.0.tgz",
@@ -6863,6 +7225,37 @@
}
}
},
"node_modules/use-sync-external-store": {
"version": "1.6.0",
"resolved": "https://registry.npmjs.org/use-sync-external-store/-/use-sync-external-store-1.6.0.tgz",
"integrity": "sha512-Pp6GSwGP/NrPIrxVFAIkOQeyw8lFenOHijQWkUTrDvrF4ALqylP2C/KCkeS9dpUM3KvYRQhna5vt7IL95+ZQ9w==",
"license": "MIT",
"peerDependencies": {
"react": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0"
}
},
"node_modules/victory-vendor": {
"version": "37.3.6",
"resolved": "https://registry.npmjs.org/victory-vendor/-/victory-vendor-37.3.6.tgz",
"integrity": "sha512-SbPDPdDBYp+5MJHhBCAyI7wKM3d5ivekigc2Dk2s7pgbZ9wIgIBYGVw4zGHBml/qTFbexrofXW6Gu4noGxrOwQ==",
"license": "MIT AND ISC",
"dependencies": {
"@types/d3-array": "^3.0.3",
"@types/d3-ease": "^3.0.0",
"@types/d3-interpolate": "^3.0.1",
"@types/d3-scale": "^4.0.2",
"@types/d3-shape": "^3.1.0",
"@types/d3-time": "^3.0.0",
"@types/d3-timer": "^3.0.0",
"d3-array": "^3.1.6",
"d3-ease": "^3.0.1",
"d3-interpolate": "^3.0.1",
"d3-scale": "^4.0.2",
"d3-shape": "^3.1.0",
"d3-time": "^3.0.0",
"d3-timer": "^3.0.1"
}
},
"node_modules/vite": {
"version": "6.4.3",
"resolved": "https://registry.npmjs.org/vite/-/vite-6.4.3.tgz",
+3 -1
View File
@@ -22,11 +22,13 @@
"leaflet.heat": "^0.2.0",
"leaflet.markercluster": "^1.5.3",
"openapi-fetch": "^0.17.0",
"qrcode.react": "^4.2.0",
"react": "^18.3.1",
"react-dom": "^18.3.1",
"react-feather": "^2.0.10",
"react-leaflet": "^4.2.1",
"react-router-dom": "^6.30.4"
"react-router-dom": "^6.30.4",
"recharts": "^3.8.1"
},
"devDependencies": {
"@eslint/js": "^9.39.4",
+41 -122
View File
@@ -6,24 +6,20 @@
*
* Route tree:
* /login LoginPage (public)
* /change-password ProtectedRoute ChangePasswordPage (T07: forced password change gate)
* / ProtectedRoute AppLayout HomePage (T09)
* /config ProtectedRoute AppLayout ConfigPage (T08)
* /change-password ProtectedRoute ChangePasswordPage (forced password change gate)
* / ProtectedRoute AppLayout HomePage
* /config ProtectedRoute AppLayout ConfigPage
* /records ProtectedRoute AppLayout RecordsPage
* /energy ProtectedRoute AppLayout EnergyPage
*
* AppLayout renders a nav with a gear-icon entry for /config and a logout button (T07).
* AppLayout renders a sidebar (AppSidebar) on the left; page content via <Outlet/> on the right.
* /login and /change-password have no sidebar layout.
*/
import { useState } from 'react'
import { QueryClient, QueryClientProvider } from '@tanstack/react-query'
import { BrowserRouter, Routes, Route, Link, Outlet, useNavigate } from 'react-router-dom'
import {
MantineProvider,
Group,
ActionIcon,
Tooltip,
useMantineColorScheme,
useComputedColorScheme,
} from '@mantine/core'
import { List, Settings, Sun, Moon, LogOut } from 'react-feather'
import { BrowserRouter, Routes, Route, Outlet } from 'react-router-dom'
import { MantineProvider, AppShell, Burger } from '@mantine/core'
// Mantine requires its CSS to be imported once.
import '@mantine/core/styles.css'
@@ -34,9 +30,9 @@ import { LoginPage } from './pages/LoginPage'
import { HomePage } from './pages/HomePage'
import { ConfigPage } from './pages/ConfigPage'
import { RecordsPage } from './pages/RecordsPage'
import { EnergyPage } from './pages/EnergyPage'
import { ChangePasswordPage } from './pages/ChangePasswordPage'
import apiClient from './api/client'
import { useQueryClient } from '@tanstack/react-query'
import { AppSidebar } from './components/AppSidebar'
// ---------------------------------------------------------------------------
// TanStack Query client (singleton, created outside render to avoid re-creation)
@@ -57,120 +53,42 @@ const queryClient = new QueryClient({
},
})
// ---------------------------------------------------------------------------
// Logout button component (needs navigate + queryClient hooks, so it's a component)
// ---------------------------------------------------------------------------
function LogoutButton() {
const navigate = useNavigate()
const qc = useQueryClient()
async function handleLogout() {
try {
await apiClient.POST('/api/auth/logout')
} catch {
// Ignore errors on logout — we clear the session regardless.
}
// Invalidate session so SessionProvider becomes unauthenticated.
await qc.invalidateQueries({ queryKey: ['session'] })
navigate('/login', { replace: true })
}
return (
<Tooltip label="Log out">
<ActionIcon
variant="default"
size="lg"
onClick={handleLogout}
aria-label="Log out"
data-testid="logout-button"
>
<LogOut size={18} />
</ActionIcon>
</Tooltip>
)
}
// ---------------------------------------------------------------------------
// Dark-mode toggle (sits next to the gear / settings icon)
// ---------------------------------------------------------------------------
function ColorSchemeToggle() {
const { setColorScheme } = useMantineColorScheme()
const computed = useComputedColorScheme('light', { getInitialValueInEffect: true })
const isDark = computed === 'dark'
return (
<Tooltip label={isDark ? 'Light mode' : 'Dark mode'}>
<ActionIcon
variant="default"
size="lg"
aria-label="Toggle color scheme"
onClick={() => setColorScheme(isDark ? 'light' : 'dark')}
data-testid="color-scheme-toggle"
>
{isDark ? <Sun size={18} /> : <Moon size={18} />}
</ActionIcon>
</Tooltip>
)
}
// ---------------------------------------------------------------------------
// App shell layout (used by all protected pages)
// ---------------------------------------------------------------------------
function AppLayout() {
// Mobile navbar open/close state — controlled here so the Burger and
// AppShell share the same state.
const [navbarOpen, setNavbarOpen] = useState(false)
return (
<div style={{ minHeight: '100vh', display: 'flex', flexDirection: 'column' }}>
{/* Top nav */}
<nav
style={{
display: 'flex',
alignItems: 'center',
justifyContent: 'space-between',
padding: '0.5rem 1rem',
borderBottom: '1px solid var(--mantine-color-default-border)',
}}
>
<Link to="/" style={{ fontWeight: 600, textDecoration: 'none' }}>
Home Automation
</Link>
<AppShell
navbar={{
width: 220,
breakpoint: 'sm',
collapsed: { mobile: !navbarOpen },
}}
header={{ height: { base: 48, sm: 0 } }}
padding="md"
>
{/* Burger shown on mobile only (hidden on sm+) */}
<AppShell.Header hiddenFrom="sm" p="xs">
<Burger
opened={navbarOpen}
onClick={() => setNavbarOpen((o) => !o)}
size="sm"
aria-label="Toggle navigation"
data-testid="nav-burger"
/>
</AppShell.Header>
<Group gap="xs">
{/* Records nav link */}
<Tooltip label="Records">
<ActionIcon
component={Link}
to="/records"
variant="default"
size="lg"
aria-label="Records"
>
<List size={18} />
</ActionIcon>
</Tooltip>
{/* Dark-mode toggle — directly beside the settings gear */}
<ColorSchemeToggle />
{/* Settings — links to config page (§5#10) */}
<Tooltip label="Settings">
<ActionIcon
component={Link}
to="/config"
variant="default"
size="lg"
aria-label="Settings"
>
<Settings size={18} />
</ActionIcon>
</Tooltip>
<LogoutButton />
</Group>
</nav>
<AppSidebar onNavClick={() => setNavbarOpen(false)} />
{/* Page content */}
<main style={{ flex: 1 }}>
<AppShell.Main>
<Outlet />
</main>
</div>
</AppShell.Main>
</AppShell>
)
}
@@ -198,7 +116,7 @@ export default function App() {
}
/>
{/* Protected routes — all nested under AppLayout */}
{/* Protected routes — all nested under AppLayout (sidebar) */}
<Route
element={
<ProtectedRoute>
@@ -209,6 +127,7 @@ export default function App() {
<Route index element={<HomePage />} />
<Route path="/config" element={<ConfigPage />} />
<Route path="/records" element={<RecordsPage />} />
<Route path="/energy" element={<EnergyPage />} />
</Route>
</Routes>
</SessionProvider>
+2771
View File
File diff suppressed because it is too large Load Diff
+77
View File
@@ -0,0 +1,77 @@
/**
* TOTP API helpers (M4-T08).
*
* All calls go through the typed apiClient (openapi-fetch + generated schema).
* CSRF tokens are injected automatically by the client middleware for write
* endpoints; GET /api/auth/totp needs no CSRF (read-only).
*
* Security notes:
* - The TotpSetupResponse contains secret + recovery_codes in plaintext.
* These are kept only in component state; callers must NOT write them to
* localStorage, sessionStorage, or log them.
* - After the setup phase is over the data leaves memory when the component
* re-renders (one-time display only).
*/
import apiClient, { ApiError } from '../api/client'
import type { components } from '../api/schema.d.ts'
export type TotpSetupResponse = components['schemas']['TotpSetupResponse']
export type TotpStatusResponse = components['schemas']['TotpStatusResponse']
// ---------------------------------------------------------------------------
// getTotpStatus — GET /api/auth/totp
// ---------------------------------------------------------------------------
/** Return the current TOTP enabled state for the authenticated user. */
export async function getTotpStatus(): Promise<TotpStatusResponse> {
const res = await apiClient.GET('/api/auth/totp')
if (!res.data) {
throw new ApiError(res.response.status, null)
}
return res.data
}
// ---------------------------------------------------------------------------
// setupTotp — POST /api/auth/totp/setup
// ---------------------------------------------------------------------------
/**
* Generate a new pending TOTP secret + recovery codes.
* Returns the one-time plaintext values; caller is responsible for displaying
* them exactly once and never persisting them outside component state.
*/
export async function setupTotp(): Promise<TotpSetupResponse> {
const res = await apiClient.POST('/api/auth/totp/setup')
if (!res.data) {
throw new ApiError(res.response.status, null)
}
return res.data
}
// ---------------------------------------------------------------------------
// enableTotp — POST /api/auth/totp/enable
// ---------------------------------------------------------------------------
/** Confirm TOTP setup with the current 6-digit code from the authenticator app. */
export async function enableTotp(code: string): Promise<void> {
// The endpoint returns 204 on success; openapi-fetch yields { data: undefined }.
await apiClient.POST('/api/auth/totp/enable', { body: { code } })
}
// ---------------------------------------------------------------------------
// disableTotp — POST /api/auth/totp/disable
// ---------------------------------------------------------------------------
/**
* Disable TOTP. Exactly one of password or code must be provided.
* On success the backend clears the secret and all recovery codes.
*/
export async function disableTotp(opts: {
password?: string | null
code?: string | null
}): Promise<void> {
await apiClient.POST('/api/auth/totp/disable', {
body: { password: opts.password ?? null, code: opts.code ?? null },
})
}
+138
View File
@@ -0,0 +1,138 @@
/**
* Tests for AppSidebar (M5-T01, updated M5-T06).
*
* Strategy: render AppSidebar inside a minimal AppShell (required by
* AppShell.Navbar) + MemoryRouter so useLocation() works.
*
* Coverage:
* 1. All four nav items render (Home, Records, Energy, Config).
* 2. Home nav item is active when pathname is '/'.
* 3. Records nav item is active when pathname is '/records'.
* 4. Config nav item is active when pathname is '/config'.
* 5. Home is NOT active when on '/records'.
* 6. Energy nav item is active when on '/energy'.
*/
import { describe, it, expect, vi } from 'vitest'
import { screen } from '@testing-library/react'
import { render } from '@testing-library/react'
import { MantineProvider, AppShell } from '@mantine/core'
import { QueryClient, QueryClientProvider } from '@tanstack/react-query'
import { MemoryRouter } from 'react-router-dom'
import { AppSidebar } from './AppSidebar'
// ---------------------------------------------------------------------------
// Mock apiClient (LogoutButton calls POST /api/auth/logout)
// ---------------------------------------------------------------------------
vi.mock('../api/client', () => ({
default: {
POST: vi.fn(),
GET: vi.fn(),
},
ApiError: class ApiError extends Error {
status: number
body: unknown
constructor(status: number, body: unknown) {
super(`API error ${status}`)
this.name = 'ApiError'
this.status = status
this.body = body
}
},
registerLoginRedirect: vi.fn(),
}))
// ---------------------------------------------------------------------------
// Helper: render AppSidebar inside the required AppShell + router providers
// ---------------------------------------------------------------------------
function renderSidebar(initialPath = '/') {
const queryClient = new QueryClient({
defaultOptions: { queries: { retry: false }, mutations: { retry: false } },
})
return render(
<MantineProvider>
<QueryClientProvider client={queryClient}>
<MemoryRouter initialEntries={[initialPath]}>
<AppShell
navbar={{ width: 220, breakpoint: 'sm', collapsed: { mobile: false } }}
header={{ height: { base: 48, sm: 0 } }}
>
<AppSidebar />
</AppShell>
</MemoryRouter>
</QueryClientProvider>
</MantineProvider>,
)
}
// ---------------------------------------------------------------------------
// Tests
// ---------------------------------------------------------------------------
describe('AppSidebar', () => {
// -------------------------------------------------------------------------
// 1. All nav items render
// -------------------------------------------------------------------------
it('renders Home, Records, Energy, and Config nav items', () => {
renderSidebar('/')
expect(screen.getByTestId('nav-home')).toBeInTheDocument()
expect(screen.getByTestId('nav-records')).toBeInTheDocument()
expect(screen.getByTestId('nav-energy')).toBeInTheDocument()
expect(screen.getByTestId('nav-config')).toBeInTheDocument()
})
// -------------------------------------------------------------------------
// 2. Home active on '/'
// -------------------------------------------------------------------------
it('marks Home nav item as active when on "/"', () => {
renderSidebar('/')
// Mantine NavLink sets data-active="true" on the active item
const homeLink = screen.getByTestId('nav-home')
expect(homeLink).toHaveAttribute('data-active', 'true')
})
// -------------------------------------------------------------------------
// 3. Records active on '/records'
// -------------------------------------------------------------------------
it('marks Records nav item as active when on "/records"', () => {
renderSidebar('/records')
const recordsLink = screen.getByTestId('nav-records')
expect(recordsLink).toHaveAttribute('data-active', 'true')
})
// -------------------------------------------------------------------------
// 4. Config active on '/config'
// -------------------------------------------------------------------------
it('marks Config nav item as active when on "/config"', () => {
renderSidebar('/config')
const configLink = screen.getByTestId('nav-config')
expect(configLink).toHaveAttribute('data-active', 'true')
})
// -------------------------------------------------------------------------
// 5. Home NOT active on '/records' (exact match guard)
// -------------------------------------------------------------------------
it('does NOT mark Home as active when on "/records"', () => {
renderSidebar('/records')
const homeLink = screen.getByTestId('nav-home')
expect(homeLink).not.toHaveAttribute('data-active', 'true')
})
// -------------------------------------------------------------------------
// 6. Energy active on '/energy'
// -------------------------------------------------------------------------
it('marks Energy nav item as active when on "/energy"', () => {
renderSidebar('/energy')
const energyLink = screen.getByTestId('nav-energy')
expect(energyLink).toHaveAttribute('data-active', 'true')
})
})
+171
View File
@@ -0,0 +1,171 @@
/**
* AppSidebar vertical sidebar navigation for all protected pages.
*
* Nav items: Home / Records / Energy / Config
* Utilities: ColorSchemeToggle + LogoutButton (at bottom)
*
* Current route is highlighted via useLocation().
* Mobile: burger button toggles the navbar open/closed (handled by AppShell context).
*/
import { NavLink, Stack, Divider, Tooltip, ActionIcon, useMantineColorScheme, useComputedColorScheme, AppShell, Text, Group } from '@mantine/core'
import { Link, useLocation, useNavigate } from 'react-router-dom'
import { Home, List, Settings, Sun, Moon, LogOut, Zap } from 'react-feather'
import { useQueryClient } from '@tanstack/react-query'
import apiClient from '../api/client'
// ---------------------------------------------------------------------------
// Individual nav entries
// ---------------------------------------------------------------------------
interface NavEntry {
to: string
label: string
icon: React.ReactNode
testId: string
}
const NAV_ENTRIES: NavEntry[] = [
{
to: '/',
label: 'Home',
icon: <Home size={18} />,
testId: 'nav-home',
},
{
to: '/records',
label: 'Records',
icon: <List size={18} />,
testId: 'nav-records',
},
{
to: '/energy',
label: 'Energy',
icon: <Zap size={18} />,
testId: 'nav-energy',
},
{
to: '/config',
label: 'Config',
icon: <Settings size={18} />,
testId: 'nav-config',
},
]
// ---------------------------------------------------------------------------
// Colour-scheme toggle
// ---------------------------------------------------------------------------
function ColorSchemeToggle() {
const { setColorScheme } = useMantineColorScheme()
const computed = useComputedColorScheme('light', { getInitialValueInEffect: true })
const isDark = computed === 'dark'
return (
<Tooltip label={isDark ? 'Light mode' : 'Dark mode'} position="right">
<ActionIcon
variant="default"
size="lg"
aria-label="Toggle color scheme"
onClick={() => setColorScheme(isDark ? 'light' : 'dark')}
data-testid="color-scheme-toggle"
>
{isDark ? <Sun size={18} /> : <Moon size={18} />}
</ActionIcon>
</Tooltip>
)
}
// ---------------------------------------------------------------------------
// Logout button
// ---------------------------------------------------------------------------
function LogoutButton() {
const navigate = useNavigate()
const qc = useQueryClient()
async function handleLogout() {
try {
await apiClient.POST('/api/auth/logout')
} catch {
// Ignore errors — clear session regardless.
}
await qc.invalidateQueries({ queryKey: ['session'] })
navigate('/login', { replace: true })
}
return (
<Tooltip label="Log out" position="right">
<ActionIcon
variant="default"
size="lg"
onClick={handleLogout}
aria-label="Log out"
data-testid="logout-button"
>
<LogOut size={18} />
</ActionIcon>
</Tooltip>
)
}
// ---------------------------------------------------------------------------
// Sidebar
// ---------------------------------------------------------------------------
interface AppSidebarProps {
/** Called when a nav link is clicked on mobile (to close the navbar). */
onNavClick?: () => void
}
export function AppSidebar({ onNavClick }: AppSidebarProps) {
const location = useLocation()
return (
<AppShell.Navbar p="xs">
{/* App title */}
<AppShell.Section>
<Group px="xs" py="sm">
<Text fw={700} size="sm" component={Link} to="/" style={{ textDecoration: 'none' }}>
Home Automation
</Text>
</Group>
<Divider />
</AppShell.Section>
{/* Navigation links */}
<AppShell.Section grow mt="sm">
<Stack gap={4}>
{NAV_ENTRIES.map(({ to, label, icon, testId }) => {
// For the home route "/" we need an exact match; others prefix-match is fine.
const isActive =
to === '/'
? location.pathname === '/'
: location.pathname.startsWith(to)
return (
<NavLink
key={to}
component={Link}
to={to}
label={label}
leftSection={icon}
active={isActive}
data-testid={testId}
onClick={onNavClick}
/>
)
})}
</Stack>
</AppShell.Section>
{/* Bottom utilities: theme toggle + logout */}
<AppShell.Section>
<Divider mb="xs" />
<Group gap="xs" px="xs" pb="xs">
<ColorSchemeToggle />
<LogoutButton />
</Group>
</AppShell.Section>
</AppShell.Navbar>
)
}
@@ -0,0 +1,381 @@
/**
* Tests for ExposeSettings (M5-T12).
*
* Strategy: vi.mock the apiClient so GET/PUT/POST responses are controlled
* without a real server or MQTT broker.
*
* Coverage:
* 1. Loading state shows a loader.
* 2. Error state shows an error alert.
* 3. Empty catalog shows the empty-state message.
* 4. Catalog with entities: renders entity names, device groups, toggle switches.
* 5. MQTT status badges render correctly (configured/connected/discovery).
* 6. Toggling a switch calls PUT /api/expose with the correct key/bool.
* 7. Re-publish button calls POST /api/expose/republish.
* 8. Republish ok=true shows success alert; ok=false shows warning.
* 9. value_getter is never rendered (no crash from non-serialisable callable).
* 10. binary_sensor entities are shown in the catalog alongside sensors.
*/
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { screen, waitFor, fireEvent } from '@testing-library/react'
import { renderWithProviders } from '../test-utils'
import { ExposeSettings } from './ExposeSettings'
// ---------------------------------------------------------------------------
// Fixtures
// ---------------------------------------------------------------------------
const MOCK_MQTT_STATUS_OFF = {
mqtt_configured: false,
mqtt_connected: false,
discovery_enabled: false,
}
const MOCK_MQTT_STATUS_ON = {
mqtt_configured: true,
mqtt_connected: true,
discovery_enabled: true,
}
const MOCK_CATALOG_EMPTY = {
catalog: [],
mqtt_status: MOCK_MQTT_STATUS_OFF,
}
const MOCK_CATALOG_ONE_DEVICE = {
catalog: [
{
entity: {
key: 'modbus.abc.voltage',
component: 'sensor',
device: { identifiers: ['modbus', 'abc'], name: 'Test Meter' },
device_class: 'voltage',
unit: 'V',
name: 'Test Meter Voltage',
state_class: 'measurement',
},
enabled: false,
},
{
entity: {
key: 'modbus.abc.current',
component: 'sensor',
device: { identifiers: ['modbus', 'abc'], name: 'Test Meter' },
device_class: 'current',
unit: 'A',
name: 'Test Meter Current',
state_class: 'measurement',
},
enabled: true,
},
{
entity: {
key: 'modbus.abc.online',
component: 'binary_sensor',
device: { identifiers: ['modbus', 'abc'], name: 'Test Meter' },
device_class: 'connectivity',
unit: '',
name: 'Test Meter Online',
state_class: null,
},
enabled: false,
},
],
mqtt_status: MOCK_MQTT_STATUS_ON,
}
// ---------------------------------------------------------------------------
// Mock apiClient
// ---------------------------------------------------------------------------
const mockGet = vi.fn()
const mockPut = vi.fn()
const mockPost = vi.fn()
vi.mock('../api/client', () => ({
default: {
GET: (...args: unknown[]) => mockGet(...args),
PUT: (...args: unknown[]) => mockPut(...args),
POST: (...args: unknown[]) => mockPost(...args),
},
ApiError: class ApiError extends Error {
status: number
body: unknown
constructor(status: number, body: unknown) {
super(`API error ${status}`)
this.name = 'ApiError'
this.status = status
this.body = body
}
},
registerLoginRedirect: vi.fn(),
}))
// ---------------------------------------------------------------------------
// Helper
// ---------------------------------------------------------------------------
function renderExpose() {
return renderWithProviders(<ExposeSettings />, { initialPath: '/config' })
}
// ---------------------------------------------------------------------------
// Tests
// ---------------------------------------------------------------------------
describe('ExposeSettings', () => {
beforeEach(() => {
vi.clearAllMocks()
})
// -------------------------------------------------------------------------
// 1. Loading state
// -------------------------------------------------------------------------
it('shows a loader while the catalog is loading', () => {
// Never resolves
mockGet.mockReturnValue(new Promise(() => {}))
renderExpose()
expect(screen.getByTestId('expose-loading')).toBeInTheDocument()
})
// -------------------------------------------------------------------------
// 2. Error state
// -------------------------------------------------------------------------
it('shows an error alert when the catalog request fails', async () => {
mockGet.mockRejectedValue(new Error('Network error'))
renderExpose()
await waitFor(() => {
expect(screen.getByTestId('expose-load-error')).toBeInTheDocument()
})
})
// -------------------------------------------------------------------------
// 3. Empty catalog
// -------------------------------------------------------------------------
it('shows empty-state message when catalog is empty', async () => {
mockGet.mockResolvedValue({ data: MOCK_CATALOG_EMPTY })
renderExpose()
await waitFor(() => {
expect(screen.getByTestId('expose-empty')).toBeInTheDocument()
})
})
// -------------------------------------------------------------------------
// 4. Catalog with entities
// -------------------------------------------------------------------------
it('renders entity names and device groups', async () => {
mockGet.mockResolvedValue({ data: MOCK_CATALOG_ONE_DEVICE })
renderExpose()
await waitFor(() => {
expect(screen.getByTestId('expose-settings')).toBeInTheDocument()
})
// Device group heading
expect(screen.getByTestId('device-group-Test Meter')).toBeInTheDocument()
// Entity names
expect(screen.getByText('Test Meter Voltage')).toBeInTheDocument()
expect(screen.getByText('Test Meter Current')).toBeInTheDocument()
expect(screen.getByText('Test Meter Online')).toBeInTheDocument()
})
it('renders toggle switches for each entity', async () => {
mockGet.mockResolvedValue({ data: MOCK_CATALOG_ONE_DEVICE })
renderExpose()
await waitFor(() => {
expect(screen.getByTestId('entity-toggle-modbus.abc.voltage')).toBeInTheDocument()
})
// voltage is disabled (enabled=false in fixture)
const voltageSwitch = screen.getByTestId(
'entity-toggle-modbus.abc.voltage',
) as HTMLInputElement
expect(voltageSwitch.checked).toBe(false)
// current is enabled (enabled=true in fixture)
const currentSwitch = screen.getByTestId(
'entity-toggle-modbus.abc.current',
) as HTMLInputElement
expect(currentSwitch.checked).toBe(true)
})
it('shows binary_sensor entities alongside sensor entities', async () => {
mockGet.mockResolvedValue({ data: MOCK_CATALOG_ONE_DEVICE })
renderExpose()
await waitFor(() => {
expect(screen.getByTestId('entity-row-modbus.abc.online')).toBeInTheDocument()
})
// binary_sensor row contains "binary_sensor" text
const onlineRow = screen.getByTestId('entity-row-modbus.abc.online')
expect(onlineRow).toHaveTextContent('binary_sensor')
})
// -------------------------------------------------------------------------
// 5. MQTT status badges
// -------------------------------------------------------------------------
it('renders MQTT status badges', async () => {
mockGet.mockResolvedValue({ data: MOCK_CATALOG_ONE_DEVICE })
renderExpose()
await waitFor(() => {
expect(screen.getByTestId('mqtt-status-badges')).toBeInTheDocument()
})
expect(screen.getByTestId('badge-mqtt-configured')).toBeInTheDocument()
expect(screen.getByTestId('badge-mqtt-connected')).toBeInTheDocument()
expect(screen.getByTestId('badge-discovery-enabled')).toBeInTheDocument()
})
it('shows "Configured" when mqtt_configured is true', async () => {
mockGet.mockResolvedValue({ data: MOCK_CATALOG_ONE_DEVICE })
renderExpose()
await waitFor(() => {
expect(screen.getByTestId('badge-mqtt-configured')).toBeInTheDocument()
})
expect(screen.getByTestId('badge-mqtt-configured')).toHaveTextContent('MQTT Configured')
})
it('shows "Not Configured" when mqtt_configured is false', async () => {
mockGet.mockResolvedValue({ data: MOCK_CATALOG_EMPTY })
renderExpose()
await waitFor(() => {
expect(screen.getByTestId('badge-mqtt-configured')).toBeInTheDocument()
})
expect(screen.getByTestId('badge-mqtt-configured')).toHaveTextContent('MQTT Not Configured')
})
// -------------------------------------------------------------------------
// 6. Toggle a switch → PUT /api/expose
// -------------------------------------------------------------------------
it('calls PUT /api/expose when a toggle is switched', async () => {
mockGet.mockResolvedValue({ data: MOCK_CATALOG_ONE_DEVICE })
mockPut.mockResolvedValue({ data: MOCK_CATALOG_ONE_DEVICE })
renderExpose()
await waitFor(() => {
expect(screen.getByTestId('entity-toggle-modbus.abc.voltage')).toBeInTheDocument()
})
// Click the voltage toggle (currently off → turning on)
const voltageSwitch = screen.getByTestId('entity-toggle-modbus.abc.voltage')
fireEvent.click(voltageSwitch)
await waitFor(() => {
expect(mockPut).toHaveBeenCalled()
})
const putCall = mockPut.mock.calls[0]
// path arg
expect(putCall[0]).toBe('/api/expose')
// body
expect(putCall[1]?.body?.toggles).toMatchObject({ 'modbus.abc.voltage': true })
})
// -------------------------------------------------------------------------
// 7. Republish button → POST /api/expose/republish
// -------------------------------------------------------------------------
it('calls POST /api/expose/republish when republish button is clicked', async () => {
mockGet.mockResolvedValue({ data: MOCK_CATALOG_ONE_DEVICE })
mockPost.mockResolvedValue({ data: { ok: true, message: 'HA Discovery re-published.' } })
renderExpose()
await waitFor(() => {
expect(screen.getByTestId('republish-button')).toBeInTheDocument()
})
fireEvent.click(screen.getByTestId('republish-button'))
await waitFor(() => {
expect(mockPost).toHaveBeenCalled()
})
const postCall = mockPost.mock.calls[0]
expect(postCall[0]).toBe('/api/expose/republish')
})
// -------------------------------------------------------------------------
// 8. Republish ok=true → success alert; ok=false → warning
// -------------------------------------------------------------------------
it('shows success alert when republish returns ok=true', async () => {
mockGet.mockResolvedValue({ data: MOCK_CATALOG_ONE_DEVICE })
mockPost.mockResolvedValue({
data: { ok: true, message: 'HA Discovery re-published successfully.' },
})
renderExpose()
await waitFor(() => {
expect(screen.getByTestId('republish-button')).toBeInTheDocument()
})
fireEvent.click(screen.getByTestId('republish-button'))
await waitFor(() => {
expect(screen.getByTestId('republish-success')).toBeInTheDocument()
})
expect(screen.queryByTestId('republish-warning')).not.toBeInTheDocument()
})
it('shows warning alert when republish returns ok=false', async () => {
mockGet.mockResolvedValue({ data: MOCK_CATALOG_ONE_DEVICE })
mockPost.mockResolvedValue({
data: { ok: false, message: 'MQTT not connected.' },
})
renderExpose()
await waitFor(() => {
expect(screen.getByTestId('republish-button')).toBeInTheDocument()
})
fireEvent.click(screen.getByTestId('republish-button'))
await waitFor(() => {
expect(screen.getByTestId('republish-warning')).toBeInTheDocument()
})
expect(screen.queryByTestId('republish-success')).not.toBeInTheDocument()
})
// -------------------------------------------------------------------------
// 9. No crash from non-serialisable value_getter (value_getter NOT rendered)
// -------------------------------------------------------------------------
it('renders without crashing even if value_getter were present (it is excluded by API)', async () => {
// The API never sends value_getter; this tests resilience of the schema
const catalogWithExtraField = {
...MOCK_CATALOG_ONE_DEVICE,
catalog: MOCK_CATALOG_ONE_DEVICE.catalog.map((entry) => ({
...entry,
entity: { ...entry.entity },
// No value_getter — confirm the component handles normal data
})),
}
mockGet.mockResolvedValue({ data: catalogWithExtraField })
renderExpose()
await waitFor(() => {
expect(screen.getByTestId('expose-settings')).toBeInTheDocument()
})
// No errors thrown, entities visible
expect(screen.getByText('Test Meter Voltage')).toBeInTheDocument()
})
})
+303
View File
@@ -0,0 +1,303 @@
/**
* ExposeSettings Home Assistant Expose entity management (M5-T12).
*
* Behaviours:
* 1. Load: GET /api/expose display catalog of exposable entities grouped by HA device,
* with per-entity toggle switches and MQTT/Discovery connection status badges.
* 2. Toggle: PUT /api/expose with the changed key bool; triggers discovery re-publish.
* 3. Republish: POST /api/expose/republish manually trigger HA Discovery re-publish.
*
* All requests go through the typed apiClient (CSRF injected automatically by
* csrfMiddleware for write operations).
*/
import { useState } from 'react'
import { useQuery, useMutation, useQueryClient } from '@tanstack/react-query'
import {
Alert,
Badge,
Button,
Center,
Group,
Loader,
Stack,
Switch,
Text,
} from '@mantine/core'
import apiClient from '../api/client'
import type { components } from '../api/schema.d.ts'
// ---------------------------------------------------------------------------
// Types
// ---------------------------------------------------------------------------
type CatalogEntry = components['schemas']['CatalogEntrySchema']
type MqttStatus = components['schemas']['MqttStatusSchema']
// ---------------------------------------------------------------------------
// Hook: load expose catalog
// ---------------------------------------------------------------------------
function useExposeCatalog() {
return useQuery({
queryKey: ['expose-catalog'],
queryFn: async () => {
const res = await apiClient.GET('/api/expose')
return res.data
},
})
}
// ---------------------------------------------------------------------------
// Hook: toggle a single entity
// ---------------------------------------------------------------------------
function useToggleEntity() {
const qc = useQueryClient()
return useMutation({
mutationFn: async ({ key, enabled }: { key: string; enabled: boolean }) => {
await apiClient.PUT('/api/expose', {
body: { toggles: { [key]: enabled } },
})
},
onSuccess: () => qc.invalidateQueries({ queryKey: ['expose-catalog'] }),
})
}
// ---------------------------------------------------------------------------
// Hook: republish discovery
// ---------------------------------------------------------------------------
function useRepublish() {
const qc = useQueryClient()
return useMutation({
mutationFn: async () => {
const res = await apiClient.POST('/api/expose/republish')
return res.data
},
onSuccess: () => qc.invalidateQueries({ queryKey: ['expose-catalog'] }),
})
}
// ---------------------------------------------------------------------------
// MqttStatusBadges — connection status indicators
// ---------------------------------------------------------------------------
function MqttStatusBadges({ status }: { status: MqttStatus }) {
return (
<Group gap="xs" wrap="wrap" data-testid="mqtt-status-badges">
<Badge
color={status.mqtt_configured ? 'blue' : 'gray'}
variant="outline"
size="sm"
data-testid="badge-mqtt-configured"
>
MQTT {status.mqtt_configured ? 'Configured' : 'Not Configured'}
</Badge>
<Badge
color={status.mqtt_connected ? 'green' : 'red'}
variant="outline"
size="sm"
data-testid="badge-mqtt-connected"
>
{status.mqtt_connected ? 'Connected' : 'Disconnected'}
</Badge>
<Badge
color={status.discovery_enabled ? 'teal' : 'gray'}
variant="outline"
size="sm"
data-testid="badge-discovery-enabled"
>
Discovery {status.discovery_enabled ? 'On' : 'Off'}
</Badge>
</Group>
)
}
// ---------------------------------------------------------------------------
// EntityRow — one entity toggle row
// ---------------------------------------------------------------------------
interface EntityRowProps {
entry: CatalogEntry
onToggle: (key: string, enabled: boolean) => void
isToggling: boolean
}
function EntityRow({ entry, onToggle, isToggling }: EntityRowProps) {
const { entity, enabled } = entry
return (
<Group justify="space-between" gap="sm" wrap="nowrap" data-testid={`entity-row-${entity.key}`}>
<Stack gap={2}>
<Text size="sm" fw={500}>
{entity.name}
</Text>
<Text size="xs" c="dimmed">
{entity.component}
{entity.device_class ? ` · ${entity.device_class}` : ''}
{entity.unit ? ` · ${entity.unit}` : ''}
</Text>
</Stack>
<Switch
checked={enabled}
onChange={(e) => onToggle(entity.key, e.currentTarget.checked)}
disabled={isToggling}
size="sm"
data-testid={`entity-toggle-${entity.key}`}
/>
</Group>
)
}
// ---------------------------------------------------------------------------
// DeviceGroup — entities grouped by device name
// ---------------------------------------------------------------------------
interface DeviceGroupProps {
deviceName: string
entries: CatalogEntry[]
onToggle: (key: string, enabled: boolean) => void
isToggling: boolean
}
function DeviceGroup({ deviceName, entries, onToggle, isToggling }: DeviceGroupProps) {
return (
<Stack gap="xs" data-testid={`device-group-${deviceName}`}>
<Text fw={600} size="sm" c="dimmed">
{deviceName}
</Text>
{entries.map((entry) => (
<EntityRow
key={entry.entity.key}
entry={entry}
onToggle={onToggle}
isToggling={isToggling}
/>
))}
</Stack>
)
}
// ---------------------------------------------------------------------------
// RepublishButton — POST /api/expose/republish
// ---------------------------------------------------------------------------
function RepublishButton() {
const [result, setResult] = useState<{ ok: boolean; message: string } | null>(null)
const republishMutation = useRepublish()
async function handleRepublish() {
setResult(null)
try {
const data = await republishMutation.mutateAsync()
if (data) {
setResult({ ok: data.ok, message: data.message })
}
} catch {
setResult({ ok: false, message: 'Republish request failed.' })
}
}
return (
<Stack gap="xs">
<Button
type="button"
variant="outline"
size="sm"
onClick={handleRepublish}
loading={republishMutation.isPending}
data-testid="republish-button"
>
Re-publish Discovery
</Button>
{result?.ok && (
<Alert color="green" data-testid="republish-success">
{result.message}
</Alert>
)}
{result !== null && !result.ok && (
<Alert color="orange" data-testid="republish-warning">
{result.message}
</Alert>
)}
</Stack>
)
}
// ---------------------------------------------------------------------------
// ExposeSettings — main exported component
// ---------------------------------------------------------------------------
export function ExposeSettings() {
const { data, isLoading, isError } = useExposeCatalog()
const toggleMutation = useToggleEntity()
function handleToggle(key: string, enabled: boolean) {
toggleMutation.mutate({ key, enabled })
}
if (isLoading) {
return (
<Center pt="md">
<Loader size="sm" data-testid="expose-loading" />
</Center>
)
}
if (isError || !data) {
return (
<Alert color="red" data-testid="expose-load-error">
Failed to load expose settings. Please refresh the page.
</Alert>
)
}
// Defensive null-coerce in case the data shape is unexpected at runtime.
const catalog = data.catalog ?? []
const mqtt_status = data.mqtt_status ?? {
mqtt_configured: false,
mqtt_connected: false,
discovery_enabled: false,
}
// Group entities by device name for display.
const deviceGroups = new Map<string, CatalogEntry[]>()
for (const entry of catalog) {
const deviceName = entry.entity.device.name
const existing = deviceGroups.get(deviceName)
if (existing) {
existing.push(entry)
} else {
deviceGroups.set(deviceName, [entry])
}
}
return (
<Stack gap="md" data-testid="expose-settings">
{/* Status row */}
<Group justify="space-between" align="center" wrap="wrap" gap="sm">
<MqttStatusBadges status={mqtt_status} />
<RepublishButton />
</Group>
{/* Entity catalog — empty state */}
{catalog.length === 0 && (
<Text c="dimmed" size="sm" data-testid="expose-empty">
No exposable entities found. Add a Modbus device in the Energy page to get started.
</Text>
)}
{/* Entity catalog — grouped by device */}
{Array.from(deviceGroups.entries()).map(([deviceName, entries]) => (
<DeviceGroup
key={deviceName}
deviceName={deviceName}
entries={entries}
onToggle={handleToggle}
isToggling={toggleMutation.isPending}
/>
))}
</Stack>
)
}
+338
View File
@@ -0,0 +1,338 @@
/**
* Tests for ContractForm component.
*
* Coverage:
* 1. Renders profile fields dynamically from profile structure (not hardcoded)
* verified by passing defaultKind so fields render automatically.
* 2. Submit in create mode calls POST /api/energy/contracts.
* 3. Submit in add-version mode calls POST /api/energy/contracts/{id}/versions.
* 4. Cancel closes modal.
*/
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { screen, waitFor } from '@testing-library/react'
import userEvent from '@testing-library/user-event'
import { renderWithProviders } from '../test-utils'
// ---------------------------------------------------------------------------
// Mock apiClient
// ---------------------------------------------------------------------------
const mockGet = vi.fn()
const mockPost = vi.fn()
vi.mock('../api/client', () => ({
default: {
GET: (...args: unknown[]) => mockGet(...args),
POST: (...args: unknown[]) => mockPost(...args),
PATCH: vi.fn(),
DELETE: vi.fn(),
},
ApiError: class ApiError extends Error {
status: number
body: unknown
constructor(status: number, body: unknown) {
super(`API error ${status}`)
this.name = 'ApiError'
this.status = status
this.body = body
}
},
registerLoginRedirect: vi.fn(),
}))
// ---------------------------------------------------------------------------
// Fixtures
// ---------------------------------------------------------------------------
const PROFILES_RESPONSE = {
profiles: [
{
kind: 'manual',
label: 'Fixed / Variable Rate (NL, dual-tariff)',
energy: {
dual_tariff: true,
buy: { normal: { unit: 'EUR/kWh' }, dal: { unit: 'EUR/kWh' } },
sell: { normal: { unit: 'EUR/kWh' }, dal: { unit: 'EUR/kWh' } },
energy_tax: { unit: 'EUR/kWh' },
ode: { unit: 'EUR/kWh', default: 0 },
},
standing: {
network_fee: { unit: 'EUR/month' },
management_fee: { unit: 'EUR/month' },
},
credits: {
heffingskorting: { unit: 'EUR/year' },
},
},
],
}
const CREATED_CONTRACT = {
id: 1,
name: 'Test Contract',
kind: 'manual',
active: false,
currency: 'EUR',
created_at: '2026-06-01T00:00:00Z',
updated_at: '2026-06-01T00:00:00Z',
versions: [],
}
// ---------------------------------------------------------------------------
// Import component
// ---------------------------------------------------------------------------
import { ContractForm } from './ContractForm'
// ---------------------------------------------------------------------------
// Tests
// ---------------------------------------------------------------------------
describe('ContractForm', () => {
beforeEach(() => vi.clearAllMocks())
it('renders profile fields dynamically from profile structure (not hardcoded)', async () => {
mockGet.mockImplementation((path: string) => {
if (path === '/api/energy/profiles') {
return Promise.resolve({ data: PROFILES_RESPONSE })
}
return Promise.resolve({ data: null })
})
const onClose = vi.fn()
const onSaved = vi.fn()
// Pass defaultKind so the profile fields render automatically in create mode
renderWithProviders(
<ContractForm defaultKind="manual" onClose={onClose} onSaved={onSaved} />,
)
// Wait for profiles to load and fields to appear
await waitFor(() => {
expect(screen.getByTestId('contract-section-energy')).toBeInTheDocument()
}, { timeout: 3000 })
// Verify sections are rendered from profile structure (not hardcoded)
expect(screen.getByTestId('contract-section-energy')).toBeInTheDocument()
expect(screen.getByTestId('contract-section-standing')).toBeInTheDocument()
expect(screen.getByTestId('contract-section-credits')).toBeInTheDocument()
// Verify specific fields exist from the profile structure
// "energy.buy.normal" → label "Buy Normal"
expect(screen.getByTestId('contract-field-energy.buy.normal')).toBeInTheDocument()
expect(screen.getByTestId('contract-field-energy.buy.dal')).toBeInTheDocument()
expect(screen.getByTestId('contract-field-standing.network_fee')).toBeInTheDocument()
})
it('calls POST /api/energy/contracts in create mode', async () => {
const user = userEvent.setup()
mockGet.mockImplementation((path: string) => {
if (path === '/api/energy/profiles') {
return Promise.resolve({ data: PROFILES_RESPONSE })
}
return Promise.resolve({ data: null })
})
mockPost.mockResolvedValue({ data: CREATED_CONTRACT })
const onClose = vi.fn()
const onSaved = vi.fn()
// Use defaultKind to ensure fields load without needing to interact with Select
renderWithProviders(
<ContractForm defaultKind="manual" onClose={onClose} onSaved={onSaved} />,
)
// Wait for form to render
await waitFor(() => {
expect(screen.getByTestId('contract-form')).toBeInTheDocument()
})
// Wait for profile fields to appear
await waitFor(() => {
expect(screen.getByTestId('contract-name')).toBeInTheDocument()
})
// Fill in contract name
await user.type(screen.getByTestId('contract-name'), 'Test Contract')
// Submit form
await user.click(screen.getByTestId('contract-form-submit'))
await waitFor(() => {
expect(mockPost).toHaveBeenCalledWith(
'/api/energy/contracts',
expect.objectContaining({
body: expect.objectContaining({
name: 'Test Contract',
kind: 'manual',
currency: 'EUR',
}),
}),
)
}, { timeout: 3000 })
})
it('calls POST /api/energy/contracts/{id}/versions in add-version mode', async () => {
const user = userEvent.setup()
mockGet.mockImplementation((path: string) => {
if (path === '/api/energy/profiles') {
return Promise.resolve({ data: PROFILES_RESPONSE })
}
return Promise.resolve({ data: null })
})
mockPost.mockResolvedValue({ data: CREATED_CONTRACT })
const onClose = vi.fn()
const onSaved = vi.fn()
renderWithProviders(
<ContractForm contractId={1} defaultKind="manual" onClose={onClose} onSaved={onSaved} />,
)
// Wait for form to be ready (add-version mode)
await waitFor(() => {
expect(screen.getByTestId('contract-form')).toBeInTheDocument()
})
// In add-version mode, no name or kind fields shown
expect(screen.queryByTestId('contract-name')).not.toBeInTheDocument()
expect(screen.queryByTestId('contract-kind')).not.toBeInTheDocument()
// Fill in effective_from (required in add-version mode)
const effectiveDateInput = screen.getByTestId('contract-effective-from')
await user.type(effectiveDateInput, '2026-07-01')
// Submit form
await user.click(screen.getByTestId('contract-form-submit'))
await waitFor(() => {
expect(mockPost).toHaveBeenCalledWith(
'/api/energy/contracts/{contract_id}/versions',
expect.objectContaining({
params: { path: { contract_id: 1 } },
body: expect.objectContaining({
values: expect.any(Object),
}),
}),
)
}, { timeout: 3000 })
})
it('calls onClose when Cancel button is clicked', async () => {
const user = userEvent.setup()
mockGet.mockImplementation((path: string) => {
if (path === '/api/energy/profiles') {
return Promise.resolve({ data: PROFILES_RESPONSE })
}
return Promise.resolve({ data: null })
})
const onClose = vi.fn()
const onSaved = vi.fn()
renderWithProviders(
<ContractForm onClose={onClose} onSaved={onSaved} />,
)
await waitFor(() => {
expect(screen.getByTestId('contract-form-cancel')).toBeInTheDocument()
})
await user.click(screen.getByTestId('contract-form-cancel'))
expect(onClose).toHaveBeenCalled()
})
it('prefills fields from the open version when in add-version mode', async () => {
// FU11: add-version mode should auto-prefill rate fields from the contract's
// current open version so the user only has to change what's different.
const CONTRACT_DETAIL = {
id: 42,
name: 'My Contract',
kind: 'manual',
active: true,
currency: 'EUR',
created_at: '2026-01-01T00:00:00Z',
updated_at: '2026-01-01T00:00:00Z',
versions: [
{
id: 1,
contract_id: 42,
effective_from: '2026-01-01T00:00:00Z',
effective_to: null, // open version
values: {
energy: {
buy: { normal: 0.133, dal: 0.127 },
sell: { normal: 0.05, dal: 0.05 },
energy_tax: 0.11,
ode: 0.0,
},
standing: {
network_fee: 30.0,
management_fee: 60.0,
},
credits: {
heffingskorting: 365.0,
},
},
created_at: '2026-01-01T00:00:00Z',
},
],
}
mockGet.mockImplementation((path: string) => {
if (path === '/api/energy/profiles') {
return Promise.resolve({ data: PROFILES_RESPONSE })
}
if (path === '/api/energy/contracts/{contract_id}') {
return Promise.resolve({ data: CONTRACT_DETAIL })
}
return Promise.resolve({ data: null })
})
const onClose = vi.fn()
const onSaved = vi.fn()
renderWithProviders(
<ContractForm contractId={42} defaultKind="manual" onClose={onClose} onSaved={onSaved} />,
)
const user = userEvent.setup()
// Wait for the form to render and profile fields to appear
await waitFor(() => {
expect(screen.getByTestId('contract-section-energy')).toBeInTheDocument()
}, { timeout: 3000 })
// Wait for prefill to apply: both profile and contract detail queries must resolve.
// Fill in the required effective_from date and submit; the submitted values
// body.values should contain the prefilled rates from the open version.
mockPost.mockResolvedValue({ data: {} })
const effectiveInput = screen.getByTestId('contract-effective-from')
await user.type(effectiveInput, '2026-07-01')
await user.click(screen.getByTestId('contract-form-submit'))
// The submitted body should contain the prefilled values from CONTRACT_DETAIL
await waitFor(() => {
expect(mockPost).toHaveBeenCalledWith(
'/api/energy/contracts/{contract_id}/versions',
expect.objectContaining({
body: expect.objectContaining({
values: expect.objectContaining({
standing: expect.objectContaining({
network_fee: 30,
management_fee: 60,
}),
}),
}),
}),
)
}, { timeout: 3000 })
})
})
+466
View File
@@ -0,0 +1,466 @@
/**
* ContractForm create a new energy contract OR add a version to an existing one.
*
* Form fields are dynamically rendered from the profile structure returned by
* GET /api/energy/profiles never hardcoded. The profile structure provides
* section names (energy, standing, credits), leaf keys, and their units.
*
* Create mode: show Name, Kind (Select from profiles), Currency, optional
* effective_from, then dynamically rendered profile value fields.
* Add-version mode (contractId provided): show only effective_from (required)
* + profile value fields for the contract's kind.
*/
import { useState, useMemo } from 'react'
import {
Modal,
Stack,
TextInput,
NumberInput,
Select,
Button,
Group,
Alert,
Loader,
Center,
Text,
Divider,
Title,
} from '@mantine/core'
import { useEnergyProfiles, useCreateContract, useAddContractVersion, useContractDetail } from './hooks'
// ---------------------------------------------------------------------------
// Props
// ---------------------------------------------------------------------------
export interface ContractFormProps {
/** When provided: add-version mode; contract must exist. */
contractId?: number
/** Existing contract kind (for add-version mode or edit). */
defaultKind?: string
onClose: () => void
onSaved: () => void
}
// ---------------------------------------------------------------------------
// Helpers for dynamic profile field rendering
// ---------------------------------------------------------------------------
/**
* A leaf in the profile structure is any object with a `unit` key.
* Returns true if the value is a leaf (renderable field).
*/
function isLeaf(val: unknown): val is { unit: string; default?: unknown } {
return typeof val === 'object' && val !== null && 'unit' in val
}
/**
* Extract all leaf fields from a profile section, returning
* [dotPath, unit, defaultValue] tuples. E.g.:
* "energy.buy.normal" "EUR/kWh"
* "energy.energy_tax" "EUR/kWh"
*/
interface LeafField {
/** Dot-separated path within the section, e.g. "buy.normal" */
fieldPath: string
unit: string
defaultValue?: number
}
function extractLeafFields(obj: Record<string, unknown>, prefix = ''): LeafField[] {
const fields: LeafField[] = []
for (const [key, val] of Object.entries(obj)) {
// Skip non-object values (e.g. dual_tariff: true)
if (typeof val !== 'object' || val === null) continue
const path = prefix ? `${prefix}.${key}` : key
if (isLeaf(val)) {
fields.push({
fieldPath: path,
unit: val.unit,
defaultValue: typeof val.default === 'number' ? val.default : undefined,
})
} else {
fields.push(...extractLeafFields(val as Record<string, unknown>, path))
}
}
return fields
}
/**
* Build a nested values object from flat field values.
* E.g. { "buy.normal": 0.133, "buy.dal": 0.127 } { buy: { normal: 0.133, dal: 0.127 } }
*/
function buildNestedValues(
sectionFields: Record<string, LeafField[]>,
fieldValues: Record<string, number | string>,
): Record<string, unknown> {
const result: Record<string, unknown> = {}
for (const [section, fields] of Object.entries(sectionFields)) {
const sectionObj: Record<string, unknown> = {}
for (const field of fields) {
const raw = fieldValues[`${section}.${field.fieldPath}`]
const numVal = typeof raw === 'number' ? raw : parseFloat(String(raw))
// Set nested path
const parts = field.fieldPath.split('.')
let current = sectionObj
for (let i = 0; i < parts.length - 1; i++) {
if (!(parts[i] in current)) current[parts[i]] = {}
current = current[parts[i]] as Record<string, unknown>
}
current[parts[parts.length - 1]] = isNaN(numVal) ? 0 : numVal
}
result[section] = sectionObj
}
return result
}
/**
* Label formatter: converts "buy.normal" "Buy Normal", "energy_tax" "Energy Tax"
*/
function formatLabel(path: string): string {
return path
.replace(/\./g, ' ')
.replace(/_/g, ' ')
.replace(/\b\w/g, (c) => c.toUpperCase())
}
// ---------------------------------------------------------------------------
// Component
// ---------------------------------------------------------------------------
export function ContractForm({ contractId, defaultKind, onClose, onSaved }: ContractFormProps) {
const isAddVersion = contractId != null
// Profiles query
const profilesQuery = useEnergyProfiles()
// Contract detail query — only used in add-version mode to prefill from the open version.
const contractDetailQuery = useContractDetail(contractId ?? 0)
// Create mode fields
const [name, setName] = useState('')
const [selectedKind, setSelectedKind] = useState<string | null>(defaultKind ?? null)
const [currency, setCurrency] = useState('EUR')
// ISO date string "YYYY-MM-DD" for effective_from; empty = default to now
const [effectiveFromStr, setEffectiveFromStr] = useState('')
// Dynamic profile field values: key = "<section>.<fieldPath>", value = number.
// User edits are stored here; for add-version mode this is merged with prefillValues.
const [userEdits, setUserEdits] = useState<Record<string, number | string>>({})
const [error, setError] = useState<string | null>(null)
const createMutation = useCreateContract()
const addVersionMutation = useAddContractVersion()
const isBusy = createMutation.isPending || addVersionMutation.isPending
// Resolve the effective kind (add-version mode uses defaultKind)
const effectiveKind = isAddVersion ? (defaultKind ?? null) : selectedKind
// Build profile options from API response
const profiles = profilesQuery.data?.profiles ?? []
const profileOptions = profiles.map((p: Record<string, unknown>) => ({
value: p.kind as string,
label: (p.label as string | undefined) ?? (p.kind as string),
}))
// Find the selected profile structure
const selectedProfile = profiles.find((p: Record<string, unknown>) => p.kind === effectiveKind)
// Extract section → leaf fields mapping from the selected profile (memoized for stable reference).
const sectionFields = useMemo<Record<string, LeafField[]>>(() => {
if (!selectedProfile) return {}
const profileObj = selectedProfile as Record<string, unknown>
const result: Record<string, LeafField[]> = {}
for (const [key, val] of Object.entries(profileObj)) {
if (key === 'kind' || key === 'label') continue
if (typeof val === 'object' && val !== null && !isLeaf(val)) {
result[key] = extractLeafFields(val as Record<string, unknown>)
}
}
return result
}, [selectedProfile])
// Add-version prefill: compute the seeded values from the contract's open version.
// Returns the flat fieldValues dict or null when data is not ready.
const prefillValues = useMemo<Record<string, number | string> | null>(() => {
if (!isAddVersion) return null
if (contractDetailQuery.isLoading || contractDetailQuery.isError) return null
if (Object.keys(sectionFields).length === 0) return null
const detail = contractDetailQuery.data
if (!detail || !detail.versions || detail.versions.length === 0) return null
const versions = detail.versions as Array<{
effective_from: string
effective_to: string | null
values: Record<string, unknown>
}>
let openVersion = versions.find((v) => v.effective_to == null)
if (!openVersion) {
openVersion = [...versions].sort((a, b) =>
a.effective_from < b.effective_from ? 1 : -1
)[0]
}
if (!openVersion?.values) return null
const seeded: Record<string, number | string> = {}
for (const [section, fields] of Object.entries(sectionFields)) {
const sectionVal = (openVersion.values as Record<string, unknown>)[section]
if (typeof sectionVal !== 'object' || sectionVal === null) continue
for (const leaf of fields) {
const parts = leaf.fieldPath.split('.')
let cursor: unknown = sectionVal
for (const part of parts) {
if (typeof cursor !== 'object' || cursor === null) { cursor = undefined; break }
cursor = (cursor as Record<string, unknown>)[part]
}
if (cursor != null && (typeof cursor === 'number' || typeof cursor === 'string')) {
const numVal = typeof cursor === 'number' ? cursor : parseFloat(String(cursor))
seeded[`${section}.${leaf.fieldPath}`] = isNaN(numVal) ? 0 : numVal
}
}
}
return Object.keys(seeded).length > 0 ? seeded : null
}, [
isAddVersion,
contractDetailQuery.isLoading,
contractDetailQuery.isError,
contractDetailQuery.data,
sectionFields,
])
// The effective field values: user edits override prefill; prefill is the base.
// In add-version mode: when the user has not yet edited any field (userEdits is empty),
// prefillValues is used as the initial base. Once the user edits any field, their
// full merged value (prefill + override) is stored in userEdits.
const fieldValues: Record<string, number | string> =
isAddVersion && prefillValues !== null && Object.keys(userEdits).length === 0
? prefillValues
: userEdits
function handleFieldChange(key: string, val: number | string) {
// On first edit in add-version mode, copy prefill into userEdits so we don't
// lose the other prefilled values when the user changes one field.
setUserEdits((prev) => {
const base = isAddVersion && prefillValues !== null && Object.keys(prev).length === 0
? { ...prefillValues }
: prev
return { ...base, [key]: val }
})
}
// Initialize default values when profile changes (create mode only)
function initDefaults(kind: string | null) {
if (!kind) return
const profile = profiles.find((p: Record<string, unknown>) => p.kind === kind)
if (!profile) return
const profileObj = profile as Record<string, unknown>
const defaults: Record<string, number | string> = {}
for (const [sectionKey, sectionVal] of Object.entries(profileObj)) {
if (sectionKey === 'kind' || sectionKey === 'label') continue
if (typeof sectionVal === 'object' && sectionVal !== null && !isLeaf(sectionVal)) {
const leaves = extractLeafFields(sectionVal as Record<string, unknown>)
for (const leaf of leaves) {
defaults[`${sectionKey}.${leaf.fieldPath}`] = leaf.defaultValue ?? 0
}
}
}
setUserEdits(defaults)
}
async function handleSubmit(e: React.FormEvent) {
e.preventDefault()
setError(null)
if (!isAddVersion && !name.trim()) {
setError('Contract name is required.')
return
}
if (!effectiveKind) {
setError('Contract kind is required.')
return
}
const values = buildNestedValues(sectionFields, fieldValues)
try {
// Convert local date string to a naive local-midnight datetime string (no Z).
// The backend interprets naive datetimes as server local wall-clock time,
// so "2026-06-25T00:00:00" → CEST local midnight → stored as UTC 22:00Z.
// When effectiveFromStr is empty, fall back to the current instant (aware,
// sent as ISO with Z) so the backend stores it unchanged.
const effectiveFromISO = effectiveFromStr
? `${effectiveFromStr}T00:00:00`
: undefined
if (isAddVersion) {
const body = {
effective_from: effectiveFromISO ?? new Date().toISOString(),
values,
}
await addVersionMutation.mutateAsync({ id: contractId, body })
} else {
const body = {
name: name.trim(),
kind: effectiveKind,
currency,
values,
...(effectiveFromISO ? { effective_from: effectiveFromISO } : {}),
}
await createMutation.mutateAsync(body)
}
onSaved()
onClose()
} catch {
setError('Failed to save. Please try again.')
}
}
// ---------------------------------------------------------------------------
// Render
// ---------------------------------------------------------------------------
const title = isAddVersion ? 'Add Contract Version' : 'New Energy Contract'
return (
<Modal
opened
onClose={onClose}
title={title}
size="md"
data-testid="contract-form-modal"
>
{(profilesQuery.isLoading || (isAddVersion && contractDetailQuery.isLoading)) && (
<Center py="md">
<Loader size="sm" data-testid="contract-form-loading" />
</Center>
)}
{profilesQuery.isError && (
<Alert color="red" mb="sm" data-testid="contract-form-profiles-error">
Failed to load pricing profiles. Cannot continue.
</Alert>
)}
{isAddVersion && contractDetailQuery.isError && !contractDetailQuery.isLoading && (
<Alert color="yellow" mb="sm" data-testid="contract-form-detail-error">
Could not load existing contract values for prefill fields start at 0.
</Alert>
)}
{!profilesQuery.isLoading && !(isAddVersion && contractDetailQuery.isLoading) && (
<form onSubmit={handleSubmit} data-testid="contract-form">
<Stack gap="sm">
{/* Create mode fields */}
{!isAddVersion && (
<>
<TextInput
label="Contract name"
required
value={name}
onChange={(e) => setName(e.currentTarget.value)}
data-testid="contract-name"
/>
<Select
label="Pricing kind"
required
data={profileOptions}
value={selectedKind}
onChange={(val) => {
setSelectedKind(val)
initDefaults(val)
}}
data-testid="contract-kind"
/>
<TextInput
label="Currency"
value={currency}
onChange={(e) => setCurrency(e.currentTarget.value)}
data-testid="contract-currency"
/>
</>
)}
{/* Effective from — required for add-version, optional for create */}
<TextInput
label={isAddVersion ? 'Effective from (required)' : 'Effective from (optional, defaults to now)'}
description="Date in YYYY-MM-DD format"
type="date"
required={isAddVersion}
value={effectiveFromStr}
onChange={(e) => setEffectiveFromStr(e.currentTarget.value)}
data-testid="contract-effective-from"
/>
{/* Dynamic profile fields */}
{effectiveKind && Object.keys(sectionFields).length > 0 && (
<>
<Divider mt="xs" />
{Object.entries(sectionFields).map(([section, fields]) => (
<Stack key={section} gap="xs" data-testid={`contract-section-${section}`}>
<Title order={6} tt="capitalize" c="dimmed">
{section.replace(/_/g, ' ')}
</Title>
{fields.map((field) => {
const key = `${section}.${field.fieldPath}`
const raw = fieldValues[key]
const numVal = typeof raw === 'number' ? raw : parseFloat(String(raw))
return (
<NumberInput
key={key}
label={formatLabel(field.fieldPath)}
description={field.unit}
value={isNaN(numVal) ? 0 : numVal}
onChange={(val) => handleFieldChange(key, val)}
decimalScale={6}
step={0.001}
data-testid={`contract-field-${key}`}
/>
)
})}
</Stack>
))}
</>
)}
{!effectiveKind && !isAddVersion && (
<Text size="sm" c="dimmed" data-testid="contract-form-kind-hint">
Select a pricing kind to see the rate fields.
</Text>
)}
{error && (
<Alert color="red" data-testid="contract-form-error">
{error}
</Alert>
)}
<Group justify="flex-end" gap="sm">
<Button
type="button"
variant="default"
onClick={onClose}
data-testid="contract-form-cancel"
>
Cancel
</Button>
<Button
type="submit"
loading={isBusy}
data-testid="contract-form-submit"
>
{isAddVersion ? 'Add Version' : 'Create Contract'}
</Button>
</Group>
</Stack>
</form>
)}
</Modal>
)
}
@@ -0,0 +1,205 @@
/**
* Tests for ContractManager component.
*
* Coverage:
* 1. Loading state rendering.
* 2. Contract list rendering.
* 3. Empty state rendering.
* 4. Activate button calls PATCH with {active: true}.
* 5. "New Contract" button opens ContractForm.
*/
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { screen, waitFor } from '@testing-library/react'
import userEvent from '@testing-library/user-event'
import { renderWithProviders } from '../test-utils'
// ---------------------------------------------------------------------------
// Mock apiClient
// ---------------------------------------------------------------------------
const mockGet = vi.fn()
const mockPost = vi.fn()
const mockPatch = vi.fn()
vi.mock('../api/client', () => ({
default: {
GET: (...args: unknown[]) => mockGet(...args),
POST: (...args: unknown[]) => mockPost(...args),
PATCH: (...args: unknown[]) => mockPatch(...args),
DELETE: vi.fn(),
},
ApiError: class ApiError extends Error {
status: number
body: unknown
constructor(status: number, body: unknown) {
super(`API error ${status}`)
this.name = 'ApiError'
this.status = status
this.body = body
}
},
registerLoginRedirect: vi.fn(),
}))
// ---------------------------------------------------------------------------
// Fixtures
// ---------------------------------------------------------------------------
const ACTIVE_CONTRACT = {
id: 1,
name: 'My Active Contract',
kind: 'manual',
active: true,
currency: 'EUR',
created_at: '2026-06-01T00:00:00Z',
updated_at: '2026-06-01T00:00:00Z',
}
const INACTIVE_CONTRACT = {
id: 2,
name: 'Inactive Contract',
kind: 'tibber',
active: false,
currency: 'EUR',
created_at: '2026-06-02T00:00:00Z',
updated_at: '2026-06-02T00:00:00Z',
}
const PROFILES_RESPONSE = {
profiles: [
{
kind: 'manual',
label: 'Fixed / Variable Rate (NL, dual-tariff)',
energy: {
dual_tariff: true,
buy: { normal: { unit: 'EUR/kWh' }, dal: { unit: 'EUR/kWh' } },
sell: { normal: { unit: 'EUR/kWh' }, dal: { unit: 'EUR/kWh' } },
energy_tax: { unit: 'EUR/kWh' },
ode: { unit: 'EUR/kWh', default: 0 },
},
standing: {
network_fee: { unit: 'EUR/month' },
management_fee: { unit: 'EUR/month' },
},
credits: {
heffingskorting: { unit: 'EUR/year' },
},
},
],
}
// ---------------------------------------------------------------------------
// Import component
// ---------------------------------------------------------------------------
import { ContractManager } from './ContractManager'
// ---------------------------------------------------------------------------
// Tests
// ---------------------------------------------------------------------------
describe('ContractManager', () => {
beforeEach(() => vi.clearAllMocks())
it('renders loading state initially', () => {
// Never resolve so we stay in loading state
mockGet.mockImplementation(() => new Promise(() => {}))
renderWithProviders(<ContractManager />)
expect(screen.getByTestId('contracts-loading')).toBeInTheDocument()
})
it('renders contract list when data loads', async () => {
mockGet.mockImplementation((path: string) => {
if (path === '/api/energy/contracts') {
return Promise.resolve({
data: { items: [ACTIVE_CONTRACT, INACTIVE_CONTRACT], total: 2 },
})
}
return Promise.resolve({ data: null })
})
renderWithProviders(<ContractManager />)
await waitFor(() => {
expect(screen.getByTestId('contracts-table')).toBeInTheDocument()
})
expect(screen.getByText('My Active Contract')).toBeInTheDocument()
expect(screen.getByText('Inactive Contract')).toBeInTheDocument()
expect(screen.getByTestId('contract-active-badge-1')).toHaveTextContent('active')
expect(screen.getByTestId('contract-active-badge-2')).toHaveTextContent('inactive')
})
it('renders empty state when no contracts exist', async () => {
mockGet.mockResolvedValue({ data: { items: [], total: 0 } })
renderWithProviders(<ContractManager />)
await waitFor(() => {
expect(screen.getByTestId('contracts-empty')).toBeInTheDocument()
})
})
it('calls PATCH with active=true when Activate button clicked', async () => {
const user = userEvent.setup()
mockGet.mockImplementation((path: string) => {
if (path === '/api/energy/contracts') {
return Promise.resolve({
data: { items: [INACTIVE_CONTRACT], total: 1 },
})
}
return Promise.resolve({ data: null })
})
mockPatch.mockResolvedValue({
data: { ...INACTIVE_CONTRACT, active: true, versions: [] },
})
renderWithProviders(<ContractManager />)
await waitFor(() => {
expect(screen.getByTestId(`contract-activate-${INACTIVE_CONTRACT.id}`)).toBeInTheDocument()
})
await user.click(screen.getByTestId(`contract-activate-${INACTIVE_CONTRACT.id}`))
await waitFor(() => {
expect(mockPatch).toHaveBeenCalledWith(
'/api/energy/contracts/{contract_id}',
expect.objectContaining({
params: { path: { contract_id: INACTIVE_CONTRACT.id } },
body: { active: true },
}),
)
})
})
it('opens ContractForm when "New Contract" button is clicked', async () => {
const user = userEvent.setup()
mockGet.mockImplementation((path: string) => {
if (path === '/api/energy/contracts') {
return Promise.resolve({ data: { items: [], total: 0 } })
}
if (path === '/api/energy/profiles') {
return Promise.resolve({ data: PROFILES_RESPONSE })
}
return Promise.resolve({ data: null })
})
renderWithProviders(<ContractManager />)
await waitFor(() => {
expect(screen.getByTestId('contract-new-button')).toBeInTheDocument()
})
await user.click(screen.getByTestId('contract-new-button'))
await waitFor(() => {
expect(screen.getByTestId('contract-form-modal')).toBeInTheDocument()
})
})
})
+331
View File
@@ -0,0 +1,331 @@
/**
* ContractManager energy contract list UI.
*
* Features:
* - Table of contracts: name, kind, active badge, currency, created date, actions.
* - Actions per row: Activate (PATCH active=true), Add Version, View History.
* - "New Contract" button at top.
* - Version history modal showing all versions with effective dates and values.
* - Loading / error / empty states.
*/
import { useState } from 'react'
import {
Table,
Button,
Group,
Text,
Loader,
Center,
Alert,
Stack,
Badge,
ScrollArea,
Modal,
Accordion,
Code,
} from '@mantine/core'
import {
useContracts,
useUpdateContract,
type ContractResponse,
type ContractDetailResponse,
} from './hooks'
import { ContractForm } from './ContractForm'
import apiClient from '../api/client'
import { formatLocalDate } from '../utils/datetime'
// ---------------------------------------------------------------------------
// Version history modal
// ---------------------------------------------------------------------------
interface VersionHistoryModalProps {
contractId: number
contractName: string
onClose: () => void
}
function VersionHistoryModal({ contractId, contractName, onClose }: VersionHistoryModalProps) {
const [loading, setLoading] = useState(true)
const [detail, setDetail] = useState<ContractDetailResponse | null>(null)
const [fetchError, setFetchError] = useState<string | null>(null)
// Fetch detail on first render
useState(() => {
apiClient
.GET('/api/energy/contracts/{contract_id}', {
params: { path: { contract_id: contractId } },
})
.then((res) => {
setDetail(res.data ?? null)
setLoading(false)
})
.catch(() => {
setFetchError('Failed to load contract details.')
setLoading(false)
})
})
return (
<Modal
opened
onClose={onClose}
title={`Version history — ${contractName}`}
size="lg"
data-testid="version-history-modal"
>
{loading && (
<Center py="md">
<Loader size="sm" />
</Center>
)}
{fetchError && (
<Alert color="red" data-testid="version-history-error">
{fetchError}
</Alert>
)}
{!loading && !fetchError && detail && (
<>
{detail.versions.length === 0 ? (
<Text c="dimmed" ta="center" size="sm" data-testid="version-history-empty">
No versions found.
</Text>
) : (
<Accordion variant="separated" data-testid="version-history-accordion">
{detail.versions.map((v) => (
<Accordion.Item key={v.id} value={String(v.id)} data-testid={`version-item-${v.id}`}>
<Accordion.Control>
<Group gap="sm">
<Text size="sm" fw={500}>
From: {formatLocalDate(v.effective_from)}
</Text>
{v.effective_to ? (
<Text size="xs" c="dimmed">
To: {formatLocalDate(v.effective_to)}
</Text>
) : (
<Badge color="green" size="xs">
current
</Badge>
)}
</Group>
</Accordion.Control>
<Accordion.Panel>
<Code block style={{ fontSize: 11 }}>
{JSON.stringify(v.values, null, 2)}
</Code>
</Accordion.Panel>
</Accordion.Item>
))}
</Accordion>
)}
</>
)}
<Group justify="flex-end" mt="md">
<Button variant="default" onClick={onClose}>
Close
</Button>
</Group>
</Modal>
)
}
// ---------------------------------------------------------------------------
// Contract table
// ---------------------------------------------------------------------------
interface ContractTableProps {
contracts: ContractResponse[]
onActivate: (id: number) => void
onAddVersion: (contract: ContractResponse) => void
onViewHistory: (contract: ContractResponse) => void
activatingId: number | null
}
function ContractTable({
contracts,
onActivate,
onAddVersion,
onViewHistory,
activatingId,
}: ContractTableProps) {
if (contracts.length === 0) {
return (
<Text c="dimmed" ta="center" size="sm" data-testid="contracts-empty">
No contracts configured yet. Click "New Contract" to add one.
</Text>
)
}
return (
<ScrollArea>
<Table striped highlightOnHover withTableBorder data-testid="contracts-table">
<Table.Thead>
<Table.Tr>
<Table.Th>Name</Table.Th>
<Table.Th>Kind</Table.Th>
<Table.Th>Active</Table.Th>
<Table.Th>Currency</Table.Th>
<Table.Th>Created</Table.Th>
<Table.Th style={{ textAlign: 'right' }}>Actions</Table.Th>
</Table.Tr>
</Table.Thead>
<Table.Tbody>
{contracts.map((contract) => (
<Table.Tr key={contract.id} data-testid={`contract-row-${contract.id}`}>
<Table.Td>
<Text fw={500} size="sm">
{contract.name}
</Text>
</Table.Td>
<Table.Td>
<Badge variant="outline" size="sm">
{contract.kind}
</Badge>
</Table.Td>
<Table.Td>
<Badge
color={contract.active ? 'green' : 'gray'}
variant="light"
size="sm"
data-testid={`contract-active-badge-${contract.id}`}
>
{contract.active ? 'active' : 'inactive'}
</Badge>
</Table.Td>
<Table.Td>{contract.currency}</Table.Td>
<Table.Td>
<Text size="xs" c="dimmed">
{formatLocalDate(contract.created_at)}
</Text>
</Table.Td>
<Table.Td>
<Group justify="flex-end" gap="xs">
{!contract.active && (
<Button
size="xs"
variant="outline"
color="green"
loading={activatingId === contract.id}
onClick={() => onActivate(contract.id)}
data-testid={`contract-activate-${contract.id}`}
>
Activate
</Button>
)}
<Button
size="xs"
variant="outline"
onClick={() => onAddVersion(contract)}
data-testid={`contract-add-version-${contract.id}`}
>
Add Version
</Button>
<Button
size="xs"
variant="subtle"
onClick={() => onViewHistory(contract)}
data-testid={`contract-history-${contract.id}`}
>
History
</Button>
</Group>
</Table.Td>
</Table.Tr>
))}
</Table.Tbody>
</Table>
</ScrollArea>
)
}
// ---------------------------------------------------------------------------
// ContractManager — top-level
// ---------------------------------------------------------------------------
export function ContractManager() {
const contractsQuery = useContracts()
const updateMutation = useUpdateContract()
const [showCreateForm, setShowCreateForm] = useState(false)
const [addVersionContract, setAddVersionContract] = useState<ContractResponse | null>(null)
const [historyContract, setHistoryContract] = useState<ContractResponse | null>(null)
const [activatingId, setActivatingId] = useState<number | null>(null)
async function handleActivate(id: number) {
setActivatingId(id)
try {
await updateMutation.mutateAsync({ id, body: { active: true } })
} finally {
setActivatingId(null)
}
}
// ---------------------------------------------------------------------------
// Render states
// ---------------------------------------------------------------------------
if (contractsQuery.isLoading) {
return (
<Center py="xl" data-testid="contracts-loading">
<Loader />
</Center>
)
}
if (contractsQuery.isError || !contractsQuery.data) {
return (
<Alert color="red" data-testid="contracts-load-error">
Failed to load contracts. Please refresh.
</Alert>
)
}
const contracts = contractsQuery.data.items
return (
<Stack gap="lg" data-testid="contract-manager">
<Group justify="space-between" align="center">
<Text fw={500}>Energy Contracts</Text>
<Button onClick={() => setShowCreateForm(true)} data-testid="contract-new-button">
New Contract
</Button>
</Group>
<ContractTable
contracts={contracts}
onActivate={handleActivate}
onAddVersion={(c) => setAddVersionContract(c)}
onViewHistory={(c) => setHistoryContract(c)}
activatingId={activatingId}
/>
{/* Create new contract */}
{showCreateForm && (
<ContractForm
onClose={() => setShowCreateForm(false)}
onSaved={() => setShowCreateForm(false)}
/>
)}
{/* Add version to existing contract */}
{addVersionContract && (
<ContractForm
contractId={addVersionContract.id}
defaultKind={addVersionContract.kind}
onClose={() => setAddVersionContract(null)}
onSaved={() => setAddVersionContract(null)}
/>
)}
{/* Version history modal */}
{historyContract && (
<VersionHistoryModal
contractId={historyContract.id}
contractName={historyContract.name}
onClose={() => setHistoryContract(null)}
/>
)}
</Stack>
)
}
+214
View File
@@ -0,0 +1,214 @@
/**
* Tests for CostView component.
*
* Coverage:
* 1. Loading state.
* 2. Empty state.
* 3. Renders costs table with data.
* 4. Shows summary values.
* 5. Recompute button triggers confirmation modal and then recompute call.
*/
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { screen, waitFor } from '@testing-library/react'
import userEvent from '@testing-library/user-event'
import { renderWithProviders } from '../test-utils'
// ---------------------------------------------------------------------------
// Mock apiClient
// ---------------------------------------------------------------------------
const mockGet = vi.fn()
const mockPost = vi.fn()
vi.mock('../api/client', () => ({
default: {
GET: (...args: unknown[]) => mockGet(...args),
POST: (...args: unknown[]) => mockPost(...args),
PATCH: vi.fn(),
DELETE: vi.fn(),
},
ApiError: class ApiError extends Error {
status: number
body: unknown
constructor(status: number, body: unknown) {
super(`API error ${status}`)
this.name = 'ApiError'
this.status = status
this.body = body
}
},
registerLoginRedirect: vi.fn(),
}))
// ---------------------------------------------------------------------------
// Fixtures
// ---------------------------------------------------------------------------
const COST_PERIOD = {
period_start: '2026-06-22T10:00:00Z',
d1_kwh: 0.1,
d2_kwh: 0.2,
r1_kwh: 0.0,
r2_kwh: 0.05,
import_cost: 0.044,
export_revenue: 0.005,
net_cost: 0.039,
currency: 'EUR',
degraded: false,
contract_version_id: 1,
}
const SUMMARY = {
currency: 'EUR',
metered_import: 10.5,
metered_export: 2.3,
metered_net: 8.2,
fixed_costs: 5.0,
credits: 50.0,
total_payable: 12.5,
}
// ---------------------------------------------------------------------------
// Import component
// ---------------------------------------------------------------------------
import { CostView } from './CostView'
// ---------------------------------------------------------------------------
// Tests
// ---------------------------------------------------------------------------
describe('CostView', () => {
beforeEach(() => vi.clearAllMocks())
it('renders loading state initially', () => {
mockGet.mockImplementation(() => new Promise(() => {}))
renderWithProviders(<CostView />)
expect(screen.getByTestId('costs-loading')).toBeInTheDocument()
})
it('renders empty state when no cost periods available', async () => {
mockGet.mockImplementation((path: string) => {
if (path === '/api/energy/costs') {
return Promise.resolve({ data: { items: [], total: 0 } })
}
if (path === '/api/energy/costs/summary') {
return Promise.resolve({ data: SUMMARY })
}
return Promise.resolve({ data: null })
})
renderWithProviders(<CostView />)
await waitFor(() => {
expect(screen.getByTestId('costs-empty')).toBeInTheDocument()
})
})
it('renders costs table when data is available', async () => {
mockGet.mockImplementation((path: string) => {
if (path === '/api/energy/costs') {
return Promise.resolve({ data: { items: [COST_PERIOD], total: 1 } })
}
if (path === '/api/energy/costs/summary') {
return Promise.resolve({ data: SUMMARY })
}
return Promise.resolve({ data: null })
})
renderWithProviders(<CostView />)
await waitFor(() => {
expect(screen.getByTestId('costs-table')).toBeInTheDocument()
})
expect(screen.getByTestId('cost-row-0')).toBeInTheDocument()
})
it('shows summary values correctly', async () => {
mockGet.mockImplementation((path: string) => {
if (path === '/api/energy/costs') {
return Promise.resolve({ data: { items: [], total: 0 } })
}
if (path === '/api/energy/costs/summary') {
return Promise.resolve({ data: SUMMARY })
}
return Promise.resolve({ data: null })
})
renderWithProviders(<CostView />)
await waitFor(() => {
expect(screen.getByTestId('summary-import')).toBeInTheDocument()
})
expect(screen.getByTestId('summary-import')).toHaveTextContent('10.500')
expect(screen.getByTestId('summary-export')).toHaveTextContent('2.300')
expect(screen.getByTestId('summary-total')).toHaveTextContent('12.50')
})
it('shows recompute confirmation modal on button click', async () => {
const user = userEvent.setup()
mockGet.mockImplementation((path: string) => {
if (path === '/api/energy/costs') {
return Promise.resolve({ data: { items: [], total: 0 } })
}
if (path === '/api/energy/costs/summary') {
return Promise.resolve({ data: SUMMARY })
}
return Promise.resolve({ data: null })
})
renderWithProviders(<CostView />)
await waitFor(() => {
expect(screen.getByTestId('cost-recompute-button')).toBeInTheDocument()
})
await user.click(screen.getByTestId('cost-recompute-button'))
await waitFor(() => {
expect(screen.getByTestId('recompute-confirm-modal')).toBeInTheDocument()
})
})
it('calls recompute mutation when confirmed', async () => {
const user = userEvent.setup()
mockGet.mockImplementation((path: string) => {
if (path === '/api/energy/costs') {
return Promise.resolve({ data: { items: [], total: 0 } })
}
if (path === '/api/energy/costs/summary') {
return Promise.resolve({ data: SUMMARY })
}
return Promise.resolve({ data: null })
})
mockPost.mockResolvedValue({ data: { recomputed: 0 } })
renderWithProviders(<CostView />)
await waitFor(() => {
expect(screen.getByTestId('cost-recompute-button')).toBeInTheDocument()
})
await user.click(screen.getByTestId('cost-recompute-button'))
await waitFor(() => {
expect(screen.getByTestId('recompute-confirm')).toBeInTheDocument()
})
await user.click(screen.getByTestId('recompute-confirm'))
await waitFor(() => {
expect(mockPost).toHaveBeenCalledWith(
'/api/energy/costs/recompute',
expect.any(Object),
)
})
})
})
+403
View File
@@ -0,0 +1,403 @@
/**
* CostView cost trends + detail + summary.
*
* Features:
* - Date range selector: Today / This month / Custom (SegmentedControl + DateInput).
* - Recharts AreaChart showing import_cost, export_revenue, net_cost per period.
* - Table showing per-15min periods with time, kWh values, costs, degraded badge.
* - Summary cards: metered import/export, fixed costs, credits, total payable.
* - "Recompute" button with confirmation.
* - Loading/error/empty states, degraded period highlighting.
*
* Recharts imports are isolated to this file only.
*/
import { useState } from 'react'
import {
Stack,
Text,
TextInput,
Loader,
Center,
Alert,
Table,
Group,
Badge,
ScrollArea,
SegmentedControl,
Button,
Paper,
SimpleGrid,
Modal,
Title,
} from '@mantine/core'
import {
AreaChart,
Area,
XAxis,
YAxis,
CartesianGrid,
Tooltip,
Legend,
ResponsiveContainer,
} from 'recharts'
import { useEnergyCosts, useEnergyCostSummary, useRecomputeCosts } from './hooks'
import { formatLocalTime } from '../utils/datetime'
// ---------------------------------------------------------------------------
// Cost limit — prevent accidental full-table pulls
// ---------------------------------------------------------------------------
const COSTS_MAX_LIMIT = 500
// ---------------------------------------------------------------------------
// Date range helpers
// ---------------------------------------------------------------------------
function getTodayRange(): { start: string; end: string } {
// Use local date boundary so "today" matches what the user sees in the display.
const now = new Date()
const start = new Date(now.getFullYear(), now.getMonth(), now.getDate())
const end = new Date(now.getFullYear(), now.getMonth(), now.getDate() + 1)
return { start: start.toISOString(), end: end.toISOString() }
}
function getThisMonthRange(): { start: string; end: string } {
// Use local date boundary so "this month" matches what the user sees in the display.
const now = new Date()
const start = new Date(now.getFullYear(), now.getMonth(), 1)
const end = new Date(now.getFullYear(), now.getMonth() + 1, 1)
return { start: start.toISOString(), end: end.toISOString() }
}
// ---------------------------------------------------------------------------
// Summary cards
// ---------------------------------------------------------------------------
interface SummaryCardProps {
label: string
value: string
testId?: string
}
function SummaryCard({ label, value, testId }: SummaryCardProps) {
return (
<Paper withBorder p="sm" data-testid={testId}>
<Stack gap={4}>
<Text size="xs" c="dimmed">
{label}
</Text>
<Text fw={600} size="lg">
{value}
</Text>
</Stack>
</Paper>
)
}
// ---------------------------------------------------------------------------
// CostView — main component
// ---------------------------------------------------------------------------
type RangePreset = 'today' | 'month' | 'custom'
export function CostView() {
const [rangePreset, setRangePreset] = useState<RangePreset>('today')
// Date strings in YYYY-MM-DD format for custom range
const [customStartStr, setCustomStartStr] = useState('')
const [customEndStr, setCustomEndStr] = useState('')
const [showRecomputeConfirm, setShowRecomputeConfirm] = useState(false)
// Compute effective date range
const { start, end } = (() => {
if (rangePreset === 'today') return getTodayRange()
if (rangePreset === 'month') return getThisMonthRange()
return {
start: customStartStr ? new Date(customStartStr).toISOString() : undefined,
end: customEndStr ? new Date(customEndStr).toISOString() : undefined,
}
})()
const costsQuery = useEnergyCosts(start, end, COSTS_MAX_LIMIT)
const summaryQuery = useEnergyCostSummary(start, end)
const recomputeMutation = useRecomputeCosts()
async function handleRecompute() {
setShowRecomputeConfirm(false)
await recomputeMutation.mutateAsync({ start, end })
}
// ---------------------------------------------------------------------------
// Render
// ---------------------------------------------------------------------------
const currency = costsQuery.data?.items[0]?.currency ?? summaryQuery.data?.currency ?? 'EUR'
return (
<Stack gap="lg" data-testid="cost-view">
{/* Date range selector */}
<Group align="flex-start" gap="md" wrap="wrap">
<Stack gap="xs">
<Text size="sm" fw={500}>
Date range
</Text>
<SegmentedControl
value={rangePreset}
onChange={(v) => setRangePreset(v as RangePreset)}
data={[
{ label: 'Today', value: 'today' },
{ label: 'This month', value: 'month' },
{ label: 'Custom', value: 'custom' },
]}
data-testid="cost-range-control"
/>
</Stack>
{rangePreset === 'custom' && (
<Group gap="sm" align="flex-end">
<TextInput
label="From"
type="date"
value={customStartStr}
onChange={(e) => setCustomStartStr(e.currentTarget.value)}
data-testid="cost-custom-start"
/>
<TextInput
label="To"
type="date"
value={customEndStr}
onChange={(e) => setCustomEndStr(e.currentTarget.value)}
data-testid="cost-custom-end"
/>
</Group>
)}
<Group gap="sm" style={{ marginLeft: 'auto' }} align="flex-end">
<Button
variant="outline"
color="orange"
size="sm"
onClick={() => setShowRecomputeConfirm(true)}
loading={recomputeMutation.isPending}
data-testid="cost-recompute-button"
>
Recompute
</Button>
</Group>
</Group>
{/* Summary cards */}
{summaryQuery.isLoading && (
<Center>
<Loader size="sm" />
</Center>
)}
{summaryQuery.isError && (
<Alert color="red" data-testid="summary-error">
Failed to load cost summary.
</Alert>
)}
{summaryQuery.data && (
<Stack gap="xs">
<Title order={6} c="dimmed">
Summary
</Title>
<SimpleGrid cols={{ base: 2, sm: 3 }} spacing="sm" data-testid="cost-summary">
<SummaryCard
label="Import (kWh)"
value={summaryQuery.data.metered_import.toFixed(3)}
testId="summary-import"
/>
<SummaryCard
label="Export (kWh)"
value={summaryQuery.data.metered_export.toFixed(3)}
testId="summary-export"
/>
<SummaryCard
label={`Fixed costs (${currency})`}
value={summaryQuery.data.fixed_costs.toFixed(2)}
testId="summary-fixed"
/>
<SummaryCard
label={`Credits (${currency})`}
value={summaryQuery.data.credits.toFixed(2)}
testId="summary-credits"
/>
<SummaryCard
label={`Total payable (${currency})`}
value={summaryQuery.data.total_payable.toFixed(2)}
testId="summary-total"
/>
</SimpleGrid>
</Stack>
)}
{/* Cost chart */}
{costsQuery.isLoading && (
<Center py="xl" data-testid="costs-loading">
<Loader />
</Center>
)}
{costsQuery.isError && (
<Alert color="red" data-testid="costs-error">
Failed to load cost periods. Please refresh.
</Alert>
)}
{costsQuery.data && costsQuery.data.items.length === 0 && (
<Alert color="gray" data-testid="costs-empty">
No cost data available for the selected period.
</Alert>
)}
{costsQuery.data && costsQuery.data.items.length > 0 && (
<>
{/* Area chart */}
<Stack gap="xs" data-testid="cost-chart">
<Title order={6} c="dimmed">
Cost trends ({currency})
</Title>
<ResponsiveContainer width="100%" height={220}>
<AreaChart
data={costsQuery.data.items.map((item) => ({
time: formatLocalTime(item.period_start),
import_cost: item.import_cost,
export_revenue: item.export_revenue,
net_cost: item.net_cost,
}))}
margin={{ top: 4, right: 16, left: 0, bottom: 4 }}
>
<CartesianGrid strokeDasharray="3 3" />
<XAxis dataKey="time" tick={{ fontSize: 10 }} interval="preserveStartEnd" />
<YAxis tick={{ fontSize: 10 }} tickFormatter={(v: number) => v.toFixed(2)} />
<Tooltip
// eslint-disable-next-line @typescript-eslint/no-explicit-any
formatter={(val: any) =>
[`${typeof val === 'number' ? val.toFixed(4) : String(val)} ${currency}`, undefined]
}
/>
<Legend />
<Area
type="monotone"
dataKey="import_cost"
stroke="#2196f3"
fill="#bbdefb"
name="Import cost"
dot={false}
/>
<Area
type="monotone"
dataKey="export_revenue"
stroke="#4caf50"
fill="#c8e6c9"
name="Export revenue"
dot={false}
/>
<Area
type="monotone"
dataKey="net_cost"
stroke="#ff9800"
fill="#ffe0b2"
name="Net cost"
dot={false}
/>
</AreaChart>
</ResponsiveContainer>
</Stack>
{/* Detail table */}
<Stack gap="xs">
<Title order={6} c="dimmed">
Period detail
</Title>
<ScrollArea>
<Table
striped
highlightOnHover
withTableBorder
withColumnBorders
style={{ fontSize: 12 }}
data-testid="costs-table"
>
<Table.Thead>
<Table.Tr>
<Table.Th>Time</Table.Th>
<Table.Th>Import kWh</Table.Th>
<Table.Th>Export kWh</Table.Th>
<Table.Th>Import cost</Table.Th>
<Table.Th>Export rev.</Table.Th>
<Table.Th>Net cost</Table.Th>
<Table.Th></Table.Th>
</Table.Tr>
</Table.Thead>
<Table.Tbody>
{costsQuery.data.items.map((item, idx) => (
<Table.Tr
key={idx}
style={item.degraded ? { opacity: 0.6 } : undefined}
data-testid={`cost-row-${idx}`}
>
<Table.Td>
<Text size="xs">
{formatLocalTime(item.period_start)}
</Text>
</Table.Td>
<Table.Td>{(item.d1_kwh + item.d2_kwh).toFixed(3)}</Table.Td>
<Table.Td>{(item.r1_kwh + item.r2_kwh).toFixed(3)}</Table.Td>
<Table.Td>{item.import_cost.toFixed(4)}</Table.Td>
<Table.Td>{item.export_revenue.toFixed(4)}</Table.Td>
<Table.Td>{item.net_cost.toFixed(4)}</Table.Td>
<Table.Td>
{item.degraded && (
<Badge color="orange" size="xs" data-testid={`cost-degraded-${idx}`}>
degraded
</Badge>
)}
</Table.Td>
</Table.Tr>
))}
</Table.Tbody>
</Table>
</ScrollArea>
</Stack>
</>
)}
{/* Recompute confirmation modal */}
{showRecomputeConfirm && (
<Modal
opened
onClose={() => setShowRecomputeConfirm(false)}
title="Recompute costs?"
size="sm"
data-testid="recompute-confirm-modal"
>
<Stack gap="md">
<Text size="sm">
This will recompute all cost periods for the selected date range. Continue?
</Text>
<Group justify="flex-end" gap="sm">
<Button
variant="default"
onClick={() => setShowRecomputeConfirm(false)}
data-testid="recompute-cancel"
>
Cancel
</Button>
<Button
color="orange"
onClick={handleRecompute}
data-testid="recompute-confirm"
>
Recompute
</Button>
</Group>
</Stack>
</Modal>
)}
</Stack>
)
}
+241
View File
@@ -0,0 +1,241 @@
/**
* Tests for energy/DeviceForm.tsx
*
* Coverage:
* 1. Renders empty form in create mode (title "New Device").
* 2. Renders pre-filled form in edit mode (title "Edit Device"), shows UUID.
* 3. Validation: missing friendly_name shows error.
* 4. Profile dropdown loads options from GET /api/modbus/profiles.
* 5. Submit in create mode calls POST /api/modbus/devices.
* 6. Submit in edit mode calls PATCH /api/modbus/devices/{uuid}.
* 7. Cancel button invokes onClose without submitting.
*/
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { screen, waitFor, fireEvent } from '@testing-library/react'
import { renderWithProviders } from '../test-utils'
import { DeviceForm } from './DeviceForm'
// ---------------------------------------------------------------------------
// Fixtures
// ---------------------------------------------------------------------------
const DEVICE = {
uuid: 'abc-123',
friendly_name: 'SDM120 Main',
transport: 'tcp',
host: '192.168.1.100',
port: 502,
unit_id: 1,
profile: 'sdm120',
poll_interval_s: 5,
enabled: true,
last_poll_at: null,
last_poll_ok: null,
created_at: '2026-06-01T00:00:00Z',
updated_at: '2026-06-01T00:00:00Z',
}
const PROFILES_RESP = {
profiles: [{ name: 'sdm120', description: 'Eastron SDM120 single-phase energy meter' }],
}
// ---------------------------------------------------------------------------
// Mock apiClient
// ---------------------------------------------------------------------------
const mockGet = vi.fn()
const mockPost = vi.fn()
const mockPatch = vi.fn()
vi.mock('../api/client', () => ({
default: {
GET: (...args: unknown[]) => mockGet(...args),
POST: (...args: unknown[]) => mockPost(...args),
PATCH: (...args: unknown[]) => mockPatch(...args),
DELETE: vi.fn(),
},
ApiError: class ApiError extends Error {
status: number
body: unknown
constructor(status: number, body: unknown) {
super(`API error ${status}`)
this.name = 'ApiError'
this.status = status
this.body = body
}
},
registerLoginRedirect: vi.fn(),
}))
// ---------------------------------------------------------------------------
// Helpers
// ---------------------------------------------------------------------------
function setupProfilesMock() {
mockGet.mockImplementation((path: string) => {
if (path === '/api/modbus/profiles') {
return Promise.resolve({ data: PROFILES_RESP })
}
// modbus-devices for invalidation
return Promise.resolve({ data: { items: [], total: 0 } })
})
}
function renderCreateForm(onClose = vi.fn(), onSaved = vi.fn()) {
return renderWithProviders(<DeviceForm onClose={onClose} onSaved={onSaved} />, {
initialPath: '/',
})
}
function renderEditForm(onClose = vi.fn(), onSaved = vi.fn()) {
return renderWithProviders(<DeviceForm device={DEVICE} onClose={onClose} onSaved={onSaved} />, {
initialPath: '/',
})
}
// ---------------------------------------------------------------------------
// Tests
// ---------------------------------------------------------------------------
describe('DeviceForm — create mode', () => {
beforeEach(() => {
vi.clearAllMocks()
setupProfilesMock()
})
it('renders with title "New Device" and empty friendly-name input', async () => {
renderCreateForm()
await waitFor(() => {
expect(screen.getByTestId('device-form-modal')).toBeInTheDocument()
})
// Modal title
expect(screen.getByText('New Device')).toBeInTheDocument()
// UUID display should NOT be present in create mode
expect(screen.queryByTestId('device-uuid-display')).not.toBeInTheDocument()
})
it('loads profile options from GET /api/modbus/profiles', async () => {
renderCreateForm()
await waitFor(() => {
expect(screen.getByTestId('device-form')).toBeInTheDocument()
})
// Profile select should be rendered
expect(screen.getByTestId('device-profile')).toBeInTheDocument()
expect(mockGet).toHaveBeenCalledWith('/api/modbus/profiles')
})
it('shows validation error when friendly name is empty', async () => {
renderCreateForm()
await waitFor(() => expect(screen.getByTestId('device-form')).toBeInTheDocument())
fireEvent.submit(screen.getByTestId('device-form'))
await waitFor(() => {
expect(screen.getByTestId('device-form-error')).toBeInTheDocument()
})
expect(screen.getByTestId('device-form-error').textContent).toContain('Friendly name')
})
it('submit calls POST /api/modbus/devices with correct body (profile auto-selected)', async () => {
mockPost.mockResolvedValue({ data: DEVICE })
const onSaved = vi.fn()
renderCreateForm(vi.fn(), onSaved)
// Wait for profiles to load; sdm120 is auto-selected as first profile.
await waitFor(() => expect(screen.getByTestId('device-form')).toBeInTheDocument())
// Fill form (profile auto-selected to 'sdm120')
fireEvent.change(screen.getByTestId('device-friendly-name'), {
target: { value: 'My Meter' },
})
fireEvent.change(screen.getByTestId('device-host'), {
target: { value: '10.0.0.1' },
})
fireEvent.submit(screen.getByTestId('device-form'))
await waitFor(() => {
expect(mockPost).toHaveBeenCalledWith(
'/api/modbus/devices',
expect.objectContaining({
body: expect.objectContaining({
friendly_name: 'My Meter',
host: '10.0.0.1',
transport: 'tcp',
profile: 'sdm120',
}),
}),
)
})
})
it('cancel calls onClose without POST', async () => {
const onClose = vi.fn()
renderCreateForm(onClose)
await waitFor(() => expect(screen.getByTestId('device-form-cancel')).toBeInTheDocument())
fireEvent.click(screen.getByTestId('device-form-cancel'))
expect(onClose).toHaveBeenCalledOnce()
expect(mockPost).not.toHaveBeenCalled()
})
})
describe('DeviceForm — edit mode', () => {
beforeEach(() => {
vi.clearAllMocks()
setupProfilesMock()
})
it('renders with title "Edit Device" and pre-filled friendly-name', async () => {
renderEditForm()
// Wait for modal to appear
await waitFor(() => {
expect(screen.getByTestId('device-form-modal')).toBeInTheDocument()
})
expect(screen.getByText('Edit Device')).toBeInTheDocument()
// Wait for form to appear (profiles must load first)
await waitFor(() => {
expect(screen.getByTestId('device-form')).toBeInTheDocument()
})
expect(screen.getByTestId('device-uuid-display').textContent).toContain('abc-123')
// Friendly name input should be pre-filled
const input = screen.getByTestId('device-friendly-name') as HTMLInputElement
expect(input.value).toBe('SDM120 Main')
})
it('submit calls PATCH /api/modbus/devices/{uuid}', async () => {
mockPatch.mockResolvedValue({ data: DEVICE })
renderEditForm()
// Wait for profiles to load so the form is visible
await waitFor(() => expect(screen.getByTestId('device-form')).toBeInTheDocument())
fireEvent.change(screen.getByTestId('device-friendly-name'), {
target: { value: 'SDM120 Updated' },
})
fireEvent.submit(screen.getByTestId('device-form'))
await waitFor(() => {
expect(mockPatch).toHaveBeenCalledWith('/api/modbus/devices/{uuid}', {
params: { path: { uuid: 'abc-123' } },
body: expect.objectContaining({ friendly_name: 'SDM120 Updated' }),
})
})
})
})
+263
View File
@@ -0,0 +1,263 @@
/**
* DeviceForm create / edit a Modbus device.
*
* Used for both new-device creation (no `device` prop) and editing an
* existing device (pass `device` prop with current values).
*
* Profile list is fetched from GET /api/modbus/profiles and rendered as a
* Select dropdown. All other fields are inline inputs.
*/
import { useState } from 'react'
import {
Modal,
Stack,
TextInput,
NumberInput,
Select,
Switch,
Button,
Group,
Alert,
Loader,
Center,
Text,
} from '@mantine/core'
import { useProfiles, useCreateDevice, useUpdateDevice } from './hooks'
import type { ModbusDevice, ModbusDeviceCreate, ModbusDeviceUpdate } from './hooks'
// ---------------------------------------------------------------------------
// Props
// ---------------------------------------------------------------------------
interface DeviceFormProps {
/** When provided, the form is in edit mode; otherwise create mode. */
device?: ModbusDevice
onClose: () => void
onSaved: () => void
}
// ---------------------------------------------------------------------------
// Component
// ---------------------------------------------------------------------------
export function DeviceForm({ device, onClose, onSaved }: DeviceFormProps) {
const isEdit = !!device
// Form state — initialise from device if editing.
const [friendlyName, setFriendlyName] = useState(device?.friendly_name ?? '')
const [host, setHost] = useState(device?.host ?? '')
const [port, setPort] = useState<number | string>(device?.port ?? 502)
const [unitId, setUnitId] = useState<number | string>(device?.unit_id ?? 1)
const [profile, setProfile] = useState<string | null>(device?.profile ?? null)
const [pollIntervalS, setPollIntervalS] = useState<number | string>(device?.poll_interval_s ?? 5)
const [enabled, setEnabled] = useState(device?.enabled ?? true)
const [error, setError] = useState<string | null>(null)
const profilesQuery = useProfiles()
const createMutation = useCreateDevice()
const updateMutation = useUpdateDevice()
const isBusy = createMutation.isPending || updateMutation.isPending
// Build select options from profiles response.
const profileOptions =
profilesQuery.data?.profiles.map((p) => ({
value: p.name,
label: `${p.name}${p.description}`,
})) ?? []
// In create mode, if no explicit selection has been made and profiles are loaded,
// derive a default from the first available profile. We avoid useEffect+setState
// (causes cascading renders) by computing the effective value here instead.
const effectiveProfile: string | null =
profile !== null
? profile
: !isEdit && profileOptions.length > 0
? profileOptions[0].value
: null
// ---------------------------------------------------------------------------
// Validation
// ---------------------------------------------------------------------------
function validate(): string | null {
if (!friendlyName.trim()) return 'Friendly name is required.'
if (!host.trim()) return 'Host is required.'
const portNum = Number(port)
if (!Number.isInteger(portNum) || portNum < 1 || portNum > 65535)
return 'Port must be an integer between 1 and 65535.'
const unitNum = Number(unitId)
if (!Number.isInteger(unitNum) || unitNum < 1 || unitNum > 247)
return 'Unit ID must be an integer between 1 and 247.'
if (!effectiveProfile) return 'Profile is required.'
const pollNum = Number(pollIntervalS)
if (!Number.isInteger(pollNum) || pollNum < 1)
return 'Poll interval must be a positive integer (seconds).'
return null
}
// ---------------------------------------------------------------------------
// Submit
// ---------------------------------------------------------------------------
async function handleSubmit(e: React.FormEvent) {
e.preventDefault()
setError(null)
const validationError = validate()
if (validationError) {
setError(validationError)
return
}
try {
if (isEdit && device) {
const body: ModbusDeviceUpdate = {
friendly_name: friendlyName.trim(),
host: host.trim(),
port: Number(port),
unit_id: Number(unitId),
profile: effectiveProfile!,
poll_interval_s: Number(pollIntervalS),
enabled,
}
await updateMutation.mutateAsync({ uuid: device.uuid, body })
} else {
const body: ModbusDeviceCreate = {
friendly_name: friendlyName.trim(),
transport: 'tcp',
host: host.trim(),
port: Number(port),
unit_id: Number(unitId),
profile: effectiveProfile!,
poll_interval_s: Number(pollIntervalS),
enabled,
}
await createMutation.mutateAsync(body)
}
onSaved()
onClose()
} catch {
setError('Failed to save device. Please try again.')
}
}
// ---------------------------------------------------------------------------
// Render
// ---------------------------------------------------------------------------
return (
<Modal
opened
onClose={onClose}
title={isEdit ? 'Edit Device' : 'New Device'}
size="md"
data-testid="device-form-modal"
>
{profilesQuery.isLoading && (
<Center py="md">
<Loader size="sm" />
</Center>
)}
{profilesQuery.isError && (
<Alert color="red" mb="sm" data-testid="profiles-load-error">
Failed to load profiles. Cannot create device.
</Alert>
)}
{!profilesQuery.isLoading && (
<form onSubmit={handleSubmit} data-testid="device-form">
<Stack gap="sm">
{isEdit && (
<Text size="xs" c="dimmed" data-testid="device-uuid-display">
UUID: {device!.uuid}
</Text>
)}
<TextInput
label="Friendly name"
required
value={friendlyName}
onChange={(e) => setFriendlyName(e.currentTarget.value)}
data-testid="device-friendly-name"
/>
<TextInput
label="Host"
description="Gateway IP address"
required
value={host}
onChange={(e) => setHost(e.currentTarget.value)}
data-testid="device-host"
/>
<NumberInput
label="Port"
required
min={1}
max={65535}
value={port}
onChange={(val) => setPort(val)}
data-testid="device-port"
/>
<NumberInput
label="Unit ID"
description="Modbus slave address (Meter ID on device panel)"
required
min={1}
max={247}
value={unitId}
onChange={(val) => setUnitId(val)}
data-testid="device-unit-id"
/>
<Select
label="Profile"
description="YAML device profile"
required
data={profileOptions}
value={effectiveProfile}
onChange={setProfile}
data-testid="device-profile"
/>
<NumberInput
label="Poll interval (seconds)"
required
min={1}
value={pollIntervalS}
onChange={(val) => setPollIntervalS(val)}
data-testid="device-poll-interval"
/>
<Switch
label="Enabled"
description="Whether this device is polled"
checked={enabled}
onChange={(e) => setEnabled(e.currentTarget.checked)}
data-testid="device-enabled"
/>
{error && (
<Alert color="red" data-testid="device-form-error">
{error}
</Alert>
)}
<Group justify="flex-end" gap="sm">
<Button variant="default" onClick={onClose} data-testid="device-form-cancel">
Cancel
</Button>
<Button type="submit" loading={isBusy} data-testid="device-form-submit">
{isEdit ? 'Save' : 'Create'}
</Button>
</Group>
</Stack>
</form>
)}
</Modal>
)
}
+82
View File
@@ -0,0 +1,82 @@
/**
* Tests for DsmrPanel latest parsed DSMR telegram view.
*
* Coverage:
* 1. Loading state.
* 2. Empty state (found=false) with a helpful hint.
* 3. Renders the payload as a key/value table; null values shown as "—".
* 4. Error state.
*/
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { screen, waitFor } from '@testing-library/react'
import { renderWithProviders } from '../test-utils'
const mockGet = vi.fn()
vi.mock('../api/client', () => ({
default: {
GET: (...args: unknown[]) => mockGet(...args),
POST: vi.fn(),
PATCH: vi.fn(),
DELETE: vi.fn(),
},
ApiError: class ApiError extends Error {
status: number
body: unknown
constructor(status: number, body: unknown) {
super(`API error ${status}`)
this.name = 'ApiError'
this.status = status
this.body = body
}
},
registerLoginRedirect: vi.fn(),
}))
import { DsmrPanel } from './DsmrPanel'
describe('DsmrPanel', () => {
beforeEach(() => vi.clearAllMocks())
it('renders loading state initially', () => {
mockGet.mockImplementation(() => new Promise(() => {}))
renderWithProviders(<DsmrPanel />)
expect(screen.getByTestId('dsmr-loading')).toBeInTheDocument()
})
it('shows an empty hint when no DSMR data has been ingested', async () => {
mockGet.mockResolvedValue({ data: { found: false, recorded_at: null, payload: null } })
renderWithProviders(<DsmrPanel />)
await waitFor(() => expect(screen.getByTestId('dsmr-empty')).toBeInTheDocument())
expect(screen.queryByTestId('dsmr-table')).not.toBeInTheDocument()
})
it('renders the latest telegram as a key/value table; null shown as dash', async () => {
mockGet.mockResolvedValue({
data: {
found: true,
recorded_at: '2026-06-23T12:16:00Z',
payload: {
electricity_delivered_1: '20915.154',
phase_voltage_l2: null,
},
},
})
renderWithProviders(<DsmrPanel />)
await waitFor(() => expect(screen.getByTestId('dsmr-table')).toBeInTheDocument())
// Field rows present (keys are sorted).
expect(screen.getByTestId('dsmr-row-electricity_delivered_1')).toHaveTextContent('20915.154')
// Null phase value rendered as an em dash, not omitted.
expect(screen.getByTestId('dsmr-row-phase_voltage_l2')).toHaveTextContent('—')
expect(screen.getByTestId('dsmr-recorded-at')).toBeInTheDocument()
})
it('renders an error state when the request fails', async () => {
mockGet.mockRejectedValue(new Error('network down'))
renderWithProviders(<DsmrPanel />)
await waitFor(() => expect(screen.getByTestId('dsmr-error')).toBeInTheDocument())
})
})
+154
View File
@@ -0,0 +1,154 @@
/**
* DsmrPanel shows the latest parsed DSMR telegram (the source of our metering data).
*
* This is the "ground truth" view: it fetches GET /api/energy/dsmr/latest and
* renders the full parsed frame as a key/value table, so you can confirm at a
* glance whether DSMR ingest is actually landing data (without reading the DB).
*
* - Auto-refresh toggle (default on) so new telegrams appear without a manual reload.
* - Loading / error / empty states; "empty" explains the likely cause.
* - Null phase values are shown as "—" (kept verbatim in the payload).
*/
import { useState } from 'react'
import {
Stack,
Group,
Text,
Loader,
Center,
Alert,
Paper,
Table,
ScrollArea,
Switch,
Divider,
Badge,
} from '@mantine/core'
import { useDsmrLatest } from './hooks'
import { formatLocalDateTime } from '../utils/datetime'
/** Default auto-refresh interval (ms) — DSMR ingest stores ~one row per 10 s. */
const AUTO_REFRESH_INTERVAL_MS = 10_000
function formatValue(value: unknown): string {
if (value === null || value === undefined) return '—'
if (typeof value === 'object') return JSON.stringify(value)
return String(value)
}
export function DsmrPanel() {
const [autoRefresh, setAutoRefresh] = useState(true)
const latestQuery = useDsmrLatest(
autoRefresh ? { refetchIntervalMs: AUTO_REFRESH_INTERVAL_MS } : undefined,
)
return (
<Stack gap="md" data-testid="dsmr-panel">
<Group justify="space-between" align="center">
<div>
<Text fw={600}>Latest DSMR reading</Text>
<Text size="xs" c="dimmed">
The most recent parsed telegram persisted to <code>dsmr_reading</code>.
</Text>
</div>
<Switch
label="Auto-refresh"
checked={autoRefresh}
onChange={(e) => setAutoRefresh(e.currentTarget.checked)}
size="sm"
data-testid="dsmr-auto-refresh-switch"
/>
</Group>
<Divider />
<DsmrContent
isLoading={latestQuery.isLoading}
isError={latestQuery.isError}
data={latestQuery.data}
/>
</Stack>
)
}
interface DsmrContentProps {
isLoading: boolean
isError: boolean
data:
| { found: boolean; recorded_at?: string | null; payload?: Record<string, unknown> | null }
| undefined
}
function DsmrContent({ isLoading, isError, data }: DsmrContentProps) {
if (isLoading) {
return (
<Center py="xl" data-testid="dsmr-loading">
<Loader />
</Center>
)
}
if (isError || !data) {
return (
<Alert color="red" data-testid="dsmr-error">
Failed to load the latest DSMR reading.
</Alert>
)
}
if (!data.found || !data.payload) {
return (
<Alert color="gray" data-testid="dsmr-empty">
No DSMR data yet. Enable <strong>DSMR ingest</strong> in Config, make sure MQTT
is connected, and confirm the DSMR Reader is publishing to the configured topic
(default <code>dsmr/json</code>). Rows are stored about once every 10 seconds.
</Alert>
)
}
const payload = data.payload
const keys = Object.keys(payload).sort()
return (
<Stack gap="sm" data-testid="dsmr-data">
<Group gap="xs">
<Badge color="teal" variant="light">
{keys.length} fields
</Badge>
{data.recorded_at && (
<Text size="sm" c="dimmed" data-testid="dsmr-recorded-at">
recorded at {formatLocalDateTime(data.recorded_at)}
</Text>
)}
</Group>
<Paper withBorder>
<ScrollArea.Autosize mah={480}>
<Table striped highlightOnHover withColumnBorders data-testid="dsmr-table">
<Table.Thead>
<Table.Tr>
<Table.Th>Field</Table.Th>
<Table.Th>Value</Table.Th>
</Table.Tr>
</Table.Thead>
<Table.Tbody>
{keys.map((key) => (
<Table.Tr key={key} data-testid={`dsmr-row-${key}`}>
<Table.Td>
<Text size="sm" ff="monospace">
{key}
</Text>
</Table.Td>
<Table.Td>
<Text size="sm">{formatValue(payload[key])}</Text>
</Table.Td>
</Table.Tr>
))}
</Table.Tbody>
</Table>
</ScrollArea.Autosize>
</Paper>
</Stack>
)
}
+375
View File
@@ -0,0 +1,375 @@
/**
* Tests for energy/EnergyCharts.tsx
*
* Coverage:
* 1. Renders chart title with device name.
* 2. Shows loading state while readings/metrics are loading.
* 3. Shows error state when either query fails.
* 4. Shows empty state when no readings are in the time window.
* 5. Renders chart container when data is available.
* 6. Tolerates missing keys in payload (no crash when key absent).
* 7. Time-range segmented control is rendered.
*
* NOTE: Recharts itself is NOT mocked we let it render (jsdom-compatible
* rendering), but we only assert on data-testid wrappers, not Recharts internals.
* Recharts SVG rendering may produce warnings in jsdom; these are expected and
* benign (jsdom lacks ResizeObserver / SVG layout).
*/
import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest'
import { screen, waitFor } from '@testing-library/react'
import { renderWithProviders } from '../test-utils'
import { EnergyCharts } from './EnergyCharts'
// ---------------------------------------------------------------------------
// Fixtures
// ---------------------------------------------------------------------------
const UUID = 'device-uuid-test'
const DEVICE_NAME = 'SDM120 Main'
const METRICS_RESP = {
profile: 'sdm120',
metrics: [
{ key: 'voltage', label: 'Voltage', unit: 'V', device_class: 'voltage' },
{ key: 'current', label: 'Current', unit: 'A', device_class: 'current' },
],
}
const READING = {
recorded_at: '2026-06-22T10:00:00Z',
payload: { voltage: 230.2, current: 1.3 },
}
const READINGS_RESP = {
items: [READING],
}
// ---------------------------------------------------------------------------
// Mock apiClient
// ---------------------------------------------------------------------------
const mockGet = vi.fn()
vi.mock('../api/client', () => ({
default: {
GET: (...args: unknown[]) => mockGet(...args),
POST: vi.fn(),
PATCH: vi.fn(),
DELETE: vi.fn(),
},
ApiError: class ApiError extends Error {
status: number
body: unknown
constructor(status: number, body: unknown) {
super(`API error ${status}`)
this.name = 'ApiError'
this.status = status
this.body = body
}
},
registerLoginRedirect: vi.fn(),
}))
// ---------------------------------------------------------------------------
// Helpers
// ---------------------------------------------------------------------------
function setupDefaultMocks() {
mockGet.mockImplementation((path: string) => {
if (path === '/api/modbus/devices/{uuid}/metrics') {
return Promise.resolve({ data: METRICS_RESP })
}
if (path === '/api/modbus/devices/{uuid}/readings') {
return Promise.resolve({ data: READINGS_RESP })
}
return Promise.resolve({ data: null })
})
}
function renderChart() {
return renderWithProviders(
<EnergyCharts uuid={UUID} deviceName={DEVICE_NAME} />,
{ initialPath: '/energy' },
)
}
// ---------------------------------------------------------------------------
// Tests
// ---------------------------------------------------------------------------
describe('EnergyCharts — rendering', () => {
beforeEach(() => {
vi.clearAllMocks()
setupDefaultMocks()
})
it('renders chart title with device name', async () => {
renderChart()
await waitFor(() => {
expect(screen.getByTestId(`energy-charts-title-${UUID}`)).toBeInTheDocument()
})
expect(screen.getByTestId(`energy-charts-title-${UUID}`).textContent).toBe(DEVICE_NAME)
})
it('renders time-range segmented control', async () => {
renderChart()
await waitFor(() => {
expect(screen.getByTestId(`energy-charts-preset-${UUID}`)).toBeInTheDocument()
})
})
it('shows loading state while data is being fetched', () => {
// Never resolve
mockGet.mockReturnValue(new Promise(() => {}))
renderChart()
expect(screen.getByTestId(`energy-charts-loading-${UUID}`)).toBeInTheDocument()
})
it('shows error state when readings query fails', async () => {
mockGet.mockRejectedValue(new Error('Network error'))
renderChart()
await waitFor(() => {
expect(screen.getByTestId(`energy-charts-error-${UUID}`)).toBeInTheDocument()
})
})
it('shows empty state when no readings in time window', async () => {
mockGet.mockImplementation((path: string) => {
if (path === '/api/modbus/devices/{uuid}/metrics') {
return Promise.resolve({ data: METRICS_RESP })
}
if (path === '/api/modbus/devices/{uuid}/readings') {
return Promise.resolve({ data: { items: [] } })
}
return Promise.resolve({ data: null })
})
renderChart()
await waitFor(() => {
expect(screen.getByTestId(`energy-charts-empty-${UUID}`)).toBeInTheDocument()
})
})
it('renders chart container when data is available', async () => {
renderChart()
await waitFor(() => {
expect(screen.getByTestId(`energy-charts-chart-${UUID}`)).toBeInTheDocument()
})
})
})
describe('EnergyCharts — truncation notice', () => {
beforeEach(() => vi.clearAllMocks())
it('shows truncation notice when readings count equals the limit cap (1000)', async () => {
// Build exactly 1000 synthetic readings to trigger the truncation hint.
const truncatedItems = Array.from({ length: 1000 }, (_, i) => ({
recorded_at: new Date(Date.now() - (999 - i) * 5000).toISOString(),
payload: { voltage: 230 + i * 0.01 },
}))
mockGet.mockImplementation((path: string) => {
if (path === '/api/modbus/devices/{uuid}/metrics') {
return Promise.resolve({ data: METRICS_RESP })
}
if (path === '/api/modbus/devices/{uuid}/readings') {
return Promise.resolve({ data: { items: truncatedItems } })
}
return Promise.resolve({ data: null })
})
renderChart()
await waitFor(() => {
expect(
screen.getByTestId(`energy-charts-truncated-${UUID}`),
).toBeInTheDocument()
})
})
it('does NOT show truncation notice when readings count is below limit', async () => {
// Default mock has exactly 1 item — well below 1000.
setupDefaultMocks()
renderChart()
await waitFor(() => {
expect(screen.getByTestId(`energy-charts-chart-${UUID}`)).toBeInTheDocument()
})
expect(screen.queryByTestId(`energy-charts-truncated-${UUID}`)).not.toBeInTheDocument()
})
})
describe('EnergyCharts — payload key tolerance', () => {
beforeEach(() => vi.clearAllMocks())
it('does not crash when payload is missing a metric key', async () => {
mockGet.mockImplementation((path: string) => {
if (path === '/api/modbus/devices/{uuid}/metrics') {
return Promise.resolve({ data: METRICS_RESP })
}
if (path === '/api/modbus/devices/{uuid}/readings') {
// payload is missing 'current' key
return Promise.resolve({
data: { items: [{ recorded_at: '2026-06-22T10:00:00Z', payload: { voltage: 230.2 } }] },
})
}
return Promise.resolve({ data: null })
})
// Should render without throwing
expect(() => renderChart()).not.toThrow()
await waitFor(() => {
expect(screen.getByTestId(`energy-charts-${UUID}`)).toBeInTheDocument()
})
})
it('does not crash when payload is entirely empty', async () => {
mockGet.mockImplementation((path: string) => {
if (path === '/api/modbus/devices/{uuid}/metrics') {
return Promise.resolve({ data: METRICS_RESP })
}
if (path === '/api/modbus/devices/{uuid}/readings') {
return Promise.resolve({
data: { items: [{ recorded_at: '2026-06-22T10:00:00Z', payload: {} }] },
})
}
return Promise.resolve({ data: null })
})
expect(() => renderChart()).not.toThrow()
// All values null → empty state (no chart) or chart with no data points shown.
await waitFor(() => {
expect(screen.getByTestId(`energy-charts-${UUID}`)).toBeInTheDocument()
})
})
})
// ---------------------------------------------------------------------------
// Auto-refresh: rolling window (A2 — the core bug fix)
// ---------------------------------------------------------------------------
describe('EnergyCharts — auto-refresh rolling window', () => {
beforeEach(() => {
vi.clearAllMocks()
})
afterEach(() => {
vi.clearAllMocks()
})
/**
* EnergyCharts now passes spanMs (not fixed start/end) to useReadings.
* This means each queryFn invocation computes end=now(), giving a rolling
* window. We verify here that:
* - The readings GET is called with an `end` param (window is computed).
* - Rendering twice (to simulate a second fetch via queryFn being called
* again) results in a second GET with a later `end` value.
*
* We test this by manually invoking the apiClient.GET mock twice with a
* small real-time delay between them, mimicking what happens when
* refetchInterval fires and the queryFn re-runs.
*/
it('uses spanMs-based rolling window so each queryFn call gets a fresh end=now()', async () => {
const endTimestamps: string[] = []
mockGet.mockImplementation((path: string, opts: { params?: { query?: { end?: string } } } = {}) => {
if (path === '/api/modbus/devices/{uuid}/metrics') {
return Promise.resolve({ data: METRICS_RESP })
}
if (path === '/api/modbus/devices/{uuid}/readings') {
const end = opts?.params?.query?.end
if (end) endTimestamps.push(end)
return Promise.resolve({ data: READINGS_RESP })
}
return Promise.resolve({ data: null })
})
// First render: triggers initial fetch (end=now at T0).
const t0 = Date.now()
renderWithProviders(
<EnergyCharts uuid={UUID} deviceName={DEVICE_NAME} />,
{ initialPath: '/energy' },
)
// Wait for the initial readings GET to fire.
await waitFor(() => expect(endTimestamps.length).toBeGreaterThanOrEqual(1))
// The recorded `end` must not be before the test started.
const firstEndMs = new Date(endTimestamps[0]).getTime()
expect(firstEndMs).toBeGreaterThanOrEqual(t0)
// The readings request must include an `end` query param (window is computed, not omitted).
expect(endTimestamps[0]).toBeTruthy()
})
it('does not freeze the window on mount — window end advances across separate renders', async () => {
// We simulate two independent hook invocations (as if refetch fired).
// Since each queryFn call uses new Date(), the end timestamps will advance
// naturally with real time. We just need to verify the concept holds.
const endTimestamps: string[] = []
mockGet.mockImplementation((path: string, opts: { params?: { query?: { end?: string } } } = {}) => {
if (path === '/api/modbus/devices/{uuid}/metrics') {
return Promise.resolve({ data: METRICS_RESP })
}
if (path === '/api/modbus/devices/{uuid}/readings') {
const end = opts?.params?.query?.end
if (end) endTimestamps.push(end)
return Promise.resolve({ data: READINGS_RESP })
}
return Promise.resolve({ data: null })
})
const { unmount } = renderWithProviders(
<EnergyCharts uuid={UUID} deviceName={DEVICE_NAME} />,
{ initialPath: '/energy' },
)
await waitFor(() => expect(endTimestamps.length).toBeGreaterThanOrEqual(1))
const firstEnd = new Date(endTimestamps[0]).getTime()
// Unmount and re-mount to simulate a new queryFn invocation after a tick.
unmount()
vi.clearAllMocks()
endTimestamps.length = 0
// Small real-time delay to ensure new Date() will be >= first.
await new Promise((resolve) => setTimeout(resolve, 5))
mockGet.mockImplementation((path: string, opts: { params?: { query?: { end?: string } } } = {}) => {
if (path === '/api/modbus/devices/{uuid}/metrics') {
return Promise.resolve({ data: METRICS_RESP })
}
if (path === '/api/modbus/devices/{uuid}/readings') {
const end = opts?.params?.query?.end
if (end) endTimestamps.push(end)
return Promise.resolve({ data: READINGS_RESP })
}
return Promise.resolve({ data: null })
})
renderWithProviders(
<EnergyCharts uuid={UUID} deviceName={DEVICE_NAME} />,
{ initialPath: '/energy' },
)
await waitFor(() => expect(endTimestamps.length).toBeGreaterThanOrEqual(1))
const secondEnd = new Date(endTimestamps[0]).getTime()
// The second invocation's end must be >= the first — window is not frozen.
expect(secondEnd).toBeGreaterThanOrEqual(firstEnd)
})
})
+322
View File
@@ -0,0 +1,322 @@
/**
* EnergyCharts self-contained Recharts wrapper for Modbus device trend charts.
*
* Design decisions (M5 decision 11):
* - Recharts is imported ONLY in this file (isolation, easy to swap later).
* - The component is fully self-contained: it owns its own data fetching via
* useReadings / useMetrics hooks and renders loading/error/empty states.
* - Time-range selection is internal; readings are always fetched with a window
* + limit cap never a full-table pull.
* - Metric labels and units come from GET /metrics; missing keys in payload are
* tolerated (a null data point is emitted so the line simply has a gap).
*/
// ---------------------------------------------------------------------------
// Recharts imports — keep ALL recharts imports inside this file.
// ---------------------------------------------------------------------------
import {
LineChart,
Line,
XAxis,
YAxis,
CartesianGrid,
Tooltip as RechartsTooltip,
Legend,
ResponsiveContainer,
} from 'recharts'
import { useState, useMemo } from 'react'
import {
Stack,
Group,
Text,
Loader,
Alert,
SegmentedControl,
Paper,
Title,
Badge,
Box,
} from '@mantine/core'
import { useReadings, useMetrics } from './hooks'
import type { MetricInfo, AutoRefreshOptions } from './hooks'
import { formatMetricValue } from './format'
import { formatLocalTime, formatLocalDateTime } from '../utils/datetime'
// ---------------------------------------------------------------------------
// Constants
// ---------------------------------------------------------------------------
/** Must match the limit passed to useReadings below. */
const CHART_READINGS_LIMIT = 1000
// ---------------------------------------------------------------------------
// Types
// ---------------------------------------------------------------------------
interface EnergyChartsProps {
/** UUID of the Modbus device to chart. */
uuid: string
/** Friendly name for the chart title. */
deviceName: string
/** Auto-refresh options forwarded to useReadings. */
autoRefresh?: AutoRefreshOptions
}
// ---------------------------------------------------------------------------
// Time-range presets
// ---------------------------------------------------------------------------
interface TimePreset {
label: string
value: string
/** How many milliseconds of history to show. */
spanMs: number
}
const TIME_PRESETS: TimePreset[] = [
{ label: '1 h', value: '1h', spanMs: 60 * 60 * 1000 },
{ label: '6 h', value: '6h', spanMs: 6 * 60 * 60 * 1000 },
{ label: '24 h', value: '24h', spanMs: 24 * 60 * 60 * 1000 },
]
const DEFAULT_PRESET = '1h'
// ---------------------------------------------------------------------------
// Metric colour palette — cycles through a fixed set
// ---------------------------------------------------------------------------
const LINE_COLORS = [
'#4c9cdb',
'#f59f00',
'#51cf66',
'#f03e3e',
'#cc5de8',
'#20c997',
'#fd7e14',
'#74c0fc',
]
function lineColor(index: number): string {
return LINE_COLORS[index % LINE_COLORS.length]
}
// ---------------------------------------------------------------------------
// Helper: format a recorded_at timestamp for the X-axis tick
// ---------------------------------------------------------------------------
function formatTimeTick(isoString: string): string {
return formatLocalTime(isoString)
}
// ---------------------------------------------------------------------------
// Helper: safe numeric read from payload — returns null if key absent/non-numeric
// ---------------------------------------------------------------------------
function safePayloadValue(
payload: Record<string, unknown> | null | undefined,
key: string,
): number | null {
if (!payload) return null
const raw = payload[key]
if (raw === null || raw === undefined) return null
const n = Number(raw)
return Number.isFinite(n) ? n : null
}
// ---------------------------------------------------------------------------
// EnergyCharts component
// ---------------------------------------------------------------------------
export function EnergyCharts({ uuid, deviceName, autoRefresh }: EnergyChartsProps) {
const [activePreset, setActivePreset] = useState<string>(DEFAULT_PRESET)
const selectedPreset = TIME_PRESETS.find((p) => p.value === activePreset) ?? TIME_PRESETS[0]
// Pass spanMs rather than fixed start/end so that every refetchInterval-triggered
// queryFn call computes end=now() and rolls the window forward — new readings
// recorded after mount will appear without a manual page refresh.
// Switching presets changes spanMs → different queryKey → immediate re-fetch.
const readingsQuery = useReadings(
uuid,
{ spanMs: selectedPreset.spanMs, limit: CHART_READINGS_LIMIT },
autoRefresh,
)
const metricsQuery = useMetrics(uuid)
// -------------------------------------------------------------------------
// Derive chart data: [{recorded_at, voltage: 230.2, current: 1.3, ...}, ...]
// -------------------------------------------------------------------------
const metricsData = metricsQuery.data?.metrics
const chartData = useMemo(() => {
const metrics: MetricInfo[] = metricsData ?? []
const readings = readingsQuery.data?.items ?? []
return readings.map((row) => {
const point: Record<string, string | number | null> = {
recorded_at: row.recorded_at,
}
for (const m of metrics) {
// Tolerate missing keys — null produces a gap in the line, not a crash.
point[m.key] = safePayloadValue(row.payload as Record<string, unknown>, m.key)
}
return point
})
}, [readingsQuery.data, metricsData])
const metrics: MetricInfo[] = metricsQuery.data?.metrics ?? []
// -------------------------------------------------------------------------
// States: loading / error / empty
// -------------------------------------------------------------------------
const isLoading = readingsQuery.isLoading || metricsQuery.isLoading
const isError = readingsQuery.isError || metricsQuery.isError
/**
* True when the returned row count equals the limit cap, meaning the window
* contains more data than was fetched. The backend returns the most-recent N
* rows in this case, so the chart shows the latest segment but a hint is
* shown so the user knows the full window is not displayed.
*/
const isTruncated =
!isLoading &&
!isError &&
(readingsQuery.data?.items.length ?? 0) >= CHART_READINGS_LIMIT
return (
<Paper withBorder p="md" data-testid={`energy-charts-${uuid}`}>
<Stack gap="sm">
{/* Header row */}
<Group justify="space-between" align="center" wrap="nowrap">
<Title order={4} data-testid={`energy-charts-title-${uuid}`}>
{deviceName}
</Title>
<SegmentedControl
size="xs"
value={activePreset}
onChange={setActivePreset}
data={TIME_PRESETS.map((p) => ({ value: p.value, label: p.label }))}
data-testid={`energy-charts-preset-${uuid}`}
/>
</Group>
{/* Loading */}
{isLoading && (
<Group justify="center" py="lg" data-testid={`energy-charts-loading-${uuid}`}>
<Loader size="sm" />
<Text size="sm" c="dimmed">
Loading readings
</Text>
</Group>
)}
{/* Error */}
{!isLoading && isError && (
<Alert color="red" data-testid={`energy-charts-error-${uuid}`}>
Failed to load readings or metrics. Please try again.
</Alert>
)}
{/* Empty */}
{!isLoading && !isError && chartData.length === 0 && (
<Text
size="sm"
c="dimmed"
ta="center"
py="lg"
data-testid={`energy-charts-empty-${uuid}`}
>
No readings in this time window.
</Text>
)}
{/* Truncation notice — shown when window has more data than the fetch limit */}
{isTruncated && (
<Text
size="xs"
c="dimmed"
ta="right"
data-testid={`energy-charts-truncated-${uuid}`}
>
Showing the most recent {CHART_READINGS_LIMIT} readings; full long-range trend pending downsampling
</Text>
)}
{/* Chart */}
{!isLoading && !isError && chartData.length > 0 && (
<Box data-testid={`energy-charts-chart-${uuid}`}>
{/* Render one chart per metric for clarity (avoids mixed Y-axis units) */}
{metrics.map((metric, idx) => {
// Check if this metric has any non-null data points; skip if all null.
const hasData = chartData.some((pt) => pt[metric.key] !== null)
if (!hasData) return null
return (
<Box key={metric.key} mb="md">
<Group gap="xs" mb={4}>
<Text size="xs" fw={600} c="dimmed">
{metric.label}
</Text>
{metric.unit && (
<Badge size="xs" variant="outline" color="gray">
{metric.unit}
</Badge>
)}
</Group>
<ResponsiveContainer width="100%" height={160}>
<LineChart
data={chartData}
margin={{ top: 4, right: 8, left: 0, bottom: 0 }}
>
<CartesianGrid strokeDasharray="3 3" opacity={0.4} />
<XAxis
dataKey="recorded_at"
tickFormatter={formatTimeTick}
tick={{ fontSize: 10 }}
minTickGap={40}
/>
<YAxis
tick={{ fontSize: 10 }}
width={50}
tickFormatter={(v: number) =>
metric.unit ? `${v} ${metric.unit}` : String(v)
}
/>
<RechartsTooltip
formatter={(value) => {
const formatted = formatMetricValue(value, metric)
if (formatted === '—') return ['—', metric.label] as [string, string]
return [
`${formatted}${metric.unit ? ' ' + metric.unit : ''}`,
metric.label,
] as [string, string]
}}
labelFormatter={(label) => {
return formatLocalDateTime(String(label))
}}
/>
<Legend />
<Line
type="monotone"
dataKey={metric.key}
name={metric.label}
stroke={lineColor(idx)}
dot={false}
isAnimationActive={false}
connectNulls={false}
/>
</LineChart>
</ResponsiveContainer>
</Box>
)
})}
</Box>
)}
</Stack>
</Paper>
)
}
+446
View File
@@ -0,0 +1,446 @@
/**
* Tests for MeterManager component.
*
* Coverage:
* 1. Loading state rendering.
* 2. Error state rendering.
* 3. Empty state when no meters exist.
* 4. Meter timeline list rendering (label, dates, active badge, reason).
* 5. "Declare New Meter" button opens form modal.
* 6. Declare meter form submit calls POST /api/energy/meters.
* 7. Declare meter 422 () error is displayed.
* 8. Edit button opens edit form modal.
* 9. Edit meter saves label/note via PATCH /api/energy/meters/{meter_id}.
* 10. Edit meter retroactive started_at triggers recompute notice.
*/
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { screen, waitFor } from '@testing-library/react'
import userEvent from '@testing-library/user-event'
import { renderWithProviders } from '../test-utils'
import { MeterManager } from './MeterManager'
// ---------------------------------------------------------------------------
// Mock apiClient
// ---------------------------------------------------------------------------
const mockGet = vi.fn()
const mockPost = vi.fn()
const mockPatch = vi.fn()
vi.mock('../api/client', () => ({
default: {
GET: (...args: unknown[]) => mockGet(...args),
POST: (...args: unknown[]) => mockPost(...args),
PATCH: (...args: unknown[]) => mockPatch(...args),
DELETE: vi.fn(),
},
ApiError: class ApiError extends Error {
status: number
body: unknown
constructor(status: number, body: unknown) {
super(`API error ${status}`)
this.name = 'ApiError'
this.status = status
this.body = body
}
},
registerLoginRedirect: vi.fn(),
}))
// ---------------------------------------------------------------------------
// Fixtures
// ---------------------------------------------------------------------------
const ACTIVE_METER = {
id: 1,
label: 'Initial 2G meter',
commodity: 'electricity',
started_at: '2024-01-15T00:00:00Z',
ended_at: null,
reason: 'initial',
note: null,
created_at: '2024-01-15T00:00:00Z',
}
const CLOSED_METER = {
id: 2,
label: 'Old 4G meter',
commodity: 'electricity',
started_at: '2023-06-01T00:00:00Z',
ended_at: '2024-01-15T00:00:00Z',
reason: 'meter_swap',
note: 'Replaced by grid company',
created_at: '2023-06-01T00:00:00Z',
}
const METERS_RESPONSE = {
items: [CLOSED_METER, ACTIVE_METER],
total: 2,
}
// ---------------------------------------------------------------------------
// Tests
// ---------------------------------------------------------------------------
describe('MeterManager — loading / error / empty states', () => {
beforeEach(() => vi.clearAllMocks())
it('renders loading state initially', () => {
mockGet.mockImplementation(() => new Promise(() => {}))
renderWithProviders(<MeterManager />)
expect(screen.getByTestId('meters-loading')).toBeInTheDocument()
})
it('renders error state when GET fails', async () => {
mockGet.mockRejectedValue(new Error('Network error'))
renderWithProviders(<MeterManager />)
await waitFor(() => {
expect(screen.getByTestId('meters-load-error')).toBeInTheDocument()
})
})
it('renders empty state when no meters exist', async () => {
mockGet.mockResolvedValue({ data: { items: [], total: 0 } })
renderWithProviders(<MeterManager />)
await waitFor(() => {
expect(screen.getByTestId('meters-empty')).toBeInTheDocument()
})
})
})
describe('MeterManager — meter list', () => {
beforeEach(() => vi.clearAllMocks())
it('renders meter timeline with label, dates, status badge, reason', async () => {
mockGet.mockResolvedValue({ data: METERS_RESPONSE })
renderWithProviders(<MeterManager />)
await waitFor(() => {
expect(screen.getByTestId('meters-table')).toBeInTheDocument()
})
// Labels
expect(screen.getByText('Initial 2G meter')).toBeInTheDocument()
expect(screen.getByText('Old 4G meter')).toBeInTheDocument()
// Status badges
expect(screen.getByTestId(`meter-status-${ACTIVE_METER.id}`)).toHaveTextContent('active')
expect(screen.getByTestId(`meter-status-${CLOSED_METER.id}`)).toHaveTextContent('closed')
// Reason badges
expect(screen.getByText('initial')).toBeInTheDocument()
expect(screen.getByText('meter_swap')).toBeInTheDocument()
})
it('renders "Declare New Meter" button', async () => {
mockGet.mockResolvedValue({ data: METERS_RESPONSE })
renderWithProviders(<MeterManager />)
await waitFor(() => {
expect(screen.getByTestId('meter-declare-button')).toBeInTheDocument()
})
})
})
describe('MeterManager — declare new meter', () => {
beforeEach(() => vi.clearAllMocks())
it('opens declare modal when "Declare New Meter" is clicked', async () => {
const user = userEvent.setup()
mockGet.mockResolvedValue({ data: { items: [], total: 0 } })
renderWithProviders(<MeterManager />)
await waitFor(() => expect(screen.getByTestId('meter-declare-button')).toBeInTheDocument())
await user.click(screen.getByTestId('meter-declare-button'))
await waitFor(() => {
expect(screen.getByTestId('declare-meter-modal')).toBeInTheDocument()
})
expect(screen.getByTestId('declare-meter-form')).toBeInTheDocument()
})
it('calls POST /api/energy/meters with correct payload including local-midnight naive datetime', async () => {
const user = userEvent.setup()
mockGet.mockResolvedValue({ data: { items: [], total: 0 } })
mockPost.mockResolvedValue({ data: ACTIVE_METER })
renderWithProviders(<MeterManager />)
await waitFor(() => expect(screen.getByTestId('meter-declare-button')).toBeInTheDocument())
await user.click(screen.getByTestId('meter-declare-button'))
await waitFor(() => expect(screen.getByTestId('declare-meter-form')).toBeInTheDocument())
// Fill form
await user.type(screen.getByTestId('meter-label'), 'New meter label')
await user.type(screen.getByTestId('meter-started-at'), '2026-01-01')
// Select reason via the combobox (Mantine Select renders a combobox)
await user.click(screen.getByTestId('meter-reason'))
await waitFor(() => screen.getByText('Initial installation'))
await user.click(screen.getByText('Initial installation'))
await user.click(screen.getByTestId('declare-meter-submit'))
await waitFor(() => {
expect(mockPost).toHaveBeenCalledWith(
'/api/energy/meters',
expect.objectContaining({
body: expect.objectContaining({
label: 'New meter label',
// FU10 local-midnight naive convention: no Z suffix
started_at: '2026-01-01T00:00:00',
reason: 'initial',
commodity: 'electricity',
}),
}),
)
})
})
it('displays error when POST fails with 422 (倒挂 / validation error)', async () => {
const user = userEvent.setup()
mockGet.mockResolvedValue({ data: { items: [ACTIVE_METER], total: 1 } })
const { ApiError } = await import('../api/client')
mockPost.mockRejectedValue(
new ApiError(422, { detail: 'started_at must be ≥ current active meter started_at' }),
)
renderWithProviders(<MeterManager />)
await waitFor(() => expect(screen.getByTestId('meter-declare-button')).toBeInTheDocument())
await user.click(screen.getByTestId('meter-declare-button'))
await waitFor(() => expect(screen.getByTestId('declare-meter-form')).toBeInTheDocument())
await user.type(screen.getByTestId('meter-label'), 'Bad meter')
await user.type(screen.getByTestId('meter-started-at'), '2020-01-01')
await user.click(screen.getByTestId('meter-reason'))
await waitFor(() => screen.getByText('Meter swap (same address)'))
await user.click(screen.getByText('Meter swap (same address)'))
await user.click(screen.getByTestId('declare-meter-submit'))
await waitFor(() => {
expect(screen.getByTestId('declare-meter-error')).toBeInTheDocument()
})
expect(screen.getByTestId('declare-meter-error').textContent).toContain(
'started_at must be ≥',
)
})
it('cancel button closes the declare modal', async () => {
const user = userEvent.setup()
mockGet.mockResolvedValue({ data: { items: [], total: 0 } })
renderWithProviders(<MeterManager />)
await waitFor(() => expect(screen.getByTestId('meter-declare-button')).toBeInTheDocument())
await user.click(screen.getByTestId('meter-declare-button'))
await waitFor(() => expect(screen.getByTestId('declare-meter-modal')).toBeInTheDocument())
await user.click(screen.getByTestId('declare-meter-cancel'))
await waitFor(() => {
expect(screen.queryByTestId('declare-meter-modal')).not.toBeInTheDocument()
})
})
})
describe('MeterManager — edit meter date initialisation', () => {
beforeEach(() => vi.clearAllMocks())
it('initialises date input from started_at (Z-suffix, UTC midnight → local date)', async () => {
// ACTIVE_METER.started_at = '2024-01-15T00:00:00Z' (UTC midnight).
// The test suite is pinned to TZ=UTC (via vite.config.ts test.env), so
// the local date is deterministically '2024-01-15' on any CI runner.
const user = userEvent.setup()
mockGet.mockResolvedValue({ data: { items: [ACTIVE_METER], total: 1 } })
renderWithProviders(<MeterManager />)
await waitFor(() => expect(screen.getByTestId(`meter-edit-${ACTIVE_METER.id}`)).toBeInTheDocument())
await user.click(screen.getByTestId(`meter-edit-${ACTIVE_METER.id}`))
await waitFor(() => expect(screen.getByTestId('edit-meter-form')).toBeInTheDocument())
const dateInput = screen.getByTestId('edit-meter-started-at') as HTMLInputElement
expect(dateInput.value).toBe('2024-01-15')
})
it('initialises date input from started_at (naive, no tz marker)', async () => {
// A naive timestamp without timezone marker — parseBackendTimestamp appends 'Z'
// so it is treated as UTC. With TZ=UTC (pinned in vite.config.ts), the local date
// equals the UTC date exactly.
const naiveMeter = { ...ACTIVE_METER, started_at: '2024-03-20T00:00:00' }
const user = userEvent.setup()
mockGet.mockResolvedValue({ data: { items: [naiveMeter], total: 1 } })
renderWithProviders(<MeterManager />)
await waitFor(() => expect(screen.getByTestId(`meter-edit-${naiveMeter.id}`)).toBeInTheDocument())
await user.click(screen.getByTestId(`meter-edit-${naiveMeter.id}`))
await waitFor(() => expect(screen.getByTestId('edit-meter-form')).toBeInTheDocument())
const dateInput = screen.getByTestId('edit-meter-started-at') as HTMLInputElement
expect(dateInput.value).toBe('2024-03-20')
})
it('initialises date input from started_at with explicit UTC offset (+02:00) — regression for old buggy regex', async () => {
// The old hand-written regex /[zZ+-]\d*$/ would fail to match '+02:00' (the ':00'
// suffix broke the pattern) and would incorrectly append 'Z', producing an Invalid Date.
// The new code uses parseBackendTimestamp which uses the correct TZ_MARKER_RE regex
// and handles explicit offsets properly.
const offsetMeter = { ...ACTIVE_METER, started_at: '2024-01-15T02:00:00+02:00' }
// UTC equivalent: 2024-01-15T00:00:00Z → with TZ=UTC (pinned) local date = '2024-01-15'
const user = userEvent.setup()
mockGet.mockResolvedValue({ data: { items: [offsetMeter], total: 1 } })
renderWithProviders(<MeterManager />)
await waitFor(() => expect(screen.getByTestId(`meter-edit-${offsetMeter.id}`)).toBeInTheDocument())
await user.click(screen.getByTestId(`meter-edit-${offsetMeter.id}`))
await waitFor(() => expect(screen.getByTestId('edit-meter-form')).toBeInTheDocument())
const dateInput = screen.getByTestId('edit-meter-started-at') as HTMLInputElement
// Must not be empty (which would indicate Invalid Date from the old buggy path)
expect(dateInput.value).not.toBe('')
expect(dateInput.value).toBe('2024-01-15')
})
it('does not include started_at in PATCH body when date is unchanged (round-trip idempotence)', async () => {
// Open the edit form and immediately submit without changing any fields except label.
// The date should be considered unchanged → no started_at in the PATCH body.
const user = userEvent.setup()
mockGet.mockResolvedValue({ data: { items: [ACTIVE_METER], total: 1 } })
mockPatch.mockResolvedValue({ data: { ...ACTIVE_METER, label: 'New label' } })
renderWithProviders(<MeterManager />)
await waitFor(() => expect(screen.getByTestId(`meter-edit-${ACTIVE_METER.id}`)).toBeInTheDocument())
await user.click(screen.getByTestId(`meter-edit-${ACTIVE_METER.id}`))
await waitFor(() => expect(screen.getByTestId('edit-meter-form')).toBeInTheDocument())
// Change only label; leave date untouched
const labelInput = screen.getByTestId('edit-meter-label')
await user.clear(labelInput)
await user.type(labelInput, 'New label')
await user.click(screen.getByTestId('edit-meter-submit'))
await waitFor(() => {
expect(mockPatch).toHaveBeenCalled()
})
const patchBody = mockPatch.mock.calls[0][1].body
expect(patchBody).not.toHaveProperty('started_at')
})
})
describe('MeterManager — edit meter', () => {
beforeEach(() => vi.clearAllMocks())
it('opens edit modal when Edit button is clicked', async () => {
const user = userEvent.setup()
mockGet.mockResolvedValue({ data: METERS_RESPONSE })
renderWithProviders(<MeterManager />)
await waitFor(() => expect(screen.getByTestId(`meter-edit-${ACTIVE_METER.id}`)).toBeInTheDocument())
await user.click(screen.getByTestId(`meter-edit-${ACTIVE_METER.id}`))
await waitFor(() => {
expect(screen.getByTestId('edit-meter-modal')).toBeInTheDocument()
})
expect(screen.getByTestId('edit-meter-form')).toBeInTheDocument()
})
it('calls PATCH /api/energy/meters/{meter_id} when label is changed', async () => {
const user = userEvent.setup()
mockGet.mockResolvedValue({ data: { items: [ACTIVE_METER], total: 1 } })
mockPatch.mockResolvedValue({ data: { ...ACTIVE_METER, label: 'Renamed meter' } })
renderWithProviders(<MeterManager />)
await waitFor(() => expect(screen.getByTestId(`meter-edit-${ACTIVE_METER.id}`)).toBeInTheDocument())
await user.click(screen.getByTestId(`meter-edit-${ACTIVE_METER.id}`))
await waitFor(() => expect(screen.getByTestId('edit-meter-form')).toBeInTheDocument())
// Clear label and type new one
const labelInput = screen.getByTestId('edit-meter-label')
await user.clear(labelInput)
await user.type(labelInput, 'Renamed meter')
await user.click(screen.getByTestId('edit-meter-submit'))
await waitFor(() => {
expect(mockPatch).toHaveBeenCalledWith(
'/api/energy/meters/{meter_id}',
expect.objectContaining({
params: { path: { meter_id: ACTIVE_METER.id } },
body: expect.objectContaining({ label: 'Renamed meter' }),
}),
)
})
})
it('shows recompute notice when started_at is changed (retroactive correction)', async () => {
const user = userEvent.setup()
mockGet.mockResolvedValue({ data: { items: [ACTIVE_METER], total: 1 } })
mockPatch.mockResolvedValue({ data: { ...ACTIVE_METER, started_at: '2024-02-01T00:00:00' } })
renderWithProviders(<MeterManager />)
await waitFor(() => expect(screen.getByTestId(`meter-edit-${ACTIVE_METER.id}`)).toBeInTheDocument())
await user.click(screen.getByTestId(`meter-edit-${ACTIVE_METER.id}`))
await waitFor(() => expect(screen.getByTestId('edit-meter-form')).toBeInTheDocument())
// Change the date field
const dateInput = screen.getByTestId('edit-meter-started-at')
await user.clear(dateInput)
await user.type(dateInput, '2024-02-01')
await user.click(screen.getByTestId('edit-meter-submit'))
await waitFor(() => {
expect(screen.getByTestId('meter-recompute-notice')).toBeInTheDocument()
})
})
it('displays error when PATCH fails', async () => {
const user = userEvent.setup()
mockGet.mockResolvedValue({ data: { items: [ACTIVE_METER], total: 1 } })
const { ApiError } = await import('../api/client')
mockPatch.mockRejectedValue(new ApiError(422, { detail: 'started_at conflict' }))
renderWithProviders(<MeterManager />)
await waitFor(() => expect(screen.getByTestId(`meter-edit-${ACTIVE_METER.id}`)).toBeInTheDocument())
await user.click(screen.getByTestId(`meter-edit-${ACTIVE_METER.id}`))
await waitFor(() => expect(screen.getByTestId('edit-meter-form')).toBeInTheDocument())
// Change label so there's something to patch
const labelInput = screen.getByTestId('edit-meter-label')
await user.clear(labelInput)
await user.type(labelInput, 'Different label')
await user.click(screen.getByTestId('edit-meter-submit'))
await waitFor(() => {
expect(screen.getByTestId('edit-meter-error')).toBeInTheDocument()
})
expect(screen.getByTestId('edit-meter-error').textContent).toContain('started_at conflict')
})
})
+503
View File
@@ -0,0 +1,503 @@
/**
* MeterManager electricity meter timeline UI.
*
* Features:
* - Table of meter epochs: label / interval (started_at ended_at or "active") /
* active badge / reason.
* - "Declare New Meter" button: form with label + date (started_at) + reason +
* optional note. Sends local-midnight naive datetime per FU10 convention.
* - Edit modal: update label, note, or correct started_at (retroactive).
* - Retroactive feedback: if started_at is changed, a success notice mentions
* that affected billing periods have been recomputed.
* - Loading / error / empty states.
*/
import { useState } from 'react'
import {
Table,
Button,
Group,
Text,
Loader,
Center,
Alert,
Stack,
Badge,
ScrollArea,
Modal,
TextInput,
Textarea,
Select,
Notification,
} from '@mantine/core'
import {
useMeters,
useDeclareMeter,
useUpdateMeter,
type MeterResponse,
type MeterReason,
} from './hooks'
import { ApiError } from '../api/client'
import { formatLocalDate, parseBackendTimestamp } from '../utils/datetime'
// ---------------------------------------------------------------------------
// Helpers
// ---------------------------------------------------------------------------
const REASON_OPTIONS: { value: MeterReason; label: string }[] = [
{ value: 'initial', label: 'Initial installation' },
{ value: 'meter_swap', label: 'Meter swap (same address)' },
{ value: 'home_move', label: 'Home / address move' },
{ value: 'other', label: 'Other' },
]
/**
* Convert a local date string "YYYY-MM-DD" to a naive local-midnight datetime
* string (no Z suffix) following the FU10 / ContractForm convention.
* The backend interprets naive datetimes as server local wall-clock time.
*/
function toLocalMidnightNaive(dateStr: string): string {
return `${dateStr}T00:00:00`
}
/**
* Format a Date object as a "YYYY-MM-DD" string using the browser's local timezone.
* Used to populate <input type="date"> fields.
* Returns '' if the Date is invalid.
*/
function toLocalDateInputString(d: Date): string {
if (isNaN(d.getTime())) return ''
const y = d.getFullYear()
const m = String(d.getMonth() + 1).padStart(2, '0')
const day = String(d.getDate()).padStart(2, '0')
return `${y}-${m}-${day}`
}
// ---------------------------------------------------------------------------
// Declare meter form (modal)
// ---------------------------------------------------------------------------
interface DeclareMeterFormProps {
onClose: () => void
onSaved: () => void
}
function DeclareMeterForm({ onClose, onSaved }: DeclareMeterFormProps) {
const [label, setLabel] = useState('')
const [dateStr, setDateStr] = useState('')
const [reason, setReason] = useState<string | null>(null)
const [note, setNote] = useState('')
const [error, setError] = useState<string | null>(null)
const declareMutation = useDeclareMeter()
async function handleSubmit(e: React.FormEvent) {
e.preventDefault()
setError(null)
if (!label.trim()) {
setError('Label is required.')
return
}
if (!dateStr) {
setError('Start date is required.')
return
}
if (!reason) {
setError('Reason is required.')
return
}
try {
await declareMutation.mutateAsync({
label: label.trim(),
started_at: toLocalMidnightNaive(dateStr),
reason: reason as MeterReason,
note: note.trim() || undefined,
commodity: 'electricity',
})
onSaved()
onClose()
} catch (err) {
if (err instanceof ApiError) {
const detail = (err.body as { detail?: string } | null)?.detail
setError(detail ?? `Error ${err.status}: failed to declare meter.`)
} else {
setError('Failed to declare meter. Please try again.')
}
}
}
return (
<Modal
opened
onClose={onClose}
title="Declare New Meter"
size="md"
data-testid="declare-meter-modal"
>
<form onSubmit={handleSubmit} data-testid="declare-meter-form">
<Stack gap="sm">
<TextInput
label="Label"
description={'Human-readable identifier, e.g. "2G meter @ Dorpsstraat 1"'}
required
value={label}
onChange={(e) => setLabel(e.currentTarget.value)}
data-testid="meter-label"
/>
<TextInput
label="Start date"
description="Date in YYYY-MM-DD format (interpreted as local midnight)"
type="date"
required
value={dateStr}
onChange={(e) => setDateStr(e.currentTarget.value)}
data-testid="meter-started-at"
/>
<Select
label="Reason"
required
data={REASON_OPTIONS}
value={reason}
onChange={setReason}
data-testid="meter-reason"
/>
<Textarea
label="Note (optional)"
value={note}
onChange={(e) => setNote(e.currentTarget.value)}
autosize
minRows={2}
data-testid="meter-note"
/>
{error && (
<Alert color="red" data-testid="declare-meter-error">
{error}
</Alert>
)}
<Group justify="flex-end" gap="sm">
<Button
type="button"
variant="default"
onClick={onClose}
data-testid="declare-meter-cancel"
>
Cancel
</Button>
<Button
type="submit"
loading={declareMutation.isPending}
data-testid="declare-meter-submit"
>
Declare Meter
</Button>
</Group>
</Stack>
</form>
</Modal>
)
}
// ---------------------------------------------------------------------------
// Edit meter form (modal)
// ---------------------------------------------------------------------------
interface EditMeterFormProps {
meter: MeterResponse
onClose: () => void
onSaved: (retroactive: boolean) => void
}
function EditMeterForm({ meter, onClose, onSaved }: EditMeterFormProps) {
const [label, setLabel] = useState(meter.label)
const [note, setNote] = useState(meter.note ?? '')
// Convert started_at to a local date string for the <input type="date">.
// parseBackendTimestamp correctly handles naive strings (no tz marker → treated as UTC,
// matching the backend's storage convention), Z-suffixed strings, and strings with
// explicit offsets like +02:00. We then extract the local-timezone date components
// so the displayed date matches the local wall-clock date of the meter start.
const initialDateStr = toLocalDateInputString(parseBackendTimestamp(meter.started_at))
const [dateStr, setDateStr] = useState(initialDateStr)
const [error, setError] = useState<string | null>(null)
const updateMutation = useUpdateMeter()
// Detect if the user changed started_at (retroactive correction).
const startedAtChanged = dateStr !== initialDateStr
async function handleSubmit(e: React.FormEvent) {
e.preventDefault()
setError(null)
const patchBody: { label?: string | null; note?: string | null; started_at?: string | null } = {}
if (label.trim() !== meter.label) patchBody.label = label.trim()
const noteVal = note.trim() || null
if (noteVal !== meter.note) patchBody.note = noteVal
if (startedAtChanged && dateStr) {
patchBody.started_at = toLocalMidnightNaive(dateStr)
}
if (Object.keys(patchBody).length === 0) {
onClose()
return
}
try {
await updateMutation.mutateAsync({ id: meter.id, body: patchBody })
onSaved(startedAtChanged)
onClose()
} catch (err) {
if (err instanceof ApiError) {
const detail = (err.body as { detail?: string } | null)?.detail
setError(detail ?? `Error ${err.status}: failed to update meter.`)
} else {
setError('Failed to update meter. Please try again.')
}
}
}
return (
<Modal
opened
onClose={onClose}
title={`Edit Meter — ${meter.label}`}
size="md"
data-testid="edit-meter-modal"
>
<form onSubmit={handleSubmit} data-testid="edit-meter-form">
<Stack gap="sm">
<TextInput
label="Label"
required
value={label}
onChange={(e) => setLabel(e.currentTarget.value)}
data-testid="edit-meter-label"
/>
<TextInput
label="Start date"
description="Retroactive correction: shifts the epoch boundary and re-judges billing periods"
type="date"
value={dateStr}
onChange={(e) => setDateStr(e.currentTarget.value)}
data-testid="edit-meter-started-at"
/>
<Textarea
label="Note (optional)"
value={note}
onChange={(e) => setNote(e.currentTarget.value)}
autosize
minRows={2}
data-testid="edit-meter-note"
/>
{error && (
<Alert color="red" data-testid="edit-meter-error">
{error}
</Alert>
)}
<Group justify="flex-end" gap="sm">
<Button
type="button"
variant="default"
onClick={onClose}
data-testid="edit-meter-cancel"
>
Cancel
</Button>
<Button
type="submit"
loading={updateMutation.isPending}
data-testid="edit-meter-submit"
>
Save
</Button>
</Group>
</Stack>
</form>
</Modal>
)
}
// ---------------------------------------------------------------------------
// Meter timeline table
// ---------------------------------------------------------------------------
interface MeterTableProps {
meters: MeterResponse[]
onEdit: (meter: MeterResponse) => void
}
function MeterTable({ meters, onEdit }: MeterTableProps) {
if (meters.length === 0) {
return (
<Text c="dimmed" ta="center" size="sm" data-testid="meters-empty">
No meters declared yet. Click "Declare New Meter" to add one.
</Text>
)
}
return (
<ScrollArea>
<Table striped highlightOnHover withTableBorder data-testid="meters-table">
<Table.Thead>
<Table.Tr>
<Table.Th>Label</Table.Th>
<Table.Th>Commodity</Table.Th>
<Table.Th>From</Table.Th>
<Table.Th>To</Table.Th>
<Table.Th>Status</Table.Th>
<Table.Th>Reason</Table.Th>
<Table.Th style={{ textAlign: 'right' }}>Actions</Table.Th>
</Table.Tr>
</Table.Thead>
<Table.Tbody>
{meters.map((meter) => {
const isActive = meter.ended_at === null
return (
<Table.Tr key={meter.id} data-testid={`meter-row-${meter.id}`}>
<Table.Td>
<Text fw={500} size="sm">
{meter.label}
</Text>
</Table.Td>
<Table.Td>
<Badge variant="outline" size="sm">
{meter.commodity}
</Badge>
</Table.Td>
<Table.Td>
<Text size="xs" c="dimmed">
{formatLocalDate(meter.started_at)}
</Text>
</Table.Td>
<Table.Td>
<Text size="xs" c="dimmed">
{meter.ended_at ? formatLocalDate(meter.ended_at) : '—'}
</Text>
</Table.Td>
<Table.Td>
<Badge
color={isActive ? 'green' : 'gray'}
variant="light"
size="sm"
data-testid={`meter-status-${meter.id}`}
>
{isActive ? 'active' : 'closed'}
</Badge>
</Table.Td>
<Table.Td>
<Badge variant="outline" size="sm" color="blue">
{meter.reason}
</Badge>
</Table.Td>
<Table.Td>
<Group justify="flex-end" gap="xs">
<Button
size="xs"
variant="outline"
onClick={() => onEdit(meter)}
data-testid={`meter-edit-${meter.id}`}
>
Edit
</Button>
</Group>
</Table.Td>
</Table.Tr>
)
})}
</Table.Tbody>
</Table>
</ScrollArea>
)
}
// ---------------------------------------------------------------------------
// MeterManager — top-level
// ---------------------------------------------------------------------------
export function MeterManager() {
const metersQuery = useMeters()
const [showDeclareForm, setShowDeclareForm] = useState(false)
const [editMeter, setEditMeter] = useState<MeterResponse | null>(null)
const [recomputeNotice, setRecomputeNotice] = useState(false)
// ---------------------------------------------------------------------------
// Render states
// ---------------------------------------------------------------------------
if (metersQuery.isLoading) {
return (
<Center py="xl" data-testid="meters-loading">
<Loader />
</Center>
)
}
if (metersQuery.isError || !metersQuery.data) {
return (
<Alert color="red" data-testid="meters-load-error">
Failed to load meters. Please refresh.
</Alert>
)
}
const meters = metersQuery.data.items
return (
<Stack gap="lg" data-testid="meter-manager">
<Group justify="space-between" align="center">
<Text fw={500}>Electricity Meters</Text>
<Button
onClick={() => setShowDeclareForm(true)}
data-testid="meter-declare-button"
>
Declare New Meter
</Button>
</Group>
{recomputeNotice && (
<Notification
color="teal"
title="Billing periods recomputed"
onClose={() => setRecomputeNotice(false)}
data-testid="meter-recompute-notice"
>
The start date was corrected. Affected billing periods have been
re-judged and cost attributions updated.
</Notification>
)}
<MeterTable meters={meters} onEdit={(m) => setEditMeter(m)} />
{/* Declare new meter */}
{showDeclareForm && (
<DeclareMeterForm
onClose={() => setShowDeclareForm(false)}
onSaved={() => setShowDeclareForm(false)}
/>
)}
{/* Edit existing meter */}
{editMeter && (
<EditMeterForm
meter={editMeter}
onClose={() => setEditMeter(null)}
onSaved={(retroactive) => {
setEditMeter(null)
if (retroactive) setRecomputeNotice(true)
}}
/>
)}
</Stack>
)
}
+138
View File
@@ -0,0 +1,138 @@
/**
* Tests for TibberPrices component.
*
* Coverage:
* 1. Loading state.
* 2. Empty state (no active contract / no kind).
* 3. Renders tibber chart when tibber kind data is available.
* 4. Shows tariff table for manual kind.
*/
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { screen, waitFor } from '@testing-library/react'
import { renderWithProviders } from '../test-utils'
// ---------------------------------------------------------------------------
// Mock apiClient
// ---------------------------------------------------------------------------
const mockGet = vi.fn()
vi.mock('../api/client', () => ({
default: {
GET: (...args: unknown[]) => mockGet(...args),
POST: vi.fn(),
PATCH: vi.fn(),
DELETE: vi.fn(),
},
ApiError: class ApiError extends Error {
status: number
body: unknown
constructor(status: number, body: unknown) {
super(`API error ${status}`)
this.name = 'ApiError'
this.status = status
this.body = body
}
},
registerLoginRedirect: vi.fn(),
}))
// ---------------------------------------------------------------------------
// Import component
// ---------------------------------------------------------------------------
import { TibberPrices } from './TibberPrices'
// ---------------------------------------------------------------------------
// Tests
// ---------------------------------------------------------------------------
describe('TibberPrices', () => {
beforeEach(() => vi.clearAllMocks())
it('renders loading state initially', () => {
mockGet.mockImplementation(() => new Promise(() => {}))
renderWithProviders(<TibberPrices />)
expect(screen.getByTestId('prices-loading')).toBeInTheDocument()
})
it('renders "no active contract" when kind is missing/null', async () => {
mockGet.mockResolvedValue({
data: {
kind: null,
currency: 'EUR',
points: [],
tariff: null,
},
})
renderWithProviders(<TibberPrices />)
await waitFor(() => {
expect(screen.getByTestId('prices-no-contract')).toBeInTheDocument()
})
})
it('renders error state when fetch fails', async () => {
mockGet.mockRejectedValue(new Error('Network error'))
renderWithProviders(<TibberPrices />)
await waitFor(() => {
expect(screen.getByTestId('prices-error')).toBeInTheDocument()
})
})
it('renders tibber chart when tibber data is available', async () => {
mockGet.mockResolvedValue({
data: {
kind: 'tibber',
currency: 'EUR',
points: [
{ starts_at: '2026-06-22T10:00:00Z', buy: 0.133, sell: 0.09, level: 'NORMAL' },
{ starts_at: '2026-06-22T10:15:00Z', buy: 0.140, sell: 0.092, level: 'NORMAL' },
],
tariff: null,
},
})
renderWithProviders(<TibberPrices />)
await waitFor(() => {
expect(screen.getByTestId('tibber-chart')).toBeInTheDocument()
})
// Check badge shows kind
expect(screen.getByText('tibber')).toBeInTheDocument()
})
it('shows tariff table for manual kind', async () => {
mockGet.mockResolvedValue({
data: {
kind: 'manual',
currency: 'EUR',
points: [],
tariff: {
buy_dal: 0.127,
buy_normal: 0.133,
sell_dal: 0.09,
sell_normal: 0.09,
},
},
})
renderWithProviders(<TibberPrices />)
await waitFor(() => {
expect(screen.getByTestId('manual-tariff-table')).toBeInTheDocument()
})
expect(screen.getByTestId('tariff-buy-normal')).toHaveTextContent('0.1330')
expect(screen.getByTestId('tariff-buy-dal')).toHaveTextContent('0.1270')
expect(screen.getByTestId('tariff-sell-normal')).toHaveTextContent('0.0900')
expect(screen.getByTestId('tariff-sell-dal')).toHaveTextContent('0.0900')
})
})
+244
View File
@@ -0,0 +1,244 @@
/**
* TibberPrices price curve visualization.
*
* - Fetches today + tomorrow price range using useEnergyPrices.
* - For tibber kind: Recharts LineChart showing buy/sell prices over time.
* - For manual kind: shows tariff table (buy_dal, buy_normal, sell_dal, sell_normal).
* - Handles: no active contract, empty data, loading, error.
*
* Recharts imports are isolated to this file only.
*/
import {
Stack,
Text,
Loader,
Center,
Alert,
Table,
Title,
Badge,
Group,
Paper,
} from '@mantine/core'
import {
LineChart,
Line,
XAxis,
YAxis,
CartesianGrid,
Tooltip,
Legend,
ResponsiveContainer,
} from 'recharts'
import { useEnergyPrices } from './hooks'
import { formatLocalTime } from '../utils/datetime'
// ---------------------------------------------------------------------------
// Time range helpers
// ---------------------------------------------------------------------------
function getTodayStart(): string {
const d = new Date()
d.setUTCHours(0, 0, 0, 0)
return d.toISOString()
}
function getTomorrowEnd(): string {
const d = new Date()
d.setUTCHours(0, 0, 0, 0)
d.setUTCDate(d.getUTCDate() + 2)
return d.toISOString()
}
// ---------------------------------------------------------------------------
// Tibber chart
// ---------------------------------------------------------------------------
interface TibberChartProps {
points: Array<{ starts_at: string; buy: number; sell: number; level?: string | null }>
currency: string
}
function TibberChart({ points, currency }: TibberChartProps) {
const data = points.map((p) => ({
time: formatLocalTime(p.starts_at),
buy: p.buy,
sell: p.sell,
}))
return (
<Stack gap="xs" data-testid="tibber-chart">
<Title order={6} c="dimmed">
Price curve ({currency})
</Title>
<ResponsiveContainer width="100%" height={260}>
<LineChart data={data} margin={{ top: 4, right: 16, left: 0, bottom: 4 }}>
<CartesianGrid strokeDasharray="3 3" />
<XAxis
dataKey="time"
tick={{ fontSize: 10 }}
interval="preserveStartEnd"
/>
<YAxis
tick={{ fontSize: 10 }}
tickFormatter={(v: number) => v.toFixed(3)}
/>
<Tooltip
// eslint-disable-next-line @typescript-eslint/no-explicit-any
formatter={(val: any) =>
[`${typeof val === 'number' ? val.toFixed(4) : String(val)} ${currency}`, undefined]
}
/>
<Legend />
<Line
type="monotone"
dataKey="buy"
stroke="#2196f3"
dot={false}
strokeWidth={2}
name="Buy"
/>
<Line
type="monotone"
dataKey="sell"
stroke="#4caf50"
dot={false}
strokeWidth={2}
name="Sell"
/>
</LineChart>
</ResponsiveContainer>
</Stack>
)
}
// ---------------------------------------------------------------------------
// Manual tariff table
// ---------------------------------------------------------------------------
interface ManualTariffTableProps {
tariff: {
buy_dal: number
buy_normal: number
sell_dal: number
sell_normal: number
}
currency: string
}
function ManualTariffTable({ tariff, currency }: ManualTariffTableProps) {
return (
<Stack gap="xs" data-testid="manual-tariff-table">
<Title order={6} c="dimmed">
Fixed tariff ({currency}/kWh)
</Title>
<Table withTableBorder withColumnBorders>
<Table.Thead>
<Table.Tr>
<Table.Th>Tariff</Table.Th>
<Table.Th>Buy</Table.Th>
<Table.Th>Sell</Table.Th>
</Table.Tr>
</Table.Thead>
<Table.Tbody>
<Table.Tr>
<Table.Td>Normal (peak)</Table.Td>
<Table.Td data-testid="tariff-buy-normal">{tariff.buy_normal.toFixed(4)}</Table.Td>
<Table.Td data-testid="tariff-sell-normal">{tariff.sell_normal.toFixed(4)}</Table.Td>
</Table.Tr>
<Table.Tr>
<Table.Td>Dal (off-peak)</Table.Td>
<Table.Td data-testid="tariff-buy-dal">{tariff.buy_dal.toFixed(4)}</Table.Td>
<Table.Td data-testid="tariff-sell-dal">{tariff.sell_dal.toFixed(4)}</Table.Td>
</Table.Tr>
</Table.Tbody>
</Table>
</Stack>
)
}
// ---------------------------------------------------------------------------
// TibberPrices — main component
// ---------------------------------------------------------------------------
export function TibberPrices() {
const start = getTodayStart()
const end = getTomorrowEnd()
const { data, isLoading, isError } = useEnergyPrices(start, end)
if (isLoading) {
return (
<Center py="xl" data-testid="prices-loading">
<Loader />
</Center>
)
}
if (isError) {
return (
<Alert color="red" data-testid="prices-error">
Failed to load energy prices. Please refresh.
</Alert>
)
}
if (!data) {
return (
<Alert color="gray" data-testid="prices-no-data">
No pricing data available.
</Alert>
)
}
// No active contract
if (!data.kind) {
return (
<Paper withBorder p="md" data-testid="prices-no-contract">
<Stack gap="xs">
<Text fw={500}>No active contract</Text>
<Text size="sm" c="dimmed">
Activate an energy contract on the Contracts tab to see pricing data.
</Text>
</Stack>
</Paper>
)
}
const currency = data.currency
return (
<Stack gap="lg" data-testid="tibber-prices">
<Group gap="sm" align="center">
<Text fw={500}>Energy Prices</Text>
<Badge variant="outline" size="sm">
{data.kind}
</Badge>
<Text size="xs" c="dimmed">
{currency}
</Text>
</Group>
{data.kind === 'tibber' && data.points && data.points.length > 0 && (
<TibberChart points={data.points} currency={currency} />
)}
{data.kind === 'tibber' && (!data.points || data.points.length === 0) && (
<Alert color="yellow" data-testid="tibber-no-prices">
No Tibber price points available for the selected time range. Check Tibber configuration.
</Alert>
)}
{data.kind === 'manual' && data.tariff && (
<ManualTariffTable tariff={data.tariff} currency={currency} />
)}
{data.kind === 'manual' && !data.tariff && (
<Alert color="yellow" data-testid="manual-no-tariff">
Manual tariff data not available.
</Alert>
)}
</Stack>
)
}
+293
View File
@@ -0,0 +1,293 @@
/**
* Tests for energy hooks in hooks.ts energy pricing API wrappers.
*
* Coverage:
* 1. useContracts GET /api/energy/contracts
* 2. useCreateContract POST /api/energy/contracts
* 3. useEnergyPrices GET /api/energy/prices
* 4. useEnergyCosts GET /api/energy/costs
* 5. useEnergyCostSummary GET /api/energy/costs/summary
* 6. useRecomputeCosts POST /api/energy/costs/recompute
*/
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { renderHook, waitFor, act } from '@testing-library/react'
import { QueryClient, QueryClientProvider } from '@tanstack/react-query'
import type { ReactNode } from 'react'
// ---------------------------------------------------------------------------
// Mock apiClient
// ---------------------------------------------------------------------------
const mockGet = vi.fn()
const mockPost = vi.fn()
const mockPatch = vi.fn()
vi.mock('../api/client', () => ({
default: {
GET: (...args: unknown[]) => mockGet(...args),
POST: (...args: unknown[]) => mockPost(...args),
PATCH: (...args: unknown[]) => mockPatch(...args),
DELETE: vi.fn(),
},
ApiError: class ApiError extends Error {
status: number
body: unknown
constructor(status: number, body: unknown) {
super(`API error ${status}`)
this.name = 'ApiError'
this.status = status
this.body = body
}
},
registerLoginRedirect: vi.fn(),
}))
// ---------------------------------------------------------------------------
// Fixtures
// ---------------------------------------------------------------------------
const CONTRACT = {
id: 1,
name: 'Test Contract',
kind: 'manual',
active: true,
currency: 'EUR',
created_at: '2026-06-01T00:00:00Z',
updated_at: '2026-06-01T00:00:00Z',
}
const PRICE_POINT = {
starts_at: '2026-06-22T10:00:00Z',
buy: 0.133,
sell: 0.09,
level: 'NORMAL',
}
const COST_PERIOD = {
period_start: '2026-06-22T10:00:00Z',
d1_kwh: 0.1,
d2_kwh: 0.2,
r1_kwh: 0.0,
r2_kwh: 0.05,
import_cost: 0.044,
export_revenue: 0.005,
net_cost: 0.039,
currency: 'EUR',
degraded: false,
contract_version_id: 1,
}
const SUMMARY = {
currency: 'EUR',
metered_import: 10.5,
metered_export: 2.3,
metered_net: 8.2,
fixed_costs: 5.0,
credits: 50.0,
total_payable: 12.5,
}
// ---------------------------------------------------------------------------
// Wrapper
// ---------------------------------------------------------------------------
function makeWrapper() {
const qc = new QueryClient({
defaultOptions: { queries: { retry: false }, mutations: { retry: false } },
})
function Wrapper({ children }: { children: ReactNode }) {
return <QueryClientProvider client={qc}>{children}</QueryClientProvider>
}
return { qc, Wrapper }
}
// ---------------------------------------------------------------------------
// Tests
// ---------------------------------------------------------------------------
describe('useContracts', () => {
beforeEach(() => vi.clearAllMocks())
it('calls GET /api/energy/contracts and returns contract list', async () => {
mockGet.mockResolvedValue({ data: { items: [CONTRACT], total: 1 } })
const { Wrapper } = makeWrapper()
const { useContracts } = await import('./hooks')
const { result } = renderHook(() => useContracts(), { wrapper: Wrapper })
await waitFor(() => expect(result.current.isSuccess).toBe(true))
expect(mockGet).toHaveBeenCalledWith('/api/energy/contracts')
expect(result.current.data?.items).toHaveLength(1)
expect(result.current.data?.items[0].id).toBe(1)
})
})
describe('useCreateContract', () => {
beforeEach(() => vi.clearAllMocks())
it('calls POST /api/energy/contracts with the provided body', async () => {
mockPost.mockResolvedValue({ data: { ...CONTRACT, versions: [] } })
mockGet.mockResolvedValue({ data: { items: [], total: 0 } })
const { Wrapper } = makeWrapper()
const { useCreateContract } = await import('./hooks')
const { result } = renderHook(() => useCreateContract(), { wrapper: Wrapper })
const body = {
name: 'New Contract',
kind: 'manual',
currency: 'EUR',
values: { energy: { buy: { normal: 0.133 } } },
}
await act(async () => {
await result.current.mutateAsync(body)
})
expect(mockPost).toHaveBeenCalledWith('/api/energy/contracts', { body })
})
})
describe('useEnergyPrices', () => {
beforeEach(() => vi.clearAllMocks())
it('calls GET /api/energy/prices with start and end params', async () => {
mockGet.mockResolvedValue({
data: {
kind: 'tibber',
currency: 'EUR',
points: [PRICE_POINT],
tariff: null,
},
})
const { Wrapper } = makeWrapper()
const { useEnergyPrices } = await import('./hooks')
const start = '2026-06-22T00:00:00Z'
const end = '2026-06-23T00:00:00Z'
const { result } = renderHook(() => useEnergyPrices(start, end), { wrapper: Wrapper })
await waitFor(() => expect(result.current.isSuccess).toBe(true))
expect(mockGet).toHaveBeenCalledWith('/api/energy/prices', {
params: { query: { start, end } },
})
expect(result.current.data?.points).toHaveLength(1)
expect(result.current.data?.kind).toBe('tibber')
})
it('calls GET /api/energy/prices without params when none provided', async () => {
mockGet.mockResolvedValue({
data: { kind: 'manual', currency: 'EUR', points: [], tariff: null },
})
const { Wrapper } = makeWrapper()
const { useEnergyPrices } = await import('./hooks')
const { result } = renderHook(() => useEnergyPrices(), { wrapper: Wrapper })
await waitFor(() => expect(result.current.isSuccess).toBe(true))
expect(mockGet).toHaveBeenCalledWith('/api/energy/prices', {
params: { query: {} },
})
})
})
describe('useEnergyCosts', () => {
beforeEach(() => vi.clearAllMocks())
it('calls GET /api/energy/costs with start, end, and limit', async () => {
mockGet.mockResolvedValue({
data: { items: [COST_PERIOD], total: 1 },
})
const { Wrapper } = makeWrapper()
const { useEnergyCosts } = await import('./hooks')
const start = '2026-06-22T00:00:00Z'
const end = '2026-06-23T00:00:00Z'
const limit = 100
const { result } = renderHook(
() => useEnergyCosts(start, end, limit),
{ wrapper: Wrapper },
)
await waitFor(() => expect(result.current.isSuccess).toBe(true))
expect(mockGet).toHaveBeenCalledWith('/api/energy/costs', {
params: { query: { start, end, limit } },
})
expect(result.current.data?.items).toHaveLength(1)
})
})
describe('useEnergyCostSummary', () => {
beforeEach(() => vi.clearAllMocks())
it('calls GET /api/energy/costs/summary with date range', async () => {
mockGet.mockResolvedValue({ data: SUMMARY })
const { Wrapper } = makeWrapper()
const { useEnergyCostSummary } = await import('./hooks')
const start = '2026-06-01T00:00:00Z'
const end = '2026-06-30T23:59:59Z'
const { result } = renderHook(
() => useEnergyCostSummary(start, end),
{ wrapper: Wrapper },
)
await waitFor(() => expect(result.current.isSuccess).toBe(true))
expect(mockGet).toHaveBeenCalledWith('/api/energy/costs/summary', {
params: { query: { start, end } },
})
expect(result.current.data?.total_payable).toBe(12.5)
})
})
describe('useRecomputeCosts', () => {
beforeEach(() => vi.clearAllMocks())
it('calls POST /api/energy/costs/recompute and invalidates cost queries', async () => {
mockPost.mockResolvedValue({ data: { recomputed: 10 } })
// Mock GET for invalidation queries (costs and summary)
mockGet.mockResolvedValue({ data: { items: [], total: 0 } })
const { Wrapper } = makeWrapper()
const { useRecomputeCosts } = await import('./hooks')
const { result } = renderHook(() => useRecomputeCosts(), { wrapper: Wrapper })
await act(async () => {
await result.current.mutateAsync({
start: '2026-06-22T00:00:00Z',
end: '2026-06-23T00:00:00Z',
})
})
expect(mockPost).toHaveBeenCalledWith('/api/energy/costs/recompute', {
params: {
query: {
start: '2026-06-22T00:00:00Z',
end: '2026-06-23T00:00:00Z',
},
},
})
})
it('calls POST /api/energy/costs/recompute without params when none provided', async () => {
mockPost.mockResolvedValue({ data: { recomputed: 0 } })
const { Wrapper } = makeWrapper()
const { useRecomputeCosts } = await import('./hooks')
const { result } = renderHook(() => useRecomputeCosts(), { wrapper: Wrapper })
await act(async () => {
await result.current.mutateAsync({})
})
expect(mockPost).toHaveBeenCalledWith('/api/energy/costs/recompute', {
params: { query: {} },
})
})
})

Some files were not shown because too many files have changed in this diff Show More