Commit Graph
92 Commits
Author SHA1 Message Date
tliu93 b65f700d56 fix(tibber): fetch forward-looking today+tomorrow via priceInfo(QUARTER_HOURLY)
pytest / test (push) Successful in 20m2s
frontend / frontend (push) Failing after 6m5s
docker-image / build-and-push (push) Successful in 13m27s
priceInfoRange is a historical cursor connection whose range ends at "now": it
never returns upcoming slots. With it, the DB only ever held prices up to the
last hourly refresh, which caused two problems:

  1. The price chart could only show history up to now, never a forward curve.
  2. Worse, per-slot billing was subtly wrong. _tibber_strategy looks up the
     price via `starts_at <= t0` (nearest slot at or before the period). Because
     a period's exact 15-min slot was not fetched until the next hourly refresh
     (~1h later), intra-hour periods were billed with the PREVIOUS quarter's
     price and then locked in by the immutability guard — never corrected.

Switch to priceInfo(resolution: QUARTER_HOURLY) { today tomorrow }, which is
forward-looking AND quarter-hourly: today is always the full local day (96
slots) and tomorrow fills in once Tibber publishes day-ahead prices (picked up
by the next hourly refresh). Every 15-min slot's exact price is now in the DB
before the slot closes, so each period finds its own slot (accurate billing)
and the live current-price entity stays fresh.

Verified against the live Tibber API: fetch_price_range returns 192 points
(96 today + 96 tomorrow), spanning local today 00:00 → tomorrow 23:45, with
future slots present (previously 0).
2026-07-18 20:20:33 +02:00
tliu93 f4cea3874b test(energy-cost): pin local_now in future-window summarize test
frontend / frontend (push) Failing after 6m38s
pytest / test (push) Successful in 20m16s
docker-image / build-and-push (push) Successful in 13m32s
test_future_window_counts_0_days relied on the real wall-clock date and assumed
7/1→8/1 2026 was entirely in the future; once that range started elapsing the
window counted fixed-fee days and the assertion failed. Pin local_now to
June 25 2026 (as the sibling window tests already do) so the case stays
deterministic.
2026-07-17 18:49:26 +02:00
tliu93 3e04b15656 feat(modbus): add DDSU666 profile and select read function code per profile
frontend / frontend (push) Successful in 2m21s
pytest / test (push) Successful in 9m50s
docker-image / build-and-push (push) Successful in 4m24s
- Add CHINT DDSU666 profile (ddsu666.yaml): FC03 holding registers, voltage/
  current/active power (kW)/reactive power (kvar)/PF/frequency, import+export
  active energy. Word/byte order left at big-endian as a documented best guess
  (manual has no float example) — to be confirmed on-device.
- Add DDSU666-Modbus-Protocol.md reference extracted from the official manual,
  plus the source PDF (parity with the SDM120 reference).
- Generalize driver.read_blocks to dispatch FC03 (holding) or FC04 (input)
  based on a function_code argument (default 4, SDM120 behaviour unchanged);
  the code is validated before any connection is attempted.
- Wire profile.function_code through the CLI read command, the background
  poller, and the device /test endpoint — previously the profile field was
  declared but never honoured (read path was hardcoded to FC04).
- Tests: default -> FC04, function_code=3 -> FC03 holding, invalid FC rejected
  before connecting.
2026-06-30 17:16:25 +02:00
tliu93 90a03e7fd6 FUE-T09: grace-shift *_today daily reset past local midnight + dedicated 00:00:10 publish
docker-image / build-and-push (push) Successful in 4m29s
frontend / frontend (push) Successful in 2m17s
pytest / test (push) Successful in 11m2s
2026-06-26 12:21:51 +02:00
tliu93 d3fc90b320 FUE-T08: defer daily fixed-fee/heffingskorting settlement to local 01:05 in summarize() 2026-06-26 11:44:34 +02:00
tliu93 d4acfc438a FUE-T07: use meter label directly as energy-cost HA device name (drop 'Energy Cost' prefix)
frontend / frontend (push) Successful in 2m14s
pytest / test (push) Successful in 10m49s
2026-06-25 21:28:13 +02:00
tliu93 c26160b10b FUE-T06: trigger HA discovery republish after meter declare/update
frontend / frontend (push) Successful in 2m11s
pytest / test (push) Successful in 10m36s
2026-06-25 20:54:42 +02:00
tliu93 efbe36d7c0 FUE-T05: anchor energy-cost HA device identity to active meter (uuid id, label name, empty when none) 2026-06-25 20:43:07 +02:00
tliu93 f663981cdb FUE-T04: add Meter.uuid (stable HA identity anchor) + backfill migration 2026-06-25 20:20:01 +02:00
tliu93 da05fd2f09 FUE-T03: correct meters POST docstring (no recomputed_periods field) and inaccurate test comments
frontend / frontend (push) Successful in 2m11s
pytest / test (push) Successful in 8m51s
2026-06-25 18:36:46 +02:00
tliu93 f1e7ce3133 FUE-T01: count billing window start day in full for fixed-fee/credit accrual (symmetric with end day) 2026-06-25 18:33:42 +02:00
tliu93 839faaf95b M7-T05: add meter CRUD API + retroactive recompute + OpenAPI regen 2026-06-25 17:01:54 +02:00
tliu93 82bb329500 M7-T04: anchor cumulative cost entities to active meter start (per-meter reset) 2026-06-25 17:01:54 +02:00
tliu93 17057ed41e M7-T03: make billing engine meter-aware (no cross-meter delta, delta sanity guard, period.meter_id) 2026-06-25 17:01:54 +02:00
tliu93 227b146c2c M7-T02: add meter service layer (declare/swap/close/edit, meter_at, mutual exclusion) 2026-06-25 17:01:54 +02:00
tliu93 32a20785ee M7-T01: add meter table + model + energy_cost_period.meter_id + backfill migration 2026-06-25 14:48:39 +02:00
tliu93 96e85fb48e FU11: anchor cumulative cost at recording start; add daily-reset cost entities; prefill add-version form
pytest / test (push) Successful in 7m32s
frontend / frontend (push) Successful in 2m6s
- MQTT import_cost_total / export_revenue_total now anchor fixed-fee / tax-credit
  accrual at max(contract start, first recorded period), so standing charges for
  untracked pre-recording days are no longer folded into the running total.
- Add energy.import_cost_today / energy.export_revenue_today (monetary,
  total_increasing) that reset at server-local midnight, for per-day cost
  aggregation in Home Assistant alongside the running totals.
- ContractForm add-version mode prefills the contract's open version values, so
  only the fields that actually change need editing.
2026-06-25 12:45:24 +02:00
tliu93 8b91146c29 FU10: localize energy time math to server tz; accrue fixed/credit per elapsed local day
frontend / frontend (push) Successful in 2m5s
pytest / test (push) Successful in 7m34s
- Store UTC, compute day boundaries in server local timezone (new app/services/timezone.py).
- summarize: fixed fee / tax credit accrue only for elapsed whole local days,
  integrated across contract versions — no future-day inflation, no version-switch collapse.
- MQTT import_cost_total / export_revenue_total getters delegate to summarize (no inline
  apportionment), anchored at the active contract's earliest version effective_from so the
  total sensors stay monotonic and never jump backward.
- effective_from: naive datetimes interpreted as server-local wall-clock, then stored UTC.
- frontend ContractForm sends local-midnight naive datetime (was UTC midnight).
- get_costs_summary default window uses server-local "today".
2026-06-25 11:13:30 +02:00
tliu93 682e06d256 modbus: add explicit cascade delete (device + readings + toggles + HA cleanup)
frontend / frontend (push) Successful in 2m4s
pytest / test (push) Successful in 7m32s
A disabled device with historical readings still cannot be deleted by default
(409 guard stays, to prevent accidental data loss). A new explicit opt-in lets
the user remove it for real:

- DELETE /api/modbus/devices/{uuid}?cascade=true removes the device's readings
  and exposed_entity_toggle rows, then the device, in one transaction (FK order:
  readings before device). Best-effort clears the HA discovery configs first
  (MQTT not connected -> no-op, never blocks the delete). Default (no cascade)
  unchanged. Returns 200 with deletion counts.
- Frontend: a 'Force Delete (all data)' button appears only after the 409, with
  a red irreversible warning; cascade is sent only on that explicit action.
- Also corrected a stale comment: FK RESTRICT IS enforced at runtime now.
2026-06-24 17:38:52 +02:00
tliu93 d7f0731d1c ha_discovery: publish state/availability under home_automation prefix
Follow the standard MQTT Discovery split: discovery config stays under the HA
discovery prefix (homeassistant/.../config, required), but the actual state and
availability are published under our own prefix (default home_automation/). The
config payload's state_topic/availability fields point at the new prefix so HA
reads state from there.

- new config ha_state_topic_prefix (default home_automation).
- build_discovery_payload + all publish_* split discovery_prefix (config topic)
  from state_prefix (state/availability). energy-cost still omits availability;
  modbus availability structure unchanged. Tests updated.
2026-06-24 16:21:15 +02:00
tliu93 9bc46f8d2d expose: buy/sell_price_now follow the live dual-tariff slot
Subscribe the separate DSMR tariff topic (dsmr/meter-stats/electricity_tariff,
1=dal/off-peak, 2=normal/peak), keep the latest slot in memory, and make the
manual-contract buy/sell_price_now sensors show the matching tariff's price
(tariff 1 -> dal, 2/unknown -> normal). Tibber (single 15-min price) unchanged.
Billing is untouched (register-split already handles tariffs correctly).

- new config dsmr_tariff_topic; subscribe managed by apply_dsmr_subscription
  (restart-free on config save). In-memory tariff is lock-guarded.
- import_cost_total / export_revenue_total cumulative getters unchanged.
2026-06-24 15:39:21 +02:00
tliu93 6f8cb05eab expose: fold standing fee into import cost, tax credit into export revenue
So HA can show the full payable: net = import - export = Total Payable, with
both entities staying monotonic-positive (no negative-value/reset pitfalls).

- import_cost_total = SUM(import_cost) + days*(network_fee+management_fee)/30
- export_revenue_total = SUM(export_revenue) + days*heffingskorting/365
  (days = calendar days since the active version's effective_from, incl. today)
- state_class total_increasing -> total (monetary-compatible; values still only
  increase). device_class/key/names: key unchanged, names clarified.
- buy/sell_price_now and DeviceInfo untouched. Decimal money math. Tests added.
2026-06-24 14:26:32 +02:00
tliu93 9d3e28a529 Billing fix: don't bill periods from stale/out-of-range DSMR readings
Root cause (found in the live DB): register_at() returned the latest reading
for any boundary beyond the DSMR data range (a future instant or a long gap),
so future periods got start==end -> delta 0 and were written as degraded=False
'ok' rows with cost 0. recompute_range() over a future end then filled the whole
month with such fake rows, which then occupied the slots so the per-minute tick
(immutability guard skips non-degraded rows) never recomputed the real data.

- register_at: ignore a reading older than one period (15min) before the
  boundary -> returns None -> period is marked degraded instead of a fake 0.
- recompute_range: only process closed periods (t1 <= now); never future ones.
- Tests cover staleness window, out-of-range -> degraded, recompute skips
  future, and a regression that normal closed periods are unaffected.
2026-06-24 12:44:05 +02:00
tliu93 7e61b0a938 HA Discovery fix: energy-cost sensors are always-available
The energy-cost device has only sensors (no online/offline heartbeat), but the
discovery config still declared an availability topic that nothing ever publishes
'online' to — so Home Assistant marked every energy-cost entity 'unavailable'
even though the state topic was being published (visible in MQTT Explorer).

- DeviceInfo gains provides_availability (default True, keeps Modbus behavior).
- _energy_cost_provider sets provides_availability=False.
- build_discovery_payload only emits availability/availability_mode for devices
  that publish a heartbeat; HA treats the rest as always-available.
- Test asserts energy-cost configs omit availability.
2026-06-24 11:54:49 +02:00
tliu93 8d4f496ff4 DSMR hardening: restart-free config + decouple from telegram id
Config hot-reload (no restart):
- MqttManager.unsubscribe(); apply_dsmr_subscription(settings) re-applies the
  DSMR subscription (subscribe/unsubscribe/topic-change, fresh snapshot so the
  sample interval also takes effect).
- Called from lifespan AND PUT /api/config, so toggling DSMR ingest in the UI
  takes effect immediately without restarting the app.

Decouple ingest from the DSMR telegram id (overflows / resets to zero):
- Migration 20260624_12: drop UNIQUE(source_id), make recorded_at UNIQUE.
- Idempotency now keys on recorded_at (telegram timestamp); the table's own
  autoincrement PK is the stable identity. source_id kept only as a reference.
- Regression test: colliding telegram ids no longer drop new data.
2026-06-24 11:02:32 +02:00
tliu93 57f2459f60 M6-T09: add Energy data API (prices/costs/summary/dsmr-latest/recompute/tibber-test)
- routes/api/energy.py + schemas/energy.py: GET prices (tibber points or manual
  tariff, buy/sell consistent with the pricing strategies), GET costs (time
  window + limit, ascending), GET costs/summary (summarize passthrough), GET
  dsmr/latest, POST costs/recompute (idempotent, <=366d guard, CSRF), POST
  tibber/test (three-state, token never echoed, CSRF).
- main.py registers router; OpenAPI re-exported; tests for all endpoints.
2026-06-23 23:05:45 +02:00
tliu93 c61fc2d4ba M6-T08: add energy_cost expose provider + post-billing state publish
- expose.py: _energy_cost_provider yields buy/sell_price_now + import_cost_total
  /export_revenue_total (total_increasing, monetary), value_getters read from
  energy_cost_period; 2-element identifiers so the existing HA Discovery builder
  (which indexes identifiers[1]) works. Other providers/mechanism untouched.
- main.py: energy-cost job calls publish_states after computing periods (no-op
  when MQTT/Discovery off). Tests added.
Note: energy-cost sensors lack an availability heartbeat (ha_discovery mechanism
limitation, out of scope here).
2026-06-23 22:44:41 +02:00
tliu93 d35ac7d752 M6-T07: add metering billing engine + period job + summary + recompute
- app/services/energy_cost.py: register_at (register values at a UTC boundary),
  compute_period (register deltas x active-version strategy, snapshot price +
  contract_version_id, immutable unless overwrite), compute_closed_periods
  (bounded lookback), recompute_range (explicit overwrite), summarize
  (sum net + standing fees month/30 - heffingskorting year/365).
- Missing tibber price -> skip; missing reading -> degraded (zeroed amounts,
  null version), consistently on both create and recompute paths.
- main.py: 1-minute energy-cost job (existing jobs/MQTT untouched). Tests added.
2026-06-23 22:26:56 +02:00
tliu93 ef48032adb M6-T06: extend MqttManager with subscribe + DSMR ingest
- mqtt.py: subscription registry, on_message dispatch (swallows handler
  errors), re-subscribe on (re)connect; publish/connect/disconnect/reconnect
  unchanged.
- app/services/dsmr_ingest.py: handle_message stores the full telegram frame
  (no allowlist; gas/phases/null preserved), 10s downsample by timestamp.second,
  source_id idempotency (select + IntegrityError rollback). Net-thread safe.
- main.py registers the DSMR subscription only when dsmr_ingest_enabled.
2026-06-23 21:52:57 +02:00
tliu93 7e266a21eb M6-T05: add Tibber GraphQL client + price refresh service + schedule
- app/integrations/tibber/client.py: fetch_price_range/fetch_current_price
  (priceInfoRange QUARTER_HOURLY, startsAt->UTC, no fixed node count assumption),
  TibberError/TibberAuthError; token never logged or put in exception text.
- app/services/tibber_prices.py: refresh_prices upserts by starts_at (idempotent),
  no-op unless an active kind=tibber contract + token exist.
- main.py: hourly tibber-refresh job (existing jobs/MQTT untouched). Tests added.
2026-06-23 21:35:29 +02:00
tliu93 4284bd40a7 M6-T04: add EnergyContract CRUD + versions + profile validation API
- app/services/contracts.py: create_contract, add_version (append-only,
  auto-closes prior version), activate_contract (mutex), and
  active_contract_version_at (half-open [from, to)) for the billing engine.
- app/schemas/energy_contract.py + routes/api/energy_contracts.py: 6 endpoints
  (GET/POST contracts, GET/PATCH contracts/{id}, POST .../versions, GET profiles).
  Values validated against pricing profiles (422 on mismatch); no DELETE.
- main.py registers router; OpenAPI re-exported; tests for CRUD/version/activate.
2026-06-23 21:19:25 +02:00
tliu93 bedea196c3 M6-T03: add pricing profile framework + strategy registry
- app/integrations/pricing: profiles.py (pydantic ManualProfile/TibberProfile,
  load_profile/list_profiles/validate_values), manual.yaml + tibber.yaml
  (structure-only, no price values), strategies.py (register/get_strategy,
  manual dual-tariff + tibber strategies, all Decimal).
- tibber strategy matches nearest tibber_price with starts_at <= t0.
- tests for profiles + strategies (hand-checked dual-tariff & tibber math).
2026-06-23 20:58:34 +02:00
tliu93 df54f5518b M6-T02: add DSMR + Tibber flat config fields
- Settings: dsmr_ingest_enabled, dsmr_mqtt_topic, dsmr_sample_interval_s,
  tibber_api_token (secret), tibber_home_id (all opt-in / off by default).
- CONFIG_FIELDS DSMR + Tibber sections; _settings_payload extended.
- .env.example examples; test_api_config coverage (sections, secret, 422).
2026-06-23 20:43:46 +02:00
tliu93 18fe18281b M6-T01: add 5 energy tables, models, and migration
- app/models/energy.py: DsmrReading, EnergyContract, EnergyContractVersion,
  TibberPrice, EnergyCostPeriod (per design §3.5; JSON blobs, RESTRICT FKs).
- alembic_app migration 20260623_11_energy_tables (head), env.py imports.
- app_db_adopt APP_BASELINE_REVISION bumped to new head.
- tests/test_energy_models.py + expose_catalog test fixups for new head.
2026-06-23 20:33:35 +02:00
tliu93 6090b98ab0 M5: enable SQLite foreign_keys pragma for DB-level FK enforcement 2026-06-22 20:45:47 +02:00
tliu93 35cdc7f22a M5: use DB-merged runtime settings for MQTT/discovery; add MQTT test button; move Expose panel into config accordion 2026-06-22 20:11:54 +02:00
tliu93 75d685f04d M5: roll Energy chart window for live auto-refresh (English label); render boolean config fields as switches 2026-06-22 19:40:18 +02:00
tliu93 45d87f36f7 M5-T12: add expose toggle API and Home Assistant Expose settings panel 2026-06-22 15:50:49 +02:00
tliu93 35b95f5c88 M5-T11: add HA discovery and state publishing wired to polling 2026-06-22 15:32:43 +02:00
tliu93 9d05f5ea62 M5-T10: add paho MQTT client with lifespan connect/reconnect and mqtt/test endpoint 2026-06-22 15:06:30 +02:00
tliu93 360cddc05b M5-T09: add exposed_entities table and ExposableEntity provider framework 2026-06-22 14:50:40 +02:00
tliu93 14b846549e M5-T08: add MQTT, HA Discovery, and Modbus polling config fields 2026-06-22 14:27:21 +02:00
tliu93 68165f0e01 M5-T07: add latest-reading cards and Recharts trend charts; readings return most-recent rows 2026-06-22 14:17:59 +02:00
tliu93 702a1cec00 M5-T05: add Modbus JSON API (device CRUD, readings, metrics, test) 2026-06-22 13:37:48 +02:00
tliu93 49d15d8ffe M5-T04: add Modbus poll service and APScheduler polling job 2026-06-22 13:24:11 +02:00
tliu93 c89cea4953 M5-T03: add Modbus TCP driver, YAML profile framework, sdm120, modbus_cli 2026-06-22 13:03:50 +02:00
tliu93 d9c82e39fb M5-T02: add modbus_device and modbus_reading tables and models 2026-06-22 12:45:50 +02:00
tliu93 51da8c5716 M4-T07: add admin_cli disable-totp and reissue-totp escape commands 2026-06-21 22:27:07 +02:00
tliu93 81bd32f613 M4-T06: add TOTP second factor to login (totp_code + recovery codes) 2026-06-21 22:08:55 +02:00
tliu93 ee3031013e M4-T05: add TOTP service and setup/enable/disable/status API 2026-06-21 21:55:17 +02:00