DSMR hardening: restart-free config + decouple from telegram id
Config hot-reload (no restart): - MqttManager.unsubscribe(); apply_dsmr_subscription(settings) re-applies the DSMR subscription (subscribe/unsubscribe/topic-change, fresh snapshot so the sample interval also takes effect). - Called from lifespan AND PUT /api/config, so toggling DSMR ingest in the UI takes effect immediately without restarting the app. Decouple ingest from the DSMR telegram id (overflows / resets to zero): - Migration 20260624_12: drop UNIQUE(source_id), make recorded_at UNIQUE. - Idempotency now keys on recorded_at (telegram timestamp); the table's own autoincrement PK is the stable identity. source_id kept only as a reference. - Regression test: colliding telegram ids no longer drop new data.
This commit is contained in:
+18
-12
@@ -22,27 +22,33 @@ from app.db import Base
|
||||
class DsmrReading(Base):
|
||||
"""One down-sampled DSMR telegram stored as a full JSON blob.
|
||||
|
||||
``recorded_at`` is a real indexed column (not inside the payload) so that
|
||||
time-range queries are efficient. ``source_id`` holds the telegram's own
|
||||
integer ``id`` and is used for idempotent upsert (skip if already present).
|
||||
The entire telegram frame is stored verbatim in ``payload``; no field
|
||||
allow-list is applied so future commodities (gas, heating, three-phase)
|
||||
are accommodated without a schema change.
|
||||
Identity & idempotency are **independent of the DSMR Reader's telegram id**
|
||||
(that field overflows and must be manually reset to zero — a known DSMR
|
||||
quirk — so relying on it for uniqueness risks silently dropping new data).
|
||||
The table's own autoincrement ``id`` PK is the stable internal identity, and
|
||||
``recorded_at`` (the telegram timestamp) is the UNIQUE de-duplication key: a
|
||||
single P1 meter emits exactly one telegram per timestamp.
|
||||
|
||||
``recorded_at`` is a real column (not inside the payload) so time-range
|
||||
queries are efficient. The entire telegram frame is stored verbatim in
|
||||
``payload``; no field allow-list is applied so future commodities (gas,
|
||||
heating, three-phase) are accommodated without a schema change.
|
||||
"""
|
||||
|
||||
__tablename__ = "dsmr_reading"
|
||||
|
||||
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
||||
|
||||
# UTC timestamp of the sample — real column so it can be indexed efficiently.
|
||||
# UTC timestamp of the sample — real column, UNIQUE (telegram-id-independent
|
||||
# idempotency key). The unique index also serves time-range queries.
|
||||
recorded_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True), nullable=False, index=True
|
||||
DateTime(timezone=True), nullable=False, unique=True
|
||||
)
|
||||
|
||||
# Telegram's own auto-increment id (DSMR Reader assigns it). Used for
|
||||
# idempotent ingestion: if the same telegram arrives twice we skip it.
|
||||
# Nullable because some DSMR sources may not emit an id.
|
||||
source_id: Mapped[int | None] = mapped_column(Integer, unique=True, nullable=True)
|
||||
# Telegram's own id (DSMR Reader assigns it). Stored only as a reference /
|
||||
# debugging aid — NOT used for uniqueness or idempotency (it overflows and
|
||||
# gets reset to zero). Nullable because some DSMR sources may not emit one.
|
||||
source_id: Mapped[int | None] = mapped_column(Integer, nullable=True)
|
||||
|
||||
# Full telegram frame as a JSON object; values are typically JSON strings
|
||||
# (e.g. "20915.154") — callers must cast to Decimal before arithmetic.
|
||||
|
||||
Reference in New Issue
Block a user