Compare commits

..
11 Commits
Author SHA1 Message Date
tliu93 f6ef99e34f MQTT-FIX-T02: bound Docker logs and document production rollout
docker-image / build-and-push (push) Successful in 1m51s
frontend / frontend (push) Successful in 1m12s
pytest / test (push) Successful in 4m31s
2026-10-11 23:59:38 +02:00
tliu93 876754be56 MQTT-FIX-T01: make main MQTT client lifecycle idempotent 2026-10-11 23:34:13 +02:00
tliu93 48071685f1 META: update orchestration model recommendations
frontend / frontend (push) Successful in 1m35s
pytest / test (push) Successful in 4m46s
2026-10-11 22:53:14 +02:00
tliu93 3cf3843af3 M8-R21: patch frontend warnings and dependencies
frontend / frontend (push) Successful in 59s
pytest / test (push) Successful in 4m19s
2026-08-28 18:16:48 +02:00
tliu93 9e93ca0db4 M8-R20: enforce warning-free frontend tests
pytest / test (push) Successful in 4m14s
frontend / frontend (push) Successful in 59s
2026-08-28 16:54:15 +02:00
tliu93 35691e08eb M8-R19: use unique thermal pagination fixtures
frontend / frontend (push) Successful in 1m0s
pytest / test (push) Successful in 4m18s
2026-08-28 15:25:06 +02:00
tliu93 7d46cb96d3 M8-R18: stabilize Mantine frontend tests
frontend / frontend (push) Successful in 1m1s
pytest / test (push) Successful in 4m19s
docker-image / build-and-push (push) Successful in 2m2s
2026-08-28 14:26:55 +02:00
tliu93 5e1545efad M8-R17: allocate thermal standing costs by service 2026-08-28 14:24:57 +02:00
tliu93 6e197d7808 M8-R16: expose hot water to HA Energy
pytest / test (push) Successful in 4m2s
docker-image / build-and-push (push) Successful in 1m39s
frontend / frontend (push) Successful in 48s
2026-08-28 10:02:53 +02:00
tliu93 018f13d73d M8-R15: fix HA discovery identities and thermal totals
frontend / frontend (push) Successful in 47s
pytest / test (push) Successful in 4m1s
docker-image / build-and-push (push) Successful in 1m38s
2026-08-28 01:20:52 +02:00
tliu93 8180082f90 M8-R14: preserve zero DSMR sample interval
docker-image / build-and-push (push) Successful in 1m35s
frontend / frontend (push) Successful in 47s
pytest / test (push) Successful in 4m15s
2026-08-27 21:29:45 +02:00
36 changed files with 2524 additions and 467 deletions

No files matched your search

+1
View File
@@ -6,3 +6,4 @@ __pycache__/
*.pyc
data/
review-notes/
development-notes/
+11 -10
View File
@@ -43,26 +43,27 @@
- **(b) 完成整个 milestone**:仅当用户在提示词里**显式要求启用 sub-agent**时,才起 implementer / reviewer / fixer sub-agent(按下方**『默认能力档位』**选择模型,用户人工指定则覆盖),按任务依赖顺序跑完整条链。
- **Sub-agent 纪律**:只在用户显式要求时才 spawn sub-agent;单步/小改动在主线内联完成。当前 harness 支持独立 sub-agent 时,使用其原生机制按下方**『默认能力档位』**派发;不支持时不得假装已创建 sub-agent,应明确说明限制,并仅在用户允许 fallback 时由主 agent 继续。
### 默认能力档位(实现模式 sub-agent;可被人工指定覆盖)
### 默认能力档位(编排器与实现模式 sub-agent;可被人工指定覆盖)
起 implementer / reviewer / fixer sub-agent 时,**默认**按下列能力档位选择当前 harness 支持的模型,无需用户每次人工指定:
每次开启编排模式时,**Orchestrator 使用开启时主线当前使用的模型**。起 implementer / reviewer / fixer sub-agent 时,**默认**按下列能力档位选择当前 harness 支持的模型,无需用户每次人工指定:
| 角色 | 通用模型要求 | 推理档位 | Harness 示例(非强制) |
| --- | --- | --- | --- |
| **Implementer** | 平衡型代码实现模型 | `medium` 或等效档位 | Claude Code:Sonnet;Codex/OpenAI:GPT-5.6 Terra (`gpt-5.6-terra`) |
| **Fixer**(返工) | 平衡型代码实现模型 | `medium` 或等效档位 | Claude Code:Sonnet;Codex/OpenAI:GPT-5.6 Terra (`gpt-5.6-terra`) |
| **Reviewer** | 当前 harness 支持的最强通用推理 / 代码模型 | `extra-high` / `xhigh` 或等效档位 | Claude Code:Opus;Codex/OpenAI:GPT-5.6 Sol (`gpt-5.6-sol`) |
| **Orchestrator**(编排器) | 每次开启编排模式时主线当前使用的模型 | 沿用主线当前设置 | 开启编排模式时的当前模型 |
| **Implementer** | 平衡型代码实现模型 | `medium` 或等效档位 | Claude Code:Sonnet;Codex/OpenAI:GPT-6.1-Luna (`gpt-6.1-luna`) |
| **Fixer**(返工) | 平衡型代码实现模型 | `medium` 或等效档位 | Claude Code:Sonnet;Codex/OpenAI:GPT-6.1-Luna (`gpt-6.1-luna`) |
| **Reviewer** | 通用推理 / 代码评审模型 | `medium` 或等效档位 | Claude Code:Opus;Codex/OpenAI:GPT-6.1-Sol (`gpt-6.1-sol`) |
- **示例非强制**:示例模型只表示当前推荐映射,不构成跨 harness 的硬性模型 ID;当前 harness 不支持时,选择最符合「通用模型要求」的可用模型。
- **选择优先级**:用户显式指定 > 当前 harness 的原生角色配置 > 上表的 harness 示例 > 按通用模型要求自动选择。
- **推理档位说明**:若 harness 提供独立的 reasoning-effort 设置,按上表设置;若不提供,在 spawn prompt 中明确 implementer/fixer 按平衡深度思考,reviewer 按对抗性外部审计强度复核。
- **推理档位说明**:若 harness 提供独立的 reasoning-effort 设置,implementer / fixer / reviewer 均设为 `medium`;若不提供,在 spawn prompt 中明确按 medium 或等效的平衡深度思考,reviewer 仍须独立、对抗性复核。
### 角色(Orchestrator → Implementer → Reviewer → Fixer)
- 我(主线)= **Orchestrator**:挑依赖已满足的下一个任务、派发、转述结果、维护任务 `Status`。
- **Implementer**(平衡型代码实现模型,medium 或等效档位):一次一个任务,严格按任务卡,不扩范围。
- **Reviewer**(最强通用推理 / 代码模型,extra-high / xhigh 或等效档位):实现完成后起 Reviewer sub-agent,按任务卡 `Acceptance criteria` + `Reviewer checklist` 复核、**独立重跑校验闸门**,驱动返工直到本轮 PASS。
- **Fixer**(平衡型代码实现模型,medium 或等效档位):按 reviewer 的编号返工清单返工;**每轮返工起一个干净的 Fixer**(与首次实现的 Implementer 分开冷启动),先读对应 `review-notes/<task>-review-<n>.md` 再改。
- 我(主线)= **Orchestrator**(每次开启编排模式时主线当前使用的模型):挑依赖已满足的下一个任务、派发、转述结果、维护任务 `Status`。
- **Implementer**(默认 GPT-6.1-Luna,medium 或等效档位):一次一个任务,严格按任务卡,不扩范围。
- **Reviewer**(默认 GPT-6.1-Sol,medium 或等效档位):实现完成后起 Reviewer sub-agent,按任务卡 `Acceptance criteria` + `Reviewer checklist` 复核、**独立重跑校验闸门**,驱动返工直到本轮 PASS。
- **Fixer**(默认 GPT-6.1-Luna,medium 或等效档位):按 reviewer 的编号返工清单返工;**每轮返工起一个干净的 Fixer**(与首次实现的 Implementer 分开冷启动),先读对应 `review-notes/<task>-review-<n>.md` 再改。
#### Reviewer 盲审纪律(M1 教训)
+151 -1
View File
@@ -345,6 +345,11 @@ CLI 工具为受控手工验证而设(设备需接市电),仅暴露读功
- 每次轮询成功后推 state topic;连接成功或勾选变更时重发 retained discovery config
- `POST /api/config/mqtt/test`:试连 broker 并发布一条测试消息(可用 MQTT Explorer 验证链路)
主 MQTT 客户端由应用 lifespan 持有。相同配置下重复调用 connect 不会创建新客户端;Paho
网络循环在短暂断线后负责自动重连,并在重连成功时恢复订阅。每分钟的 HA 发布任务会幂等地
确保客户端存在。配置变更或应用关闭时,旧客户端先从管理器解绑,再断开并停止网络循环;旧
客户端的迟到回调不会改变新连接状态或分发消息。DSMR source 使用各自独立的客户端生命周期。
**Config 页配置流程**:
1. 在 `/config` 的「MQTT」section 填写 broker host/port/username/password,启用 `MQTT_ENABLED`
2. 点「发送测试消息」确认 broker 链路通
@@ -585,10 +590,155 @@ docker compose -f docker-compose.yml pull
docker compose -f docker-compose.yml up -d
```
### Docker 日志轮转与生产应用
源码 Compose 为 `app` 和一次性 `migration` 服务配置 Docker `json-file` 日志轮转:单个
日志文件最多约 10 MiB,最多保留 3 个文件。开发 override 继承这项设置。Docker 管理这些
文件,通常位于宿主机
`/var/lib/docker/containers/<完整容器ID>/<完整容器ID>-json.log`;路径取决于 Docker data
root,`docker logs` / `docker compose logs` 是查看容器日志的接口。轮转会限制每个容器的
日志保留量,较旧内容会被 Docker 轮转淘汰。不要手动删除或编辑 Docker 管理的日志文件。
`docker restart` 只重启现有容器,不会更新容器创建时保存的 logging 配置;须重建容器。
本机开发可重新执行上面的 Compose 命令。生产设备的 Compose 位于
`/home/tianyu/.local/share/homeautomation`,与本仓库源码 Compose 分离。部署时只把两个
服务的 `logging` 块合并到生产 Compose 对应服务中,并保留生产端已有的 `.env`、设备映射、
`group_add`、volume、端口和其他部署设置;不要用完整源码 Compose 覆盖生产文件。
要同步的配置块如下,分别放在生产文件的 `services.app` 和 `services.migration` 下:
```yaml
logging:
driver: json-file
options:
max-size: "10m"
max-file: "3"
```
生产变更只供人工维护窗口执行,需在生产目录操作。先备份生产 Compose(以及已有的环境文件),
并且只把 `app`、`migration` 的 `logging` 块同步到生产文件;不要复制或覆盖整份生产 Compose。
保留原有设备映射、`group_add`、volume、端口、环境变量和其他部署设置。编辑后用生产实际使用的
Compose 文件组合及合成环境值运行 `docker compose ... config --quiet` 检查;后续所有 Compose 命令都要带上相同生产文件组合和
`.mqtt-fix-image-pin.yml`,任何重建命令都带 `--pull never --no-build`;不要把展开后的配置
(可能含凭据)打印或贴入报告。以下命令在生产目录执行,示例假设生产基础文件名为
`docker-compose.yml`;若实际启动时显式使用其他 `-f` 文件,应在每条命令中原样保留该文件组合。
先确定本次要走的镜像路径,并创建仅用于固定镜像的持久 override。该文件只设置两个服务的
`image`,生产 Compose 仍负责其余设置。只应用日志配置时,不能假设本地 `latest` 标签仍指向
正在运行的镜像:先从运行中的 app 容器读取不可变 image ID,将其记录并加上受控本地标签,
然后固定这个标签。命令示例:
```bash
cd /home/tianyu/.local/share/homeautomation
MQTT_FIX_BACKUP="compose.before-mqtt-fix-$(date +%Y%m%d-%H%M%S).yml"
MQTT_FIX_ENV_BACKUP="env.before-mqtt-fix-$(date +%Y%m%d-%H%M%S).bak"
MQTT_FIX_RECORD="mqtt-fix-maintenance-record.txt"
MQTT_FIX_PIN_BACKUP="image-pin.before-mqtt-fix.yml"
MQTT_FIX_APP_ID="$(docker inspect -f '{{.Image}}' home-automation-app)"
MQTT_FIX_OLD_REPODIGESTS="$(docker image inspect -f '{{json .RepoDigests}}' "$MQTT_FIX_APP_ID")"
printf 'previous app image ID: %s\nprevious app RepoDigests: %s\n' "$MQTT_FIX_APP_ID" "$MQTT_FIX_OLD_REPODIGESTS" | tee -a "$MQTT_FIX_RECORD"
MQTT_FIX_APP_IMAGE="home-automation:prod-before-mqtt-fix"
docker image tag "$MQTT_FIX_APP_ID" "$MQTT_FIX_APP_IMAGE"
printf 'previous app local pin: %s\n' "$MQTT_FIX_APP_IMAGE" | tee -a "$MQTT_FIX_RECORD"
cp docker-compose.yml "$MQTT_FIX_BACKUP"
if [ -f .env ]; then cp .env "$MQTT_FIX_ENV_BACKUP"; fi
if [ -f .mqtt-fix-image-pin.yml ]; then cp .mqtt-fix-image-pin.yml "$MQTT_FIX_PIN_BACKUP"; fi
printf 'Compose backup: %s\n.env backup (if present): %s\nprevious image pin backup (if present): %s\n' "$MQTT_FIX_BACKUP" "$MQTT_FIX_ENV_BACKUP" "$MQTT_FIX_PIN_BACKUP" | tee -a "$MQTT_FIX_RECORD"
cat > .mqtt-fix-image-pin.yml <<YAML
services:
app:
image: ${MQTT_FIX_APP_IMAGE}
migration:
image: ${MQTT_FIX_APP_IMAGE}
YAML
```
`.mqtt-fix-image-pin.yml` 保存已核验的镜像选择;不要删除它或在后续 Compose 操作中省略它。
只有明确部署另一个已核验的镜像时才更新该文件。
在编辑生产文件的 `logging` 块后,用同一镜像 pin 和合成环境变量静态检查配置,并通过显式
override 固定镜像身份。`--env-file /dev/null` 可避免从部署 `.env` 取值;命令行变量只用于
Compose 插值。不要输出 `config` 展开结果:
```bash
WARMTELINK_DEVICE_PATH=/dev/synthetic-device WARMTELINK_SERIAL_GID=12345 TZ=Europe/Amsterdam \
docker compose --env-file /dev/null -f docker-compose.yml -f .mqtt-fix-image-pin.yml config --quiet
```
若同时部署 MQTT 源码修复,须先确认修复已合入并发布。显式拉取已发布的版本 tag(不要用
可变 `latest`),从本地镜像记录 registry digest,并在 override 中用 digest 固定目标;先核验
该 tag/digest 对应包含修复的发布,再继续。示例中的 `<published-tag>` 应替换为已核验的版本:
```bash
MQTT_FIX_RELEASE_TAG='code.wanderingbadger.dev/tliu93/home-automation:REPLACE_WITH_VERIFIED_TAG'
docker pull "$MQTT_FIX_RELEASE_TAG"
MQTT_FIX_NEW_REPODIGESTS="$(docker image inspect -f '{{json .RepoDigests}}' "$MQTT_FIX_RELEASE_TAG")"
printf 'target release tag: %s\ntarget RepoDigests: %s\n' "$MQTT_FIX_RELEASE_TAG" "$MQTT_FIX_NEW_REPODIGESTS" | tee -a "$MQTT_FIX_RECORD"
MQTT_FIX_APP_IMAGE='code.wanderingbadger.dev/tliu93/home-automation@sha256:REPLACE_WITH_VERIFIED_DIGEST'
cat > .mqtt-fix-image-pin.yml <<YAML
services:
app:
image: ${MQTT_FIX_APP_IMAGE}
migration:
image: ${MQTT_FIX_APP_IMAGE}
YAML
printf 'selected fixed app image: %s\n' "$MQTT_FIX_APP_IMAGE" | tee -a "$MQTT_FIX_RECORD"
```
将镜像 digest 与发布记录中的目标源码提交对应关系核验后,把确认过的
`MQTT_FIX_APP_IMAGE` 用于上述 override 和 `config --quiet` 检查。源码修复尚未发布时,
只走前述运行中 image ID 固定路径,不执行 `docker pull`。纯应用代码修复不需要启动 migration;
配置更新时可用 `create` 重建 migration 容器但不启动一次性迁移任务。
容器必须重建才能采用新日志策略。重建 app 会造成短暂采集与 API 中断;安排维护窗口并确认
设备采集可恢复。重建前如需留存旧 Docker 日志,可将 `docker logs --timestamps
home-automation-app` 输出重定向到受控备份文件,并确认文件可读;此步骤只导出日志,不删除
Docker 文件。Docker 可能在移除旧容器时一并移除其 json-file 日志,因此不要假设重建后旧
日志仍可从新容器读取。
确认配置、logging 块和固定镜像后,在生产目录应用容器配置(避免启动 migration)。每条重建
命令都显式使用 image pin 并禁止隐式拉取,且保留生产基础 Compose 与原有部署设置:
```bash
docker compose -f docker-compose.yml -f .mqtt-fix-image-pin.yml create --pull never --no-build --force-recreate migration
docker compose -f docker-compose.yml -f .mqtt-fix-image-pin.yml up -d --no-deps --pull never --no-build --force-recreate app
docker inspect -f '{{.HostConfig.LogConfig.Type}} {{json .HostConfig.LogConfig.Config}}' home-automation-app
docker inspect -f '{{.Image}}' home-automation-app
docker compose -f docker-compose.yml -f .mqtt-fix-image-pin.yml ps
docker compose -f docker-compose.yml -f .mqtt-fix-image-pin.yml logs --tail=100 app
```
检查 `inspect` 结果为 `json-file`、`max-size` 为 `10m`、`max-file` 为 `3`,app 为运行状态,
近期日志无启动错误,之后确认 MQTT 与采集恢复。回退时恢复备份的生产 Compose(或恢复旧
logging 块)。两种镜像路径都可用维护记录中的旧 app image ID 回退:确认本地旧镜像仍在后,
将持久 pin 文件恢复为记录的旧本地 pin(或由记录的 image ID 重新打回该标签),并继续使用该文件:
```bash
docker image inspect home-automation:prod-before-mqtt-fix >/dev/null
cat > .mqtt-fix-image-pin.yml <<'YAML'
services:
app:
image: home-automation:prod-before-mqtt-fix
migration:
image: home-automation:prod-before-mqtt-fix
YAML
WARMTELINK_DEVICE_PATH=/dev/synthetic-device WARMTELINK_SERIAL_GID=12345 TZ=Europe/Amsterdam \
docker compose --env-file /dev/null -f docker-compose.yml -f .mqtt-fix-image-pin.yml config --quiet
docker compose -f docker-compose.yml -f .mqtt-fix-image-pin.yml create --pull never --no-build --force-recreate migration
docker compose -f docker-compose.yml -f .mqtt-fix-image-pin.yml up -d --no-deps --pull never --no-build --force-recreate app
```
源码更新路径也使用开头记录的旧 app image ID/pin 回退,无需依赖已变化的 `latest`。若本地
旧 image ID 已不存在,则用维护记录中保存的旧 RepoDigest 显式拉取旧镜像,核对身份后打回原 pin
标签,再按上面的回退步骤操作。记录本次和回退使用的 image ID/digest、固定 pin 文件内容、生产
Compose 及 `.env` 备份文件名、logging inspect 结果及验证时间。回退同样会重建容器、造成短暂
中断;重建前需另行导出想保留的当前容器日志。保留 `.mqtt-fix-image-pin.yml` 供后续 Compose
操作继续使用;不要删除 Docker 日志、备份或其他生产文件。
持续查看日志:
```bash
docker compose logs -f app
docker compose -f docker-compose.yml -f .mqtt-fix-image-pin.yml logs -f app
```
## Container Image CI
+62 -17
View File
@@ -45,12 +45,15 @@ class DeviceInfo:
"""Grouping metadata that maps an entity to a logical HA device.
``identifiers`` corresponds to ``device.identifiers`` in HA Discovery
config — a stable tuple used to group entities under one device card.
config. HA merges device cards if *any* identifier overlaps, so every
logical device deliberately has exactly one, globally namespaced value.
``identity`` is an independent internal seed for MQTT topics and unique
ids; it must never be inferred from the HA identifier list.
``name`` is the human-readable device name (friendly_name).
"""
identifiers: tuple[str, ...]
"""Stable identifiers for HA device grouping (e.g. ``("modbus", "<uuid>")``).
"""Stable HA identifiers, normally a one-item tuple.
These must not change over the device lifetime; they anchor the HA device
card even when friendly_name changes.
@@ -59,6 +62,14 @@ class DeviceInfo:
name: str
"""Human-readable device name (may change; triggers HA discovery re-publish)."""
identity: Optional[str] = None
"""Stable internal topic/unique-id identity, separate from HA identifiers.
The fallback preserves compatibility for third-party providers still using
the historic ``("kind", "uuid")`` construction while first-party providers
use this field explicitly.
"""
provides_availability: bool = True
"""Whether this device publishes an availability ("online"/"offline") heartbeat.
@@ -85,6 +96,11 @@ class DeviceInfo:
)
"""Return whether the source behind this device is currently usable."""
@property
def internal_identity(self) -> str:
"""Return the topic/unique-id seed without exposing HA grouping details."""
return self.identity or self.identifiers[-1]
@dataclass
class ExposableEntity:
@@ -276,8 +292,9 @@ def _modbus_provider(session: Session) -> list[ExposableEntity]:
for device in devices:
device_info = DeviceInfo(
identifiers=("modbus", device.uuid),
identifiers=(f"home-automation:modbus:{device.uuid}",),
name=device.friendly_name,
identity=device.uuid,
)
# Load the profile to get metric metadata.
@@ -407,9 +424,10 @@ def _energy_cost_provider(session: Session) -> list[ExposableEntity]:
HA device identity (换表 → 新 sensor)
--------------------------------------
``identifiers[1]`` is set to the active meter's **uuid** (not the fixed
string ``"energy-cost"``). ``ha_discovery.py`` uses ``identifiers[1]`` as
the MQTT node_id and as part of the ``unique_id`` for every entity.
The independent internal identity is set to the active meter's **uuid**.
``ha_discovery.py`` uses that internal identity as the MQTT node_id and as
part of the ``unique_id`` for every entity; HA receives one separate,
namespaced device identifier.
Declaring a new active electricity meter produces a new uuid → new node_id /
unique_id → HA creates a brand-new sensor, cleanly isolating post-swap data.
@@ -460,9 +478,9 @@ def _energy_cost_provider(session: Session) -> list[ExposableEntity]:
DeviceInfo identifiers
----------------------
**Two-element tuple** ``("energy-cost", meter.uuid)`` so that
``ha_discovery.py``'s ``entity.device.identifiers[1]`` resolves to the
meter uuid (used as the MQTT node_id and unique_id seed throughout).
One-element namespaced tuple ``("home-automation:energy-cost:<uuid>",)``.
The meter uuid used for MQTT topics and unique IDs is carried independently
in ``DeviceInfo.identity``.
"""
from app.models.energy import EnergyCostPeriod, Meter # local import to avoid circular
from sqlalchemy import select
@@ -499,14 +517,15 @@ def _energy_cost_provider(session: Session) -> list[ExposableEntity]:
currency = latest_period.currency
# --- Shared DeviceInfo anchored to the active meter's uuid ---
# identifiers[1] = meter.uuid drives the MQTT node_id and unique_id in
# ha_discovery.py. Swapping the meter produces a new uuid → new HA sensor.
# The internal identity is meter.uuid, while the HA identifier is a single
# namespaced value. Swapping the meter produces a new HA sensor/card.
# provides_availability=False: the energy-cost device has only sensors and no
# online/offline heartbeat, so its entities must be "always available" in HA.
# (Otherwise HA shows them unavailable despite state being published.)
device_info = DeviceInfo(
identifiers=("energy-cost", active_meter.uuid),
identifiers=(f"home-automation:energy-cost:{active_meter.uuid}",),
name=active_meter.label,
identity=active_meter.uuid,
provides_availability=False,
)
@@ -967,7 +986,8 @@ def _m8_energy_provider(session: Session) -> list[ExposableEntity]:
for source in sources:
source_info = DeviceInfo(
identifiers=("meter-source", source.uuid), name=source.name,
identifiers=(f"home-automation:meter-source:{source.uuid}",), name=source.name,
identity=source.uuid,
availability_id=source.uuid,
availability_getter=lambda sess, source_id=source.id: _source_online_by_id(sess, source_id),
)
@@ -994,7 +1014,8 @@ def _m8_energy_provider(session: Session) -> list[ExposableEntity]:
continue
binding, channel, source = bound
info = DeviceInfo(
identifiers=("meter", meter.uuid), name=meter.label,
identifiers=(f"home-automation:meter:{meter.uuid}",), name=meter.label,
identity=meter.uuid,
# Keep this opaque and Meter-anchored. In particular, do not use a
# source UUID here: two channels of one source can be independently
# stale/invalid and must not overwrite each other's availability.
@@ -1007,7 +1028,7 @@ def _m8_energy_provider(session: Session) -> list[ExposableEntity]:
elif meter.commodity == "heating":
unit, device_class = "GJ", "energy"
else:
unit, device_class = "m³", "volume"
unit, device_class = "m³", "water"
for suffix, getter in (
("total", _meter_total_getter(binding.id, source.id, channel.id)),
("today", _meter_today_getter(binding.id, source.id, channel.id)),
@@ -1021,15 +1042,20 @@ def _m8_energy_provider(session: Session) -> list[ExposableEntity]:
heating, hot_water = active_by_commodity.get("heating"), active_by_commodity.get("hot_water")
if heating is not None and hot_water is not None:
# Keep the v1.6.1 canonical identity/key seed stable. Discovery topic
# segments are sanitised independently by ha_discovery, so this dot is
# never emitted in a topic while existing toggle rows remain usable.
identity = ".".join(sorted((heating.uuid, hot_water.uuid)))
currency = _thermal_currency(session)
cost_info = DeviceInfo(
identifiers=("thermal-cost", identity), name="Thermal Energy Cost",
identifiers=(f"home-automation:thermal-cost:{identity}",), name="Thermal Energy Cost",
identity=identity,
provides_availability=False,
)
labels = {
"heating": "Heating", "hot_water_heating": "Hot Water Heating", "water": "Water",
"water_tax": "Water Tax", "fixed": "Fixed", "all_in": "All-in",
"water_tax": "Water Tax", "hot_water_total": "Hot Water",
"fixed": "Fixed", "all_in": "All-in",
}
for suffix, window in (("total", None), ("today", "today")):
for metric, label in labels.items():
@@ -1203,6 +1229,25 @@ def _thermal_cost_getter(metric: str, window: str | None) -> Callable[[Session],
return result["breakdown"]["hot_water"]
if metric == "water_tax":
return result["breakdown"]["hot_water_tax"]
if metric == "hot_water_total":
# Keep the existing component entities variable-only, while making
# the two consumer-facing totals partition the all-in summary.
# Standing costs are assigned by their contractual commodity:
# hot-water network belongs to hot water; the remaining named
# standing fees belong to heating.
return (
result["breakdown"]["hot_water_heating"]
+ result["breakdown"]["hot_water"]
+ result["breakdown"]["hot_water_tax"]
+ result["fixed_breakdown"]["hot_water_network"]
)
if metric == "heating":
return result["breakdown"]["heating"] + sum(
(result["fixed_breakdown"][key] for key in (
"heating_network", "metering", "delivery_set", "other"
)),
0,
)
return result["breakdown"][metric]
return _getter
+79
View File
@@ -2,10 +2,14 @@ from __future__ import annotations
import json
import logging
from time import monotonic
from dataclasses import dataclass, field
from typing import Any
from urllib import error, parse, request
from websockets.exceptions import WebSocketException
from websockets.sync.client import connect
from app.config import Settings
logger = logging.getLogger(__name__)
@@ -57,6 +61,81 @@ class HomeAssistantClient:
self._post_json(f"/api/webhook/{webhook_id}", body, operation="trigger_webhook")
def discovery_registry_bindings(self, unique_ids: set[str]) -> dict[str, set[str]]:
"""Return HA device identifiers currently bound to MQTT unique IDs.
This is deliberately a read-only WebSocket query. MQTT only confirms
broker receipt; the entity/device registries are the authoritative HA
observation that a discovery unload/re-add was actually processed.
"""
self._require_config()
if not unique_ids:
return {}
try:
deadline = monotonic() + self.timeout_seconds
with connect(self._websocket_url(), open_timeout=self.timeout_seconds,
close_timeout=self.timeout_seconds) as websocket:
greeting = json.loads(self._websocket_recv(websocket, deadline))
if greeting.get("type") != "auth_required":
raise HomeAssistantRequestError("Unexpected Home Assistant WebSocket greeting")
websocket.send(json.dumps({"type": "auth", "access_token": self.settings.home_assistant_auth_token}))
auth = json.loads(self._websocket_recv(websocket, deadline))
if auth.get("type") != "auth_ok":
raise HomeAssistantRequestError("Home Assistant WebSocket authentication failed")
entities = self._websocket_command(websocket, 1, "config/entity_registry/list", deadline)
devices = self._websocket_command(websocket, 2, "config/device_registry/list", deadline)
except (OSError, WebSocketException, TimeoutError, ValueError, KeyError, TypeError) as exc:
raise HomeAssistantRequestError("Home Assistant registry query failed") from exc
devices_by_id = {
device["id"]: {
identifier[1]
for identifier in device.get("identifiers", [])
if (
isinstance(identifier, (list, tuple))
and len(identifier) == 2
and identifier[0] == "mqtt"
and isinstance(identifier[1], str)
and identifier[1]
)
}
for device in devices
if isinstance(device, dict) and isinstance(device.get("id"), str)
}
return {
entity["unique_id"]: devices_by_id.get(entity.get("device_id"), set())
for entity in entities
if entity.get("platform") == "mqtt" and entity.get("unique_id") in unique_ids
}
@staticmethod
def _websocket_recv(websocket: Any, deadline: float) -> str:
remaining = deadline - monotonic()
if remaining <= 0:
raise TimeoutError("Home Assistant WebSocket registry query timed out")
return websocket.recv(timeout=remaining)
@classmethod
def _websocket_command(
cls, websocket: Any, message_id: int, command: str, deadline: float
) -> list[dict[str, Any]]:
websocket.send(json.dumps({"id": message_id, "type": command}))
while True:
response = json.loads(cls._websocket_recv(websocket, deadline))
if response.get("id") != message_id:
continue
if not response.get("success"):
raise HomeAssistantRequestError(f"Home Assistant WebSocket {command} failed")
return response.get("result", [])
def _websocket_url(self) -> str:
parsed = parse.urlsplit(self.settings.home_assistant_base_url)
if parsed.scheme not in {"http", "https"} or not parsed.netloc:
raise HomeAssistantConfigError("HOME_ASSISTANT_BASE_URL must be an HTTP(S) URL")
scheme = "wss" if parsed.scheme == "https" else "ws"
path = f"{parsed.path.rstrip('/')}/api/websocket"
return parse.urlunsplit((scheme, parsed.netloc, path, "", ""))
def _require_config(self) -> None:
if self.is_configured():
return
+3 -1
View File
@@ -98,8 +98,10 @@ def _validate_field_value(kind: str, field: SourceConfigField, value: Any) -> No
_check_type(field, value)
if field.name == "path" and not value.startswith("/dev/"):
raise SourceProfileError("warmtelink_serial config path must start with '/dev/'.")
if field.name in {"broker_port", "sample_interval_s", "baudrate"} and value <= 0:
if field.name in {"broker_port", "baudrate"} and value <= 0:
raise SourceProfileError(f"Config field {field.name!r} must be greater than zero.")
if field.name == "sample_interval_s" and value < 0:
raise SourceProfileError("Config field 'sample_interval_s' must not be negative.")
if field.name == "data_bits" and value != 7:
raise SourceProfileError("warmtelink_serial data_bits must be 7.")
if field.name == "parity" and value != "N":
+155 -57
View File
@@ -94,6 +94,12 @@ class MqttManager:
def __init__(self) -> None:
self._client: mqtt.Client | None = None
self._lock = threading.RLock()
# Serialize ownership changes without holding the callback lock while
# loop_stop() joins Paho's network thread.
self._lifecycle_lock = threading.Lock()
self._client_generation = 0
self._client_settings: tuple[object, ...] | None = None
self._client_loop_started = False
# Source replacement/removal is serialized independently from callback
# bookkeeping. In particular, paho's loop_stop() joins its network
# thread, whose callback completion also needs ``_lock``.
@@ -121,7 +127,8 @@ class MqttManager:
@property
def is_connected(self) -> bool:
"""True if the underlying paho client is currently connected."""
return self._connected
with self._lock:
return self._connected
# ------------------------------------------------------------------
# Public interface
@@ -141,21 +148,29 @@ class MqttManager:
logger.debug("MQTT not configured or not enabled — skipping connect.")
return
with self._lock:
self._start_client(settings)
with self._lifecycle_lock:
with self._lock:
if self._client is not None and self._client_settings == self._settings_key(settings):
thread = getattr(self._client, "_thread", None)
is_alive = getattr(thread, "is_alive", None)
if thread is None and not self._client_loop_started:
return
if thread is not None and (not callable(is_alive) or is_alive()):
return
self._replace_client(settings)
def disconnect(self) -> None:
"""Disconnect from the broker and stop the background loop thread.
No-op if no client is active.
"""
with self._lock:
with self._lifecycle_lock:
self._stop_client()
with self._source_lifecycle_lock:
with self._lock:
source_ids = list(self._source_clients)
for source_id in source_ids:
self._stop_source_client(source_id)
with self._source_lifecycle_lock:
with self._lock:
source_ids = list(self._source_clients)
for source_id in source_ids:
self._stop_source_client(source_id)
def reconnect(self, settings: Settings) -> None:
"""Disconnect the current client (if any) and reconnect with *settings*.
@@ -163,9 +178,23 @@ class MqttManager:
Call this after saving updated MQTT configuration values.
No-op if MQTT is not enabled or broker host is not set.
"""
with self._lock:
self._stop_client()
self.connect(settings)
if not _is_configured(settings):
# A configuration update only owns the application-wide client.
# DSMR sources have independent settings and lifecycle; only app
# shutdown via disconnect() is allowed to stop them all.
with self._lifecycle_lock:
self._stop_client()
return
with self._lifecycle_lock:
with self._lock:
if self._client is not None and self._client_settings == self._settings_key(settings):
thread = getattr(self._client, "_thread", None)
is_alive = getattr(thread, "is_alive", None)
if thread is None and not self._client_loop_started:
return
if thread is not None and (not callable(is_alive) or is_alive()):
return
self._replace_client(settings)
def publish(
self,
@@ -174,23 +203,34 @@ class MqttManager:
*,
retain: bool = False,
qos: int = 0,
) -> None:
) -> bool:
"""Publish a message to *topic*.
If the client is not connected the call is silently skipped.
Errors are logged but do not raise.
Return whether paho accepted the message for publication. If the
client is not connected, or paho raises/rejects it, return ``False``;
errors still do not escape this best-effort boundary.
"""
with self._lock:
client = self._client
connected = self._connected
if client is None or not self._connected:
if client is None or not connected:
logger.debug("MQTT publish skipped — not connected (topic=%s).", topic)
return
return False
try:
client.publish(topic, payload=payload, qos=qos, retain=retain)
result = client.publish(topic, payload=payload, qos=qos, retain=retain)
# paho returns MQTTMessageInfo with an integer rc. Keep fakes and
# alternate clients compatible by treating an absent/non-int rc as
# accepted after the call itself succeeded.
rc = getattr(result, "rc", None)
if isinstance(rc, int) and rc != mqtt.MQTT_ERR_SUCCESS:
logger.warning("MQTT publish rejected (topic=%s, rc=%s).", topic, rc)
return False
return True
except Exception:
logger.exception("MQTT publish error (topic=%s).", topic)
return False
def subscribe(self, topic: str, handler: Callable[[bytes], None]) -> None:
"""Register *handler* to be called when a message arrives on *topic*.
@@ -202,7 +242,8 @@ class MqttManager:
Handler exceptions are swallowed by the ``on_message`` dispatcher so
that a buggy handler can never crash the paho loop thread.
"""
self._subscriptions[topic] = handler
with self._lock:
self._subscriptions[topic] = handler
# Grab the client reference outside the lock to avoid holding the lock
# while calling back into paho (which may itself acquire internal locks).
with self._lock:
@@ -225,8 +266,8 @@ class MqttManager:
immediately; either way the handler is removed from the registry so it
will not be re-subscribed on the next ``_on_connect``.
"""
self._subscriptions.pop(topic, None)
with self._lock:
self._subscriptions.pop(topic, None)
client = self._client
connected = self._connected
@@ -378,13 +419,36 @@ class MqttManager:
# Internal helpers
# ------------------------------------------------------------------
def _start_client(self, settings: Settings) -> None:
"""Build a fresh paho Client, configure it, and call loop_start + connect."""
client = mqtt.Client(
callback_api_version=mqtt.CallbackAPIVersion.VERSION2,
client_id=settings.mqtt_client_id,
@staticmethod
def _settings_key(settings: Settings) -> tuple[object, ...]:
return (
settings.mqtt_enabled,
settings.mqtt_broker_host,
settings.mqtt_broker_port,
settings.mqtt_username,
settings.mqtt_password,
settings.mqtt_tls_enabled,
settings.mqtt_client_id,
)
def _replace_client(self, settings: Settings) -> None:
self._stop_client()
self._start_client(settings)
def _start_client(self, settings: Settings) -> None:
"""Install one generation before starting Paho's reconnecting loop."""
with self._lock:
self._client_generation += 1
generation = self._client_generation
try:
client = mqtt.Client(
callback_api_version=mqtt.CallbackAPIVersion.VERSION2,
client_id=settings.mqtt_client_id,
)
except Exception:
logger.exception("MQTT client initialization failed.")
return
# Callbacks — VERSION2 on_connect signature:
# (client, userdata, connect_flags, reason_code, properties)
def _on_connect(
@@ -394,21 +458,21 @@ class MqttManager:
reason_code: mqtt.ReasonCode,
_properties: mqtt.Properties | None,
) -> None:
if reason_code.is_failure:
logger.warning("MQTT connection refused: %s", reason_code)
self._connected = False
else:
logger.info("MQTT connected (reason_code=%s).", reason_code)
with self._lock:
if self._client is not _client or self._client_generation != generation:
return
if reason_code.is_failure:
self._connected = False
logger.warning("MQTT connection refused: %s", reason_code)
return
self._connected = True
# Re-establish all registered subscriptions after (re-)connect.
for sub_topic in self._subscriptions:
try:
_client.subscribe(sub_topic)
logger.debug("MQTT re-subscribed to topic=%s.", sub_topic)
except Exception:
logger.exception(
"MQTT re-subscribe failed (topic=%s).", sub_topic
)
topics = tuple(self._subscriptions)
logger.info("MQTT connected (reason_code=%s).", reason_code)
for sub_topic in topics:
try:
_client.subscribe(sub_topic)
except Exception:
logger.exception("MQTT re-subscribe failed (topic=%s).", sub_topic)
# VERSION2 on_disconnect signature:
# (client, userdata, disconnect_flags, reason_code, properties)
@@ -419,7 +483,10 @@ class MqttManager:
reason_code: mqtt.ReasonCode,
_properties: mqtt.Properties | None,
) -> None:
self._connected = False
with self._lock:
if self._client is not _client or self._client_generation != generation:
return
self._connected = False
if reason_code.is_failure:
logger.warning("MQTT disconnected unexpectedly (reason_code=%s).", reason_code)
else:
@@ -431,7 +498,10 @@ class MqttManager:
_userdata: object,
message: mqtt.MQTTMessage,
) -> None:
handler = self._subscriptions.get(message.topic)
with self._lock:
if self._client is not _client or self._client_generation != generation:
return
handler = self._subscriptions.get(message.topic)
if handler is None:
logger.debug(
"MQTT on_message: no handler for topic=%s.", message.topic
@@ -459,35 +529,57 @@ class MqttManager:
# Credentials — never log the password
if settings.mqtt_username:
client.username_pw_set(
username=settings.mqtt_username,
password=settings.mqtt_password or None,
)
# Start background loop thread *before* connect so paho can handle
# the connection handshake asynchronously.
client.loop_start()
try:
client.username_pw_set(
username=settings.mqtt_username,
password=settings.mqtt_password or None,
)
except Exception:
logger.exception("MQTT credentials setup failed.")
return
# Publish ownership before network processing/callbacks can begin.
with self._lock:
self._client = client
self._client_settings = self._settings_key(settings)
self._connected = False
self._client_loop_started = False
try:
client.connect(
client.connect_async(
host=settings.mqtt_broker_host,
port=settings.mqtt_broker_port,
keepalive=60,
)
result = client.loop_start()
if isinstance(result, int) and result != mqtt.MQTT_ERR_SUCCESS:
raise RuntimeError(f"MQTT loop_start failed (rc={result})")
with self._lock:
if self._client is client and self._client_generation == generation:
self._client_loop_started = True
except Exception:
# Remove ownership before joining the loop; callbacks from this
# generation become inert immediately.
with self._lock:
if self._client is client and self._client_generation == generation:
self._client = None
self._client_settings = None
self._connected = False
self._client_loop_started = False
# Log without leaking the password
logger.exception(
"MQTT connect failed (host=%s, port=%s). Stopping loop.",
settings.mqtt_broker_host,
settings.mqtt_broker_port,
)
try:
client.disconnect()
except Exception:
pass
try:
client.loop_stop()
except Exception:
pass
return
self._client = client
logger.info(
"MQTT client started (host=%s, port=%s).",
settings.mqtt_broker_host,
@@ -496,11 +588,17 @@ class MqttManager:
def _stop_client(self) -> None:
"""Disconnect and stop the background loop. Idempotent."""
client = self._client
if client is None:
return
self._client = None
self._connected = False
with self._lock:
client = self._client
if client is None:
self._connected = False
self._client_settings = None
return
self._client = None
self._client_settings = None
self._connected = False
self._client_loop_started = False
self._client_generation += 1
try:
client.disconnect()
except Exception:
+8 -4
View File
@@ -38,7 +38,7 @@ from app.services.config_page import build_runtime_settings, seed_missing_config
from app.services.dsmr_ingest import apply_dsmr_subscription
from app.services.public_ip import check_public_ipv4_and_notify
from app.services.modbus_poll import poll_all_enabled_devices, BASE_POLL_TICK_SECONDS
from app.services.ha_discovery import publish_discovery, publish_states
from app.services.ha_discovery import initialize_legacy_thermal_cleanup, publish_discovery, publish_states
from app.services.tibber_prices import run_tibber_refresh_best_effort
from app.services.energy_cost import compute_closed_periods
from app.services.meter_cost import compute_closed_periods as compute_closed_meter_cost_periods
@@ -169,9 +169,12 @@ def _run_scheduled_ha_state_publish() -> None:
session: Session = session_local()
try:
runtime_settings = build_runtime_settings(session, get_settings())
# Reconnect if MQTT is configured (in DB) but not yet connected.
if mqtt_manager.is_configured(runtime_settings) and not mqtt_manager.is_connected:
logger.info("_run_scheduled_ha_state_publish: MQTT configured but not connected — attempting connect.")
# Ensure the configured MQTT client is installed; the manager avoids
# replacing it while Paho is already retrying a transient disconnect.
if mqtt_manager.is_configured(runtime_settings):
# connect() is idempotent for the same settings and retains a Paho
# client while its own loop is retrying a transient disconnect.
logger.info("_run_scheduled_ha_state_publish: ensuring MQTT client is active.")
mqtt_manager.connect(runtime_settings)
publish_discovery(session)
publish_states(session)
@@ -203,6 +206,7 @@ def ensure_auth_db_ready() -> None:
initialize_auth_schema(session, get_settings())
seed_missing_config_from_bootstrap(session, get_settings())
sync_app_hostname_from_bootstrap(session, get_settings())
initialize_legacy_thermal_cleanup(session)
except AppDatabaseAdoptionError as exc:
raise RuntimeError(str(exc)) from exc
except AuthBootstrapError as exc:
+448 -56
View File
@@ -34,16 +34,30 @@ from __future__ import annotations
import json
import logging
import re
from datetime import UTC, datetime
from typing import Any
from sqlalchemy.orm import Session
from sqlalchemy import inspect
from app.integrations.expose import ExposableEntity, build_catalog
from app.models.config import AppConfigEntry
from app.integrations.mqtt import mqtt_manager
from app.services.config_page import build_runtime_settings
logger = logging.getLogger(__name__)
_HA_SEGMENT_RE = re.compile(r"[^A-Za-z0-9_-]+")
# These durable, versioned application metadata entries prevent the minute
# publisher from repeatedly emitting already accepted known-bad v1.6.1 thermal
# topics. A failed publish is not acknowledged and remains retryable across
# restarts. They are intentionally application metadata, rather than broker
# state. An empty retained publish to an illegal v1.6.1 topic cannot be used
# as an acknowledgement: HA rejects that topic before it processes its payload.
_LEGACY_THERMAL_CLEANUP_KEY = "HA_DISCOVERY_LEGACY_THERMAL_CLEANUP_V1"
_REGISTRY_REPAIR_KEY = "HA_DISCOVERY_REGISTRY_REPAIR_V2"
# ---------------------------------------------------------------------------
# Internal helpers
@@ -59,14 +73,25 @@ def _should_publish(settings: Any) -> bool:
)
def _node_id(device_uuid: str) -> str:
"""Stable MQTT node_id derived from device uuid (replace hyphens for safety)."""
return device_uuid.replace("-", "_")
def _safe_segment(value: str) -> str:
"""Return a deterministic HA discovery topic segment.
Home Assistant accepts only ``[A-Za-z0-9_-]+`` for node/object ids.
Replacing each run of other characters keeps ordinary UUID/key output
readable while ensuring composite identities cannot produce invalid topics.
"""
result = _HA_SEGMENT_RE.sub("_", value).strip("_")
return result or "home_automation"
def _node_id(identity: str) -> str:
"""Stable, strictly valid MQTT node_id from an internal identity."""
return _safe_segment(identity.replace("-", "_"))
def _object_id(entity: ExposableEntity) -> str:
"""Stable MQTT object_id derived from entity key (dots + hyphens → underscores)."""
return entity.key.replace(".", "_").replace("-", "_")
"""Stable, strictly valid MQTT object_id derived from the entity key."""
return _safe_segment(entity.key.replace("-", "_"))
def _discovery_topic(entity: ExposableEntity, prefix: str) -> str:
@@ -74,7 +99,7 @@ def _discovery_topic(entity: ExposableEntity, prefix: str) -> str:
Format: ``<prefix>/<component>/<node_id>/<object_id>/config``
"""
node = _node_id(entity.device.identifiers[1]) # device uuid
node = _node_id(entity.device.internal_identity)
obj = _object_id(entity)
return f"{prefix}/{entity.component}/{node}/{obj}/config"
@@ -84,7 +109,7 @@ def _state_topic(entity: ExposableEntity, prefix: str) -> str:
Format: ``<prefix>/<component>/<node_id>/<object_id>/state``
"""
node = _node_id(entity.device.identifiers[1])
node = _node_id(entity.device.internal_identity)
obj = _object_id(entity)
return f"{prefix}/{entity.component}/{node}/{obj}/state"
@@ -104,16 +129,133 @@ def _availability_id(entity: ExposableEntity) -> str:
M8 meters deliberately retain their own UUID as HA node/unique identity,
while their availability is supplied by a MeterSource UUID.
"""
return entity.device.availability_id or entity.device.identifiers[1]
return entity.device.availability_id or entity.device.internal_identity
def _unique_id(entity: ExposableEntity) -> str:
"""Stable unique_id — device uuid + metric key (never from mutable fields)."""
device_uuid = entity.device.identifiers[1]
device_uuid = entity.device.internal_identity
# entity.key is already "modbus.<uuid>.<metric_key>" — use it as the seed
return f"{device_uuid}_{entity.key.replace('.', '_')}"
def _migration_state(session: Session, key: str, default: str) -> str:
"""Read a versioned discovery-repair acknowledgement from ``app_config``."""
entry = session.query(AppConfigEntry).filter(AppConfigEntry.key == key).one_or_none()
return entry.value if entry is not None else default
def _has_repair_store(session: Session) -> bool:
"""Whether this session uses the app schema (not a lightweight unit DB)."""
return inspect(session.get_bind()).has_table("app_config")
def _set_migration_state(session: Session, key: str, value: str) -> None:
"""Durably acknowledge a completed discovery-repair step.
The state is operational metadata only; it never alters meters, readings,
contracts, costs, or expose toggles.
"""
entry = session.query(AppConfigEntry).filter(AppConfigEntry.key == key).one_or_none()
if entry is None:
session.add(AppConfigEntry(key=key, value=value, updated_at=datetime.now(UTC)))
else:
entry.value = value
entry.updated_at = datetime.now(UTC)
session.commit()
def _migration_json(session: Session, key: str) -> dict[str, Any]:
"""Read a versioned JSON migration ledger, treating corrupt values as pending."""
raw = _migration_state(session, key, "{}")
try:
value = json.loads(raw)
except json.JSONDecodeError:
logger.warning("Ignoring malformed HA discovery migration ledger %s", key)
return {}
return value if isinstance(value, dict) else {}
def _set_migration_json(session: Session, key: str, value: dict[str, Any]) -> None:
_set_migration_state(session, key, json.dumps(value, sort_keys=True))
def _frozen_legacy_inventory(ledger: dict[str, Any]) -> list[str] | None:
"""Return a structurally valid immutable cleanup inventory, if present.
The ledger predates a schema migration, so it must tolerate both an absent
entry and old ``topics``-only values. An inventory becomes immutable only
once it is a list of non-empty topic strings; anything else is compatibility
data to be frozen during startup, not state a publisher may reinterpret.
"""
value = ledger.get("inventory")
if not isinstance(value, list) or not all(isinstance(topic, str) and topic for topic in value):
return None
return list(dict.fromkeys(value))
def _publish_ok(topic: str, payload: str | bytes, *, retain: bool = True) -> bool:
"""Publish best-effort while treating only an explicit ``False`` as failure.
Production ``MqttManager.publish`` returns ``bool``. Accepting ``None``
keeps existing third-party/mock publishers best-effort compatible.
"""
return mqtt_manager.publish(topic, payload, retain=retain) is not False
def initialize_legacy_thermal_cleanup(session: Session) -> None:
"""Create the legacy-cleanup ledger before expose toggles can be edited.
A newly installed database has no v1.6.1 retained thermal topics. Marking
that case complete during startup prevents a later first-time toggle from
manufacturing an old, illegal topic. Conversely, an upgraded database
freezes its exact enabled legacy topics before the UI can change a toggle.
"""
if not _has_repair_store(session):
return
ledger = _migration_json(session, _LEGACY_THERMAL_CLEANUP_KEY)
if ledger.get("complete") is True:
return
inventory = _frozen_legacy_inventory(ledger)
if inventory is not None:
# A previously frozen non-empty inventory must never be expanded or
# recomputed from mutable toggle/prefix state. A zero-item inventory
# is just the fresh-install terminal state written by an older build.
if not inventory:
_set_migration_json(
session,
_LEGACY_THERMAL_CLEANUP_KEY,
{"complete": True, "inventory": [], "topics": []},
)
return
# This deliberately propagates. Startup runs before the expose UI can
# write a toggle, so swallowing an enumeration or durable-write error would
# create a window in which the cleanup scope could be changed or lost.
legacy_entities = _legacy_thermal_entities(session)
from app.config import get_settings
settings = build_runtime_settings(session, get_settings())
inventory = list(dict.fromkeys(
_legacy_discovery_topic(entity, settings.ha_discovery_prefix)
for entity in legacy_entities
))
_set_migration_json(
session,
_LEGACY_THERMAL_CLEANUP_KEY,
{
"complete": not inventory,
"inventory": inventory,
"topics": sorted({
topic for topic in ledger.get("topics", [])
if isinstance(topic, str) and topic in inventory
}),
},
)
# ---------------------------------------------------------------------------
# Public: build discovery payload
# ---------------------------------------------------------------------------
@@ -220,34 +362,90 @@ def publish_discovery(session: Session) -> None:
logger.exception("publish_discovery: failed to build catalog; aborting")
return
# Meter UUIDs are intentionally identity-changing epochs. Discovery config
# is retained, so clear only the precisely enumerable old M8 identities;
# never wildcard a provider/topic and risk removing another source's card.
try:
stale_entities = _stale_m8_entities(session)
except Exception:
logger.exception("publish_discovery: unable to enumerate old M8 identities")
stale_entities = []
for old_entity in stale_entities:
# Clear the *illegal* v1.6.1 thermal keys once. Startup freezes the exact
# inventory while the legacy toggle state still describes v1.6.1. Thus a
# later UI toggle/prefix/meter change cannot erase or expand this repair.
# ``topics`` is only durable success progress; ``inventory`` is immutable.
# Startup freezes compatibility ledgers before the UI can mutate the
# toggle/prefix inputs. The publisher only consumes that frozen snapshot.
repaired_keys: set[str] = set()
if _has_repair_store(session):
ledger = _migration_json(session, _LEGACY_THERMAL_CLEANUP_KEY)
if not ledger.get("complete"):
inventory = _frozen_legacy_inventory(ledger)
if inventory is None:
logger.error(
"publish_discovery: legacy cleanup ledger was not frozen at startup; refusing cleanup"
)
if inventory is not None:
completed = {
topic for topic in ledger.get("topics", [])
if isinstance(topic, str) and topic in inventory
}
for topic in (topic for topic in inventory if topic not in completed):
try:
if _publish_ok(topic, b""):
completed.add(topic)
# Commit each accepted illegal-topic tombstone before
# attempting another one: a crash must not replay it.
_set_migration_json(
session,
_LEGACY_THERMAL_CLEANUP_KEY,
{
"complete": False,
"inventory": inventory,
"topics": sorted(completed),
},
)
else:
logger.warning("publish_discovery: broker rejected v1.6.1 cleanup for %s", topic)
except Exception:
logger.exception("publish_discovery: unable to clear v1.6.1 topic %s", topic)
if set(inventory).issubset(completed):
_set_migration_json(
session,
_LEGACY_THERMAL_CLEANUP_KEY,
{"complete": True, "inventory": inventory, "topics": sorted(completed)},
)
# Current-format stale topics are legal and deliberately remain retryable:
# they cover post-repair meter swaps and include all 14 thermal metrics.
try:
old_topic, _ = build_discovery_payload(old_entity, discovery_prefix, state_prefix)
mqtt_manager.publish(old_topic, b"", retain=True)
stale_entities = _stale_m8_entities(session)
except Exception:
logger.exception("publish_discovery: unable to clear old identity %r", old_entity.key)
logger.exception("publish_discovery: unable to enumerate stale M8 identities")
stale_entities = []
for old_entity in stale_entities:
try:
if not _publish_ok(_discovery_topic(old_entity, discovery_prefix), b""):
logger.warning("publish_discovery: broker rejected stale cleanup for %r", old_entity.key)
except Exception:
logger.exception("publish_discovery: unable to clear stale identity %r", old_entity.key)
# v1.6.1 used the same legal topics/unique_ids for Meter, Source, Modbus
# and electricity-cost entities, but attached their registry entries to
# wrongly merged HA devices. HA does not move those entries when only a
# discovery ``device`` block changes. One durable unload -> re-add cycle
# lets HA rebuild the existing unique_id entries against the new singleton
# identifiers, preserving entity keys, toggles and user customizations.
repaired_keys = _run_registry_repair(session, catalog, discovery_prefix, state_prefix, settings)
for entry in catalog:
entity = entry.entity
if entity.key in repaired_keys:
continue
try:
topic, config = build_discovery_payload(entity, discovery_prefix, state_prefix)
if entry.enabled:
payload = json.dumps(config)
mqtt_manager.publish(topic, payload, retain=True)
_publish_ok(topic, payload)
logger.debug(
"publish_discovery: published config for %r → %s", entity.key, topic
)
else:
# Clear retained config for disabled entities.
mqtt_manager.publish(topic, b"", retain=True)
_publish_ok(topic, b"")
logger.debug(
"publish_discovery: cleared config for disabled entity %r → %s",
entity.key,
@@ -259,39 +457,163 @@ def publish_discovery(session: Session) -> None:
)
def _stale_m8_entities(session: Session) -> list[ExposableEntity]:
"""Return synthetic discovery entries for superseded thermal identities.
def _is_registry_repair_entity(entity: ExposableEntity) -> bool:
"""Whether a legal v1.6.1 config needs an unload/re-add device split."""
return entity.key.startswith(("modbus.", "source.", "meter.", "energy.", "thermal_cost."))
This is deliberately a narrow, best-effort cleanup: ended Meter UUIDs and
historically possible thermal combinations only; current identities are excluded.
def _ha_registry_bindings(settings: Any, unique_ids: set[str]) -> dict[str, set[str]] | None:
"""Read HA's registry, returning ``None`` when it is not observable."""
from app.integrations.homeassistant import (
HomeAssistantClient,
HomeAssistantConfigError,
HomeAssistantRequestError,
)
# Runtime settings from old callers/tests may not expose outbound HA
# fields. Treat absent/non-string credentials as intentionally optional.
if not isinstance(getattr(settings, "home_assistant_base_url", None), str) or not isinstance(
getattr(settings, "home_assistant_auth_token", None), str
):
return None
client = HomeAssistantClient(settings)
if not client.is_configured():
return None
try:
return client.discovery_registry_bindings(unique_ids)
except (HomeAssistantConfigError, HomeAssistantRequestError, TypeError, ValueError):
logger.warning("HA registry repair remains pending: HA registry is unavailable")
return None
def _run_registry_repair(
session: Session,
catalog: list[Any],
discovery_prefix: str,
state_prefix: str,
settings: Any,
) -> set[str]:
"""Repair each v1.6.1 entity only after HA confirms every phase.
MQTT acknowledges broker receipt, not Home Assistant processing. The
ledger therefore holds each target at ``unloaded`` until HA's entity
registry no longer contains its stable unique_id, then at ``republished``
until HA reports the expected singleton device identifier. A partial
catalog simply adds targets on a later run; it cannot complete others.
When HA's registry is unavailable, ordinary discovery remains untouched.
"""
from app.integrations.expose import DeviceInfo
targets = [entry for entry in catalog if _is_registry_repair_entity(entry.entity)]
if not targets:
return set()
ledger = _migration_json(session, _REGISTRY_REPAIR_KEY)
pending_targets = [
entry for entry in targets if ledger.get(_unique_id(entry.entity), {}).get("phase") != "complete"
]
if not pending_targets:
return set()
unique_ids = {_unique_id(entry.entity) for entry in pending_targets}
bindings = _ha_registry_bindings(settings, unique_ids)
if bindings is None:
return set()
blocked: set[str] = set()
for entry in pending_targets:
entity = entry.entity
unique_id = _unique_id(entity)
expected = set(entity.device.identifiers)
phase = ledger.get(unique_id, {}).get("phase", "pending")
observed = bindings.get(unique_id)
if phase == "unloaded":
if observed is not None:
# HA was disconnected or otherwise missed the retained
# tombstone. Keep it retained until HA itself confirms delete.
try:
_publish_ok(_discovery_topic(entity, discovery_prefix), b"")
except Exception:
logger.exception("publish_discovery: unable to repeat registry unload for %r", entity.key)
blocked.add(entity.key)
continue
if not entry.enabled:
_set_registry_phase(session, ledger, unique_id, "complete", expected)
continue
_publish_registry_config(session, ledger, entry, discovery_prefix, state_prefix, expected, blocked)
elif phase == "republished":
if observed == expected:
_set_registry_phase(session, ledger, unique_id, "complete", expected)
elif observed is None and entry.enabled:
_publish_registry_config(session, ledger, entry, discovery_prefix, state_prefix, expected, blocked)
else:
# A stale/wrong device binding must pass through deletion again.
try:
if _publish_ok(_discovery_topic(entity, discovery_prefix), b""):
_set_registry_phase(session, ledger, unique_id, "unloaded", expected)
except Exception:
logger.exception("publish_discovery: unable to unload wrong registry entity %r", entity.key)
blocked.add(entity.key)
elif phase != "complete":
if observed == expected or (observed is None and not entry.enabled):
_set_registry_phase(session, ledger, unique_id, "complete", expected)
elif observed is None:
_publish_registry_config(session, ledger, entry, discovery_prefix, state_prefix, expected, blocked)
else:
try:
if _publish_ok(_discovery_topic(entity, discovery_prefix), b""):
_set_registry_phase(session, ledger, unique_id, "unloaded", expected)
blocked.add(entity.key)
else:
logger.warning("publish_discovery: broker rejected registry unload for %r", entity.key)
except Exception:
logger.exception("publish_discovery: unable to unload registry entity %r", entity.key)
return blocked
def _set_registry_phase(
session: Session, ledger: dict[str, Any], unique_id: str, phase: str, expected: set[str]
) -> None:
ledger[unique_id] = {"phase": phase, "identifier": sorted(expected)}
_set_migration_json(session, _REGISTRY_REPAIR_KEY, ledger)
def _publish_registry_config(
session: Session, ledger: dict[str, Any], entry: Any, discovery_prefix: str,
state_prefix: str, expected: set[str], blocked: set[str],
) -> None:
entity = entry.entity
topic, config = build_discovery_payload(entity, discovery_prefix, state_prefix)
try:
if _publish_ok(topic, json.dumps(config)):
_set_registry_phase(session, ledger, _unique_id(entity), "republished", expected)
else:
logger.warning("publish_discovery: broker rejected registry re-add for %r", entity.key)
except Exception:
logger.exception("publish_discovery: unable to re-add registry entity %r", entity.key)
blocked.add(entity.key)
def _legacy_discovery_topic(entity: ExposableEntity, prefix: str) -> str:
"""Return the exact v1.6.1 config topic for a known stale M8 entity.
This intentionally preserves the old hyphen-only conversion because the
point is to remove that exact retained broker key once, never to publish a
wildcard or manufacture a new invalid topic.
"""
node = entity.device.internal_identity.replace("-", "_")
obj = entity.key.replace(".", "_").replace("-", "_")
return f"{prefix}/{entity.component}/{node}/{obj}/config"
def _overlapping_thermal_pairs(session: Session) -> list[tuple[Any, Any]]:
"""Return only heating/water Meter epochs that could have coexisted."""
from app.models.energy import Meter
from sqlalchemy import select
meters = session.execute(select(Meter).where(
Meter.commodity.in_(("electricity", "heating", "hot_water"))
)).scalars().all()
current = {meter.commodity: meter for meter in meters if meter.ended_at is None}
old = [meter for meter in meters if meter.ended_at is not None]
entities: list[ExposableEntity] = []
for meter in old:
info = DeviceInfo(identifiers=("meter", meter.uuid), name=meter.label)
for suffix in ("total", "today"):
entities.append(ExposableEntity(
key=f"meter.{meter.uuid}.{suffix}", component="sensor", device=info,
device_class=None, unit="", name="obsolete",
))
heatings = [meter for meter in meters if meter.commodity == "heating"]
waters = [meter for meter in meters if meter.commodity == "hot_water"]
current_identity = (
".".join(sorted((current["heating"].uuid, current["hot_water"].uuid)))
if current.get("heating") is not None and current.get("hot_water") is not None else None
)
pairs: list[tuple[Any, Any]] = []
for heating in heatings:
for water in waters:
if heating.ended_at is None and water.ended_at is None:
continue
# A thermal identity can only have been published when both Meter
# epochs were current at the same instant. Do not form a Cartesian
# product of historical records: that would tombstone identities
@@ -302,16 +624,86 @@ def _stale_m8_entities(session: Session) -> list[ExposableEntity]:
water_end is not None and heating_start >= water_end
):
continue
identity = ".".join(sorted((heating.uuid, water.uuid)))
if identity == current_identity:
continue
info = DeviceInfo(identifiers=("thermal-cost", identity), name="obsolete")
for metric in ("heating", "hot_water_heating", "water", "water_tax", "fixed", "all_in"):
for suffix in ("total", "today"):
entities.append(ExposableEntity(
key=f"thermal_cost.{identity}.{metric}_{suffix}", component="sensor", device=info,
device_class=None, unit="", name="obsolete",
))
pairs.append((heating, water))
return pairs
def _thermal_cleanup_entities(
pairs: list[tuple[Any, Any]], *, include_hot_water_total: bool
) -> list[ExposableEntity]:
"""Build exact synthetic config entries for known thermal identities."""
from app.integrations.expose import DeviceInfo
metrics = ["heating", "hot_water_heating", "water", "water_tax", "fixed", "all_in"]
if include_hot_water_total:
metrics.insert(2, "hot_water_total")
entities: list[ExposableEntity] = []
for heating, water in pairs:
identity = ".".join(sorted((heating.uuid, water.uuid)))
info = DeviceInfo(
identifiers=(f"home-automation:thermal-cost:{identity}",),
name="obsolete",
identity=identity,
)
for metric in metrics:
for suffix in ("total", "today"):
entities.append(ExposableEntity(
key=f"thermal_cost.{identity}.{metric}_{suffix}", component="sensor", device=info,
device_class=None, unit="", name="obsolete",
))
return entities
def _legacy_thermal_entities(session: Session) -> list[ExposableEntity]:
"""Return only v1.6.1 illegal topics that could retain a non-empty config.
v1.6.1 published an illegal thermal config only while its expose toggle was
enabled. A disabled toggle published its own tombstone, so querying the
durable toggle state prevents a fresh install from manufacturing warnings
for twelve never-used illegal topics.
"""
from app.models.expose import ExposedEntityToggle
entities = _thermal_cleanup_entities(_overlapping_thermal_pairs(session), include_hot_water_total=False)
keys = [entity.key for entity in entities]
enabled_keys = {
row.key for row in session.query(ExposedEntityToggle).filter(
ExposedEntityToggle.key.in_(keys), ExposedEntityToggle.enabled.is_(True)
)
}
return [entity for entity in entities if entity.key in enabled_keys]
def _stale_m8_entities(session: Session) -> list[ExposableEntity]:
"""Return *safe-format* configs that became stale after a later epoch swap.
Unlike the v1.6.1 cleanup, this intentionally excludes the active thermal
pair and includes the post-repair ``hot_water_total`` metric (14 configs
per stale pair). All generated topics use ``_discovery_topic``.
"""
from app.integrations.expose import DeviceInfo
from app.models.energy import Meter
from sqlalchemy import select
meters = session.execute(select(Meter).where(
Meter.commodity.in_(("electricity", "heating", "hot_water"))
)).scalars().all()
entities: list[ExposableEntity] = []
for meter in (meter for meter in meters if meter.ended_at is not None):
info = DeviceInfo(
identifiers=(f"home-automation:meter:{meter.uuid}",), name=meter.label, identity=meter.uuid
)
for suffix in ("total", "today"):
entities.append(ExposableEntity(
key=f"meter.{meter.uuid}.{suffix}", component="sensor", device=info,
device_class=None, unit="", name="obsolete",
))
stale_pairs = [
(heating, water)
for heating, water in _overlapping_thermal_pairs(session)
if heating.ended_at is not None or water.ended_at is not None
]
entities.extend(_thermal_cleanup_entities(stale_pairs, include_hot_water_total=True))
return entities
@@ -468,7 +860,7 @@ def publish_device_state(session: Session, device: Any) -> None:
continue
entity = entry.entity
# Only process entities belonging to this device.
if entity.device.identifiers[1] != device_uuid:
if entity.device.internal_identity != device_uuid:
continue
# Skip the online binary_sensor itself (availability handled above).
if entity.component == "binary_sensor" and "online" in entity.key:
@@ -543,7 +935,7 @@ def clear_device_discovery(session: Session, device_uuid: str) -> None:
for entry in catalog:
entity = entry.entity
# Only clear entities belonging to this device.
if entity.device.identifiers[1] != device_uuid:
if entity.device.internal_identity != device_uuid:
continue
try:
topic, _config = build_discovery_payload(entity, discovery_prefix, state_prefix)
+10
View File
@@ -5,6 +5,11 @@ services:
user: "1000:1000"
restart: "no"
init: true
logging:
driver: json-file
options:
max-size: "10m"
max-file: "3"
command: ["python", "-m", "scripts.run_migrations"]
environment:
TZ: "${TZ:-Europe/Amsterdam}"
@@ -19,6 +24,11 @@ services:
user: "1000:1000"
restart: unless-stopped
init: true
logging:
driver: json-file
options:
max-size: "10m"
max-file: "3"
environment:
TZ: "${TZ:-Europe/Amsterdam}"
depends_on:
+20
View File
@@ -1214,6 +1214,26 @@ M8 收尾的前置条件。agent 不得执行、记录为已执行,或以 mock
## 14. Post-M8 lifecycle repair(M8-R08~R10)
### M8-R16 — HA Water Energy Discovery metadata
active `hot_water` Meter 的 total/today MQTT Discovery metadata 从通用 `volume` 修正为 Home
Assistant Energy Water 所要求的 `water`;保留 `m³` 与 `total_increasing`。区域供暖继续使用合法的
`energy + GJ` 组合,thermal EUR 成本实体以及既有 key、identity、unique_id 和 topic 均不变。
### M8-R15 — HA Discovery identity/topic repair
修复 HA 将多值 `device.identifiers` 任一匹配合并的风险:Source、Meter、Modbus 与成本 epoch 均使用单一完整 namespaced identifier,内部 MQTT identity 独立保存。thermal 的既有点号 identity/key 保持兼容,node/object topic segment 单独规范化;仅对 v1.6.1 实际发布过的 dot topic 做可重试、成功后抑制的精确 retained cleanup。新增默认关闭的 `Thermal Hot Water Total/Today`;其后由 R17 明确为 hot-water 的 all-in 分配值。
### M8-R17 — Thermal HA standing-cost allocation
保持全部 thermal HA entity key、unique_id 与名称不变,只调整 `Thermal Heating Total/Today` 与
`Thermal Hot Water Total/Today` 的数值语义。Heating 为 `heating` variable 加上
`heating_network`、`metering`、`delivery_set`、`other` 四项 fixed breakdown;Hot Water 为
`hot_water_heating + hot_water + hot_water_tax` 加上 `hot_water_network`。因此两者之和精确等于
`Thermal All-in Total/Today`,每项 standing 只分配一次。component 实体(hot-water heating、water、
water tax)继续只显示 variable,`Fixed` 继续显示全部 standing;provider 只读取 `meter_cost.summarize`
在 01:05 结算后的结果,不改 15 分钟 ledger、API/schema 或数据库。
M8 交付后的 Meter lifecycle 修复链记录在本地 `review-notes/M8-meter-lifecycle-repair-plan.md`。
它不新增 ORM / **数据库** schema 或 Alembic revision,不做启动自动修复、一次性数据脚本或历史删除;R08 虽然
更新了 API/Pydantic schema 及 OpenAPI/codegen,但没有变更 ORM 或数据库 schema。已有的 stranded binding 只能
+124
View File
@@ -0,0 +1,124 @@
# MQTT 可靠性修复 — 2026-10-11
本次范围为 MQTT-FIX-T01 → MQTT-FIX-T02,依赖顺序执行,两张卡自动继续。
按 [AGENTS.md](../../AGENTS.md) 与 [协作契约](README.md) 实现、独立审查及返工。
`done` 仅代表源码自动化技术验收完成;**源码修复完成后仍须人工部署与生产验证**。
中文实现、审查及返工简报存放于已忽略的 `review-notes/`,不入库。
## 证据与边界
排查日期 2026-10-11,Europe/Amsterdam。以下生产观察由用户提供,未在本轮访问生产确认:
1. 应用当天约 01:00 启动,故障前未自动重启、未发生 OOM。
2. 16:50:43 起出现 `MQTT disconnected unexpectedly (reason_code=Unspecified error).`。
3. 截至约 22:34 累计约 270 万条断线日志;22:34–22:35 一分钟内 15,283 次断线、101 次 publish rejected。
4. 一个 uvicorn 进程、284 个总线程;并非已经确认 284 个 MQTT 客户端。
5. 主 client ID 为 `home-automation`;配置最后更新时间为 2026-08-28,当天未观察到变更。
6. Modbus、WarmteLink 串口及独立 DSMR MQTT source 仍有新数据,不能描述为 broker 整体停止服务。
7. 生产镜像的 `app/main.py`、`app/integrations/mqtt.py` SHA256 与源码一致。
8. 用户在 broker 日志中看到大量新连接及 `session taken over`。
9. 22:37 用户授权重启应用后,观察超过一分钟未见新异常断线或发布失败,各采集源恢复;仅属临时恢复。
10. 首次断线触发原因未知,不能将网络故障或 broker 重启写成事实。
待独立核实的链路:60 秒 HA 发布任务在未连接时调用 `connect()`;它无条件创建同 ID Paho
client/线程、覆盖引用但未停旧 client;旧回调能覆盖共享连接状态。已有离线 fake 复现只是
排查线索,不能代替本轮回归测试。
部署背景:生产日志配置为 `{"Type":"json-file","Config":{}}`;约 276 万行日志,按内容
估算 JSON 文件约 353 MiB(非 stat 精确大小),当时约 666 GB 空闲,无即时容量危机。
生产 Compose 在 `/home/tianyu/.local/share/homeautomation`,与源码 Compose 是两个文件。
**本轮禁止**:push、tag、改写已推送历史、修改生产配置、重启/重建生产容器、连接真实
MQTT broker(包括 `10.238.75.82:1883`)、访问真实串口、改动生产数据库或日志;禁止
清空历史日志、删除 Docker 文件、prune、变更全局 Docker daemon。仅提供部署步骤。
允许源码/测试/部署文档修改、本地 commit、fixup 及整理本轮尚未推送提交。保留已有无关改动。
### MQTT-FIX-T01 — 修复主 MQTT 客户端生命周期
- **Status**: `done`
- **Depends**: none
- **Context**: 防止短暂断线后定时任务创建同 ID 客户端累积,避免旧回调污染新连接。
**Files**
- `modify app/integrations/mqtt.py`
- `modify app/main.py`
- `modify tests/test_mqtt_client.py`
- `modify tests/test_mqtt_subscribe.py`
- `create tests/test_mqtt_scheduler.py`(必要的调度任务回归)
- `modify README.md`(MQTT 生命周期说明)
- `modify .gitignore`(编排过程报告目录保护)
- `create docs/design/mqtt-reliability-fixes.md`(主线维护任务与 Status)
**Steps**
1. 独立核实当前生命周期、回调及调度代码,结合安装的 Paho 2.1.0 源码核对启动顺序和失败清理。
2. 使同配置 `connect()` 幂等,区分正在连接、等待自动重连、无客户端;现有 client 能自动重连时不再创建额外 client。
3. 初始连接失败、初始化失败、后台循环失效均须可恢复,失败对象/循环清理,不能永久跳过重试。
4. 配置变更安全停止、清理旧 client 后替换,受管理的主 client 只有一个。
5. connect/disconnect/message 回调核验身份或 generation,拒绝旧状态及过期消息。
6. 处理创建、替换、关闭与回调并发;不能持有回调需要的锁调用等待网络线程退出的 `loop_stop()`。
7. 保持独立 DSMR source、订阅恢复、MQTT 测试身份及现有 publish 行为兼容。
8. 增加离线故障与并发回归,更新生命周期说明,运行适用闸门并保存中文实现简报。
**Out of scope / 不要碰**
- 不使用随机 ID、单纯换 ID 或降低日志级别掩盖问题。
- 不改变 HTTP 路由/schema/docstring;如确有必要变更,须同步 OpenAPI 和前端类型。
- 不改变旧 milestone 完成状态;不访问生产或真实设备;不处理已有 Compose 串口改动。
**Acceptance criteria**
- [x] 同配置重复 connect 及断线后多轮定时任务不累积 client 或后台循环。
- [x] 初次连接/初始化失败后能重试,失败对象和循环被清理,循环失效也有恢复路径。
- [x] 配置重连后旧 client 停止,旧 connect/disconnect/message 回调不能影响新状态或处理过期消息。
- [x] 当前 client 重连成功后恢复订阅与发布。
- [x] shutdown、reconnect 与回调交错时能完成、不死锁。
- [x] 新测试使用 mock/fake 及合成配置,不连接生产 broker 或真实串口;覆盖行为与并发边界。
- [x] DSMR source、测试 client ID 及现有 publish 行为兼容。
- [x] pytest、ruff 全绿,独立 Reviewer 重跑;如适用同步契约/检查构建上下文。
**Reviewer checklist**
- 独立核查 Paho 2.1.0 生命周期顺序、线程泄漏、状态污染、死锁及失败恢复,不仅断言内部实现细节。
- 核查调度触发路径、多轮断线自动重连、配置替换与关闭交错,以及旧消息拒绝。
- 核查 DSMR 独立客户端及订阅/publish/test identity 回归风险。
- 先读对应实现/返工简报,独立重跑闸门;报告 PASS 或编号返工清单,不改实现或 Git。
### MQTT-FIX-T02 — 为 Docker 日志配置轮转
- **Status**: `done`
- **Depends**: MQTT-FIX-T01
- **Context**: 为本项目容器限定日志保留,明确源码修改与生产容器应用配置是两个操作。
**Files**
- `modify docker-compose.yml`
- `modify docker-compose.dev.yml`(仅必要时)
- `modify README.md`(部署说明)
- `modify tests/test_deployment.py`(必要的静态检查)
- `modify docs/design/mqtt-reliability-fixes.md`(主线维护 Status)
**Steps**
1. 为 app 与 migration 均配置 `logging.driver: json-file`,字符串 options `max-size: "10m"`、`max-file: "3"`。
2. 验证开发 override 合并后保留相同合理的轮转配置,不修改全局 daemon 或其他项目。
3. 用 `docker compose config --quiet` 或等效静态检查验证基础与合并配置;不输出含凭据的完整配置。
4. 说明宿主机日志路径 `/var/lib/docker/containers/<完整ID>/<完整ID>-json.log`、`docker logs`、保留策略。
5. 明确仅 `docker restart` 不应用 logging 配置,需要重建容器。
6. 给出生产目录 `/home/tianyu/.local/share/homeautomation` 中的最小配置同步、重建、验证与回退步骤。
7. 写清发布源码修复镜像与仅应用日志配置各自前提,以及短暂采集中断;不覆盖生产设备/volume/env 等配置。
8. 保存中文简报,跑 pytest / ruff 和配置验证,独立 Reviewer 核查可执行性。
**Out of scope / 不要碰**
- 不自动部署,不清空或删除日志,不 prune,不触碰设备历史数据。
- 不修改全局 Docker daemon;不把源码配置变更描述为生产已生效。
- 不纳入用户已有的串口映射/group_add 改动。
**Acceptance criteria**
- [x] app、migration 最终 Compose 均含指定字符串 logging options。
- [x] 开发 override 合并后日志配置正确。
- [x] Compose 经 config --quiet 或等效静态检查,不输出凭据。
- [x] 文档说明日志路径、docker logs、保留策略、restart 无效而须重建。
- [x] 文档明确生产路径、最小同步、镜像更新前提、采集中断、验证与回退;区分镜像发布和日志配置应用。
- [x] 无生产操作、数据/日志删除,已有无关改动保留。
- [x] pytest、ruff 全绿,独立 Reviewer 重跑并核查合并及部署可执行性。
**Reviewer checklist**
- 核查 logging options 类型及基础/开发配置合并;静态配置检查的范围和凭据保护。
- 核查生产/source Compose 分离,镜像构建发布前提与 logging 重建步骤是否明确。
- 核查只涉及 app/migration,不变更串口、volume、生产 DB、全局 daemon;说明旧日志在重建时的 Docker 生命周期影响及可回退方式。
- 先读对应实现/返工简报,独立重跑闸门,给 PASS 或编号返工清单。
+3 -1
View File
@@ -55,7 +55,9 @@
Expose 框架还可以把已勾选的 Energy 实体通过 MQTT Home Assistant Discovery 发布;开关位于应用 Config 页的 HA Expose 面板,默认均为关闭。M8 增加了 source online、按 Meter UUID 锚定的累计量/today,以及 heating、hot-water-heating、water、water-tax、fixed、all-in total/today 等 thermal 实体。
- source 和 Meter identity 不依赖可变 label;换表会产生新 Meter UUID identity。
- thermal 组合成本 identity 由当前 heating/hot_water Meter UUID 的有序组合锚定,任一换表都会产生新 identity,避免不同累计域拼接。
- thermal 组合成本 identity 由当前 heating/hot_water Meter UUID 的有序组合锚定,任一换表都会产生新 identity,避免不同累计域拼接。`Thermal Hot Water Total/Today` 精确为 `hot_water_heating + hot_water`,不包含 `hot_water_tax`。
- 每个 HA device 只发布一个完整、`home-automation:` namespaced identifier;内部 topic/unique-id seed 与该 identifier 分离。所有 discovery node/object segment 都会转为 HA 允许的 `[A-Za-z0-9_-]+` 字符集。
- v1.6.1 的 dot-containing thermal retained topics 会在启动、UI 尚未能修改 toggle 前,按当时实际重叠的 Meter epoch、enabled toggle 和 runtime discovery prefix 冻结为精确清单;每个成功 topic 都会持久记账,失败/未尝试项才会重试。冻结清单完成后跨重启不再枚举或发布旧 topic;fresh install 的空清单也会立即完成。没有 wildcard,也不会清理任何应用数据。旧 device 合并修复则以已配置的 Home Assistant WebSocket entity/device registry 实际观察 unload、重建和正确 device identifier;HA 不可达时普通 discovery 继续发布,repair 保持 pending。
- availability、unit、device/state class 与 today reset 由 provider 声明;operator 应在 HA 中核对,而不应假设同名实体可跨换表连续。
- 关闭 toggle 后 retained discovery 会被清理;关闭暴露不删除 source、Meter、合同、读数或成本历史。
+7 -1
View File
@@ -77,7 +77,13 @@ docker compose -f docker-compose.yml run --rm migration
成本页的 15 分钟 ledger 分开显示 heating 与 hot-water 三项 variable breakdown;fixed 费只在合同级 summary 按本地自然日计提一次,all-in = variable + fixed。用显式 recompute 来验证测试时间窗时,应手算并核对 Decimal 金额,保留原有 electricity 合同和数字不变。
在 Config 的 HA Expose 中只开启需要的 source、Meter 与 thermal entities。核对 unit、state class、availability、today reset 和换表后 identity;关闭 toggle 后应用会清理 retained discovery。不要把 source secret、设备 identity 或合同金额放进 HA entity 名称、日志或截图。
在 Config 的 HA Expose 中只开启需要的 source、Meter 与 thermal entities。`Thermal Heating Total/Today`
显示 heating variable 加上 heating network、metering、delivery set 与 other 固定费;`Thermal Hot Water
Total/Today` 显示 hot-water heating、water、water tax variable 加上 hot-water network 固定费。两条合计
恰好等于 `Thermal All-in Total/Today`;component entities 仍仅显示各自 variable,`Fixed` 仍显示全部
standing,避免重复计费。数值仍来自在 01:05 后结算的 summary。核对 unit、state class、availability、today
reset 和换表后 identity;关闭 toggle 后应用会清理 retained discovery。不要把 source secret、设备 identity
或合同金额放进 HA entity 名称、日志或截图。
## 安全回滚
+99 -221
View File
@@ -22,13 +22,14 @@
"react-dom": "^18.3.1",
"react-feather": "^2.0.10",
"react-leaflet": "^4.2.1",
"react-router-dom": "^6.30.4",
"react-router-dom": "^7.18.3",
"recharts": "^3.8.1"
},
"devDependencies": {
"@eslint/js": "^9.39.4",
"@testing-library/dom": "10.4.1",
"@testing-library/jest-dom": "^6.9.1",
"@testing-library/react": "^14.3.1",
"@testing-library/react": "16.3.0",
"@testing-library/user-event": "^14.6.1",
"@types/react": "^18.3.31",
"@types/react-dom": "^18.3.7",
@@ -1390,9 +1391,9 @@
"license": "MIT"
},
"node_modules/@redocly/openapi-core": {
"version": "1.34.15",
"resolved": "https://registry.npmjs.org/@redocly/openapi-core/-/openapi-core-1.34.15.tgz",
"integrity": "sha512-HAwCnNyKcs5XGQqms+9t7OdAPM/5TDstmhF+0i7tdCFato2QKuYIlyWETwkXd8c5zbltr1oB+6y9NTeQLr2d6Q==",
"version": "1.34.19",
"resolved": "https://registry.npmjs.org/@redocly/openapi-core/-/openapi-core-1.34.19.tgz",
"integrity": "sha512-o/0VgsBXgwcY1lyeqcVtSGdTQAPnVggo0fbFVPlxl5XVDKUcVH0OLRqt3CbkwByT5FU305E0iE0O7MzThjDblw==",
"dev": true,
"license": "MIT",
"dependencies": {
@@ -1401,7 +1402,7 @@
"colorette": "1.4.0",
"https-proxy-agent": "7.0.6",
"js-levenshtein": "1.1.6",
"js-yaml": "4.1.1",
"js-yaml": "4.3.1",
"minimatch": "5.1.9",
"pluralize": "8.0.0",
"yaml-ast-parser": "0.0.43"
@@ -1412,9 +1413,9 @@
}
},
"node_modules/@redocly/openapi-core/node_modules/brace-expansion": {
"version": "2.1.1",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-2.1.1.tgz",
"integrity": "sha512-WR1cURNjuvBLMZBMbqM0UoE+WAfdUcEV1ccD8PVBVOI+Z3ND4+SZbN8RsfT2bMuG1qwz5RFvPukSZm5fF2D5eA==",
"version": "2.1.4",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-2.1.4.tgz",
"integrity": "sha512-hGfVzPxthbf3+2yjg/RBs60cB0FhqBS/zvdV/4wn4/BmN0bNMMHPc4V/BbFieqf1TKAGGAHnY4eSjajCl0f2Xg==",
"dev": true,
"license": "MIT",
"dependencies": {
@@ -1422,10 +1423,20 @@
}
},
"node_modules/@redocly/openapi-core/node_modules/js-yaml": {
"version": "4.1.1",
"resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-4.1.1.tgz",
"integrity": "sha512-qQKT4zQxXl8lLwBtHMWwaTcGfFOZviOJet3Oy/xmGk2gZH677CJM9EvtfdSkgWcATZhj/55JZ0rmy3myCT5lsA==",
"version": "4.3.1",
"resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-4.3.1.tgz",
"integrity": "sha512-CY6crGq313MX8GkwvB7tzgp99vjQxY1++5y10/BKN/GUfHqWaOGQMNZkBvqSzsZKWk/ijwHlWzzkLulsGHhjWQ==",
"dev": true,
"funding": [
{
"type": "github",
"url": "https://github.com/sponsors/puzrin"
},
{
"type": "github",
"url": "https://github.com/sponsors/nodeca"
}
],
"license": "MIT",
"dependencies": {
"argparse": "^2.0.1"
@@ -1483,15 +1494,6 @@
"url": "https://opencollective.com/immer"
}
},
"node_modules/@remix-run/router": {
"version": "1.23.3",
"resolved": "https://registry.npmjs.org/@remix-run/router/-/router-1.23.3.tgz",
"integrity": "sha512-4An71tdz9X8+3sI4Qqqd2LWd9vS39J7sqd9EU4Scw7TJE/qB10Flv/UuqbPVgfQV9XoK8Np6jNquZitnZq5i+Q==",
"license": "MIT",
"engines": {
"node": ">=14.0.0"
}
},
"node_modules/@rolldown/pluginutils": {
"version": "1.0.0-beta.27",
"resolved": "https://registry.npmjs.org/@rolldown/pluginutils/-/pluginutils-1.0.0-beta.27.tgz",
@@ -1591,9 +1593,6 @@
"arm"
],
"dev": true,
"libc": [
"glibc"
],
"license": "MIT",
"optional": true,
"os": [
@@ -1608,9 +1607,6 @@
"arm"
],
"dev": true,
"libc": [
"musl"
],
"license": "MIT",
"optional": true,
"os": [
@@ -1625,9 +1621,6 @@
"arm64"
],
"dev": true,
"libc": [
"glibc"
],
"license": "MIT",
"optional": true,
"os": [
@@ -1642,9 +1635,6 @@
"arm64"
],
"dev": true,
"libc": [
"musl"
],
"license": "MIT",
"optional": true,
"os": [
@@ -1659,9 +1649,6 @@
"loong64"
],
"dev": true,
"libc": [
"glibc"
],
"license": "MIT",
"optional": true,
"os": [
@@ -1676,9 +1663,6 @@
"loong64"
],
"dev": true,
"libc": [
"musl"
],
"license": "MIT",
"optional": true,
"os": [
@@ -1693,9 +1677,6 @@
"ppc64"
],
"dev": true,
"libc": [
"glibc"
],
"license": "MIT",
"optional": true,
"os": [
@@ -1710,9 +1691,6 @@
"ppc64"
],
"dev": true,
"libc": [
"musl"
],
"license": "MIT",
"optional": true,
"os": [
@@ -1727,9 +1705,6 @@
"riscv64"
],
"dev": true,
"libc": [
"glibc"
],
"license": "MIT",
"optional": true,
"os": [
@@ -1744,9 +1719,6 @@
"riscv64"
],
"dev": true,
"libc": [
"musl"
],
"license": "MIT",
"optional": true,
"os": [
@@ -1761,9 +1733,6 @@
"s390x"
],
"dev": true,
"libc": [
"glibc"
],
"license": "MIT",
"optional": true,
"os": [
@@ -1778,9 +1747,6 @@
"x64"
],
"dev": true,
"libc": [
"glibc"
],
"license": "MIT",
"optional": true,
"os": [
@@ -1795,9 +1761,6 @@
"x64"
],
"dev": true,
"libc": [
"musl"
],
"license": "MIT",
"optional": true,
"os": [
@@ -1932,7 +1895,6 @@
"integrity": "sha512-o4PXJQidqJl82ckFaXUeoAW+XysPLauYI43Abki5hABd853iMhitooc6znOnczgbTYmEP6U6/y1ZyKAIsvMKGg==",
"dev": true,
"license": "MIT",
"peer": true,
"dependencies": {
"@babel/code-frame": "^7.10.4",
"@babel/runtime": "^7.12.5",
@@ -1975,52 +1937,31 @@
"license": "MIT"
},
"node_modules/@testing-library/react": {
"version": "14.3.1",
"resolved": "https://registry.npmjs.org/@testing-library/react/-/react-14.3.1.tgz",
"integrity": "sha512-H99XjUhWQw0lTgyMN05W3xQG1Nh4lq574D8keFf1dDoNTJgp66VbJozRaczoF+wsiaPJNt/TcnfpLGufGxSrZQ==",
"version": "16.3.0",
"resolved": "https://registry.npmjs.org/@testing-library/react/-/react-16.3.0.tgz",
"integrity": "sha512-kFSyxiEDwv1WLl2fgsq6pPBbw5aWKrsY2/noi1Id0TK0UParSF62oFQFGHXIyaG4pp2tEub/Zlel+fjjZILDsw==",
"dev": true,
"license": "MIT",
"dependencies": {
"@babel/runtime": "^7.12.5",
"@testing-library/dom": "^9.0.0",
"@types/react-dom": "^18.0.0"
"@babel/runtime": "^7.12.5"
},
"engines": {
"node": ">=14"
"node": ">=18"
},
"peerDependencies": {
"react": "^18.0.0",
"react-dom": "^18.0.0"
}
},
"node_modules/@testing-library/react/node_modules/@testing-library/dom": {
"version": "9.3.4",
"resolved": "https://registry.npmjs.org/@testing-library/dom/-/dom-9.3.4.tgz",
"integrity": "sha512-FlS4ZWlp97iiNWig0Muq8p+3rVDjRiYE+YKGbAqXOu9nwJFFOdL00kFpz42M+4huzYi86vAK1sOOfyOG45muIQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"@babel/code-frame": "^7.10.4",
"@babel/runtime": "^7.12.5",
"@types/aria-query": "^5.0.1",
"aria-query": "5.1.3",
"chalk": "^4.1.0",
"dom-accessibility-api": "^0.5.9",
"lz-string": "^1.5.0",
"pretty-format": "^27.0.2"
"@testing-library/dom": "^10.0.0",
"@types/react": "^18.0.0 || ^19.0.0",
"@types/react-dom": "^18.0.0 || ^19.0.0",
"react": "^18.0.0 || ^19.0.0",
"react-dom": "^18.0.0 || ^19.0.0"
},
"engines": {
"node": ">=14"
}
},
"node_modules/@testing-library/react/node_modules/aria-query": {
"version": "5.1.3",
"resolved": "https://registry.npmjs.org/aria-query/-/aria-query-5.1.3.tgz",
"integrity": "sha512-R5iJ5lkuHybztUfuOAznmboyjWq8O6sqNqtK7CLOqdydi54VNbORp49mb14KbWgG1QD3JFO9hJdZ+y4KutfdOQ==",
"dev": true,
"license": "Apache-2.0",
"dependencies": {
"deep-equal": "^2.0.5"
"peerDependenciesMeta": {
"@types/react": {
"optional": true
},
"@types/react-dom": {
"optional": true
}
}
},
"node_modules/@testing-library/user-event": {
@@ -2441,16 +2382,16 @@
}
},
"node_modules/@typescript-eslint/typescript-estree/node_modules/brace-expansion": {
"version": "5.0.6",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.6.tgz",
"integrity": "sha512-kLpxurY4Z4r9sgMsyG0Z9uzsBlgiU/EFKhj/h91/8yHu0edo7XuixOIH3VcJ8kkxs6/jPzoI6U9Vj3WqbMQ94g==",
"version": "5.0.9",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.9.tgz",
"integrity": "sha512-ScQ4IuvIEF1TMlP7Zt+vjJ//9zlPb2SDcxWxM3bk8s6t6GGdJ7KO1dCcTidOPJKePW30LE/2cT7wCyPho9/Wxg==",
"dev": true,
"license": "MIT",
"dependencies": {
"balanced-match": "^4.0.2"
},
"engines": {
"node": "18 || 20 || >=22"
"node": "20 || >=22"
}
},
"node_modules/@typescript-eslint/typescript-estree/node_modules/minimatch": {
@@ -2979,9 +2920,9 @@
}
},
"node_modules/brace-expansion": {
"version": "1.1.15",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.15.tgz",
"integrity": "sha512-EwOCDEex4quD37XhqM3omwtMoJjr//isUZz1JopUNWms+4Z2ViyM/k1YIRePpoVNnQhENnxtFjLaxNHrT7xIUg==",
"version": "1.1.18",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.18.tgz",
"integrity": "sha512-Edep/X9fGqVNmzKBVsDYIOtD+z1tuezV70LBjdCst9Tqu76lsnvRiZ6oTic1n+/BIwX6QDGAO94PN4N2SADvtw==",
"dev": true,
"license": "MIT",
"dependencies": {
@@ -3188,6 +3129,19 @@
"dev": true,
"license": "MIT"
},
"node_modules/cookie": {
"version": "1.1.1",
"resolved": "https://registry.npmjs.org/cookie/-/cookie-1.1.1.tgz",
"integrity": "sha512-ei8Aos7ja0weRpFzJnEA9UHJ/7XQmqglbRwnf2ATjcB9Wq874VKH9kfjjirM6UhU2/E5fFYadylyhFldcqSidQ==",
"license": "MIT",
"engines": {
"node": ">=18"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/express"
}
},
"node_modules/cross-spawn": {
"version": "7.0.6",
"resolved": "https://registry.npmjs.org/cross-spawn/-/cross-spawn-7.0.6.tgz",
@@ -3451,39 +3405,6 @@
"integrity": "sha512-qIMFpTMZmny+MMIitAB6D7iVPEorVw6YQRWkvarTkT4tBeSLLiHzcwj6q0MmYSFCiVpiqPJTJEYIrpcPzVEIvg==",
"license": "MIT"
},
"node_modules/deep-equal": {
"version": "2.2.3",
"resolved": "https://registry.npmjs.org/deep-equal/-/deep-equal-2.2.3.tgz",
"integrity": "sha512-ZIwpnevOurS8bpT4192sqAowWM76JDKSHYzMLty3BZGSswgq6pBaH3DhCSW5xVAZICZyKdOBPjwww5wfgT/6PA==",
"dev": true,
"license": "MIT",
"dependencies": {
"array-buffer-byte-length": "^1.0.0",
"call-bind": "^1.0.5",
"es-get-iterator": "^1.1.3",
"get-intrinsic": "^1.2.2",
"is-arguments": "^1.1.1",
"is-array-buffer": "^3.0.2",
"is-date-object": "^1.0.5",
"is-regex": "^1.1.4",
"is-shared-array-buffer": "^1.0.2",
"isarray": "^2.0.5",
"object-is": "^1.1.5",
"object-keys": "^1.1.1",
"object.assign": "^4.1.4",
"regexp.prototype.flags": "^1.5.1",
"side-channel": "^1.0.4",
"which-boxed-primitive": "^1.0.2",
"which-collection": "^1.0.1",
"which-typed-array": "^1.1.13"
},
"engines": {
"node": ">= 0.4"
},
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/deep-is": {
"version": "0.1.4",
"resolved": "https://registry.npmjs.org/deep-is/-/deep-is-0.1.4.tgz",
@@ -3687,27 +3608,6 @@
"node": ">= 0.4"
}
},
"node_modules/es-get-iterator": {
"version": "1.1.3",
"resolved": "https://registry.npmjs.org/es-get-iterator/-/es-get-iterator-1.1.3.tgz",
"integrity": "sha512-sPZmqHBe6JIiTfN5q2pEi//TwxmAFHwj/XEuYjTuse78i8KxaqMTTzxPoFKuzRpDpTJ+0NAbpfenkmH2rePtuw==",
"dev": true,
"license": "MIT",
"dependencies": {
"call-bind": "^1.0.2",
"get-intrinsic": "^1.1.3",
"has-symbols": "^1.0.3",
"is-arguments": "^1.1.1",
"is-map": "^2.0.2",
"is-set": "^2.0.2",
"is-string": "^1.0.7",
"isarray": "^2.0.5",
"stop-iteration-iterator": "^1.0.0"
},
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/es-iterator-helpers": {
"version": "1.3.3",
"resolved": "https://registry.npmjs.org/es-iterator-helpers/-/es-iterator-helpers-1.3.3.tgz",
@@ -4650,23 +4550,6 @@
"node": ">=12"
}
},
"node_modules/is-arguments": {
"version": "1.2.0",
"resolved": "https://registry.npmjs.org/is-arguments/-/is-arguments-1.2.0.tgz",
"integrity": "sha512-7bVbi0huj/wrIAOzb8U1aszg9kdi3KN/CyU19CTI7tAoZYEZoL9yCDXpbXN+uPsuWnP02cyug1gleqq+TU+YCA==",
"dev": true,
"license": "MIT",
"dependencies": {
"call-bound": "^1.0.2",
"has-tostringtag": "^1.0.2"
},
"engines": {
"node": ">= 0.4"
},
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/is-array-buffer": {
"version": "3.0.5",
"resolved": "https://registry.npmjs.org/is-array-buffer/-/is-array-buffer-3.0.5.tgz",
@@ -5121,9 +5004,9 @@
"license": "MIT"
},
"node_modules/js-yaml": {
"version": "4.2.0",
"resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-4.2.0.tgz",
"integrity": "sha512-ePWsvanv0DWuDRsW8dnt+R4jQ31SCRCQ7hhNcPXZPsoBZiemuZNYGf7adZdqX2D86j6rvKp3RpCxVTSb8WQlOw==",
"version": "4.3.2",
"resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-4.3.2.tgz",
"integrity": "sha512-SFNOvSJ+Dgf/9An904Yx+CgSlIPCkIpao4qo51lpee25TIRejdH3rhR4EZMGoNx3/TP3O+wzWuiTFl4sqbltzA==",
"dev": true,
"funding": [
{
@@ -5414,9 +5297,9 @@
"license": "MIT"
},
"node_modules/nanoid": {
"version": "3.3.12",
"resolved": "https://registry.npmjs.org/nanoid/-/nanoid-3.3.12.tgz",
"integrity": "sha512-ZB9RH/39qpq5Vu6Y+NmUaFhQR6pp+M2Xt76XBnEwDaGcVAqhlvxrl3B2bKS5D3NH3QR76v3aSrKaF/Kiy7lEtQ==",
"version": "3.3.18",
"resolved": "https://registry.npmjs.org/nanoid/-/nanoid-3.3.18.tgz",
"integrity": "sha512-DTg4MJbGMWkfi6VZFdNt2/caMbQy4Ou+Op/hJQvGEWcnVfoA1QA+xzRKAzw9jD6+GVOOeYr/mIcuDSdug6F6+w==",
"dev": true,
"funding": [
{
@@ -5490,23 +5373,6 @@
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/object-is": {
"version": "1.1.6",
"resolved": "https://registry.npmjs.org/object-is/-/object-is-1.1.6.tgz",
"integrity": "sha512-F8cZ+KfGlSGi09lJT7/Nd6KJZ9ygtvYC0/UYYLI9nmQKLMnydpB9yvbv9K1uSkEu7FU9vYPmVwLg328tX+ot3Q==",
"dev": true,
"license": "MIT",
"dependencies": {
"call-bind": "^1.0.7",
"define-properties": "^1.2.1"
},
"engines": {
"node": ">= 0.4"
},
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/object-keys": {
"version": "1.1.1",
"resolved": "https://registry.npmjs.org/object-keys/-/object-keys-1.1.1.tgz",
@@ -5842,9 +5708,9 @@
}
},
"node_modules/postcss": {
"version": "8.5.15",
"resolved": "https://registry.npmjs.org/postcss/-/postcss-8.5.15.tgz",
"integrity": "sha512-FfR8sjd4em2T6fb3I2MwAJU7HWVMr9zba+enmQeeWFfCbm+UOC/0X4DS8XtpUTMwWMGbjKYP7xjfNekzyGmB3A==",
"version": "8.5.26",
"resolved": "https://registry.npmjs.org/postcss/-/postcss-8.5.26.tgz",
"integrity": "sha512-u82N74LFzG8ca+dD8puPnplTXoGH4fTPpVGuIbt36G3qvNlkvfD0lEAZSxaly3KX8TS/L1A1gsCEmvKmBcVbkQ==",
"dev": true,
"funding": [
{
@@ -5862,7 +5728,7 @@
],
"license": "MIT",
"dependencies": {
"nanoid": "^3.3.12",
"nanoid": "^3.3.17",
"picocolors": "^1.1.1",
"source-map-js": "^1.2.1"
},
@@ -6092,35 +5958,41 @@
}
},
"node_modules/react-router": {
"version": "6.30.4",
"resolved": "https://registry.npmjs.org/react-router/-/react-router-6.30.4.tgz",
"integrity": "sha512-SVUsDe+DybHM/WmYKIVYhZh1o5Dcuf16yM6WjG02Q9XVFMZIJyHYhwrr6bFBXZkVP6z69kNkMyBCujt8FaFLJA==",
"version": "7.18.3",
"resolved": "https://registry.npmjs.org/react-router/-/react-router-7.18.3.tgz",
"integrity": "sha512-gyXgtdr5uACJ5b1Q4udzjVV+tb/rlHIMJKuJ0e89R4Kzgz47z/rgP0dIKxktqIEUhDHluGTPJJH/wRha7CyqsA==",
"license": "MIT",
"dependencies": {
"@remix-run/router": "1.23.3"
"cookie": "^1.0.1",
"set-cookie-parser": "^2.6.0"
},
"engines": {
"node": ">=14.0.0"
"node": ">=20.0.0"
},
"peerDependencies": {
"react": ">=16.8"
"react": ">=18",
"react-dom": ">=18"
},
"peerDependenciesMeta": {
"react-dom": {
"optional": true
}
}
},
"node_modules/react-router-dom": {
"version": "6.30.4",
"resolved": "https://registry.npmjs.org/react-router-dom/-/react-router-dom-6.30.4.tgz",
"integrity": "sha512-q4HvNl+mmDdkS0g+MqiBZNteQJCuimWoOyHMy4T/RQLAn9Z29+E91QXRaxOujeMl2HTzRSS0KFPd7lxX3PjV0Q==",
"version": "7.18.3",
"resolved": "https://registry.npmjs.org/react-router-dom/-/react-router-dom-7.18.3.tgz",
"integrity": "sha512-ytVbyBBM7vMfRCam25r0WMhSVSom909A8p+8m0/f1w853dz/xfFu6etAT2SEbVoSnI+ZoPRDqIsQXVT89gp7kg==",
"license": "MIT",
"dependencies": {
"@remix-run/router": "1.23.3",
"react-router": "6.30.4"
"react-router": "7.18.3"
},
"engines": {
"node": ">=14.0.0"
"node": ">=20.0.0"
},
"peerDependencies": {
"react": ">=16.8",
"react-dom": ">=16.8"
"react": ">=18",
"react-dom": ">=18"
}
},
"node_modules/react-style-singleton": {
@@ -6447,6 +6319,12 @@
"semver": "bin/semver.js"
}
},
"node_modules/set-cookie-parser": {
"version": "2.7.2",
"resolved": "https://registry.npmjs.org/set-cookie-parser/-/set-cookie-parser-2.7.2.tgz",
"integrity": "sha512-oeM1lpU/UvhTxw+g3cIfxXHyJRc/uidd3yK1P242gzHds0udQBYzs3y8j4gCCW+ZJ7ad0yctld8RYO+bdurlvw==",
"license": "MIT"
},
"node_modules/set-function-length": {
"version": "1.2.2",
"resolved": "https://registry.npmjs.org/set-function-length/-/set-function-length-1.2.2.tgz",
@@ -7080,9 +6958,9 @@
}
},
"node_modules/undici": {
"version": "7.27.2",
"resolved": "https://registry.npmjs.org/undici/-/undici-7.27.2.tgz",
"integrity": "sha512-uZsKNuzQxDMUY6M3pIMvy5tvlGmtq8XJ2oLAkfRKGNu+1VQAIvLy2xIVG5ATZl5wDXl/tddByAWCizRbOme+TA==",
"version": "7.29.0",
"resolved": "https://registry.npmjs.org/undici/-/undici-7.29.0.tgz",
"integrity": "sha512-IDxfleLmmbSskfWSUATiN1nfn2rDuvnMOqb5CWR92iIfojA0Ud+ulOAAEQ57LPr9rWmsreUyf5lwyao+7GNNVw==",
"dev": true,
"license": "MIT",
"engines": {
+3 -2
View File
@@ -27,13 +27,14 @@
"react-dom": "^18.3.1",
"react-feather": "^2.0.10",
"react-leaflet": "^4.2.1",
"react-router-dom": "^6.30.4",
"react-router-dom": "^7.18.3",
"recharts": "^3.8.1"
},
"devDependencies": {
"@eslint/js": "^9.39.4",
"@testing-library/dom": "10.4.1",
"@testing-library/jest-dom": "^6.9.1",
"@testing-library/react": "^14.3.1",
"@testing-library/react": "16.3.0",
"@testing-library/user-event": "^14.6.1",
"@types/react": "^18.3.31",
"@types/react-dom": "^18.3.7",
+7 -2
View File
@@ -462,12 +462,17 @@ describe('CostView', () => {
it('paginates the complete thermal ledger and resets offset when its range or scope changes', async () => {
const user = userEvent.setup()
const lastRow = { ...THERMAL_ROW, period_start: '2026-06-22T12:00:00Z', quantity: '501' }
const firstPageRows = Array.from({ length: 500 }, (_, index) => {
const periodStart = new Date(Date.UTC(2026, 5, 22, 10, index * 15))
const periodEnd = new Date(Date.UTC(2026, 5, 22, 10, (index + 1) * 15))
return { ...THERMAL_ROW, period_start: periodStart.toISOString(), period_end: periodEnd.toISOString() }
})
const lastRow = { ...THERMAL_ROW, period_start: '2026-06-27T15:00:00Z', period_end: '2026-06-27T15:15:00Z', quantity: '501' }
mockGet.mockImplementation((path: string, options?: { params?: { query?: { offset?: number } } }) => {
if (path === '/api/energy/meters') return Promise.resolve({ data: { items: [ACTIVE_HEATING_METER, ACTIVE_HOT_WATER_METER], total: 2 } })
if (path === '/api/energy/meter-costs') {
const offset = options?.params?.query?.offset ?? 0
return Promise.resolve({ data: offset === 0 ? { items: Array.from({ length: 500 }, () => THERMAL_ROW), total: 501 } : { items: [lastRow], total: 501 } })
return Promise.resolve({ data: offset === 0 ? { items: firstPageRows, total: 501 } : { items: [lastRow], total: 501 } })
}
if (path === '/api/energy/meter-costs/summary') return Promise.resolve({ data: THERMAL_SUMMARY })
if (path === '/api/energy/costs/summary') return Promise.resolve({ data: SUMMARY })
+11 -6
View File
@@ -15,7 +15,7 @@
*/
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { screen, waitFor } from '@testing-library/react'
import { screen, waitFor, waitForElementToBeRemoved } from '@testing-library/react'
import userEvent from '@testing-library/user-event'
import { renderWithProviders } from '../test-utils'
import { MeterManager } from './MeterManager'
@@ -1197,20 +1197,24 @@ describe('MeterManager — lifecycle modal submissions', () => {
let input: HTMLInputElement
let submit: HTMLElement
let modal: HTMLElement
if (entry === 'close') {
await user.click(await screen.findByRole('button', { name: 'Close meter' }))
input = screen.getByTestId(`close-meter-modal-${ACTIVE_METER.id}`).querySelector('input[type="datetime-local"]')!
modal = screen.getByTestId(`close-meter-modal-${ACTIVE_METER.id}`)
input = modal.querySelector('input[type="datetime-local"]')!
submit = screen.getAllByRole('button', { name: 'Close meter' })[1]
} else if (entry === 'unbind') {
await user.click(await screen.findByRole('button', { name: 'Unbind' }))
input = screen.getByTestId('unbind-modal-pending-source').querySelector('input[type="datetime-local"]')!
modal = screen.getByTestId('unbind-modal-pending-source')
input = modal.querySelector('input[type="datetime-local"]')!
submit = screen.getAllByRole('button', { name: 'Unbind' })[1]
} else {
await user.click(await screen.findByRole('button', { name: entry === 'direct bind' ? 'Bind source' : entry === 'same-meter transfer' ? 'Transfer source' : 'Recover binding' }))
await chooseChannel(user)
input = entry === 'direct bind'
? screen.getByTestId(`direct-bind-modal-${ACTIVE_METER.id}`).querySelector('input[type="datetime-local"]')!
: screen.getByTestId('transfer-effective-at')
modal = entry === 'direct bind'
? screen.getByTestId(`direct-bind-modal-${ACTIVE_METER.id}`)
: screen.getByTestId('transfer-modal-pending-source')
input = entry === 'direct bind' ? modal.querySelector('input[type="datetime-local"]')! : screen.getByTestId('transfer-effective-at')
const submitButtons = screen.getAllByRole('button', { name: entry === 'direct bind' ? 'Bind source' : 'Transfer binding' })
submit = submitButtons[submitButtons.length - 1]
}
@@ -1224,6 +1228,7 @@ describe('MeterManager — lifecycle modal submissions', () => {
await user.type(input, '{Enter}')
expect(entry === 'unbind' ? mockPatch : mockPost).toHaveBeenCalledTimes(1)
release()
await waitForElementToBeRemoved(modal)
},
)
+5 -5
View File
@@ -10,7 +10,7 @@
*/
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { render, screen, fireEvent, waitFor } from '@testing-library/react'
import { act, render, screen, fireEvent, waitFor } from '@testing-library/react'
import { MantineProvider } from '@mantine/core'
import { QueryClient, QueryClientProvider } from '@tanstack/react-query'
import { MemoryRouter } from 'react-router-dom'
@@ -199,7 +199,7 @@ describe('HomePage', () => {
altitude: null,
}
expect(capturedOnSelectLocation).toBeDefined()
capturedOnSelectLocation!(record)
act(() => capturedOnSelectLocation!(record))
// EditLocationModal should appear
await waitFor(() => screen.getByTestId('edit-location-modal'))
@@ -216,7 +216,7 @@ describe('HomePage', () => {
longitude: 116.41,
}
expect(capturedOnSelectPoo).toBeDefined()
capturedOnSelectPoo!(record)
act(() => capturedOnSelectPoo!(record))
await waitFor(() => screen.getByTestId('edit-poo-modal'))
expect(screen.getByTestId('edit-poo-modal')).toBeTruthy()
@@ -232,7 +232,7 @@ describe('HomePage', () => {
longitude: 116.4,
altitude: null,
}
capturedOnSelectLocation!(record)
act(() => capturedOnSelectLocation!(record))
await waitFor(() => screen.getByTestId('edit-location-modal'))
fireEvent.click(screen.getByTestId('edit-location-cancel'))
@@ -248,7 +248,7 @@ describe('HomePage', () => {
latitude: 39.91,
longitude: 116.41,
}
capturedOnSelectPoo!(record)
act(() => capturedOnSelectPoo!(record))
await waitFor(() => screen.getByTestId('edit-poo-modal'))
fireEvent.click(screen.getByTestId('edit-poo-cancel'))
+22
View File
@@ -7,6 +7,28 @@
* - ResizeObserver (Mantine uses it for responsive components)
*/
import '@testing-library/jest-dom'
// Import RTL here so its automatic cleanup hook is registered before the
// warning assertion below; later test imports reuse the same module instance.
import { cleanup } from '@testing-library/react'
import { afterEach, beforeEach } from 'vitest'
import { assertNoReactTestWarnings, createReactWarningGuard } from './test-warning-guard'
const originalConsoleError = console.error
const reactTestWarnings: unknown[][] = []
console.error = createReactWarningGuard(originalConsoleError, reactTestWarnings)
beforeEach(() => {
reactTestWarnings.length = 0
})
afterEach(() => {
// Run teardown before checking the guard. RTL's automatic cleanup is also
// registered, but cleanup is idempotent and this ordering keeps a guard
// failure from preventing a later test from starting with stale portals.
cleanup()
assertNoReactTestWarnings(reactTestWarnings)
})
// ---------------------------------------------------------------------------
// window.matchMedia polyfill (jsdom does not implement this)
+16
View File
@@ -0,0 +1,16 @@
import { screen } from '@testing-library/react'
import { useMantineEnv } from '@mantine/core'
import { describe, expect, it } from 'vitest'
import { renderWithProviders } from './test-utils'
function MantineEnvironmentProbe() {
return <output>{useMantineEnv()}</output>
}
describe('renderWithProviders', () => {
it('uses Mantine test environment', () => {
renderWithProviders(<MantineEnvironmentProbe />)
expect(screen.getByText('test')).toBeInTheDocument()
})
})
+1 -1
View File
@@ -51,7 +51,7 @@ export function renderWithProviders(ui: ReactNode, options: RenderOptions = {})
function Wrapper() {
return (
<MantineProvider>
<MantineProvider env="test">
<QueryClientProvider client={queryClient}>
<MemoryRouter initialEntries={entries}>
<Routes>
+40
View File
@@ -0,0 +1,40 @@
import { describe, expect, it, vi } from 'vitest'
import {
assertNoReactTestWarnings,
createReactWarningGuard,
formatReactTestWarnings,
isReactTestWarning,
} from './test-warning-guard'
describe('React test warning guard', () => {
it('recognizes React act and duplicate-key warnings only', () => {
expect(isReactTestWarning(['Warning: An update to Example inside a test was not wrapped in act(...).'])).toBe(true)
expect(isReactTestWarning(['Warning: Each child in a list should have a unique "key" prop.'])).toBe(true)
expect(isReactTestWarning(['Warning: Encountered two children with the same key, `duplicate`.'])).toBe(true)
expect(isReactTestWarning(['network request failed'])).toBe(false)
})
it('collects target warnings for one compact diagnostic and forwards unrelated errors', () => {
const originalConsoleError = vi.fn()
const warnings: unknown[][] = []
const guardedConsoleError = createReactWarningGuard(originalConsoleError, warnings)
guardedConsoleError('Warning: An update to Example inside a test was not wrapped in act(...)')
guardedConsoleError('Warning: An update to Other inside a test was not wrapped in act(...)')
guardedConsoleError('Warning: Encountered two children with the same key, `duplicate`.')
guardedConsoleError('network request failed', { status: 500 })
expect(originalConsoleError).toHaveBeenCalledTimes(1)
expect(originalConsoleError).toHaveBeenCalledWith('network request failed', { status: 500 })
expect(formatReactTestWarnings(warnings)).toBe(
'React act warning: 2, React duplicate-key warning: 1 (Example, Other)',
)
expect(() => assertNoReactTestWarnings(warnings)).toThrow(
'React test warning guard: React act warning: 2, React duplicate-key warning: 1 (Example, Other).',
)
})
it('passes only when no target warnings were captured', () => {
expect(() => assertNoReactTestWarnings([])).not.toThrow()
})
})
+55
View File
@@ -0,0 +1,55 @@
/**
* Fail tests that emit React warnings which otherwise only reach console.error.
*
* The guard intentionally forwards every unrelated console error unchanged.
* It collects only the two warnings that make React test results unreliable and
* reports one compact error per test instead of flooding CI logs with stacks.
*/
export function isReactTestWarning(args: unknown[]): boolean {
const message = args.map((arg) => String(arg)).join(' ')
return (
message.includes('not wrapped in act(...)') ||
message.includes('Each child in a list should have a unique "key"') ||
message.includes('Encountered two children with the same key')
)
}
export function createReactWarningGuard(
originalConsoleError: (...args: unknown[]) => void,
warnings: unknown[][],
): (...args: unknown[]) => void {
return (...args: unknown[]) => {
if (isReactTestWarning(args)) {
warnings.push(args)
return
}
originalConsoleError(...args)
}
}
export function formatReactTestWarnings(warnings: unknown[][]): string {
const kinds = new Map<string, number>()
for (const args of warnings) {
const kind = args.map((arg) => String(arg)).join(' ').includes('not wrapped in act(...)')
? 'React act warning'
: 'React duplicate-key warning'
kinds.set(kind, (kinds.get(kind) ?? 0) + 1)
}
const summaries = [...kinds.entries()].map(([kind, count]) => `${kind}: ${count}`).join(', ')
const components = [...new Set(warnings
.map((args) => {
const message = String(args[0])
return message.includes('An update to %s inside a test') ? String(args[1]) : message.match(/An update to (.+?) inside a test/)?.[1]
})
.filter((component): component is string => Boolean(component)))].slice(0, 3)
return components.length > 0 ? `${summaries} (${components.join(', ')})` : summaries
}
export function assertNoReactTestWarnings(warnings: unknown[][]): void {
if (warnings.length > 0) {
throw new Error(
`React test warning guard: ${formatReactTestWarnings(warnings)}. ` +
'Await the observable update, close/removal, or query settlement that caused it.',
)
}
}
+7
View File
@@ -19,6 +19,13 @@ export default defineConfig({
test: {
environment: 'jsdom',
globals: true,
// Keep test output identical in interactive, agent, and CI environments.
// Vitest otherwise selects its minimal agent reporter when CODEX_CI is set,
// which hides console output from passing tests.
reporters: ['default'],
// Mantine-heavy UI tests allocate substantial jsdom resources. Keep the
// worker pool bounded so concurrent CI jobs remain reliable.
maxWorkers: 2,
setupFiles: ['./src/test-setup.ts'],
env: {
// Lock the test timezone to UTC so that date-formatting assertions
+29
View File
@@ -118,6 +118,35 @@ def test_app_start_seeds_missing_config_from_env_without_overwriting_existing_va
reset_db_caches()
def test_startup_initializes_fresh_legacy_discovery_cleanup_ledger(
tmp_path, monkeypatch: pytest.MonkeyPatch
) -> None:
"""The cleanup ledger exists before the expose UI can write its first toggle."""
import app.main as main
app_database_url = _prepare_app_db(tmp_path)
monkeypatch.setenv("APP_DATABASE_URL", app_database_url)
monkeypatch.setenv("AUTH_BOOTSTRAP_USERNAME", "admin")
monkeypatch.setenv("AUTH_BOOTSTRAP_PASSWORD", "test-password")
get_settings.cache_clear()
reset_db_caches()
main.ensure_auth_db_ready()
conn = sqlite3.connect(tmp_path / "app_ready.db")
try:
value = conn.execute(
"SELECT value FROM app_config WHERE key = ?",
("HA_DISCOVERY_LEGACY_THERMAL_CLEANUP_V1",),
).fetchone()[0]
finally:
conn.close()
assert value == '{"complete": true, "inventory": [], "topics": []}'
get_settings.cache_clear()
reset_db_caches()
def test_app_start_syncs_app_hostname_from_env_even_when_db_has_old_value(
tmp_path, monkeypatch: pytest.MonkeyPatch
) -> None:
+17
View File
@@ -87,6 +87,23 @@ def test_compose_defaults_business_timezone_and_dev_inherits_it() -> None:
assert "TZ" not in dev["services"].get(service_name, {}).get("environment", {})
def test_compose_log_rotation_is_string_typed_and_dev_inherits_it() -> None:
"""Both services use bounded json-file logs; dev layering keeps base policy."""
base = _read_yaml("docker-compose.yml")
dev = _read_yaml("docker-compose.dev.yml")
expected = {
"driver": "json-file",
"options": {"max-size": "10m", "max-file": "3"},
}
for service_name in ("migration", "app"):
base_logging = base["services"][service_name]["logging"]
assert base_logging == expected
assert all(isinstance(value, str) for value in base_logging["options"].values())
# The override does not define logging, so Compose retains the base value.
assert "logging" not in dev["services"].get(service_name, {})
def test_compose_keeps_app_non_root_and_maps_minimal_warmtelink_serial_access() -> None:
"""Base Compose maps the configured device with only pyserial's required access.
+8 -3
View File
@@ -12,6 +12,8 @@ from alembic import command
from alembic.config import Config
from sqlalchemy import create_engine, event, inspect, text
from app.integrations.meter_sources import sanitize_source_config, validate_source_config
def _config(database_url: str) -> Config:
config = Config("alembic_app.ini")
@@ -108,7 +110,7 @@ def test_populated_revision_14_adopts_dsmr_history_at_revision_16(tmp_path: Path
{"key": "DSMR_INGEST_ENABLED", "value": "true", "at": start},
{"key": "DSMR_MQTT_TOPIC", "value": "historic/dsmr", "at": start},
{"key": "DSMR_TARIFF_TOPIC", "value": "historic/tariff", "at": start},
{"key": "DSMR_SAMPLE_INTERVAL_S", "value": "15", "at": start},
{"key": "DSMR_SAMPLE_INTERVAL_S", "value": "0", "at": start},
{"key": "MQTT_BROKER_HOST", "value": "mqtt.example.invalid", "at": start},
{"key": "MQTT_BROKER_PORT", "value": "1884", "at": start},
{"key": "MQTT_USERNAME", "value": "historic-user", "at": start},
@@ -159,11 +161,14 @@ def test_populated_revision_14_adopts_dsmr_history_at_revision_16(tmp_path: Path
text("SELECT id, enabled, config FROM meter_source WHERE kind = 'dsmr_mqtt'")
).one()
assert source.enabled == 1
assert json.loads(source.config) == {
source_config = json.loads(source.config)
assert source_config == {
"broker_host": "mqtt.example.invalid", "broker_port": 1884,
"username": "historic-user", "password": "historic-password", "tls_enabled": True,
"topic": "historic/dsmr", "tariff_topic": "historic/tariff", "sample_interval_s": 15,
"topic": "historic/dsmr", "tariff_topic": "historic/tariff", "sample_interval_s": 0,
}
assert validate_source_config("dsmr_mqtt", source_config) == source_config
assert sanitize_source_config("dsmr_mqtt", source_config)["sample_interval_s"] == 0
assert connection.execute(text("SELECT value FROM app_config WHERE key = 'DSMR_MQTT_TOPIC'")).scalar_one() == "historic/dsmr"
assert dict(connection.execute(text("SELECT key, value FROM app_config")).all()) == config_before
assert connection.execute(text("SELECT group_concat(telegram_id) FROM dsmr_reading")).scalar_one() == "77,78,77"
+131 -48
View File
@@ -13,7 +13,7 @@ Coverage:
7. value_getter returns None when no non-degraded period exists.
8. MQTT not enabled → publish_states is a no-op (no raises, no publish calls).
9. Integration: build_discovery_payload on an energy_cost entity does NOT raise
IndexError (validates 2-element identifiers).
IndexError (validates one-item HA identifiers and independent internal identities).
10. Keys are stable fixed strings (not derived from mutable data or DB ids).
11. Provider registered: energy_cost entities appear alongside modbus entities
in the full catalog.
@@ -767,9 +767,8 @@ def test_publish_states_noop_when_not_connected() -> None:
def test_build_discovery_payload_no_index_error_for_energy_entities(energy_db) -> None:
"""build_discovery_payload must NOT raise IndexError for energy_cost entities.
FUE-T05: identifiers is now ('energy-cost', meter.uuid).
Validates that the 2-element identifiers tuple satisfies ha_discovery.py's
requirement to access identifiers[1] as node_id.
The HA grouping identifier and internal MQTT identity are deliberately
separate; a one-item HA identifier must therefore remain sufficient.
"""
from app.integrations.expose import build_catalog
from app.services.ha_discovery import build_discovery_payload
@@ -800,15 +799,11 @@ def test_build_discovery_payload_no_index_error_for_energy_entities(energy_db) -
assert len(energy_entries) == 6, "Expected 6 energy_cost entities in catalog"
for entry in energy_entries:
# identifiers[1] must be the meter uuid (not "energy-cost").
assert entry.entity.device.identifiers[1] == meter_uuid, (
f"identifiers[1] must be meter uuid {meter_uuid!r}, "
f"got {entry.entity.device.identifiers[1]!r}"
)
# Must not raise — specifically no IndexError from identifiers[1]
assert entry.entity.device.internal_identity == meter_uuid
assert entry.entity.device.identifiers == (f"home-automation:energy-cost:{meter_uuid}",)
topic, config = build_discovery_payload(entry.entity, "homeassistant")
# node_id = identifiers[1] with hyphens → underscores
# node_id = internal meter identity with hyphens → underscores
node_id = meter_uuid.replace("-", "_")
assert node_id in topic, (
f"Expected meter uuid node_id {node_id!r} in discovery topic, got {topic!r}"
@@ -817,13 +812,12 @@ def test_build_discovery_payload_no_index_error_for_energy_entities(energy_db) -
f"Discovery topic must end with /config, got {topic!r}"
)
assert "unique_id" in config
# unique_id seed is identifiers[1] (meter uuid) + entity key
# unique_id seed is the internal meter identity + entity key
assert meter_uuid in config["unique_id"], (
f"unique_id must contain meter uuid, got {config['unique_id']!r}"
)
assert "device" in config
assert "energy-cost" in config["device"]["identifiers"]
assert meter_uuid in config["device"]["identifiers"]
assert config["device"]["identifiers"] == [f"home-automation:energy-cost:{meter_uuid}"]
def test_energy_cost_entities_omit_availability_so_ha_shows_them(energy_db) -> None:
@@ -859,18 +853,18 @@ def test_energy_cost_entities_omit_availability_so_ha_shows_them(energy_db) -> N
def test_energy_entity_discovery_topics_contain_correct_node_id() -> None:
"""Discovery topic node_id for energy entities must be derived from meter uuid.
FUE-T05: identifiers[1] is now the active meter's uuid.
ha_discovery._node_id() replaces hyphens with underscores in identifiers[1]
to build the MQTT node_id. This test verifies that the topic reflects the
The internal identity is the active meter's uuid, independent from the
singleton HA identifier. This test verifies that the topic reflects the
meter uuid (not the old fixed 'energy-cost' string).
"""
from app.integrations.expose import DeviceInfo, ExposableEntity
from app.services.ha_discovery import build_discovery_payload
meter_uuid = "12345678-abcd-ef00-1234-567890abcdef"
# identifiers[1] = meter uuid — this is what FUE-T05 sets.
# ``identity`` is the meter UUID; HA grouping is a separate one-item tuple.
device = DeviceInfo(
identifiers=("energy-cost", meter_uuid),
identifiers=(f"home-automation:energy-cost:{meter_uuid}",),
identity=meter_uuid,
name="Test Meter",
provides_availability=False,
)
@@ -886,7 +880,7 @@ def test_energy_entity_discovery_topics_contain_correct_node_id() -> None:
topic, config = build_discovery_payload(entity, discovery_prefix="homeassistant")
# node_id: identifiers[1] = meter_uuid, hyphens → underscores
# node_id: internal identity = meter_uuid, hyphens → underscores
expected_node = meter_uuid.replace("-", "_")
assert f"/{expected_node}/" in topic, (
f"Expected meter uuid node_id {expected_node!r} in topic {topic!r}"
@@ -2345,12 +2339,11 @@ def test_energy_cost_provider_returns_empty_when_no_active_meter(energy_db) -> N
)
def test_energy_cost_provider_identifiers_match_meter_uuid(energy_db) -> None:
"""FUE-T05 ②: with an active meter, identifiers[1] == meter.uuid.
def test_energy_cost_provider_identity_matches_meter_uuid(energy_db) -> None:
"""The active meter anchors the internal identity and namespaced HA identifier.
The HA device identity is anchored to the active meter's uuid. ha_discovery.py
uses identifiers[1] as the MQTT node_id and unique_id seed; changing the active
meter (meter swap) produces a new uuid → new node_id → new HA sensor.
The HA device identity and MQTT internal identity are both anchored to the
active meter's uuid. Changing the meter produces a new node_id and HA sensor.
"""
from app.integrations.expose import _energy_cost_provider
@@ -2367,11 +2360,8 @@ def test_energy_cost_provider_identifiers_match_meter_uuid(energy_db) -> None:
assert len(entities) == 6, f"Expected 6 entities, got {len(entities)}"
for entity in entities:
assert entity.device.identifiers == ("energy-cost", meter_uuid), (
f"identifiers must be ('energy-cost', meter.uuid); "
f"expected ('energy-cost', {meter_uuid!r}), "
f"got {entity.device.identifiers!r}"
)
assert entity.device.internal_identity == meter_uuid
assert entity.device.identifiers == (f"home-automation:energy-cost:{meter_uuid}",)
assert entity.device.name == meter_label, (
f"device.name must be exactly the meter label {meter_label!r}, "
f"got {entity.device.name!r}"
@@ -2382,7 +2372,7 @@ def test_energy_cost_entity_keys_do_not_contain_meter_uuid(energy_db) -> None:
"""FUE-T05 ③: entity keys remain stable 'energy.*' strings (no uuid injected).
Keys are the anchor for the toggle table; they must NOT change when the meter
changes. Only identifiers[1] (node_id / unique_id) changes on a meter swap.
changes. Only the internal identity (node_id / unique_id) changes on a meter swap.
"""
from app.integrations.expose import _energy_cost_provider
@@ -2418,7 +2408,7 @@ def test_energy_cost_entity_keys_do_not_contain_meter_uuid(energy_db) -> None:
def test_energy_cost_toggle_survives_meter_swap(energy_db) -> None:
"""FUE-T05 ③ (toggle stability): enabled toggle on 'energy.buy_price_now' survives meter swap.
After a meter swap the provider queries a new active meter → new identifiers[1] /
After a meter swap the provider queries a new active meter → new internal identity /
unique_id / topic in HA. But the entity key stays 'energy.buy_price_now', so the
existing toggle row (keyed by 'energy.buy_price_now') is still found → enabled=True.
@@ -2475,11 +2465,7 @@ def test_energy_cost_toggle_survives_meter_swap(energy_db) -> None:
(e for e in catalog if e.entity.key == "energy.buy_price_now"), None
)
assert buy_entry is not None, "energy.buy_price_now must be present in catalog"
# identifiers[1] must now be the NEW meter's uuid
assert buy_entry.entity.device.identifiers[1] == new_meter_uuid, (
f"After swap, identifiers[1] must be new meter uuid {new_meter_uuid!r}, "
f"got {buy_entry.entity.device.identifiers[1]!r}"
)
assert buy_entry.entity.device.internal_identity == new_meter_uuid
# Toggle state must still be enabled (key unchanged → same toggle row found)
assert buy_entry.enabled is True, (
"energy.buy_price_now toggle must remain enabled after meter swap "
@@ -2488,7 +2474,7 @@ def test_energy_cost_toggle_survives_meter_swap(energy_db) -> None:
def test_energy_cost_identifiers_change_after_meter_swap(energy_db) -> None:
"""FUE-T05 ④: after meter swap, provider produces new identifiers[1] (new meter uuid).
"""After a meter swap, provider produces a new internal meter identity.
Old uuid's entities are no longer produced → HA sensor for old uuid is frozen.
New uuid's entities appear → HA creates fresh sensors for the new meter.
@@ -2544,12 +2530,9 @@ def test_energy_cost_identifiers_change_after_meter_swap(energy_db) -> None:
f"Expected 6 entities after swap, got {len(entities_after_swap)}"
)
for entity in entities_after_swap:
assert entity.device.identifiers[1] == new_uuid, (
f"After swap, identifiers[1] must be new uuid {new_uuid!r}, "
f"got {entity.device.identifiers[1]!r}"
)
assert entity.device.internal_identity == new_uuid
# Old uuid must not appear in identifiers
assert entity.device.identifiers[1] != old_uuid, (
assert entity.device.internal_identity != old_uuid, (
f"After swap, old uuid {old_uuid!r} must not appear in identifiers"
)
@@ -2805,16 +2788,59 @@ def test_m8_catalog_has_source_meter_and_thermal_entities_disabled(energy_db) ->
assert entries[f"meter.{heating.uuid}.total"].entity.unit == "GJ"
assert entries[f"meter.{heating.uuid}.total"].entity.device_class == "energy"
assert entries[f"meter.{water.uuid}.total"].entity.unit == "m³"
assert entries[f"meter.{water.uuid}.total"].entity.device_class == "volume"
for suffix in ("total", "today"):
water_entity = entries[f"meter.{water.uuid}.{suffix}"].entity
assert water_entity.device_class == "water"
assert water_entity.unit == "m³"
assert water_entity.state_class == "total_increasing"
assert entries[f"meter.{electricity.uuid}.total"].entity.unit == "kWh"
assert entries[f"meter.{electricity.uuid}.total"].entity.device_class == "energy"
assert entries[f"meter.{electricity.uuid}.today"].entity.state_class == "total_increasing"
device_ids = {
entries[f"source.{heating_source.uuid}.online"].entity.device.identifiers[0],
entries[f"meter.{heating.uuid}.total"].entity.device.identifiers[0],
entries[f"meter.{water.uuid}.total"].entity.device.identifiers[0],
entries[f"meter.{electricity.uuid}.total"].entity.device.identifiers[0],
}
assert len(device_ids) == 4
assert all(len(entry.entity.device.identifiers) == 1 for entry in entries.values())
thermal = [entry for key, entry in entries.items() if key.startswith("thermal_cost.")]
assert len(thermal) == 12
assert len(thermal) == 14
assert {entry.entity.key.rsplit(".", 1)[-1] for entry in thermal} >= {
"hot_water_total_total", "hot_water_total_today"
}
assert {entry.entity.name for entry in thermal if ".hot_water_total_" in entry.entity.key} == {
"Thermal Hot Water Total", "Thermal Hot Water Today"
}
assert all(entry.enabled is False for entry in thermal)
assert all(entry.entity.unit == "EUR" and entry.entity.device_class == "monetary" for entry in thermal)
def test_m8_hot_water_discovery_payload_is_ha_energy_compatible(energy_db) -> None:
"""Hot-water Meter discovery exposes the exact HA Energy Water metadata."""
from app.integrations.expose import build_catalog
from app.services.ha_discovery import build_discovery_payload
now = datetime.now(timezone.utc)
with Session(energy_db) as session:
_source, _channel, meter = _make_thermal_source_and_meter(session, "hot_water", now)
session.commit()
entries = {item.entity.key: item.entity for item in build_catalog(session)}
for suffix in ("total", "today"):
entity = entries[f"meter.{meter.uuid}.{suffix}"]
topic, payload = build_discovery_payload(entity, "homeassistant")
assert payload["device_class"] == "water"
assert payload["unit_of_measurement"] == "m³"
assert payload["state_class"] == "total_increasing"
assert payload["unique_id"] == f"{meter.uuid}_meter_{meter.uuid}_{suffix}"
assert payload["device"]["identifiers"] == [f"home-automation:meter:{meter.uuid}"]
assert topic == (
f"homeassistant/sensor/{meter.uuid.replace('-', '_')}/"
f"meter_{meter.uuid.replace('-', '_')}_{suffix}/config"
)
def test_m8_meter_getter_hides_stale_or_offline_source(energy_db) -> None:
from app.integrations.expose import build_catalog
@@ -2972,9 +2998,15 @@ def test_m8_thermal_today_summary_ends_at_frozen_now(energy_db) -> None:
captured: dict[str, datetime] = {}
result = {
"period_count": 1, "fixed_cost": Decimal("0"), "total_cost": Decimal("0"),
"breakdown": {key: Decimal("0") for key in (
"heating", "hot_water_heating", "hot_water", "hot_water_tax",
)},
"breakdown": {
"heating": Decimal("0"), "hot_water_heating": Decimal("1.25"),
"hot_water": Decimal("2.75"), "hot_water_tax": Decimal("9.99"),
},
"fixed_breakdown": {
"heating_network": Decimal("0"), "metering": Decimal("0"),
"delivery_set": Decimal("0"), "hot_water_network": Decimal("0"),
"other": Decimal("0"),
},
}
def summarize_spy(_session: Session, start: datetime, end: datetime, *, now: datetime) -> dict:
@@ -2994,4 +3026,55 @@ def test_m8_thermal_today_summary_ends_at_frozen_now(energy_db) -> None:
entity = next(item.entity for item in build_catalog(session)
if item.entity.key.endswith(".heating_today"))
assert entity.value_getter(session) == Decimal("0")
hot_water_total = next(item.entity for item in build_catalog(session)
if item.entity.key.endswith(".hot_water_total_today"))
assert hot_water_total.value_getter(session) == Decimal("13.99")
assert captured["end"] == now
@pytest.mark.parametrize("suffix", ("total", "today"))
def test_m8_thermal_totals_allocate_standing_costs_and_match_all_in(energy_db, suffix: str) -> None:
"""Heating and hot water each receive their assigned costs, without overlap."""
from app.integrations.expose import build_catalog
from app.services import timezone as tz
now = datetime(2026, 1, 15, 10, tzinfo=timezone.utc)
summary = {
"period_count": 2,
"fixed_cost": Decimal("35"),
"total_cost": Decimal("54"),
"breakdown": {
"heating": Decimal("10"), "hot_water_heating": Decimal("2"),
"hot_water": Decimal("3"), "hot_water_tax": Decimal("4"),
},
"fixed_breakdown": {
"heating_network": Decimal("5"), "metering": Decimal("6"),
"delivery_set": Decimal("7"), "hot_water_network": Decimal("9"),
"other": Decimal("8"),
},
}
with Session(energy_db) as session:
_make_thermal_source_and_meter(session, "heating", now)
_make_thermal_source_and_meter(session, "hot_water", now)
session.commit()
with (
patch("app.integrations.expose._utc_now", return_value=now),
patch.object(tz, "local_tz", return_value=ZoneInfo("Europe/Amsterdam")),
patch.object(tz, "local_now", return_value=now.astimezone(ZoneInfo("Europe/Amsterdam"))),
patch("app.services.meter_cost.summarize", return_value=summary),
):
entities = {item.entity.key.rsplit(".", 1)[-1]: item.entity
for item in build_catalog(session) if item.entity.key.startswith("thermal_cost.")}
heating = entities[f"heating_{suffix}"].value_getter(session)
hot_water = entities[f"hot_water_total_{suffix}"].value_getter(session)
assert heating == Decimal("36") # 10 + heating_network + metering + delivery_set + other
assert hot_water == Decimal("18") # 2 + 3 + 4 + hot_water_network
assert entities[f"fixed_{suffix}"].value_getter(session) == Decimal("35")
assert entities[f"all_in_{suffix}"].value_getter(session) == Decimal("54")
assert heating + hot_water == entities[f"all_in_{suffix}"].value_getter(session)
# Component entities deliberately remain variable-only.
assert entities[f"hot_water_heating_{suffix}"].value_getter(session) == Decimal("2")
assert entities[f"water_{suffix}"].value_getter(session) == Decimal("3")
assert entities[f"water_tax_{suffix}"].value_getter(session) == Decimal("4")
+15 -1
View File
@@ -362,7 +362,21 @@ def test_build_catalog_device_grouping(expose_db):
assert len(identifiers_set) == 1, (
"All entities for one device must share the same DeviceInfo identifiers"
)
assert identifiers_set.pop() == ("modbus", test_uuid)
own_identifiers = identifiers_set.pop()
assert own_identifiers == (f"home-automation:modbus:{test_uuid}",)
# A second Modbus device must never overlap its singleton HA identifier.
other_uuid = "aaaaaaaa-0000-0000-0000-000000000099"
with Session(expose_db) as session:
_make_modbus_device(session, friendly_name="SDM120 E", uuid=other_uuid)
session.commit()
with Session(expose_db) as session:
second_catalog = build_catalog(session)
other_identifiers = {
entry.entity.device.identifiers for entry in second_catalog if other_uuid in entry.entity.key
}
assert other_identifiers == {(f"home-automation:modbus:{other_uuid}",)}
assert {own_identifiers} != other_identifiers
def test_build_catalog_metric_metadata_from_profile(expose_db):
+500 -5
View File
@@ -15,6 +15,7 @@ Coverage:
from __future__ import annotations
import json
from datetime import datetime, timedelta, timezone
from pathlib import Path
from typing import Any
@@ -443,14 +444,14 @@ def test_publish_discovery_enabled_vs_disabled_payload(disco_db) -> None:
voltage_config_topic = f"homeassistant/sensor/{node_id}/{voltage_obj_id}/config"
# The voltage entity's config topic should have non-empty JSON payload
voltage_call = next(
((t, p, r) for t, p, r in config_calls if t == voltage_config_topic), None
)
assert voltage_call is not None, (
voltage_calls = [(t, p, r) for t, p, r in config_calls if t == voltage_config_topic]
assert voltage_calls, (
f"Expected config publish for voltage topic {voltage_config_topic!r}. "
f"Got topics: {[t for t, _, _ in config_calls]}"
)
_t, payload, retain = voltage_call
# First v1.6.1 repair run deliberately unloads the old config before it
# re-adds the same unique_id under the corrected HA device identifier.
_t, payload, retain = voltage_calls[-1]
assert payload not in (b"", "", None), "Enabled entity should get non-empty config payload"
assert retain is True, "Discovery config must be retained"
@@ -1378,3 +1379,497 @@ def test_stale_m8_entities_includes_ended_electricity_meter(disco_db) -> None:
session.commit()
keys = {entity.key for entity in _stale_m8_entities(session)}
assert keys == {f"meter.{old.uuid}.total", f"meter.{old.uuid}.today"}
def test_discovery_uses_single_namespaced_identifier_and_safe_thermal_topic() -> None:
"""HA device grouping is independent from a dot-containing internal seed."""
from app.integrations.expose import DeviceInfo, ExposableEntity
from app.services.ha_discovery import build_discovery_payload
identity = "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa.bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb"
device = DeviceInfo(
identifiers=(f"home-automation:thermal-cost:{identity}",),
identity=identity,
name="Thermal Energy Cost",
provides_availability=False,
)
entity = ExposableEntity(
key=f"thermal_cost.{identity}.heating_total", component="sensor", device=device,
device_class="monetary", unit="EUR", name="Thermal Heating Total", state_class="total",
)
topic, config = build_discovery_payload(entity, "homeassistant")
assert config["device"]["identifiers"] == [f"home-automation:thermal-cost:{identity}"]
assert all(part.replace("-", "").replace("_", "").isalnum()
for part in topic.split("/")[2:4])
assert "." not in topic
def test_legacy_thermal_cleanup_persists_each_success_and_retries_only_failure(disco_db) -> None:
"""Illegal v1.6.1 cleanup never re-sends a durably accepted topic."""
from app.integrations.expose import DeviceInfo, ExposableEntity
from app.services import ha_discovery
from app.models.config import AppConfigEntry
identity = "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa.bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb"
legacy = ExposableEntity(
key=f"thermal_cost.{identity}.heating_total", component="sensor",
device=DeviceInfo(identifiers=("thermal-cost", identity), identity=identity, name="obsolete"),
device_class=None, unit="", name="obsolete",
)
second_legacy = ExposableEntity(
key=f"thermal_cost.{identity}.heating_today", component="sensor",
device=legacy.device, device_class=None, unit="", name="obsolete",
)
settings = _make_settings()
manager = _make_mock_manager()
with (
patch("app.services.ha_discovery.build_runtime_settings", return_value=settings),
patch("app.services.ha_discovery.mqtt_manager", manager),
patch("app.services.ha_discovery.build_catalog", return_value=[]),
patch("app.services.ha_discovery._legacy_thermal_entities", return_value=[legacy, second_legacy]),
patch("app.services.ha_discovery._stale_m8_entities", return_value=[]),
):
with Session(disco_db) as session:
ha_discovery.initialize_legacy_thermal_cleanup(session)
manager.publish.side_effect = [True, False]
with Session(disco_db) as session:
ha_discovery.publish_discovery(session)
assert manager.publish.call_count == 2
with Session(disco_db) as session:
progress = session.query(AppConfigEntry).filter_by(
key=ha_discovery._LEGACY_THERMAL_CLEANUP_KEY
).one()
assert ha_discovery._legacy_discovery_topic(legacy, "homeassistant") in progress.value
manager.publish.reset_mock()
manager.publish.side_effect = None
manager.publish.return_value = True
with Session(disco_db) as session:
ha_discovery.publish_discovery(session)
assert manager.publish.call_args_list == [
((ha_discovery._legacy_discovery_topic(second_legacy, "homeassistant"), b""), {"retain": True}),
]
# A fresh Session simulates a process restart: no illegal topic is sent.
manager.publish.reset_mock()
with Session(disco_db) as session:
ha_discovery.publish_discovery(session)
manager.publish.assert_not_called()
def test_legacy_thermal_enumeration_failure_does_not_advance_marker(disco_db) -> None:
"""An inventory error is pending work, never an empty successful cleanup."""
from app.models.config import AppConfigEntry
from app.services import ha_discovery
settings = _make_settings()
manager = _make_mock_manager()
with (
patch("app.services.ha_discovery.build_runtime_settings", return_value=settings),
patch("app.services.ha_discovery.mqtt_manager", manager),
patch("app.services.ha_discovery.build_catalog", return_value=[]),
patch("app.services.ha_discovery._legacy_thermal_entities", side_effect=RuntimeError("enumeration failed")),
patch("app.services.ha_discovery._stale_m8_entities", return_value=[]),
):
with Session(disco_db) as session:
ha_discovery.publish_discovery(session)
with Session(disco_db) as session:
assert session.query(AppConfigEntry).filter_by(
key=ha_discovery._LEGACY_THERMAL_CLEANUP_KEY
).one_or_none() is None
def test_legacy_cleanup_compatibility_freeze_keeps_previous_success_progress(disco_db) -> None:
"""Startup freezes a pre-inventory ledger and preserves its success progress."""
from app.integrations.expose import DeviceInfo, ExposableEntity
from app.models.config import AppConfigEntry
from app.services import ha_discovery
device = DeviceInfo(identifiers=("legacy",), identity="old", name="obsolete")
first = ExposableEntity(key="thermal_cost.old.heating_total", component="sensor", device=device,
device_class=None, unit="", name="obsolete")
second = ExposableEntity(key="thermal_cost.old.heating_today", component="sensor", device=device,
device_class=None, unit="", name="obsolete")
first_topic = ha_discovery._legacy_discovery_topic(first, "homeassistant")
with Session(disco_db) as session:
session.add(AppConfigEntry(
key=ha_discovery._LEGACY_THERMAL_CLEANUP_KEY,
value=json.dumps({"complete": False, "topics": [first_topic]}),
updated_at=datetime.now(timezone.utc),
))
session.commit()
with (
patch("app.services.ha_discovery.build_runtime_settings", return_value=_make_settings()),
patch("app.services.ha_discovery._legacy_thermal_entities", return_value=[first, second]),
):
with Session(disco_db) as session:
ha_discovery.initialize_legacy_thermal_cleanup(session)
manager = _make_mock_manager()
with (
patch("app.services.ha_discovery.build_runtime_settings", return_value=_make_settings()),
patch("app.services.ha_discovery.mqtt_manager", manager),
patch("app.services.ha_discovery.build_catalog", return_value=[]),
patch("app.services.ha_discovery._stale_m8_entities", return_value=[]),
patch("app.services.ha_discovery._ha_registry_bindings", return_value=None),
):
with Session(disco_db) as session:
ha_discovery.publish_discovery(session)
assert manager.publish.call_args_list == [
((ha_discovery._legacy_discovery_topic(second, "homeassistant"), b""), {"retain": True}),
]
with Session(disco_db) as session:
ledger = ha_discovery._migration_json(session, ha_discovery._LEGACY_THERMAL_CLEANUP_KEY)
assert ledger == {
"complete": True,
"inventory": [first_topic, ha_discovery._legacy_discovery_topic(second, "homeassistant")],
"topics": sorted((first_topic, ha_discovery._legacy_discovery_topic(second, "homeassistant"))),
}
def test_legacy_cleanup_startup_marks_fresh_install_complete_and_short_circuits(disco_db) -> None:
"""A later first toggle cannot make a v1.6.1 topic after fresh startup."""
from app.models.config import AppConfigEntry
from app.services import ha_discovery
manager = _make_mock_manager()
settings = _make_settings()
with patch("app.services.ha_discovery._legacy_thermal_entities", return_value=[]):
with Session(disco_db) as session:
ha_discovery.initialize_legacy_thermal_cleanup(session)
with Session(disco_db) as session:
ledger = session.query(AppConfigEntry).filter_by(
key=ha_discovery._LEGACY_THERMAL_CLEANUP_KEY
).one()
assert ledger.value == '{"complete": true, "inventory": [], "topics": []}'
# A fresh Session is equivalent to a restarted process. If a user now
# enables a thermal entity, the completed ledger prevents re-enumeration.
with (
patch("app.services.ha_discovery.build_runtime_settings", return_value=settings),
patch("app.services.ha_discovery.mqtt_manager", manager),
patch("app.services.ha_discovery.build_catalog", return_value=[]),
patch("app.services.ha_discovery._legacy_thermal_entities") as legacy,
patch("app.services.ha_discovery._stale_m8_entities", return_value=[]),
patch("app.services.ha_discovery._ha_registry_bindings", return_value=None),
):
with Session(disco_db) as session:
ha_discovery.publish_discovery(session)
legacy.assert_not_called()
manager.publish.assert_not_called()
def test_legacy_cleanup_startup_completes_empty_compat_ledger_before_later_enable(disco_db) -> None:
"""An Alembic-head empty compat ledger cannot manufacture a later old topic."""
from app.models.config import AppConfigEntry
from app.models.energy import Meter
from app.services import ha_discovery
now = datetime.now(timezone.utc)
with Session(disco_db) as session:
heating = Meter(label="heating", commodity="heating", started_at=now, ended_at=None,
reason="initial", note=None, created_at=now)
water = Meter(label="water", commodity="hot_water", started_at=now, ended_at=None,
reason="initial", note=None, created_at=now)
session.add_all((heating, water))
session.add(AppConfigEntry(
key=ha_discovery._LEGACY_THERMAL_CLEANUP_KEY,
value=json.dumps({"complete": False, "topics": []}),
updated_at=now,
))
session.commit()
settings = _make_settings(ha_discovery_prefix="startup_prefix")
with patch("app.services.ha_discovery.build_runtime_settings", return_value=settings):
with Session(disco_db) as session:
ha_discovery.initialize_legacy_thermal_cleanup(session)
with Session(disco_db) as session:
ledger = ha_discovery._migration_json(session, ha_discovery._LEGACY_THERMAL_CLEANUP_KEY)
assert ledger == {"complete": True, "inventory": [], "topics": []}
active_meters = session.query(Meter).filter(Meter.ended_at.is_(None)).all()
pair = ha_discovery._thermal_cleanup_entities(
[(next(meter for meter in active_meters if meter.commodity == "heating"),
next(meter for meter in active_meters if meter.commodity == "hot_water"))],
include_hot_water_total=False,
)
_enable_entity(session, pair[0].key)
session.commit()
manager = _make_mock_manager()
with (
patch("app.services.ha_discovery.build_runtime_settings",
return_value=_make_settings(ha_discovery_prefix="changed_prefix")),
patch("app.services.ha_discovery.mqtt_manager", manager),
patch("app.services.ha_discovery.build_catalog", return_value=[]),
patch("app.services.ha_discovery._stale_m8_entities", return_value=[]),
patch("app.services.ha_discovery._ha_registry_bindings", return_value=None),
):
with Session(disco_db) as session:
ha_discovery.publish_discovery(session)
manager.publish.assert_not_called()
def test_legacy_cleanup_startup_propagates_inventory_failure(disco_db) -> None:
"""Startup must fail closed instead of exposing a mutable cleanup window."""
from app.services import ha_discovery
with patch("app.services.ha_discovery._legacy_thermal_entities", side_effect=RuntimeError("boom")):
with Session(disco_db) as session:
with pytest.raises(RuntimeError, match="boom"):
ha_discovery.initialize_legacy_thermal_cleanup(session)
def test_legacy_cleanup_startup_propagates_durable_write_failure(disco_db) -> None:
"""A failed freeze write is fatal; UI must not open with an unfrozen ledger."""
from app.services import ha_discovery
with (
patch("app.services.ha_discovery._legacy_thermal_entities", return_value=[]),
patch("app.services.ha_discovery._set_migration_json", side_effect=RuntimeError("disk full")),
):
with Session(disco_db) as session:
with pytest.raises(RuntimeError, match="disk full"):
ha_discovery.initialize_legacy_thermal_cleanup(session)
def test_legacy_cleanup_startup_keeps_upgrade_with_enabled_topic_pending(disco_db) -> None:
"""Existing enabled v1.6.1 inventory is not mistaken for a fresh install."""
from app.integrations.expose import DeviceInfo, ExposableEntity
from app.models.config import AppConfigEntry
from app.services import ha_discovery
legacy = ExposableEntity(
key="thermal_cost.old.heating_total", component="sensor",
device=DeviceInfo(identifiers=("legacy",), identity="old", name="obsolete"),
device_class=None, unit="", name="obsolete",
)
with patch("app.services.ha_discovery._legacy_thermal_entities", return_value=[legacy]):
with Session(disco_db) as session:
ha_discovery.initialize_legacy_thermal_cleanup(session)
with Session(disco_db) as session:
ledger = session.query(AppConfigEntry).filter_by(
key=ha_discovery._LEGACY_THERMAL_CLEANUP_KEY
).one()
assert ledger.value == (
'{"complete": false, "inventory": ["homeassistant/sensor/old/'
'thermal_cost_old_heating_total/config"], "topics": []}'
)
def test_legacy_cleanup_freezes_startup_upgrade_inventory_across_toggle_changes(disco_db) -> None:
"""Alembic-head compat ledger survives UI disable and runtime prefix changes."""
from app.models.config import AppConfigEntry
from app.models.energy import Meter
from app.models.expose import ExposedEntityToggle
from app.services import ha_discovery
now = datetime.now(timezone.utc)
settings = _make_settings(ha_discovery_prefix="frozen_prefix")
with Session(disco_db) as session:
heating = Meter(label="heating", commodity="heating", started_at=now, ended_at=None,
reason="initial", note=None, created_at=now)
water = Meter(label="water", commodity="hot_water", started_at=now, ended_at=None,
reason="initial", note=None, created_at=now)
session.add_all((heating, water))
session.flush()
enabled = ha_discovery._thermal_cleanup_entities(
[(heating, water)], include_hot_water_total=False
)[:2]
for entity in enabled:
_enable_entity(session, entity.key)
first_topic = ha_discovery._legacy_discovery_topic(enabled[0], "frozen_prefix")
session.add(AppConfigEntry(
key=ha_discovery._LEGACY_THERMAL_CLEANUP_KEY,
value=json.dumps({"complete": False, "topics": [first_topic]}),
updated_at=now,
))
session.commit()
with patch("app.services.ha_discovery.build_runtime_settings", return_value=settings):
with Session(disco_db) as session:
ha_discovery.initialize_legacy_thermal_cleanup(session)
with Session(disco_db) as session:
ledger = ha_discovery._migration_json(session, ha_discovery._LEGACY_THERMAL_CLEANUP_KEY)
inventory = ledger["inventory"]
assert len(inventory) == 2
assert all(topic.startswith("frozen_prefix/") for topic in inventory)
assert ledger["topics"] == [first_topic]
# This mirrors PUT /api/expose: persist the UI change before it invokes
# publish_discovery in the same request.
toggle = session.query(ExposedEntityToggle).filter_by(key=enabled[0].key).one()
toggle.enabled = False
session.commit()
manager = _make_mock_manager()
failed_topic = inventory[-1]
published: list[str] = []
manager.publish.side_effect = lambda topic, _payload, **_kwargs: (
published.append(topic) or topic != failed_topic
)
with (
patch("app.services.ha_discovery.build_runtime_settings",
return_value=_make_settings(ha_discovery_prefix="changed_prefix")),
patch("app.services.ha_discovery.mqtt_manager", manager),
patch("app.services.ha_discovery.build_catalog", return_value=[]),
patch("app.services.ha_discovery._stale_m8_entities", return_value=[]),
patch("app.services.ha_discovery._ha_registry_bindings", return_value=None),
):
with Session(disco_db) as session:
ha_discovery.publish_discovery(session)
assert published == [failed_topic]
with Session(disco_db) as session:
pending = ha_discovery._migration_json(session, ha_discovery._LEGACY_THERMAL_CLEANUP_KEY)
assert pending["complete"] is False
assert pending["inventory"] == inventory
assert pending["topics"] == [first_topic]
manager.publish.reset_mock()
manager.publish.side_effect = lambda topic, _payload, **_kwargs: published.append(topic) or True
published.clear()
with (
patch("app.services.ha_discovery.build_runtime_settings",
return_value=_make_settings(ha_discovery_prefix="changed_prefix")),
patch("app.services.ha_discovery.mqtt_manager", manager),
patch("app.services.ha_discovery.build_catalog", return_value=[]),
patch("app.services.ha_discovery._legacy_thermal_entities") as enumerate_legacy,
patch("app.services.ha_discovery._stale_m8_entities", return_value=[]),
patch("app.services.ha_discovery._ha_registry_bindings", return_value=None),
):
with Session(disco_db) as session:
ha_discovery.publish_discovery(session)
assert published == [failed_topic]
enumerate_legacy.assert_not_called()
manager.publish.reset_mock()
with (
patch("app.services.ha_discovery.build_runtime_settings", return_value=settings),
patch("app.services.ha_discovery.mqtt_manager", manager),
patch("app.services.ha_discovery.build_catalog", return_value=[]),
patch("app.services.ha_discovery._legacy_thermal_entities") as enumerate_legacy,
patch("app.services.ha_discovery._stale_m8_entities", return_value=[]),
patch("app.services.ha_discovery._ha_registry_bindings", return_value=None),
):
with Session(disco_db) as session:
ha_discovery.publish_discovery(session)
enumerate_legacy.assert_not_called()
manager.publish.assert_not_called()
def test_registry_repair_waits_for_registry_observation_and_partial_catalog_recovers(disco_db) -> None:
"""The V2 repair is per entity and never mistakes broker ACK for HA ACK."""
from app.integrations.expose import CatalogEntry, DeviceInfo, ExposableEntity
from app.services import ha_discovery
def entity(kind: str, identity: str, metric: str) -> ExposableEntity:
return ExposableEntity(
key=f"{kind}.{identity}.{metric}" if kind != "energy" else f"energy.{metric}",
component="sensor",
device=DeviceInfo(
identifiers=(f"home-automation:{kind}:{identity}",), identity=identity, name=identity,
provides_availability=False,
),
device_class=None, unit="", name=metric,
)
entities = [
*(entity("meter", f"meter-{number}", "total") for number in range(3)),
*(entity("source", f"source-{number}", "online") for number in range(2)),
*(entity("modbus", f"modbus-{number}", "voltage") for number in range(2)),
entity("energy", "electricity-epoch", "import_cost_total"),
]
catalog = [CatalogEntry(entity=item, enabled=True) for item in entities]
manager = _make_mock_manager()
manager.publish.return_value = True
settings = _make_settings()
calls: list[tuple[str, object]] = []
manager.publish.side_effect = lambda topic, payload, **_kwargs: calls.append((topic, payload)) or True
bindings: dict[str, set[str]] = {ha_discovery._unique_id(item): {"old"} for item in entities}
with (
patch("app.services.ha_discovery.build_runtime_settings", return_value=settings),
patch("app.services.ha_discovery.mqtt_manager", manager),
patch("app.services.ha_discovery.build_catalog", return_value=catalog),
patch("app.services.ha_discovery._legacy_thermal_entities", return_value=[]),
patch("app.services.ha_discovery._stale_m8_entities", return_value=[]),
patch("app.services.ha_discovery._ha_registry_bindings", side_effect=lambda _settings, ids: {
key: value for key, value in bindings.items() if key in ids
}),
):
with Session(disco_db) as session:
ha_discovery.publish_discovery(session)
topics = [ha_discovery._discovery_topic(item, "homeassistant") for item in entities]
assert [payload for _topic, payload in calls[:len(entities)]] == [b""] * len(entities)
assert [topic for topic, _payload in calls[:len(entities)]] == topics
assert len({ha_discovery._unique_id(item) for item in entities}) == len(entities)
# HA confirms every tombstone; only then is each target re-added.
calls.clear()
bindings.clear()
with (
patch("app.services.ha_discovery.build_runtime_settings", return_value=settings),
patch("app.services.ha_discovery.mqtt_manager", manager),
patch("app.services.ha_discovery.build_catalog", return_value=catalog),
patch("app.services.ha_discovery._stale_m8_entities", return_value=[]),
patch("app.services.ha_discovery._ha_registry_bindings", side_effect=lambda _settings, _ids: dict(bindings)),
):
with Session(disco_db) as session:
ha_discovery.publish_discovery(session)
assert len(calls) == len(entities)
assert all(payload not in (b"", "", None) for _topic, payload in calls)
def test_registry_repair_unavailable_keeps_normal_discovery_publishing(disco_db) -> None:
"""A broken optional HA WS link cannot leave legal configs tombstoned."""
from app.integrations.expose import CatalogEntry, DeviceInfo, ExposableEntity
from app.services import ha_discovery
entity = ExposableEntity(
key="meter.meter-1.total", component="sensor",
device=DeviceInfo(identifiers=("home-automation:meter:meter-1",), identity="meter-1", name="m1"),
device_class=None, unit="", name="total",
)
manager = _make_mock_manager()
with (
patch("app.services.ha_discovery.build_runtime_settings", return_value=_make_settings()),
patch("app.services.ha_discovery.mqtt_manager", manager),
patch("app.services.ha_discovery.build_catalog", return_value=[CatalogEntry(entity, True)]),
patch("app.services.ha_discovery._legacy_thermal_entities", return_value=[]),
patch("app.services.ha_discovery._stale_m8_entities", return_value=[]),
patch("app.services.ha_discovery._ha_registry_bindings", return_value=None),
):
with Session(disco_db) as session:
ha_discovery.publish_discovery(session)
assert manager.publish.call_args.args[1] != b""
def test_stale_thermal_cleanup_uses_safe_topics_and_all_fourteen_metrics(disco_db) -> None:
"""Later thermal meter swaps clear only ended safe-format pair configs."""
from app.models.energy import Meter
from app.services import ha_discovery
now = datetime.now(timezone.utc)
with Session(disco_db) as session:
old_heating = Meter(label="old heat", commodity="heating", started_at=now - timedelta(days=2),
ended_at=now - timedelta(days=1), reason="meter_swap", note=None, created_at=now)
old_water = Meter(label="old water", commodity="hot_water", started_at=now - timedelta(days=2),
ended_at=now - timedelta(days=1), reason="meter_swap", note=None, created_at=now)
active_heating = Meter(label="new heat", commodity="heating", started_at=now - timedelta(days=1),
ended_at=None, reason="meter_swap", note=None, created_at=now)
active_water = Meter(label="new water", commodity="hot_water", started_at=now - timedelta(days=1),
ended_at=None, reason="meter_swap", note=None, created_at=now)
session.add_all((old_heating, old_water, active_heating, active_water))
session.commit()
stale = ha_discovery._stale_m8_entities(session)
old_identity = ".".join(sorted((old_heating.uuid, old_water.uuid)))
thermal = [item for item in stale if item.key.startswith("thermal_cost.")]
assert len(thermal) == 14
assert {item.key for item in thermal} == {
f"thermal_cost.{old_identity}.{metric}_{suffix}"
for metric in ("heating", "hot_water_heating", "hot_water_total", "water", "water_tax", "fixed", "all_in")
for suffix in ("total", "today")
}
assert all("." not in ha_discovery._discovery_topic(item, "homeassistant") for item in thermal)
assert not any(active_heating.uuid in item.key and active_water.uuid in item.key for item in thermal)
+84
View File
@@ -111,3 +111,87 @@ def test_homeassistant_client_raises_on_invalid_arguments() -> None:
with pytest.raises(ValueError, match="webhook_id"):
client.trigger_webhook(webhook_id="", body={})
def test_discovery_registry_bindings_reads_entity_and_device_registry(monkeypatch: pytest.MonkeyPatch) -> None:
"""The repair confirmation reads HA's authoritative registry bindings."""
sent: list[dict] = []
class _Socket:
replies = iter((
'{"type":"auth_required"}',
'{"type":"auth_ok"}',
'{"id":1,"success":true,"result":[{"platform":"mqtt","unique_id":"u1","device_id":"d1"}]}',
'{"id":2,"success":true,"result":[{"id":"d1","identifiers":[["mqtt","home-automation:meter:m1"]]}]}',
))
def __enter__(self):
return self
def __exit__(self, *_args):
return None
def recv(self, *, timeout=None):
assert timeout is not None
assert timeout <= 1.5
return next(self.replies)
def send(self, payload):
sent.append(json.loads(payload))
monkeypatch.setattr("app.integrations.homeassistant.connect", lambda *_args, **_kwargs: _Socket())
bindings = HomeAssistantClient(_configured_settings()).discovery_registry_bindings({"u1", "missing"})
assert bindings == {"u1": {"home-automation:meter:m1"}}
assert [message.get("type") for message in sent] == [
"auth", "config/entity_registry/list", "config/device_registry/list"
]
def test_discovery_registry_bindings_uses_one_deadline_and_ignores_non_mqtt_identifiers(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Only official ["mqtt", value] pairs participate in repair matching."""
received_timeouts: list[float] = []
class _Socket:
replies = iter((
'{"type":"auth_required"}',
'{"type":"auth_ok"}',
'{"id":1,"success":true,"result":['
'{"platform":"mqtt","unique_id":"u1","device_id":"d1"},'
'{"platform":"mqtt","unique_id":"u2","device_id":"d2"},'
'{"platform":"mqtt","unique_id":"u3","device_id":"d3"}]}',
'{"id":2,"success":true,"result":['
'{"id":"d1","identifiers":[["mqtt","expected"],["esphome","aux"]]},'
'{"id":"d2","identifiers":[["esphome","expected"],"expected",["mqtt",3],[]]},'
'{"id":"d3","identifiers":[["mqtt","expected"],["mqtt","old"]]}]}',
))
def __enter__(self): return self
def __exit__(self, *_args): return None
def send(self, _payload): return None
def recv(self, *, timeout=None):
received_timeouts.append(timeout)
return next(self.replies)
monkeypatch.setattr("app.integrations.homeassistant.connect", lambda *_args, **_kwargs: _Socket())
bindings = HomeAssistantClient(_configured_settings()).discovery_registry_bindings({"u1", "u2", "u3"})
assert bindings == {"u1": {"expected"}, "u2": set(), "u3": {"expected", "old"}}
assert len(received_timeouts) == 4
assert all(timeout is not None and 0 < timeout <= 1.5 for timeout in received_timeouts)
def test_discovery_registry_bindings_silent_socket_times_out(monkeypatch: pytest.MonkeyPatch) -> None:
class _Socket:
def __enter__(self): return self
def __exit__(self, *_args): return None
def send(self, _payload): return None
def recv(self, *, timeout=None):
assert timeout is not None and timeout > 0
raise TimeoutError("silent")
monkeypatch.setattr("app.integrations.homeassistant.connect", lambda *_args, **_kwargs: _Socket())
with pytest.raises(HomeAssistantRequestError, match="registry query failed"):
HomeAssistantClient(_configured_settings()).discovery_registry_bindings({"u1"})
+26
View File
@@ -68,6 +68,32 @@ def test_secret_sanitize_and_mask_merge_keep_old_value():
assert merged["topic"] == "new/topic"
def test_dsmr_zero_interval_validates_sanitizes_and_merges_unchanged():
config = validate_source_config("dsmr_mqtt", {"sample_interval_s": 0})
assert config["sample_interval_s"] == 0
assert sanitize_source_config("dsmr_mqtt", config)["sample_interval_s"] == 0
assert merge_source_config("dsmr_mqtt", config, {"topic": "new/topic"})["sample_interval_s"] == 0
@pytest.mark.parametrize(
("kind", "config"),
[
("dsmr_mqtt", {"sample_interval_s": -1}),
("dsmr_mqtt", {"sample_interval_s": False}),
("dsmr_mqtt", {"broker_port": 0}),
("dsmr_mqtt", {"broker_port": -1}),
("dsmr_mqtt", {"broker_port": False}),
("warmtelink_serial", {"path": "/dev/warmtelink", "baudrate": 0}),
("warmtelink_serial", {"path": "/dev/warmtelink", "baudrate": -1}),
("warmtelink_serial", {"path": "/dev/warmtelink", "baudrate": False}),
],
)
def test_numeric_source_profile_constraints_still_reject_invalid_values(kind, config):
with pytest.raises(SourceProfileError):
validate_source_config(kind, config)
@pytest.fixture()
def session(tmp_path):
engine = create_engine(f"sqlite:///{tmp_path / 'source_services.db'}")
+287 -14
View File
@@ -4,6 +4,7 @@ All paho interaction is mocked — no real broker is required.
"""
from __future__ import annotations
import threading
from unittest.mock import MagicMock, patch
import pytest
@@ -103,7 +104,7 @@ def test_connect_creates_paho_client_with_version2() -> None:
settings = _make_settings()
mock_client = MagicMock()
mock_client.connect.return_value = None
mock_client.connect_async.return_value = None
with patch("app.integrations.mqtt.mqtt.Client", return_value=mock_client) as mock_cls:
with patch("app.integrations.mqtt.mqtt.CallbackAPIVersion"):
@@ -117,7 +118,7 @@ def test_connect_creates_paho_client_with_version2() -> None:
# loop_start must have been called
mock_client.loop_start.assert_called_once()
# connect must have been called with the right host/port
mock_client.connect.assert_called_once_with(
mock_client.connect_async.assert_called_once_with(
host="broker.test", port=1883, keepalive=60
)
@@ -133,6 +134,220 @@ def test_connect_uses_configured_client_id() -> None:
assert mock_cls.call_args.kwargs["client_id"] == "home-automation-dev"
def test_connect_same_settings_is_idempotent_even_when_disconnected() -> None:
manager = MqttManager()
settings = _make_settings()
client = MagicMock()
with patch("app.integrations.mqtt.mqtt.Client", return_value=client) as cls:
manager.connect(settings)
manager.connect(settings)
cls.assert_called_once()
client.loop_start.assert_called_once()
client.connect_async.assert_called_once()
def test_connect_replaces_client_when_paho_loop_thread_has_exited() -> None:
manager = MqttManager()
settings = _make_settings()
old_client, recovered_client = MagicMock(), MagicMock()
with patch("app.integrations.mqtt.mqtt.Client", side_effect=[old_client, recovered_client]):
manager.connect(settings)
old_client._thread = None
manager._client_loop_started = True
manager.connect(settings)
old_client.disconnect.assert_called_once()
old_client.loop_stop.assert_called_once()
recovered_client.loop_start.assert_called_once()
def test_loop_start_failure_cleans_installed_client_for_retry() -> None:
manager = MqttManager()
failed_client = MagicMock()
failed_client.loop_start.return_value = -1
recovered_client = MagicMock()
with patch("app.integrations.mqtt.mqtt.Client", side_effect=[failed_client, recovered_client]):
manager.connect(_make_settings())
assert manager._client is None
manager.connect(_make_settings())
failed_client.disconnect.assert_called_once()
failed_client.loop_stop.assert_called_once()
assert manager._client is recovered_client
def test_concurrent_same_config_connect_creates_one_client() -> None:
manager = MqttManager()
settings = _make_settings()
client = MagicMock()
barrier = threading.Barrier(3)
def connect() -> None:
barrier.wait(timeout=1)
manager.connect(settings)
with patch("app.integrations.mqtt.mqtt.Client", return_value=client) as cls:
workers = [threading.Thread(target=connect) for _ in range(2)]
for worker in workers:
worker.start()
barrier.wait(timeout=1)
for worker in workers:
worker.join(timeout=2)
assert all(not worker.is_alive() for worker in workers)
cls.assert_called_once()
client.loop_start.assert_called_once()
@pytest.mark.parametrize("operation", ["disconnect", "reconnect"])
def test_teardown_can_join_loop_while_disconnect_callback_runs(operation: str) -> None:
"""A real callback thread must finish while teardown joins the fake loop."""
manager = MqttManager()
callback_go = threading.Event()
callback_entered = threading.Event()
callback_done = threading.Event()
clients: list[object] = []
class LoopClient:
def __init__(self) -> None:
self._thread: threading.Thread | None = None
self.on_disconnect = None
self.loop_joined = threading.Event()
def connect_async(self, **_kwargs: object) -> None:
pass
def loop_start(self) -> int:
def network_callback() -> None:
callback_go.wait(timeout=2)
callback_entered.set()
try:
self.on_disconnect(self, None, None, MagicMock(is_failure=True), None)
finally:
callback_done.set()
self._thread = threading.Thread(target=network_callback, daemon=True)
self._thread.start()
return 0
def disconnect(self) -> None:
callback_go.set()
def loop_stop(self) -> None:
assert callback_entered.wait(timeout=1), "callback thread did not start"
assert self._thread is not None
self._thread.join(timeout=1)
assert not self._thread.is_alive(), "callback thread did not exit before loop_stop returned"
self.loop_joined.set()
def cleanup(self) -> None:
callback_go.set()
if self._thread is not None:
self._thread.join(timeout=1)
def make_client(**_kwargs: object) -> LoopClient:
client = LoopClient()
clients.append(client)
return client
try:
with patch("app.integrations.mqtt.mqtt.Client", side_effect=make_client):
manager.connect(_make_settings())
old_client = clients[0]
if operation == "disconnect":
manager.disconnect()
assert manager._client is None
else:
manager.reconnect(_make_settings(mqtt_client_id="replacement"))
assert manager._client is clients[1]
assert callback_done.is_set()
assert old_client.loop_joined.is_set()
assert old_client._thread is not None and not old_client._thread.is_alive()
finally:
for client in clients:
client.cleanup()
@pytest.mark.parametrize("failure", ["constructor", "tls", "credentials", "connect_async", "loop_start"])
def test_initialization_failure_is_cleaned_and_next_connect_recovers(failure: str) -> None:
manager = MqttManager()
broken, recovered = MagicMock(), MagicMock()
broken.connect_async.return_value = None
if failure == "tls":
broken.tls_set.side_effect = OSError("synthetic TLS failure")
elif failure == "credentials":
broken.username_pw_set.side_effect = OSError("synthetic credential failure")
elif failure == "connect_async":
broken.connect_async.side_effect = OSError("synthetic connect setup failure")
elif failure == "loop_start":
broken.loop_start.side_effect = OSError("synthetic loop failure")
settings = _make_settings(
mqtt_tls_enabled=failure == "tls",
mqtt_username="user" if failure == "credentials" else "",
mqtt_password="synthetic",
)
first = (
[RuntimeError("synthetic constructor failure"), recovered]
if failure == "constructor"
else [broken, recovered]
)
with patch("app.integrations.mqtt.mqtt.Client", side_effect=first):
manager.connect(settings)
assert manager._client is None
manager.connect(settings)
assert manager._client is recovered
recovered.loop_start.assert_called_once()
recovered.connect_async.assert_called_once()
if failure in {"connect_async", "loop_start"}:
broken.disconnect.assert_called_once()
broken.loop_stop.assert_called_once()
elif failure in {"tls", "credentials"}:
broken.loop_start.assert_not_called()
@pytest.mark.parametrize("old_is_failure", [False, True])
def test_old_connect_success_and_failure_callbacks_cannot_change_replacement(
old_is_failure: bool,
) -> None:
manager = MqttManager()
old_client, new_client = MagicMock(), MagicMock()
with patch("app.integrations.mqtt.mqtt.Client", side_effect=[old_client, new_client]):
manager.connect(_make_settings())
manager.reconnect(_make_settings(mqtt_client_id="replacement"))
manager._connected = True
old_client.on_connect(
old_client, None, None, MagicMock(is_failure=old_is_failure), None
)
assert manager._client is new_client
assert manager.is_connected
new_client.subscribe.assert_not_called()
def test_old_client_callbacks_cannot_change_replacement_state_or_dispatch() -> None:
manager = MqttManager()
settings = _make_settings()
old_client, new_client = MagicMock(), MagicMock()
received: list[bytes] = []
manager.subscribe("sensor/state", received.append)
with patch("app.integrations.mqtt.mqtt.Client", side_effect=[old_client, new_client]):
manager.connect(settings)
manager.reconnect(_make_settings(mqtt_client_id="replacement"))
manager._connected = True
failed = MagicMock()
failed.is_failure = True
old_client.on_disconnect(old_client, None, None, failed, None)
old_client.on_message(old_client, None, MagicMock(topic="sensor/state", payload=b"stale"))
assert manager.is_connected
assert received == []
new_client.on_message(new_client, None, MagicMock(topic="sensor/state", payload=b"fresh"))
assert received == [b"fresh"]
def test_deployment_client_id_variants_are_distinct() -> None:
production_ids = {
"home-automation",
@@ -155,7 +370,7 @@ def test_connect_sets_credentials_when_username_provided() -> None:
settings = _make_settings(mqtt_username="user", mqtt_password="s3cr3t")
mock_client = MagicMock()
mock_client.connect.return_value = None
mock_client.connect_async.return_value = None
with patch("app.integrations.mqtt.mqtt.Client", return_value=mock_client):
manager.connect(settings)
@@ -170,7 +385,7 @@ def test_connect_skips_credentials_when_username_empty() -> None:
settings = _make_settings(mqtt_username="", mqtt_password="")
mock_client = MagicMock()
mock_client.connect.return_value = None
mock_client.connect_async.return_value = None
with patch("app.integrations.mqtt.mqtt.Client", return_value=mock_client):
manager.connect(settings)
@@ -183,7 +398,7 @@ def test_connect_calls_tls_set_when_tls_enabled() -> None:
settings = _make_settings(mqtt_tls_enabled=True)
mock_client = MagicMock()
mock_client.connect.return_value = None
mock_client.connect_async.return_value = None
with patch("app.integrations.mqtt.mqtt.Client", return_value=mock_client):
manager.connect(settings)
@@ -200,7 +415,7 @@ def test_connect_does_not_raise_on_connection_failure() -> None:
settings = _make_settings()
mock_client = MagicMock()
mock_client.connect.side_effect = OSError("Connection refused")
mock_client.connect_async.side_effect = OSError("Connection refused")
with patch("app.integrations.mqtt.mqtt.Client", return_value=mock_client):
# Must not raise — failures are caught and logged
@@ -221,7 +436,7 @@ def test_disconnect_stops_loop_and_clears_client() -> None:
settings = _make_settings()
mock_client = MagicMock()
mock_client.connect.return_value = None
mock_client.connect_async.return_value = None
with patch("app.integrations.mqtt.mqtt.Client", return_value=mock_client):
manager.connect(settings)
@@ -249,7 +464,7 @@ def test_publish_passes_topic_payload_retain_to_paho() -> None:
settings = _make_settings()
mock_client = MagicMock()
mock_client.connect.return_value = None
mock_client.connect_async.return_value = None
with patch("app.integrations.mqtt.mqtt.Client", return_value=mock_client):
manager.connect(settings)
@@ -257,7 +472,7 @@ def test_publish_passes_topic_payload_retain_to_paho() -> None:
manager._connected = True
manager._client = mock_client
manager.publish("test/topic", '{"key": "value"}', retain=True, qos=1)
assert manager.publish("test/topic", '{"key": "value"}', retain=True, qos=1) is True
mock_client.publish.assert_called_once_with(
"test/topic", payload='{"key": "value"}', qos=1, retain=True
@@ -267,8 +482,7 @@ def test_publish_passes_topic_payload_retain_to_paho() -> None:
def test_publish_is_noop_when_not_connected() -> None:
manager = MqttManager()
# No connect — should silently skip
manager.publish("topic", "payload", retain=False)
# No exception raised
assert manager.publish("topic", "payload", retain=False) is False
def test_publish_does_not_raise_on_paho_error() -> None:
@@ -279,7 +493,17 @@ def test_publish_does_not_raise_on_paho_error() -> None:
manager._connected = True
# Must not raise
manager.publish("topic", "payload")
assert manager.publish("topic", "payload") is False
def test_publish_returns_false_when_paho_rejects_message() -> None:
manager = MqttManager()
mock_client = MagicMock()
mock_client.publish.return_value.rc = 1
manager._client = mock_client
manager._connected = True
assert manager.publish("topic", "payload") is False
# ---------------------------------------------------------------------------
@@ -297,7 +521,7 @@ def test_reconnect_stops_old_client_and_starts_new_one() -> None:
manager._connected = True
new_client = MagicMock()
new_client.connect.return_value = None
new_client.connect_async.return_value = None
with patch("app.integrations.mqtt.mqtt.Client", return_value=new_client):
manager.reconnect(settings)
@@ -308,11 +532,60 @@ def test_reconnect_stops_old_client_and_starts_new_one() -> None:
# New client must have been connected
new_client.loop_start.assert_called_once()
new_client.connect.assert_called_once_with(
new_client.connect_async.assert_called_once_with(
host="broker.test", port=1883, keepalive=60
)
@pytest.mark.parametrize(
"settings",
[
_make_settings(mqtt_enabled=False),
_make_settings(mqtt_broker_host=""),
],
ids=["disabled", "empty-host"],
)
def test_unconfigured_reconnect_stops_main_client_but_keeps_dsmr_source(settings) -> None:
"""Changing main MQTT config must not interrupt independent DSMR clients."""
manager = MqttManager()
main_client = MagicMock()
source_client = MagicMock()
received: list[bytes] = []
with patch(
"app.integrations.mqtt.mqtt.Client",
side_effect=[main_client, source_client],
):
manager.connect(_make_settings())
assert manager.replace_source(
17,
host="source.test",
port=1883,
username="",
password="",
tls_enabled=False,
subscriptions={"meter/17": received.append},
)
manager.reconnect(settings)
assert manager._client is None
main_client.disconnect.assert_called_once()
main_client.loop_stop.assert_called_once()
assert manager._source_clients[17] is source_client
assert manager.source_is_active(17)
source_client.disconnect.assert_not_called()
source_client.loop_stop.assert_not_called()
message = MagicMock()
message.topic = "meter/17"
message.payload = b"reading"
source_client.on_message(source_client, None, message)
assert received == [b"reading"]
manager.disconnect()
# ---------------------------------------------------------------------------
# POST /api/config/mqtt/test — session + CSRF guards
# ---------------------------------------------------------------------------
+68
View File
@@ -0,0 +1,68 @@
"""Offline regression coverage for scheduler-driven MQTT recovery."""
from __future__ import annotations
from unittest.mock import MagicMock, patch
from app.integrations.mqtt import MqttManager
from app.main import _run_scheduled_ha_state_publish
def test_scheduler_reuses_manager_across_disconnects_then_subscribes_and_publishes() -> None:
session = MagicMock()
session_local = MagicMock(return_value=session)
settings = MagicMock(
mqtt_enabled=True,
mqtt_broker_host="synthetic-broker",
mqtt_broker_port=1883,
mqtt_username="",
mqtt_password="",
mqtt_tls_enabled=False,
mqtt_client_id="offline-test",
)
manager = MqttManager()
clients: list[MagicMock] = []
def make_client(*_args: object, **_kwargs: object) -> MagicMock:
client = MagicMock()
clients.append(client)
return client
manager.subscribe("homeassistant/status", lambda _payload: None)
def publish_states(_session: object) -> None:
manager.publish("homeassistant/state", b"state", retain=True)
with (
patch("app.main.get_session_local", return_value=session_local),
patch("app.main.build_runtime_settings", return_value=settings),
patch("app.main.get_settings", return_value=MagicMock()),
patch("app.main.mqtt_manager", manager),
patch("app.main.publish_discovery") as publish_discovery,
patch("app.main.publish_states", side_effect=publish_states),
patch("app.integrations.mqtt.mqtt.Client", side_effect=make_client),
):
for _ in range(3):
_run_scheduled_ha_state_publish()
assert len(clients) == 1
assert clients[0].loop_start.call_count == 1
clients[0].on_disconnect(
clients[0], None, None, MagicMock(is_failure=True), None
)
assert not manager.is_connected
# A subsequent scheduler turn sees Paho's existing loop retrying; it
# must not replace that client. Simulate the successful CONNACK.
_run_scheduled_ha_state_publish()
assert len(clients) == 1
clients[0].on_connect(clients[0], None, None, MagicMock(is_failure=False), None)
assert manager.is_connected
clients[0].subscribe.assert_called_with("homeassistant/status")
_run_scheduled_ha_state_publish()
assert len(clients) == 1
clients[0].publish.assert_called_once_with(
"homeassistant/state", payload=b"state", qos=0, retain=True
)
assert publish_discovery.call_count == 5
assert session.close.call_count == 5
manager.disconnect()
+11 -11
View File
@@ -143,7 +143,7 @@ def test_unsubscribe_unknown_topic_is_noop() -> None:
# ---------------------------------------------------------------------------
def _extract_on_message(manager: MqttManager, settings) -> object:
def _extract_on_message(manager: MqttManager, settings) -> tuple[object, MagicMock]:
"""Connect with a mocked paho client and return the on_message callback that
was registered on the client mock."""
mock_client = MagicMock()
@@ -153,7 +153,7 @@ def _extract_on_message(manager: MqttManager, settings) -> object:
manager.connect(settings)
# The callback was assigned as: client.on_message = _on_message
return mock_client.on_message
return mock_client.on_message, mock_client
def test_on_message_dispatches_to_registered_handler() -> None:
@@ -168,10 +168,10 @@ def test_on_message_dispatches_to_registered_handler() -> None:
manager.subscribe("dsmr/json", handler)
on_message = _extract_on_message(manager, settings)
on_message, client = _extract_on_message(manager, settings)
msg = _make_mqtt_message("dsmr/json", b'{"id": 1}')
on_message(MagicMock(), None, msg) # simulate paho calling the callback
on_message(client, None, msg) # simulate paho calling the callback
assert received_payloads == [b'{"id": 1}']
@@ -183,11 +183,11 @@ def test_on_message_ignores_unknown_topic() -> None:
handler = MagicMock()
manager.subscribe("other/topic", handler)
on_message = _extract_on_message(manager, settings)
on_message, client = _extract_on_message(manager, settings)
msg = _make_mqtt_message("unknown/topic", b"data")
# Must not raise
on_message(MagicMock(), None, msg)
on_message(client, None, msg)
handler.assert_not_called()
@@ -201,11 +201,11 @@ def test_on_message_swallows_handler_exception() -> None:
raise RuntimeError("handler exploded")
manager.subscribe("dsmr/json", bad_handler)
on_message = _extract_on_message(manager, settings)
on_message, client = _extract_on_message(manager, settings)
msg = _make_mqtt_message("dsmr/json", b"{}")
# This must not raise
on_message(MagicMock(), None, msg)
on_message(client, None, msg)
def test_on_message_does_not_crash_on_handler_exception_multiple_calls() -> None:
@@ -221,13 +221,13 @@ def test_on_message_does_not_crash_on_handler_exception_multiple_calls() -> None
raise ValueError("first call fails")
manager.subscribe("dsmr/json", flaky_handler)
on_message = _extract_on_message(manager, settings)
on_message, client = _extract_on_message(manager, settings)
msg1 = _make_mqtt_message("dsmr/json", b"first")
msg2 = _make_mqtt_message("dsmr/json", b"second")
on_message(MagicMock(), None, msg1) # should not raise despite exception in handler
on_message(MagicMock(), None, msg2) # handler called again
on_message(client, None, msg1) # should not raise despite exception in handler
on_message(client, None, msg2) # handler called again
assert call_count[0] == 2