Desired and Observed State Should Never Share a Field

Operator intent and device-reported reality can disagree legitimately during rollout, reboot, outage or rollback.

I keep this as a field note because the failure mode is easy to misclassify: assignment overwrites the same field later used for device-reported running state. The useful move is to identify the boundary first, then change only the layer that owns it.

What I model

The system is easier to debug when intent, observation and transport are not collapsed into one state. For this case, my rule is simple: Model intent and observation separately, then reason about convergence.

Implementation pattern

Keep desired_release_id and running_release_id independent and derive status from their relationship.

I prefer a small explicit contract over a clever implicit one. That gives logs, tests and dashboards something concrete to verify and keeps unrelated layers from compensating for each other.

What I verify

• assignment does not alter observed state
• heartbeat cannot erase intent
• UI shows pending convergence

Failure handling

When one of those checks fails, I preserve the failing evidence before restarting or changing configuration. The first broken contract determines the next investigation. That keeps troubleshooting causal instead of turning it into a sequence of guesses.

What I keep

Model intent and observation separately, then reason about convergence. The specific tools can change, but that ownership boundary remains useful across firmware, networks and infrastructure.