Service Port Mismatches: The Quiet Networking Failure

Symptom

I parameterized service ports and used the same values in both Deployment and Service templates.

The symptom pointed at the wrong layer. A pod can be running and DNS can resolve while traffic still fails because the transport contract is misaligned.

My first rule: identify the stage of failure

controller state -> Kubernetes state -> process state -> dependency state -> product outcome

What I checked

I checked ArgoCD diff/sync, Kubernetes events, pod logs, Service endpoints, workload-specific latency or backlog before changing anything.

I traced this as controller state -> Kubernetes state -> process state -> dependency state -> product outcome and checked ArgoCD diff/sync, Kubernetes events, pod logs, Service endpoints, workload-specific latency or backlog at each handoff. That kept the debugging path concrete.

Where the problem actually was

The root cause was this: A pod can be running and DNS can resolve while traffic still fails because the transport contract is misaligned. The practical lesson was simple: Trace the exact port path end to end instead of assuming matching numbers.

Prevention checklist

• Use commit history to preserve incident context.
• Instrument the workload-specific failure mode, not just CPU and memory.
• Classify the failing layer before changing anything.
• Capture events and logs before restarting.
• Keep deployment health separate from product health.