Readiness Is Not the Same as Liveness

The load balancer calls /health. The process returns 200 as long as the HTTP server is up, including while it cannot reach the database and while it is shutting down. Traffic keeps arriving. Requests fail. Someone adds the database to the same check. The database blips, the orchestrator restarts every instance, and the blip becomes an outage.

Kubernetes documents the split. A liveness probe that fails causes the kubelet to restart the container. A readiness probe that fails removes the pod from Service endpoints so it stops receiving traffic, without a restart. Startup probes exist so a slow start does not get killed by liveness. You can use the same split on a virtual machine with a load balancer. The names are less important than the two questions: should this process be replaced, and should it receive work right now?

What each check is allowed to depend on

During deploy, mark the instance not ready before you stop it, and wait until in-flight requests finish. Readiness is that drain. Liveness is not. Killing a process that is mid-request because a probe was slow is how deploys lose work.

Illustrative scenario

This scenario is illustrative. An API needs its database to serve any route. Readiness checks a short database query and fails when that query cannot run. Liveness only checks that the process can answer at all. A database failover makes instances unready. They are not restarted. When the database returns, they become ready without a cold start. A deadlock that stops the event loop fails liveness, and that instance is replaced. A model provider used by one route does not sit in either check. That route returns a clear error. The rest of the API stays ready.

Checklist

• Separate the restart check from the traffic check.
• Keep liveness local. Do not chain it to every dependency.
• Fail readiness during startup and drain.
• Give probes a timeout longer than a normal response and shorter than you are willing to serve errors.
• Alert when readiness stays failed. A restart loop is a different alert from a drained instance.
• Do not use a public status page as the probe. Probe the instance, not the marketing URL.

Questions that usually come next

Should the ready check hit the database every second?

A cheap query on an interval of a few seconds is enough to leave a bad instance. A heavy query on a tight interval becomes load. If the check is slower than the traffic, you are measuring the check.

What if we do not use Kubernetes?

The load balancer still needs a URL that means "send work," and the supervisor still needs a signal that means "replace this process." Two URLs, or one URL with two meanings you have written down, are enough. One URL that means both will eventually do the wrong one.