A Green Build Must Mean the Checks Ran

The pipeline is green. The test job says "no tests found" and exits zero, because the test project was renamed and the script still looks for the old path. The branch protection requires the pipeline. It does not require that a test ran. The merge is allowed. The phrase "CI passed" is now false in the only sense that matters.

Jez Humble’s description of continuous delivery treats the deployment pipeline as the mechanism that proves a change is releasable, every time, from version control. That proof is only as strict as the jobs. A job that swallows a missing tool, a job marked optional, or a status check that is not required, is a way around the proof. Fail closed: if the check cannot run, the build fails.

What a required check has to guarantee

• The commands ran. A count of zero tests is a failure if the repository is supposed to contain tests.
• The tools exist in the pipeline image. A linter that is not installed must not be reported as clean.
• The same commands a reviewer would trust locally are the ones in the pipeline, or the pipeline is stricter.
• Branch protection lists those jobs. A green job that is not required does not gate the merge.
• Flaky failures are fixed or quarantined explicitly. Ignoring a red job by rerunning until it is green hides the flake.
• Secrets for the pipeline are not printed in the log when a command fails.

Generated code does not change the rule. AI Code Review Gates adds human review where judgment matters. The pipeline still has to run. A bypass for "small" generated edits is how an untested change becomes the exception that is always used.

Illustrative scenario

This scenario is illustrative. The pipeline installs dependencies, runs the test command, and fails if the test runner reports zero tests or a non-zero exit. A lint job fails if the binary is missing. Branch protection requires both jobs. A pull request cannot merge on a pipeline that was skipped by a label. When a test is quarantined, the quarantine is a named list in the repo with an owner, not a click in the UI that only one person remembers. The next red build is a stop, not a suggestion.

Checklist

• Fail the build when tests do not run.
• Fail the build when a required tool is absent.
• Require those jobs on the protected branch.
• Keep the pipeline definition in the repo, reviewed like code.
• Quarantine flakes explicitly and with an expiry.
• Do not grant a bypass for urgency. Urgency is when a skip hurts most.

Questions that usually come next

Should every repository block merges on day one?

Block on the checks you have. A repository with no tests should fail a job that claims to run tests, and it should add a small set that covers the path you fear breaking. An optional pipeline teaches people to ignore it.

Are manual test passes a substitute?

A person can check what automation cannot see yet. That pass should be recorded. It should not replace a compile and a test run that a script can do every time. If the manual step is the only step, the green build is a mood.