When I audited the CI of my own SaaS, I tested the gates before the code. One of them, a front-end coverage threshold of 95% lines, 95% functions, 90% branches and 95% statements, had never been evaluated: the CI job ran the tests without the --coverage flag. I fixed the job and set the floor to 91/93/88/91, a threshold the pipeline now actually enforces, so it fails the moment coverage drops below it. The pipeline comment now records why: Previously ran without --coverage = decorative.
That is verification debt in one line of YAML. Writing code got cheap. Knowing the code is right did not. I build MailHive with an AI-assisted workflow, so the volume of code that needs checking is high, and a check that only looks like a check is worse than no check, because you stop worrying.
What Verification Debt Actually Is
Sonar calls the distance between distrusting AI-generated code and actually checking it the verification gap, and says that gap creates what Amazon CTO Werner Vogels has called verification debt. Technical debt is a shortcut you know you took. Verification debt is code nobody can vouch for. It can be well structured, pass review at a glance and still be wrong.
Sonar's survey of more than 1,100 developers, published in January 2026, put numbers on the gap: 96% do not fully trust that AI-generated code is functionally correct, and only 48% say they always check it before committing. 38% say reviewing it takes more effort than reviewing a colleague's code. Sonar sells code quality tooling, so read those as directional.
Your Sense of Speed Is Not Evidence
METR's 2025 randomized trial is the best warning I know. 16 experienced open source developers worked on 246 real issues in repositories they knew. With AI allowed they took 19% longer, and afterwards they still believed AI had made them 20% faster. METR now calls that result out of date, and its February 2026 follow-up says the newer data is an unreliable signal, mainly because developers increasingly chose not to take part if it meant working without AI. Its newer estimates lean toward a speedup, with wide uncertainty. So do not quote the 19% as the current state of AI tools. The lasting finding is the gap between perception and measurement. Feeling fast proves nothing.
Two Findings from Auditing My Own Guardrails
Neither was exotic, and both looked fine on a normal day. I found them by auditing the guardrails themselves, not the code they protect.
- Coverage thresholds that were never evaluated. The numbers sat in vitest.config.ts and CI did not run coverage. Fixed by making the CI script run with coverage on.
- A linter that only ran locally. ESLint ran in a pre-push hook, which --no-verify skips, so no pipeline evaluated the module boundary rule or no-explicit-any. I moved it into one CI job that lints both apps.
Both looked covered. What was missing was proof that they could fail.
Coverage Tells You What Ran, Not What Was Checked
Even a working coverage gate answers the wrong question. It tells you a line executed. It does not tell you a test would notice if that line were wrong. Here is a small illustrative example, not from any real codebase, and it reaches 100% coverage:
export function isOverLimit(spent: number, limit: number): boolean {
return spent > limit;
}
it('blocks over the limit', () => expect(isOverLimit(150, 100)).toBe(true));
it('allows under the limit', () => expect(isOverLimit(50, 100)).toBe(false));
// Change > to >= and both tests still pass.
// Nothing checks spent === limit, which is the case the business cares about.Mutation testing finds exactly this. A tool like Stryker, which is open source, changes your code in small ways and reruns the tests. If the tests still pass, that mutant survived and you have found a check that is not there. On the service that holds MailHive's email state, the first run produced a precise list of surviving mutants, each one a specific change the tests would not notice. A coverage report, even at a high floor, will never give you that list, and each entry points to the test worth writing next.
Do not start by making it a merge gate. Mutation runs are slow and some surviving mutants are harmless. Mine is a manual job, which can also run on a schedule, scoped to one service, it cannot block a merge, and the score is a signal to read. Widen the scope as you harden each slice.
Test the Guards Themselves
A rule can sit in the config and be set up in a way that never reports a violation. A rule that never fails looks identical to a codebase that never breaks it. So MailHive's lint job runs a guardrail canary: a script that plants one deliberate violation per key rule and fails if ESLint does not report it. A canary covers the rules it plants, so any guard outside ESLint, like a grep for focused tests, needs its own planted violation.
Decide What Each Check Is Allowed to Do
The mistake I would warn you about most is treating an AI reviewer as a gate. MailHive has an AI review job on merge requests that posts a note with its findings, and it never blocks a merge. That is deliberate. Deterministic checks can block, because when they fail you can see why and reproduce it. A model's opinion is an extra reader, useful and sometimes sharp, and it does not get a veto or the final word.
That split is the same one I wrote about for regulated environments: instructions are a request, and the pipeline is what enforces. If you also use Copilot or Claude Code with project instructions, keep the same line. Instructions reduce the bad code that arrives. They verify nothing.
What to Do This Week
Everything below is free and works on a normal CI runner. Do it in this order, because the first step is the one that exposes the others.
- Prove each gate can fail. Open a throwaway branch, break one thing per gate (a failing test, a coverage drop, a lint error, a secret), and watch the pipeline go red. Any gate that stays green is decoration. Then automate the ones you can with a canary.
- Make sure every check that matters also runs in CI. Local hooks are a courtesy to yourself. They are skippable by design.
- Fail on focused tests. On Vitest, check that allowOnly is not on in CI. On Jest, add a grep, and plant a violation to prove the grep runs in the right directory.
- Turn on the rule you have been avoiding, using a baseline. ESLint's bulk suppressions let existing error-level violations stay, counted per file and rule, so new ones fail. You stop the debt growing today and pay it down later.
- Run mutation testing once on your riskiest module. Pick the money, permissions or state code, read the survivors, and write the tests they point to.
- Keep a small end-to-end smoke suite with Playwright for the flows that must never break. Keep it small enough that people do not start ignoring it.
- Size human review by risk, not by diff. A generated test file and a change to an authorization check do not deserve the same attention, even at the same line count.
Where Gates Stop
Even when every gate works, each one has a boundary. Mutation testing is slow, so I scope it to the riskiest slices first and widen it as they harden. An end-to-end suite that needs protected secrets often cannot run on every merge request, so decide explicitly which flows must pass before a release. And no gate can tell you a business rule is the wrong business rule. That still takes a person who knows the product.
A gate you have never seen fail is an assumption.
References
- Sonar. Verification gap press release, January 2026. https://www.sonarsource.com/company/press-releases/sonar-data-reveals-critical-verification-gap-in-ai-coding/
- METR. Early-2025 AI and experienced open source developers study. https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/
- METR. February 2026 update on the experiment design. https://metr.org/blog/2026-02-24-uplift-update/
- StrykerJS mutation testing. https://stryker-mutator.io/docs/stryker-js/introduction/
- ESLint bulk suppressions. https://eslint.org/docs/latest/use/suppressions
- Vitest allowOnly. https://vitest.dev/config/allowonly
- Vitest coverage configuration. https://vitest.dev/config/coverage