Your LLM Doesn't Need a Judge. It Needs a Linter.

Confidence: hypothesis. The gate failures below are drawn from published LLM-judge evaluation research and industry patterns in regulated document work, not firsthand deployment. The gating pattern is an argument I have not shipped at scale. Argue with it.

Somewhere in your pipeline right now, one language model is grading another. LLM-as-judge is the default quality gate of 2026: cheap to stand up, plausible in demos, and backed by a fast-growing benchmarking literature. That same literature is starting to document the problem — a 2026 large-scale study of LLM-judge agreement found that the standard scoring method used to validate these tools overstates how well they actually discriminate, once you correct for chance agreement (Norman et al., 2026). Judge models earn their keep — in the right lane. The problem is which lane teams put them in.

When the judge's score decides whether output ships — whether a drafted document section enters the record, whether a generated answer reaches a customer — you have made a model's opinion into a compliance gate. And an opinion, however well prompted, has a property that disqualifies it from that job: ask it twice and you can get two answers.

Clinical drug development — one of the most heavily audited industries on earth — shows what a quality gate has to survive. And it points to an uncomfortable conclusion: the load-bearing gate in any high-stakes LLM system should be the most boring technology you own — a deterministic linter.

What a gate has to survive

In clinical research, the document that runs a trial — the protocol — is the trial's operating contract. Regulators inspect not just the document but the process that produced it. An inspector's questions about any quality gate are always the same three: