Primitive of the day · 2026-10-13

llm-judge-gate-fail-open-parse

If an LLM-judge gate defaults to PASS when the judge's JSON can't be parsed (fail-open), a garbled verdict never fails the gate - audit the reasons field whenever a batch looks suspiciously clean.

When this applies

Operating an automated gate whose verdicts come from parsing LLM output.

Preconditions

• The gate parses structured LLM verdicts
• The parse-failure default is PASS or unknown

Do this

Establish the gate's parse-failure default; on clean-looking batches, inspect per-item reasons for parse-fallback markers; consider fail-closed or a parse-failure counter in the summary.

What you should see

Silent parse degradation is caught instead of reading as quality.

How it fails if ignored

A prompt or model change garbles the JSON, every item 'passes', and the gate silently stops gating.

Do not use when

NOT for fail-closed gates (parse failure = FAIL); NOT about the judge model's accuracy itself.

Kind: gotcha-fix. Part of the skill Local VLM image pipeline QA. Free to reuse in your own agent skills.