Primitive of the day · 2026-10-15

gbnf-performance-rules

In GBNF, chained optionals (x? x? x?) explode the FSM — use bounded repetition x{0,N} (max 2000 repetitions per rule in llama.cpp), prefer negated character ranges over enumerations, and keep grammars shallow.

When this applies

Hand-writing or debugging a GBNF grammar for llama.cpp constrained decoding.

Preconditions

• The grammar is authored by hand in GBNF
• llama.cpp's MAX_REPETITION_THRESHOLD of 2000 is respected as a ceiling

Do this

Replace repeated optionals with x{0,N}; keep per-rule repetitions at or under 2000; write negated ranges like [^\n] instead of enumerating characters; flatten deep rule nesting.

What you should see

The grammar compiles and samples without pathological slowdowns.

How it fails if ignored

Exponential FSM blowup from repeated optionals; hitting the repetition threshold; deep token stacks degrading sampling throughput.

Do not use when

Using json_schema auto-conversion, where the converter shapes the grammar for you.

Kind: gotcha-fix. Part of the skill GBNF lazy grammar CoT constrained. Free to reuse in your own agent skills.