Grounding an LLM in a Codebook: RAG for Regulated Work

When the cost of a wrong answer is a failed inspection or an unsafe install, "mostly right" is not a product. This is how we approach grounding models in standards documents.

Why the naive approach fails

The instinct is to dump the document into a vector store, retrieve the top few chunks, and let the model answer. That works well enough for a demo and badly enough in production to be dangerous.

Three reasons:

Codebooks are cross-referential. An article means little without its exceptions, and the exceptions live somewhere else entirely. Retrieve the article alone and you get a confidently incomplete answer.

Tables are not prose. A naive chunker turns an ampacity table into unreadable soup. The model then invents plausible numbers, which is the worst failure mode available.

Questions do not match document language. A tech asks "what wire for a 200 amp service." The document says "conductor ampacity." Pure semantic similarity gets you close, not correct.

What we do instead

Chunk on document structure, not token count. Articles, sections, and tables are the natural units. A chunk that splits a table in half is a bug.

Keep tables intact and separately addressable. Tables get their own representation and are retrieved whole. If a table is the answer, the model should be reading the actual table.

Carry the reference through the entire pipeline. Every chunk knows where it came from, and that provenance survives into the answer. The user should always be able to check the work.

Retrieve the neighbourhood. Pull the surrounding context — the parent section, the referenced exceptions — not just the single best-matching chunk.

Teaching the model to refuse

This is the part most teams skip, and the part professionals notice immediately.

A grounded system must be willing to say: I do not have a basis for that. Three cases matter.

• Out of scope. The question is not about this domain. Say so.
• Retrieved nothing relevant. Say that too, rather than falling back on memory.
• The answer depends on something only the user can see. This is the big one for field work. Equipment-specific values are not in any standard. The correct answer names what to go measure.

That last case is why our HVAC assistant ends charging answers by pointing at the unit's data plate. The honest answer is a procedure, not a number.

Evaluating it

Retrieval metrics are necessary and insufficient. What we care about:

• Citation accuracy — does the referenced article genuinely support the claim? This needs review by someone who knows the trade. There is no shortcut.
• Refusal rate on unanswerable questions — deliberately ask things the corpus cannot support and confirm the system declines.
• Numeric fidelity — anything read from a table must match the table exactly.

We treat a confidently wrong citation as a severity-one defect. It is worse than no answer, because it manufactures false trust.

The takeaway

Grounding is not a retrieval feature you bolt on. It is a product stance: the system is accountable for every claim, and where it cannot be accountable, it says so.

That stance is the difference between a tool professionals adopt and a novelty they try once.