Verified Citations and Abstention in RAG

A RAG system that cannot say 'I don't know' is a liability, not a feature. Inside a US real-estate private-equity fund with roughly $3B in assets under management, I shipped one that verifies every citation and abstains when the evidence is thin, because a confident wrong answer near the books costs more than a slow one.

The lesson landed the day a demo hallucinated in front of a compliance officer. The assistant answered a question about a fund document, cited a source, and sounded completely sure. The citation was real. It just did not say what the answer claimed it said.

The room went quiet, and the person whose whole job is to catch that kind of error had just watched a machine produce it with a straight face. So I stopped optimizing for coverage and started optimizing for something narrower and harder: never be confidently wrong. This is the design that came out of it, and why I would build it the same way again.

What a verified citation actually means

Start with the word everyone uses loosely. A citation, in most RAG systems, is a link the model placed next to its answer. Nobody checked that the link supports the claim. It looks like evidence, which is worse than no evidence, because it buys trust it did not earn.

The pipeline: retrieve, cite, verify, abstain

The system runs the same four steps on every question, and the last one is a fork, not a finish line.

This ran as the shared answer layer underneath the fund's 11-agent platform, so every agent that read a document inherited the same citation checks instead of reinventing them.

When the system is allowed to refuse

Abstention is a policy, not a mood. The system does not refuse because it feels unsure. It refuses when the evidence is in a specific state, and those states are written down.

Here is the honest version of what a vendor bot does versus what a system allowed near the books does, by the state of the evidence.

How you verify a citation, not just attach one

Attaching a citation is easy. Verifying it is the actual work, and it is two checks, not one.

The first is entailment, or NLI: given the claim and the cited span, does the span support the claim, contradict it, or neither? A 'neither' is not good enough. Neutral means unsupported, and unsupported routes to abstention.

The second is numbers. Language models are fluent with figures and wrong with them. If a claim states an amount, a date, or a count, it has to match the source exactly, not 'about right'. In finance, 'about right' is how you book a wrong number with confidence.

type Verdict = 'supported' | 'contradicted' | 'insufficient'

// A claim ships only if the cited span survives BOTH checks.
async function verifyClaim(claim: Claim, span: SourceSpan): Promise<Verdict> {
  // 1. Entailment (NLI): does the source actually support the
  //    sentence, or does it just sit near it and look relevant?
  const relation = await nliCheck(claim.text, span.text)
  if (relation === 'contradiction') return 'contradicted'
  if (relation === 'neutral') return 'insufficient'

  // 2. Numbers: a figure in the claim must match the source to the
  //    digit. "Close" is a fabrication with good manners.
  if (claim.hasNumber && !numbersMatch(claim, span)) {
    return 'contradicted'
  }

  return 'supported'
}

// One failing claim abstains the WHOLE answer. No partial credit.
function decide(verdicts: Verdict[]): 'answer' | 'abstain' {
  return verdicts.every((v) => v === 'supported') ? 'answer' : 'abstain'
}

Two lines carry the whole design. The entailment call is what separates a verified citation from a decorative one. The numeric check is what keeps a fluent paragraph from quietly changing a figure.

And the decision function is deliberately unforgiving. One unsupported claim abstains the entire answer. There is no 'answer the easy part and hedge the rest', because a half-answer with one confident error is the exact failure the compliance officer was paid to catch.

The failure modes that survive, and how we scored them

None of this is trustworthy because I say so. It is trustworthy because it was evaluated and red-teamed before it went near a real question, and the evaluation scored the things that actually fail.

I am not going to quote you a single accuracy number, and you should be suspicious of anyone who does. The useful artifact is the rubric: what we measured, and why each dimension gates a release.

The version of this rubric that decides whether a release ships, thresholds and all, is the eval suite that turns abstention into a release gate. This post is the mechanism; that one is the verdict.

Why regulated finance is the proving ground

You could ask why bother, when most RAG assistants just answer and move on. The answer is the setting. Regulated finance is the proving ground precisely because the cost of a confident wrong answer is not embarrassment, it is a finding in an audit.

That constraint is a gift. It forces the honest version of the system, the one that would survive anywhere the cost of being wrong is real.

If you have to prove that accuracy to a regulator rather than assert it, the abstention log and the citation checks are the evidence, which is the same argument I make about evidencing accuracy under the EU AI Act.

FAQ