RAG Is a Governance Problem: Access Rights, Provenance and Citation Checks
In a bank or an insurer, most honesty and data protection failures of a RAG system happen in retrieval, not in the model. Here is how I govern that step.
When a client asks me to review a retrieval-augmented generation system, I spend the first hour on the model and the rest of the week on retrieval. Nearly every incident I have seen in regulated companies had the same shape: the model behaved as designed, the problem was the material it was fed. A chunk from a policy withdrawn two years ago. A passage from a file the user was never cleared to read. A confident answer with no source anyone could open. None of that is a model failure. It is an AI governance failure in the retrieval layer.
Why RAG failures are an AI governance problem
A language model does not know your rules. It knows the text in its context window. Whatever lands there, restricted, outdated or irrelevant, it will work with, and it will sound sure. People call that hallucination. The honest diagnosis is that the system handed the model bad material.
That is why retrieval is a component in our governance layer like any model: registered, monitored, with a defined response when it misbehaves. Control before model. You can swap the model next year. The retrieval rules, the provenance and the evaluation sets stay.
Access rights are enforced at retrieval time, or not at all
The rule I give every team is simple: the model must never see what the user may not see. Not filtered afterwards, not hidden by a prompt instruction, but excluded before the query reaches the vector store. In practice this means three things.
Inherited permissions. Every chunk carries the access list of its source document, copied at indexing time and refreshed whenever the source changes.
Identity in the query. The search runs as the asking user, and the permission filter is applied inside the search, not on the result list.
Logged exposure. Every chunk that reaches the context window is logged with user ID, document ID, version and the rule that allowed it.
Post-filtering fails quietly. Retrieve fifty chunks and drop the restricted ones afterwards, and the ranking has already been shaped by documents the user may not see, while one bug in the filter leaks all of them. Pre-filtering costs recall and some work on the index. In a bank, that trade is not a discussion.
Provenance travels with every chunk
A chunk without provenance is a rumour. For every piece of text we index, these fields are mandatory.
Source. System, document identifier and the path an auditor can open.
Version and hash. So that two people reading the same answer six months apart can prove whether the document changed.
Effective date and expiry. When the rule applied, and when it stopped applying.
Owner. The business unit responsible, because someone has to answer when the document is wrong.
Classification. Public, internal, confidential, or personal data.
This metadata is not decoration. The citation check, the outdated-document rule and the audit trail all run on it. Missing metadata means the chunk does not enter the index. No provenance, no retrieval, just as no registration means no deployment.
Citation checks: does the passage carry the claim?
Models cite fluently, and that is exactly the problem. A footnote looks like evidence even when the quoted passage says something else. So every answer in a regulated process passes two checks.
The automated check splits the answer into claims and asks, for each one, whether the cited chunk actually supports it. A second model does this, with a different prompt and no access to the conversation, and its output is a score, not a verdict. Claims below the threshold are marked unverified. The whole answer is held back when more than a set share of claims fail, or when a failing claim touches an amount, a deadline or a legal obligation.
The sampled human review is the part people try to skip. A fixed share of answers per week, weighted towards low scores and new sources, goes to a reviewer from the unit that owns the documents. They record whether the citation supports the claim, whether it is the right document, and whether they would have answered differently. The disagreement rate between judge model and reviewers is monitored. When it drifts, the retrieval owner gets a ticket. When it crosses a second threshold, the use case is paused.
The principle behind this: a RAG answer without a checkable source is unverified by default. Not wrong, unverified. The interface says so, and no downstream process may treat it as a fact.
Outdated policies and the designed “I do not know”
Policy documents in banks and insurers have versions, transitional periods and successors. The index has to know that. A chunk whose validity has ended is excluded unless the question explicitly asks about a historical state, and even then the citation carries the validity period. Documents past their review date with no successor trigger a notice to the owner. An expired rule is never silently quoted as current.
And when retrieval returns nothing above the relevance threshold, or only chunks the citation check rejects, the correct output is: I do not know, here is who to ask. I design that outcome, test for it and count it. A system that never says it does not know is not well informed. It is badly controlled.
Pseudonymised retrieval for personal data
Many of the most valuable internal questions touch personal data: a claims file, a complaint, a loan history. We handle retrieval the way we handle training: the model works with IDs, not names. Identifiers are replaced by pseudonyms before indexing, and the mapping lives in a separate system that neither the model nor the vector store can reach. The answer comes back with IDs, and the application resolves them only for a user authorised to see the person behind them, and logs that resolution.
This maps onto the red, gray and green zones I use with clients. Raw personal data in a general-purpose index is red. Pseudonymised retrieval with access control, logged resolution and a documented purpose is gray turning green, and the only setup I would defend in front of a data protection officer and a supervisor at once. The EU AI Act, in force since August 2024, phases in its obligations over the coming years, with data governance and traceability at their centre. Provenance per chunk and pseudonymised retrieval are concrete building blocks for that.
Evaluation sets built from real internal questions
Public benchmarks tell you how a model performs on someone else’s documents. Your supervisor cares about yours. So we build the evaluation set from questions employees actually asked, taken from the logs and cleaned of personal data, plus the hard cases the business unit brings in: answers that changed last year, documents that contradict each other, questions no document answers.
Each question has an expected source, an expected answer or an expected “I do not know”, and an expected access outcome for at least two user roles. The set runs on every change to the index, prompts, embedding model or generator. Release is blocked when access violations are not zero or citation support drops below the previous run. Every reviewer disagreement that turns out to be a system error becomes a new test case.
Open questions
Oversight at scale. Sampled human review works at hundreds of answers a day. At hundreds of thousands, which answers deserve a human, and how do we know the sample is not blind to exactly the failures that matter?
Robustness of the judge. The model that checks citations is itself a model. How strongly do its errors correlate with the generator’s, and how do we measure that without a third model?
Interpretability of retrieval. A similarity score is not a reason. Can we give an auditor a legible account of why one chunk was retrieved and another was not?
Honesty under pressure. When business users penalise “I do not know”, how do we stop that feedback, through prompts, fine-tuning or ranking, from training abstention out of the system?
Frequently asked questions
Is it enough to filter retrieved documents by permission after the search?
No. The filter has to be part of the query so that restricted documents never influence the ranking and never enter the context window. Post-filtering depends on one piece of code never failing, which is not an acceptable control.
Who checks the citations in a RAG system?
Two layers: an automated judge that scores whether each cited passage supports the claim, and a weekly sample reviewed by people from the unit that owns the documents. Rising disagreement between the two pauses the use case.
What happens when the system finds no reliable source?
It says so and names a human contact. Any answer without a source the user can open is shown as unverified and may not serve as the basis for a decision.
RAG Is a Governance Problem: Access Rights, Provenance and Citation Checks
In a bank or an insurer, most honesty and data protection failures of a RAG system happen in retrieval, not in the model. Here is how I govern that step.
When a client asks me to review a retrieval-augmented generation system, I spend the first hour on the model and the rest of the week on retrieval. Nearly every incident I have seen in regulated companies had the same shape: the model behaved as designed, the problem was the material it was fed. A chunk from a policy withdrawn two years ago. A passage from a file the user was never cleared to read. A confident answer with no source anyone could open. None of that is a model failure. It is an AI governance failure in the retrieval layer.
Why RAG failures are an AI governance problem
A language model does not know your rules. It knows the text in its context window. Whatever lands there, restricted, outdated or irrelevant, it will work with, and it will sound sure. People call that hallucination. The honest diagnosis is that the system handed the model bad material.
That is why retrieval is a component in our governance layer like any model: registered, monitored, with a defined response when it misbehaves. Control before model. You can swap the model next year. The retrieval rules, the provenance and the evaluation sets stay.
Access rights are enforced at retrieval time, or not at all
The rule I give every team is simple: the model must never see what the user may not see. Not filtered afterwards, not hidden by a prompt instruction, but excluded before the query reaches the vector store. In practice this means three things.
Post-filtering fails quietly. Retrieve fifty chunks and drop the restricted ones afterwards, and the ranking has already been shaped by documents the user may not see, while one bug in the filter leaks all of them. Pre-filtering costs recall and some work on the index. In a bank, that trade is not a discussion.
Provenance travels with every chunk
A chunk without provenance is a rumour. For every piece of text we index, these fields are mandatory.
This metadata is not decoration. The citation check, the outdated-document rule and the audit trail all run on it. Missing metadata means the chunk does not enter the index. No provenance, no retrieval, just as no registration means no deployment.
Citation checks: does the passage carry the claim?
Models cite fluently, and that is exactly the problem. A footnote looks like evidence even when the quoted passage says something else. So every answer in a regulated process passes two checks.
The automated check splits the answer into claims and asks, for each one, whether the cited chunk actually supports it. A second model does this, with a different prompt and no access to the conversation, and its output is a score, not a verdict. Claims below the threshold are marked unverified. The whole answer is held back when more than a set share of claims fail, or when a failing claim touches an amount, a deadline or a legal obligation.
The sampled human review is the part people try to skip. A fixed share of answers per week, weighted towards low scores and new sources, goes to a reviewer from the unit that owns the documents. They record whether the citation supports the claim, whether it is the right document, and whether they would have answered differently. The disagreement rate between judge model and reviewers is monitored. When it drifts, the retrieval owner gets a ticket. When it crosses a second threshold, the use case is paused.
The principle behind this: a RAG answer without a checkable source is unverified by default. Not wrong, unverified. The interface says so, and no downstream process may treat it as a fact.
Outdated policies and the designed “I do not know”
Policy documents in banks and insurers have versions, transitional periods and successors. The index has to know that. A chunk whose validity has ended is excluded unless the question explicitly asks about a historical state, and even then the citation carries the validity period. Documents past their review date with no successor trigger a notice to the owner. An expired rule is never silently quoted as current.
And when retrieval returns nothing above the relevance threshold, or only chunks the citation check rejects, the correct output is: I do not know, here is who to ask. I design that outcome, test for it and count it. A system that never says it does not know is not well informed. It is badly controlled.
Pseudonymised retrieval for personal data
Many of the most valuable internal questions touch personal data: a claims file, a complaint, a loan history. We handle retrieval the way we handle training: the model works with IDs, not names. Identifiers are replaced by pseudonyms before indexing, and the mapping lives in a separate system that neither the model nor the vector store can reach. The answer comes back with IDs, and the application resolves them only for a user authorised to see the person behind them, and logs that resolution.
This maps onto the red, gray and green zones I use with clients. Raw personal data in a general-purpose index is red. Pseudonymised retrieval with access control, logged resolution and a documented purpose is gray turning green, and the only setup I would defend in front of a data protection officer and a supervisor at once. The EU AI Act, in force since August 2024, phases in its obligations over the coming years, with data governance and traceability at their centre. Provenance per chunk and pseudonymised retrieval are concrete building blocks for that.
Evaluation sets built from real internal questions
Public benchmarks tell you how a model performs on someone else’s documents. Your supervisor cares about yours. So we build the evaluation set from questions employees actually asked, taken from the logs and cleaned of personal data, plus the hard cases the business unit brings in: answers that changed last year, documents that contradict each other, questions no document answers.
Each question has an expected source, an expected answer or an expected “I do not know”, and an expected access outcome for at least two user roles. The set runs on every change to the index, prompts, embedding model or generator. Release is blocked when access violations are not zero or citation support drops below the previous run. Every reviewer disagreement that turns out to be a system error becomes a new test case.
Open questions
Frequently asked questions
Is it enough to filter retrieved documents by permission after the search?
No. The filter has to be part of the query so that restricted documents never influence the ranking and never enter the context window. Post-filtering depends on one piece of code never failing, which is not an acceptable control.
Who checks the citations in a RAG system?
Two layers: an automated judge that scores whether each cited passage supports the claim, and a weekly sample reviewed by people from the unit that owns the documents. Rising disagreement between the two pauses the use case.
What happens when the system finds no reliable source?
It says so and names a human contact. Any answer without a source the user can open is shown as unverified and may not serve as the basis for a decision.
Related: AI Governance · EU AI Act · AI Consultant
Recent Posts
Recent Comments