AI applications3 min readBy Revot
What makes a RAG assistant trustworthy
Retrieval-augmented assistants fail in predictable ways. Trust comes from retrieval quality, grounding, access control, evaluation and a clear path to a human — not from a better prompt alone.
In short
- Most wrong answers start as retrieval problems: measure retrieval before tuning the model.
- Grounding means every claim can be traced to a source the user can see.
- Access control has to apply at retrieval time, not just at login.
- Evaluate before launch and after every model or prompt change, and route uncertainty to a person.
A retrieval-augmented generation (RAG) assistant answers questions using your own documents. When someone asks a question, the system searches your knowledge base for relevant passages, then asks a language model to answer using those passages. Done well, it lets staff or customers find answers buried in policies, procedures, manuals and recordings.
Done badly, it produces fluent, confident answers that are wrong — which is worse than no assistant, because people stop checking. Trustworthiness is not a property of the model. It comes from how the whole system is designed, measured and bounded.
Start with retrieval quality
If the right passage is not retrieved, the model cannot answer correctly — at best it will say it does not know, at worst it will improvise. So the first thing to measure is not the answer but the retrieval: for a set of real questions, did the system find the passages a knowledgeable person would have used?
Retrieval quality depends on unglamorous decisions. How documents are split into chunks. Whether headings and context travel with each chunk. How tables, scanned PDFs and transcripts are handled. Whether outdated versions are removed. How results are ranked. These are worth testing directly, because tuning a prompt cannot fix a passage that never arrived.
Ground every answer in visible sources
A grounded answer is one where every claim is supported by a retrieved source. In practice that means instructing the model to answer only from the provided passages, showing citations the user can open, and checking — automatically where possible — whether the answer is actually supported by what it cites.
Citations do two jobs. They let users verify an answer that matters, and they make errors diagnosable: if an answer is wrong, you can see whether the source was wrong, outdated, or misread.
Apply access control at retrieval time
Not everyone should see every document. If permissions are only checked at login, an assistant can happily quote a passage the user was never allowed to read. Access rules — per user, per role, and per customer in multi-tenant systems — need to filter what can be retrieved in the first place. Tenant isolation deserves its own tests: can one customer’s data ever surface in another customer’s answer?
Assume documents can contain instructions
Prompt injection is the risk that text inside a document, email or web page is treated as an instruction — “ignore previous instructions and…”. Any assistant that reads content it did not write is exposed to this. Test for it with deliberately hostile content, keep the model’s permissions narrow, and do not give an assistant tools that can take consequential actions without a human approval step.
Evaluate before launch — and after every change
An evaluation set is a collection of real questions with known good answers and known correct sources. Run the assistant against it and score what matters:
- Correctness — is the answer right?
- Groundedness — is it supported by the cited sources?
- Source quality — were the best passages retrieved?
- Refusal and escalation — does it decline or hand over when it should?
- Latency and cost — is it fast and affordable enough for how it will be used?
Then keep the evaluation set and re-run it whenever the model, prompt, chunking or retrieval settings change. Model providers update models; a change that improves one kind of question can quietly break another. Versioning prompts and evaluation sets alongside the code makes these regressions visible instead of anecdotal.
Design the path to a human
A trustworthy assistant knows when not to answer. When retrieval finds nothing relevant, when sources conflict, when the question is out of scope, or when the stakes are high, it should say so and route the question to a person — with the question and what it found attached, so nobody starts from scratch.
This is also how the system improves. Escalated questions show you where documents are missing, outdated or unclear, which is often more valuable than any change to the model.
What trust actually looks like
No assistant is right every time, and any vendor who promises otherwise is describing a demo. Trust comes from a system whose errors are measured, whose answers can be checked, whose access rules hold, and whose uncertainty reaches a person. Build those in from the start and you get an assistant people can rely on — and know when not to.