On this page6
Anyone who has tried an off-the-shelf AI chatbot on a real business site has seen it: a confident answer about a refund policy that doesn’t exist. The cause is simple. A language model answers from what it learned in training, and it learned nothing about your company. Retrieval-augmented generation, usually shortened to RAG, is the standard fix.
What RAG actually does
Instead of asking the model a question cold, the system first looks up the most relevant passages from your own content, then hands those to the model along with the question. The model is told to answer from that material. It’s the difference between asking a stranger and asking someone who has the handbook open in front of them.
The pieces behind it
- Ingestion. Your help articles, product pages or PDFs are pulled in and split into sensible chunks.
- Embeddings. Each chunk is turned into numbers that capture its meaning, and stored in a vector database.
- Retrieval. When someone asks a question, the closest chunks are found by meaning, not by exact keywords.
- Answering. The model writes a reply from those chunks, ideally with links back to the sources.
Where projects go wrong
Most poor chatbots fail at step one or two. Chunks that cut a sentence in half, outdated pages left in the index, or three versions of the same policy all produce muddled answers. The language model usually gets the blame, but the content pipeline is often the real problem.
The other common mistake is skipping testing. Before launch, collect real questions from your support inbox and check the answers against them. Keep that list and rerun it every time you change something.
Teach it to say “I don’t know”
A good assistant has a defined way out. If nothing relevant is retrieved, it should say so and offer to pass the conversation to a person, not improvise. That one behaviour does more for trust than any amount of clever prompting.
Does RAG remove hallucinations?
No. It reduces them a lot, particularly when you require citations and keep the scope narrow. It won’t make a system perfect, and anyone who promises that is selling something.
RAG or fine-tuning?
For most business cases, start with RAG. Your content changes, and with RAG an updated help page shows up in answers right away. Fine-tuning bakes knowledge into the model and needs retraining when things change. It has its uses, mostly around tone and format, but it’s rarely the first thing to reach for.
If you want a support or knowledge assistant grounded in your own content, send us a sample of your documentation and the ten questions people ask most. We’ll tell you what it would take.
You can read more about our Generative AI & LLM Solutions service on the main site.
