How to build an AI agent that answers from your own documents (RAG) in n8n

An AI agent knows a lot about everything, but nothing about your business: your prices, your shipping terms or how your service works. When someone asks, it either doesn't know or, worse, makes something up. The fix has a slightly technical name, RAG, but the idea is simple: before answering, the agent searches your documents for the part that covers the question and answers from that.

In this guide we build it in n8n, with no coding, by connecting blocks on screen.

How it works, in plain words

Think of a library with a good librarian:

You set up the library: you take your documents, split them into small chunks and store each chunk so it can be found later by meaning, not just exact words. That way of storing text is called embeddings, and the storage is a vector database.
Someone asks a question: the agent looks up the chunks closest to the question, reads them and answers using that information.

So in n8n you'll have two parts: a workflow that loads the documents and another with the agent that searches them.

What you need

n8n up and running (if you don't have it yet, see how to install n8n for free with Docker).
An API key from OpenAI or another provider, used for both the chat model and the embeddings.
Your documents in a simple format: PDF, plain text or a Google Doc. Ideally tidy and up to date.

Step 1: load your documents

Create a new workflow. The easiest start is On form submission (an n8n form with a file field) or a manual trigger followed by Google Drive in Download mode to fetch a specific file.

Then add a vector store node. For testing, use Simple Vector Store in Insert Documents mode: it keeps the data in n8n's memory, with nothing else to install. Give it a memory key, for example business_docs, which you'll use later to search it.

Step 2: chunking and embeddings

Two pieces plug into the vector store node:

Embeddings: add Embeddings OpenAI with your credential. This turns each chunk of text into something that can be searched by meaning.
Document: add Default Data Loader. Set the data type to binary (the file) and, in the Text Splitter slot, add Recursive Character Text Splitter. A chunk size of a few hundred to a thousand characters, with some overlap, is a good place to start.

Run the workflow and upload your document. If it works, the node shows how many chunks were stored.

Step 3: the agent that searches

Create another workflow (or a second branch) with a Chat Trigger and an AI Agent node, just like in the guide to your first AI agent in n8n: a chat model plus Simple Memory.

In the agent's Tool slot, add Simple Vector Store again, this time in tool mode (Retrieve Documents as Tool for AI Agent), with the same memory key (business_docs) and the same embeddings model. Give it a clear name and description, for example:

Name: business_docs
Description: Searches the business's official documents
(prices, terms, shipping, opening hours and FAQs).
Always use it before answering any question about the business.

Step 4: instructions that prevent made-up answers

In the agent's System Message, make it clear it can only answer from what it finds:

You are the assistant for [BUSINESS NAME].
Before answering anything about the business, always search
the "business_docs" tool.
Only answer with information found in the documents.
If it isn't there, say so clearly and offer to have the team get in touch.
Never make up prices, timelines or terms.
Reply in the user's language, in 2-4 sentences.

To fine-tune the tone and tricky cases, the customer service AI agent prompt has a fuller template.

Step 5: test it and go live

Open the chat and ask things that are in the document, things phrased differently and things that aren't there. In the agent's panel you can see which chunks it retrieved for each answer, so you'll understand why it replied the way it did.

Simple Vector Store is for testing only: the data is lost if n8n restarts. Once you're happy, swap both nodes for a permanent vector store that n8n supports, such as Supabase, Qdrant, Pinecone or Postgres with PGVector, and load the documents again.

Then connect the agent to the channel you use: your website, WhatsApp or Telegram.

Common mistakes

It answers without searching. Strengthen the tool description and mention the tool by name in the instructions.
It finds nothing. Almost always the loading and searching sides use a different memory key or a different embeddings model. They must match.
Incomplete answers. Chunks may be too small, or too few are retrieved. Try slightly larger chunks or ask the tool for more results.
Outdated information. If you change a price, reload the document. The easiest setup is a single "official" document you reload whenever it changes.
Messy documents. A PDF full of tables or scanned pages reads badly. Clear text with headings and one question per section works much better.

Shortcut: agents that are already built

If you'd rather start with something that already works, the AgentKiosk pack includes agents ready to import into n8n, with their instructions written and rules to avoid making things up. And if you'd like to try for free, download the WhatsApp agent and take a look at 7 AI automations for small businesses.