The short answer
A rule-based chatbot matches keywords and walks the customer down a flow somebody drew in advance, so it only handles questions that were anticipated. An AI agent retrieves the relevant passage from your documents and composes an answer, so it handles phrasings nobody anticipated - and refuses when your documents do not cover the question.
Two products both call themselves chatbots. One shows the customer four buttons and moves them through a flow somebody drew last quarter. The other reads the question, searches your documents, and answers in the customer's own words. In a fifteen-minute demo they look similar. In production they behave nothing alike, and the gap shows up exactly when a customer asks something nobody anticipated.
How a rule-based chatbot actually works
A decision-tree bot is a flowchart with a chat interface. Someone - usually you - draws the branches: if the message contains 'refund', show the refund node; if the customer picks button two, go to the shipping node; otherwise show the fallback. Every path a customer can take is a path a human drew in advance.
This is not a criticism of the technology. Flows are precise, auditable and completely predictable, and for a transactional journey - collect an order number, book a slot, confirm an address - they are the right tool. The problem is that a support inbox is not a transaction. It is a stream of open-ended questions, and a flowchart can only answer questions somebody predicted.
Where the flow breaks
The failure is always the same and it is always the customer's fault, structurally speaking: they phrased it wrong. Your flow fires on 'refund'; the customer typed 'paisa wapas'. Your flow has a shipping branch; the customer asked whether the shipping delay applies to their pincode during the sale, which is two branches crossed. Your flow has a fallback; the customer hits it and leaves.
The maintenance answer vendors offer is to add more triggers. Add 'money back'. Add 'return karna hai'. Add the Tamil phrasing. This works, in the sense that a bucket with more patches holds more water, and it is a permanent job for someone. For a mixed-language inbox it is not a job anyone finishes - we wrote about why.
A decision tree can only answer the questions someone thought of first. Your customers are not consulting that list.
How an AI agent works instead
A grounded agent inverts the work. Instead of you enumerating questions, you upload the documents that contain your answers. When a message arrives, the system finds the passages most relevant to what was actually asked - matching on meaning, not on shared words - and composes a reply from those passages, citing the file it used.
That means 'return kaise hoga', 'can I send this back', and 'what is your return policy' all land on the same paragraph of your returns document, without anyone writing three triggers. And it means a question that combines two topics gets an answer that combines two passages, rather than falling into a fallback because it did not fit a branch.
The mechanism has a name - retrieval-augmented generation - and it is worth understanding before you buy anything, because it is the piece that decides whether the bot can lie to your customers. The plain-language version is here.
Side by side, on the things that matter
| Rule-based chatbot | Grounded AI agent | |
|---|---|---|
| Setup work | Draw every flow and branch by hand | Upload the documents you already have |
| Unanticipated phrasing | Falls back to a menu | Matches on meaning, answers anyway |
| Mixed languages | One trigger per phrasing, per language | One document, all phrasings |
| Ongoing maintenance | Editing flows as policies change | Editing documents as policies change |
| When it doesn't know | Generic fallback or a loop | Says so and escalates with context |
| Auditing an answer | Trace the branch | Read the cited document |
| Good at | Fixed transactional journeys | Open-ended questions from documents |
The third category: the ungrounded LLM
There is a middle option that is worse than both, and it is increasingly common because it is easy to ship: a raw language model with a prompt saying 'you are a helpful support assistant for Acme'. No retrieval, no documents, no citations.
This demos brilliantly. It handles any phrasing, in any language, with warmth and fluency. It also invents your return policy, because it has never seen your return policy, and it does so in confident, well-structured prose that customers believe. A rules bot that fails is annoying. A fluent bot that fabricates a delivery promise is a liability, and you find out when a customer holds you to it.
The distinguishing test takes thirty seconds: ask about something your documents genuinely do not cover. An agent says it does not know and offers a person. An ungrounded model writes you a policy on the spot.
What each one costs you to keep running
Purchase price is the smaller number. The ongoing cost is where these diverge.
A flow-based bot costs you maintenance in proportion to how varied your customers are. Every new product, every new phrasing, every language creates work in the builder, and that work has no natural end. Teams typically keep it current for about a quarter and then let it drift, at which point the fallback rate climbs and nobody notices because nobody is reading fallback transcripts.
A grounded agent costs you document hygiene: keeping your policies accurate and current, which is work you should be doing anyway. When a fact changes, you edit one file. The knowledge-base guide covers what that discipline looks like in practice - it is about twenty minutes a month for most businesses.
Four questions to ask any vendor
The words on the pricing page have converged - everyone says AI now. These four separate the categories quickly.
- Show me the setup. If the answer is a flow builder with branches, it is a rules bot regardless of what the marketing says.
- Ask it something phrased unusually. A rules bot menus you; an agent answers.
- Ask where an answer came from. If it cannot name a document, nothing was retrieved.
- Ask about something outside your documents. Honest refusal is the correct behaviour; a confident invented answer is the failure mode you cannot afford.
Where this lands
Flows are not obsolete - they are just a narrower tool than they were sold as. If your problem is a fixed journey with valid states, build the flow. If your problem is that customers ask you things all day and the answers already exist in documents you have written, a flowchart is the wrong shape for it, and no amount of trigger words will fix that.
If you want to see the difference on your own content rather than take our word for it, the homepage demo retrieves from whatever text you paste into it, so you can see how it handles your own wording rather than a scripted example.
Frequently asked questions
Is a chatbot the same as an AI agent?
No. A chatbot follows rules a human authored - keyword triggers and decision trees. An AI agent retrieves relevant passages from your knowledge base and generates an answer from them, which means it handles questions nobody wrote a rule for.
Do I still need to build flows with an AI agent?
No. There is no flow builder step. You upload the documents that contain your answers, set the tone, and paste one script tag. Maintenance is editing documents, not maintaining branches.
Are decision-tree bots ever the right choice?
Yes - for transactional journeys with a fixed set of valid states, like collecting an order number or booking a slot. They are the wrong tool for open-ended questions, which is most of a support inbox.
How do I test whether a vendor is really an AI agent?
Ask it a question phrased in a way nobody would have anticipated, then ask where the answer came from. A rules bot falls back to a menu; a grounded agent names the document. Then ask something your documents do not cover - it should decline rather than improvise.
