TL;DR
A customer agent on WhatsApp or web chat will meet questions it cannot answer from your records, and a fluent guess can commit your business to something nobody approved. Asking the model to judge its own confidence does not fix that. Decide the stop points before launch, check them in code on every message, and store who owns each conversation so the agent stays silent while a person has it. Then staff the queue, and review a weekly sample to tune the stops.
In this article
8 min read
Decide where the agent must stop before it goes live, and enforce those stops in the code around the model rather than in its instructions. Some stops are fixed by topic, some fire when a lookup fails or the customer asks for a person, and the model’s own sense of uncertainty is at most one signal among them. Once a stop fires, the conversation belongs to a named person until they hand it back, and the agent checks that before every reply.
Three public questions show the two halves of the problem. In January 2026, a small business owner on the Make community forum asked how to stop a WhatsApp assistant when personal data is needed or the client asks for a person, let a secretary continue the same chat, and bring the AI back after a day of silence. They had tried filters and flags and found no clean way to store whether the AI or the human was in charge. Two years earlier, on the OpenAI developer forum, a builder asked how to switch a session to human agents when the assistant cannot answer from its knowledge base, without waiting for the customer to ask. And in August 2026, a post on Reddit’s r/LLMDevs forum asked where a small-business support agent should be forced to stop, proposing hard stops by risk, reversibility and data reliability instead of the model’s confidence.
A model’s confidence is the weakest stop signal you have
The intuitive design asks the model to escalate when it is unsure. The research gives three reasons not to rely on that. OpenAI’s GPT-4 technical report (March 2023) found the pre-trained model well calibrated on a multiple-choice benchmark and reported that post-training “hurts calibration significantly”. Xiong and colleagues (ICLR 2024, arXiv June 2023) found that models asked to state their confidence in words tend to be overconfident, and that every method they tested struggled on questions needing professional knowledge. Kalai and colleagues at OpenAI (September 2025) argue that models guess because training and evaluation reward a guess over “I don’t know”.
The cost of a guess falls on the business. In Moffatt v. Air Canada (2024 BCCRT 149, 14 February 2024), a British Columbia tribunal ordered the airline to compensate a customer after its website chatbot said a bereavement fare could be claimed after travel, which the airline’s policy did not allow. The tribunal rejected the argument that the chatbot was responsible for its own words and held that the airline is responsible for all the information on its website, whether it comes from a static page or a chatbot.
So treat the model’s uncertainty as a hint that can add a handoff, never as the gate that must catch the dangerous ones.
Write the stop list by topic, facts and signals
Before launch, the owner and whoever builds the agent go through real or expected conversations and write down where the agent must not answer alone. The list falls into three kinds of trigger, each decided and tested differently.
Comparison
| Kind of trigger | Examples | Who decides it | How to test it |
|---|---|---|---|
| Fixed by topic | Refunds or exceptions to policy, complaints and legal threats, collecting personal data, anything about health or safety, any price, date or promise not in your records | The owner, in writing, before launch | One test conversation per topic, including indirect wording |
| Facts the agent cannot back up | An order or booking lookup failed or returned nothing, a required field is missing, two sources disagree | Code, from the tool results | Make each lookup fail in a test and check the agent stops |
| Conversation signals | The customer asks for a person, the same question comes back, the agent fails to understand twice in a row | Code, with counters per conversation | Scripted conversations that repeat and rephrase |
A request for a person should always win, immediately. Microsoft’s Copilot Studio documentation (updated 30 April 2026) treats a customer typing “talk to agent”, or a question the agent cannot place, as an implicit trigger for its hand off to a live agent topic. Google’s Dialogflow CX numbers its built-in no-match events from one to six, so a builder can escalate on the second or third failure to understand instead of the first.
Topic stops need a classifier, which may itself be a model, but the decision about what happens next is ordinary code. The model’s job is to read the message and fill in the topic and the facts; the code compares them with the list. The reasoning in stop rules kept outside the agent’s discretion applies here as much as to coding agents.
Make “a person has this chat” a state the agent checks first
The Make forum poster’s real difficulty was state. A handoff needs one record per conversation that says who is in charge, and the agent’s code reads it before any model call. If a person owns the conversation, the agent does nothing: no reply, no “let me help while you wait”.
The platforms leave this to you. Dialogflow CX’s live agent handoff response only signals that a conversation should be handed off; your integration has to act on it. The WhatsApp Cloud API documentation describes no bot-to-human handover at all. If staff answer from the WhatsApp Business app on the same number, Meta’s coexistence documentation says each message they send arrives at your webhook as an smb_message_echoes event, and the agent should treat that as a takeover even when no stop fired.
def handle_incoming(conversation, message, now):
state = store.get(conversation.id) # mode, owner, reason, since
if state.mode == "human":
return None # the person has it; stay silent
reading = model.read(message, conversation) # topic, facts asked, request for a person
trigger = stop_rules.first_match(reading, tool_results(reading), counters(conversation))
if trigger:
store.set(conversation.id, mode="human", reason=trigger, since=now, owner=rota.on_duty(now))
notify(rota.on_duty(now), handoff_brief(conversation, reading, trigger))
return customer_message_for(trigger, rota.next_pickup(now))
return model.reply(message, conversation)This sketch is illustrative: it omits storage, retries and error handling. Resuming is the other half. The poster wanted the AI back after a day of silence, which lines up with WhatsApp’s customer service window: 24 hours from the customer’s last message, after which a business can send only pre-approved templates. Let the person end the takeover with a short note, and let the timer apply only to conversations the person marked as done. What state must survive a human decision pause covers what the agent must read before it speaks again.
Hand over a brief, and tell the customer what happens next
A transcript alone makes the person read the whole chat while the customer waits. Copilot Studio passes the conversation history plus context variables such as the last topic, the customer’s last phrases and a private message to the live agent. A small business can do the same in a WhatsApp group message or a shared inbox note:
- What the customer wants, in one sentence
- The trigger that fired and why
- What the agent has already told the customer, word for word
- What the agent looked up, including lookups that failed
- What is missing before anyone can answer
- When the customer service window closes
The customer needs one honest message: a person will take over, and when that is likely. Base the time on the rota, not on hope; outside staffed hours, say when someone will be back. The handoff message must not promise the outcome. “A colleague will process your refund” is the guess the stop was meant to prevent.
Staff the queue before you switch the stops on
A handoff nobody picks up is worse than no handoff, because the agent has gone silent. Before launch, name who is on duty for each hour the agent runs, who covers when that person is busy, and what happens to a handoff still unanswered after a set time: a second person is notified, and the customer is told.
Then measure the stops once a week. Take a random sample of conversations the agent finished alone and look for the ones that should have stopped: those missed escalations are the cases that matter. Look at the handed-over ones for stops that were not needed, and record how long customers waited for a person. Each missed escalation becomes a new test conversation and, where needed, a new rule; each unnecessary one is a candidate for narrowing a rule. One person, usually whoever owns customer service, owns that review.
When Eliyce builds an agent, the stop conditions and their acceptance checks are agreed with the owner before building, the handoff rules and conversation state are written as tested code, and the model only interprets language. Eliyce runs its own agents in production, with sub-agents and retrieval doing real work; it does not staff the human side of support, and no customer agent built by Eliyce is in production yet. If your agent needs a clear point where a person takes over, talk about your agent.
Frequently Asked Questions
How do I make an AI agent stop and hand the conversation to a person instead of guessing?
Write the stop conditions down, check them in code on every message, and keep a per-conversation record of who is in charge. The part most teams skip is testing: run one scripted conversation per stop before launch and again after every prompt or model change, because a stop that worked last month can quietly stop firing.
Where does my agent guess today, and which guesses cause complaints?
Read a sample of recent conversations and mark each answer that was not backed by a record: missing data, a policy question, a promise, or a request outside the agent’s job. Then match them against complaints and corrections from staff. The categories that produced complaints become fixed stops first; the rest can wait for evidence.
Which triggers should be fixed rules, and what does each handoff cost?
Anything that can commit the business, money, personal data or safety is a fixed rule. Repeated questions and failed lookups can be signals with counters. Each handoff costs staff minutes and customer waiting time, so count both from the first week; if a rule produces many handoffs that staff resolve in seconds, it is a candidate for a narrower rule or a better answer source.
Where does a handoff fail in practice?
Three places: the agent replies during a takeover because the state was not checked or a second workflow also answers; the handoff arrives after hours and nobody sees it; or the agent resumes on a timer and contradicts what the person said. Test each one deliberately, including a staff reply sent from a phone.
Which measures show that escalation is working, and who owns them?
Missed escalations found in the weekly sample, unnecessary escalations, and time from handoff to the first human reply. Track them per trigger so a change can be traced to one rule. One named person owns the weekly review and signs off any change to the stop list, and the change is tested before it goes live.
