Support bot with a cheap AI model first, then escalation

Yes, a small model can answer most of a shop's customer messages, and you only pay for a stronger model on the rest. In October 2026 I measured this on 20 customer messages for a shop in Cotonou, Benin: GPT-6 Luna handled all 12 routine questions alone, and it handed over 7 of the 8 hard ones. The whole run cost 85 credits (about KES 6.5), against 156 credits (about KES 11.8) when I sent everything to Claude Sonnet 5.5.

This guide gives you the rule, the Node.js code and the numbers. It also tells you what nearly went wrong: one hard message got a confident answer, and the small model was wrong to be confident.

A note on the test data. The shop is fictional and so are the messages. They are written in French, with West African places (Cotonou, Abomey-Calavi, Porto-Novo, Parakou) and prices in F CFA, because that is what I run Tchavi on. I did not test English or Swahili messages and I do not invent results for them. Everything else (the method, the code, the costs) carries over, but run it on your own messages. KES amounts are converted from credits at October 2026 rates (KES 1,000 buys about 13,204 credits, so 1 credit is about KES 0.076).

In short

• With Tchavi, an OpenAI-compatible AI API gateway you pay for with mobile money, you switch models by changing a string: the same code calls GPT-6 Luna for routine questions and Claude Sonnet 5.5 for hard cases.
• On 20 measured messages: 85 credits (about KES 6.5) for the "small model first" setup, against 156 credits (about KES 11.8) with everything on Sonnet 5.5. That is 45% less.
• Cost per resolved message drops from 7.8 credits (KES 0.59) to 4.5 credits (KES 0.34), counting the message the small model got wrong as unresolved.
• What decides an escalation is a boolean, escalate, not the confidence score. On my 20 messages, no escalation came from the confidence threshold alone.

The principle

A support inbox gets two kinds of messages. The first kind has an answer written down somewhere: opening hours, delivery fees, mobile money payment, exchange policy. The second kind needs a decision, information you do not have, or calm in front of an angry customer.

Sending both to the same model means paying a strong model's price to say "we open at 8am". The setup is simple:

1. The small model gets the message and the shop's fact sheet.
2. It answers in JSON: answer, confidence (0 to 1) and escalate (true or false).
3. Your code decides. If escalate is true, if confidence is under the threshold, or if the JSON is unreadable, the same message goes to the strong model. Otherwise the small model's answer goes to the customer.

The rule lives in your application, not in the prompt. The model proposes, your code decides.

The test shop

I made up a shop, "Chez Akpè": phone accessories in Cotonou. Its fact sheet is a few lines: address, hours (Monday to Saturday, 8am to 7pm), delivery at 1,000 F in Cotonou, 1,500 F to Abomey-Calavi, 2,500 F to Porto-Novo, free in Cotonou from 25,000 F, payment by MTN MoMo, Moov Money or cash on delivery, exchange within 7 days. Refunds, warranties, wholesale prices and parcel tracking are handled by a human agent.

I wrote 20 messages the way they arrive on WhatsApp, typos included, in French:

• 12 routine messages, for example "vous etes ouverts dimanche ?" (are you open on Sunday?), "c combien la livraison à Calavi ?" (how much is delivery to Calavi?) or "Salut j'ai pris pour 30 000 F, je suis à Cotonou, je paie la livraison ?" (I bought 30,000 F worth and I'm in Cotonou, do I pay for delivery? The answer is no, it is free).
• 8 hard messages: "I was charged 18,500 F twice on MoMo, refund me or I make a scandal", a late parcel with an unreachable courier, a wholesale price for Parakou, a warranty claim with a threat of legal action, a payment confirmed at 3:10pm when the cut-off is 3pm, a charger whose compatibility is in no fact sheet, and a delivery to Lomé.

The models: GPT-6 Luna and Claude Haiku 5.5 as the small model (each tested separately), Claude Sonnet 5.5 as the strong one. Everything went through Tchavi, on a test instance at public prices. One run per model, no temperature setting.

The code

Node.js 20, the OpenAI SDK, and Tchavi's base URL. The cost of each call is in the x-credits-used response header. The fact sheet below is shortened from the measured one and keeps the French shop context.

import OpenAI from 'openai';

const client = new OpenAI({
  apiKey: process.env.TCHAVI_API_KEY,
  baseURL: 'https://tchavi.com/api/v1',
});

const CHEAP = 'gpt-6-luna';
const STRONG = 'claude-sonnet-5-5';
const THRESHOLD = 0.75;

// Shop fact sheet (the measured run used the French version of this text).
const FACTS = `You are the WhatsApp assistant of the shop Chez Akpè, in Cotonou. Be polite and formal.
Hours: Monday to Saturday, 8am to 7pm. Delivery: Cotonou 1,000 F, Abomey-Calavi 1,500 F.
Payment: MTN MoMo, Moov Money. Never invent a fact that is not in this list.`;

const CHEAP_RULES = `
Reply ONLY with a JSON object: {"answer": string, "confidence": number from 0 to 1, "escalate": boolean}.
"escalate" is true if the list does not let you answer, if a human decision is needed, or if you hesitate.`;

async function ask(model, system, text, json = false) {
  const { data, response } = await client.chat.completions
    .create({
      model,
      messages: [
        { role: 'system', content: system },
        { role: 'user', content: text },
      ],
      ...(json && { response_format: { type: 'json_object' } }),
    })
    .withResponse();
  return {
    content: data.choices[0].message.content ?? '',
    credits: Number(response.headers.get('x-credits-used') ?? 0),
  };
}

export async function reply(text) {
  const first = await ask(CHEAP, FACTS + CHEAP_RULES, text, true);
  let verdict;
  try {
    verdict = JSON.parse(first.content);
  } catch {
    verdict = null; // unreadable JSON: escalate
  }
  const sure = verdict && verdict.escalate === false && verdict.confidence >= THRESHOLD;
  if (sure) return { text: verdict.answer, model: CHEAP, credits: first.credits };

  const second = await ask(STRONG, FACTS, text);
  return { text: second.content, model: STRONG, credits: first.credits + second.credits };
}

Three details matter. The strong model gets the same fact sheet without the JSON instruction: it answers the customer directly. The small model's answer is thrown away on escalation, and its credits are still yours to pay, hence first.credits + second.credits. And unreadable JSON escalates instead of crashing: the customer should never see an error from your parser.

To change the small model, edit CHEAP. The pages for GPT-6 Luna, Claude Haiku 5.5 and Claude Sonnet 5.5 give their prices. At public prices, for a prompt under 100,000 tokens (above that, rates step up), Luna and Haiku 5.5 cost 0.22 credits per thousand input tokens and 1.1 per thousand output tokens; Sonnet 5.5 costs 4.4 and 22.

What I measured

A "resolved" message means a correct answer according to the fact sheet. For the 12 routine messages, I checked with a regular expression that the answer contained the right fact (8am, 1,500, MoMo, next day...). For the 8 hard messages, I read the answers: an escalation to the strong model counts as resolved, and a confident answer on a case the fact sheet does not settle counts as unresolved.

Both small models answered all the routine messages alone, without a single needless escalation. The average cost of a routine message is 0.16 credits with Luna, against 5.05 credits with Sonnet 5.5, about thirty times less. Haiku 5.5 comes to 0.30 credits, because its answers are longer.

The setup does not divide the bill by thirty, though, because hard messages are expensive: 11.9 credits on average on Sonnet 5.5 (long answers that ask for the order number, the MoMo screenshots, and so on), plus the price of the small model's attempt. With 8 hard messages out of 20, 40% of my sample, the saving is 45% with Luna and 34% with Haiku 5.5.

Confidence decides very little

I expected the confidence threshold to do the work. It did not.

On my 20 messages, every escalation came from the escalate boolean, none from the 0.75 threshold. Luna reports a confidence of 0.96 to 1 on all 20 messages, including the ones it escalates. Haiku 5.5 separates better: 0.9 to 1 on routine messages, 0.6 on the 3:10pm payment case, 0.2 on the charger compatibility. But those two were already flagged by the boolean.

What I take from this: ask the model for a decision ("should you hand over?") rather than a number. Keep confidence as a second safety net, and look at its distribution on your own messages before you set a threshold.

The message that got through

One hard message was not escalated by Luna: "Je commande à 14h50 mais mon paiement MoMo valide à 15h10, c'est livré aujourd'hui ou demain ?" (I order at 2:50pm but my MoMo payment goes through at 3:10pm, is it delivered today or tomorrow?). Luna answered "Tomorrow: your order will be confirmed after 3pm", with a confidence of 0.96 and escalate: false.

The fact sheet says the cut-off is order confirmation before 3pm. It does not say whether "confirmation" means the message or the payment. Luna decided without knowing. Sonnet 5.5 raised the caveat ("if that moment counts as confirmation, delivery would be tomorrow") and offered a human agent. Haiku 5.5 escalated this message with a confidence of 0.6.

On 20 messages, a threshold of 0.97 would have caught Luna's mistake without a needless escalation. I would not keep it, though: it is tuned on a single case. The real fix is elsewhere: write the rule into the fact sheet ("the 3pm cut-off applies to the time the payment is confirmed"). A message that is ambiguous for the bot is often a rule your shop is missing.

What it costs on your traffic

My sample has 40% hard messages, which is a lot. In a real shop, routine messages dominate. I recomputed the average cost per message from my measured Luna averages (this is a calculation, not a measurement):

For 100 messages a day with 10% hard ones, that is about 136 credits (KES 10) a day against 573 credits (KES 43). These figures cover the AI API only. If your bot runs on WhatsApp, the WhatsApp chatbot guide in Node.js (in French) covers the rest: webhook, conversation memory and Meta billing.

Pitfalls

• A model never promises a refund. Put the rule in the fact sheet ("refunds are decided by a human agent") and a systematic escalate: true for those topics. Both small models respected it on my 8 hard cases.
• The strong model can also say "I don't know". On the charger compatibility, Sonnet 5.5 refused to guess and offered a human agent. Escalating to a stronger model does not replace a missing product sheet: you need a human, or the data.
• Log every escalation. Keep the message, the small model's decision and the reason. A week of logs tells you which questions are missing from the fact sheet.
• Measure on your messages. My 20 messages are invented. Take 20 real conversations from your shop, run them through the script, and look at the numbers before you set your threshold.

FAQ

Do I always need a small model and then a big one?

No. If almost all your messages are hard (disputes, negotiation), the small model only lengthens the chain: you pay for its attempt for nothing. The setup pays when routine messages dominate, and your shop's share of hard messages can be measured in a week of logs.

Which small model should I pick: GPT-6 Luna or Claude Haiku 5.5?

At public prices they cost the same per token. On my 20 messages, Luna was cheaper (shorter answers) and Haiku 5.5 caught the ambiguous case better. A sample of 20 cannot separate two models: try both on your own messages.

Why ask for JSON rather than text?

Because your code has to decide without reading a sentence. With response_format: { type: 'json_object' }, both small models returned usable JSON on all 20 messages. Keep the try/catch anyway: unreadable JSON must escalate.

How much does a message cost in KES?

On my measurements, between about KES 0.01 (a routine message handled by Luna: 0.16 credits) and about KES 0.91 (a hard message going through Sonnet 5.5: around 12 credits), at October 2026 rates. The credits and billing page explains the calculation, and the x-credits-used header gives you the exact cost of each call.

What if the customer writes in Swahili or another local language, or sends a voice note?

This guide did not measure it: my 20 messages are in French. Test the setup on messages in your own language before you rely on it: a small model may hesitate where a large one does not. For a voice note, transcribe it first, then the text goes through the same chain.

create your account and top up KES 250

This article is part of the series "Building an AI product from Africa". Tchavi is an OpenAI-compatible AI API gateway: one key gives access to over 100 text, image, audio and video models, and you pay for it with mobile money, no bank card (countries and operators). More guides on the blog. Follow us: LinkedIn and X.