Customer Support Agent
A voice and chat support agent for a payments company. It answers from the knowledge base, checks a payout only for its owner, and hands real cases to a person. It never moves money.
A RelayPay customer with a question about fees or a stuck payout waited for a person, who looked up the record by hand.
- Caller · voice or chat (trigger)
- Vapi: speech ⇄ text · custom LLM (step)
- Scrub card numbers + passwords (step)
- Agent turn: Sonnet 5 · 9 MCP tools (AI)
- search_knowledge (step)
- Read back + confirm: reference · email · time (human)
- Owner’s email? (decision)
- Says nothing about the record (failure)
- Lookup status (step)
- Sentence guard: promises · unconfirmed · internal (step)
- Ticket + escalation (step)
- Calendar callback: weekdays 9–5 Lagos (output)
- Discord · no caller details (output)
- Spoken reply (output)
- Supabase log: turns · tool_calls · retrieval_logs (store)
- Staff dashboard (human)
- Step 01
Vapi handles speech in and out, and sends every turn to my backend as a custom LLM, so the backend decides every word. A typed chat works without Vapi if the mic is down.
- Step 02
Card numbers and passwords are scrubbed before storage or Claude, then the Agent SDK runs with my MCP server as its only tools.
- Step 03
A knowledge answer must come from search_knowledge that turn. Each sentence is checked before it’s spoken, and a blocked one becomes a fixed safe line.
- Step 04
A reference is read back first (“I heard T X N nine zero zero one”), and a status is only given once the account email is read back, confirmed and matches.
- Step 05
If it can’t solve it, it opens a ticket, escalates, books a weekday callback against the calendar and posts to Discord with no caller details.
- Step 06
Vapi’s end-of-call report closes the call with a status and reason, and a sweeper catches calls that never report.
A reference isn’t proof of who’s calling
At first anyone who knew TXN-9001 heard its status. Now it needs the owner’s confirmed email, and without it the agent won’t even say the record exists.
The model judges the yes, the code checks the value
read_back stores the exact value, and a write needs a heard read-back of it from an earlier turn. If the prompt is ignored, the tools still refuse.
“Booked” only when the calendar says so
If Google fails, the case stands, #callbacks says “Needs manual booking” and #system-alerts gets the error.
One live call at a time
A held session is about 400 MB on a 512 MB server, so a second caller hears a busy line instead of crashing both calls.
The read-back passed every test, then failed the first real call
Every simulated case passed. On the first real call, the write was blocked.
Vapi stores the spoken line in its own wording (“typed; it’s”, “1:00 PM”), so the heard read-back never matched the stored value. It failed closed, with nothing invented.
Fix: Both sides are now compared as letters and digits only. The retest on Render booked TKT-1546.
Simulated input passes tests. Test against what the real provider actually sends.
- Get a real call through the real provider on day one, before writing the simulated test cases.
- Require an emailed code from the start. The email check stops guessing, but not someone who already knows a customer’s email.
- Benchmark Nigerian English speech to text early, since the callers are in Lagos.