AI Voice Agent Saying Wrong Things? Fixes for Each Problem

In short: Open the call's transcript, find the moment it went wrong and change the one setting behind it: the Language for the wrong language, the System Prompt for names, length and how calls end, Conversation Rules for interruptions, and linked Knowledge Base documents or Data Catalogs for wrong facts. Save, call yourself with the same question and compare.

A step-by-step guide for BookAICall, an AI voice agent platform that makes and answers phone calls for businesses in Hindi, English, Hinglish and other Indian languages. Updated .

Steps

  1. Find the call: open Unified CRM, click the Voice Calls tab and click Available under Transcript, or open the caller's lead. Read the transcript and play the recording around the line that went wrong.

  2. Match it to a symptom below. Then open AI Agents, click Edit on the agent and change only the setting that the symptom points to.

  3. Click Save Changes. Calls that start after the save use it; a call already in progress keeps the earlier settings.

  4. Test with the same question: click Call AI on your own lead in Unified CRM and talk as that caller did. Read the new transcript.

  5. If the change made calls worse, scroll to Prompt Version History & 1-Click Rollback in the agent's panel and click Rollback on the version that worked.

It speaks the wrong language, or keeps switching

  • Cause: the Language under Voice & Personality is not the one your callers speak, or the greeting and prompt are written in another language.
  • Fix: choose the Language (Bilingual (Hinglish) for a Hindi-English mix), write the Greeting Message in that language, and add one rule to the prompt, for example: "Speak Hindi. If the caller speaks English, reply in English."

It says your brand, a product or a name wrongly

  • Cause: the agent reads names the way they are written. Short forms, capital letters and unusual spellings come out wrong.
  • Fix: tell it in the prompt how to say the word, for example: "Our name is said Shar-ma Pro-per-teez" or "Say B H K as three letters." A Hindi agent says Hindi names best when you write them in Devanagari.
  • Callers' names come from the lead's Name in the CRM: correct the spelling there. A lead still named Inbound Caller is not called by name.

Its answers are too long

  • Cause: no length rule, or long paragraphs in the prompt that the agent repeats.
  • Fix: in Conversation Rules set Longest reply (words) to about 20, and add "One or two short sentences, then listen" to the prompt. Move long details, such as a project description, into the Knowledge Base.

It talks over the caller, or stops in the middle of a sentence

  • It answers before the caller has finished: in Conversation Rules make the Pause that ends the caller's turn longer, or choose Low: waits for the caller to finish. Callers reading out an order or an address need the longest pause.
  • It stops talking when the caller says "haan" or when there is noise: the agent stops whenever it hears the caller. A TV, traffic or a speakerphone can do the same. Test from a quiet place; if it keeps happening on quiet lines, raise a ticket with the time of the call.
  • On incoming calls the caller's "hello?" cuts the greeting: set Inbound greeting to Finish the greeting first.

It makes up prices, offers or facts

  • Cause: the answer is not in the prompt, in a document linked to this agent, or in a catalog it can search. A document that shows Failed (for example a scanned PDF) has no text.
  • Fix: add the facts to the Knowledge Base and tick the document under Linked Knowledge Base Documents in the agent's panel. Lists that change, such as stock or rates, go in Data Catalogs, ticked under Connected Data Catalogs.
  • Add to the prompt: "Answer only from the knowledge base. If you do not know, say our team will call back with the details. Never guess a price or an offer."

It repeats itself or asks the same question again

  • Cause: the same instruction appears twice in the prompt, the greeting is pasted into the prompt as well, or a noisy line made the agent miss the answer.
  • Fix: remove the repeated lines, keep the greeting only in Greeting Message, and add "Do not ask again for something the caller has already told you."

It does not end the call, or ends it too early

  • How it works: the agent ends the call when the conversation is done or the caller says goodbye, and when it reaches voicemail or a recorded message. If the line goes silent it asks whether the caller can hear it, then says goodbye. No call runs longer than 15 minutes.
  • Never ends: add an End step to the prompt, for example "After confirming the visit, thank them and end the call."
  • Ends too early: say what must happen before the end, for example "Do not end the call until you have offered a site visit and asked if they have any other question."

It sounds robotic, or the voice does not suit your callers

  • Choose another voice under Voice & Personality and a Tone such as Friendly, then call it. The agent speaks as a woman or a man to match the voice, for example "kar rahi hoon" with a female voice.

It greets a caller as if it had called them

  • The Greeting Message is also said on incoming calls. Write a greeting that works both ways, or give incoming calls their own agent and number (see "Make an agent answer incoming calls").

Example: A coaching institute's agent, fixed in one afternoon

Apex Classes in Kota reads three transcripts from its admissions agent. In the first, the agent told a parent there are weekend batches; there are none. The batch list was in a PDF that was never ticked for the agent, so Meera ticks it under Linked Knowledge Base Documents. In the second, the agent said "A-P-E-X" letter by letter; she adds "Our name is said Ay-pex Classes" to the prompt. In the third, the agent cut off a father who was reading out his son's roll number; she sets the Pause that ends the caller's turn longer. She saves, calls herself with the same three questions, and all three now go right.

Tips

  • Change one thing at a time and test it, so you know which change fixed the call.
  • Keep a list of the ten questions your customers ask most and ask all ten after every change.
  • Read calls with a poor outcome first: they show what to fix fastest.

Frequently asked questions

Why did the agent tell a caller the wrong price?

It did not have the right one. Put prices in a Data Catalog or a Knowledge Base document, tick it for the agent, and add "Never guess a price" to the prompt.

Agent Hindi ki jagah English bol raha hai, kya karein?

Set the Language to Hindi or Bilingual (Hinglish) under Voice & Personality, write the greeting in Hindi, and add a line to the prompt about which language to speak.

Can I make the agent say our brand name correctly?

Yes. Write in the prompt how the name is said, split into sounds, and test with a call.

The agent keeps interrupting customers. What do I change?

In Conversation Rules, make the Pause that ends the caller's turn longer or choose Low sensitivity, then call it again.

The agent does not hang up after the customer says bye.

Add an End step to the prompt that says when to thank the caller and end the call. The agent ends calls itself once the conversation is over.

I changed the prompt but the agent still says the old thing.

Check that you clicked Save Changes and that the old line is not also in a linked document or the greeting. Calls already in progress keep the earlier settings, and during Holiday mode the holiday greeting is used.

Can I undo a prompt change?

Yes. Prompt Version History & 1-Click Rollback in the agent's panel keeps your last 15 prompts and greetings; click Rollback on the one you want.

Nothing I change fixes it. Who do I tell?

Raise a ticket from Support with the caller's number and the time of the call, and say what the agent should have said.

Questions this guide answers

  • Why is my AI calling agent giving wrong answers?
  • How do I stop an AI voice agent from interrupting customers?
  • Why does my AI agent speak the wrong language?
  • How do I make an AI agent pronounce my brand name?

Related guides

Start the free trial · Book a demo · All guides

BookAICall provides enterprise AI voice agents with real-time conversational capabilities.

Contact: [email protected] | +91-8502044044 | Mansarovar Jaipur Rajasthan