I have an agent which helps callers book hotel rooms and has a fallback to transfer to a human on request or if something isn’t going quite right. The destination number I have configured is the hotel reception, which is usually answered by a person but sometimes can go to voicemail.
I originally configured my call transfer to be a warm transfer so the AI agent would summarise the call with the human so they didn’t need to repeat themselves. I encountered 2 issues with this configuration when the call went to voicemail:
The agent would start speaking and summarising the call immediately, while the voicemail greeting was playing, and by the time the voicemail greeting got to "leave a message " the agent had almost finished its summary so the voicemail was often similar to “then they wanted to speak to a person, I’ll put them through now”.
The call was put through, with the caller unaware they were actually now in a voicemail recording, so there was just silence.
What I really want is the following:
If a person picks up the call, the AI agent will summarise the call and transfer the call for the human to complete
If a voicemail picks up the call, the AI agent should wait for the voicemail to start recording and then leave a message. The caller should not be put through (clearly nobody is there to help) and should instead be told the AI agent couldn’t get through to a person and they have left a message for a callback.
I couldn’t see an obvious way to achieve this with the regular warm transfer options so I changed the transfer method to be an agentic warm transfer to see if I could get my desired behaviour in the transfer agent prompt. As a bonus, if the human missed something they are able to ask the transfer agent to repeat it, so I quite like the idea of an agentic transfer. I set up a new transfer agent and wrote my instructions, but whatever I wrote in my system prompt for the transfer agent it seemed to just ignore everything. In the end I changed my transfer agent prompt to be the following:
Say exactly “Apples” and then call the function `cancel_transfer`.
When my original agent started the transfer process it put the caller on hold, called my destination number, summarised the call and bridged the call immediately afterwards and then said the handoff message - no mention of apples and the transfer was not cancelled. It is behaving like the transfer agent is not actually being switched to?
If there is a particular way I should handle my desired outcome and my approach is incorrect, I am happy to explore the correct way to achieve it and modify my agents accordingly.
Original agent ID: agent_763fcbd4a24cd839b93878aa08
Transfer agent ID: agent_757a87ae82de9f42fc3a0c62ac
Example call: call_bc0140856398cf7323e78f5b4c6
Hey @alacer.ben thanks for providing the details and the Agent ID and Call ID. Let me check this with the team, and I’ll get back to you with an update as soon as possible.
For detecting voicemail vs. human, you’ll need to use the on_call_ended webhook to check the call status and handle each case separately rather than relying on warm transfer alone. On human pickup, do warm transfer with your summary. On voicemail detection, hang up and use get_call to confirm it’s voicemail, then tell the caller you’ve left a message. The transfer agent ignoring prompts usually means the system directives are overriding your custom instructions—check if you’re passing conflicting parameters in the transfer config itself.
Hey @mark1, thanks for your reply. That setting does seem to help with the standard warm transfer. The agentic warm transfer still doesn’t seem to be working at all though.
Putting aside the voicemail detection for a moment and focusing on just when a real person does answer the transferred call, when I configure my original inbound agent to use agentic warm transfer, the transfer agent doesn’t appear to do anything (like it’s not even called) and the inbound call is immediately bridged when I pick up the transferred call. From the description of agentic warm transfer, I am expecting to have a conversation with the transfer agent and then have the call bridged.
Hey @alacer.ben The call ID you shared appears to be for a warm transfer. Could you please provide the call ID for the agentic transfer case instead? This will help us better understand and investigate the issue.
My apologies @mark1, I tried various calls earlier and I have lost track of the IDs so I have just made another test call with the following IDs.
Inbound call: call_1bdf42b785d40285e9918685076
Transfer agent call: call_5c99b79d3b9a1a67b9d93d0d
This time the prompt for the transfer agent was just
Tell a one line joke. Then ask if you want to speak to the caller.
but as before when I answered the call from the inbound agent I was not told a joke or asked if I wanted to speak to the caller, it was just immediately bridged once I answered.
Hey @mark1 maybe I am just misunderstanding how this is supposed to work, because I would expect the transfer agent to be doing some kind of work, otherwise I am confused about the purpose of the transfer agent and why I am required to have one if it doesn’t seem to do anything?
I was expecting that when the destination (hotel reception) answered the call, the transfer agent would follow its system prompt (which in the testing example is just to tell a joke) and then ask if they should put the call through. So in the transcript there would be a joke (and in the real world possible conversation between the transfer agent and the receptionist) in between the tool invocation and result. The process seems to skip over the middle part (the transfer agent) and go straight to the handoff message.
In a made up scenario this is something like what I would expect to happen based on the description of an agentic transfer, so please tell me if I have it wrong and I just misunderstand the concept.
The caller calls the inbound hotel AI agent and starts making a booking
The agent calls our MCP and determines there is no availability for the dates requested and explains this to the caller
The caller wants to speak to a person
The inbound agent puts the caller on hold and starts the agentic transfer process (which I guess is some form of agent switch from the inbound agent to the dedicated transfer agent)
The agent calls the destination number (hotel reception) and the receptionist answers the phone
The transfer agent explains they have a guest on hold that was trying to book a room for [ date ] but there was no availability and they wanted to speak to a person
The receptionist was distracted and says sorry I didn’t catch the date, can you say it again
The transfer agent confirms the date which was discussed with the caller
The receptionist says ok thanks, put them through
The transfer agent then bridges the call and plays the handoff message (you’re now connected to a human, I’ll leave you two to it)
Sorry just to add. I understand if the conversation between the receptionist and the transfer agent happens in a different call record and doesn’t show in the original inbound call (that’s not really a problem), but if you look at the call transfer agent call ID I provided it is just me saying hello to the agent and it immediately bridging the call - no joke was told and I wasn’t asked if I wanted to speak to the caller.
Hey @alacer.ben I’ve checked with the team, and it appears to be a hallucination from the AI. The behavior should be as expected, but the agent hallucinated in this call.
Hey @mark1 unfortunately it is not just the call I mentioned that exhibited this behaviour and I don’t agree this was a one-off hallucination, I still think there is something not right with the agentic transfer process in general. Below are a bunch of calls I made this morning, some of which worked as expected and others which just bridged immediately when I said hello.
I updated my transfer prompt to do something useful instead of just telling a joke (that was just me trying to get the transfer agent to say something), and when it follows the prompt it works exactly how I would expect and exactly how we would want, it just seems quite unpredictable as to whether the transfer agent bridges the call immediately or follows its prompt.
Inbound call_6067636066cbbed352aaf59f5be, transfer call call_2af0a6af59253ebb5b22a2fe - this was bridged immediately.
Inbound call_3e620f370c81a64e5fa97c14c2f, transfer call call_b47bcf450f8d394bd3345de6 - this followed the transfer agent prompt.
Inbound call_669e78655a578107573da08bc48, transfer call call_ebd193513ef4f96cf6380e03 - this followed the transfer agent prompt.
Inbound call call_a65fe0fb480051c58cf058b9a7e, transfer call call_f3beace90ae68b65fe3b1d60 - this was bridged immediately.
Inbound call call_b982eff7ee8b9e01cdb20816bb1, transfer call call_868247abd4a671d5af1c3e7a - this was bridged immediately.
Hey @alacer.ben What’s happening on the failing calls:
The transfer agent is being switched to. On every “immediate bridge” call we can see the bridge_transfer tool call firing — that tool only exists on the transfer agent, so the sub-agent is definitely running.
The failure is that the transfer LLM is calling bridge_transfer on its very first turn, before speaking anything. Your prompt asks it to summarise first and then call the function, but that’s a soft instruction and gpt-5.1 doesn’t always honour it. Same published version, 3 of your 5 calls skipped straight to the tool call, 2 followed the prompt — so it’s a repeatable non-compliance pattern, not a one-off, also I would suggest to use GPT 4.1.
Two changes on the transfer agent that reliably fix this:
**Give bridge_transfer a real description that gates it.** Something like: “Call this ONLY after you have finished speaking the summary to the person who answered and they have confirmed they are ready to take the call. Never call this on the first turn.” Empty descriptions let gpt-5.1 treat the tool as freely callable — a real description acts as a much stronger guardrail than the same wording in the system prompt.
**Re-order the prompt as explicit numbered steps**, e.g. "Step 1: introduce yourself and speak a one-line summary out loud. Step 2: ask if they need anything clarifying. Step 3: only after they respond, callbridge_transfer." Numbered steps with the tool call as the final step reduce skipping much more reliably than descriptive prose.
Thanks for the explanation and clarification @mark1, I appreciate it. I will work on the prompt as suggested and bear in mind what you have said about coercing the LLM into doing the right thing
