Override agent settings in web/sms chat?

Support Help
1

Hey there! I’m hoping to override Knowledge Base variables in our SMS / Web Chat agent workflow so we can have one main prompt and dynamically use the right knowledge bases based on our own logic.

I’m not able to get that to work by providing an agent_override in our API call to start a new chat.

How can I accomplish this behavior? The AI helper on the forum said it’s possible, but it didn’t work.

curl -i -sS https://api.retellai.com/create-chat \

  -X POST \                            

  -H "Authorization: Bearer $RETELL_API_KEY" \

  -H "Content-Type: application/json" \

  --data @- <<JSON      

{                                         

  "agent_id": "$AGENT_ID",

  "agent_version": $AGENT_VERSION,

  "metadata": {

    "test": "web-chat-kb-override-probe"

  },

  "agent_override": {

    "retell_llm": {

      "knowledge_base_ids": ["$KB_ID"],

      "kb_config": {

        "top_k": 3,

        "filter_score": 0.8

      }

    }

  }

}

The above block returns a new chat but the agent has no idea how to answer questions about what would have been in the custom KB_ID that we pass in. Here’s an example chat ID created by this method: chat_5165779fefc361f75c6fb6b4a30

2

Hi @anthony3

Agent overrides for chat/SMS use the field name il, not agent_override. The agent_override field name you’re using isn’t recognized by the API. The correct field for per-session overrides is il, and it does support retell_llm.knowledge_base_ids and retell_llm.kb_config.

our request should look like:

{
  "agent_id": "$AGENT_ID",
  "agent_version": "$AGENT_VERSION,
  "metadata": { "test": "web-chat-kb-override-probe" },
  "il": {
    "retell_llm": {
      "knowledge_base_ids": ["$KB_ID"],
      "kb_config": {
        "top_k": 3,
        "filter_score": 0.8
      }
    }
  }
}

Replace agent_override with il and the knowledge base override should take effect for that chat session.

You can see the full details here (Inbound webhook - Retell AI) and ( Outbound Calls (Make Calls) - Retell AI)

Thank You

3

Thanks, just tried it out, but still not seeing the retrieval from my KB. Can you check chat_e28fabe28219b4bf8caad8ddcb7 to see if knowledge_base_31232fb26ad71ef6 was set with the “il” field correctly?

4

Hi @anthony3

I’ve shared it with our team for review.

We’ll keep you updated as soon as we have more information.

Best regards

1 Like
5

Hi @anthony3

Team checked the chat session chat_e28fabe28219b4bf8caad8ddcb7 — the knowledge base knowledge_base_31232fb26ad71ef6 was not applied to this chat. The LLM configuration used by your chat agent currently has no knowledge bases attached.

The reason is that the Create Chat API does not currently support the agent_override parameter — this feature is only available for voice call creation endpoints (Create Phone Call, Create Web Call, Register Phone Call). So even if you pass agent_override with knowledge_base_ids when creating a chat, it won’t take effect.

To use a knowledge base in your chat agent, you’d need to add it directly to the LLM/Response Engine configuration in the dashboard (or via the Update Retell LLM API) so it’s baked into the agent itself.

Thank You

6

Hm. This would be extremely useful for our texting use case and will have to do that entire dynamic knowledge base workflow outside of Retell, which we do not prefer.