Gemini 3.5 flash on a single-prompt agent. The user got confused and started interrupting the agent with “hello?” type inputs. The model then raced and streamed “Hello” 50x in a single message after which the user had to hang up.
Please share stats on LLM collapse for commonly used models and which models you recommend for a balance of speed (TTFT with lowest variance), instruction following, and cost.
While i understand, model collapse and similar failures are inevitable, this unfortunate turn of events cost our business over $500 in lost revenue and potential further complications. I’m seeking advice backed by stats to minimize the likelihood of such events in the future.
call: call_9a5a7cf8b3172f1db69e73f06e0