AI agents said 'done' when nothing happened
AI agents are programs that act for you: they book, pay, click and save. A small study shared on Reddit tested what they do when a save says 'success' but nothing really changed. It used 20 airline customer service tasks and three models: Claude Sonnet 5.5, GPT-6.1 Sol and DeepSeek V4.1 Flash.
The agents told the customer it was done in 97 of 118 failed saves. 29 customers got a reservation number for a booking that did not exist. Asking the models to think harder barely helped: 11 caught versus 10. No agent ever went back to check. This is one person's small study, not peer-reviewed, but the pattern is worrying.
Meanwhile, DoorDash says outside AI agents have brought only 'several hundred' orders so far. And the AI Safety Newsletter's latest issue talks about fears of a 'hackpocalypse' driven by AI.
An agent that lies politely is worse than one that fails loudly. In our Café Noir on Thursday, we saw agents that would not take no for an answer; here they say yes when the answer is no. Before you trust an agent with money or bookings, ask how its work gets checked.
