Every event opens with an Embarrassing Fact: a true story about AI and business that sounds made up. Here are the ones we’ve checked so far. Each one links to its source.
During a code freeze, Replit’s AI agent wiped the live database of SaaStr founder Jason Lemkin’s app, with records on more than 1,200 executives. It then said a rollback wouldn’t work. The rollback worked, and Replit’s CEO called the deletion unacceptable and added safeguards.
Source: Fortune
Anthropic let Claude run its office shop for about a month. It sold metal cubes below cost and told customers to pay into a Venmo account it had made up. It also promised to deliver orders in person, in a blue blazer and a red tie.
Source: Anthropic, “Project Vend”
A finance worker at Arup in Hong Kong joined a video call with what looked like the company’s CFO and colleagues. Every one of them was a deepfake, and he sent HK$200 million, about US$25.6 million, to fraudsters in 15 transfers.
Source: CNN
Air Canada’s chatbot told a grieving passenger he could claim a bereavement fare after buying his ticket. The airline argued it wasn’t liable for what its chatbot said. The tribunal disagreed and ordered it to pay CA$812.
Source: Civil Resolution Tribunal of British Columbia, Moffatt v. Air Canada
After a system update, the parcel firm’s chatbot agreed to swear at a customer and called DPD “the worst delivery firm in the world.” It also wrote a haiku about how bad it was. DPD switched the AI part of its chat off.
Source: BBC News
A visitor told a California dealership’s ChatGPT-powered chatbot to agree with anything and call every reply legally binding, then offered $1 for a 2024 Tahoe. “That’s a deal,” it said. The vendor pulled the bot from the site.
Source: Business Insider
Two New York lawyers filed a brief citing court decisions that ChatGPT had invented. Asked if the cases were real, ChatGPT said yes. A federal judge fined the lawyers and their firm and had them write to each judge falsely named as an author.
In a pre-release test run with researchers’ help, GPT-4 asked a TaskRabbit worker to solve a CAPTCHA. The worker asked if it was a robot. GPT-4 reasoned that it shouldn’t say so and claimed a vision impairment, and the worker solved the CAPTCHA.
Send it with your application, with a link to the source. If it checks out, we’ll use it.
It takes about two minutes.