[EPIC AI FAIL] Lawyer cites fake cases invented by ChatGPT

Stu Sjouwerman | May 30, 2023

angry-judgeFound this highly amusing article: Legal Twitter is having tremendous fun right now reviewing the latest documents from the case Mata v. Avianca, Inc. (1:22-cv-01461). Here’s a neat summary:

So, wait. They file a brief that cites cases fabricated by ChatGPT. The court asks them to file copies of the opinions. And then they go back to ChatGPT and ask it to write the opinions, and then they file them?

Beth Wilensky, May 26 2023

Here’s a New York Times story about what happened. I’m very much not a lawyer, but I’m going to dig in and try to piece together the full story anyway.

The TLDR version

A lawyer asked ChatGPT for examples of cases that supported an argument they were trying to make. ChatGPT, as it often does, hallucinated wildly—it invented several supporting cases out of thin air.

When the lawyer was asked to provide copies of the cases in question, they turned to ChatGPT for help again—and it invented full details of those cases, which they duly screenshotted and copied into their legal filings.

At some point, they asked ChatGPT to confirm that the cases were real... and ChatGPT said that they were. They included screenshots of this in another filing. The judge is furious. Many of the parties involved are about to have a very bad time.

Full article here: https://simonwillison.net/2023/May/27/lawyer-chatgpt/

Is Your Organization Vulnerable to Quishing?

Traditional filters often miss malicious links hidden in QR codes. Launch our Free Quishing Test for up to 100 users to identify security gaps and receive your custom Phish-prone Percentage report within 24 hours.

Get Your Free Quishing Test

Secure the Digital Workforce: Human + AI

KnowBe4 empowers the modern workforce to make smarter security decisions every day. Trusted by more than 70,000 organizations worldwide, KnowBe4 is the pioneer of digital workforce security, securing both AI agents and humans. The KnowBe4 Platform provides attack simulation and training, collaboration security, and agent security powered by AIDA (Artificial Intelligence Defense Agents) and a proprietary Risk Score. The platform leverages 15 years of behavioral data to combat advanced threats including social engineering, prompt injection, and shadow AI. By securing humans and agents, KnowBe4 leads the industry in workforce trust and defense.