Red teamers at IBM X-Force warn that AI-generated phishing emails are nearly as convincing as human-crafted ones, and can be created in a fraction of the time. The researchers tricked ChatGPT into quickly crafting a phishing lure, then tested the lure against real employees.
Stephanie Carruthers, Chief People Hacker for IBM X-Force Red, wrote in a blog post, “With only five simple prompts we were able to trick a generative AI model to develop highly convincing phishing emails in just five minutes — the same time it takes me to brew a cup of coffee. It generally takes my team about 16 hours to build a phishing email, and that’s without factoring in the infrastructure set-up."
"So, attackers can potentially save nearly two days of work by using generative AI models. And the AI-generated phish was so convincing that it nearly beat the one crafted by experienced social engineers, but the fact that it’s even that on par, is an important development.”
The researchers found that the human-crafted phish was only three percent more successful than the AI-generated one.
“Humans may have narrowly won this match, but AI is constantly improving,” Carruthers says. “As technology advances, we can only expect AI to become more sophisticated and potentially even outperform humans one day. As we know, attackers are constantly adapting and innovating. Just this year we’ve seen scammers increasingly use voice clones generated by AI to trick people into sending money, gift cards or divulge sensitive information.”
The researchers add that organizations should teach employees that AI-generated phishing emails may have flawless grammar and spelling.
“Dispel the myth that phishing emails are riddled with bad grammar and spelling errors,” Carruthers writes. “AI-driven phishing attempts are increasingly sophisticated, often demonstrating grammatical correctness. That’s why it’s imperative to re-educate our employees and emphasize that grammatical errors are no longer the primary red flag. Instead, we should train them to be vigilant about the length and complexity of email content. Longer emails, often a hallmark of AI-generated text, can be a warning sign.”
KnowBe4 enables your workforce to make smarter security decisions every day. Over 65,000 organizations worldwide trust the KnowBe4 platform to strengthen their security culture and reduce human risk.
IBM SecurityIntelligence has the story.
Here's how it works:
