Identifying AI-Enabled Phishing

Stu Sjouwerman | Mar 22, 2023

AI phishingUsers need to adapt to an evolving threat landscape in which attackers can use AI tools like ChatGPT to craft extremely convincing phishing emails, according to Matthew Tyson at CSO.

“A leader tasked with cybersecurity can get ahead of the game by understanding where we are in the story of machine learning (ML) as a hacking tool,” Tyson writes. “At present, the most important area of relevance around AI for cybersecurity is content generation. This is where machine learning is making its greatest strides and it dovetails nicely for hackers with vectors such as phishing and malicious chatbots. The capacity to craft compelling, well-formed text is in the hands of anyone with access to ChatGPT, and that’s basically anyone with an internet connection.”

Tyson quotes Conal Gallagher, CIO and CISO at Flexera, as saying that since attackers can now write grammatically correct phishing emails, users will need to pay attention to the circumstances of the emails.

“Looking for bad grammar and incorrect spelling is a thing of the past — even pre-ChatGPT phishing emails have been getting more sophisticated,” Gallagher said. “We must ask: ‘Is the email expected? Is the from address legit? Is the email enticing you to click on a link?' Security awareness training still has a place to play here.”

Tyson explains that technical defenses have become very effective, so attackers focus on targeting humans to bypass these measures.

“Email and other elements of software infrastructure offer built-in fundamental security that largely guarantees we are not in danger until we ourselves take action,” Tyson writes. “This is where we can install a tripwire in our mindsets: we should be hyper aware of what it is we are acting upon when we act upon it. Not until an employee sends a reply, runs an attachment, or fills in a form is sensitive information at risk. The first ring of defense in our mentality should be: ‘Is the content I’m looking at legit, not just based on its internal aspects, but given the entire context?’ The second ring of defense in our mentality then has to be, ‘Wait! I’m being asked to do something here.’”

New-school security awareness training with simulated phishing tests can enable your employees to recognize increasingly sophisticated phishing attacks.

CSO has the story.

Discover Your Organization’s Phish-prone™ Percentage

Ninety-one percent of data breaches begin with spear phishing. Launch our Free Phishing Security Test for up to 100 users to uncover your team's vulnerability and see how your security posture stacks up against industry benchmarks.

Get Your Free Phishing Security Test

Secure the Digital Workforce: Human + AI

KnowBe4 empowers the modern workforce to make smarter security decisions every day. Trusted by more than 70,000 organizations worldwide, KnowBe4 is the pioneer of digital workforce security, securing both AI agents and humans. The KnowBe4 Platform provides attack simulation and training, collaboration security, and agent security powered by AIDA (Artificial Intelligence Defense Agents) and a proprietary Risk Score. The platform leverages 15 years of behavioral data to combat advanced threats including social engineering, prompt injection, and shadow AI. By securing humans and agents, KnowBe4 leads the industry in workforce trust and defense.