Researchers at Sophos have found that the criminal market for malicious generative AI tools is still disorganized and contentious.
While there are obvious ways to abuse generative AI, such as crafting phishing emails or writing malware, criminal versions of these tools are still unreliable.
The researchers found numerous malicious generative AI tools on the market, including WormGPT, FraudGPT, XXXGPT, Evil-GPT, WolfGPT, BlackHatGPT, DarkGPT, HackBot, PentesterGPT, PrivateGPT. These tools have received mixed reactions in the criminal community, however.
“Some users were very keen to trial or purchase them, but many were doubtful about their capabilities and novelty,” the researchers write. “And some were outright hostile, accusing the tools’ developers of being scammers.”
WormGPT, one of the earlier malicious ChatGPT alternatives, shut down in August 2023, with its developers stating, “At the end of the day, WormGPT is nothing more than an unrestricted ChatGPT. Anyone on the internet can employ a well-known jailbreak technique and achieve the same, if not better, results.”
The researchers conclude, “On the whole – at least in the forums we examined for this research, and counter to our expectations – LLMs don’t seem to be a huge topic of discussion, or a particularly active market relative to other products and services. Most threat actors are continuing to go about their usual day-to-day business, while only occasionally dipping into generative AI. That being said, the number of GPT-related services we found suggests that this is a growing market, and it’s possible that more and more threat actors will start incorporating LLM-enabled components into other services too."
"Ultimately, our research reveals that many threat actors are wrestling with the same concerns about LLMs as the rest of us, including apprehensions about accuracy, privacy, and applicability. But they also have concerns specific to cybercrime, which may inhibit them, at least at the moment, from adopting the technology more widely.”
KnowBe4 enables your workforce to make smarter security decisions every day. Over 65,000 organizations worldwide trust the KnowBe4 platform to strengthen their security culture and reduce human risk.
Sophos has the story.
Here's how it works:
