Threat actors are increasingly leveraging AI hallucinations to plant phishing links and other malicious content in AI output, IEEE Spectrum reports. Large language models (LLMs) sometimes fabricate information, including web domains, when answering users’ questions. Attackers are registering these hallucinated web domains to host phishing pages.
Security researcher Seth Michael Larson, who dubbed this technique “slop squatting,” told IEEE Spectrum that these attacks exploit the trust users place in AI tools.
“It’s targeting a pattern in what an LLM is recommending,” Larson explained. “For lots of users asking for a particular resource, the model will answer with the same hallucination some percentage of the time. Users implicitly will take what an AI tells them to be authoritative. They’ll assume some security trust decisions have happened. But they haven’t.”
Larson advises users to treat AI results with vigilance, just as they would with unknown websites that appeared in search engine results.
“My recommendation with any AI output is: You should not be trusting this,” Larson says. “As in any engineering field, you can’t just go off of vibes. You have to have a way to verify. How do we know that this result is correct? Have that in your mind when you’re using these tools.”
Larson added, “We’re not giving enough warnings to users that the output is not something you should just blindly accept. People need to understand: The trust boundary between this information and your computer is you.”
IEEE has the story: https://spectrum.ieee.org/ai-cyberattacks-llm-slop-squatting
