As I was writing my latest book, How AI and Quantum Impact Cyber Threats and Defenses, I was hit by how many theoretical and real attacks there are involving AI. There are attacks committed by AI and attacks committed agsinst AI, and I’m not sure which category is bigger.
Every attack type we have ever had (e.g., social engineering, vulnerability exploitation, authentication attacks, side channel attacks, etc.) is going to be worsened by AI-enabled attack tools and methodologies. They will be more persuasive, faster and more successful. AI-enabled social engineering, especially adding AI-created deepfake videos, is going to significantly ramp up social engineering. AI hack bots are going to exploit more vulnerabilities, create and find more zero days, and exploit a larger percentage of them (which currently sits at only 4% of total publicly announced vulnerabilities). And that’s saying a lot, because we had over 48,000 publicly announced vulnerabilities last year.
Another large category of attacks is attacks against AI technologies. While researching for the book, I became overwhelmed by all the traditional and new attacks against AI. AI will not only be attacking us, but will also be attacked by traditional methods and tools, and by AI-enabled tools. In fact, most of the news of new attacks involving AI are about attacks against AI, not by it.
Attacks against AI include:
- Prompt injections
- Data poisoning
- Context poisoning
- AI identity attacks
- Supply chain attacks
- Jailbreaking
- Abusing AI system prompts
- Model/weight manipulation
- Label poisoning
- Memory poisoning
- Improper input handling
- Improper output handling
- Excessive agency
- Unbounded consumption
- Attacks against AI browsers
- Attacks against AI-browser add-ins
- Privacy risks
- Ad-driven attacks
- API attacks
- MCP attacks
- A2A attacks
- Malicious models
- and more
There are so many attacks against AI that I had to break up AI-related attacks into two different chapters.
Conversely, quantum attacks are fairly straightforward. There are far fewer of them, mostly against quantum-susceptible cryptography, but widely applicable.
The sheer complexity of how AI is going to work (and is now already working) is going to make threat modeling and defending a lot harder. Just look at the list above. And that’s just the new stuff. You have to add all of that on top of all the existing traditional attacks, which will be used both by and againstAI technologies.
It is really why I decided to write this book.
Thinking about AI-related attacks, both by and against AI, really hurt my head. Trying to figure out all the needed defenses took a year of research and four months of heads-down writing.
My wife laughs recounting this story, but when I finally finished half the book on AI and started writing the quantum half, I told my wife how glad I was to get back to something I knew better, understood more, and could more easily write about. She replied, “Quantum is the easier part?”
Yeah, it was.
