[SCARY] Research Shows Weaponized GenAI Worm That Gets Distributed Via A Zero Click Phishing Email

ComPromptMizedIsraeli researchers came out with a hell of a thing just now. Here is a bit of the abstract and a video. YIKES.

In the past year, numerous companies have incorporated Generative AI (GenAI) capabilities into new and existing applications, forming interconnected Generative AI (GenAI) ecosystems consisting of semi/fully autonomous agents powered by GenAI services.

While ongoing research highlighted risks associated with the GenAI layer of agents (e.g., dialog poisoning, privacy leakage, jailbreaking), a critical question emerges: Can attackers develop malware to exploit the GenAI component of an agent and launch cyber-attacks on the entire GenAI ecosystem?

This is a 3-minute video that shows the whole thing.

Their paper introduces Morris II, the first worm designed to target GenAI ecosystems through the use of adversarial self-replicating prompts. The study demonstrates that attackers can insert such prompts into inputs that, when processed by GenAI models, prompt the model to replicate the input as output (replication) and engage in malicious activities (payload).

Additionally, these inputs compel the agent to deliver them (propagate) to new agents by exploiting the connectivity within the GenAI ecosystem. We demonstrate the application of Morris II against GenAI-powered email assistants in two use cases (spamming and exfiltrating personal data), under two settings (black-box and white-box accesses), using two types of input data (text and images).

The worm is tested against three different GenAI models (Gemini Pro, ChatGPT 4.0, and LLaVA), and various factors (e.g., propagation rate, replication, malicious activity) influencing the performance of the worm are evaluated.

Here is the site itself with the video and also the paper. You may need an incognito window to get there.



Download the 2024 Security Culture Report

Dive into the largest Security Culture Report, with insights from over 800,000 employees in 4,078 organizations across 18 industries. See where you stand in a global comparison of security practices.

Security Culture Report

Download Now!

PS: Don't like to click on redirected buttons? Cut & Paste this link in your browser:


Subscribe to Our Blog

Comprehensive Anti-Phishing Guide

Get the latest about social engineering

Subscribe to CyberheistNews