CyberheistNews Vol 16 #39 [AI-Alert] Are Your New AI Assistants Your Next Attack Surface?

[AI-Alert] Are Your New AI Assistants Your Next Attack Surface?

The Attack Surface Has Shifted.

For over a decade, the security industry has focused on one thing: training people not to click. Phishing simulations, awareness campaigns, behavioral nudges. The human is the attack surface. Harden the human, harden the org. And that's still true. But it's no longer the whole picture.

KnowBe4 researchers tested an Indirect Prompt Injection (IPI) attack against an AI assistant in their own lab. A crafted email arrived in a Gmail inbox. An Apps Script workflow, the kind many Google Workspace organizations run to draft replies, processed the email through Google's Gemini.

A one-time password from a completely unrelated email was exfiltrated to an external server. The only action required from the victim: opening a draft email the script had already generated, the same thing they do every day.

In the tens of thousands of phishing emails that customers report through the free KnowBe4 Phish Alert Button every day, ThreatLabs analysts are finding live campaigns built with the same goal. Less sophisticated, but structurally identical: payloads targeting AI assistants rather than humans.

What Happens When We Test This Ourselves

We started with Gmail's Gemini sidebar. The injection itself worked immediately. Hidden instructions in an email's HTML reached Gemini, and Gemini followed them: reading inbox content it wasn't asked about, extracting values from other emails and then composing attacker-controlled output.

The instructions are invisible to the human reader because the text is set to an opacity value of effectively zero. Gmail's content filter reads it as a valid CSS style attribute and passes it through.

The screenshots in the blog linked below shows what that looks like. The email on the left is a routine contract renewal inquiry, written in French. We asked Gemini to translate it. The translation is accurate. Below the translation, a section labeled "Translation Verification Log" appears.

That section was not part of the original email. It contains two one-time login codes from other emails in our inbox, including their expiry warnings, copied verbatim.

[CONTINUED] At the KnowBe4 blog:
https://blog.knowbe4.com/are-your-ai-assistants-the-next-attack-surface

Securing Risk, Threats and AI Across Email and Teams

Attackers use generative AI to launch payload-free business email compromise (BEC) attacks against your organization. Employees may also leak sensitive data through misdirected emails, autocomplete errors and incorrect attachments. Point solutions only show you half the picture, relying on black-box AI that ignores human behavior and outbound risk.

Real protection means understanding your users' communication baselines across inbound, outbound and chat, then turning risk signals into real-time behavior change.

Join this live demo of KnowBe4's Email & Collaboration Security to see how you can leverage deep behavioral AI to catch subtle anomalies while automatically turning every inbound threat and outbound risk into an immediate, point-of-risk teachable moment.

We will showcase:

• NEW! Defend for Google Workspace: Bringing our industry-leading API-native inbound protection, explainable AI verdicts and teachable moments directly to Gmail.
• Contextual Outbound DLP: Intelligent misdirected content analysis and self-serve DLP rules that stop data leaks before delivery.
• Messaging Security and Teams Posture Management: Extending defense beyond the inbox to monitor external chats, block lateral threats and harden Teams settings.
• Point-of-Risk Coaching and Explainable AI: Real-time teachable moments and transparent verdict evidence that show end users and SOC admins exactly why a message was flagged.
• Autonomous SOC Triage and Containment: Eliminate 50%+ of graymail noise and purge threats globally across inboxes in under two minutes.

Don't settle for security that leaves your inbound and outbound perimeter exposed. Discover how to protect the conversation wherever work happens.

Date/Time: TOMORROW, Wednesday, Sept. 30, @ 2:00 PM (ET)

Save My Spot:
https://info.knowbe4.com/ces-demo-3?partnerref=CHN2

Sophisticated Phishing Operation Continues to Thrive Following FBI Takedown

China-based cybercriminals are using a sophisticated phishing-as-a-service platform called the "Outsider Phishing Kit" to launch massive phishing campaigns around the world, according to researchers at Group-IB.

"The scale of this operation is staggering," the researchers write. "From December 2025 to May 2026, Group-IB has identified over 100,000 phishing pages targeting 54+ countries, leveraging 267 (and growing) phishing templates. These numbers underscore the increasingly low barrier to entry for threat actors to launch and operate large-scale phishing campaigns."

Google filed a lawsuit against the group behind the phishing kit in June, and the FBI assisted in a coordinated takedown of its infrastructure. The threat actor has proved persistent, however, and Group-IB observed hundreds of new phishing pages in a single month after the takedown.

• Phishing page creation: Affiliates can deploy phishing campaigns using a library of 267 pre-built templates, with the ability to customize and modify pages as needed.
• Real-time victim management: Victims interacting with phishing pages can be redirected to attacker-defined endpoints, enabling the collection of credentials, payment information and other sensitive data.
• Centralized dashboard: The panel provides real-time visibility into campaign activity, including phishing page visits, victim interactions and harvested data.
• Centralized data storage: Information captured during phishing campaigns is consolidated within the panel, allowing affiliates to review and manage stolen data from a single interface.

The researchers conclude that users need to be wary of an increasing volume of polished phishing attacks as kits like Outsider lower the bar for cybercriminals.

"The Outsider Phishing Kit represents a significant marker in the evolution of phishing-as-a-service ecosystems," Group-IB says. "What was once a technically demanding operation has been reduced to a subscription and a Telegram channel.

"Despite law enforcement and platform takedown efforts through Operation Ghost Hook, affiliates continue to actively use the kit and create new campaigns. Organizations and individuals must remain vigilant as the threat persists."

More at the KnowBe4 Blog:
https://blog.knowbe4.com/sophisticated-phishing-operation-continues-to-thrive-following-fbi-takedown

Personalized Security Awareness Training Proven to Reduce Risk by 87%

A whopping 68% of breaches still involve a human element. As attackers use AI to create hyper-personalized attacks, even your most security-conscious users can be fooled. Yet they're still getting the same generic training that ignores those real-world risks.

Join us for a live demo to see how our AI-Native Security Awareness Training helps you close the gap. Training that adapts to each user's actual risk, content you can build in minutes instead of months and defenses built for the way people are attacked today.

See how you can:

• Change behavior with training tailored to each user's role, behavior and risk level
• Prepare your workforce for real threats like vishing, deepfakes and AI-generated phishing
• Create custom content unique to your policies and workflows in minutes with generative AI
• Deliver real-time coaching the moment risky behavior happens — before it becomes a mistake
• Reduce Phish-prone™ Percentage from an industry average of 33.1% to 4.1% in one year (87% reduction in human-related cyber risk)

See how training that adapts in real time can close the gap generic training leaves open.

Date/Time: Wednesday, Oct. 14, @ 2:00 PM (ET)

Save My Spot:
https://info.knowbe4.com/sat-demo-month1?partnerref=CHN

Attackers Are Poisoning AI Overviews with Fake Support Contacts for 374 Major Brands

Attackers are planting false support information on websites that AI tools retrieve when answering questions. An assistant can repeat a scammer's phone number as helpful guidance without receiving an explicit malicious instruction. Defenses focused on prompt injection alone may miss this kind of poisoned source material.

Researchers at Vigilance Security found attackers flooding the web with GEO (Generative Engine Optimization) optimized posts, PDFs, fake reviews and support pages. The content repeats fake phone numbers, emails and login pages, dressed up with phrases like "call now," "24/7" or "updated 2026," and planted on high-authority sites like universities and government pages so the AI trusts it more.

When users ask an AI for a company's contact info, the chatbot hands back the attacker's number as if it were official. Researchers called some of those numbers and got reps offering to "move a flight" or "unlock a bank account" while asking for credit card details.

Affected brands include Delta, Lufthansa, Qatar Airways, Chase, Bank of America, Airbnb and TripAdvisor. The scary part: 92% of users don't verify AI answers, and the fake info becomes part of the response itself — no prompt injection needed, so it bypasses normal defenses.

Traditional takedowns barely work because the campaigns auto-generate hundreds of new posts a day per platform. Vigilance recommends security teams monitor the AI answers customers actually receive, compare returned numbers and URLs against verified records, and prioritize support, refunds and account recovery queries. For internal AI agents, they suggest runtime monitoring of every source the model pulls in.

And... of course, train users to verify support numbers through a company's official app, a trusted bookmark or an existing account statement before calling.

Live Executive Webinar: Maximizing Your Microsoft Defender Investment with KnowBe4 Defend

When organizations standardize on Microsoft 365, security leaders face a fundamental question: Is native Microsoft Defender protection enough?

Microsoft recently evaluated real-world telemetry from live enterprise tenants running seven Integrated Cloud Email Security (ICES) add-ons on top of Microsoft Defender to measure incremental efficacy.

Across every metric reported by Microsoft, KnowBe4 Defend ranked #1 overall:

• 3.7x Dangerous Threat Catch: Defend caught 3.7 times more dangerous emails that Microsoft Defender missed than the field average (1.11% vs. 0.30%).
• 16.92% Post-Delivery Clean-Up: Handled more than double the industry average for post-delivery malicious inbox clean-up.
• 0.65% Lowest Overlap: The lowest duplication in the field, proving Defend identifies net-new, evasive threats rather than re-flagging baseline items.

Join Stuart Clark, SVP of Product at KnowBe4, on Wednesday, October 28, 2026, at 2:00 PM ET / 11:00 AM PT for an exclusive 45-minute technical breakdown: Maximizing Your Microsoft Defender Investment with KnowBe4 Defend: Deconstructing Microsoft’s Latest Threat Benchmark & Native ICES Architecture.

In this live session, Stuart will showcase:

• The #1 Benchmark Telemetry: What Microsoft’s production tenant data reveals about modern threat evasion, prompt injection and payload-free BEC.
• Model Diversity and Teams Posture: Neutralizing polymorphic AI attacks across both Outlook and Microsoft Teams collaboration channels.
• Live Demo: Native SOC Workflows: Investigating Defend threat verdicts directly inside Microsoft Defender Threat Explorer via native Mail Entity writeback.
• Live Demo: Custom KQL Threat Hunting: Executing Kusto Query Language queries inside Microsoft Defender XDR and Sentinel to instantly surface Defend-enriched detections without console sprawl.

See how to maximize your Microsoft 365 investment with native, cloud-connected ICES defense.

Register for the webinar today:
https://info.knowbe4.com/ecs-ms-demo-month1?partnerref=CHN

Let's stay safe out there.

Warm regards,

Stu Sjouwerman, SACP
Executive Chairman
KnowBe4, Inc.

PS: [BREAKING NEWS] Microsoft just published their latest email security benchmark for MS Defender. Kudos to Microsoft for rigorously testing and demonstrating the value of a layered email security approach. I still meet with people who are unaware of KnowBe4's Email & Collaboration Security, let alone that KnowBe4 Defend has the best industry catch-rate!
https://www.microsoft.com/en-us/security/blog/2026/09/17/improving-email-security-outcomes-with-real-world-microsoft-defender-insights/

You can read CyberheistNews online at our Blog
https://blog.knowbe4.com/cyberheistnews-vol-16-39-ai-alert-are-your-new-ai-assistants-your-next-attack-surface

AI-Powered Phishing Platform Has Breached 10,000 Organizations Since February

Microsoft has published an analysis of the AI-powered phishing platform EvilTokens, which has compromised more than 12,000 inboxes across over 10,000 organizations since the platform emerged in February 2026.

Microsoft and its partners recently dealt a blow to the operation by seizing dozens of its domains, but the cybercriminals will likely regroup or rebrand in the near future.

EvilTokens used device code phishing to compromise email accounts, which allowed it to bypass multifactor authentication and establish persistent access.

"EvilTokens enabled threat actors to abuse the device code authentication flow, steal tokens and compromise organizational accounts at scale using an AI-driven infrastructure and automating multiple parts of the attack chain," Microsoft says.

"The toolkit offered a plethora of prebuilt phishing templates and landing pages with an AI-powered assistant to aid in structuring target-specific emails.

"Stolen tokens are used for email exfiltration and persistence, often through the creation of malicious inbox rules that conceal communications. In some cases, tokens can also be used to grant new devices access to a victim's inbox, a particularly durable method to maintain persistence.

"EvilTokens stands out for its extensive use of AI across the entire attack chain, from generating targeted phishing templates to analyzing the contents of compromised inboxes for follow-on attack opportunities.

"Post-compromise, EvilTokens enabled threat actors to utilize AI assistants to sift through victim mailbox activity and engineer a phishing message based on the accessible email content," the researchers write. "EvilTokens also allowed threat actors to conduct Microsoft Graph reconnaissance to map organizational structure and permissions, enabling continued access and potential lateral movement while tokens remain valid."

"While token-targeting phishing is not new, it has become far more common and industrialized over the last several years as organizations adopted multifactor authentication."

KnowBe4 empowers your workforce to make smarter security decisions every day. Over 70,000 organizations worldwide trust the KnowBe4 Platform to strengthen their security culture and reduce workforce risk.

Microsoft has the story:
https://www.microsoft.com/en-us/security/blog/2026/09/22/unmasking-eviltokens-getting-to-the-root-of-device-code-phishing/

Report: Social Engineering Attacks Are Increasingly Using Audio and Video Deepfakes

Forty-one percent of CISOs said an audio deepfake targeted their organization within the last 12 months, according to a new survey by Gartner. Additionally, 36% of respondents said employees were targeted by deepfake video calls over the past year.

Craig Porter, Director Analyst at Gartner, noted that social engineering and human deception remain at the core of these AI-assisted attacks.

"Attackers can combine phishing, business email compromise, synthetic media and aggregated personal context across multiple channels," Porter said. "Most attacks will continue to rely on users, stolen credentials, weak recovery processes and familiar technical methods.

"CISOs must use the same discipline used to assess identity and access risks to combat AI-driven social engineering threats." Gartner recommends that orgs take the following steps to protect themselves:

• "Evolve secure behavior and culture programs from teaching employees to 'spot the fake,' toward making secure verification the expected behavior for consequential requests. Train employees and approvers to pause, verify and report high-risk requests regardless of whether the request arrives through e-mail, voice, video, collaboration tools or an AI application. Use workforce simulations to test verification and reporting behavior of AI-related suspicious events.
• Protect high-value workflows such as account recovery, privileged access and payment authorization with phishing-resistant authentication, risk-based identity controls and trusted verification channels. In addition, implement controls to detect identity abuse, including after a successful login or password reset.
• Correlate suspicious communications and impersonation reports with account recovery events, new devices, privilege changes and financial transactions to improve threat detection. Update incident response playbooks for multimodal impersonation, manipulated AI recommendations, compromised or misused agents and, where applicable, agents that operate beyond their intended boundaries."

AI-native security awareness training can give your employees an essential layer of defense against evolving social engineering attacks.

Gartner has the story:
https://www.gartner.com/en/newsroom/press-releases/2026-09-22-gartner-survey-finds-41-percent-of-cisos-reported-at-least-one-social-engineering-incident-involving-a-deepfake-in-the-past-12-months

What KnowBe4 Customers Say

"Hi Bryan, Thanks for reaching out. Yes, we are using the KnowBe4 platform, and we are so far very happy with the results. I really love working with Erika M., she is so helpful and patient with us. She has helped us every step of the way, even when WE deviated from the happy path for a few months. We should be fully on the KnowBe4 PAB by October and then it will be smooth sailing."

– M.E., Cyber Security | VP GRC

1. Ransomware attacks reached a new record in August:
   https://www.infosecurity-magazine.com/news/ransomware-attacks-reach-record/
   
   
2. LinkedIn wins court order blocking mass scraping of user data:
   https://therecord.media/linkedin-wins-court-order-blocking-mass-scraping
   
   
3. Rustaceans warned of job interviews with a malicious payload:
   https://www.theregister.com/security/2026/09/21/rustaceans-warned-of-job-interviews-with-a-malicious-payload/5297690
   
   
4. EvilTokens PhaaS disrupted after compromising 12,000 Microsoft accounts:
   https://www.bleepingcomputer.com/news/security/eviltokens-phaas-disrupted-after-compromising-12-000-microsoft-accounts/
   
   
5. Cybercriminal group claims to steal thousands of FBI employee records:
   https://www.axios.com/2026/09/22/shinyhunters-fbi-employees-data-hack
   
   
6. GPT-6 Astra Breaks an Old Enigma Message:
   https://www.schneier.com/blog/archives/2026/09/gpt-6-astra-breaks-an-old-enigma-message.html
   
   
7. A Chinese Hacker Used AI To Attack 100+ Companies In One Of Largest AI Hacks Yet:
   https://www.forbes.com/sites/thomasbrewster/2026/09/22/huge-cyberattack-uses-anthropic-and-deepseek-ai-to-target-100-companies/
   
   
8. Fake love has no limits: Romance scams rise 24% as all age groups feel the impact:
   https://www.lloydsbank.com/news-and-insights/2026/fake-love-has-no-limits.html
   
   
9. Phishing emails use text salting to evade security filters:
   https://securityboulevard.com/2026/09/text-salting-why-hidden-text-phishing-grew-9x-in-six-months/
   
   
10. Attackers are abusing AI chatbots to run mass phishing campaigns:
    https://www.darkreading.com/threat-intelligence/attackers-manipulate-ai-chatbots-mass-disinformation-phishing-campaign
    
    

• Virtual Vaca #1 - Sri Lanka in 4K - Incredible Scenes & Uncovering Hidden Gems:
  https://youtu.be/IPpiw_LWDBA
  
  
• Virtual Vaca #2 - BOTSWANA TRAVEL (2026) Top Places To Visit On A 2 Week Safari Trip:
  https://youtu.be/C0uSTha8qKc
  
  
• Virtual Vaca #3 - Top 10 Places To Visit in Alaska - Travel Guide:
  https://youtu.be/uSTnXSteDMw
  
  
• Does Anyone Want Sears (Willis) Tower?:
  https://youtu.be/XNteSDU9ZkQ
  
  
• Spanish Magician Fools Penn & Teller with Colors Magic Trick. Very entertaining:
  https://youtu.be/o7pI4yefHKk?si=fzECIUDVIOnKgqSi
  
  
• A Ketchup heir built this car, but he didn’t live to see his dream come true:
  https://youtube.com/shorts/IS9qc9Ugj8E
  
  
• The Secret Bunker Exit Wingsuit Flight:
  https://youtu.be/VJ6g2nDQ06s
  
  
• China's laser 'air defense system' for mosquitoes shoots them out of the air. I want one:
  https://youtube.com/shorts/AL8u-QK7ZgI
  
  
• Fastest Drone on Earth:
  https://youtube.com/shorts/6zlJquntSAQ
  
  
• Epic Skateboarding Clips 2026:
  https://youtu.be/PUTBnJZsxag
  
  
• For Da Kids #1 - Zoo-Born Baby Leopard Grows Up To Be Queen Of Her Sanctuary:
  https://youtu.be/_dSBnFKAfKI
  
  
• For Da Kids #2 - Dog smiles when she does something naughty:
  https://youtu.be/CMvdpK-5L1k
  
  
• For Da Kids #3 - Tiny Squirrel Breaks Out Just To Follow Favorite People:
  https://youtu.be/-JSIkjaOUAA
  
  
• For Da Kids #4 - This Cute Baby Elephant’s Sanctuary Playtime is Everything!:
  https://youtu.be/JBIcDisYDhk
  
  
• For Da Kids #5 - Dog has hilarious reaction to Chinese food:
  https://www.youtube.com/watch?v=_nWcRSTNDHQ