AI-generated fraud schemes are now the dominant type of fraud, according to a new report from AU10TIX. AI-assisted forgeries overtook physical manipulation for the first time, as these tools allow attackers to fool humans and technology with very little manual effort.
“AI tools have made it possible to fabricate entire documents, generate convincing text fields, and produce synthetic facial imagery at scale with no physical access to a genuine document, no specialist equipment, and no physical trace,” the report says. “What distinguishes this category is not sophistication in isolation but production volume: these are not individual incidents but industrialized operations, often running in coordinated batches across multiple platforms simultaneously.”
Dating services are particularly vulnerable to AI-generated deepfake attacks. Threat actors target users of these sites to conduct a variety of long-running, targeted social engineering schemes.
“Entertainment and online dating platforms carry the highest deepfake detection failure rate of any industry: 4.29%. More than double the crypto industry's 1.84%. All five of the top attack subtypes are digital and AI-generated: Synthetic Pattern, Quality Manipulation, Text Deepfake, and selfie-based deepfake techniques. The motivation is specific. Dating platforms attract fraud because the incentive is not financial. It is social engineering. AI-generated profile photos and matching identity documents allow fraudsters to construct convincing fake personas at scale, for romance scams, catfishing, and account farming. The sophistication of deepfake attacks here is disproportionate to its relative size, which suggests a highly motivated and technically capable fraud population.”
To help combat this industrialization of synthetic identity fraud, organizations must deploy targeted, automated technical defenses alongside specialized simulation. To address this rapidly scaling operational vector, the KnowBe4 Platform provides the Deepfake Training Content Agent—part of our production-ready AIDA suite—which automatically generates custom deepfakes featuring a leader from your own organization to directly teach users how convincing AI-powered impersonations look and feel in real time.
AU10TIX has the story: Identity Fraud Has Industrialized: AU10TIX Finds AI-Generated Fraud Surpassed Physical Forgery for the First Time
