The AI revolution of the 2020s has positioned Large Language Models (LLMs) as the new foundation for digital transformation. With unparalleled data processing power and text comprehension, LLMs can automate complex tasks that were previously manual and time-consuming, making them highly scalable. Unfortunately, this same efficiency is being applied to social engineering.
Key takeaways
- Dark LLMs such as WormGPT, FraudGPT and KawaiiGPT are built without safeguards and sold on Telegram and dark web forums to automate phishing, BEC and malware.
- The same capability now produces voices and faces, and CrowdStrike recorded a 134% rise in vishing intrusions between 2024 and 2025.
- Spotting bad grammar no longer protects anyone, so train employees to notice requests that skip a step and test them against current AI-driven attacks.
Recent research from Palo Alto Networks’ Unit 42 highlights that the emergence of "Dark LLMs", malicious AI models, is transforming the cybercrime landscape, enhancing social engineering in ways never before achieved.
What are Dark LLMs?
As Unit 42 points out, "Any tool powerful enough to build a complex system can also be repurposed to break one." This underscores the dual-use dilemma of AI. While LLMs are built for positive innovation, they can be weaponized by bad actors to streamline malicious workflows.
LLMs excel at linguistic precision, crafting grammatically perfect, contextually relevant, and psychologically manipulative lures that elevate Phishing, Vishing (voice phishing), and Business Email Compromise (BEC). They also demonstrate code fluency, allowing attackers to generate and debug harmful scripts or custom malware rapidly.
Models like WormGPT, FraudGPT, and KawaiiGPT are purposely built without ethical safeguards or security filters. Sold on Telegram and Dark Web forums, these "Dark LLMs" allow cybercriminals to bypass traditional guardrails to build sophisticated, massive-scale attacks.
Top Benefits of Dark LLMs for Cybercriminals
As it’s getting smarter, AI is also getting easier to abuse. The same tools that write emails and code for legit work are now being twisted into weapons.
- AI produces highly accurate language to mimic the communication styles of executives or financial institutions, exploiting victim trust. This allows bad actors to deploy hyper-realistic social engineering schemes, at scale
- These tools allow for the instant production of phishing kits and data theft scripts, lowering the technical barrier to entry. Threat groups can engineer and commercialize malware at unprecedented scales.
- Low-capacity actors with limited resources can now carry out digital scams and extortion, transforming cybercrime into a low-cost, high-frequency operation. Dark LLMs are democratizing cybercrime.
Weaponized LLMs Are Coming for You
Dark AI models like WormGPT and KawaiiGPT strip out ethics and safeguards, making it easy for criminals to launch real attacks. You don’t need to be elite anymore, just willing. The biggest risks break down into three buckets:
1. Social Engineering on Steroids
These LLMs are really good at crafting convincing social engineering messages. They can produce phishing and business email compromise messages that sound polished, confident, and totally legit, at scale. WormGPT can fake a CEO’s tone well enough to slip past email filters. KawaiiGPT pumps out “Urgent: Verify Your Account” messages with fake links that steal passwords. Without broken spelling or obvious, usual red flags, easily getting past firewalls.
WormGPT’s interface works like ChatGPT to craft social engineering artifacts.
2. Ransomware Made Easy
Dark LLMs don’t just write threats, they write the malware too. WormGPT can spit out ransomware code in seconds, encrypt files, phone home through Tor, and drop scary ransom notes with countdown timers. KawaiiGPT does the same thing, complete with “YOUR FILES ARE ENCRYPTED” messages and Bitcoin payment instructions. What used to take real skill now takes a prompt.
3. Malware for the Masses
These tools are basically malware starter kits. Even low-skill attackers can retrieve chunks of malicious code and stitch them together. Earlier versions of WormGPT already helped people write harmful scripts; newer ones go further, handing out ready-to-use building blocks for custom malware. The bar to entry has fallen through the floor.
Dark LLMs are making the cyber threat landscape nastier by the day. Tools like KawaiiGPT quietly spit out Python scripts to snoop through EML files (plain text versions of emails) and leak them out without raising alarms. Even worse, Palo Alto Networks recently caught an AI-powered attack where shady webpages hijack LLM APIs to crank out polymorphic JavaScript in real time right inside a victim’s browser. With carefully engineered prompts that dodge safety rails, attackers assemble full-blown phishing pages on the fly. Bottom line: LLMs are supercharging stealthy, hard-to-kill cyber attacks, and defenders are stuck playing catch-up with tech that cuts both ways.
4. Beyond text: voice and video
The three buckets above all end in something written. The more consequential shift is that the same generative capability now produces a voice and a face.
Voice synthesis removes the constraint that has capped voice phishing until now, which was operator hours. One person could run one call at a time, in one accent, with one script. That limit is gone, and the numbers follow: CrowdStrike recorded a 134% rise in vishing intrusions between 2024 and 2025, and Google attributes 23% of cloud compromises to voice phishing calls.
Video is the same story one step further on. A deepfaked executive on a Teams or Zoom call defeats the verification habit most finance teams actually rely on, which is not a policy but a glance: I saw them, it was them.
The defensive consequence matters more than the technology. An employee trained only to spot a badly written email has been trained against the one weakness Dark LLMs removed first.
How to protect your team
While cybersecurity gatekeepers highlight the importance of establishing standards and frameworks to govern the proliferation of dangerous AI models, the first and most practical layer of defense remains human awareness. Organizations must train employees on a recurring basis to proactively identify sophisticated phishing attacks and continuously strengthen their cybersecurity posture.
Phishing detection has become significantly more challenging with the rise of LLMs. Despite increasingly efficient and sophisticated attacks, however, the core indicators of modern phishing remain unchanged:
- Emotional triggers: Attackers exploit fear, urgency, or curiosity to override rational decision-making.
- Requests that bypass process: Victims are pressured to break policy: share credentials, ignore multi-party approvals, or install software outside approved IT channels.
Both indicators survive the arrival of AI precisely because neither depends on the quality of the writing. This is the single most useful thing to teach: stop training people to spot bad grammar, and start training them to notice when a request asks them to skip a step.
This is why continuous training, paired with advanced simulations that mirror real-world attacker techniques, is critical. By combining awareness of emerging AI-driven threat vectors, social engineering tactics, and clearly defined security processes, organizations can materially reduce their cyber risk.
What this means for regulated firms
For financial services the exposure is doubled: AI-assisted BEC is aimed directly at payment authority, and the regulatory expectation is that awareness training reflects current threats rather than a fixed annual curriculum.
Under DORA, ICT risk awareness training is an explicit obligation and firms are expected to evidence it. A program that tests employees on the phishing patterns of three years ago, in a year when the lures are machine-generated and the calls are synthetic, is difficult to present as current. The practical answer is to test against techniques in circulation now and keep the susceptibility and reporting data as the evidence.
Learn how to detect modern phishing →
Frequently Asked Questions
-
A Dark LLM is a large language model built or modified without safety guardrails, sold to cybercriminals for generating phishing emails, malware and fraud scripts. WormGPT, FraudGPT and KawaiiGPT are the best-known examples, distributed through Telegram and dark web forums.
-
Producing business email compromise and phishing messages that convincingly imitate an executive's tone, and generating ransomware and data theft code. Its interface works like a standard chat assistant, which is what removes the skill barrier.
-
Not by the signals most people were taught. Spelling errors, awkward phrasing and generic greetings are exactly what generative models fix. What survives is behavioral: emotional pressure and requests that ask someone to bypass an established process.
-
By testing them against AI-generated attacks. Simulations that use the same generative techniques, across email, voice and video, build a verification reflex that does not depend on spotting a writing mistake.
Train against the attacks being generated this quarter
The lures your team was trained to spot no longer look like the lures they will receive. Test them on AI-written phishing, synthetic voice calls with vishing simulation and deepfake video meetings with deepfake simulation, and see who verifies.
