The AI revolution of the 2020s has positioned Large Language Models (LLMs) as the new foundation for digital transformation. With unparalleled data processing power and text comprehension, LLMs can automate complex tasks that were previously manual and time-consuming, making them highly scalable. Unfortunately, this same efficiency is being applied to social engineering.
Recent research from Palo Alto Networks’ Unit 42 highlights that the emergence of "Dark LLMs"—malicious AI models—is transforming the cybercrime landscape, enhancing social engineering in ways never before achieved.
What are Dark LLMs?
As Unit 42 points out, "Any tool powerful enough to build a complex system can also be repurposed to break one." This underscores the dual-use dilemma of AI. While LLMs are built for positive innovation, they can be weaponized by bad actors to streamline malicious workflows.
LLMs excel at linguistic precision, crafting grammatically perfect, contextually relevant, and psychologically manipulative lures that elevate Phishing, Vishing (voice phishing), and Business Email Compromise (BEC). They also demonstrate code fluency, allowing attackers to generate and debug harmful scripts or custom malware rapidly.
Models like WormGPT, FraudGPT, and KawaiiGPT are purposely built without ethical safeguards or security filters. Sold on Telegram and Dark Web forums, these "Dark LLMs" allow cybercriminals to bypass traditional guardrails to build sophisticated, massive-scale attacks.
Top Benefits of Dark LLMs for Cybercriminals
As it’s getting smarter, AI is also getting easier to abuse. The same tools that write emails and code for legit work are now being twisted into weapons.
- AI produces highly accurate language to mimic the communication styles of executives or financial institutions, exploiting victim trust. This allows bad actors to deploy hyper-realistic social engineering schemes, at scale
- These tools allow for the instant production of phishing kits and data theft scripts, lowering the technical barrier to entry. Threat groups can engineer and commercialize malware at unprecedented scales.
- Low-capacity actors with limited ressources can now carry out digital scams and extortion, transforming cybercrime into a low-cost, high-frequency operation. Dark LLMs are democratizing cybercrime.
Weaponized LLMs Are Coming for You
Dark AI models like WormGPT and KawaiiGPT strip out ethics and safeguards, making it easy for criminals to launch real attacks. You don’t need to be elite anymore, just willing. The biggest risks break down into three buckets:
1. Social Engineering on Steroids
These LLMs are really good at crafting convincing social engineering messages. They can produce phishing and business email compromise messages that sound polished, confident, and totally legit, at scale. WormGPT can fake a CEO’s tone well enough to slip past email filters. KawaiiGPT pumps out “Urgent—-Verify Your Account” messages with fake links that steal passwords. Without broken spelling or obvious, usual red flags, easily getting past firewalls.
WormGPT’s interface works like ChatGPT to craft social engineering artifacts.
2. Ransomware Made Easy
Dark LLMs don’t just write threats, they write the malware too. WormGPT can spit out ransomware code in seconds, encrypt files, phone home through Tor, and drop scary ransom notes with countdown timers. KawaiiGPT does the same thing, complete with “YOUR FILES ARE ENCRYPTED” messages and Bitcoin payment instructions. What used to take real skill now takes a prompt.
3. Malware for the Masses
These tools are basically malware starter kits. Even low-skill attackers can retrieve chunks of malicious code and stitch them together. Earlier versions of WormGPT already helped people write harmful scripts; newer ones go further, handing out ready-to-use building blocks for custom malware. The bar to entry has fallen through the floor.
Dark LLMs are making the cyber threat landscape nastier by the day. Tools like KawaiiGPT quietly spit out Python scripts to snoop through EML files (plain text versions of emails) and leak them out without raising alarms. Even worse, Palo Alto Networks recently caught an AI-powered attack where shady webpages hijack LLM APIs to crank out polymorphic JavaScript in real time right inside a victim’s browser. With carefully engineered prompts that dodge safety rails, attackers assemble full-blown phishing pages on the fly. Bottom line: LLMs are supercharging stealthy, hard-to-kill cyber attacks, and defenders are stuck playing catch-up with tech that cuts both ways.
How to protect your team
While cybersecurity gatekeepers highlight the importance of establishing standards and frameworks to govern the proliferation of dangerous AI models, the first and most practical layer of defense remains human awareness. Organizations must train employees on a recurring basis to proactively identify sophisticated phishing attacks and continuously strengthen their cybersecurity posture.
Phishing detection has become significantly more challenging with the rise of LLMs. Despite increasingly efficient and sophisticated attacks, however, the core indicators of modern phishing remain unchanged:
- Emotional triggers: Attackers exploit fear, urgency, or curiosity to override rational decision-making.
- Requests that bypass process: Victims are pressured to break policy: share credentials, ignore multi-party approvals, or install software outside approved IT channels.
This is why continuous training, paired with advanced simulations that mirror real-world attacker techniques, is critical. By combining awareness of emerging AI-driven threat vectors, social engineering tactics, and clearly defined security processes, organizations can materially reduce their cyber risk.
