The Machines Are Phishing Us: How Generative AI Is Fueling the Next Generation of Cyberattacks

Phishing emails used to be easy to spot. They were riddled with spelling mistakes, awkward phrasing, and improbable claims—often promising a fortune from a foreign prince or asking for help transferring large sums of money. Most people learned to dismiss them with a roll of the eyes.

Today, those poorly written scams have evolved into polished, personalized, and highly persuasive messages that can fool even seasoned professionals. Increasingly, these phishing emails aren’t written by humans—they’re crafted by generative AI.

Generative AI has brought tremendous benefits across industries, but it’s also changing the face of cybercrime. From automated social engineering to deepfake impersonations, cybercriminals now leverage generative AI to launch smarter, faster, and more convincing attacks. For organizations of every size, the question isn’t if they’ll be targeted, but whether they’re prepared for this new breed of threats.

Phishing Reimagined by Generative AI

Traditional phishing relied on scale: cast a wide net, catch a few victims. Generative AI has flipped that approach. Machine learning models now enable attackers to target individuals with precision—pulling data from public profiles, previous breaches, or company websites to create messages that appear both relevant and urgent.

Generative AI is fueling the rise of Phishing-as-a-Service platforms, where polished kits are sold for just a few hundred dollars, making industrial-scale cybercrime accessible to almost anyone. One such kit, Darcula, now supports phishing lures in over 100 languages and includes preloaded templates for more than 200 global brands. These kits are incredibly effective—they’re engineered to pass as real login pages for major platforms like Apple, Facebook, and Microsoft, dramatically increasing the likelihood of victim engagement. Recently, Darcula received a major generative AI-powered upgrade that allows attackers to customize campaigns, evade detection, and generate dynamic messages that appear entirely authentic.

Another growing threat is Flowerstore, a fully operational phishing marketplace complete with a slick interface, subscription options, and customer support. This platform offers customizable lures, automated delivery, and analytics dashboards that mirror legitimate SaaS marketing tools. Flowerstore even features affiliate programs—turning cybercrime into a scalable, revenue-generating business. In one recent campaign, Flowerstore operators managed to steal over 20,000 credentials in just a few days.

But the deception doesn’t stop at email. Generative AI-generated voice phishing, or “vishing,” is becoming a frontline tactic. Attackers now use voice clones of company executives to call employees in real time—giving instructions to transfer funds, share credentials, or override internal controls. These aren’t pre-recorded messages; they’re dynamic conversations, powered by synthetic voice models trained on public video or audio.

The result is a surge in more sophisticated social engineering tactics, blurring the line between real and fake. Recent reports indicate 1 in 3 employees have been targeted by generative AI-driven phishing or vishing attacks, and losses linked to these campaigns are rapidly climbing worldwide.

Understanding the Risk

The financial and reputational damage from generative AI-enhanced attacks is staggering. Beyond the immediate cost of fraud, organizations face regulatory fines, operational disruptions, and a loss of trust among customers and partners.

What makes generative AI-powered phishing especially dangerous is their subtlety. These attacks often bypass traditional security filters, appear perfectly legitimate, and exploit the human instinct to trust familiar names and patterns. The margin for error is slim—and a moment of inattention is all it takes.

Why Layered Security Matters

There is no silver bullet in cybersecurity—especially against intelligent, adaptive threats. That’s why experts emphasize layered security: combining controls across people, processes, and technology to create depth in defense.

Think of it as safety nets. If an attacker gets past one layer—say, an email filter—multifactor authentication (MFA), behavioral monitoring, and real-time incident response help contain the damage. The more layers in place, the harder it is for threats to succeed.

This approach is critical against generative AI-driven attacks, which exploit every possible angle. Layered defenses ensure that even if one control fails, others stand ready to respond.

Defending Against Machine-Driven Threats

Fortunately, the same generative AI powering attacks can be leveraged to stop them—if applied strategically.

Modern security platforms analyze email content, behavior, and context in real time, detecting subtle signs of manipulation. Identity protections like MFA—especially hardware tokens resistant to phishing—thwart attackers from abusing stolen credentials. Behavior-based monitoring learns “normal” activity for users and devices, automatically detecting and responding to anomalies.

Yet technology alone isn’t enough. Cybersecurity is as much about people and strategy as it is about systems.

Cybersecurity Is a Team Effort

Firewalls and filters are necessary, but must be supported by governance, response planning, user education, and a deep understanding of evolving threats.

That’s why partnering with experienced cybersecurity experts is critical. Organizations—especially those without in-house security teams—benefit from providers specializing in offensive and defensive strategies, real-time threat monitoring, penetration testing, and incident response.

A strong cybersecurity partner helps organizations understand their unique risks, align defenses with business goals, and ensure they’re prepared—not just protected.

Staying Ahead of the Curve

Generative AI in cybercrime doesn’t just raise the stakes—it changes the rules. Attackers now scale, adapt, and exploit both digital and human weaknesses with alarming precision. But we’re not powerless.

By embracing intelligent, adaptive defenses—both technological and human—we can level the playing field. Organizations must stop treating cybersecurity as a passive shield and start treating it as an active, evolving capability.

This means using systems that understand language and behavior, strengthening identity verification, training employees to be skeptical (not paranoid), and accepting that the fight against generative AI-driven attacks must itself be powered by generative AI and guided by experience.

Trust is one of the most valuable currencies in business today. Protecting it requires strategy, partnerships, and vigilance. Layered security, combined with the right cybersecurity partner, ensures that protection is proactive, practical, and built for what’s next.

CustosIQ, for example, works with enterprises of all sizes to build adaptive cybersecurity programs rooted in risk-driven frameworks. By combining offensive testing, continuous monitoring, and compliance remediation, CustosIQ helps organizations not just react to threats but anticipate and outmaneuver them. Their approach emphasizes that cybersecurity is a continuous, evolving discipline—not just a checklist or a set it and forget it tech stack.

About the authors:

Robert R. Ragan, Jr. and Isabelle Syring are experienced cybersecurity leaders at CustosIQ, a firm specializing in offensive security, proactive risk management, and adaptive defense strategies. CustosIQ helps organizations stay ahead of emerging threats through real-time monitoring, incident response, and strategic risk frameworks