AI Threat Vectors Part 1: Voice Is the Next Attack Surface, and It Needs a Firewall

Voice Is the New Keyboard, and No One Is Guarding It

For thirty years, “input” meant a keyboard, and security grew up around that idea: sanitise the field, validate the payload, rate-limit the endpoint. The voice broke the assumption. It now sits in call centers, banking apps, cars, wearables, and the AI agents companies ship to talk to customers directly. 

The next wave, physical AI and robotics, will take instructions almost entirely through voice, because a robot working alongside people cannot depend on a keyboard.

New AI attacks appear faster than anyone can catalog them, and the ones that work rarely look like the ones you defended against last quarter. Guessing which technique lands next is a losing game, so the only defense that holds is one that assumes the attack gets through and limits what it can do.

We bolted voice onto systems that were never designed to treat sound as a security boundary. A typed command arrives as structured text you can inspect against a schema. A spoken command arrives as a raw analog signal bundling three things at once: who is speaking, what they are saying, and what they actually intend. 

From $243K to $25M: Five Years of Voice Fraud

The losses compounded a hundredfold in five years while the technical bar dropped the whole time. Same playbook, same trust exploited.

Three documented voice-driven frauds. The 2019 case was called the first known case of AI voice fraud.

Deloitte’s Center for Financial Services projects that generative-AI-enabled fraud in the US will climb from $12.3 billion in 2023 to $40 billion by 2027, a 32% compound annual growth rate. Deepfake vishing is a big part of that curve, and it is aimed squarely at the workflows voice agents now run: approvals, transfers, and credential resets.

US Gen-AI fraud losses, 2023 actual and 2024 to 2027 projected at a 32% CAGR.
Source: Deloitte Center for Financial Services.

Five Attack Families Already Defeat Voice Authentication

A decade of research shows almost every layer of a voice pipeline has a working exploit. They cluster into five families.

Voice-attack families and what each one defeats. Not one of them is stopped by confirming the voice at the start of a call.

Executive Voices Are Cloneable From Three Seconds of Audio

McAfee’s researchers cloned a voice to an 85% match from just three seconds of audio, and to a 95% match with a hand audio band, using tools they found free online.

Three seconds is less audio than any executive produces in one earnings call.

Earnings calls, keynotes, podcast interviews, all-hands recordings, voicemail greetings. All of it is harvestable, none of it can be recalled, and most of it was published deliberately because visibility is part of the job. The same study found that one in four adults had already encountered an AI voice scam, and 77% of those targeted lost money.

You cannot patch a CEO’s voice the way you patch a server. The only lever left is what happens on the receiving end of the call, which is why detection has to shift from “is this the real voice” to “does this request make sense given everything said so far.”

The Four-Step Anatomy of a Voice Takeover

Most voice fraud runs the same four steps. Verification happens once, in step 3. The manipulation runs across all of them.

The four-step voice takeover, with the three documented cases that used it. Three different targets, three different payloads, one identical flaw: the system trusted the voice and never questioned the intent.

Liveness Detection Loses the Cloning Arms Race

Every mainstream voice defense scores a single moment and then trusts what follows.

Liveness detection and cloning are on the same treadmill, and the side generating the fakes moves first. Every time detection gets better at spotting synthetic artifacts, the next generation of cloning model is trained to remove exactly those artifacts. Defenders who only invest in “detect the fake” will always be one model generation behind.

How a Stateful Prompt Firewall Secures Voice Agents

This is where AccuKnox Prompt Firewall changes the model. After speech is transcribed to text, the firewall sits inline between that input and the AI, inspecting every prompt and every response against your policies before either side passes.

It carries context across the whole session instead of returning a single pass or fail, it carries context across the whole session and scores cumulative risk across turns, so it catches a request that only looks dangerous once you account for everything said before it. It links each prompt to its response as one audited interaction, applies Zero Trust decisions on every turn rather than once at the wake word, and can block, sanitize, or monitor in real time.

AccuKnox’s own research on stateful Defenses found that layering session context reduced multi-turn attack success from roughly 73% to under 9%, while keeping false positives below 0.5%. That is the exact trade a voice deployment needs: catch the slow-building fraud without locking out real customers.

The AccuKnox Prompt Firewall sits inline between your application and the model, inspecting traffic in both directions and logging every interaction.

Here are the two approaches scored against the same fraudulent call.

The same six moments of one call, scored two ways. The stateless check finishes its job before the attack starts. Risk values are illustrative of how cumulative scoring behaves.

Every transcribed turn is checked against a set of policy classes before it reaches the model, including prompt injection, PII and secrets anonymisation, toxicity, and banned topics. For a voice agent, that means a caller cannot smuggle in an injection or coax out sensitive data just because they got past the wake word.

A sample of the Prompt Firewall policy classes applied to every turn, on both the incoming
prompt and the model’s response.

A Five-Question Audit for Voice Agent Deployments

The voice agent audit. If more than one row gets a weak answer, you have the same exposure that cost Arup $25 million.

Voice Is One Surface. The Same Patterns Repeat Everywhere.

The same blind spot shows up everywhere an AI system takes input it did not generate. Voice is simply the surface where the money moves first.

The AI attack surface map used across this series. Read the third column top to bottom.
Every row says the same thing about stateless filtering.

Part 2 of this series turns to the attack that hides inside the conversation itself: multi-turn jailbreaks.

FAQs