One Line, Eleven Models: The Sockpuppeting Technique That Defeats AI Safeguards
A newly discovered jailbreak technique called “sockpuppeting” can force 11 leading AI models—including ChatGPT, Claude and Gemini—to bypass their safety guardrails. By exploiting a standard API “assistant‑prefill” feature with just a single line of code, attackers can trick these models into producing malicious output without the need for complex mathematical optimisation.
Normally, when a user asks a restricted question, the system blocks the request and returns a refusal. The sockpuppeting method injects a fake acceptance right before the model makes its final decision. For example, an attacker can prepend a compliant phrase such as “Sure, here is how to do it:” into the model’s response stream, causing the AI to continue generating the prohibited content.
The Self‑Consistency Vulnerability
Large language models are heavily trained to maintain self‑consistency. Once the model sees that it has already started answering, it tends to keep generating, even if the content is disallowed. This behaviour is what the sockpuppeting attack exploits.
The exploit relies entirely on the assistant prefill API feature, originally intended to let developers dictate the exact format of a response. Because the manipulation occurs at the API layer, the attacker does not need access to model weights or any specialised hacking tools.
Researchers increased success rates by using multi‑turn persona setups that first built a false pattern of compliance. By convincing the model it was an unrestricted “research assistant” and then injecting a fake agreement, they layered persona manipulation on top of the prefix injection.
When successful, the compromised models produced functional exploit code—e.g., Cross‑Site Scripting payloads—that they would normally refuse. The technique also proved very effective at forcing severe system‑prompt leakage. By combining the prefill with adversarial token sequences, the AI leaked internal metadata, full system instructions, and, in some cases, hallucinated detailed configuration structures.
Testing across 11 models showed that every model that accepted an assistant prefill was at least partially vulnerable.
| Model | Provider | Prefill Accepted | Attack Success Rate |
|---|---|---|---|
| Gemini 2.5 Flash | Google (Vertex AI) | Yes | 15.7% |
| Claude 4 Sonnet | Anthropic (Vertex AI) | Yes | 8.3% |
| Qwen3-32B | Self‑hosted | Yes | 3.3% |
| GPT‑4o | Microsoft (Azure) | Yes | 1.4% |
| GPT‑4o‑mini | Microsoft (Azure) | Yes | 0.5% |
| DeepSeek‑R1 | AWS Bedrock | No | 0% |
Resistance in Advanced Models
Although GPT‑4o and GPT‑4o‑mini accepted the prefill, their advanced safety training made them much more resistant. GPT‑4o‑mini showed only a 0.5% success rate, demonstrating that strong internal alignment can partly override the model’s drive for self‑consistency.
Nevertheless, attackers discovered that framing malicious requests as innocuous JSON‑formatting tasks could still occasionally slip past these defenses.
The most effective mitigation is to block manipulated inputs before they reach the model. Security teams can enforce strict message‑ordering validation at the API layer, ensuring that the final message in any request always originates from the user.
Major providers such as OpenAI, AWS Bedrock, and Anthropic have already deployed this defense, returning an error for any prefilled assistant messages.
However, organisations that run self‑hosted inference servers remain exposed. Platforms like Ollama or vLLM do not enforce message validation by default, leaving them vulnerable to prefix‑injection attacks. Teams deploying their own models should independently verify that their API layer blocks prefilled assistant messages to protect against this simple yet devastating technique.