Critical RCE Vulnerability Discovered in SGLang: How Malicious GGUF Models Can Compromise Inference Servers

In an era where AI infrastructure speed is prioritized, a significant security oversight has been uncovered within SGLang, a high-performance framework designed for Large Language Model (LLM) serving. Security researchers have identified a critical vulnerability that allows remote attackers to move from merely hosting a model to achieving full Remote Code Execution (RCE) on the host machine.

Tracked as CVE-2026-5760, this flaw presents a classic supply-chain attack vector. By leveraging the widely used GGUF model format, a threat actor can weaponize a model file and host it on public repositories like HuggingFace. Once an unsuspecting developer or automated pipeline pulls this model into an SGLang environment, the attacker gains a foothold that can lead to total system compromise.

The Root Cause: Insecure Template Rendering

The technical crux of this vulnerability lies in a Server-Side Template Injection (SSTI) oversight within the framework’s reranking logic. When SGLang processes a model, it must interpret chat templates—instructions that format how a model perceives conversation history. In the case of SGLang, these templates are often ingested directly from the model’s metadata.

Deep-diving into the codebase, the vulnerability is located within the serving_rerank.py module. The framework utilizes the Jinja2 templating engine to render these chat templates. However, instead of employing the hardened ImmutableSandboxedEnvironment, the code calls the standard Jinja2.Environment() function.

This is a critical architectural error. While a sandboxed environment restricts the template’s ability to access dangerous Python attributes or system calls, the standard environment provides no such boundaries. This allows an attacker to craft a template that “escapes” the intended logic and executes arbitrary Python commands with the same privileges as the SGLang process.

Anatomy of an Attack: Step-by-Step Execution

Recent proof-of-concept (PoC) research has mapped out the exact lifecycle of this exploit:

• Weaponization: The attacker crafts a malicious GGUF file. Inside the file, they modify the tokenizer.chat_template metadata to include a sophisticated SSTI payload.
• The Trigger Phrase: To ensure the payload executes, the attacker embeds a specific string (e.g., “The answer can only be ‘yes’ or ‘no’”). This phrase is strategically chosen to trigger SGLang’s internal Qwen3 reranker detection logic, forcing the framework to process the malicious template.
• Deployment: The compromised model is uploaded to a public platform. The attacker waits for a victim to download and execute the model using SGLang version 0.5.9.
• The Payload Delivery: When a user makes an API request to the /v1/rerank endpoint, the framework pulls the template from the model to process the request.
• Code Execution: As the template is processed, the Jinja2 engine hits the un-sandboxed payload. Using known Python bypass techniques, the attacker breaks out of the template context and executes operating system commands, effectively handing control of the server to the attacker.

Broader Context and Risk Assessment

This vulnerability is not an isolated incident but part of a growing trend of security friction in the AI ecosystem. The exploit maps directly to two major weakness classes: CWE-1336 (Improper Neutralization of Special Elements Used in a Template Engine) and CWE-94 (Improper Control of Generation of Code).

The attack surface is remarkably similar to previous high-profile vulnerabilities in the AI space, such as the “Llama Drama” bug (CVE-2024-34359) that targeted llama-cpp-python, and the recent Denial-of-Service (DoS) concerns in the vLLM framework (CVE-2025-61620).

Mitigation and Best Practices

If you are currently running SGLang version 0.5.9, you are at risk. We strongly recommend the following immediate actions:

1. Verify Model Sources: Avoid downloading GGUF models from unverified or unknown users on public repositories. Stick to official or highly trusted contributors.
2. Isolate Inference Environments: Run your inference servers in highly restricted, low-privilege containers or sandboxed environments to limit the blast radius of a potential RCE.
3. Monitor for Updates: Prepare to upgrade SGLang as soon as a patch is released that implements ImmutableSandboxedEnvironment for all template rendering paths.