Security Flaw 'Shadow MQ' Exposes AI Inference Servers to Arbitrary Code Execution

Security researcher Avi Lumelsky from Oligo Security presented a three-year investigation (2024–2025) into 'Shadow MQ,' a design flaw affecting AI inference servers that use ZeroMQ with unsafe Python object deserialization via the Pickle protocol. The vulnerability, found in at least six open-source projects—including Meta’s Llama, vLLM, TensorRT-LLM, SGLang, Modular Max, and PyTorch Monarch—allows arbitrary code execution when attackers send malicious Pickle payloads to exposed TCP ports. Lumelsky demonstrated exploits across these frameworks, showing immediate RCE with root access on misconfigured servers, often deployed in cloud environments with GPUs. The flaw stems from copying insecure code patterns, such as PyZMQ’s receive_pyobj() function, without implementing mitigations like HMAC validation or switching to safer serialization methods like MessagePack. Over 20,000 internet-facing hosts were identified using ZeroMQ’s unique protocol banner, though not all were confirmed vulnerable. Maintainers of projects like Ray and PyTorch Monarch disputed or ignored fixes, classifying the issue as 'intended behavior' despite confirmed RCE risks. The talk emphasized that AI infrastructure’s competitive, performance-driven development prioritizes speed over security, leaving critical systems exposed.