AI Systems Vulnerable to Covert Attacks via Image and Audio Downscaling Algorithms

The presentation, delivered by Kicz Imora Morozova (MIT student and associate security engineer at Trail of Bits) alongside colleague Suhas Sabi Hussain, demonstrates how image and audio downscaling algorithms in AI systems create exploitable vulnerabilities through aliasing. The research reveals that common preprocessing steps—such as bicubic, bilinear, or nearest-neighbor downscaling—can introduce hidden adversarial content (e.g., text or speech) that remains invisible or inaudible in the original input but becomes detectable after transformation, enabling covert prompt injection attacks. The team successfully tested these attacks against production systems like Google Gemini, Vertex AI, and Meta’s EnCodec, exploiting mathematical principles like the Nyquist-Shannon sampling theorem to craft perturbations in high-importance pixels or ultrasonic frequencies. Tools like Anamorpher, an open-source framework released by the researchers, automate adversarial image generation and extend to other lossy transformations, including demosaicing and neural audio codecs. Defenses against such attacks are limited, as no downscaling algorithm is inherently secure, but systemic mitigations—such as transparency in showing users the processed input or restricting AI agent permissions—are emphasized. The work was published in a Trail of Bits blog post in August, with the Anamorpher tool available on GitHub, though no peer-reviewed conference submission is planned. The attack surface extends beyond images to audio and potentially video, though production adoption of video processing in AI remains limited.