Security Vulnerabilities in Large Model Caching Infrastructures Exposed

The presentation, delivered by Shanu (a student at Australian OCU and InterLab), examines security vulnerabilities in large model (LM) caching infrastructures, specifically prefix cache, multimodal cache, and semantic cache systems like Weaviate and LanceDB. The talk highlights that attackers can exploit these caches to manipulate responses without directly compromising the model, using techniques such as set collision, semantic injection, and multimodal cache poisoning to deliver malicious content or bypass security checks. Key attack vectors include populating caches with precomputed harmful outputs, leveraging low-cost hash collisions (e.g., $0.05 per collision on AWS with 128GB RAM), and exploiting semantic similarity thresholds (e.g., 0.8 default in GBD cache) to force incorrect cache hits. The research demonstrates real-world impacts, including system integrity breaches, automated workflow bypasses, and multimodal attacks where different inputs map to the same cache entry. Mitigation strategies proposed include using secure hashing (e.g., SHA-256), stronger embeddings (e.g., text-embedding-ada-002), post-cache filtering, and incorporating metadata like image dimensions into cache keys. The work, already disclosed to affected vendors, reveals that default caching mechanisms in modern LM systems introduce significant attack surfaces across frameworks like Weaviate, LangChain, and GPTCache. Testing showed attack success rates up to 72% with 500 queries costing $0.75, while defenses reduced exploitability to 27% with minimal latency overhead.