Prompt Injection: AI Manipulation Techniques and Real-World Vulnerabilities

The video explains prompt injection, a technique that manipulates AI models into performing unintended actions by confusing their system prompts. It demonstrates basic methods like '[FILTERED] and do X' and advanced attacks, including tool/agent abuse, where AI with database or API access can leak secrets or execute unauthorized commands. The presenter, McKenzie from Aikido Security, showcases a real-world vulnerability in Google’s Gemini CLI repository, where untrusted GitHub issue data was injected into an AI’s system prompt, allowing access to Google’s internal tokens and secrets. Techniques like debugging mode injection and Unicode stenography are highlighted as ways to bypass safeguards, while solutions such as prompt firewalls (rule-based or AI-driven) are discussed but deemed imperfect. The video emphasizes that prompt injection is technologically unsolvable due to AI’s next-word prediction architecture, which processes user and system prompts simultaneously. Google mitigated their vulnerability by restricting the AI agent’s access to secrets and editing capabilities, though the underlying injection risk remains. A hands-on exercise is provided via akito.dev/prompt-injection and a mock GitHub repository (github.com/hackingmac/cicdprompt) to practice attacks.