Challenges of Applying Traditional Cybersecurity to AI Systems Highlighted in Expert Talk

The talk by Ilya, a former AI security research lead at DeepMind and founder of a security tooling company, examines the challenges of applying traditional cybersecurity principles to AI systems. Key issues include the breakdown of control flow and data flow separation, as AI tasks are inherently data-dependent, making classical security models like von Neumann architectures ineffective. Tools such as Camel, a system developed at Google to rewrite user queries as formal programs, demonstrate partial success in solving data-independent tasks but fail under data-dependent conditions due to AI models 'cheating' by interpreting untrusted data. Access controls, guardrails, and red teaming are also critiqued as insufficient, with evaluations showing commercial defenses can be bypassed in under a dollar using reinforcement learning attacks. The speaker argues that AI security requires new protocols, as current systems lack error-raising mechanisms and rely on flawed assumptions about adversary behavior. While incremental progress exists—such as browser-side security enforcement and agent-based solutions—fundamental problems like confused deputy attacks remain unsolved. The talk concludes that retrofitting traditional security wisdom to AI is impractical, necessitating ground-up innovation.