Enterprise AI Agents Require Continuous Red Teaming for Security

Joan Vendrell, NeuralTrust CEO and cofounder, said traditional penetration tests provide only a snapshot in time while agentic AI systems operate continuously and interact with live data. He described a conversation with a CISO preparing for a production rollout of an autonomous customer service agent who had passed conventional tests but could not explain how the agent would respond to an evolving multi-step prompt injection attack.

Vendrell stated that AI agents are non-deterministic, meaning their behavior changes based on context, memory, and available tools. This creates opportunities for what he called adversarial reasoning, where attacks target the agent's logic rather than its input.

He noted that the OWASP Top 10 for LLM Applications has evolved to include agentic hijacking and indirect prompt injection. Gartner predicts that by 2028 more than 50 percent of enterprises will use dedicated AI security platforms to manage these risks.

Vendrell said traditional red teaming, which typically involves a human team spending two weeks testing a system, cannot match the speed of AI development.

Vendrell recommended deploying adversarial agents to operate 24/7 and stress-test production agents using the MITRE ATLAS framework. He said red teaming must focus on insecure output handling and simulate scenarios where agents receive malicious commands through trusted tools such as compromised email or poisoned database entries.

He stated that continuous red teaming should align with the NIST AI risk management framework, specifically its Measure and Manage functions. Vendrell added that red teams should continuously feed agents poisoned data to test resilience against indirect prompt injection and should track identity lineage to ensure accountability remains traceable even when an agent's reasoning is compromised.