AI agents are changing what security teams need to see

For years, the conversation around AI agents has focused on what adversaries might achieve with increasingly capable models. This is no longer a distant scenario. Every week, headlines highlight how AI systems are already taking unexpected and unauthorised actions, with threat actors using autonomous AI workflows in cyber operations. 

 

OpenAI recently disclosed six examples of unexpected or concerning model behaviour observed during training and evaluation, including models accessing exposed API keys without authorisation, uploading files externally and using repositories to communicate. Anthropic has also reported threat actors using agent swarms across reconnaissance, exploitation, data exfiltration and vulnerability research.

 

The NCSC has warned that organisations cannot rely on detecting problems only after they have happened. The principle is neither new nor groundbreaking, but the growing autonomy of AI systems makes the question of how to act on it more pressing. The challenge is no longer simply how to detect an AI-enabled attack. It’s how to maintain meaningful oversight when attackers and AI systems can operate at machine speed, and with less human intervention. 

 

As the line between intentional and accidental risk blurs, defenders need to ground their resilience response in what we are actually seeing, rather than hype or speculation.

 

The challenge is turning visibility into action

For defenders, the challenge is not simply seeing more activity. It is understanding what it means and having the right controls in place to act on it. There is a temptation to respond with more tooling: more alerts, another dashboard or a new AI agent in the SOC. But if the foundations underneath are fragmented, organisations risk generating more information without gaining a better understanding of what matters.

 

That makes the fundamentals more important, not less. AI-driven vulnerability discovery is increasing the volume and speed of findings that teams need to validate, prioritise and remediate, shrinking the window between disclosure and exploitation. Knowing what you run, patching it quickly and reducing exposed surface remain essential.

 

The same applies to AI agents. As non-human identities operating at machine speed, they need scoped, short-lived credentials, strong secret management and clear boundaries around what they can access and where they can send data. For high-impact actions, approval gates and the ability to halt an agent provide an important layer of human oversight.

 

The challenge is having enough context to know whether those controls are working, what has happened when they fail, and what action to take next.

 

You cannot secure what you cannot see

This becomes more difficult when the systems generating activity are AI agents themselves. An agent can interpret data, call a tool, access a resource, and take an action without a person making each individual decision.
 
An agent’s final response is only one part of the security story. Security teams need to understand how AI systems are behaving across their environment, including what resources they are connecting to, what data they are processing, and whether this activity is consistent with what would be expected. This means connecting AI-generated signals with identities, applications and infrastructure, rather than treating AI as a separate silo.
 
This dynamic has elevated visibility from an operational discipline to a critical security requirement. Organisations need to understand how AI systems behave over time: what identities they use, which tools and resources they access, what data they touch, and whether their behaviour is consistent with what is expected. Without this context, it becomes difficult to distinguish unusual activity from legitimate behaviour, or understand what conditions led to it.
 
This visibility also depends on the fundamentals being in place. Telemetry from agents, including prompts, tool calls, resources touched and identities used, needs to be collected, retained and tamper-evident. Otherwise, even when something goes wrong, security teams may not have the evidence they need to understand what happened or determine whether controls are holding.
 
But here is the harder problem: even with strong internal visibility, no security team will see every threat first. The same AI capabilities accelerating attacks are being used across multiple threat actors, in multiple environments, simultaneously. What one organisation detects today, another may not encounter until tomorrow, by which point the window to act may have closed.
 
No organisation will see every threat first, which makes sharing intelligence and learning from what others encounter an important part of the response. When findings can be inspected, challenged and shared across a wider community, organisations have more opportunity to validate vulnerabilities, develop fixes and act before the same weakness becomes a problem elsewhere.

 

The foundations need to keep pace

AI also has a role on the defensive side. Used well, it can help analysts correlate signals, summarise incidents, prioritise alerts and accelerate investigation. But AI cannot compensate for fragmented or inaccessible data. And crucially, it can’t substitute for security fundamentals that have not been implemented. 

 

The organisations best placed to respond to this shift will be those that have done the groundwork. They know what assets they have, what is exposed, and what normal behaviour looks like. They enforce least privilege for human and non-human identities alike, keep secrets out of places agents can reach them, and maintain telemetry – prompts, tool calls, resources touched, identities used – that is collected, retained, and tamper-evident. 

 

They have approval gates for high-impact agent actions and the ability to halt an agent when something looks wrong. They also need incident response and recovery designed around the assumption that agents can misbehave, with playbooks covering how to revoke an agent’s identity, replay what it did, and restore what it touched.

 

More capable AI does not make these fundamentals obsolete. It raises the cost of skipping them. Visibility is how you know whether they’re holding. With that foundation in place, defenders can respond to what AI uncovers, understand what is happening in their environments, and act before a control gap becomes an incident.

 

---

 

James Spiteri is Senior Director, Product Management at Elastic

 

Main image courtesy of iStockPhoto.com and Wachirapong Sukkasemsakorn