AI worms: Aggressive Autonomous Agents

Max Heinemeyer at Darktrace describes a cyber-security threat so dangerous, even criminals won’t even use it (hopefully) 

 

Imagine a team of hackers that never sleeps, never takes a break and its only goals are to break into your organization and cause disruption. 

 

Recent research in machine learning, such as in large language models (LLMs), has led to nefarious advances in the use of autonomous agent teams, which work together for hacking purposes. While this type of research is still in its infancy, it shows the potential direction of travel for cyber-security threats of the future. 

 

How do Aggressive Autonomous Agents work? 

A team of aggressive autonomous agents are given a target and then persistently attempt to compromise the target like a team of human attackers would. This could be by trying to find and exploit web application vulnerabilities, infrastructure vulnerabilities or via text-based social engineering such as phishing. All they need is to be pointed at a victim organization to get going. 

 

Teams of such aggressive autonomous agents show a lot of promise in being able to compromise infrastructure without human oversight or intervention. While public research on this is still in the early stages, it points to the direction of where the threat landscape could be heading. 

 

These aggressive autonomous agents show promise in hacking web applications, exploiting vulnerabilities or fulfilling other typical offensive security tasks. While some of the early prototypes are coming out of research and academia, there are startups that experiment with automation & AI to create more autonomous Penetration Testing and hacking capabilities.  

 

Turning Aggressive Autonomous Agents into AI worms 

Now imagine a team of aggressive autonomous agents which do not stop at breaching the target organization of choice. Imagine they identify suppliers of the initial victim organization and uses them as further starting points to repeat the same process. For example, by breaching the email server and understanding their victim’s supply chain in that way. 

 

But this time, the agents are armed with additional ammunition; access to emails from the initial victim organization allows them to exploit the trust between the initial victim and its suppliers, making subsequent compromises much more likely. Digital supply chains, such as shared documents or infrastructure between the initial victim and their suppliers, makes pivoting between initial victim and supplier infrastructure much easier. 

 

This process of moving from one target to the next is reminiscent of traditional worming malware – with the big difference that the teams of autonomous agents are context-aware to a certain level have a higher chance of moving between targets. 

 

With that in mind, the comparison of a team of autonomous aggressive agents to an AI worm doesn’t seem too far-fetched. 

 

Of course, this is only speculation at this point – and there could be limitations in terms of capability and cost.  

 

Because an AI worm could spread so quickly and more easily evade detection, it could cause widespread damage. For instance, a nation-state could potentially use it in cyber-warfare to take down critical infrastructure and harm financial institutions, as well as hinder government functions. Restricting unintended lateral movement between targets could pose a difficult challenge here – you don’t want an AI worm to move to your own or allied territory.  

 

Unfortunately, the ingredients for a prototype for this type of threat already exist and while nobody has put them together publicly yet, there is a lot of investment happening at the intersection of military, offensive cyber-security and AI across the globe. It’s not unreasonable to assume that nation states are researching this type of capability already behind closed doors. 

 

Why criminals aren’t interested in AI worms 

The main thing to consider is that it’s likely only nation-states that have the technical prowess or motivation to build an AI worm right now. Building an AI worm would require a significant resource investment, and criminal groups likely don’t have the tech capability to add AI into their existing toolset.  

 

However, there are other reasons cyber-criminals won’t unleash AI worms: because of the damage the worm would cause, any bad actor that did use one would come under intense law enforcement pressure. Besides, their existing tools and techniques are effective enough and make lots of money. There’s no incentive for them to innovate for innovation’s sake.   

 

So, we need to be alert, but not alarmed. The reality is there’s no guarantee that the conceptual ideas outlined here won’t eventually be turned into real-world threats by bad actors with the resources and motivation to do so. Hopefully the technological barriers and resources required to develop an AI worm, along with the knowledge - even by cyber-criminals and rogue nation-states - of the damage and associated repercussions that would occur if an AI worm was unleashed, will prevent it from happening.  

 

But even if an AI worm remains nothing more than a lab exercise or thought experiment, what it demonstrates is that the threat landscape is constantly evolving. Cyber-criminals don’t sit still and while an AI worm might be off the table for them, AI is inevitably a tool they will integrate into their operations.  

 

The only way for enterprise to proactively combat these emerging threats is to implement cyber-security that augments the strategic and creative potential of a human security team with AI’s ability to spot and understand regular business behaviour. Then, humans and machines can work together to deal with those threats before business operations are impacted and real damage is done. 

 

---

 

Max Heinemeyer is Chief Product Officer at Darktrace 

 

Main image courtesy of iStockPhoto.com and Orhan Turan