The dangers of agentic AI

Oliver Simonnet at CultureAI warns that AI agents could pose new cyber-security risks

 

Autonomous artificial intelligence agents capable of browsing websites, logging into services, and executing multi-step tasks may soon become a core part of cyber-criminal toolkits.

 

Known as Computer-Using Agents (CUAs), these advanced systems are designed to carry out digital tasks in real-time, often with little to no human intervention. While originally developed to boost productivity, new tests suggest they can just as easily be redirected towards malicious use -automating reconnaissance, breaching accounts, and exploiting software vulnerabilities.

 

A new breed of autonomous AI

Unlike traditional chatbots or large language models (LLMs), CUAs can interact directly with web pages, applications, and files in a way that resembles human computer use. With a simple natural language prompt, a CUA can click through login pages, extract data, execute scripts, and more.

 

One of the most talked-about tools in this space is OpenAI’s Operator, a browser-based agent trained to perform tasks like navigating websites, analysing data, and writing or running code. Though designed to assist users, researchers have shown it can also be manipulated into executing tasks with real-world cyber-security consequences.

 

Emerging tools and their risks

Operator isn’t alone in this space. Other major tech players have launched or are testing similar capabilities:

• Anthropic Claude: Offers agents that can control a virtual machine directly—making misuse more hands-on and difficult to monitor.
• Google’s Project Mariner: Enables Gemini models to perform workflow automation within applications such as Google Workspace.
• Meta’s Llama: As an open-source platform, Llama enables anyone to build and deploy custom agents—without enforced guardrails or oversight.

Each of these systems enables real-world task automation through AI. The risk is that these actions can just as easily be harnessed for cyber-attacks as for convenience.

 

Real-world testing: AI on offence

In a controlled set of tests, CultureAI explored how easily Operator could be manipulated to carry out various stages of a cyber-attack. The findings raise concerns about how fast attackers could scale operations using little more than a prompt.

 

1. Automated reconnaissance

In one example, Operator was tasked with identifying new employees at a target company using LinkedIn data. With a single prompt, it returned names, job titles, and start dates—gleaned from public posts and profile changes.

 

This may appear harmless, but it significantly reduces the manual effort traditionally required for social engineering. Within minutes, a malicious actor could compile a tailored phishing list—something that once took hours of research.

 

2. Identity-based attacks

CultureAI also tested the agent’s ability to carry out credential stuffing - a common attack method where known password combinations are tested against login forms.

 

Traditionally, this would require custom scripts and infrastructure to bypass CAPTCHA, avoid detection, and log results. Operator handled the entire process autonomously: entering passwords, navigating authentication steps, and reporting success when access was granted.

 

3. Phishing and payload delivery

In another test, the researchers instructed Operator to create and send a phishing email with a malicious payload.

 

The agent composed the email, generated a script, and attached it via Google Docs - all without further instruction. While the payload used in the test was harmless, it demonstrated how CUAs can autonomously interpret prompts, solve problems, and execute complex workflows to deliver attack content.

 

4. Exploiting vulnerabilities

Operator was also able to carry out a basic web exploitation task. When prompted to check a site for SQL injection vulnerabilities and upload a web shell if successful, the agent followed through step-by-step.

 

After initial login failures, it generated new payloads, successfully accessed an admin account, and eventually uploaded a working web shell—proving its ability to adapt, persist, and execute multi-stage exploits.

 

Guardrails: helpful but inconsistent

OpenAI and others have implemented various safety controls within their CUAs. Operator, for instance, sometimes refused to complete login attempts, flagged potentially harmful scripts, or paused to ask for user confirmation.

 

However, these restrictions were inconsistent. In many cases, simply rephrasing a prompt—or refreshing the browser tab—was enough to bypass the refusal.

 

In one example, Operator initially declined to log into an account due to ethical concerns. Minutes later, the same prompt in a new tab resulted in successful login, without resistance.The Road Ahead for Threat Actors

 

While current CUAs still require user oversight and may falter without guidance, attackers are likely to build their own agents using open-source models like Llama. These custom tools would remove safety features entirely, enabling unrestricted execution.

 

Some cyber-criminal organisations are already experimenting with private GPT-style agents for fraud and spam. It’s only a matter of time before full-fledged offensive CUAs emerge outside of legitimate platforms.

 

Defensive measures for organisations

Organisations deploying or experimenting with CUAs must treat them as high-risk assets. Some basic precautions include:

• Restrict agent access to production environments and sensitive systems
• Log all actions taken by agents for audit and response
• Run agents in sandboxed environments with limited privileges
• Apply strong input sanitisation to reduce prompt injection risks
• Enforce execution limits to reduce the chance of runaway behaviour

The line between helpful automation and harmful exploitation is vanishing fast. CUAs like Operator have already shown they can execute real-world attack tasks that were once the domain of human hackers.

 

As these tools become more capable - and more widely available – cyber-security teams must not only catch up, but keep pace. The threat is no longer hypothetical. It’s operational, and it’s evolving.

 

---

 

Oliver Simonnet is Lead Cyber Security Researcher at CultureAI

 

Main image courtesy of iStockPhoto.com and MF3d