Friend and foe? The impact of AI on attackers and defenders

David Higgins at CyberArk explores how AI is accelerating both threat and defence, while exposing critical blind spots in organisational security strategies

 

In just a few short years, AI has become woven into the fabric of modern business, powering everything from customer engagement to supply chain management. Business leaders cannot shy away from the technology and its huge potential for delivering value and providing a competitive edge. However, the reality is that every leap in capability opens new doors for cyber-attackers.

 

As a result, with high-profile breaches dominating our headlines, the UK continues to be exposed to increasingly sophisticated cyber-threats. CyberArk’s recent research warned that AI is a triple-edged sword. It can be harnessed as a weapon for attackers, a tool for defenders, and exploited as a growing source of new vulnerabilities.

 

A recent warning from the UK’s cyber-security agency, the NCSC stated that AI ‘will almost certainly continue to make elements of cyber-intrusion operations more effective’. While it’s a daunting prospect for businesses, the most reliable way to build resilience is to place identity security at the heart of AI strategies. This ensures that the very systems designed to advance and improve business operations are not exploited for malicious intent.

 

Old tactics, but with AI powered precision

Cyber-criminals have always relied on the exploitation of trust, but now AI enables them to do so with unprecedented speed and capability. Phishing remains the leading cause of identity-related breaches and has evolved beyond clumsy scams into highly sophisticated impersonations. Attackers can now generate life-like voice clones, photorealistic deepfakes, and perfectly worded messages that mimic colleagues or suppliers. Just last year, one UK firm transferred $25 million after receiving a fake video call from a “CEO.”

 

Traditional training and technical filters cannot shoulder this burden alone, especially when attackers combine AI with human psychology for their advantage. Instead, these challenges call for a fundamental shift in organisational mindset, where verifying identity becomes second nature and security awareness is seamlessly woven into everyday decision-making.

 

AI on the defensive

Although AI is helping attackers scale and adapt, it is also completely transforming how security teams protect valuable data and operate. Nearly nine in ten UK businesses now use AI tools to analyse network behaviour, detect emerging threats in real time, and automate previously time-consuming security tasks. This rapid adoption is also mirrored on the buyer side, as more than 70% of cyber-security decision-makers at large organisations report they are “highly willing” to invest in AI cyber-security tools. The finding reflects an increased understanding that human analysts alone can’t keep up with modern attacks.

 

However, gains in efficiency come with a caveat - AI systems can miss subtle anomalies or generate false positives if left unchecked. Without strong human oversight, there is a risk of overconfidence in the technology. To mitigate this, security leaders must apply rigorous governance and ensure algorithms are trained on relevant, high-quality data, validated regularly and deployed alongside skilled human analysts who can interpret results in context.

 

The rise and risk of machine identities

One of the fastest growing and least visible challenges in the AI era is the explosion of machine identities, from automated processes to AI agents within corporate IT estates. These digital entities now outnumber human identities by 100 to one. Many hold elevated privileges and lack adequate lifecycle management, creating attractive entry points for intruders.

 

The rise of shadow AI makes matters worse. More than half (57%) of employees use unapproved AI services to speed up work, usually to automate tasks or generate content quickly. These unapproved AI tools add another layer of complexity. While efficient, they often operate without appropriate safeguards. This has the potential to expose sensitive data, breach compliance regulations and risk reputational damage.

 

For organisations embracing AI, this risk requires more than technical controls. Businesses must establish clear usage policies as well as educating employees on the risk of bypassing security. In addition to this, organisations should offer robust and secure tools that enable employees to use AI effectively, while safeguarding against emerging and potential risks.

 

Securing identity is the foundation for trustworthy AI

For businesses to realise AI’s benefits without the potential risk, identity protection cannot be an afterthought and must be a strategic priority. This starts with having complete, real-time visibility over all identities, including human, machine, or AI agents, and applying the principle of least privilege to limit access.

 

Many companies are adapting their identity and access management frameworks to address AI-specific risks, granting temporary, on-demand permissions to machine accounts and holding AI systems to the same trust and verification standards as people.

 

It’s undeniable that AI drives efficiency, insight and innovation, delivering huge benefits for organisations that embrace it responsibly. But without strong identity controls, it also magnifies risk. The most successful businesses will be those that understand that resilience is the basis of sustainable growth. As both defenders and attackers actively leverage AI to sharpen their capabilities, one principle stands firm: securing AI across an organisation begins with securing identity. 

 

---

 

David Higgins is Senior Director, Field Technology Office at CyberArk

 

Main image courtesy of iStockPhoto.com and XH4D