Deepfakes, cyber-security and identity management

Akif Khan at Gartner explains why deepfakes are a critical threat to Identity and Access Management systems and their leaders

 

Imagine a scenario where a financial institution’s customer service representative receives a call from what appears to be a long-time client. The caller’s voice is familiar, their details check out, and they request a significant transfer of funds. Unbeknownst to the representative, the caller is a fraudster using a sophisticated deepfake audio to mimic the client’s voice. This is a plausible scenario in an age where deepfakes are becoming increasingly sophisticated.

 

The rise of generative AI (GenAI), and with it, deepfakes, has introduced unprecedented security challenges for identity and access management (IAM) leaders. Deepfake technology, which uses AI to generate hyper-realistic audio, video, and images, has become a critical threat vector. It undermines the reliability of biometric verification systems and fuels increasingly convincing social engineering attacks.

 

For IAM leaders, this shift means that trusted identity processes are no longer inherently secure, exposing them to potential financial losses, reputational damage, and operational disruption.

 

The escalating challenge

Deepfake technology has rapidly advanced, making it more accessible and affordable for malicious actors to create convincing synthetic media. This has led to a proliferation of deepfakes that are more realistic and harder to detect.

 

The consequences can be severe, ranging from financial fraud to the erosion of trust in our digital institutions.

 

Biometric vulnerabilities

Processes that rely on automated biometric voice recognition are threatened by attackers using deepfake audio. This is particularly concerning for financial institutions and organisations using voice recognition for authentication in contact centres, IT service desks, and mobile applications.

 

The risk is not merely theoretical – sophisticated deepfake audio can convincingly mimic an individual’s voice, potentially granting unauthorised access to sensitive accounts or systems.

 

Similarly, identity verification (IDV) processes that include automated biometric face recognition are threatened by attackers using deepfake images and videos. This affects a broad range of applications, including customer onboarding, citizen identification, and workforce management. Attackers can use deepfake videos or images to bypass facial recognition systems, compromising the security of these processes.

 

Social engineering amplified

Deepfakes add a new layer of sophistication to social engineering attacks. Executives have been impersonated in real-time calls, leading employees to approve fraudulent transactions or divulge confidential information. These attacks exploit trust in human interactions, eroding the foundation of organisational communication.

 

Fortifying IAM and building resilience

Despite these challenges, the rise of deepfakes also presents an opportunity to strengthen our cybersecurity defences. It forces us to rethink our approach to security, pushing us to adopt more robust authentication methods and invest in advanced detection technologies. Moreover, it underscores the importance of digital literacy, highlighting to IAM leaders the need for greater awareness and education about the risks associated with deepfakes in their organisation.

 

Enhancing biometric verification: Deploying deepfake audio detection in combination with other security measures is crucial to improve the resilience of biometric voice recognition. Layered security measures, such as liveness detection, metadata inspection, and emulator detection, are essential for identifying anomalies and suspicious patterns.

 

Liveness detection, for example, can verify that a biometric sample is being captured from a live person. Metadata inspection can uncover inconsistencies in file data, while emulator detection can identify attempts to spoof biometric systems. Techniques like screen detection, watermarks, and payload integrity should also be considered

 

Hardening business processes: Audit critical processes to identify vulnerabilities and implement multi-factor authentication (MFA) and multi-party approvals for high-risk transactions. For example, financial transactions above a certain threshold might require approval from multiple managers.

 

Reduce reliance on voice or face recognition as a standalone authentication method. Instead, integrate these tools into a broader security framework that includes behavioural analysis, device telemetry, and contextual risk signals.

 

Investing in employee awareness and training: Employees are often the first line of defense against deepfake-augmented attacks. For example, employees can be trained to verify unusual requests through out-of-band communication channels.

 

Establishing clear escalation procedures for suspected deepfake attempts can minimise operational disruption. This might involve designating specific personnel to handle potential deepfake incidents and ensuring that employees know how to report suspicious activity.

 

Exploring emerging technologies: Real-time deepfake detection for videoconferencing platforms is an area of growing interest. While still nascent, these tools offer the potential to monitor and flag suspicious activity during virtual meetings.

 

Additionally, collaborate with IAM solution vendors to develop more effective detection standards, improving the scalability and interoperability of anti-deepfake tools.

 

Securing identity in a synthetic age

The rise of deepfakes is a stark reminder of the evolving nature of cybersecurity threats. The risks are clear: compromised biometric systems, amplified social engineering attacks, and ultimately, a loss of trust.

 

While the technological arms race against deepfake creation will continue, waiting for a "silver bullet" is a risky strategy. A robust and resilient IAM strategy requires a combination of innovative tools and organisation-wide efforts to reduce vulnerabilities. As threat actors become increasingly adept, the ability to effectively detect and neutralise deepfake impersonation will be a pivotal factor in safeguarding organisational integrity and maintaining stakeholder confidence.

 

By acting now, IAM leaders can not only defend against current threats but also establish a foundation for navigating the future of AI-driven security risks. 

 

---

 

Akif Khan is VP Analyst at Gartner.  Gartner analysts will further discuss the use of deepfake content to subvert IAM at the Gartner Identity & Access Management Summit, taking place from 24-25 March in London, UK

 

Main image courtesy of iStockPhoto.com and wildpixel