teissTalk: Building a trusted security model for Generative and Agentic AI

Views on news

AI assistants with web browsing features can be repurposed as covert command-and-control (C2) channels, allowing malicious traffic to blend into routine enterprise communications. According to new findings from Check Point Research (CPR), platforms including Grok and Microsoft Copilot can be manipulated through their public web interfaces to fetch attacker-controlled URLs and return responses. This approach shifts AI from a development aid for attackers into an operational component of malware itself. With this technique, criminals are able to abuse the entire AI system. With the breakneck speed of AI development, the speed of the catch-up game with criminals is becoming unmanageable. Risks are further increased by shadow AI. To proactively identify these risks, cyber security teams now need not only rigour but also imagination. 

Using monitoring agentic AI to supervise production AI agents

Today AI is integrated into various enterprise systems – emails, APIs, digital tools, data bases. There are five pillars to AI security. First, you should ensure that AI’s access to accounts and workflows is secure. Second, make sure that data classification, access controls, DLP for the prompts are in place. Third, the systems that the AI system interacts with should be mapped out, checking how runtime controls can prevent prompt injection and jailbreaking. All these should be built on a governance model – the fifth pillar of the scheme.  The LLM model won’t know when it’s abused or compromised and can distinguish only the types of inputs it’s fed.

 
Building in a kill switch is also essential. Testing of AI’s outputs must become much more dynamic and continuous. Security teams, meanwhile, should move beyond static rules, procedures and standards to a much more agile system that carries out continuous tracking and monitoring. AI is now built into most of the SaaS services businesses use and to get transparency about this is really challenging. Embedding monitoring and security into the AI pipeline – security by design – could be the most effective way of reducing exposure to bad actors. A new approach to security can be using independent AI agents to monitor agentic AI. Vulnerability assessment and penetration testing can also be carried out today by agentic AI embedded in VPNs or MCPs. In the agentic AI context, third party risk gets elevated to a new level. 

The panel’s advice

• AI is a double threat – it helps criminals scale their attacks while also introducing new attack surfaces, which businesses need new safeguards against.
• Humans are best at being diligent about previous risks, not the upcoming ones.
• Prompt injection through PDFs or tickets sent to the AI is one of the top gen AI-related attack vectors today.
• To get started with agentic AI security, look at existing frameworks.