teissTalk: Reducing your security debt in a GenAI-driven world

On 14 November 2024, Teiss Talk host Thom Langford was joined by Lee Munson, Principal Research Analyst, Information Security Forum; Zia Ush Shamszaman, Senior Lecturer , Computer Science, Teesside University; and John Smith, EMEA CTO, Veracode.

Views on news 

Developers in almost all (83%) organizations use AI to generate code, causing security leaders to fear it could fuel a major security incident, according to a new Venafi survey, 78% of whom  is convinced that AI-generated code will lead their organization to a security reckoning and 59% are losing sleep over the security implications of AI. GenAI, however, is very effective in writing code and can save a huge amount of time for developers, but whether that code is safe is another matter. It can get more secure, though, if prompts include security requests or compliance with guidelines and frameworks too. There is also a misconception that AI-generated code is better than that written by humans. Quality assurance teams must be trained differently in this new world.

Although new AI models are also being developed to automate testing, currently, companies, in general, probably spend more on developing than testing with this type of software.

Making GenAI-generated code better

What will be a game changer is developers starting to use domain-specific, more specialised LLMs. Codes generated now will become training data for new models, which also makes the security of generated code paramount. What the technology can be most securely used for is threat detection, threat intelligence and vulnerability management. GeneticAI is an emerging approach, where small AI models are interacting with each other and which can become a major use case in the future. The fact that GenAI can mimic human communication so well can also give a false sense of security to users. 

Running the same programme on two different AI models and seeing if they have the same output can also serve as a kind of verification tool. Policy has a crucial role to play too, which should also include guardrails regarding who can have access to code generating tools, as well as the data sets that the model is trained on.

To train a model rather than fine-tuning an LLM requires an astonishing amount of data, which will enhance the need for more data in the future in order to create new base models. The interconnectedness of GenAI models can result in using the wrong code for years without even realising it. The remedy for poisonous data can be training your model to deal with the wrong data via adversarial learning. It’s important to remember that if a developer uses open source code their contribution becomes open source too. There are also tools now that intellectual property owners use to deliberately poison models. Whether the GenAi model will use your proprietary code depends on whether you use the tool at no cost or pay for it and what’s in the agreement.

The panel’s advice

• Fully autonomous Ai decision making is still very much futuristic and scary. It’s still the co-pilot phase.
• With rapid software roll-out you can very easily make a massive mistake.
• Developers should get some training on how to write code and make security checks on it.