Security and the AI inference pipeline

Most enterprise AI governance is built around access. Who can use which tools? What data are employees permitted to submit in a prompt? Which vendors clear procurement? These controls are necessary, but they stop well short of where the actual risk sits. The inference pipeline, where inputs are transformed into outputs, is largely ungoverned in most organisations. That gap is becoming harder to justify.

 

Access controls feel tangible because they map onto existing security thinking. Identity, permissioning, data loss prevention — security teams have managed these for years, so extending them to AI feels like a natural next step. The problem is that AI systems don’t behave like the systems those controls were designed for.

 

Large language models don’t retrieve records. They reconstruct responses from fragments of context. A user blocked from opening a sensitive CRM record directly can still receive elements of that record through an AI-generated summary, if the retrieval layer isn’t subject to the same permissions. The restriction exists on paper. The exposure happens anyway.

 

The inference pipeline includes context retrieval, prompt assembly, model execution, and response validation. Each step has its own attack surface. In most businesses, none of it is treated as requiring boundary control.

 

The problem compounds with agentic AI. When models move from answering questions to triggering actions, the attack surface expands into the orchestration layer. Prompt injection stops being a theoretical concern and becomes an operational one. Malformed or malicious inputs can manipulate how a model interprets instructions, accesses tools, or sequences tasks. A system can be pushed into bypassing its own safeguards without anyone noticing until something breaks, or doesn’t break in the way you’d expect.

 

Auditability is where this starts to become a regulatory problem rather than just a security one. In regulated industries, businesses must demonstrate what data was used, how decisions were made, and whether controls were consistently applied. Most AI deployments can’t meet that standard. Inference calls aren’t logged in structured, traceable ways. Context is opaque. Model versions change without corresponding records. When outputs are stored, the metadata needed to reconstruct how they were generated usually isn’t. When something goes wrong, there’s no reliable audit trail to investigate it.

 

The underlying difficulty is that governance is still being treated as a policy problem. In traditional deterministic systems, defining rules at the boundary and relying on consistent internal behaviour was often enough. AI doesn’t work that way. Behaviour emerges from how data, models, and orchestration interact. That means governance has to be embedded into the infrastructure itself: access controls extended into context retrieval and output filtering, sensitive data handled before it reaches the model, inputs sanitised systematically, tool usage constrained by explicit rules, and every inference producing an auditable record.

 

These are engineering problems as much as security problems. Organisations that treat them as such, and build controls into the inference layer rather than around it, are the ones that will have something approaching a defensible AI security posture. The ones that don’t will continue to accumulate risk beneath the surface, in the part of the stack nobody’s watching.

 

---

 

Satish Thiagarajan is the founder of Brysa, a Salesforce and data consultancy focused on closing the loop between insight and execution in sales, marketing, and service

 

Main image courtesy of iStockPhoto.com and pcess609