Why risk thinking transfers to AI vulnerability scanning

I come from the cybersecurity world, where the word “risk” is very common. Risk avoidance, risk acceptance, risk appetite. All these risk-related tasks exist because there is a chance that something bad will happen. And when it does – a vulnerability is exploited, an attack is launched, an insider goes rogue – you are already prepared.

In the world of shift-left, more developers and companies are pushing vulnerability detection sooner in the development chain. If possible, the developer should run the vulnerability scanning tooling in the development process, so that code that is pushed to the company repositories is already clean of any “bad code”. It is a good practice, but harder to implement than it sounds.

If you plan to use LLM-based security scanning (be it Codex Security from OpenAI, or Claude Security Review or other LLMs), there are some best practices that you must take into consideration. The one that I want to point out here is that you need a harness on top of the model to make sure that the guardrails, the company policy, the expectations that you have are applied correctly.

Why you need a harness

Why should you consider using a harness for your model when you do vulnerability scanning of your code?

The model does not know your guardrails

The LLM has its own built-in guardrails, but they are probably broader than your use case requires. Some examples that might be problematic:

  • you don’t allow connecting to other applications – maybe you want the vulnerability scan to be limited to the application being scanned, and not to scan connected applications.
  • you don’t allow connecting to the live application – maybe you don’t want the model to connect to the live application to validate the findings
  • you want the model to stop after every finding – maybe you want each finding reported the moment it is found, instead of batched up until the scan finishes

The guardrails don’t necessarily have to be complicated, but they must apply to your organization and your needs. Make sure that these guardrails are pushed to the model with each prompt and that they have a way to be enforced.

The model does not know your PII rules

The model’s settings might allow or disallow printing, storing and displaying certain categories of personal information. Those defaults belong to the model, not to you, so you may want to explicitly exclude certain types of content (API keys, family names, addresses, etc.) from what the scan reads and reports.

Maybe you want the PII data to not be completely hidden, but tokenized, so you are able to track its usage over time. This allows for analysis of data, without revealing the original information. All of this is context the model needs to handle your data the way your organization expects.

The model does not know your budget

The model has no notion of your budget – cost control is something the company has to enforce. As with any precaution, maybe you want the model to keep track of its current session and token usage and stop when something drifts off course: a task runs far longer than expected, a worker spawns too many subagents, or usage crosses a threshold you set.

This is not a hard spending limit, but it is another layer of protection against over-spending or spending on tasks that are not very important to your organization.

The model does not know your data formats

The model is not aware of how you want your data and findings structured, or what should happen once a finding is reported. The harness allows the model to respect that data structure and follow the same principles that you already have in place regarding data formats, schedules, notifications and other internal policies.

The harness can verify the model’s work

The model will do its work, hunt for vulnerabilities, and create a report. A harness can double-check and make sure that the model has really done its job and that the required artifacts have been generated. The harness can also check from time to time that the model executes what it was requested to do and that it remains within its time or budget limits. It’s the four-eyes principle applied to the actions of the model.

The harness can automate some of the approval work

Working with the model might be tedious, at least at the beginning and for certain workloads. For example, the model might constantly ask for permission to read documentation or to run certain tasks. Or, maybe you want to allow the model to do multiple passes over the same code, in case it finds certain types of vulnerabilities. A harness can pre-approve these routine requests, so you are only asked about the ones that matter.

The harness can run work in parallel and compare outcomes

Maybe you want multiple passes over the code instead of one, with the same model or with different models, and then the results compared. Or maybe you want to scan different parts of the codebase at different depths (e.g., a light scan for low-risk modules and a deeper scan where the risk is higher).

Conclusion

A model can find vulnerabilities. It cannot know your risk appetite. Guardrails, PII handling, spend limits, data formats, approvals, and second-pass verification are all organizational decisions, and none of them live inside the model.

That is what the harness is for. It is the layer where your policy becomes enforceable rather than aspirational – and where you get the evidence that the scan did what you asked. Start small: one guardrail, one budget cap, one correctness check. Then grow it as your scanning matures. Running the model without it is not shift-left security; it is risk acceptance you never explicitly agreed to.