Your AI pilot will fail its security review
The pilot works. Then legal asks where the data goes, and the answer is an API in another jurisdiction with a sub-processor list nobody can enumerate. The project stops there — usually after the budget is spent.
We have now seen this sequence enough times to predict it. A team identifies a genuine opportunity, builds a proof of concept in a few weeks, demonstrates it to enthusiastic stakeholders, and gets approval to productionise. Somewhere between that approval and go-live, the project meets the security review, and the review asks a question the architecture cannot answer.
The question is rarely sophisticated. It is usually a version of: where does our data go, and who else can see it?
What makes this failure expensive is not that it happens. It is that it happens last. By the time the review runs, the money is spent, the internal expectations are set, and the only options are to abandon the work or rebuild it on a different foundation. Both outcomes are worse than the version where somebody asked in week one.
Why proofs of concept mislead
A proof of concept is designed to answer one question: is this possible? The fastest way to answer it is to call a hosted model API with a handful of representative documents or transcripts. That is a reasonable engineering decision at that stage, and it is also the decision that determines everything that follows.
The problem is that the proof of concept has now demonstrated capability under conditions that will never be approved. The prompt engineering, the retrieval design, the evaluation set, the integration assumptions — all of it was tuned against a model the organisation cannot lawfully or contractually use with real data. The work is not wasted, but the conclusion is: you have proven that a system you are not allowed to run would work.
The pilot did not fail because the technology was immature. It failed because the deployment constraint was treated as a detail rather than a requirement.
The five questions that stop projects
In our experience, security reviews of AI systems converge on the same short list. None of them are difficult to answer if the system was designed with them in mind, and all of them are close to unanswerable if it was not.
Not which cloud region the vendor advertises — which jurisdiction the inference actually happens in, including for logging, caching and any fallback path.
Most model providers reserve the right to add sub-processors with notice. A data protection officer reads that clause as an open-ended commitment they cannot control.
Not a usage dashboard. A record, in your own systems, of the actual content sent and returned — because that is what an incident investigation needs.
Vendor policies on this are usually adequate and always subject to change. The question behind the question is whether you are relying on a policy or on an architecture.
If a model is deprecated, repriced or restricted, does your system degrade, stop, or need rebuilding? A dependency you cannot substitute is a business risk, not a technical one.
Design for the constraint first
The fix is unglamorous. Before any code, hold one session with the people who will run the review — security, legal, and whoever owns the data. Establish what data is in scope, where it is permitted to sit, who must be able to inspect it, and what the organisation has already committed to its own customers. That conversation takes an afternoon and it determines the architecture.
Then build the proof of concept on the deployment you will actually be allowed to use. If that means open-weight models hosted privately, prove the capability there. It is marginally slower to set up and it eliminates the entire class of failure described above.
This is the order we work in, and it is why our engagements start with a conversation about constraints rather than a demonstration. On a recent engagement for a European CRM company, the deployment position was settled before the first line of code — and it was the reason the project shipped at all. Their support tickets contain their customers' customer records, and their own contractual commitments made a hosted model API unusable. A conventional build would have produced an impressive pilot and no product.
When public models are the right answer
None of this is an argument that hosted models are wrong. For workloads where the data is not sensitive and time to market matters more than control, they are usually the correct choice, and we use them. Drafting public marketing copy, summarising already-public material, prototyping against synthetic data — the constraint does not bind, and paying for private infrastructure would be waste.
The mistake is not choosing a hosted model. It is discovering, at the review gate, that you needed to have chosen something else.
A stalled pilot is usually recoverable. The retrieval design, the evaluation work and the integration knowledge transfer to a private deployment; what changes is where the inference runs.
How we deploy privately →