Sovereign AI: three decisions before the model

Generative AI is moving beyond the pilot stage. It reads supplier invoices and feeds the ERP, searches decades of legal archives, checks bills of exchange and answers the customers of a banking service. As it enters critical processes, the question changes in nature: it is no longer about which model performs best, but about which data it may see, on which infrastructure, and at what cost to the company's independence.

Yet most projects still begin by choosing the model. That is approaching the problem backwards. In an enterprise AI project, the model is not a starting decision: it is the consequence of three structural decisions that come before it.

At a glance

• Sovereignty: data determines hosting, flow by flow, and the target architecture is hybrid by design.
• Capacity: on controlled infrastructure, resources are finite; the right model is the smallest one that meets the business requirement.
• Reversibility: models change every quarter, applications last for years; the architecture must absorb that gap.

From pilot to critical process

A pilot tolerates almost anything: a cloud API, an oversized model, an ad hoc integration. A critical process does not. It commits the company's responsibility for data confidentiality, it demands predictable costs, and it must run for years, well beyond the lifespan of any single model.

These three requirements map onto three decisions. Each one narrows the range of possible choices. Taken in order, they leave a small number of candidate models at the end; taken in reverse, they force the architecture to be rebuilt as soon as the system goes into production.

Sovereignty is decided per flow

The first decision is not technical: it is legal and strategic. Where is each piece of data allowed to be processed? A supplier invoice, a litigation file and a public FAQ do not follow the same rules, and it would be as costly to bring everything on-premises as it would be risky to entrust everything to the cloud.

The unit of decision is therefore the data flow, not the project. Each flow receives a hosting rule based on its sensitivity, the transfers allowed and the required location: internal infrastructure, operated sovereign cloud, or public API. The result is a hybrid architecture in which each workload sits where it is allowed to be.

This decision belongs to executive management, legal and compliance before IT. Left to technical teams, it gets settled by default, often in favour of whichever tool is easiest to integrate.

Capacity determines the model

As soon as a flow must stay on controlled infrastructure, the logic is reversed. In the cloud, computing power seems unlimited and is paid for as you go. On-premises, it is finite, bought upfront and shared between several services. The decisive constraint is GPU memory: a model that does not fit entirely within it slows down by an order of magnitude.

The question is no longer "what is the best model?" but "what is the smallest model that meets the business requirement within the available resource budget?". Public leaderboards cannot answer it: they measure generic tasks. Only a benchmark on the real task, with real data, can settle it. This is not an acceptance-testing step; it is the decision tool that balances accuracy, latency and infrastructure cost.

Done well, this decision avoids the most common expense in on-premises AI projects: hardware sized for a model that never needed to run.

Reversibility protects the investment

The model chosen today will not be next year's. Models succeed one another every quarter, and so do inference engines, while business applications are amortized over several years. Binding one to the other amounts to scheduling their obsolescence.

The answer is architectural: a single gateway between applications and models, with model names treated as configuration. Switching models or engines, or moving a flow to the cloud, then becomes a setting rather than a project. It is also the best protection against lock-in, whether to a model provider or to an integrator.

Finally, reversibility means running a local model like any other production service: monitoring, capacity management, controlled updates. An LLM rarely fails outright; it degrades through silent slowdowns that only appropriate monitoring will reveal.

Illustration: a real deployment

These three decisions hold up in our own deployments, deliberately run on modest hardware: two consumer-grade 16 GB graphics cards, shared between several services.

Sovereignty. Sensitive document flows were classified for local processing; the others could stay in the cloud. The on-premises scope was thus limited to what genuinely needed to be there.

Capacity. Quantized to 4–5 bits, Qwen3.6-27B takes up 18.2 GB with a 2,048-token context: it does not fit on one card, and spreading it across both would deprive the other services of their memory. The benchmark confirmed the value of not aiming bigger. On the same JSON data-extraction task, Qwen3-14B returns an accurate result in 33 seconds, Qwen3.6-27B the same result in 156 seconds, and Gemma 4 12B does not complete this task.

The larger model was not more accurate: it generated three times as many reasoning tokens to reach the same result. The 14B delivers the same accuracy nearly five times faster, on a single card.

Reversibility. All applications go through an OpenAI-compatible gateway, such as LiteLLM, with model aliases. Operations revealed the importance of monitoring: short of free memory, one model was running three-quarters on the CPU, with no error and no alert. Once each service was assigned its own card, the pipeline became eleven times faster. We now monitor the share of each model actually loaded into GPU memory, not just its availability.

What decision-makers must settle

Before launching an AI project on sensitive data, three questions deserve an explicit answer at executive level:

• Which flows are not allowed to leave our infrastructure, and who decided?
• What business requirement are we targeting, and have we measured it on our task before sizing the hardware?
• What would switching models next quarter cost: a setting or a project?

Our approach

At KAIROS Impulse, we work through these three decisions in this order, on the client's real data. The model comes last, and the architecture lets the client change it without depending on us. Our principle: transfer a capability, not create a dependency.

The right model is the one your infrastructure can run without incident for years.