200.000 Euro Gewinn: In die eigene Tasche oder in die Firma? | Umair Zaffar
Umair Zaffar September 29, 2026 0 Comments

Quality Gates Before Deployment: How an AI Model Earns Its License Plate

Nobody lets a car onto the road without registration and an inspection. At SIFAMO, no model talks to a customer before it has passed nine gates and received its license plate. What those gates check, what a failure looks like, and why AI control needs an expiry date on every plate.

I have compared AI models to cars without number plates often enough. Today I want to be concrete, because the comparison only helps as a process. “No registration, no deployment” is the rule in our governance layer. The registration is the last step of a pre-deployment process, and that process is where control is actually decided. Monitoring and response only work on a model that was admitted properly. This is the version we run with banks, insurers and public-sector clients. It is not a legal framework, and it is not complete.

Why a model has to earn the right to talk to a customer

The default in many companies is the opposite. A team builds a prototype, it works in a demo, somebody connects it to the customer portal, and it is in production. Nobody has written down what it is for, what trained it or who switches it off. When the supervisor asks why a customer was told something wrong, the answer is a shrug.

A gate turns that around. The model starts with no rights. Each gate grants one: to see pseudonymised data, to be evaluated, to reach a small group, finally to speak to the public. Control comes before the model, not after it. In a regulated environment the question is not whether you can build it. It is whether you can prove that it does only what it is allowed to do. That proof is the core of AI governance as we practise it.

The nine gates in our pre-deployment process

We run these gates in order. A model that fails one does not proceed, because the later gates assume the earlier ones hold.

  • Purpose and risk classification. One page: what the model does, whom it affects, which decisions it prepares or takes, and the risk class the business owner assigns. The EU AI Act, in force since August 2024, phases in its obligations over the coming years, and this page is where you later show how you classified the system. No page, no owner, no next gate.
  • Data lineage and pseudonymisation proof. For every training and retrieval source: where it came from, on which legal basis, and evidence that personal data was replaced by IDs before the model saw it. We check this by sampling the data, not by reading a policy.
  • An evaluation set owned by the business. The vendor’s benchmark tells you how the model does on the vendor’s questions. We need a few hundred cases the business unit wrote itself, with the answers it would accept, including the ugly edge cases from real complaints. If the vendor supplies the test, the vendor grades itself.
  • Regression tests on every change. Every new model version, prompt edit or change to the retrieval index runs the full evaluation set again. We log the result per case, not just the average, because a flat average can hide ten cases that flipped from correct to wrong.

The first four gates establish what the model is and whether it works. The next five establish whether it can be trusted with people.

  • Adversarial testing relevant to the use case. Prompt injection through documents the model reads, attempts to pull another customer’s data out of the context, jailbreaks that make a claims assistant promise a payout. We write the attacks for the specific use case; a generic jailbreak list says little about a model that reads invoices.
  • Bias checks where decisions about people are involved. We run otherwise identical cases with protected attributes varied. If the outcome shifts with a name, a postcode or a date of birth, the model stops here. This is the Ahmed and Stefan test, done before the first real Ahmed applies.
  • A rollback plan that has been rehearsed. Which previous version takes over, who triggers it, how long it takes, what happens to conversations in flight. We want to have seen the switch flipped in staging.
  • Sign-off by business owner, risk and IT. Three names, three dates, the evaluation report attached. Business confirms purpose and evaluation set, risk confirms classification and bias results, IT confirms that monitoring and rollback are wired up.
  • The registration record. Model and version, owner, permitted scope, data sources, expiry and re-evaluation date. This record is the license plate. The governance layer reads it at runtime: a model whose scope does not cover a request, or whose plate has expired, does not get the call.

What a gate failure looks like in practice

Failures are rarely dramatic. A team adjusts the tone of a customer assistant, the regression run shows the same overall score as before, and the per-case log shows four cases about cancellation deadlines have flipped. Gate four fails, the change goes back, and the four cases become permanent fixtures in the evaluation set.

Another pattern: a retrieval index gets a new source, and the lineage check finds a shared drive with unredacted customer letters. The model has already seen names. In our zones that is red. The index is rebuilt from pseudonymised sources, and the model version that touched raw data is retired, not patched.

And the one that hurts most: the bias run on a credit pre-check shifts when only the first name varies. Everything else passed. The business owner wants to ship with a human in the loop. Risk says no, because a human shown a biased recommendation tends to follow it. No plate until the training data is rebalanced and the run is clean. That costs weeks. It is still cheaper than the alternative.

Why AI control needs an expiry date on the plate

A plate without expiry is a one-time inspection, and that is not control, it is a snapshot. Models drift because the vendor updates them underneath you. Data drifts because customers, products and laws change. An evaluation set from spring tests a world that no longer exists in autumn.

So every registration record carries a re-evaluation date, typically every few months for customer-facing models, and immediately after any vendor model change. On that date the full gate run is repeated. If it passes, the plate is renewed. If nobody does the work, the plate expires and the governance layer stops routing calls to it. An expiry that nobody enforces is a reminder, not a control.

Open questions

  • Oversight at scale. Our gates assume a handful of models per business unit. With dozens of agents calling each other, who holds the plate for the composition, and how do you evaluate behaviour that emerges only in the chain?
  • Robustness. An adversarial suite covers the attacks we know. We have no good way to estimate how much of the real attack surface it covers, and that estimate is what risk would like to see.
  • Interpretability. A bias check tells us that outcomes shift, not why. If we could see which features the model leans on, rebalancing would be targeted instead of trial and error.
  • Honesty. The evaluation set checks whether answers are correct, not whether the model reports its own uncertainty truthfully. That would deserve a gate of its own, and we do not yet know how to test for it reliably.

Frequently asked questions

Who should own the evaluation set?

The business unit whose customers the model talks to. IT can run the set and the vendor can suggest cases, but ownership and sign-off stay with the business. Otherwise the test measures what the builder thinks matters.

How often should a model be re-evaluated?

On every change to the model, prompt or data, and on a fixed date regardless of changes. For customer-facing models we work with intervals of a few months, and the honest answer is that it should be shorter than feels comfortable.

What happens to a model that fails a gate?

It stays where it is. If it was already in production, the previous registered version takes over through the rollback plan. Failure, fix and retest go into the registration record, so an auditor later sees not only that the model passed, but what it failed on first.

Related: AI Governance · EU AI Act · AI Consultant

Leave Comment