EU AI Act Article 14: Human Oversight, Implemented in Practice
EU AI Act Article 14 requires that humans can effectively oversee a high-risk system while it is in use: understand it, override it and stop it. We build that not as a process document but as tiered review, a stop flag in the registry and a feedback loop inside the governance layer.
Article 14 is the article everyone cites and hardly anyone builds. “A human looks at it” appears in every project deck. Article 14 asks for more: the system has to be designed so that oversight still works at twenty thousand cases a month. It is addressed first to the provider, who designs and develops the system. The deployer gets its share through Art. 14(3)(b) and through Art. 26(2), which obliges it to assign oversight to people with competence, training and authority. Whoever builds a high-risk system in-house on a foundation model and uses it is, under Art. 3(3), usually both.
The EU AI Act has been in force since 1 August 2024. Prohibitions and AI literacy have applied since February 2025, the obligations for GPAI models since August 2025, the transparency obligations under Article 50 since August 2026. After the postponement by the amending Regulation (EU) 2026/1744 (Digital Omnibus), the obligations for high-risk systems under Annex III apply from 2 December 2027, for Annex I products from 2 August 2028. Article 14 is one of the high-risk requirements in Chapter III Section 2, so those dates apply directly. The Omnibus moved deadlines; it removed no obligation. How the article fits the overall structure is in the overview EU AI Act in practice.
What Article 14 requires
The article has five paragraphs. The first three say what oversight is for and how much is needed, the fourth says what the humans must concretely be able to do, the fifth covers a special case.
Art. 14(1): High-risk systems must be designed and developed, including with appropriate human-machine interface tools, so that natural persons can effectively oversee them “during the period in which they are in use”. This is a design obligation on the provider, not an operating instruction.
Art. 14(2): The aim of oversight is to prevent or minimise risks to health, safety and fundamental rights, both under intended use and under reasonably foreseeable misuse. It targets in particular the risks that remain despite the other requirements of Section 2.
Art. 14(3): Measures must be commensurate with the risks, the level of autonomy and the context of use. Two routes: point (a), the provider builds them in where technically feasible; point (b), the provider identifies them before placing the system on the market and the deployer implements them. What the deployer has to do belongs in the instructions for use under Art. 13(3)(d).
Art. 14(4): The persons assigned to oversight must, as appropriate and proportionate, be able to do five things: (a) understand the capacities and limitations of the system and monitor its operation, including detecting anomalies, dysfunctions and unexpected performance; (b) remain aware of the tendency to rely automatically on the output (automation bias), in particular for systems that provide information or recommendations for decisions taken by humans; (c) correctly interpret the output, using the available interpretation tools and methods; (d) decide not to use the system, or disregard, override or reverse its output; (e) intervene or interrupt the system through a stop button or a similar procedure that brings it to a halt in a safe state.
Art. 14(5): For remote biometric identification under Annex III point 1(a), a two-person rule applies: no action or decision may be based on the identification unless at least two competent natural persons have separately verified and confirmed it. Exception: law enforcement, migration, border control and asylum, where Union or national law considers this disproportionate.
Paragraph 4(b) names automation bias explicitly and aims at systems that provide recommendations for human decisions. That is exactly the case of a pre-ranking assistant, which is why I chose this example.
The technical example
An insurer uses an assistant in recruiting that matches incoming applications against the requirement profile of a position and pre-ranks them. AI for recruitment or selection, in particular to filter applications and evaluate candidates, is listed in Annex III point 4(a). The system is therefore high-risk, no matter how small the team that built it.
The data flow: the application lands in the applicant tracking system. The policy gateway pseudonymises it before any model sees it: name, photo, date of birth and address are replaced by an applicant ID, and free text is scrubbed for names and addresses. The model compares the documents with the versioned requirement profile the business has approved and returns, per must-have criterion, “met”, “not met” or “unclear”, each with the passage it relies on. From that comes a proposal: shortlist or not. The gateway writes the evidence, maps the ID back and delivers the proposal into the applicant tracking system.
Where the obligation bites: at the invisible rejection. An application the assistant puts below the line never meets a human unless the design forces it. And at the recruiter with four hundred applications in front of her and a green score. She clicks. That is exactly what paragraph 4(b) warns about. Add the systematic failure pattern: a model that regularly reads a parental leave or a foreign degree as “not met” is not noticed case by case, only in the pattern.
How we implement it
At SIFAMO we treat Article 14 as a property of the governance layer, not as a chapter in the procedures manual. Every measure has a place in the registry, the gateway or the evidence log.
Tiered review per decision type. The registry carries an oversight field per model. For the assistant it holds two decision types. “Do not forward” is tier 3: full review, no application is rejected without a recruiter having read it and decided themselves. Until then it stays “pending”. “Shortlist” is tier 2: sampled review at a rate we calculate from the error rate we want to detect, not from free capacity. The system draws the sample, not the team. On top comes the blind re-check: a second reviewer receives a share of already decided cases without seeing the model score or the first decision. The disagreement rate between the two is a production metric.
Understanding and monitoring (Art. 14(4)(a)). The review interface shows what the model can judge and what it cannot: formal criteria yes, motivation and soft skills no. The known weaknesses from the evaluation sit directly in the interface, not in a PDF. Monitoring reads the evidence log and reports to the business owner, here the head of recruiting: distribution of scores per position and per application channel, the rate of “unclear” outputs, disagreement with a simple rules check for hard must-have criteria such as language level or licence.
Against automation bias (Art. 14(4)(b)). Sequence is the most important tool: the recruiter sees the application and the requirement profile first, records their own judgment, and only then sees the model’s assessment. There are no traffic-light colours and no score without reasons and passage. We inject cases with known-wrong assessments into the review queue and measure per reviewer whether they catch them. That is unpopular and the only reliable test I know. Time per case and approval rate run as metrics with alerts, because when both head toward zero seconds and one hundred percent, the loop is dead. Cases per reviewer per hour are capped.
Interpreting (Art. 14(4)(c)). Every criterion comes with the passage it relies on, and “unclear” is a legitimate output. A model that is allowed to abstain produces fewer confident errors than one that must always deliver a score.
Not using, overriding, reversing (Art. 14(4)(d)). The reviewer has four actions: confirm, move up, move down, discard. Each needs a reason category, none needs an approval. And the applicant tracking system works without the assistant, because it never depended on it. The team can switch any position to the mode without pre-ranking.
Stopping (Art. 14(4)(e)). The stop button is a flag on the registry record. The head of recruiting sets it personally, without a deployment and without a change request, as can risk and IT operations. The gateway reads the flag on every call. The safe state is defined: applications stay in the queue, no proposal and no rejection is produced, nothing is lost. Resuming requires the same sign-off a new version requires. We learned that a stop button only engineers can press is not a control the business trusts.
Art. 14(5) as contrast. The two-person rule applies only to remote biometric identification. We do not copy it onto recruiting. Two signatures on every application do not double the quality, they halve the attention. We apply four eyes where the second person changes the outcome: cases overturned on a protected ground, and the first hundred cases after a model or prompt change.
Feedback. Every reviewer action lands in the evidence log with a reason category. The category points to one of three places for the fix: the prompt, the requirement profile as policy, or the evaluation set. Changes to the prompt or the profile get a new version and go through the same registration as a new model, and the triggering cases become regression tests. Oversight that does not change the system is theatre.
What the auditor sees: the registry record with tiers, sampling rates and who may stop, per application the evidence with model and prompt version, policy version, output, reviewer action and time, the catch rate on injected cases, and every stop event with person and timestamp. Where the law leaves room, this is what we chose: we read “effectively” in paragraph 1 as measurable. Oversight whose effect we cannot measure counts, for us, as absent. I am an engineer, not a lawyer, but this reading has so far survived every discussion with a legal department.
Typical mistakes
Oversight per system instead of per decision type. The same assistant drafts rejection letters and filters candidates out. Two consequences, two tiers. Whoever classifies the system wholesale reviews either too much or the wrong thing.
The green score. A number with a colour, without reasons and before the reviewer’s own judgment. That is automation bias as a feature.
The stop button in the ticket system. If halting needs a change request, you do not have a stop button, you have a process. Paragraph 4(e) means something else.
Review with no way back. Hundreds of overrides, no changed prompt. Then the review was a cost centre, not a control.
Missing the provider role. The in-house team builds on a foundation model and considers itself a pure deployer. But paragraphs 1 to 4 address the provider, and the provider then sits in your own building.
EU AI Act Article 14 checklist
Every decision type of the system has a registered oversight tier (full, sampled, monitoring only) with a rationale.
The sample is drawn by the system, the rate is derived from the error rate to be detected and fixed as a floor.
A blind re-check runs, and the disagreement rate is a metric with an alert.
The reviewer sees the model’s capacities, limitations and known weaknesses in the interface, and sees the reasons before the score.
Injected error cases measure whether reviewers are still reviewing. Time per case and approval rate have alerts.
The reviewer can override, discard and decline to use the system without approval. The business application runs without the model.
The business owner can set the stop flag personally, the gateway reads it on every call, the safe state is defined and tested.
Every reviewer action is in the evidence log with a reason category and demonstrably leads to prompt, policy or test-set changes.
Open questions
“Effectively” in paragraph 1 is not defined. The harmonised standards meant to make it concrete are not yet available in a form we would rely on. We measure catch rate, time per case and disagreement rate and hope the standards go in that direction.
The reasoning a model shows the reviewer is itself a model output. Whether it describes what drove the assessment or is a plausible story written afterwards is something we cannot reliably verify today. How much it protects against automation bias depends on that.
How sampling rates and thresholds should adapt when models change weekly, without a human recalculating them, is open.
Frequently asked questions
Does this apply to us as a deployer if we only buy the assistant?
Yes, through two routes. Art. 14(3)(b) lets the provider identify oversight measures that you as deployer have to implement, and they appear in the instructions for use. Art. 26(2) additionally obliges you to assign oversight to people with competence, training, authority and support. Whoever substantially modifies the product or deploys it under their own name moves into the provider role under Art. 25.
Is it enough if a human approves every output?
Formally perhaps, practically not. Article 14 requires that the person understands outputs, detects anomalies and remains aware of automation bias. An approval rate near one hundred percent at review times below reading time proves the opposite. Tiered review with sampling and blind re-checks serves the purpose better than a signature on everything. More on this under AI governance.
Does the stop button have to be a real button?
No. Paragraph 4(e) speaks of a stop button or a similar procedure that brings the system to a halt in a safe state. Three properties matter: one authorised person can trigger it alone, it takes effect immediately on every call, and the safe state is defined in advance. For us it is a flag in the registry. The other articles of the series are here: all articles at a glance.
EU AI Act Article 14: Human Oversight, Implemented in Practice
EU AI Act Article 14 requires that humans can effectively oversee a high-risk system while it is in use: understand it, override it and stop it. We build that not as a process document but as tiered review, a stop flag in the registry and a feedback loop inside the governance layer.
Article 14 is the article everyone cites and hardly anyone builds. “A human looks at it” appears in every project deck. Article 14 asks for more: the system has to be designed so that oversight still works at twenty thousand cases a month. It is addressed first to the provider, who designs and develops the system. The deployer gets its share through Art. 14(3)(b) and through Art. 26(2), which obliges it to assign oversight to people with competence, training and authority. Whoever builds a high-risk system in-house on a foundation model and uses it is, under Art. 3(3), usually both.
The EU AI Act has been in force since 1 August 2024. Prohibitions and AI literacy have applied since February 2025, the obligations for GPAI models since August 2025, the transparency obligations under Article 50 since August 2026. After the postponement by the amending Regulation (EU) 2026/1744 (Digital Omnibus), the obligations for high-risk systems under Annex III apply from 2 December 2027, for Annex I products from 2 August 2028. Article 14 is one of the high-risk requirements in Chapter III Section 2, so those dates apply directly. The Omnibus moved deadlines; it removed no obligation. How the article fits the overall structure is in the overview EU AI Act in practice.
What Article 14 requires
The article has five paragraphs. The first three say what oversight is for and how much is needed, the fourth says what the humans must concretely be able to do, the fifth covers a special case.
Paragraph 4(b) names automation bias explicitly and aims at systems that provide recommendations for human decisions. That is exactly the case of a pre-ranking assistant, which is why I chose this example.
The technical example
An insurer uses an assistant in recruiting that matches incoming applications against the requirement profile of a position and pre-ranks them. AI for recruitment or selection, in particular to filter applications and evaluate candidates, is listed in Annex III point 4(a). The system is therefore high-risk, no matter how small the team that built it.
The data flow: the application lands in the applicant tracking system. The policy gateway pseudonymises it before any model sees it: name, photo, date of birth and address are replaced by an applicant ID, and free text is scrubbed for names and addresses. The model compares the documents with the versioned requirement profile the business has approved and returns, per must-have criterion, “met”, “not met” or “unclear”, each with the passage it relies on. From that comes a proposal: shortlist or not. The gateway writes the evidence, maps the ID back and delivers the proposal into the applicant tracking system.
Where the obligation bites: at the invisible rejection. An application the assistant puts below the line never meets a human unless the design forces it. And at the recruiter with four hundred applications in front of her and a green score. She clicks. That is exactly what paragraph 4(b) warns about. Add the systematic failure pattern: a model that regularly reads a parental leave or a foreign degree as “not met” is not noticed case by case, only in the pattern.
How we implement it
At SIFAMO we treat Article 14 as a property of the governance layer, not as a chapter in the procedures manual. Every measure has a place in the registry, the gateway or the evidence log.
Tiered review per decision type. The registry carries an oversight field per model. For the assistant it holds two decision types. “Do not forward” is tier 3: full review, no application is rejected without a recruiter having read it and decided themselves. Until then it stays “pending”. “Shortlist” is tier 2: sampled review at a rate we calculate from the error rate we want to detect, not from free capacity. The system draws the sample, not the team. On top comes the blind re-check: a second reviewer receives a share of already decided cases without seeing the model score or the first decision. The disagreement rate between the two is a production metric.
Understanding and monitoring (Art. 14(4)(a)). The review interface shows what the model can judge and what it cannot: formal criteria yes, motivation and soft skills no. The known weaknesses from the evaluation sit directly in the interface, not in a PDF. Monitoring reads the evidence log and reports to the business owner, here the head of recruiting: distribution of scores per position and per application channel, the rate of “unclear” outputs, disagreement with a simple rules check for hard must-have criteria such as language level or licence.
Against automation bias (Art. 14(4)(b)). Sequence is the most important tool: the recruiter sees the application and the requirement profile first, records their own judgment, and only then sees the model’s assessment. There are no traffic-light colours and no score without reasons and passage. We inject cases with known-wrong assessments into the review queue and measure per reviewer whether they catch them. That is unpopular and the only reliable test I know. Time per case and approval rate run as metrics with alerts, because when both head toward zero seconds and one hundred percent, the loop is dead. Cases per reviewer per hour are capped.
Interpreting (Art. 14(4)(c)). Every criterion comes with the passage it relies on, and “unclear” is a legitimate output. A model that is allowed to abstain produces fewer confident errors than one that must always deliver a score.
Not using, overriding, reversing (Art. 14(4)(d)). The reviewer has four actions: confirm, move up, move down, discard. Each needs a reason category, none needs an approval. And the applicant tracking system works without the assistant, because it never depended on it. The team can switch any position to the mode without pre-ranking.
Stopping (Art. 14(4)(e)). The stop button is a flag on the registry record. The head of recruiting sets it personally, without a deployment and without a change request, as can risk and IT operations. The gateway reads the flag on every call. The safe state is defined: applications stay in the queue, no proposal and no rejection is produced, nothing is lost. Resuming requires the same sign-off a new version requires. We learned that a stop button only engineers can press is not a control the business trusts.
Art. 14(5) as contrast. The two-person rule applies only to remote biometric identification. We do not copy it onto recruiting. Two signatures on every application do not double the quality, they halve the attention. We apply four eyes where the second person changes the outcome: cases overturned on a protected ground, and the first hundred cases after a model or prompt change.
Feedback. Every reviewer action lands in the evidence log with a reason category. The category points to one of three places for the fix: the prompt, the requirement profile as policy, or the evaluation set. Changes to the prompt or the profile get a new version and go through the same registration as a new model, and the triggering cases become regression tests. Oversight that does not change the system is theatre.
What the auditor sees: the registry record with tiers, sampling rates and who may stop, per application the evidence with model and prompt version, policy version, output, reviewer action and time, the catch rate on injected cases, and every stop event with person and timestamp. Where the law leaves room, this is what we chose: we read “effectively” in paragraph 1 as measurable. Oversight whose effect we cannot measure counts, for us, as absent. I am an engineer, not a lawyer, but this reading has so far survived every discussion with a legal department.
Typical mistakes
EU AI Act Article 14 checklist
Open questions
Frequently asked questions
Does this apply to us as a deployer if we only buy the assistant?
Yes, through two routes. Art. 14(3)(b) lets the provider identify oversight measures that you as deployer have to implement, and they appear in the instructions for use. Art. 26(2) additionally obliges you to assign oversight to people with competence, training, authority and support. Whoever substantially modifies the product or deploys it under their own name moves into the provider role under Art. 25.
Is it enough if a human approves every output?
Formally perhaps, practically not. Article 14 requires that the person understands outputs, detects anomalies and remains aware of automation bias. An approval rate near one hundred percent at review times below reading time proves the opposite. Tiered review with sampling and blind re-checks serves the purpose better than a signature on everything. More on this under AI governance.
Does the stop button have to be a real button?
No. Paragraph 4(e) speaks of a stop button or a similar procedure that brings the system to a halt in a safe state. Three properties matter: one authorised person can trigger it alone, it takes effect immediately on every call, and the safe state is defined in advance. For us it is a flag in the registry. The other articles of the series are here: all articles at a glance.
Related: EU AI Act · All articles · AI Governance · AI Consultant
Recent Posts
Recent Comments