EU AI Act Article 60: Real-World Testing, Implemented in Practice
EU AI Act Article 60 allows a high-risk system to be tested in real-world conditions before it is placed on the market, but only with an approved testing plan, registration in the EU database, informed consent of the subjects and reversible results. For us the testing plan is a registry record with an expiry date, and every test starts in shadow mode: the model computes, humans decide, and only the comparison opens the pilot.
The most dangerous gap in an AI project lies between the evaluation set in the lab and the first real customer: different data, different users, different errors. Article 60 of the AI Regulation closes that gap with a regulated intermediate step, testing in real-world conditions outside a regulatory sandbox. It is addressed to providers and prospective providers of high-risk systems under Annex III; the consolidated text also names Annex I Section A, while Section B is covered by the new Article 60a. Deployers are testing partners (paragraph 2), subjects consent under Article 61, and the registration fields are in Annex IX.
The EU AI Act has been in force since 1 August 2024. Prohibitions and AI literacy have applied since February 2025, the obligations for GPAI models since August 2025, the transparency obligations under Article 50 since August 2026. After the postponement by the amending Regulation (EU) 2026/1744 (Digital Omnibus), the obligations for high-risk systems under Annex III apply from 2 December 2027, for Annex I products from 2 August 2028. Article 60 sits in Chapter VI, which has applied since August 2026; it matters precisely in the time until the Annex III obligations start. The Omnibus moved deadlines; it removed no obligation. The overview is in EU AI Act in practice.
What Article 60 requires
Art. 60(1) to (3): Providers or prospective providers may test high-risk systems in real-world conditions outside a regulatory sandbox, following a testing plan whose detailed elements the Commission will specify by implementing act; the prohibitions under Art. 5 are unaffected (paragraph 1). At any time before placing on the market or putting into service, alone or with deployers (paragraph 2). The test does not replace an ethical review (paragraph 3).
Art. 60(4)(a) to (c): Testing plan submitted to the market surveillance authority (a); approved by it, where silence for 30 days counts as approval unless national law requires an explicit authorisation (b); testing registered in the EU database under Art. 71(4), with a Union-wide unique identification number and the information in Annex IX, for Annex III points 1, 2, 6 and 7 in the non-public section (c).
Points (d) to (f): Provider established in the Union or with a legal representative there (d); data goes to third countries only with appropriate safeguards (e); duration as short as possible, at most six months, extendable once by six months after prior notification with justification (f).
Points (g) to (i): Vulnerable groups, by age or disability, protected (g); participating deployers informed, with the instructions for use under Art. 13 and an agreement on roles (h); subjects have given informed consent under Art. 61, and only in law enforcement is it sufficient instead that the testing and its outcome do them no harm and that their data is deleted afterwards (i).
Points (j) and (k): Effective oversight by persons with the qualification, capacity, training and authority (j); predictions, recommendations and decisions can be effectively reversed and disregarded (k).
Paragraphs 5 to 9: Subjects withdraw at any time without detriment or justification and may request immediate deletion of their data (paragraph 5). Authorities may request information and inspect unannounced (paragraph 6), and under Art. 76(3) suspend, terminate or have the test modified. Serious incidents are reported under Art. 73; the provider mitigates immediately, otherwise suspends or terminates, with a recall procedure (paragraph 7). Suspension, termination and final outcomes go to the authority (paragraph 8). The provider is liable for damage (paragraph 9).
Art. 61: Freely given, informed consent before participation, after concise information on nature, objectives and possible inconvenience (a), conditions and duration (b), rights, above all withdrawal at any time (c), the possibility to request that results be reversed or disregarded (d), the identification number of the test and the provider’s contact details (e). Dated, documented, copy to the subjects (paragraph 2).
Annex IX: Identification number, name and contact details of the provider and participating deployers, a brief description of the system and its intended purpose including the context of use, a summary of the testing plan, and the status suspended or terminated. To be kept up to date; not public under Art. 71(4).
The technical example
A provider has built a claims-triage model: claims at a health insurer are sorted by complexity, fraud suspicion and expected payout, with a recommendation to settle directly or send to expert review. The provider classifies it, as a precaution, as high-risk under Annex III point 5, because the triage prepares benefit decisions about insured persons; the insurer is the deployer. Before the obligations start, both want to know whether the model holds up on real claims what the evaluation set promises. “Just let it run alongside for a while” is exactly the case Article 60 regulates.
The data flow: the claim comes from the claims system to our policy gateway, is pseudonymised, checked against scope and data zone and passed to the model. Phase one, shadow mode: the output lands only in the evidence log, the handler does not see it and decides as before. Phase two, limited pilot: for consenting insured persons she sees the recommendation, still decides herself, and every decision with model involvement is tagged and reversible. Article 60 bites already in shadow mode, because real data of real people is processed there too; the consent under Art. 61 must be in place before the first claim.
How we implement it
At SIFAMO we treat a real-world test like a production system on a shorter leash: a registry record in the governance layer, no registration, no route, except that the record carries the status test and contains the testing plan. Simplified:
{
"model_id": "DE-V-2026-0212",
"status": "real_world_test",
"test_id": "(identification number from the EU database)",
"provider": "provider, legal representative in the Union",
"deployer": "health insurer, Head of Claims",
"purpose": "Triage of claims: recommendation, not a decision",
"phases": [{"name": "shadow", "weeks": 8, "live_effect": false},
{"name": "pilot", "weeks": 12, "live_effect": "recommendation visible"}],
"duration": {"weeks_max": 26, "extension": "once, prior notice with justification"},
"subjects": {"max": 3000, "consent": "Art. 61, status per subject ID",
"excluded": "minors, guardianship cases"},
"kill_criteria": ["disagreement with handlers above threshold, 5 days",
"overturned pilot decision linked to a protected attribute",
"serious incident under Art. 3(49)",
"request of the authority"],
"oversight": {"handlers": "named, trained", "blind_recheck": "share of pilot cases"},
"reversal": "every pilot decision tagged, reopen without model",
"deletion": "at end of test and on withdrawal",
"expiry": "end of test, then no route"
}
Three fields carry the article. duration is point (f): the expiry date is the end of the test, after which the model loses its route automatically; an extension is a new version with the notification to the authority attached. subjects is points (g) and (i). kill_criteria is our answer to paragraph 7 and Art. 76(3): the kill switch is held by the insurer’s business owner, and a request from the authority fires without discussion. The summary under Annex IX point 4 is generated from the record.
Approval and registration as a gate. The release gate for test status checks three things a production system does not know: plan submitted to the authority, with date; approval or expiry of the 30 days documented; identification number from the EU database in the test_id field. If one is missing, the route stays closed, even for shadow mode. Annex IX point 5 is a status field; when it flips to suspended or terminated, the database entry and the notification under paragraph 8 follow automatically.
Consent with pseudonyms. The information under Art. 61(1) is a versioned document with the five points, including the identification number and the way to request that a decision be reversed. The consent file with name, date and copy is kept by the insurer in the red data zone. The gateway receives only subject ID and status: consented with date and document version, or withdrawn. No status, no model call; the claim bypasses the model. Withdrawal under paragraph 5 triggers deletion of the pseudonymised test data; only the hash remains in the evidence log.
Shadow mode as the safe default. In shadow mode the gateway writes input hash, model and version, triage class and confidence to the evidence log per claim, and later adds the handler’s actual decision. The comparison of the two columns is the test result: agreement per class, deviations by direction, bias check with varied protected attributes. If it holds the thresholds from the plan across the whole shadow phase, business owner, risk and provider sign the transition to the pilot. A model that does not convince in the shadow has no business in a pilot.
Reversibility and oversight in the pilot. Point (k) is a data structure for us: every decision with a visible recommendation carries the test tag and the decision ID in the claims system. One query returns all affected claims; a reopen puts them back into expert review without the model, rehearsed in staging like the rollback. Point (j) is met by named, trained handlers, a blind re-check of a share of pilot cases and an escalation path with names up to the authority. A serious incident under Art. 3(49) triggers the report under Art. 73 and pulls the kill switch. The auditor sees plan, approval, identification number, consent status per subject, the shadow comparison, every pilot decision with its tag, and the final notification under paragraph 8.
Typical mistakes
“Just a shadow test, no subjects.” Real data of real people is processed in shadow mode too. Consent under Art. 61 is a precondition, not a pilot accessory.
Pilot without a way back. Recommendations flow into decisions, but nobody can say afterwards which ones. Point (k) requires that each one can be pulled back out.
Consent held by the provider. The provider collects names and signatures it does not need. The file belongs to the deployer; the provider needs the status per ID.
Registry not updated. Test suspended, entry says running. Annex IX point 5 and paragraph 8 require the notification.
EU AI Act Article 60 checklist
Classification settled: high-risk under Annex III or Annex I Section A, otherwise Art. 60 is not the framework.
Testing plan as a registry record: phases, duration up to six months, ceiling on subjects, kill criteria, deletion rule.
Plan submitted, approval or expiry of the deadline documented, identification number from the EU database in the record, agreement with the deployer on roles, oversight and kill switch.
Information under Art. 61 versioned, consent status per subject ID in the gateway, withdrawal triggers deletion.
Shadow mode before any live effect, comparison against the human decision with thresholds and bias check.
Every pilot decision tagged, reopen rehearsed in staging, blind re-check in place.
Incident reporting under Art. 73, suspension and final notification under paragraph 8 as a process with names.
Open questions
The Commission’s implementing act on the detailed elements of the testing plan under paragraph 1 is still pending. Our record anticipates it and will be adjusted.
Whether claims triage falls under Annex III at all is disputed. We classify high when in doubt; whoever classifies lower does not need Art. 60 but should run the same mechanics.
Whether shadow mode with no effect on the subjects is already testing under Art. 3(57), to which Art. 61 applies, is my reading, not a settled one. We obtain consent regardless.
Frequently asked questions
Does this apply to us as a deployer?
Article 60 is addressed to the provider. As an insurer that allows the test in your own house, you are a testing partner under paragraph 2 and point (h): you receive the instructions for use, sign the agreement on roles and, in practice, provide oversight and consents. If you develop the model yourself, you are the provider. What a deployer needs anyway is under AI Governance.
Is shadow mode without consent enough?
Not in my reading. In shadow mode too the system processes real data in real-world conditions, and point (i) ties admissibility to consent under Art. 61; the exception applies only to law enforcement. Without consent, only the lab remains, with synthetic or historical data on a suitable legal basis.
What about our test is public?
Nothing. Under Art. 71(4) the information on tests under Art. 60 is excluded from the public part of the database. The authority sees it, competitors do not. You still have to keep the entry up to date. The other articles in the series: all articles at a glance.
EU AI Act Article 60: Real-World Testing, Implemented in Practice
EU AI Act Article 60 allows a high-risk system to be tested in real-world conditions before it is placed on the market, but only with an approved testing plan, registration in the EU database, informed consent of the subjects and reversible results. For us the testing plan is a registry record with an expiry date, and every test starts in shadow mode: the model computes, humans decide, and only the comparison opens the pilot.
The most dangerous gap in an AI project lies between the evaluation set in the lab and the first real customer: different data, different users, different errors. Article 60 of the AI Regulation closes that gap with a regulated intermediate step, testing in real-world conditions outside a regulatory sandbox. It is addressed to providers and prospective providers of high-risk systems under Annex III; the consolidated text also names Annex I Section A, while Section B is covered by the new Article 60a. Deployers are testing partners (paragraph 2), subjects consent under Article 61, and the registration fields are in Annex IX.
The EU AI Act has been in force since 1 August 2024. Prohibitions and AI literacy have applied since February 2025, the obligations for GPAI models since August 2025, the transparency obligations under Article 50 since August 2026. After the postponement by the amending Regulation (EU) 2026/1744 (Digital Omnibus), the obligations for high-risk systems under Annex III apply from 2 December 2027, for Annex I products from 2 August 2028. Article 60 sits in Chapter VI, which has applied since August 2026; it matters precisely in the time until the Annex III obligations start. The Omnibus moved deadlines; it removed no obligation. The overview is in EU AI Act in practice.
What Article 60 requires
The technical example
A provider has built a claims-triage model: claims at a health insurer are sorted by complexity, fraud suspicion and expected payout, with a recommendation to settle directly or send to expert review. The provider classifies it, as a precaution, as high-risk under Annex III point 5, because the triage prepares benefit decisions about insured persons; the insurer is the deployer. Before the obligations start, both want to know whether the model holds up on real claims what the evaluation set promises. “Just let it run alongside for a while” is exactly the case Article 60 regulates.
The data flow: the claim comes from the claims system to our policy gateway, is pseudonymised, checked against scope and data zone and passed to the model. Phase one, shadow mode: the output lands only in the evidence log, the handler does not see it and decides as before. Phase two, limited pilot: for consenting insured persons she sees the recommendation, still decides herself, and every decision with model involvement is tagged and reversible. Article 60 bites already in shadow mode, because real data of real people is processed there too; the consent under Art. 61 must be in place before the first claim.
How we implement it
At SIFAMO we treat a real-world test like a production system on a shorter leash: a registry record in the governance layer, no registration, no route, except that the record carries the status test and contains the testing plan. Simplified:
Three fields carry the article. duration is point (f): the expiry date is the end of the test, after which the model loses its route automatically; an extension is a new version with the notification to the authority attached. subjects is points (g) and (i). kill_criteria is our answer to paragraph 7 and Art. 76(3): the kill switch is held by the insurer’s business owner, and a request from the authority fires without discussion. The summary under Annex IX point 4 is generated from the record.
Approval and registration as a gate. The release gate for test status checks three things a production system does not know: plan submitted to the authority, with date; approval or expiry of the 30 days documented; identification number from the EU database in the test_id field. If one is missing, the route stays closed, even for shadow mode. Annex IX point 5 is a status field; when it flips to suspended or terminated, the database entry and the notification under paragraph 8 follow automatically.
Consent with pseudonyms. The information under Art. 61(1) is a versioned document with the five points, including the identification number and the way to request that a decision be reversed. The consent file with name, date and copy is kept by the insurer in the red data zone. The gateway receives only subject ID and status: consented with date and document version, or withdrawn. No status, no model call; the claim bypasses the model. Withdrawal under paragraph 5 triggers deletion of the pseudonymised test data; only the hash remains in the evidence log.
Shadow mode as the safe default. In shadow mode the gateway writes input hash, model and version, triage class and confidence to the evidence log per claim, and later adds the handler’s actual decision. The comparison of the two columns is the test result: agreement per class, deviations by direction, bias check with varied protected attributes. If it holds the thresholds from the plan across the whole shadow phase, business owner, risk and provider sign the transition to the pilot. A model that does not convince in the shadow has no business in a pilot.
Reversibility and oversight in the pilot. Point (k) is a data structure for us: every decision with a visible recommendation carries the test tag and the decision ID in the claims system. One query returns all affected claims; a reopen puts them back into expert review without the model, rehearsed in staging like the rollback. Point (j) is met by named, trained handlers, a blind re-check of a share of pilot cases and an escalation path with names up to the authority. A serious incident under Art. 3(49) triggers the report under Art. 73 and pulls the kill switch. The auditor sees plan, approval, identification number, consent status per subject, the shadow comparison, every pilot decision with its tag, and the final notification under paragraph 8.
Typical mistakes
EU AI Act Article 60 checklist
Open questions
Frequently asked questions
Does this apply to us as a deployer?
Article 60 is addressed to the provider. As an insurer that allows the test in your own house, you are a testing partner under paragraph 2 and point (h): you receive the instructions for use, sign the agreement on roles and, in practice, provide oversight and consents. If you develop the model yourself, you are the provider. What a deployer needs anyway is under AI Governance.
Is shadow mode without consent enough?
Not in my reading. In shadow mode too the system processes real data in real-world conditions, and point (i) ties admissibility to consent under Art. 61; the exception applies only to law enforcement. Without consent, only the lab remains, with synthetic or historical data on a suitable legal basis.
What about our test is public?
Nothing. Under Art. 71(4) the information on tests under Art. 60 is excluded from the public part of the database. The authority sees it, competitors do not. You still have to keep the entry up to date. The other articles in the series: all articles at a glance.
Related: EU AI Act · All articles · AI Governance · AI Consultant
Recent Posts
Recent Comments