Javascript is required
logo-dastralogo-dastra

AI system - High risk

Zelie Aderic
Zelie Aderic
6 minutes read time

The European Regulation on Artificial Intelligence (AI Act) creates a classification of AI systems according to their level of risk, with tailored obligations:

  • Unacceptable risk: prohibited systems (e.g. cognitive manipulation, social scoring).

  • High risk: systems subject to obligations relating to documentation, risk management, transparency, and registration in a European database (e.g. recruitment, HR management, or credit systems).

  • Limited risk: enhanced transparency obligations (e.g. generative AI, chatbots).

  • Minimal risk: no direct regulatory obligation, but good practices are encouraged.


There are two scenarios that allow an AI system to be qualified as high risk:

**1) If the AI system meets the following two cumulative criteria under Article 6 of the AI Act:

  • The product containing the AI system is subject to a mandatory conformity assessment by a third-party body before it is placed on the market or put into service;

  • The AI system is intended to be used as a safety component of a product covered by Union harmonisation legislation, or it is itself a product (for example, the AI system itself is a medical device).

For example, AI systems used for recruitment or medical software. **2) The AI system falls within the areas listed in Annex III: The intended use of the AI system falls within one of the use cases listed in Annex III of the Regulation:

No.

Area

High-risk use cases (AI systems intended for such use)

1

Biometrics (if use is permitted under EU or national law)

- Remote biometric identification (in real time or post-remote identification)
- Use for biometric categorisation based on sensitive or protected attributes, or their inference

- Or emotion recognition

Excludes simple biometric verification (e.g. facial authentication)

2

Critical infrastructure

- AI system used as a safety component for the management and operation of critical digital infrastructure for road traffic, water supply, gas, electricity, heating

3

Education & vocational training

AI systems intended to:
- Determine access to, admission to, or assignment to education or vocational training institutions
- Assess learning outcomes
- Assess the appropriate level of education
- Detect fraud or other prohibited behaviour during examinations

4

Employment & human resources

- Automated recruitment or selection (targeted offers, screening, evaluation)
- AI systems intended to make decisions affecting professional conditions, including promotion, dismissal, task allocation, performance evaluation

5

Essential public/private services

- Assessing eligibility for essential social benefits (healthcare, aid)
- Creditworthiness assessment / credit scoring
- Pricing in health / life insurance
- Emergency response handling (ranking emergency calls or prioritising dispatch of emergency health services)

6

Law enforcement (if use is permitted under EU or national law)

- Assessment of the risk of victimisation, offence, or reoffending
- AI systems intended to be used as polygraphs / similar tools
- Reliability of evidence in criminal investigations or prosecutions

- Assessment of the characteristics or criminal records of persons or groups
- Profiling in criminal investigations / sanctions

7

Migration, asylum, borders (if use is permitted under EU or national law)

- AI systems intended to be used as polygraphs / similar tools
- Assessment of risks posed by a person seeking entry into the territory
- Assessment of asylum / visa / residence permit applications and related complaints
- Identification or recognition (other than document verification) in the context of migration, asylum and border control management

8

Justice & democracy

- Assisting courts in interpreting and applying the law
- AI systems intended to influence the outcome of elections or referendums, or voting behaviour (excluding logistical tools)


**Exception: Exceptionally, the AI system is not considered “high-risk” even if it falls within Annex III, provided that it does not present a significant risk to the health, safety or fundamental rights of natural persons, and at least one of the following conditions is met:

  • The AI system is intended to perform a narrow procedural task (a technical or administrative task with no decision-making autonomy);

  • The AI system is designed to improve the outcome of an already performed human activity, without replacing it or influencing the final decision;

  • The AI system is intended to detect patterns of decision-making or deviations from prior human decisions, without replacing or influencing the human assessment already made, and without appropriate human oversight;

  • The AI system is intended to perform a preparatory task for an assessment carried out in the context of a use case recognised as “high-risk” by the European Commission (e.g. pre-screening files without autonomous decision-making).

However, an AI system that performs profiling of natural persons is always considered high-risk.

Subscribe to our newsletter

We'll send you occasional emails to keep you informed about our latest news and updates to our solution

* You can unsubscribe at any time using the link provided in each newsletter.