Artificial Intelligence

The classification of an AI system as high-risk

The AI Act under Article 6 indicates the AI systems classified as high-risk:​

• The AI system is intended to be used as a safety component of a product, or the AI system is itself a product, covered by EU harmonisation legislation listed in Annex 1
• The product whose safety component is the AI system, or the AI system itself as a product, is required to undergo a third-party conformity assessment, with a view to the placing on the market or putting into service of that product pursuant to EU harmonisation legislation listed in Annex I
• AI Systems listed in Annex III, namely:
  • Biometrics
  • Critical infrastructures
  • Education and vocational training
  • Employment and workforce management
  • Essential public and private services
  • Law enforcement
  • Migration, asylum, and border control management
  • Administration of justice and democratic processes
• AI systems which are referred to in Annex III and perform profiling of natural persons

It is pertinent to note that the AI Act also provides a filter mechanism that excludes the AI systems listed as high-risk under Annex III, from being high-risk, where they do not pose a significant risk of harm to the health, safety or fundamental rights of natural persons, including by not materially influencing the outcome of decision making. This applies if the AI system performs narrow procedural tasks, improves the result of previous human activities, does not influence human decisions, or does purely preparatory tasks.

Please refer to the AI Act for the complete list of high-risk AI Systems, including any applicable exceptions.

Obligations with respect to high-risk AI systems

High-risk AI systems must meet stringent requirements before they can be placed on the market, or put into service, including:

• Having a Risk Management System​ and mitigation and control measures
• Ensuring Data governance and management practices ​
• Keeping technical documentation and logs
• Assuring transparency and provide information​
• Establishing adequate Human Oversight
• Achieving an appropriate level of accuracy, robustness, and cybersecurity
• Affixing the CE Marking​

Transparency is at the core of the EU AI Act. There are obligations tailored to the High-Risk AI Systems, and under Article 50, the AI Act establishes a general set of transparency requirements with a wider scope.

Providers shall ensure that natural persons are notified that they are interacting with an AI system, unless this is evident.

Providers of AI systems, including, general-purpose AI systems (GPAIs), which generate synthetic audio, image, video or text content, shall ensure that the outputs are marked in a machine readable format and detectable as artificially generated or manipulated.

Deployers shall inform natural persons that emotion recognition or biometric categorisation systems are applied to them.

Deployers shall label as artificially generated image, audio or video content constituting a deep fake, as well as text which is published with the purpose of informing the public on matters of public interest.

For AI systems which do not fall under the specified categories (prohibited, high risk, and transparency risk), the AI Act does not impose any mandatory obligations.

Nonetheless, the AI Act under Article 95, encourages the involved parties, such as providers and deployers, to voluntarily adopt the codes of conduct which will be issued by the AI Office and the Member States. Codes of conduct may also be drawn up by providers or deployers of AI systems.

‘General-purpose AI model’ means an AI model, including where such an AI model is trained with a large amount of data using self-supervision at scale, that displays significant generality and is capable of competently performing a wide range of distinct tasks regardless of the way the model is placed on the market and that can be integrated into a variety of downstream systems or applications, except AI models that are used for research, development or prototyping activities before they are placed on the market.

The definition of a GPAI Model can be found in Article 3 of the AI Act, and if any of the conditions listed in Article 51 of the AI Act are met, a GPAI Model shall be classified as a GPAI Model with systemic risk, and in particular instances, as referred to in Article 52, the relevant provider is obliged to notify the Commission within the stipulated time frame.

GPAI Models
Article 53 of the AI Act lists the obligations that providers of GPAI models must fulfil. These obligations include, but are not limited to: drawing up and keeping up-to-date technical documentation, and putting into place a policy respecting EU copyright law. Exceptions to these obligations apply.

Article 54 of the AI Act obliges providers of GPAI models established in third countries to appoint, by written mandate, an authorised representative to perform the task specified in the mandate received from the provider. The authorised representative may be requested by the AI Office to provide a copy of the same mandate. The Article also gives a list of the minimum requirements which the mandate should have.

GPAI Models with systemic risks
In addition to the obligations listed in Articles 53 and 54, providers of GPAI models with systemic risks, must also fulfil the obligations listed under Article 55 of the AI Act, such as, performing model evaluation in accordance with standardised protocols and tools.

A conformity assessment should be performed before a high-risk AI system is placed on the market or put into service as stipulated under Article 16 of the AI Act.

Further, by virtue of Article 43 of the AI Act, whenever a substantial change occurs that may affect the compliance of a high-risk AI system with the AI Act or when the high-risk AI system’s intended purpose changes, that high-risk AI system should undergo a new conformity assessment.

Article 43 of the AI Act also distinguishes between when an Internal Conformity Assessment Procedure and when an External Conformity Assessment Procedure are to be done.

Article 43(1)(b) of the AI Act lays down that where any of the following 4 points apply, the demonstration of conformity of high-risk AI systems must be done with the involvement of a notified body, as referred to in Annex VII of the AI Act:

• The harmonised standards referred to in Article 40 of the AI Act do not exist, and the common specifications referred to in Article 41 of the AI Act are not available;
• The provider has not applied, or has applied only part of, the harmonised standard;
• The common specifications referred to in the first point exist, but the provider has not applied them;
• One or more of the harmonised standards referred to in the first point has been published with a restriction, and only on the part of the standard that was restricted

A Provider may choose any notified body, except for when the system is intended to be put into service by law enforcement, immigration, or asylum authorities or by Union institutions, bodies, offices or agencies, the market surveillance authority referred to in Article 74(8) or (9) of the AI Act, as applicable, shall act as a notified body.

Article 43(2) of the AI Act stipulates that for high-risk AI systems referred to in points 2 to 8 of Annex III of the AI Act, a conformity assessment procedure based on internal control shall be followed, as referred to in Annex VI of the AI Act.

The AI Act was published in the Official Journal of the European Union on the 12th of July 2024, indicating that its entry into force will be in August 2024. However, not all aspects of the AI Act will apply from the said date. Article 113 of the AI Act provides the dates of entry into force of the different parts of the AI Act.

Member states shall establish or designate as national competent authorities at least one notifying authority and at least one market surveillance authority.

The MDIA and the IDPC (Information Data Protection Commission) have already been identified as market surveillance authorities.

The MDIA and the National Accreditation Board have been identified as notifying authorities.

It is planned that the process of designation is completed within the stipulated deadline of 2nd August 2025.

Deployers, providers, importers, distributors, product manufactures and authorised representatives all have specific roles and responsibilities under the AI Act. Deployers, who use AI systems under their authority (excluding personal activities), must ensure compliance with the AI Act. Deployers should in particular take appropriate technical and organisational measures to ensure they use high-risk AI systems in accordance with the instructions of use and certain other obligations should be provided for with regard to monitoring of the functioning of the AI systems and with regard to record-keeping, as appropriate.

Providers, who develop or put on the market or into service AI systems, must also ensure compliance with AI Act requirements, including, conducting conformity assessments, affixing CE markings, maintaining documentation, and take corrective actions with respect to high risk AI systems, when required. Once the high risk AI system is on the market, providers must have a post-market monitoring system in place. Providers must also report serious incidents and malfunctions of high risk AI systems.

Importers, who place on the market AI systems that bears the name or trademark of a natural or legal person established in a third country, must verify compliance with the AI Act. Some of the obligations imposed require the importer to ensure that the high-risk AI system bears the required CE marking, ensure to indicate their contact details on the AI high-risk systems, and maintain records for 10 years.

Distributors, who are persons within the supply chain that make AI systems available on the Union market. Distributors must also be in line with the AI Act. For example, before making  high-risk AI systems available, deployers shall store and transport them properly and take corrective actions for those that are non-compliant.

Authorised Representatives should be appointed by written mandate by providers established in third countries prior to making their AI systems available in the Union. This authorised representative plays a pivotal role in ensuring the compliance of the high-risk AI systems placed on the market or put into service in the Union by those providers who are not established in the Union and in serving as their contact person established in the Union.

The Product Manufacturer shall be considered to be the provider of the high-risk AI system for high-risk AI systems that are safety components of products covered by the Union harmonisation legislation listed in Section A of Annex I of the AI Act, and shall be subject to the obligations under Article 16 of the AI Act under either of the following circumstances: (a) the high-risk AI system is placed on the market together with the product under the name or trademark of the product manufacturer; (b) the high-risk AI system is put into service under the name or trademark of the product manufacturer after the product has been placed on the market.

The AI Office is established by Article 64 of the AI Act within the Commission. It plays a crucial role in supporting the accelerated development of AI systems and provide advice on the implementation of the AI Act. Its functions include developing voluntary model terms for contracts, supporting the detection and labelling of AI-generated content investigating potential infringements of rules on GPAI models and systems, and contributing to international cooperation related to AI Act and governance.

By virtue of Article 65 of the AI Act, the AI Board is established and consists of one representative per Member State. The European Data Protection Supervisor shall also participate as an observer and the AI Office shall also attend the meetings of the Board, without taking part in votes. Other national and Union authorities, bodies, or experts may be invited to the meetings without taking part in the votes. Its functions include providing advice and to facilitate the effective application of the AI Act, supporting national authorities in implementation of this Regulation, and collect and share technical and regulatory expertise among Member States.

For further information, please refer to the EU Commision’s Website.