Overview of the operators
Art. 3(8) of the AI Act lists the following operators:
- Providers
- Product manufacturers
- Deployers
- Authorised representatives
- Importers
- Distributors
The precise definitions of the individual operators may be found here.
Although “users” and “affected persons” are also mentioned in the AI Act, they are not operators within the meaning of the Act.
The Commission Guidelines on prohibited artificial intelligence practices and the Guidelines on the implementation of the transparency obligations for certain AI systems under Art. 50 of the AI Act explain the roles of provider and deployer, in particular, in more detail. The explanations below are based on these guidelines.
Who is a provider?
A provider is a natural or legal person, public authority, agency or other body that develops an AI system or has such a system developed and places it on the market or puts it into service under its own name or trademark in the EU. This is regardless of whether the provider is located in the EU or in a third country.
Providers can take various roles in the AI value chain, including the development, training, evaluation and optimisation of AI systems. Even though the development of an AI system often takes an interdisciplinary approach with various experts working together, the person distributing or using the system under their own name or trademark is the provider.
An important point is that providers must ensure that their AI systems meet all relevant requirements before placing it on the market or putting it into service.
The responsibilities along the AI value chain are set out in Art. 25. Importers, distributors, deployers and other third parties may also be considered providers if they
- put their name or trademark on an AI system already placed on the market,
- make substantial modifications (Art. 3(23)) to an AI system that has already been placed on the market that affect its compliance or modify its intended purpose,
- modify the intended purpose of an AI system in such a way that it becomes a high-risk AI system.
In these cases, they are considered providers and are subject to the same obligations as the original providers.
Who is a product manufacturer?
The AI Act refers to product manufacturers as one of the parties involved but does not give a precise explanation of them. The reference to EU product legislation in Annex I to the AI Act, however, makes clear that product manufacturer within the meaning of the AI Act means the manufacturer of the product in question.
For example, a manufacturer is defined in the Radio Equipment Directive as “any natural or legal person who manufactures radio equipment or has radio equipment designed or manufactured, and markets that equipment under his name or trade mark” (Art. 2(1)(12) RED).
If an AI system is integrated into a product as a safety component, the manufacturer of the product must comply with the obligations for providers of AI systems set out in the AI Act. In these situations, product manufacturers are considered providers.
Who is a deployer?
A deployer is a natural or legal person, public authority, agency or other body using an AI system under its authority except where the AI system is used in the course of a personal non-professional activity.
“Authority” over an AI system should be understood as assuming responsibility over the decision to deploy the system and over the manner of its actual use, including the results.
Deployers fall within the scope of the AI Act if their place of establishment or location is within the EU or, if they are located in a third country, the output of the AI system is used in the EU.
Where the deployer of an AI system is a legal person under whose authority the system is used, the individual employees that act within the procedures and under the control of that person should not be considered to be the deployer.
A legal person also remains a deployer if it involves third parties (eg contractors, external staff) in the operation of the system on its behalf and under its responsibility and control.
Deployers may also be treated as or considered providers in the circumstances described in Art. 25.
Who is an authorised representative?
An authorised representative is a natural or legal person located or established in the EU who has received and accepted a written mandate from a provider of an AI system or a general-purpose AI model (GPAI) to, respectively, perform and carry out on its behalf the obligations and procedures established by the AI Act.
The written mandate clearly documents who is authorised to act on behalf of the provider and the responsibility remains within European jurisdiction. The authorised representative thus takes on all the tasks required by the AI Act, such as carrying out risk and conformity assessments and reporting incidents, and ensures that these requirements are met correctly and on time. In this way, providers – especially if they are not based in the EU themselves – can ensure that the legal requirements are met in the EU, while the authorised representative only acts within the bounds of the mandate received and bears responsibility for the proper fulfilment of obligations.
Who is an importer?
An importer is a natural or legal person located or established in the EU that places on the market an AI system that bears the name or trademark of a person established in a third country (Art. 3(6)).
The importer is the link in the AI value chain that imports an AI system from an external provider into the EU.
Importers may also be treated as or considered providers in the circumstances described in Art. 25. Like distributors, importers might therefore have to fulfil all the obligations of the AI Act; if they are acting in both roles simultaneously, the requirements apply cumulatively. As soon as an operator puts their name on a high-risk AI system already placed on the market, substantially modifies it or modifies its intended purpose in such a way that it remains a high-risk AI system, the operator is considered a provider and must take on all related obligations, regardless of contractual arrangements. The same applies if the intended purpose of an AI system that has already been placed on the market and was not previously classified as high-risk is modified in such a way that it becomes a high-risk AI system. These provisions apply in addition to and together with the existing EU harmonisation legislation.
Who is a distributor?
A distributor is a natural or legal person, other than the provider or the importer, that makes an AI system already placed on the market available on the EU market (Art. 3(7)). The distributor’s activities thus occur after the system has been placed on the market and cover the mere distribution to deployers, end-users or resellers.
Just like importers, distributors may also be treated as or considered providers in the circumstances described in Art. 25. The obligations referred to here are in addition to the regulations described in the section on importers and highlight the fact that operators acting in multiple roles at the same time (importers, distributors, providers) have to view their obligations cumulatively.
Who are “affected persons”?
The AI Act does not define “affected persons” as a separate operator group but rather refers to all natural persons exposed to the decisions, output or operational procedures of an AI system in any way without being part of the delivery or usage chain themselves.
They are particularly mentioned in the AI Act in the following contexts:
- Information obligation – deployers must inform the natural persons exposed thereto of the operation of the AI system and the processing of data (in accordance with the GDPR) (Art. 50(3)).
- Risk and impact assessment – the consequences of the use of “real-time” remote biometric identification (RBI) systems for all persons concerned (Art. 5(2)) must be examined.
- Right to explanation – any person subject to a decision taken by a high-risk AI system which produces legal effects or affects fundamental rights has the right to obtain a clear explanation of the role of the AI system in the decision-making procedure (Art. 86).
Overview of the actions “placing on the market”, “putting into service” and “use”
Different operators carry out different actions in the AI value chain. These include the “placing on the market”, “making available” and “putting into service” of AI systems as defined in the AI Act. Explanations of these activities may be found in the Commission Guidelines Commission Guidelines on prohibited artificial intelligence parctices.
Art. 3(9) defines “placing on the market” as the first making available of an AI system or a GPAI model on the EU market, which also refers to the next definition.
Art. 3(10) defines “making available on the market” as the supply of an AI system for distribution or use in the EU in the course of a commercial activity, whether in return for payment or free of charge. This is regardless of the means of supply, such as through an application programming interface (API), via cloud, direct downloads, as physical copies, or embedded in physical products.
Example: an
AI system developed by a provider outside the
EU is placed on the
EU market for the first time when it is offered in return for payment or free of charge in one or more
EU Member States. Such placing on the market may occur by providing access to the system online through an
API or other user interface.
Art. 3(11) defines “putting into service” as the supply of an AI system for first use in the EU. This covers both supply for first use to third parties as well as in-house development and deployment. The use must correspond to the intended purpose of the AI system.
Art. 3(12) defines “intended purpose” as the use for which an AI system is intended by the provider, including the specific context and conditions of use, as specified in the information supplied by the provider in the instructions for use, promotional or sales materials and statements, as well as in the technical documentation.
Examples:
1. A provider builds an
AI system outside the
EU and supplies that system to a law enforcement authority or to a company in the
EU to be used for the first time, thereby putting it into service.
2. A public authority develops a scoring system in-house and deploys it to predict the risk of fraud of household allowance beneficiaries, thereby putting it into service.
While the “use” of an AI system is not explicitly defined in the AI Act, it should be understood in a broad manner to cover the use or deployment of the system at any moment of its lifecycle after having been placed on the market or put into service. This may also cover the integration of the AI system in other services, processes or infrastructure.
While providers of AI systems must consider the conditions of use which may be reasonably foreseen (intended use and possible misuse), deployers remain responsible for taking the lawful conditions for the use of the system into account.
Example: an
AI system used by an employer to infer emotions at the workspace is prohibited, except when used for medical or safety purposes. The prohibition applies to deployers regardless of whether the provider of the system has excluded such use in its terms of use.
Obligations along the AI value chain
The AI Act does not only place requirements on AI systems themselves but assigns obligations along the entire value chain. For its practical implementation, therefore, it is first necessary to clarify which role an operator is taking in the lifecycle of an AI system and what the resulting obligations are.
Operators’ obligations
Figure 1 below shows the specific Art. that have to be observed according to the AI system’s/GPAI model’s risk category.
List of provider obligations by risk category. The scope of obligations increases as the risk classification of the AI system/GPAI model rises.
AI systems with minimal risk
AI literacy
The most basic obligation of any provider is AI literacy (Art. 3(56)): it requires the knowledge and understanding necessary to make an informed deployment of AI systems, assess opportunities and risks and avoid possible harm.
On this basis, Art. 4 requires providers to take specific measures to promote AI literacy among their staff and other persons dealing with the AI system on their behalf.
The ability to understand AI systems, look at them critically and deploy them responsibly is vital today. AI literacy is essential to make informed decisions, minimise risks and comply with statutory requirements. At the same time, AI literacy enables the best possible use to be made of the potential of AI and innovations to be developed.
The AI Act does not specify what the support should look like. No standardised training measures are laid down, nor must any specific level of AI literacy be ensured for individual persons.
The individual context of an organisation is fundamental: the organisation’s role as a provider or deployer developing or using AI systems, together with the associated risk, and the staff’s previous knowledge and experience are decisive factors.
Developing AI literacy is an ongoing process. AI literacy should be regularly updated and adapted to technological developments.
The AI Act does not require certification for participation in training or qualification measures. Measures can be carried out internally or externally.
Measures taken to build AI literacy should be documented.
Corrective actions
A further feature that all providers have in common is that they have to take certain corrective actions under certain circumstances. There are considerable differences in the specific nature of this obligation depending on the type of provider and the trigger.
- Art. 20 places a proactive obligation only on providers of high-risk AI systems: if they find that their system is not or is no longer in conformity with the AI Act, they must take action immediately without waiting for intervention by the authorities.
- Art. 79, on the other hand, applies to all risk categories and all operators (providers, importers, distributors, deployers). The obligation in this case is reactive, triggered by the finding of a national market surveillance authority that an AI system presents a risk to health, safety or fundamental rights.
- Art. 93 relates specifically to providers of GPAI models and comes under the competence of the European Commission rather than national authorities. Instead of “corrective actions”, this provision refers to measures to comply with the obligations set out in Art. 53 and 54 and mitigation measures in the case of systemic risks.
AI systems with limited risk
The AI systems mentioned in Art. 50 are classed as “limited risk”. Unlike high-risk AI systems, they are not subject to comprehensive conformity requirements but merely have to fulfil certain transparency obligations. The term “limited risk” is a practical classification used by the European Commission and other operators and not linguistically defined in the AI Act. More detailed descriptions of the requirements and recommended actions relating to Art. 50 may be found here.
Paragraphs 1, 2 and 5 are relevant to providers:
AI systems intended to interact directly with natural persons
The obligation in Art. 50(1) is that affected persons must be able to recognise that they are interacting with an AI system, unless this is obvious anyway from the context or the system is being used to combat criminal offences with the appropriate safeguards.
AI systems (including GPAI systems) generating synthetic content
Art. 50(2) requires the outputs to be marked in a machine-readable format as artificially generated. The technical solution must be effective, interoperable, robust and reliable. There are exemptions for assistive functions for standard editing without a substantial alteration of content, or, again, for combating criminal offences.
Accessibility requirements
The obligation found in Art. 50(5) is not a standalone requirement but accompanies all the information obligations in paragraphs 1 to 4, whether these originally apply to providers or deployers. Paragraph 5 sets out that the information referred to in paragraphs 1 to 4 must be provided in good time (at the latest at the time of the first interaction or exposure) and in a clear, distinguishable and accessible manner. For the provider obligations in paragraphs 1 and 2, this means that it is not enough for there to be marking, the marking itself must also meet these qualitative requirements.
GPAI models
A general-purpose AI (GPAI) model is a model that can serve a variety of purposes. It must be broadly applicable and able to be integrated into different systems or applications.
The obligations for providers of GPAI models are set out in Art. 53 and 54. A more detailed explanation of these and classification rules for AI models may be found here.
Transparency towards downstream operators
In accordance with Art. 53(1)(b), providers of GPAI models must make available to downstream providers, such as providers of AI systems, comprehensive information and documentation to make transparent the capabilities and limitations of their model. This information should enable the downstream providers to comply with their obligations pursuant to the AI Act and to integrate the model into their systems safely. The documentation must contain, at a minimum, the elements set out in Annex XII and be made available to the Artificial Intelligence Office and the competent national authorities upon request.
Data requirements
In accordance with Art. 53(1)(c) and (d), providers of GPAI models must ensure that the data used for training is representative, diverse and sufficient to guarantee a high model performance and generalisability. The data must also be sourced so that they enable compliance with the requirements of the AI Act and do not promote impermissible biases or discrimination. Providers are further obliged to draw up, and comply with, a policy to comply with copyright law.
Technical documentation
In accordance with Art. 53(1)(a), providers of GPAI models are required to draw up and keep up-to-date technical documentation, which must contain, at a minimum, the information set out in Annex XI and include, among other things, the technical specifications of the model.
Cooperation with authorities
Art. 53(3) sets out that providers of GPAI models must cooperate with the European Commission and the national competent authorities in the exercise of their competences and powers.
Appointment of authorised representative
In accordance with Art. 54, providers of GPAI models established in third countries must appoint an authorised representative in the EU prior to placing their model on the market in the EU. The authorised representative is responsible for monitoring compliance with the obligations of the Act, keeping the technical documentation and cooperating with the authorities. The authorised representative receives a written mandate enabling it to perform these tasks and empowering it to be addressed by the authorities. If the authorised representative suspects that the provider is acting contrary to the Act, it must terminate the mandate and inform the AI Office. The obligation set out in this Art. does not apply to providers of AI models that are released under a free and open-source licence, unless the models present systemic risks.
GPAI models with systemic risk
A GPAI model with systemic risk is a general-purpose AI model that is classified as a model with systemic risk due to its capabilities and impact. This may be the case if it has high-impact capabilities evaluated on the basis of technical tools and methodologies or if the European Commission or the scientific panel has made a decision to this effect. In addition to the obligations set out in Art. 53 and 54, these providers must fulfil the obligations set out in Art. 55.
A more detailed explanation of these and classification rules for AI models may be found here.
Risk management
In accordance with Art. 55(1)(a), providers of GPAI models must carry out risk management encompassing a model evaluation in accordance with standardised protocols and tools, including adversarial testing with a view to identifying and mitigating systemic risks. In addition, in accordance with Art. 55(1)(b), they must assess and mitigate possible systemic risks at EU level, including their sources, that may stem from the development, distribution or use of GPAI models.
Accuracy, robustness and cybersecurity
In accordance with Art. 55(1)(d), providers of GPAI models must ensure an adequate level of cybersecurity protection for their models and the physical infrastructure. This includes implementing a Safety and Security Framework covering the entire lifecycle of the model, including development, placing on the market and use.
Reporting obligations to authorities
In accordance with Art. 55(1)(c), providers of GPAI models must keep track of, document, and report, without undue delay, to the AI Office and, as appropriate, to national competent authorities, serious incidents and possible corrective measures to address them.
Registration and reporting obligations
In accordance with Art. 52(1), providers of GPAI models must notify the European Commission without delay if their model meets the conditions for a model with systemic risk, especially if it has high-impact capabilities. They must provide the information necessary to demonstrate that the conditions have been met.
High-risk AI systems
Providers of high-risk AI systems must ensure that their systems meet the requirements of Art. 9 to 15 and also meet the other obligations under Art. 16 to 27.
A more detailed description of the rules for high-risk AI systems under Art. 9 to 15 may be found here.
Transparency towards downstream operators
In accordance with Art. 13, providers of high-risk AI systems must ensure that their systems are transparent and enable deployers to interpret the output and use it appropriately. To do so, they must provide instructions for use that include concise, complete and correct information about the system, such as its characteristics, capabilities and limitations of performance and information about possible risks and restrictions.
Data requirements
Art. 10 sets out that providers of high-risk AI systems must ensure that the training, validation and testing data sets used are of high quality and meet the specific requirements of the AI Act. To do so, they must put in place data governance and management practices to ensure that the data sets are relevant, representative and free of errors and do not contain biases that are likely to affect the health and safety of persons or have a negative impact on fundamental rights.
Technical documentation
In accordance with Art. 11, providers of high-risk AI systems must draw up technical documentation containing all the information necessary to demonstrate that the system complies with the requirements of the AI Act. The documentation must be drawn up before the system is placed on the market or put into service and must be kept up-to date to provide competent authorities and notified bodies with the necessary information in a clear and comprehensive form to assess the AI system.
Cooperation with authorities
In accordance with Art. 21, providers of high-risk AI systems must cooperate with the competent authorities and, upon request, provide them all the information and documentation necessary to demonstrate the conformity of the system with the requirements of the AI Act. This includes the automatically generated logs of the system (see Art. 12) to the extent such logs are under the provider’s control. The competent authorities must treat the information as confidential.
Appointment of authorised representative (for third countries)
In accordance with Art. 22, prior to making their high-risk AI systems available on the EU market, providers established in third countries must appoint an authorised representative which is established in the EU, acts as a point of contact for the competent authorities and carries out certain tasks, such as verifying the declaration of conformity and the technical documentation, providing information and documentation, and cooperating with the authorities on risk mitigation measures.
Risk management
In accordance with Art. 9, providers of high-risk AI systems must establish a risk management system that identifies, analyses and evaluates risks that may arise from the use of their AI system. The risk management system must be applied and updated throughout the entire lifecycle of the AI system and is intended to ensure that risks are minimised and health, safety and fundamental rights are upheld.
Accuracy, robustness and cybersecurity
In accordance with Art. 15, high-risk AI systems must be designed and developed in such a way that they achieve an appropriate level of accuracy, robustness and cybersecurity and that they perform consistently in those respects throughout their lifecycle. This includes the development of benchmarks and measurement methodologies, the declaration of accuracy metrics in the instructions for use and the implementation of technical and organisational measures to ensure the robustness and cybersecurity of the system.
Registration obligations
In accordance with Art. 16(i), providers of high-risk AI systems must register themselves and their AI systems in the EU database on the basis of Art. 49 before placing their system on the market or putting it into service. This also applies to providers that have concluded that their AI system is not high-risk. The registration takes place in a public section of the EU database except in the following circumstances:
- AI systems in the areas of law enforcement, migration, asylum and border control management must be registered in a secure non-public section of the EU database.
- High-risk AI systems referred to in point 2 of Annex III are instead registered at national level.
Reporting obligations to authorities
In addition, in accordance with Art. 73, providers of high-risk AI systems must report serious incidents to the competent market surveillance authorities immediately, or in any case within certain time limits. Following the reporting, they must perform investigations, conduct risk assessments and take corrective action in cooperation with the competent authorities. This process is part of the quality management system pursuant to Art. 17.
Recording of events
In accordance with Art. 12, providers of high-risk AI systems must ensure that their system automatically records events over the lifetime of the system to ensure traceability. The recordings must include the identification of risks, post-market monitoring and monitoring the operation of the AI system.
Ensuring human oversight
In accordance with Art. 14, providers of high-risk AI systems must ensure that their systems are designed and developed in such a way that they can be effectively overseen by natural persons during the period in which they are in use in order to prevent or minimise the risks to health, safety or fundamental rights. To do so, they must take suitable measures such as establishing tools for human oversight or the provision of information and training for deployers to ensure that they can correctly interpret the system’s output and are able to take the necessary measures.
Marking obligations
In accordance with Art. 16(b), providers of high-risk AI systems must indicate on the high-risk AI system, on its packaging or its accompanying documentation, their name, registered trade name or registered trade mark and the address at which they can be contacted.
Ensuring accessibility requirements
In accordance with Art. 16(l), providers of high-risk AI systems must ensure that their high-risk AI system complies with accessibility requirements in accordance with Directives (EU) 2016/2102 and (EU) 2019/882.
Quality management
In accordance with Art. 16(c), providers of high-risk AI systems must set up a quality management system (QMS) which complies with Art. 17 and ensures compliance with the requirements of the AI Act. The QMS must be documented in writing and cover at least aspects such as the design, development, testing, validation, data management, risk management, post-market monitoring and reporting of serious incidents.
Documentation-keeping obligations
Art. 16(d) and (e) specify that providers of high-risk AI systems must keep the documentation referred to in Art. 18 and the logs referred to in Art. 19 for a certain period in order to demonstrate the compliance of their AI systems with the requirements of the AI Act. This includes the technical documentation, the documentation concerning the QMS, the EU declaration of conformity and the automatically generated logs of the AI system that must be kept for at least six months, unless provided otherwise in the applicable EU or national law.
Conformity assessment procedure, declaration and marking
In accordance with Art. 16(f), providers of high-risk AI systems must ensure that their system undergoes a conformity assessment procedure as referred to in Art. 43, prior to its being placed on the market or put into service. Art. 16(g) then specifies that they must draw up an EU declaration of conformity in accordance with Art. 47 confirming that the system meets the requirements of the AI Act. Finally, Art. 16(h) sets out that providers of high-risk AI systems must affix the CE marking to the system or its packaging to indicate conformity with the AI Act, in accordance with Art. 48.
Deployers’ obligations
Like providers, deployers must fulfil certain obligations along the AI value chain.
Figure 2 below shows the specific Art. that have to be observed according to the AI system’s risk category.
Tabellarische Aufstellung der Betreiberverpflichtungen nach Risikostufen. Der Umfang der Verpflichtungen nimmt entsprechend der steigenden Risikoklassifizierung des KI-Systems zu
AI systems with minimal risk
AI literacy
Just as for providers of AI systems, AI literacy (Art. 3(56)) is of decisive importance for deployers as well. Pursuant to Art. 4, they, too, must have the knowledge and understanding necessary to make an informed deployment of AI systems, assess opportunities and risks and avoid possible harm.
Corrective actions
Like providers, deployers are required to take corrective actions under certain circumstances. In accordance with Art. 79, where a market surveillance authority finds that an AI system does not comply with the requirements of the AI Act, deployers must take appropriate corrective actions to bring the AI system into compliance or stop using it.
AI systems with limited risk
The AI systems mentioned in Art. 50 are classed as “limited risk”. Unlike high-risk AI systems, they are not subject to comprehensive conformity requirements but merely have to fulfil certain transparency obligations. The term “limited risk” is a practical classification used by the European Commission and other operators and not linguistically defined in the AI Act. More detailed descriptions of the requirements and recommended actions relating to Art. 50 may be found here.
Paragraphs 3, 4 and 5 are relevant to deployers:
Emotion recognition systems and biometric categorisation systems
In accordance with Art. 50(3), deployers of emotion recognition systems or biometric categorisation systems must inform the natural persons exposed thereto of the operation of the system and must process the personal data in accordance with the applicable data protection legislation. However, this obligation does not apply to AI systems permitted by law to detect, prevent or investigate criminal offences, subject to appropriate safeguards for the rights and freedoms of third parties.
AI systems that generate or manipulate content, especially deep fakes
In accordance with Art. 50(4), deployers of AI systems that generate or manipulate image, audio or video content, such as deep fakes as defined in Art. 3(60), must disclose that the content has been artificially generated or manipulated unless the exemption conditions are met. Equally, deployers of AI systems that generate or manipulate text which is published with the purpose of informing the public on matters of public interest must disclose that the text has been artificially generated or manipulated unless the use is authorised by law or where the text has undergone a process of human review or editorial control.
Accessibility requirements
The obligation found in Art. 50(5) is not a standalone requirement but accompanies all the information obligations in paragraphs 1 to 4, whether these originally apply to providers or deployers. Paragraph 5 sets out that the information referred to in paragraphs 1 to 4 must be provided in good time (at the latest at the time of the first interaction or exposure) and in a clear, distinguishable and accessible manner. For the provider obligations in paragraphs 1 and 2, this means that it is not enough for there to be marking, the marking itself must also meet these qualitative requirements.
High-risk AI systems
The AI Act places an obligation on deployers of high-risk AI systems to use them in a particularly diligent, monitored and documented manner to minimise risks to fundamental rights, security and transparency.
Transparency towards affected persons
Art. 26(11) states that deployers of high-risk AI systems that make decisions or assist in making decisions related to natural persons must inform the natural persons that they are subject to the use of the system.
For example, a company using an AI system to evaluate applications must inform applicants of the use. Equally, a school using an AI system to supervise examinations must inform the pupils of this.
For high-risk AI systems used for law enforcement purposes, Art. 26(11) imposes, in addition to the general information obligation on deployers, the detailed information and transparency provisions of Art. 13 of Directive (EU) 2016/680, pursuant to which data subjects must be fully informed of the controller, purposes of the processing, legal basis, period of storage, recipients and their rights (access, rectification, erasure, complaint). In exceptional cases, for example to maintain the secrecy of investigations or protect public or national security, this information may be restricted or omitted subject to proportionate, democratic legislative measures.
This provision is without prejudice to the transparency obligations of Art. 50.
Use of the high-risk AI system in accordance with the instructions for use
In accordance with Art. 26(1), deployers of high-risk AI systems must use the systems exactly in accordance with the accompanying instructions for use and take appropriate technical and organisational measures to ensure the application meets the intended purposes and the specified safety and quality standards. Art. 26(3) gives them the freedom to organise their internal resources and activities as allowed under EU or national law provided they implement the human oversight measures indicated by the provider. In addition, under Art. 26(4), they must ensure that input data is relevant and sufficiently representative in view of the intended purpose of the high-risk AI system so as to avoid incorrect decisions.
Human oversight
In accordance with Art. 26(2), deployers of high-risk AI systems must assign human oversight to suitable natural persons who have the necessary competence, training and authority, as well as the necessary support. This obligation is in harmony with the obligation of providers to design and develop their systems in such a way that effective human oversight is possible (Art. 14). These obligations on deployers are without prejudice to other deployer obligations and to the deployer’s commercial freedom to organise its own resources and activities (Art. 26(3)).
Monitoring the AI system
In accordance with Art. 26(5), deployers of high-risk AI systems must continually monitor the operation of the system on the basis of the instructions for use and, where deployers find indications of risk within the meaning of Art. 79(1), they must, without undue delay, inform the provider, the distributor and the relevant market surveillance authority. In such cases, the use of the system must be immediately suspended. For financial institutions, these monitoring obligations are deemed to be fulfilled by complying with the requirements of EU financial service law.
Serious incident reporting
Where deployers have identified a serious incident, Art. 26(5) obliges them to inform first the provider and then the relevant market surveillance authorities. The reporting must take place without delay but the urgency depends on the severity of the incident. If the provider cannot be reached, Art. 73 applies mutatis mutandis.
Keeping generated logs
In accordance with Art. 26(6), deployers of high-risk AI systems must keep the logs automatically generated by the system for a period of at least six months, to the extent such logs are under their control. The obligation to keep the logs applies unless provided otherwise in applicable EU or national law, in particular in EU law on the protection of personal data or regarding the financial services sector.
For financial institutions, the requirements of the relevant EU financial service law additionally apply.
Data protection impact assessment, where applicable
In accordance with Art. 26(9), deployers of high-risk AI systems can use the information made available by providers under Art. 13 to comply with their obligation to carry out a data protection impact assessment.
This assessment must be carried out under Art. 35 of Regulation (EU) 2016/679 (GDPR) or Art. 27 of Directive (EU) 2016/680, if the relevant AI system is affected. The instructions for use of high-risk AI systems contain relevant information that may be of use for assessing data protection risks.
Cooperation with competent national authorities
In accordance with Art. 26(12), deployers of high-risk AI systems must cooperate with the relevant competent authorities in their implementation of the AI Act.
Right to explanation of individual decision-making
Deployers of high-risk AI systems as referred to in Annex III (with the exception of systems in critical infrastructure listed in point 2 of the Annex), must also, in accordance with Art. 86(1), provide affected persons with clear and meaningful explanations of the role of the AI system in the decision-making procedure and the main elements of the decision taken, provided the decision produces legal effects or significantly affects that person.
Information obligations about use at the workplace
In accordance with Art. 26(7), deployers of high-risk AI systems who are employers must inform workers’ representatives and the affected workers that they will be subject to the use of the high-risk AI system before it is put into service or used at the workplace. This obligation applies only to specific high-risk AI systems and in accordance with the rules and procedures laid down in EU and national law and practice on information of workers and their representatives.
Registration obligation for public authorities and EU institutions, bodies, offices or agencies
In accordance with Art. 26(10), certain deployers of high-risk AI systems which are public authorities, EU institutions, bodies, offices or agencies must register the systems in the EU database on the basis of Art. 49(3) before putting them into service or using them. They may not use a high-risk AI system that is not registered in the EU database and must inform the provider or distributor.
Authorisation by a judicial or administrative authority for post-RBI
In accordance with Art. 26(10), certain deployers of high-risk AI systems for post-remote biometric identification (RBI) in the framework of an investigation for the targeted search of a person suspected or convicted of having committed a criminal offence must request authorisation, ex ante, or without undue delay and no later than 48 hours, by a judicial authority or an administrative authority. This authorisation is necessary except when the system is used for the initial identification of a potential suspect based on objective and verifiable facts directly linked to an offence. Each use must be limited to what is strictly necessary for the investigation of a specific criminal offence.
If the authorisation is rejected, the use of the system must be stopped with immediate effect and the personal data linked to its use must be deleted. In no case may the system be used in an untargeted way, without any link to a criminal offence, a criminal proceeding, a genuine and present or foreseeable threat of a criminal offence, or the search for a specific missing person. It must be ensured that no decision that produces an adverse legal effect on a person may be taken by the law enforcement authorities based solely on the output of such systems.
Each use of the system must be documented in the relevant police file and made available to the competent authorities upon request, excluding the disclosure of sensitive operational data. Deployers must submit annual reports to the competent authorities on their use of the systems. The reports may be aggregated to cover more than one deployment. These provisions apply without prejudice to the GDPR and Directive (EU) 2016/680 for the processing of biometric data.
Fundamental rights impact assessment
In accordance with Art. 27, certain deployers of high-risk AI systems are obliged to carry out a fundamental rights impact assessment prior to deploying the system. This obligation is directed at bodies governed by public law, private entities providing public services, and deployers of high-risk AI systems referred to in points 5(b) and (c) of Annex III. The assessment consists of a description of the processes, the period of time and the frequency of use of the system, the groups of persons affected, specific risks of harm, human oversight measures and measures to be taken in the case of harm.
The assessment is necessary for the first use of the system. Deployers may, in similar cases, rely on previously conducted assessments or existing impact assessments carried out by the provider. If relevant information changes, it must be updated. The market surveillance authority must be notified of the results of the assessment unless there is an exemption under Art. 46(1) of the AI Act.
If a data protection impact assessment was already conducted pursuant to the GDPR or Directive (EU) 2016/680, the fundamental rights impact assessment may complement that data protection impact assessment. The AI Office provides a template for a questionnaire to facilitate deployers in complying with their obligations. These provisions do not apply to high-risk AI systems to be used in the area mentioned in point 2 of Annex III.
