General Purpose AI Systems in the AI Act: Trying to Fit a Square Peg Into a Round Hole

In 2021, the European Union (EU) published its first draft of the AI Act, a comprehensive artificial intelligence (AI) regulation. Emphasizing the traditional European focus on ex ante safety, the AI Act imposes safety requirements that AI operators must meet before bringing their systems to market. These requirements depend on the assessed risk of the AI system. AI systems posing unacceptable risks are banned; high-risk systems are subject to ex-ante safety requirements; systems presenting medium risk must meet transparency obligations; and low-risk systems are allowed with no restrictions, although companies are encouraged to adopt codes of conduct. The primary criterion for determining whether an AI system is high-risk is the context of its deployment. The EU Commission proposed a list of areas of application for the classification of AI systems as high-risk, including critical infrastructure, education, employment, law enforcement and immigration (Annex III to the AI Act).The AI Act relies heavily on the notion of an "intended purpose" of AI systems. According to the AI Act, the "intended purpose" refers to the use for which an AI system is intended by the provider, including the specific context and conditions of use, as specified in the information supplied by the provider in the instructions for use, promotional or sales materials and statements, as well as in the technical documentation (art. 3). The intended purpose of a system influences whether it is considered high-risk (art. 7.2.a) and is a factor in assessing important aspects such as an AI system's consistency (art. 9), the training requirements to be imposed (art. 10), and the duration for which training logs should be kept (art. 20).After the AI Act was published, some stakeholders argued that it presented a loophole. An increasing number of AI systems released did not have a single intended purpose. For example, foundation models can be adapted to various downstream tasks. This category of model includes systems such as PALM, DALL-E 2, Stable Diffusion, and GPT-4. Given that these systems are not specifically created for deployment in high-risk contexts, they would not be subject to the safety requirements. However, some of these requirements, such as data validation, can only be implemented during the training phase. Consequently, it would have been impossible for end-users wanting to use these systems in high-risk settings to comply with the requirements. In fact, users and deployers of foundation models sometimes discover some of their capabilities only after they have been placed on the market, making end uses of such models even more difficult to foresee.Civil society actors began advocating for the AI Act to explicitly include systems without a single predefined purpose, arguing that otherwise, these systems would either evade regulation,or the burden of making them safer would fall onto end-users, disadvantaging smaller European companies. On the other hand, the industry lobbied to exempt these systems from the AI Act requirements altogether. The consequences of the AI Act remaining in its initial form are unknown, as the inability to meet safety requirements might have prevented end-users from deploying systems without an intended purpose in high-risk settings in the first place. Following extensive debate, the concept of "General Purpose AI systems" (GPAIS) was introduced to the AI Act. In a draft circulated by the European Parliament co-rapporteurs, a GPAIS is defined as “an AI system that is trained on broad data at scale, is designed for generality of output, and can be adapted to a wide range of tasks,” which is the definition of foundation models from the seminal paper on the topic.Much of the current legal discussion focuses on the chain of responsibility to ensure that operators who transform GPAIS into high-risk systems can comply with the necessary requirements. The Parliament co-rapporteurs' latest draft also proposes that some articles initially intended for high-risk systems, particularly article 9 requiring a risk management framework and article 15 addressing accuracy, robust ness, and cybersecurity requirements, be extended to GPAIS.The ongoing legal discussion on GPAIS highlights that the AI Act was not drafted in a technology-agnostic manner and is ill-suited to address the latest advancements in AI. Attempting to adapt it to GPAIS a posteriori, despite its structure being entirely built around end uses and intended purposes, is like trying to fit a square peg into a round hole.In the first section of our paper, we will show that the AI Act was influenced by a conception of AI systems as non-autonomous statistical software and of potential harms as stemming mostly from datasets. We will demonstrate that the notion of intended purpose, which draw from product safety, was suited for that paradigm. In the second section, we will discuss AI systems that do not have an intended purpose, such as GPAIS, foundation models, and others. We will clarify what types of harms they can cause and why the AI Act in its initial version is not prepared to address them. In the third section, we will present the provisions proposed by the EU Parliament to regulate these models and propose additional policy recommendations for adapting the AI Act to GPAIS and future AI systems.