C 465/78
EN
Official Journal of the European Union
6.12.2022
Tuesday 3 May 2022
51.
Stresses that AI technologies, especially those that have not been designed and developed with the explicit control procedures in place and are used improperly and without oversight in military command centres or in missile launch facilities, entail particularly significant risks and could escalate an automated reciprocal conflict;
52.
Notes that the use of AI systems in defence-related developments is considered a game changer in military operations through the analysis of data, the ability to reflect greater situational complexity, the potential to improve target accuracy, optimise logistics and engage in armed conflicts with a reduced risk of physical harm to civilian populations and one’s own military personnel, as well as using data for the development of modes of action such as wargaming; cautions, however, that this could lead to a lower threshold for the use of force and therefore more conflicts; affirms that machines cannot make human-like decisions involving the legal principles of distinction, proportionality and precaution; affirms that humans should be kept in control of the decision to deploy and use weapons and remain accountable for the use of lethal force and for decisions over life and death; is of the opinion that AI-based weapons systems should be subject to global standards and an international ethical code of conduct to underpin the deployment of AI technologies in military operations, with full respect for international humanitarian law and human rights law and in compliance with Union law and values;
53.
Is concerned about the military research and technological developments being pursued in some countries with regard to lethal autonomous weapons systems without meaningful human control; observes that lethal autonomous weapons systems are already used in military conflicts; recalls that Parliament has repeatedly called for an international ban on the development, production and use of lethal autonomous weapons systems and for effective negotiations to begin on their prohibition; stresses that AI-enabled systems can under no circumstances be allowed to replace human decision-making involving the legal principles of distinction, proportionality and precaution;
54.
Notes, in particular, that AI technology may entail potential risks as a means of pursuing various forms of hybrid warfare and foreign interference; specifies that it could for instance be mobilised to trigger disinformation, by using bots or fake social media accounts, to weaponise interdependence, by gathering valuable information or denying network access to adversaries, to create disturbances in the economic and financial systems of other countries, to pollute the political debate and favour extremist groups, or to manipulate elections to destabilise democracies;
55.
Highlights that AI technologies could also include AI-powered malware, identity theft, data poisoning or other forms of adversarial machine learning that cause other AI systems to misinterpret input; points, in particular, to the rise in deepfakes, which are not necessarily cyberattacks but lead to doubts over the authenticity of all digital content, including videos, and therefore require particular attention in terms of transparency requirements; warns that deepfakes could contribute to a broad climate of public mistrust in AI, as well as a deeper socio-political polarisation within our societies;
56.
Elaborates that the use of AI systems in a significant amount of key critical infrastructure such as energy and transport grids, the space sector, the food chain, banking and financial infrastructure, and hospital facilities has created new vulnerabilities that require robust cybersecurity measures to prevent threats; points out, in this regard, the importance of cooperation and information sharing and action both at EU level as well as among Member States; underlines the importance of fostering the resilience of critical entities to hybrid threats;
57.
Warns that the capabilities of AI may also pose security risks, as they may lead humans to place such confidence in AI that they trust it more than their own judgement; notes that using a human-in-the-loop approach as a corrective mechanism is not feasible in all cases; notes that experiments have shown that this can elevate the level of autonomy of AI beyond the supporting role for which it was originally designed and means that humans miss opportunities to gain experience and refine their skills and knowledge of AI systems; stresses, therefore, that safety by design and meaningful human oversight based on appropriate training as well as appropriate security and privacy safeguards are required in high-risk AI systems in order to overcome such automation bias;