# EESC-2019-01830-00-01-AC-TRA-EN.docx

EN
European Economic and Social Committee INT/887 Human-centric artificial intelligence OPINION
European Economic and Social Committee
Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions – Building trust in human-centric artificial intelligence 
[COM(2019) 168 final] Rapporteur: Franca Salis-Madinier Referral
European Commission, 03/06/2019 Legal basis
Article 304 of the Treaty on the Functioning of the European Union Section responsible
Single Market, Production and Consumption Adopted in section
18/10/2019 Adopted at plenary session
30/10/2019 Plenary session No
547 Outcome of vote
(for/against/abstentions)
198/1/4 1. Conclusions and recommendations 1.1 Artificial intelligence (AI) is not an end in itself, but a tool that can deliver far-reaching positive change and involve risk, which is why its use must be regulated. 1.2 The Commission should take measures with regard to forecasting, preventing and prohibiting the malicious use of AI and machine learning and better regulate the placing of products with malicious intent on the market.
1.3 The EU should, in particular, promote the development of AI systems that focus on specific applications to speed up the ecological and climate transition. 1.4 It is important to identify which challenges can be met by means of codes of ethics, self-regulation and voluntary commitments and which need to be tackled by regulation and legislation supported by oversight and, in the event of non-compliance, penalties. AI systems must always comply with existing legislation. 1.5 AI requires an approach which covers technical as well as societal and ethical aspects. The EESC is pleased that the EU intends to build a human-centric AI approach which is in line with its fundamental values: respect for human dignity, freedom, democracy, equality and non-discrimination, the rule of law and respect for human rights. 1.6 The EESC reiterates 1 the need to consult and inform workers and their representatives when AI systems are introduced that are likely to alter the way work is organised, supervised and overseen, as well as worker evaluation and recruitment systems. The Commission must promote social dialogue with a view to involving workers in the uses of AI systems. 1.7 The EESC stresses 2 that trustworthy AI presupposes that humans have control of machines and that citizens are informed about its uses. AI systems must be explainable or, where this is not possible, citizens and consumers must be informed about their limitations and risks. 1.8 The EU needs to address the emerging risks 3 in the area of health and safety in the workplace. Standards must be established to avoid autonomous systems causing harm or damage to people. Workers must be trained to work with machines and to stop them in an emergency. 1.9 The EESC calls for the development of a robust certification system based on test procedures that enable companies to state that their AI systems are reliable and safe. The transparency, traceability and explainability of algorithmic decision-making processes are a technical challenge which needs to be supported by EU instruments such as Horizon Europe. 1.10 Privacy and data protection will determine how far citizens and consumers trust AI. Data ownership and the control and use of data by companies and organisations have yet to be resolved (particularly in relation to the Internet of Things). The EESC urges the Commission to review the General Data Protection Regulation (GDPR) and related legislation on a frequent basis in the light of developments in technology. 1.11 The EESC believes that consideration must be given to the contribution that AI systems can make to reducing greenhouse gas emissions, particularly in industry, transport, energy, construction and agriculture. It calls for the climate and digital transitions to be interlinked. 1.12 The EESC believes that oversight of AI systems may not be sufficient to define who is responsible and build trust. The EESC recommends that, as a priority, clear rules be drawn up assigning responsibility to natural persons or legal entities in the event of non-compliance. The EESC also calls on the Commission, as a priority, to examine the fundamental question of the insurability of AI systems.
1.13 The EESC proposes developing, for companies which comply with the rules, a European trusted-AI Business Certificate based partly on the assessment list put forward by the high-level experts' group on AI (high-level group). 1.14 By promoting work in this area in the G7 and G20 and in bilateral dialogues, the EU must endeavour to ensure that AI regulation goes beyond the EU's borders. We need an international agreement on trustworthy AI, which will develop international standards and carry out frequent checks on the relevancy of those standards. 2. Summary of the Commission proposal 2.1 This communication builds on the work of the high-level group which the Commission appointed in June 2018. In this communication, the Commission identifies seven key requirements for achieving trustworthy AI, which are listed in point 4, 2.2 The Commission has launched a pilot phase involving stakeholders on a broad scale. This exercise focuses in particular on the assessment list drawn up by the high-level group for each of the key requirements. At the beginning of 2020, this group will review and update the assessment list and if appropriate the Commission will propose further measures. 2.3 The Commission wants to take its AI approach international and will continue to play an active role, including in the G7 and G20. 3. General comments 3.1 Human-centric AI needs an approach covering technical, societal and ethical issues. The EESC is pleased that the European institutions intend to build an AI approach which is in line with the values underpinning the EU: respect for human dignity, freedom, democracy, equality and non-discrimination, the rule of law and respect for human rights. As the Commission points out 4 , AI is not an end in itself, but a tool that can deliver far-reaching positive change. Like any tool, it creates both opportunities and risks, which is why the EU has to regulate its use and clearly establish just who is responsible. 3.2 Trust in human-centric AI will be forged by affirming values and principles and providing a well-established regulatory framework and ethical guidelines setting out key requirements. 3.3 It is important to work with all stakeholders to identify which of the many challenges posed by AI need to be tackled by regulation and legislation supported by regulatory oversight mechanisms and, in the event of non-compliance, penalties, and which can be tackled by means of codes of ethics, self-regulation and voluntary commitments. The EESC is pleased that the Commission has taken on board some of the principles originally raised by the EESC, but considers it unfortunate that it has not yet proposed any specific measures to address legitimate concerns (as regards consumer rights, system security and liability).
3.4 AI systems must comply with the existing regulatory framework, particularly as regards protection of personal data, product liability, consumer protection, non-discrimination, professional qualifications and information and consultation of workers in the workplace. It is important to make sure that this legislation is adapted to the new challenges of digitalisation and AI. 3.5 As the Commission notes, "processes to clarify and assess potential risks associated with the use of AI systems, across various application areas, should be put in place" 5 . The EESC attaches the utmost importance to the future arrangements for this assessment and to the establishment of indicators that could be used to perform it. The assessment list proposed by the high-level group is a starting point for implementing these processes. 3.6 This also concerns the question of fair distribution of the expected added value of AI systems. The EESC believes that the beneficial transformation which AI has the potential to bring in terms of economic development, sustainability of (particularly energy) production and consumption processes and better use of resources must benefit all countries and all citizens. 4. Specific comments 4.1 Human agency and oversight 4.1.1 The Commission wants to be sure that the use of AI systems will never undermine human autonomy or give rise to adverse effects. The EESC supports this approach of human oversight of machines, as it has already stated in previous opinions. 4.1.2 Under this approach, citizens also have to be properly informed about the uses of these systems. The systems have to be explainable or, where this is not possible (in the case of deep learning, for instance), the user has to be informed about the system’s limitations and risks. In any event, people have to retain the freedom to decide differently from the AI system. 4.1.3 In businesses and public administrations, workers and their representatives must be properly informed and consulted when AI systems are introduced that are likely to alter the way work is organised and to affect them (in terms of supervision, oversight, evaluation and recruitment). The Commission must promote social dialogue with a view to involving workers in the uses of AI systems. 4.1.4 With regard to human resources, particular attention must be paid to the risks of misuse of AI systems, such as unlimited surveillance, collection of personal and health data, and sharing of these data with third parties, and to the emerging risks in terms of health and safety in the workplace 6 . Clear standards must be established to ensure that human-machine collaboration does not cause damage to humans. The International Organization for Standardization (ISO) standard on collaborative robots 7 , which is aimed at manufacturers, integrators and users, provides guidelines for the design and organisation of a collaborative workspace and the reduction of the risks to which people can be exposed. Workers must be trained to use AI and robotics, to work with them and, in particular, to stop them in an emergency ("emergency brake principle"). 4.2 Technical robustness and safety
4.2.1 The EESC calls for the introduction of European security standards and the development of a robust certification procedure based on test procedures that would enable companies to state that their AI systems are reliable. The EESC would also like to stress the importance of the insurability of AI systems. 4.2.2 The Commission pays scant attention to the issue of forecasting, preventing and prohibiting the malicious use of AI and machine learning, against which many researchers have issued warnings 8 . Their recommendations should be taken into account, particularly those concerning the dual use of these technologies which can potentially touch on digital security (increase in cyber attacks, exploitation of human and AI vulnerabilities, data poisoning), physical security (hacking of autonomous systems, including autonomous vehicles, drones and automatic weapons) and political security (mass collection of personal data, targeted propaganda, video manipulation, etc.). Researchers, engineers and public authorities must work closely to prevent these risks; for their part, experts and other stakeholders such as users and consumers must be involved in discussions on these issues. 4.3 Privacy and data governance 4.3.1 The Commission calls for access to data to be "adequately governed and controlled" 9 . The EESC believes that we need to go further than general statements. The degree of trust that people have in AI systems will also determine their development. The issues of data ownership, and the control and use of data by companies and organisations have yet to be resolved. The amount of data transmitted for example by cars to car manufacturers and the type of data transmitted are startling 10 . Despite the concept of privacy by design, with which connected objects have to comply under the GDPR, we can see that consumers have very little or no information on this subject and no means of controlling these data. The EESC therefore urges the Commission to review the GDPR and related legislation in the light of developments in technology 11 . 4.4 Transparency 4.4.1 The EESC believes that the explainability of algorithmic decision-making processes is key to understanding not the mechanisms but the underlying logic of the decision-making processes and how they are influenced by AI systems. Developing standard test procedures for machine learning systems continues to be a technical challenge which needs to be supported by EU instruments such as Horizon Europe. 4.4.2 The EESC agrees with the Commission that AI systems must be identifiable as such, "ensuring that users know they are interacting with an AI system" 12 , including in the context of relations between patients and health professionals and professional services linked to citizens' health and well-being. The EESC also stresses that users and consumers must also be able to be informed about the services performed by human beings. Many AI systems actually involve large amounts of human work, which is often hidden from end-users 13 . There is the underlying issue here of the lack of transparency towards users and consumers of services, and a form of usage of concealed and unrecognised work. 4.4.3 In addition, the EESC believes that consumers must always be informed when AI systems are integrated into the products they buy, and must always be able to access and control their data. 4.5 Diversity, non-discrimination and fairness 4.5.1 Risks in the form of discrimination are present in some AI applications which profile citizens, users and consumers (for example for recruitment, letting property and certain personal services). The EU has adopted a body of legislation on equal treatment and non-discrimination 14 and AI systems must comply with it. However, this legislation must also be adapted and, if appropriate, bolstered (including in terms of enforcement) in order to cope with new practices. There is a real danger that algorithmic profiling could become a new and powerful tool of discrimination. The EU must prevent this danger.
4.5.2 The Anti-Racism Directive and the Directive on equal treatment for men and women beyond the workplace provide for the creation of special bodies responsible for promoting gender equality. The EESC calls for these bodies to play an active role in monitoring and overseeing AI systems with regard to the risks of direct or indirect discrimination. 4.6 Societal and environmental well-being 4.6.1 The Commission does not propose any specific ways to link up the climate transition and the digital transformation, particularly as regards the use of AI systems. Consideration must be given to the contribution that AI systems can make to reducing greenhouse gas emissions, particularly in industry, transport, energy, construction and agriculture. 4.6.2 The Commission points out that AI systems can be used to enhance social skills but they could also lead to a deterioration in this area. The EESC feels that the EU must be more proactive in gauging certain societal challenges. For example, studies have shown that some applications incorporating AI systems are designed to keep users of online services (social networks, games, videos, etc.) connected for as long as possible. The aim is to be able to collect as much data as possible on their behaviour; the strategies used range from endless transmitting of algorithmic recommendations to reminders and notifications, games, etc. The effects on children of the excesses of connection and solicitation have been studied 15 and the findings have shown an increase in anxiety, aggression, sleeplessness and an impact on education, social interaction, health and well-being. In order to build trustworthy AI, the EU must take these effects into account and prevent them. 4.6.3 Lastly, one of the elements of societal well-being is related to a sense of security at work. The effects of digitalisation can undermine security and cause stress 16 , and so strategies are needed to anticipate change before any restructuring occurs and provide ongoing training for all workers. This requires a high standard of social dialogue in companies between employers and workers' representatives, involving in particular inclusive deployment of new technologies, especially AI and robotics. To consolidate trust between management and workers, IA systems in the area of management, evaluation and oversight of workers must be explainable, their parameters must be known and the way they work must be transparent. 4.7 Accountability 4.7.1 The decisions taken by machine learning systems cannot be explained in simple terms; moreover, they are updated regularly. The EESC believes that oversight of AI systems may not be sufficient to define who is responsible and build trust. It therefore recommends that rules be drawn up assigning responsibility to natural persons or legal entities in the event of non-compliance. The EESC recommends relying more on trustworthy companies or professionals than on algorithms, and proposes developing, for companies which comply with all the rules, a European trusted-AI Business Certificate based partly on the assessment list suggested by the high-level group. 4.7.2 The Product Liability Directive 17 establishes the principle of strict liability for European producers: where a defective product causes harm to a consumer, the producer can be held liable even when there is no fault or negligence on their part. The increasingly widespread design, deployment and use of AI systems mean that the EU needs to adopt adapted liability rules for situations where products with digital content and consumer services can be dangerous and harmful. Consumers must be able to take legal action in the event of harm caused by an AI system. 5. The need for regulation beyond Europe 5.1 In a global context, AI regulation must go beyond Europe’s borders. Europe should promote a broad consensus on AI in the G7 and G20 and keep up bilateral dialogue so that a majority of countries can participate in AI standardisation processes and verify their relevance on a regular basis.
Brussels, 30 October 2019 Luca Jahier The president of the European Economic and Social Committee * *
- N.B:
The appendix to this document (the supplementary opinion of the Consultative Commission on Industrial Change – CCMI/170 – "Artificial intelligence in the automotive sector" – EESC-2019-02501-00-00-AS-TRA) can be found on the following pages. European Economic and Social Committee
CCMI/170 Artificial intelligence in the automotive sector SUPPLEMENTARY OPINION
Consultative Commission on Industrial Change (CCMI)
to the opinion of the Section for the Single Market, Production and Consumption on
Human-centric artificial intelligence
(INT/887) [COM(2019) 168 final] Rapporteur: Ulrich Samm Co-rapporteur: Monika Sitarová Hrušecká Contact
Plezer@eesc.europa.eu Administrator
Adam Plezer Document date
09/10/2019 Bureau decision
14/05/2019 Legal basis
Rule 37(2) of the Rules of Procedure Supplementary opinion Body responsible
Consultative Commission on Industrial Change (CCMI) Adopted in CCMI
02/10/2019 1. Conclusions and recommendations 1.1 The CCMI is convinced about the benefits of connected and automated mobility for our society with regard to safety and better services. While the EU automotive industry is well positioned with its expertise in developing vehicle technologies, there are serious concerns about future competitiveness and jobs with regard to the possibly fast changing vehicle market with strong competitors worldwide. 1.2 AI is playing a crucial role in these developments and the EU has still to find its place in this field. The aim is to define regulations and rules based on European values such as respect for human dignity, freedom, democracy, equality, the rule of law and human rights, including the rights of persons belonging to minorities and the EU Charter of Fundamental Rights. The CCMI welcomes Europe's ethical approach with regard to AI, which should strengthen citizens' trust in digital development and create a competitive advantage for European businesses.
1.3 The CCMI points out that even semi-automatic vehicles are contributing to a reduction in fatalities, but also warns that the introduction of such assistance systems is being hampered because, firstly, the cost of a car increases and, secondly, driving becomes more complex and some drivers then simply turn off these assistance features. 1.4 The CCMI suggests that the automotive industry, together with the municipalities, provide the possibilities for training for private and professional drivers in order to learn about the features, limits and risks of semi-automatic vehicles. 1.5 The CCMI believes that autonomous vehicles will only succeed if parts of the road system are adapted and certain network infrastructures (such as 5G) are upgraded, which affects all transport sectors (road, rail, air, water) crucial for the development of an integrated multimodal transport system. 1.6 When human error is eliminated, automatic transport systems must be almost 100% secure. The CCMI believes that this is a necessary condition for the success of autonomous vehicles and warns against negligent testing of risky technologies. 1.7 The CCMI calls on the European Commission to develop and implement as soon as possible regulations for the access to vehicle data. The huge amount of data offers plenty of untapped potential. However, it also brings significant risks and challenges regarding safety, security and privacy. 1.8 There is little in-depth analysis of the full implications of moving to autonomous mobility. Big data companies and the automobile industry have a mixed record on transparency, compliance and co-operation. Therefore, policy commitments to any form of extensive implementation of autonomous mobility should be approached with great caution. 1.9 The CCMI welcomes the Commission's approach in giving priority to regulating the protection of vehicles against cyber-attacks, ensuring secure and trustworthy communication between vehicles and infrastructure and providing a sound data protection level in compliance with the General Data Protection Regulation. 1.10 The CCMI warns that decisions made through complex machine learning systems cannot be explained with simple means. The CCMI, therefore, recommends that EU standard testing procedures be developed to assess the functionality and limitations of such systems (such as prejudice, bias, discrimination, resilience, robustness, etc.). 1.11 The CCMI believes that the "explainability" of the decision-making process is not enough to allocate responsibility. The concept of responsibility is linked to intentionality and personality and cannot be transferred to machines. The CCMI therefore recommends that clear rules be outlined which unambiguously attribute the responsibility in the event of failures to legal persons: humans or companies in charge. 1.12 The CCMI recommends that greater reliance be put on trustworthy enterprises or professionals, rather than on trustworthy algorithms. It therefore proposes that a European Trusted-AI Business Certificate be developed based on the final European Ethics Guidelines.
1.13 Automation and robots have a significant impact on the future of work and they will have the potential to stabilise the economy in an ageing society. In this respect the CCMI recommends that the role and the provisions of the European social pillar have to be taken into account. It is essential to clearly involve the social partners in this. 1.14 The CCMI is aware of the fact that the evolution towards autonomous driving is happening at the same time as several other investment programmes (towards electrification, towards Industry 4.0 manufacturing), and at a moment of an economic downturn for the sector. These simultaneous challenges deserve to be anticipated and accompanied by strong social and re-conversion/up-skilling policies. 1.15 The CCMI want to highlight the role of educational training programmes and lifelong learning in protecting European workers operating in an environment that is being profoundly changed by the gradual emergence of AI. 1.16 Future skills should match societal needs and the demands of the labour market. This can only be achieved through close cooperation between social partners and the public. 2. Introduction 2.1 This opinion focusses on the automotive sector only and supplements the general opinion INT/887 "Building Trust in Human-Centric Artificial Intelligence (AI)" referring to Communication COM(2019) 168 final. Other strategic communications on this topic were also considered, including "AI for Europe 18 ", "Coordinated Plan on AI 19 ", "Connected and Automated Mobility 20 " and "Cooperative Intelligent Transport Systems 21 ". 2.2 Today's widespread availability of high computing power and big data generation enable the use of powerful AI-based tools and machine-learning tools in almost all sectors of our economy. In modern automobiles we already have significant AI technologies on board, and the ongoing developments promise much more. 2.3 The Commission requested a "High-level expert group on AI" to develop ethical guidelines that build on the existing regulatory framework and that should be applied by developers, suppliers and users of AI in the internal market. The Group has recently presented a draft of its Ethics Guidelines for Trustworthy AI 22 , which includes an assessment list that operationalises the key requirements. 2.4 Under the umbrella of the European AI Alliance the Commission has launched a targeted piloting phase in summer 2019 involving stakeholders in order to test the practical implementation of the ethical guidance for AI development and use. 2.5 This opinion focusses on the automotive sector and analyses the sustainable use and ethical guidelines on AI. Distinctions will be made between semi-automatic and fully autonomous driving. The issues of responsibility and trust are addressed as well as safety and data access and the requirements and needs for a fully integrated intermodal transport system. Because of its importance, the opinion also addresses the impact of AI on manufacturing processes and the labour market, in the awareness that these phenomena are not specific to the automotive industry.
3. General comments 3.1 The CCMI welcomes Europe's ethical approach with regard to AI, which should strengthen citizens' trust in digital development and create a competitive advantage for European businesses. The approach must be based on European values such as respect for human dignity, freedom, democracy, equality, the rule of law, social rights and human rights, including the rights of persons belonging to minorities and the EU Charter of Fundamental Rights. In addition, the impact of connected and automated mobility must be considered in the light of the EU's commitment to a carbon-neutral, circular economy. 3.2 The high-level expert group on AI believes that trustworthy AI can only be achieved if it complies with European law, follows ethical principles based on European values and fulfils the highest safety requirements. The group identified seven key requirements that AI applications should respect to be considered trustworthy: 1) Human agency and oversight, 2) Technical robustness and safety, 3) Privacy and data governance, 4) Transparency, 5) Diversity, non-discrimination and fairness, 6) Societal and environmental well-being, and 7) Accountability. 3.3 The CCMI acknowledges these key requirements as a valuable basis on which to develop new sector specific recommendations. To this end, the CCMI is contributing with the specific recommendations given below. The CCMI also recognises that at the Gothenburg council in 2018 23 the Member States, in close cooperation with the Commission, already moved ahead with the establishment of a task force on ethical aspects of automated and connected driving 24 . It encourages more Member States to join this effort. 3.4 The EU automotive industry is well positioned with its expertise in developing vehicle technologies. The CCMI is convinced about the benefits of connected and automated mobility for our society, which will also open the way towards a gradual move from personally-owned vehicles to integrated, shared mobility solutions. 3.5 The CCMI is concerned at the lack of data and analysis on the cost and impact of obtaining, integrating and handling the massive data flows involved with autonomous vehicles. In addition, the record on transparency, compliance and cooperation with big data companies and the automobile industry has often been against the public interest. In this context, policy commitments to the immense resources needed to support the infrastructure and technology platforms required for any form of extensive implementation of autonomous mobility should be approached with great caution. 3.6 The CCMI welcomes, in particular, the initiative to allocate more funding to AI-related innovation, infrastructure, education and training via EU financing instruments, as set out, for example, by the coordinated plan on AI to increase joint investments, foster cross-border cooperation and promote significant investments in R&D within AI Horizon 2020, Horizon Europe and the Digital Europe Programme. Horizon Europe's support for innovation centres is particularly helpful for SMEs using cutting-edge technologies. Funding education through research projects is an essential way to promote talent and provide high-level skills. 4. Specific comments 4.1 Semi-automatic driving (with driver) 4.1.1 The CCMI emphasises that semi-automatic vehicles (level 1-4) help to reduce fatalities and therefore supports the Commission's approach of enhancing the number of new safety features for vehicles, including those based on in-vehicle AI, as part of the revision of the General Safety Regulation for motor vehicles.
4.1.2 The CCMI notes two problematic areas, apart from uncertainty and lack of trust related to failures of complex technology, which may be a hurdle for public acceptance: a) additional technical features can increase the cost of a car significantly, and b) with a growing number of assistance systems, driving a car can become significantly different. Obviously, there is a need for additional training for new as well as experienced drivers. The CCMI proposes that the automotive industry, together with municipalities, offer training courses and training areas for private and professional drivers. 4.1.3 Consumers must also be given clear and unambiguous information about the features of a modern vehicle (including limits and risks) at the time of purchase, rental or car-sharing. The CCMI emphasises that drivers in difficult situations (extreme weather, unusual traffic situations, off-road) can easily switch off the assistance systems at any time, thus retaining control of the vehicle. 4.2 Autonomous driving (driverless) 4.2.1 The technology for autonomous road vehicles is quite mature, but its integration is hampered as long as these vehicles are driving on the same roads as conventional cars and other road users. As a first step one could restrict autonomous driving to motorways only, where traffic conditions are simpler and easier to control. However, as far as can be ascertained, no full-cost estimates exist of converting existing motorways to be enabled for autonomous driving in a mixed-vehicle environment. 4.2.2 When human error is eliminated, automatic transport systems must be almost 100% secure and meet the same safety standards as other passenger transportation systems such as trains or aviation. The CCMI believes that this is a necessary condition for the success of autonomous vehicles and warns against negligent testing of risky technologies. It recommends, therefore, that all pilot projects and test procedures with automatic driving be performed under the highest safety standards possible, even when this may slow down development compared to competitors outside the EU. On the positive side, it will enhance public acceptance and deliver better products in the long term. 4.3 Towards an integrated traffic system 4.3.1 The use of AI technologies is important for all transport sectors (road, rail, air, water) and will be crucial to the development of an integrated multimodal transport system. 4.3.2 The CCMI believes that automated vehicles will only become a success with a significant re-designing of parts of the road system. An integral approach and cooperation is necessary between different sectors such as transport, energy and telecommunications, as well as regions, cities and municipalities across borders. 4.3.3 The CCMI encourages the Commission to support the 5G cross-border corridors for the large scale testing of connected and automated mobility. However, this cannot be regarded as a priority when many areas, rural areas in particular, are still excluded from high-speed internet connection. 4.3.4 The CCMI welcomes the Commission's intention to further develop the Galileo services and related vehicle navigation technologies for semi-autonomous mobility.
4.4 Data protection 4.4.1 With increased connectivity, vehicle data can be accessed from every corner of the world. This possibility opens the door to plenty of untapped potential. However, it also brings significant risks and challenges regarding safety, security and privacy. Given the strong interest of third parties (such as Google) in having comprehensive access to all vehicle data, EU regulation seems urgently needed. 4.4.2 The CCMI welcomes the Commission's approach in giving priority to regulating the protection of vehicles against cyber-attacks, ensuring secure and trustworthy communication between vehicles and infrastructure and providing a sound data protection level in compliance with the General Data Protection Regulation. 4.5 Trust and responsibility 4.5.1 Any AI technology deployed should have ethics by design. Procedures of formal verification or evaluation for any autonomous decision-making and autonomous behaviour in the physical environment should be put in place. 4.5.2 The draft Ethical Guidelines for Trustworthy AI state that the traceability of AI systems should be ensured. This means logging and documenting the decisions made by the systems, as well as the entire process that yielded the decisions (explainability). Risk assessment and management protocols should be put in place for any autonomous decision-making technology. 4.5.3 Decisions made through complex machine learning systems cannot be explained with simple means. For this reason, the CCMI believes that "explainability" requires a broader definition in order to gain confidence. The CCMI recommends that EU standard testing procedures be developed to assess the functionality and limitations of such systems (such as prejudice, bias, discrimination, resilience, robustness, etc.). The introduction of such official standard test procedures could in future form a basis of trust for AI systems. 4.5.4 The CCMI thinks that the explainability of the decision-making process is not sufficient to attribute responsibility. The concept of responsibility is linked to intentionality and personality and cannot be transferred to machines
25 . The CCMI therefore recommends that clear rules be outlined which unambiguously attribute the responsibility in the event of failures to legal persons: humans or companies in charge. A standard liability must be established for cases where the process leading to the faulty decision cannot be traced back (i.e. in the absence of explainability). 4.5.5 The CCMI recommends that greater reliance be put on trustworthy enterprises or professionals, rather than on trustworthy algorithms. It therefore proposes that a European Trusted-AI Business Certificate be developed based on the final European Ethics Guidelines for AI. Such a certificate would enable consumer confidence to be attributed to a company and its products. 4.5.6 The CCMI calls on the European Commission to review the Product Liability Directive and to integrate specific liability rules for the operation of autonomous systems into it.
4.6 Manufacturing, services, jobs and education 4.6.1 Automation and robots will have a significant impact on the future of work. While present-day robots are mainly replacing blue-collar work, white-collar professions will also be affected when artificial intelligence is introduced. Automation and robots have the potential to stabilise the economy in an ageing society. In this respect the role and the provisions of the European social pillar have to be taken in account. 4.6.2 Artificial Intelligence can automate some of a company's interaction with its workers: (1) upon recruitment, and for promotion/training, and (2) to constantly monitor work and automatically detect deviations from the prescribed process. In all these developments, regulation via collective agreements or law should ensure fairness, prevent discrimination and preserve workers' privacy and autonomy. It is essential to clearly involve social partners in this process and ensure that the AI will be working for the Human and not the other way around. 4.6.3 The CCMI believes that artificial intelligence ("AI") and automation processes have enormous potential to improve European society in terms of process efficiency, quality and reliability, but they also pose significant challenges, risks and concerns. 4.6.4 The possible evolution towards autonomous driving is happening at the same time as several other investment programmes (towards electrification, towards Industry 4.0 manufacturing), and at a moment of an economic downturn for the sector. These simultaneous challenges are a heavy burden for the sector, which can lead to companies going bankrupt because they can't cope with all these investments at a time when their sales are plummeting. All these transformations and risks to workers deserve to be anticipated and accompanied by strong social and re-conversion/up-skilling policies. 4.6.5 Humans and robots working side by side will bring with itself a lot of challenges, the responsibility for the different steps in the production process will have to be clarified, in particular by involving social partners and strong social dialogue in this respect. 4.6.6 The European Commission and the Member States will also have to carry out a careful evaluation of the effects of AI on the labour market. The main feature of Artificial Intelligence, and specifically of machine learning, is to learn the hidden patterns from a large amount of data, just as an experienced and skilled worker does. The more controlled the environment, such as that found in manufacturing, the better the performance of the machine-learning system. This entails significant risks of having the skills of experienced industrial workers being captured by machine-learning systems, with a strong risk of de-skilling the workforce. The impact on the skills profiles of professions should be carefully evaluated. 4.6.7 It is important to highlight the role of educational training programmes in protecting European workers operating in an environment that is being profoundly changed by the gradual emergence of AI. These programmes have to be developed in cooperation between all the involved actors in order to secure not only the demands of the market and business, but also of the workforce of this sector. 4.6.8 Lifelong learning, particularly related to digital skills, is a necessity and a right for everyone, requiring more and more flexibility and massively increased resources from individuals, companies and all education and training systems. The sharing of this massive effort and of its benefits must be discussed among all stakeholders: trade unions, employers' organisations, professional organisations, regional and national education and training authorities. 4.6.9 Future skills should match societal needs and the demands of the labour market. This can only be achieved through close cooperation between social partners and public and private education systems. More volatile markets will give rise to challenges, as businesses and workers will have to adapt quickly enough. This, in particular, is a challenge for professional training systems.
Brussels, 2 October 2019 Lucie Studničná The president of the Consultative Commission on Industrial Change _____________
(1)
OJ C 440, 6.12.2018, p. 1.
(2)
OJ C 288, 31.8.2017, p. 1

OJ C 440, 6.12.2018, p. 1
(3)
https://osha.europa.eu/en/emerging-risks.
(4) COM(2019) 168 final.
(5) COM(2019) 168 final, p. 5.
(6) See in particular OSH and the future of work: benefits and risks of artificial intelligence tools in workplaces.
(7) ISO/TS 15066, 2016.
(8) See report on The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation, February 2018.
(9) COM(2019) 168 final, p. 6.
(10) Your car knows when you gain weight, The New York Times (International Edition), 22.5.2019.
(11)
OJ C 190, 5.6.2019, p. 17
.
(12) COM(2019) 168 final, p. 6.
(13) See for instance A white-collar sweatshop: Google Assistant contractors allege wage theft, The Guardian, 29.5.2019 and Bot technology impressive, except when it’s not the bot, The New York Times (International Edition), 24.5.2019.
(14)
OJ L 180, 19.7.2000, p. 22
;
OJ L 303, 2.12.2000, p.16,
;
OJ L 373, 21.12.2004 p. 37
;
OJ L 204, 26.7.2006, p. 23
.
(15) See Kidron, Evans, Afia (2018), Disrupted Childhood – The Cost of Persuasive Design, 5Rights Foundation.
(16) Report by the high-level group on the impact of the digital transformation on EU labour markets, 2019.
(17)
OJ L 210, 7.8.1985, p. 29
.
(18) COM(2018) 237 and
OJ C 440, 6.12.2018, p. 51
.
(19) COM(2018) 795 final and
OJ C 240, 16.7.2019, p. 51
.
(20) COM(2018) 283 final and
OJ C 62, 15.2.2019, p. 274
.
(21) COM(2016) 766 final and
OJ C 288, 31.8.2017, p. 85
.
(22)
https://ec.europa.eu/digital-single-market/en/high-level-expert-group-artificial-intelligence
.
(23)
https://www.regeringen.se/4a990c/contentassets/55d3db45d6804a4c96df4604ad8ef80c/180619_gothenburg-conclusions_webb.pdf
.
(24) Germany (chair), Austria, Luxembourg, United Kingdom, EC, ACEA, CLEPA.
(25) Deutscher Ethikrat, Deutsche Akademie der Technikwissenschaften.