# Joint memo from the Inserm Ethics Committee, the LORIER program, and the Inserm Scientific Advisory Board

Henri Atlan, Catherine Bourgain, Herve Chneiweiss, François Eisinger, Catherine Vidal, Frédérique Lesaulnier, Marion Abecassis, Philippe Ravaud, Ghislaine Filliatreau, Ioana Andreescu, et al.

► To cite this version:

Henri Atlan, Catherine Bourgain, Herve Chneiweiss, François Eisinger, Catherine Vidal, et al. Joint memo from the Inserm Ethics Committee, the LORIER program, and the Inserm Scientific Advisory Board. Best practice recommendations using AI in Inserm Research / February 2025, 2025. inserm-05003733

HAL Id: inserm-05003733

https://inserm.hal.science/inserm-05003733v1

Submitted on 24 Mar 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

RÉPUBLIQUE
FRANÇAISE

Liberté

Égalité

Fraternité

Inserm

La science pour la santé

From science to health

Inserm Ethics Committee

<figure><img src="image_3.png" /></figure>

"Neurotechnologies
and
Artificial Intelligence"
Working Group

Best practice recommendations
using AI in Inserm Research

Joint memo from the Inserm Ethics
Committee, the LORIER program, and
the Inserm Scientific Advisory Board

February 2025
# Joint memo from the Inserm Ethics Committee, the LORIER program, and the Inserm Scientific Advisory Board

## Members of the Working Group (July 2024 - November 2024)

**For the Inserm Ethics Committee:** Henri Atlan, Catherine Bourgain, Hervé Chneiweiss, François Eisinger, Catherine Vidal, Frédérique Lesaulnier, and with the legal contribution of Marion Abecassis

**For the LORIER program:** Philippe Ravaud, Ghislaine Filliatreau, and Ioana Andreescu

**For the Inserm Scientific Advisory Board:** Rodolphe Thiebaut, Isabelle Remy-Jouet, Lotfi Senhadji, Valérie Urbach, and Murielle Gaudry

**External experts:**

- Anita Burgun, University Professor and Hospital Practitioner (PUPH) in Medical Information Technology, Imagine Institute, Paris Cité University, Paris Public Hospitals Group (AP-HP) & Inserm, PRAIRIE Chair.

- Jean Charlet, AP-HP Research Director, LIMICS Sorbonne University & Inserm U1142.

- Pierre-Antoine Gourraud, PUPH in Cell Biology, Nantes University, Inserm Joint Research Unit 1064 & Healthcare Data Warehouse Manager at Nantes University Hospital "Data Clinic". Member of the scientific leadership team of Ouest Data Hub.
Best practice recommendations following analysis of the ethical
questions raised by using Artificial Intelligence in Inserm research

"... unless we can worry about a technology, it is not revolutionary enough"

"the engineer of tomorrow will be an apprentice sorcerer not by negligence or incompetence;
he will be one deliberately"

Jean-Pierre Dupuy - The Mark of the Sacred

"[I]t could be that we [...] will forever be unable to understand [...] the things which
nevertheless we are able to do"

Hannah Arendt cited by Jean-Pierre Dupuy in The Mark of the Sacred
# Summary of the main recommendations

## 1- Disclosure and transparency:

1.1 Naive disclosure: Scientists must be careful about not just the uncontrolled dissemination of data (loss of data confidentiality), but also the dissemination of uncontrolled data (hallucinations, unsourced data), regardless of the digital tool used.

1.2 Scientific publication: Researchers must explicitly mention the use of artificial intelligence (AI) systems in their work (tools, algorithms, parameters) and distinguish the contributions obtained using AI systems from those derived from their creative activity. In publications, a "Use of AI systems" section must detail the role of AI systems, like the software used for statistical analyses. A practical guide and references will be prepared and made available on the LORIER portal.

1.3 Research administration and support: Transparency must also apply to all other uses of AI systems at Inserm, particularly in the Human Resources sector (recruitment, careers, etc.).

## 2- Attribution and transparent models:
Scientists must publish the details of the AI system models (training data, versions) they create or use, and ensure their long-term archiving for replication studies. They must also promote models that enable working in compliance with good ethical practices – particularly the citation of data sources, and encourage academic uses of open science.

## 3- Use of synthetic data:
Inserm scientists are encouraged to develop the use of synthetic data that would limit the risks of research participant re-identification (anonymity within the meaning of CNIL/G29¹/EDPB<fnref n="2" />), while ensuring the verisimilitude of the data, their suitability for a set of secondary uses and their diversity, avoiding biases that could compromise the research. This use of qualitative anonymous synthetic data also makes it possible in certain cases to enrich datasets, particularly in areas where real data are scarce or sensitive.

## 4- Verification of AI system findings:
Responsibility for the accuracy of the analyses generated using AI systems lies with their users and particularly the researchers, who must validate their reliability and identify potential biases. Staff are invited to thoroughly test the

<footnote n="1">¹ G29 refers to Article 29 of the European Directive of 24 October 1995 on the protection of data and the free movement of such data, which has set up a working group bringing together the representatives of each independent national data protection authority.</footnote>

<footnote n="2">² The European Data Protection Board (EDPB) is an independent European body that has succeeded the G29. It brings together the national data protection authorities of the European Economic Area countries, as well as the European Data Protection Supervisor (EDPS). https://www.edpb.europa.eu/edpb_en.</footnote>
reproducibility and reliability of AI models by 1) comparing the findings obtained with different datasets and 2) testing the findings obtained with different AI algorithms. The creation of a national cross-cutting unit on the use of digital technologies for science for health at Inserm<fnref n="3" /> could centralize follow-up information (collection of bias detections and solutions found) and help formulate the training provision in order to support this approach. Given the rapid changes occurring in AI systems and practices, continuous reflection is necessary – involving the Ethics Committee, the Scientific Advisory Board, and the LORIER program.

**5- Documentation of AI system data:** Data generated using AI systems must be clearly identified to avoid any confusion with real observations. Researchers must guarantee the traceability of the AI system data used in the studies.

**6- Integrity and equity:** Researchers must seek to anticipate the social impacts of AI systems. They must be trained in the legal rules, particularly regarding data protection and respect for property rights. Particular vigilance is required for under-represented or historically discriminated groups.

**7- Monitoring, alternative solutions and public engagement:** Continuous surveillance of the societal impact of AI system use is essential. The use of AI systems in research has an environmental impact. Such systems must therefore be used sparingly and in contexts of certain relevance. AI systems using a reduced dataset and fewer parameters, such as Small Language Models (SLMs), represent frugal and sober AI. This development of responsible AI systems can also contribute to improving digital sovereignty. The national cross-cutting unit on the use of digital technologies for science for health at Inserm could gather feedback and develop a culture of transparency surrounding the uses of AI systems.

**8- Continuous training:** Training all staff, both scientific and administrative, in digital technologies and the use of AI systems, their principles and limits, is crucial for informed and responsible use.

**9- Health data passport:** A "data passport" documenting the origin, quality and biases of the data would enable their responsible reuse.

<footnote n="3">³ During his 2025 New Year's address to head office staff, CEO Didier Samuel mentioned the creation of an Inserm unit dedicated to AI systems.</footnote>
10- **Transparency portal**: A portal would give participants in research conducted at Inserm visibility over the use of their health data, thereby ensuring transparency, regulatory compliance and flexibility for research.
# Why this document?

Artificial intelligence (AI) systems have become commonplace. They are a field of research and enable the development of research. An example? AI system developers were awarded two Nobel prizes in 2024! The Physics prize went to John J. Hopfield (Princeton University) and Geoffrey Hinton (University of Toronto) “*for foundational discoveries and inventions that enable machine learning with artificial neural networks*”<fnref n="4" />. The Chemistry prize went to Demis Hassabis and John Jumper from the Google DeepMind team that designed the AlphaFold protein structure prediction software<fnref n="5" />.

The growing use of AI systems in medical and health research, particularly in institutions such as Inserm, raises major challenges that are both promising and invite questions. While AI systems are obviously used by Inserm to support the scientific approach, their use is also developing within administration, which may eventually involve decision-making support in terms of projects, management or human resources. AI systems offer undeniable advantages: they enable massive amounts of data to be processed, accelerate the discovery of new mechanisms responsible for diseases, they can help reveal new biomarkers, help predict the onset of diseases, and enable us to envisage increasingly customizable medicine. On a daily basis, AI systems become remarkable tools to help write text by producing abstracts, document plans, proposing style improvements, correcting spelling and grammar, helping to generate and verify computer codes or as a translation tool, all of which save time on tasks with low added value. However, this technological revolution also comes with harmful impacts, for example energy costs and environmental repercussions, and numerous questions about the more or less long-term effects of these technologies, the associated uncertainties and risks that particularly concern the quality and confidentiality of data, algorithmic biases, the interpretability of results, the dependence on proprietary technologies, the weakening of practices and infrastructures in the event of malicious attacks, and the loss of sovereignty.

Inserm staff and governance must therefore organize themselves in order to address the ethical, scientific, legal and social challenges posed by AI systems: how to enable transparency and

<footnote n="4">⁴ “This year’s two Nobel Laureates in Physics have used tools from physics to develop methods that are the foundation of today’s powerful machine learning. John Hopfield created an associative memory that can store and reconstruct images and other types of patterns in data. Geoffrey Hinton invented a method that can autonomously find properties in data, and so perform tasks such as identifying specific elements in pictures.”
https://www.nobelprize.org/prizes/physics/2024/press-release/</footnote>

<footnote n="5">⁵ https://www.nobelprize.org/prizes/chemistry/</footnote>
non-discrimination in often opaque algorithms? What place should (or should not) be
maintained for human expertise in the face of systems capable of "predicting" and then
confirming diagnoses or prescribing treatments? In terms of public health, how can we control
the risks of increasing heterogeneity in care and deepening inequalities in access to care,
possibly associated with AI systems, particularly for populations under-represented in
databases? How can AI systems be used in a rational way to limit the ecological impact to what
is strictly essential?

Our ability to learn how to use, monitor and develop the most rigorous and upstanding practices
of these technologies will be a crucial factor in minimizing the negative effects that can be
anticipated and maximizing their impact on biomedical innovation and improved care for all
patients. This arbitration work must be the subject of debates and be able to be conducted in a
collective and transparent manner.

At Inserm, the integration of AI systems represents both an opportunity to advance knowledge, but also a collective responsibility, in order to ensure that these technologies are used equitably, ethically, effectively, and transparently. A distinction must also be made depending on whether Inserm is using or designing such systems.

Like human intelligence, which is expressed in multiple forms and particularly through the
creativity of scientific researchers, artificial intelligence has many dimensions. Although the
term appears everywhere these days, its origin is already old, since it was in 1956 at the
Dartmouth conference that the term "artificial intelligence" was officially used for the first time
by John McCarthy, Marvin Minsky, Nathaniel Rochester, and Claude Shannon<fnref n="6" />. Public research
is a central player in the development of AI systems. Today, many algorithmic tools are used in
the highly diverse daily lives of most scientific disciplines, whose practices and perceptions
they have profoundly changed. The recent rise of generative AI systems and the provision, at
no direct apparent cost to users (unaware of the aspiration of their data by these systems and
the associated environmental impacts), of Large Language Model (LLM) systems such as
ChatGPT from Open IA, or DeepSeek-R1 from DeepSeek, are a new source of disruption that
concerns even more numerous dimensions of scientific work. The rapid adoption of generative
AI systems makes it necessary to reflect on the conditions and effects on scientific work of

<footnote n="6">⁶ https://co-ia.fr/chronologie-histoire/.</footnote>
these new technologies, which will very likely continue to undergo active developments in the
years to come. Questions relating to the integrity and ethics of research arise in particular.

With Regulation (EU) 2024/1689 laying down harmonised rules on artificial intelligence (AI Act), the European Union has positioned itself at the forefront of the regulation of AI tools.

The AI Act came into force on August 2, 2024, and the implementation of its provisions is staggered over time, depending on the level of risk associated with the AI systems concerned. The AI Act defines an "AI system" as "the process of obtaining the outputs, such as predictions, content, recommendations, or decisions, which can influence physical and virtual environments, and the] capability of AI systems to derive models or algorithms, or both, from inputs or data"<fnref n="7" />. The European Commission published guidelines on the definition of AI systems on February 6, 2025⁸.

It should be noted that the AI Act excludes from its scope AI systems or AI models specifically developed and commissioned solely for scientific research and development, as well as their output<fnref n="9" />. Indeed, the European legislator believes that the Regulation “should support innovation, should respect freedom of science, and should not undermine research and development activity”<fnref n="10" />.

However, the outlines of this important research exemption have not yet been precisely defined.
Some players, such as the European Federation of Pharmaceutical Industries and Associations (EFPIA), have already alerted the European authorities to the necessity for clarification in this regard.

It should be remembered that the AI Act is designed as a regulation applicable to a "product",
very much inspired by the European regulation on medical devices (MD). This means that there
are regulatory markers to put in place throughout the life cycle of the product and that if the
result presents a risk, the responsibility of the players involved in the emergence of this risk

<footnote n="7">⁷ https://www.vie-publique.fr/questions-reponses/292157-intelligence-artificielle-le-cadre-juridique-europeen-en-6-questions.</footnote>

<footnote n="8">⁸ https://digital-strategy.ec.europa.eu/fr/library/commission-publishes-guidelines-ai-system-definition-facilitate-first-ai-acts-rules-application.</footnote>

<footnote n="9">⁹ Article 2.6 of the AI Act.</footnote>

<footnote n="10">¹⁰ Recital 25 of the AI Act.</footnote>
will be sought throughout the value chain. Given its research activities, Inserm must therefore closely monitor these future clarifications, as well as the governance arrangements that will be put in place to manage its projects. In any case, while the AI Act does not apply to certain AI systems developed and used by Inserm exclusively for research purposes, other regulatory frameworks remain relevant – particularly the GDPR in the event of the processing of personal data.

Techniques that enable inference when constructing an AI system include:

* machine learning approaches, which learn from data how to achieve certain objectives;

* logic- and knowledge-based approaches that make inferences from the encoded knowledge or the symbolic representation of the task to be resolved.

These techniques go beyond basic data processing in that the inference capacity of an AI system enables it to learn, reason<fnref n="11" /> or model.

Because AI systems are designed to operate at different levels of autonomy, they benefit from a variable degree of independence in their action from human intervention and the ability to operate without human intervention.

The European regulation also defines:

* a remote biometric identification system as "an AI system intended for the identification of natural persons without their active involvement, typically at a distance, through the comparison of a person's biometric data with the biometric data contained in a reference database";

* a real-time remote biometric identification system as "a remote biometric identification system, whereby the capturing of biometric data, the comparison and the identification all occur without a significant delay, comprising not only instant identification, but also limited short delays in order to avoid circumvention".

With research being one of the sectors that generative AI can transform very quickly, the European Commission has also published guidelines on the responsible use in research, "which aim to preserve research integrity". Intended for researchers, research organizations and funding agencies, these guidelines are based on pre-existing charters and guides (COPE, ALLEA, etc.). For the writing of articles, we can cite reporting guidelines such as TRIPOD-

<footnote n="11">¹¹ Reasoning versus the application of logical rules. While the term “reasoning” is always used in AI, it is indeed the application of logical rules by algorithms.</footnote>
AI, which are required by many journals<fnref n="12" />. While encouraging scientists to make the most of these new tools, they are asked to avoid using them in certain practices such as the peer review of articles and the evaluation of research projects. Research institutions must provide or facilitate training on these tools and monitor how they are developed and used within them.

A research institution such as Inserm must address this issue in order to determine a clear policy regarding the use of AI systems in both research and administration, and to enable its personnel to know its recommendations and promote debate. Such a best practice guide is all the more important as the quality of scientific output using AI systems will go hand in hand with the responsibility of the scientists who will use these systems. In fact, here ethics is already part of the practice of law. As lawyer Juliette Sénéchal points out in a recent analysis<fnref n="13" />: "The technological disruption resulting from the 'general use' capacity of these new AI systems and the enthusiasm of users that immediately resulted from it, first and foremost distanced these 'systems' from the conventional concept of a product, envisaged as a simple technical 'tool' with a certain number of specific functions, in order to move closer to more flexible and versatile concepts, such as those of digital services and illicit content within the meaning of the Digital Services Act (DSA)<fnref n="14" />, for two reasons. The first is that these general-purpose AI systems were immediately offered as complementary services by providers of digital search engine services, social networks, or marketplaces within the meaning of the DSA. The second is that these general-purpose AI systems, as generative AI, can produce, as a result of their output, deep fakes<fnref n="15" /> that could, for some, be qualified as illegal content under the DSA and which will, in addition, potentially be intended to circulate massively on online platform services. Take, for example, the pornographic deep fakes of US singer Taylor Swift, which were recently viewed 47 million times within a few hours on a digital social network service<fnref n="16" />." We must therefore take particular care to seek to prevent (by detecting and reporting) the use of doctoring, plagiarism, usurpation, etc. which result in violation of the rules of scientific integrity.

<footnote n="12">¹² https://www.tripod-statement.org/wp-content/uploads/2019/12/TRIPOD-Abstract_AI_Paper.pdf.</footnote>

<footnote n="13">¹³ L'AI Act dans sa version finale – provisoire -, une hydre à trois têtes Juliette Sénéchal, Dalloz Actualité, March 11, 2023 https://www.dalloz-actualite.fr/flash/l-ai-act-dans-sa-version-finale-provisoire-une-hydre-troistetes.</footnote>

<footnote n="14">¹⁴ Regulation (EU) 2022/2065 of 19 October 2022 on a Single Market For Digital Services and amending Directive 2000/31/EC, OJ L 277, 27.10.2022, p. 1–102.</footnote>

<footnote n="15">¹⁵ AI Act: “deep fake means AI generated or manipulated image, audio or video content that resembles existing persons, objects, places or other entities or events and would falsely appear to a person to be authentic or truthful”.</footnote>

<footnote n="16">¹⁶ Le Monde, January 26, 2024, Taylor Swift victime de fausses images pornographiques, l'intelligence artificielle en accusation. https://www.lemonde.fr/pixels/article/2024/01/26/taylor-swift-victim-de-fausses-images-pornographiques-lacommunaute-des-swifties-a-la-rescousse_6213215_4408996.html.</footnote>
Furthermore, the increased use of this type of tool may lead to an increased dependence on proprietary systems, subject to logic (particularly financial, aspiration of data, etc.) that may not be compatible with compliance with good scientific practices, such as consent, respect for intellectual property, ad hoc citations, verification of findings.

In Appendix 2 of this document, we recap the operating principles of the major AI systems, with the main body of the document itself highlighting the limitations and constraints before proposing a certain number of recommendations for Inserm governance and staff.

In order to develop recommendations for personnel in the Inserm units and beyond for health and life sciences research, the Ethics Committee has mobilized a working group in partnership with leaders from the LORIER program and the Scientific Advisory Board, together with AI system experts (see Appendix 1 for a full list of the Working Group members).

The use of AI systems must comply with the four main principles of scientific integrity as defined by the European Code of Conduct<fnref n="17" />:

* *Reliability* in ensuring the quality of research, reflected in the design, methodology, analysis and use of resources.

* *Honesty* through transparent, honest and unbiased research from the beginning, peer review and communication.

* *Respect* for scientific colleagues, research participants, society, including its cultural heritage and ecosystems.

* *Accountability* for the research from idea to publication, for its management and organization, for training, supervision and mentoring, and for its wider societal impacts.

Finally, it should be noted that several research institutes comparable to Inserm have developed strategies to support their staff in the ethical and upstanding use of AI systems<fnref n="18" />.

**While AI systems are not limited to LLMs, the very architecture of LLMs raises specific questions**

<footnote n="17">¹⁷ https://european-research-area.ec.europa.eu/news/living-guidelines-responsible-use-generative-ai-research-published.</footnote>

<footnote n="18">¹⁸ https://www.helmholtz.ai.</footnote>
The development of generative AI thanks to Large Language Models (LLMs), which are
developed from the embedding of large quantities of texts and the resulting avatars,
conversational agents such as ChatGPT have highlighted an easily accessible, seemingly
inexpensive AI full of promise (and fantasies)<fnref n="19" />, which leads us to immediately highlight
several areas of questioning:

- AI systems use and especially reuse data. This reuse is the credo of such systems. AI systems would make it possible to re-use data graveyards that can and should be used, to "get them out" and utilize them. A term fairly reminiscent of marketing<fnref n="20" />, it requires us to think about the validity of the data when the context of use changes. Thus, the secondary use of medical and administrative data highlights a pronounced discrepancy between the initial intention of their collection in a specific context and the re-uses, sometimes idealized, to which they are subject, whether in isolation or in combination, taking into account potential changes over time and the particularities of each collection and/or processing center.

- While an LLM searches large quantities of texts and answers general questions very well, the relevance of its response is often limited for questions related to specific medical or biological expertise. While retraining with more specific data can make it possible to refine the results, the quality of the responses may remain unsatisfactory insofar as the reasoning methods, experience and knowledge mobilized are not always available or can even be explained in the form of text.

- (Re)training carried out in medicine raises more acutely than elsewhere the question of the veracity of the answers given by the agent: the LLM was trained to produce statements that are “plausible” for a human reader by displaying combinations of signs (e.g. text) but cannot discern what is true<fnref n="21" /> from what is not. Therefore, regardless of the size and quality

<footnote n="19">¹⁹ See also the Inserm page maintained by LIMICS: https://www.Inserm.fr/dossier/intelligence-artificielle-et-sante/.</footnote>

<footnote n="20">²⁰ https://grapheneai.com/the-ai-analyst-strategic-insights-and-data/ or https://medium.com/@renhaosu_12442/why-some-ai-ideas-end-up-in-the-graveyard-lessons-learned-c399854abbf7.</footnote>

<footnote n="21">²¹ "Discerning the truth" is taken here as the possibility to verify, particularly by means of IT procedures, the reality of a statement and therefore evaluate its accuracy.</footnote>
of the corpuses on which it relies, we can never be sure of the reliability of the response produced. “Hallucinations” by LLMs remain possible.

For learning and LLMs alike, if the data or texts are linked to patients, the rights of these patients must be respected: professional secrecy and data confidentiality, non-use for research without the prior notification or consent of the parties concerned, application of the General Data Protection Regulation (GDPR), the Digital Act, the European Artificial Intelligence Regulation known as the “AI Act” and other provisions of national law, etc. This issue of respect for the rights of patients or healthy volunteers is also present in the case of a clinical trial or in the reuse of original data collected as part of medical care or previous research. Hospitals are responding to this problem by organizing the reuse of data in protected Healthcare Data Warehouses (HDW). This reuse in the context of community medicine or in community-hospital collaborations is currently under construction: see the Health Data Hub (HDH) and the HDW reference framework/requests for authorization from the French Data Protection Authority (CNIL) for the implementation of community medicine HDW (in progress). A proposal for an interministerial *national strategy* on the secondary use of health data<fnref n="22" />, was opened for public consultation in October and November 2024. The use of health data may be particularly intrusive when it concerns patients or research participants, but also when it reflects the practices of the healthcare and research professionals who produce it. It is therefore important to note that there are several levels of confidentiality, both for patients and professionals, and that confidentiality is only one of the many issues that can make this data sensitive.

- The capacity of AI systems (symbolic or numerical) to propose findings (of reasoning or calculation) must not render them independent of human intervention<fnref n="23" />. Especially in medicine where patients’ lives are at stake (even image-based screening, which is highly effective, incurs human liability). Ethical reasoning, both professional ethical and consequentialist, supports this conclusion by asking what would be the price to pay to do without a human being (decision quality gradient). This is set out in the French Data

<footnote n="22">²² Stratégie interministérielle pour construire notre patrimoine national des données de santé 2025-2028 (sante.gouv.fr) and https://esante.gouv.fr/actualites/consultation-publique-sur-lutilisation-secondaire-des-donnees-de-sante.</footnote>

<footnote n="23">²³ Some people are considering the adoption of numerous legal liability arrangements. We would like to point out here that until now it has always been the doctor or researcher who is liable. This is implied in the bioethics law but not explicitly written.</footnote>
Protection (Art. 47) and Bioethics Acts (Art. 17)<fnref n="24" /> which, in addition, requires a precise
explanation of how it works for the users and explicit information for the patient. Therefore
an obligation of explainability weighs on designers, so that user healthcare professionals
can exercise control over the results of the processing in compliance with the guarantee
provided by human intervention (French Public Health Code (CSP), art. L. 4001-3) (see
Appendix 3).

- Finally, recent advances in AI are largely the result of the significant reduction in data access costs and in the cost of on-demand computation mobilization. However, it is crucial to emphasize that, like any technology, these tools are never neutral, be it in their design phases or in their uses, and that they may have been trained on data that do not reflect the principles of equity. Indeed, every stage of their development, from the basic algorithms and the quality of the initial data used to the final applications – often with enormous human work that cannot be neutral – is imbued with choices and values that reflect the potential biases of the designers and their inclusion in a social context. In addition, it is important to keep in mind that many of these tools are being developed, at an extremely rapid rate, by private companies for commercial purposes, in a competitive market, which raises questions about the ability of legislators to guarantee the quality of services provided in the “public” interest and the absence of biases brought about by economic, political or other issues.

<footnote n="24">²⁴ https://www.legifrance.gouv.fr/jorf/id/JORFTEXT000043884384.</footnote>
# Ethical questions regarding the use of AI systems at Inserm

AI systems are already transforming and will continue to profoundly transform the day-to-day practices in Inserm's laboratories and administrative departments alike. We therefore wish to consider the consequences of these uses, the potential benefits and risks, and the open questions in order to develop in the following chapter the proposals for ethical and responsible use of AI systems.

## 1. Evaluation of the real carbon cost and ecological footprint of using AI systems at Inserm:

* Environmental responsibility: The AI systems boom is associated with a huge increase in the ecological footprint, which seriously undermines our ability to meet the French and European commitment to net zero emissions by 2050²⁵. The overflow of digital technology and AI systems in all our spheres leads to growing unease in the face of an asynchronism between the hyper-acceleration of the present under the effect of the digital transition and the speed of development of AI systems, and a blockage of the future due to climate and ecological disruption<fnref n="26" />. There is a marked opposition between the digital transition – and even more so with the development of AI systems (a Web query mobilizing pre-indexed data is much less costly in computing power than the AI system called upon for the same service) – and the socio-ecological transition<fnref n="27" />. Indeed, dematerialization does not lead to a reduction in the exploitation of natural resources – such exploitation has tripled since the 1970s<fnref n="28" />. In France, “by 2030, the power of five to seven nuclear reactors could be mobilized by the data center sector alone¹”. Add to this the energy needed to extract the metals and rare earths used in the terminals, and the result is catastrophic and lasting ecological damage<fnref n="29" />. Moreover, this outrageous development places demands on an invaluable resource: water.

Water is transformed into polluted ecocidal effluents by the extraction methods, particularly for rare earths<fnref n="30" />. The amount of water used to cool equipment is also on the rise, with an increase of 17% between 2022 and 2023 for Google and 34% between

<footnote n="25">²⁵ L'insoutenable coût écologique du boom de l'IA, Nastasia Hadjadji, July 2024 REPORTERRE</footnote>

<footnote n="26">²⁶ IPCC 2023 Synthesis Report.</footnote>

<footnote n="27">²⁷ L'impasse collaborative. Pour une véritable économie de la coopération. Éloi Laurent. October 2018 ed. LLL</footnote>

<footnote n="28">²⁸ International Institute for Sustainable Development 2021 BRIEF #16 Sustainable Use of Natural Resources: The Governance Challenge, Jennifer Bansard and Mika Schröder.</footnote>

<footnote n="29">²⁹ Evaluation de l'impact environnemental du numérique en France et analyse prospective. Ademe Arcep, January 2022.</footnote>

<footnote n="30">³⁰ Les terres rares : Quels impacts ? EcolInfo pour une informatique éco-responsable.</footnote>
2021 and 2022 (i.e. the volume of more than 2500 Olympic swimming pools<fnref n="31" />) for Microsoft. A recent study shows that AI systems could consume between 4.2 and 6.6 billion cubic meters of water in 2027, a consumption that is slightly higher than that of Denmark and equivalent to half that of the UK<fnref n="32" />. We call upon these two AI system giants through many queries without being fully aware of the impacts on ecology and in turn on health.

*   **Social responsibility:** The recommendation to evaluate and regulate greenhouse gas (GHG) emissions induced by the use of AI systems at international level⁸ should lead to recommendations for frugality in the creation and use of new AI system algorithms, combined with communication on the ecological impact that would enable users to make reasoned choices⁸. These reasoned choices also require being fully informed about the life cycle analysis of each terminal. While frugality is a start, it nevertheless remains insufficient when faced with the scale of the climate challenge. It is now imperative to make a transition to sobriety. The reasoned use of AI systems with a view to social well-being and taking into account that the limits of the resources necessitate an urgent change of direction.

## 2. The question of AI system size

The principle of minimization should lead us to ask ourselves: “Do we still need to use large systems?”

There are LLMs that use a reduced dataset and fewer parameters, representing AI that is both frugal (in terms of learning data) and sober (in terms of electricity consumption in use). We are talking about Small Language Models (SLMs). They are easier to deploy with limited resources. We can compensate for a smaller learning sample and therefore theoretically poorer performance by the fact that we can make it more efficient by developing it on more specific data that we control. Several healthcare data warehouses are based on these systems.

This addresses several ethical and normative challenges:

*   in terms of digital sovereignty, the use of SLMs can strengthen the technological independence of institutions and nations, reducing dependence on large international tech companies;

<footnote n="31">³¹ Artificial intelligence technology behind ChatGPT was built in Iowa — with a lot of water. https://apnews.com/.</footnote>

<footnote n="32">³² Impacts de l'intelligence artificielle : risques et opportunités pour l'environnement. CESE Rapporteur.es Fabienne Tatot et Gilles Vermot Desroches September 2024.</footnote>
* in terms of privacy, smaller models may be easier to monitor and secure, providing better safeguards for the protection of patient and research participant data.

### 3. The design approaches for AI systems: bottom-up or top-down?

While many within Inserm are AI system users, more and more players are also AI system designers, either alone or in collaboration with other research institutes such as INRIA, CNRS or with private companies. We have identified the following ethical issues:

○ Inclusion and participation: a bottom-up design approach can encourage greater involvement of end users and stakeholders, ensuring that AI systems meet real needs and respect community values.

○ the risk of imposed technologies: a top-down approach can impose technological solutions without a real consensus or understanding of local impacts, which can lead to resistance or unethical uses. Support from the public authorities for the plurality of the solutions proposed, avoiding monopoly situations, can help protect against these risks.

### 4. Data quality, conditions of access and qualification

The missions of a national public research organization such as Inserm oblige us to consider the following:

* Equity and accessibility: Ensure equitable access to data, encourage the production of data, including in areas that are not immediately economically viable, avoiding discrimination based on technological resources or capabilities.

* Integrity and reliability: Maintain high standards for data quality and qualification to ensure accurate and reliable results.

* Equality of access: As with any technological innovation, it is possible that only the privileged few benefit from it, or even exploit new AI tools to the detriment of those who do not have access to them. Access to the data and computing resources necessary for AI systems raises this kind of dilemma. It is essential to ensure that the emergence of these technologies does not contribute to increasing inequalities. Let us take the example of the introduction of a fee for access for secondary use to data already generated and funded by French national solidarity, such as those of the National Health Data System (SNDS). Such a levy could benefit large players with significant financial resources, to the detriment of the smaller ones, thereby hindering the emergence of new companies or initiatives in the sector. Special vigilance on the part of the public
authorities is therefore necessary to prevent economic barriers from increasing existing
disparities.

* Open Science: The developments of generative AI systems are largely dependent on and benefit from the strong support of public institutions for the opening up of public data, including research data. It is important that this movement of openness, motivated by the desire to share knowledge in society, is not hampered but amplified. AI algorithms and tools developed using open data must remain accessible, at a lower cost, for public research institutions.

**5. Vigilance on the shift from expertise to execution, from knowledge to know-how**

AI systems are a research subject at Inserm with scientists working on theoretical questions, some of which go as far as the development of “tools” or demonstrators. However, the majority of the staff using AI systems at Inserm are not the designers. Particular attention must therefore be paid to:

*   **Loss of expertise:** Excessive displacement of human expertise towards AI systems can lead to a loss of critical skills, an excessive dependence on automated systems. This can also lead to situations of professional vulnerability when faced with a tool whose perfect mastery is a matter for specialists. The challenges of training and acquisition of expertise are major. How do we acquire expertise complementary to that of the AI systems, if we always let those systems “think and analyze” in our place? How can we support changes in learning practices?

*   **Critical evaluation:** It is crucial to maintain the ability to question and understand datasets, in order to avoid bias and errors in analyses and decisions based on AI systems. To avoid the total delegation of tasks to AI systems and the latter placing the user under supervision, the regulations oblige the industrial developers of AI systems to be transparent about the methods they use, since clarity and intelligibility are necessary for monitoring and for explainability by the user. With regard to development in an academic environment, Inserm will have to define the precise framework to be used.

*   **Verification of the transposability of the algorithm to the research in question:** It is mandatory that researchers evaluate the models they use and ensure that they are Fair, Appropriate, Valid, Effective and Safe (FAVES) in the context of their research. In the same way that hospitals must evaluate AI systems locally, it is up to the teams to ensure and document the correct use of the algorithms.
## 6. Place of the scientific hypothesis in the generation and questioning of datasets

Scientific integrity requires special attention to:

*   **Scientific rigor:** Maintaining the importance of the scientific hypothesis and rigorous methodology in the use of data and AI systems to avoid hasty, incorrect conclusions or the reproduction of discriminatory social norms.

*   **Transparency and reproducibility:** Ensuring transparency of the processes and models used so that the hypotheses and findings can be verified and reproduced. Reproducibility is one of the essential pillars of scientific integrity. Inserm has made this one of its priority areas within the LORIER program.

## 7. Vigilance on Intellectual Property (IP)

Ethical challenges:

*   **Justice and access:** Ensure that AI innovations are accessible and beneficial to all, and not restricted by excessive intellectual property barriers. The question of accessibility to source codes raises multiple complex issues. Inserm could issue recommendations aimed at fostering Open Science and Transparency by encouraging the publication of data, models and algorithms in open formats, while respecting standards of ethics and confidentiality. This approach would facilitate the sharing of advances in AI while strengthening collaboration with other researchers and institutes. Inserm could also recommend using Open-Source Licenses, developing and disseminating AI tools under open-source licenses, allowing other researchers and organizations to use, modify and distribute them, while avoiding access restrictions imposed by restrictive patents. Inserm could also promote partnerships with public and private stakeholders in order to build alliances with other research organizations, tech companies and public authorities to support the development of accessible innovations, while avoiding exclusive licensing agreements that could limit public access to developed technologies. Lastly, Inserm could establish a balanced patents policy. If patents are filed to protect certain innovations, Inserm could favor non-exclusive licenses or patent pool type licenses that facilitate access to the technology to several players in exchange for a low cost, or even free of charge for public bodies.

*   **Recognition:** Ensure equitable recognition and compensation for individual and collective contributions to the development of AI technologies.
Legal challenges:

* Risk of losing the priority criterion or the "innovative" criterion. Disclosure of research findings, or reasoned hypotheses on preliminary findings, by injecting a query into an LLM (a prompt containing all or part of the patentable discovery), threatens to cause loss of "originality" or "precedence" and compromises the obtaining or recognition of an intellectual property right in the case of data considered as already published in an AI system. If the results obtained are too directly "inspired" by protected content (computer code posted on a forum, etc.), the risk of counterfeiting, without the user's knowledge, is increased.

## 8. AI system-induced ways of life and consequences of AI systems on research and care practices

One of the ethical aspects to consider is the way in which technology shapes its users just as much as or more than the other way around:

* Societal changes: Examine how the use of AI systems modifies lifestyles, teaching practices, work practices and social relationships. Evaluate the ethical implications of these changes and prevent situations of vulnerability that can be detected upstream (ethics-by-design/Reflection-Anticipation-Deliberation).

* Quality of care: Ensure that the AI tools developed at Inserm improve quality of care, for example by giving healthcare staff more time to better personalize care, without dehumanizing interactions between patients and healthcare professionals. As a result, there are financial and organizational challenges, particularly with regard to equity of care. While the recruitment of IT specialists is becoming necessary to monitor and maintain critical distance from AI systems, vigilance regarding adverse effects on care – such as the consequent reduction in the number of healthcare staff – is required.

* Prevention and social surveillance: If the use of AI systems accompanies prevention practices or earlier diagnosis, the risk will need to be measured in terms of new widespread surveillance practices<fnref n="33" /> as well as the well-known risk of overdiagnosis in the context of screening (see also Michel Foucault’s concept of biopower: exchanging more life for more surveillance constraints).

<footnote n="33">³³ David Armstrong The rise of surveillance medicine Sociology of Health and illness Volume17, June 1995 Pages 393-404.</footnote>
* Research practices: Anticipate the effects of an ability to rapidly process ever-increasing quantities of data and bibliographic references on the quality and quantity of the scientific publications produced, and on the methods used to evaluate research.

* Administrative practices: Anticipate the effects of the use of AI systems in the field of human resources or recruitment.

## 9. Immediate benefits versus potentially harmful longer-term effects

AI systems are a recent, rapidly evolving technology whose impact over time has not been evaluated:

* Prudence: Weigh up the immediately perceived benefits of AI technologies with a prudent evaluation of the long-term effects, to control the benefit/risk balance. The evaluation of AI system-based approaches should incorporate efficacy, tolerability and social impact, as should changes in the organization of care. The ethical challenge is significant given our desire to not delay access to high-performance innovations. To supplement the human guarantee principle contained in the AI Act and evaluation "by design", larger-scale post-market evaluations are recommended, as is the case for real-world Phase 4 studies for medicinal products. The creation at Inserm of a national cross-cutting unit on the use of digital technologies for science for health will also make it possible to develop long-term impact scenarios.

* Intergenerational responsibility: Consider the long-term impacts on future generations, ensuring that current benefits do not compromise their well-being.

## 10. Protection of privacy versus the expected relevance of the research: The relevance of synthetic data

* The General Data Protection Regulation (GDPR) has rightly increased the focus on the protection of privacy. The utilization of personal data remains possible, provided that appropriate measures are put in place to guarantee confidentiality. In the field of health and biomedical research, these issues are even more critical given the sensitive nature of the data processed. The difference between "anonymous data" and "pseudonymous data" is not always well understood:

* Pseudonymization consists of removing directly identifying information (last name, first name, etc.) from a dataset, but these retain indirectly identifying elements (such as a unique combination of characteristics), making it possible to re-identify individuals by
crossing data or deciphering an identity hidden by a code kept in a correspondence
table<fnref n="34" />.

° However, anonymization is a one-way process: An irreversible process that prevents someone from being identified. The European data protection authorities (former G29, EDPB, and the French CNIL) have reaffirmed three criteria which, if they are jointly met, make it possible to guarantee that a dataset is truly anonymous: individualization, correlation and inference<fnref n="35" />. It should be noted here that the regulations in force are almost 10 years old with the G29 criteria<fnref n="36" />.

° It is in the context of anonymization that the generation of synthetic data takes on its full meaning: Recreating training data for LLMs with no confidentiality problems by exploiting the computing power of mathematical models. There are several methods. Some are based on mathematical principles, others are designed to meet the regulatory criteria of individualization, correlation and inference. Some approaches use models based on the data themselves, going as far as creating a local model for each observation, while others are based on purely theoretical equations. *Ultimately*, the synthetic data are fully simulated but realistic. They reproduce the characteristics of the original data, enable similar analyses, while ensuring that it is impossible in practice to trace real individuals with certainty. As a concrete example, several French university hospitals are involved in the PARTAGES project coordinated by the Health Data Hub, an LLM project with a common database of summary reports written by interns (and other hospital doctors). The shared database is a "common digital" file that can be used anywhere. Local reconfiguration will be done with the local document databases at each university hospital and cannot be used externally. The use of anonymous synthetic data whenever possible will quickly become an ethical requirement. They also constitute a privileged tool to encourage the opening up of data in research<fnref n="37" />. Finally, the use of

<footnote n="34">³⁴ Defined under article 4 of the GDPR, pseudonymisation is “the processing of personal data in such a manner that the personal data can no longer be attributed to a specific data subject without the use of additional information, provided that such additional information is kept separately and is subject to technical and organisational measures to ensure that the personal data are not attributed to an identified or identifiable natural person”.</footnote>

<footnote n="35">³⁵ For more information, see the [guidelines](https://www.cnil.fr/fr/le-g29-publie-un-avis-sur-les-techniques-d-anonymisation). They are currently being updated but the three criteria set out remain valid.</footnote>

<footnote n="36">³⁶ Singularity correlation inference. ([https://www.cnil.fr/fr/le-g29-publie-un-avis-sur-les-techniques-d-anonymisation](https://www.cnil.fr/fr/le-g29-publie-un-avis-sur-les-techniques-d-anonymisation); [https://ec.europa.eu/justice/article-29/documentation/opinion-recommendation/files/2014/wp216](https://ec.europa.eu/justice/article-29/documentation/opinion-recommendation/files/2014/wp216), en.pdf).</footnote>

<footnote n="37">³⁷ In accordance with this, the white paper *Données de santé artificielles : analyse et pistes de réflexion*, April 2024.</footnote>
synthetic data can improve the representativeness of a dataset and the correction of qualitative and/or quantitative bias by augmentation.

It should be stated here that colleagues consider in the current phase that expectations regarding synthetic data appear to be very uncertain, excessive and not based on sufficient evidence in most cases. These colleagues warn of the risk of inappropriate commercial development of these synthetic datasets.

These reservations lead to clarifying the contexts in which synthetic data are used:

Firstly, there are synthetic data that can be qualified as anonymous, and others not, some of which are produced by differential confidentiality, which is mathematically comfortable and has in fact been the subject of the creation of several companies, including, for example, Hazy recently acquired by SAS. While there are many uses, just that of publishing data that enable the analysis of an article to be reproduced<fnref n="38" /> seems to justify relevance for Inserm: codes, anonymous data within the meaning of the GDPR and comparison with the pseudonymized data available<fnref n="39" />.

Secondly, care must be taken not to introduce confusion: In synthetic data, the essential criterion is the more or less sensitive nature of the data and, first and foremost, their potential to re-identify research participants. This potential justifies their legal regulation and demands that we qualify them as anonymous or not. In fact, even if anonymized in appearance, they can remain sensitive. It will therefore also be a question of working on the best way to "desensitize" them. As there are several techniques for producing synthetic data (more or less "data-driven", more or less "model-driven"), there are always scientific and technical hypotheses in the data and modeling production processes<fnref n="40" />. It is therefore necessary for each case to identify where a security breach may exist. There are also several techniques for monitoring data usage: blockchain, federation, etc. Synthetic data can also make it possible to desensitize personal data. Thus, in the PRIMUS university hospital research in health (RHU) project, anonymous synthetic data are used to desensitize clinical trial data in Multiple Sclerosis: their quality is determined by the use, in this case the feeding for intensive use of a decision-

<footnote n="38">³⁸ https://jnnp.bmj.com/content/92/2/122.</footnote>

<footnote n="39">³⁹ https://github.com/ICAN-aneurysms/RIA-predict/tree/master/notebooks.</footnote>

<footnote n="40">⁴⁰ Guillaudeux et al. Patient-centric synthetic data generation, no reason to risk re-identification in biomedical data analysis, https://www.nature.com/articles/s41746-023-00771-5.</footnote>
making tool. In addition, a federated model proposes monitoring the use of synthetic
data by the legal data managers.

11. Language or voice bias and hallucinations

As the findings of the AI systems are based on the representativeness of the data, what
is poorly represented is poorly processed. A study published in early October in BMJ^41^
warns against the use of chatbots based on health LLMs due to their linguistic bias.
Researchers from the universities of Hong Kong, Singapore and RMIT in Vietnam have
found that the ChatGPT-3.5 (OpenAI) generative artificial intelligence (AI) module
provides erroneous information when questioned in a less common language. When
questioned in Vietnamese, the chatbot misinterpreted the symptoms of atrial fibrillation
as being those of Parkinson's disease. The researchers recommend improving the
models' translation capacities through the open-source sharing of diverse linguistic data.
This is the law of series for OpenAI, which must also respond to criticism of its voice
recognition tool Whisper, which is accused of frequent "hallucinations." Notably used
by French start-up Nabla, which designs AI-powered medical transcription software,
the tool frequently invents entire text passages when faced with moments of silence.
These hallucinations distort medical records and ultimately increase the workload of
doctors, who then have to meticulously check the content generated... wasting the time
allegedly saved. OpenAI does not recommend "the use of Whisper in high-risk
domains".

<footnote n="41">⁴¹ Kwok et al. Health information for all: do large language models bridge or widen the digital divide? *BMJ* 2024;387:e080208, doi: https://doi.org/10.1136/bmj-2024-080208 (Published 11 October 2024).</footnote>
# Our recommendation proposals<sup>42</sup>

## 1- Transparent disclosure and attribution

Inserm staff must clearly state any use of AI systems in their research, particularly the tools, algorithms and specific parameters used, including in responses to calls for proposals and in the "Materials and Methods" section of articles. Any sources of information or ideas derived directly from AI systems should be indicated precisely, clearly distinguishing contributions derived from human ideas from those directly proposed by AI systems. The concern to duly cite human expertise and previous literature must be maintained, especially when machines do not provide such citations in their results. In order to support Inserm personnel in their day-to-day tasks, whether they are researchers, technical or administrative staff, we recommend setting up a training course to address their professional challenges.

In practice: In the "Materials and Methods" section of submitted articles, provide for a "Use of AI systems" section with, as sub-sections, the contexts of their use

- To define the question asked

- To define the method chosen and/or to answer this question

- To perform the bibliographic search

- To summarize it

- To write all or part of the article

As in the statistical section where it is mentioned that the calculations were carried out with such-and-such a software application, the AI systems used would be indicated. Also specify which human authors oversaw the eventual AI system work.

In a recent JAMA editorial, a highly developed proposal was put forward.<fnref n="43" />

In dialogue with AI system designers and developers, Inserm staff must:

- ensure that the designers of the AI systems used for their research respect the ethical rules and regulations in force;

- provide publicly accessible details on the conditions for obtaining models produced by AI systems, and particularly the data used to create or refine them;

<footnote n="42">⁴² We indicate having taken up some of the proposals made by Blau et al. https://www.pnas.org, May 29, 2024 by adapting them to the specific situation of health research.</footnote>

<footnote n="43">⁴³ Flanagin et al. JAMA April 2, 2024 Volume 331, Number 13.</footnote>
* carefully manage and publish information about the models and their variants in order to provide other scientists with a way to cite the use of particular models with specificity;

* provide long-term archives of models to enable replication studies;

* dismiss a finding when the generated content cannot be correctly attributed;

* pursue innovations in learning, reasoning and information search machines aimed at providing users of these models with the ability to assign/recognize the sources and authorship of data used in the content generated by artificial intelligence. Be transparent with developments and explain their cause and nature;

* enable free or low-cost use for academic research institutions when the developments have mobilized data that are freely available or made available by public actors.

## 2. Verification of AI system-generated content and analysis

Inserm staff are responsible for the accuracy of the data, images and deductions they derive from their use of generative AI models. Responsible use of AI demands the use of appropriate methods to validate the accuracy and reliability of the deductions made by or with the help of AI systems, as well as a full disclosure of the elements of proof relevant to these deductions. This includes monitoring and testing biases in AI system algorithms and findings, with the aim of identifying and correcting biases that may distort research findings or interpretations.

Inserm staff must know and report the limitations of the systems' capacity to confirm the truthfulness of AI system-generated data, texts or images. Inserm should give them the means to do so. The creation of a national cross-cutting unit on the use of digital technologies for science for health at Inserm shall make it possible to collect this information and contribute to training. This new structure could be integrated into the LORIER program. Where it is not possible to verify the truthfulness of the content generated, the findings of the model should provide clear and well-calibrated evaluations of confidence. The creators of models must proactively identify, report and correct AI algorithm biases that may distort research results or their interpretations. It should be emphasized here that it is necessary to distinguish between two types of bias: those of the AI systems themselves, linked to the database and its processing method, and those linked to the users (who express themselves in the questions they ask of the AI systems).
### 3. Documentation of the findings produced by AI systems

Inserm staff must mark (incorporation of watermarks) the data, deductions and images generated with the help of AI systems or synthetic data with source information on the role of AI systems in their creation, so that they are not confused with observations collected in the real world. Scientists must not present AI system-generated content as observations collected in the real world.

Inserm staff in dialogue with the model designers must clearly identify, annotate and retain the source of synthetic data used in their training procedures and monitor problems, concerns and behaviors arising from the reuse of computer-generated content for the training of future models.

### 4. The importance of ethics and equity

Inserm staff are called upon to exercise caution in their use of AI systems, taking into account their limitations, potential biases and impacts, and to implement verification strategies to ensure that their use of AI systems produces scientifically valid and socially beneficial findings, while taking appropriate measures to mitigate the risk of harm. Once again, the training of all research stakeholders, including students and administrative personnel, is essential.

Inserm staff must respect the ethical guidelines relating to the use of AI systems, in particular regarding the clear attribution of the sources of data derived from observation compared to those generated with the help of AI systems, respect of copyright and other intellectual property rights, respect of the law relating to personal data and the protection of privacy, respect of the rules of the disclosure of certain specific data and respect of the rules concerning consent to access such data, as well as the detection and correction of potential bias in the construction and use of AI systems.

Inserm staff should also remain vigilant regarding the societal consequences that may arise alongside the development and deployment of AI systems, and update the practices and rules that promote beneficial uses and mitigate the prospect of ecological and social damage.
5. Special attention to personal data used with or generated from AI systems

Inserm staff are asked to use the personal data of cohort participants or patients only when their research requires it. One way to develop research without the risk of breaching sensitive personal data is to use synthetic data. Three dimensions of synthetic data are particularly important: confidentiality, verisimilitude and augmentation.

a. Confidentiality is ensured by the simulated generation of anonymous synthetic data, thereby making it possible to share data without being subject to the constraints of personal data regulations, such as the GDPR and the G29/EDPB criteria in Europe. The work of generating synthetic data must focus on ensuring against any form of re-identification and on the rigorous documentation of these efforts. Indeed, in the case of small numbers of people, for example in rare diseases, even mathematically simulated data can be considered personal if they are too close to real data;

b. Verisimilitude consists of producing simulated data that are as realistic as possible. They faithfully reproduce the sets of real data, preserving multiple subsequent uses. Well-developed synthetic data must appear plausible while avoiding representativeness bias. Achieving a high degree of verisimilitude guarantees the value of synthetic datasets, making confidentiality an additional benefit, rather than a primary objective.

c. Finally, data augmentation is another essential aspect, perhaps more longstanding, where synthetic data enrich existing data by generating additional context-specific data.

Inserm and the researchers guarantee the confidentiality of the sensitive data in their possession and must under no circumstances inject potentially-identifying health data relating to patients or research participants into a Chat without being able to control how they may be reused and who will receive them (to ask the AI system to generate similar fictitious data, etc.). This obligation should be clearly stated in the institution's IT Charter.

Inserm staff and governance are asked to promote equity by being vigilant about the questions for which AI systems are used. They are also asked to promote equitable access to AI tools and AI system training. These efforts should enable a diversified community of scientific researchers to effectively utilize AI systems and respond to the diversity of health research needs, including those of groups of people or patients who have traditionally been poorly considered or marginalized. In addition, methods to solicit meaningful public participation in
the evaluation of equity and justice in the use of AI system technologies should be explored and
used.

AI systems should not be used without attentive human oversight during the decision-making stages of peer review processes or decisions on career advancement and funding allocation.

Within the framework controlled by the governance of the processed data, promote a policy or even a service offer for research teams favoring environmentally sober and secure AI systems hosted in trusted infrastructures compliant with the regulations in force.

**6. Continuous monitoring, surveillance and public engagement**

Inserm staff, as well as representatives of academia, industry, government and civil society, are asked to continuously monitor and evaluate the impact of AI systems on greenhouse gas emissions, the scientific process, scientific working conditions, the quality of published scientific findings and, in full transparency, to adapt strategies if necessary to maintain integrity. With AI system technologies evolving rapidly, researcher communities must continue to examine and understand the contributions, deficiencies and influences of AI systems, strive to anticipate and prevent harmful uses and exploit their potential to address critical societal challenges. At the same time, AI system specialists must strive to improve the effectiveness of AI systems for the sciences, particularly by addressing the issues of truthfulness, attribution, explanation and transparency of the training data and the models generated. Efforts must be made in the different scientific fields to continue the ongoing study of the state and dynamics of the use of AI systems in research and to develop new procedures to integrate the viewpoints and experiences of the data subjects (patients, professionals, users, wider public) as the AI systems are developed, applied and regulated. The existence of these participatory approaches and their findings should be widely disseminated.

As part of the establishment of the national cross-cutting unit on the use of digital technologies for science for health at Inserm, we recommend that its governance establish a web portal in which it is possible to report instances of drift or unsuitability.

Inserm staff are called upon to collectively evaluate (in labs, communities of practice, etc.) the transposability of AI systems to their research, the positive or problematic impacts of these tools on their individual and collective practices, particularly with regard to the rationales for creating value from/publishing research, the impact on their expertise and its transmission. The national cross-cutting unit on the use of digital technologies for science for health at Inserm will be
tasked with collecting and sharing within the scientific community the new expertise thus created.

Given the environmental impact of the use of AI systems, research focused on sobriety (frugal AI) is a major strategic element to consider in the development and use of AI systems at Inserm.

## 7. Personnel training at Inserm

It is essential to ensure that Inserm personnel are regularly trained as AI users, developers and researchers. This covers AI systems as research subjects or as tools for writing/conducting research projects. The aim is to train students and tenured staff in the principles, limits and challenges of AI systems.

## 8. Creation of a health data passport

The quality and relevance of data are central to the effectiveness of the AI systems used in biomedical research and health. It is therefore essential to be able to have the associated metadata available. Research is beginning to report on the ethical consequences of misuse or inappropriate use of data in an AI context. Similarly, the questions raised by the reuse of data that are collected for a specific purpose and then used in another context should be examined, and the scientific, ethical and legal dimensions should be explored. Indeed, the limits of reuse, which are based in particular on controlling the differences between the context of data collection and that of their reuse, are difficult for a human to identify, and even more so for AI.

This is why we recommend generalizing the association of metadata with any database to describe its characteristics, particularly the origin and methodology for obtaining the data. The aim would be to create a data passport, which would constitute an ethical and technical framework aimed at documenting the origin, quality, potential biases and transformations operated over the life of the data (standardization, filtering, etc.) to ensure appropriate and responsible reuse. This passport would support the granting of data visas, which would act as a filter each time the data are reused. The visa would make it possible not only to validate the suitability of the data for new contexts of use, but also to trace their path each time they are reused. It would include information on modifications, transformations or validations applied, problems or successes encountered, and indications on possible future uses.
This system would provide:

* greater transparency and traceability: the data passport and visa guarantee complete traceability of the origin and uses of the data. This increases accountability in the way data are processed, thereby avoiding blind reuse that could lead to bias or error;

* better prevention of bias: evaluating and documenting potential biases starting from the initial collection (population, context, granularity) helps to avoid their amplification during reuse, a crucial ethical aspect, especially in medical contexts;

* improved legal and ethical compliance: the visa could also include information on the legal and ethical constraints linked to the data (consent, GDPR), ensuring that they are not reused illegally or unethically;

* better protection against misuse: by limiting the use and reuse of data to compatible contexts, the passport and visa would limit the development of false results and erroneous conclusions that could harm, for example, populations that are poorly represented in the dataset.

In summary, the creation of data passports and visas can improve the uses and re-uses of AI system-assisted data by helping to make them more transparent and accountable, thereby minimizing the risks of bias, misinterpretation and non-compliance which, in biomedical research, may have potentially dramatic ethical consequences.

# 9 - Creation of a data transparency portal at Inserm

The creation of a transparency portal for a biomedical research organization like Inserm could be a major step forward in building trust with biomedical research health data donors, while meeting regulatory requirements and promoting agile research:

## 1. Transparency

The portal would above all be a transparency tool for study participants and health data providers, referred to hereafter as "donors". It would make it possible to:

- Clearly inform data producers and donors about the use of their data, the objectives of the research and the potential benefits.

- Present ongoing research projects in an accessible manner, with understandable summaries and regular updates on the progress of the work.
- Provide information on data management (retention period, types of analysis, level of confidentiality, etc.), specifying the security measures put in place to protect the privacy of donors.

- Offer complete traceability of data by enabling individuals and organizations having consented to the (re)use of their data for research to see when and how their data are used, as well as the findings and publications resulting from this work.

## 2. Regulatory tool

The portal would serve as a regulatory compliance tool by integrating key elements related to personal data laws and regulations, including GDPR. This includes:

- Facilitating the collection and management of informed consent, with online monitoring of authorizations and a simple and rapid opt-out option.

- Ensure access to legal information: the portal would display information on donor rights (rights of access, rectification, opposition, etc.) and complaints management procedures.

- Ensure legal transparency on partnerships and collaborations by informing donors of the entities involved and any data transfers, especially outside the EU if they have been the subject of a specific agreement.

## 3. Research agility instrument

In parallel, this transparency portal would be a lever for research agility by enabling more reactive and collaborative management of health data, in particular thanks to:

- Rapid updates to research protocols and regulations, which donors could follow in real time. If there are changes to a project, the participants could be informed and reconfirmed in just a few clicks.

- Open and interactive communication with participants: the portal could include functionalities enabling researchers to seek the opinion or consent of donors for new studies or the wider use of their data.

- Easy access to data for researchers in compliance with security standards, in order to accelerate the implementation of projects without complex administrative timelines, while complying with regulatory obligations.
# APPENDIX 1

## Members of the Working Group (July 2024 - November 2024)

**For the Inserm Ethics Committee:** Henri Atlan, Catherine Bourgain, Hervé Chneiweiss, François Eisinger, Catherine Vidal, Frédérique Lesaulnier, and with the legal contribution of Marion Abecassis

**For the LORIER program:** Philippe Ravaud, Ghislaine Filliatreau, and Ioana Andreescu

**For the Inserm Scientific Advisory Board:** Rodolphe Thiebaut, Isabelle Remy-Jouet, Lotfi Senhadji, Valérie Urbach, and Murielle Gaudry

**External experts:**

- Anita Burgun, University Professor and Hospital Practitioner (PUPH) in Medical Information Technology, Imagine Institute, Paris Cité University, Paris Public Hospitals Group (AP-HP) & Inserm, PRAIRIE Chair.

- Jean Charlet, AP-HP Research Director, LIMICS Sorbonne University & Inserm U1142.

- Pierre-Antoine Gourraud, PUPH in Cell Biology, Nantes University, Inserm Joint Research Unit 1064 & Healthcare Data Warehouse Manager at Nantes University Hospital "Data Clinic". Member of the scientific leadership team of Ouest Data Hub.
# APPENDIX 2

## Recap of the different types of Artificial Intelligence

While Large Language Models (LLMs) are now attracting all the attention due to the very rapid development of their use, it must be remembered that they are only a subcategory of deep neural (Deep Learning) networks, more specifically Transformers. They deserve special mention due to their complexity and broad use in natural language processing. They form part of a wider range of models<fnref n="44" />:

1. **Supervised learning:** The system receives the correct answer and is asked to retrieve it from other datasets

* Neural networks: Used for various tasks such as image and speech recognition. Examples: Multilayer Perceptron (MLP), Convolutional Neural Networks (CNN).

* Support Vector Machines (SVM): Used for classification and regression.

* Decision trees and random forests: Used for classification and regression, aggregating several decision trees to improve accuracy.

2. **Unsupervised learning:** This uses machine learning algorithms to analyze and cluster unlabeled datasets. These algorithms discover hidden patterns or data groupings without the need for human intervention.

* Clustering algorithms: Used to identify structures hidden in data. Examples: K-means, DBSCAN.

* Autoencoding neural networks: Used for dimensionality reduction and detection of abnormalities.

* Principal Component Analysis (PCA): Used for dimensionality reduction and trend identification in data.

3. **Reinforcement learning:** Each response from the machine is assigned a "true" or "false" value

* Q-learning: An algorithm based on tables to find optimal policies.

<footnote n="44">⁴⁴ Section prepared from what ChatGPT 4.0 responds when the prompt (July 2024) asks it to inform us about the main models of artificial intelligence. It should be noted that it omits section 8 of this appendix.</footnote>
* Deep Q-Networks (DQN): An approach using neural networks to approximately learn Q values.

* Policy algorithms: Such as the Proximal Policy Optimization (PPO) algorithm and the Deep Deterministic Policy Gradient (DDPG).

4. Self-supervised learning: Deep Learning

* Convolutional Neural Networks (CNN): Mainly used for image recognition.

* Recurrent Neural Networks (RNN): Used for natural language processing and time series. Examples include Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU).

* Generative Adversarial Network (GAN): Used to generate new data similar to training data, such as images and text.

* Transformers: Used for natural language processing and other tasks. Examples: BERT, GPT, GPT-3, GPT-4 (Large Language Models, LLMs). LLMs are trained on massive corpuses of text and are capable of understanding and generating natural language in a very sophisticated way.

5. Probabilistic models

* Restricted Boltzmann Machines (RBMs): Used for dimensionality reduction, classification, regression and modeling of data.

* Bayesian graphical models: Used for probabilistic modeling of complex data. Examples: Bayesian networks, Hidden Markov models.

6. Evolutionary algorithms

* Genetic algorithms: Used for optimization and search of solutions in large research spaces.

7. Large Language Models (LLMs)

* Transformers: LLMs such as BERT, GPT-3 and GPT-4 are based on the Transformer architecture. They are able to understand and generate human text in a highly advanced way. These models have revolutionized Natural Language Processing (NLP) and are used in various applications ranging from machine translation to text generation.

* LLMs are distinguished by their capacity to process and generate natural language with unprecedented accuracy and fluidity. They are trained on extremely large datasets and
use hundreds of billions of parameters, enabling them to capture a wide range of
linguistic knowledge and contexts.

8. Formal models

* Ontologies: These models organize the knowledge and data of an area of activity in the form of a hierarchical structure. They are, among other things, the basis of decision support systems whose reasoning is explicit and can be combined with other models (LLM or others) to develop "hybrid" AI systems.

* Knowledge graphs (KG): These data- and/or ontology-based models enable information to be structured to make the relationships between different data explicit, thereby facilitating their use in various fields, including information search, artificial intelligence, and data analysis.
# APPENDIX 3

## Article L4001-3 of the French Health Code

### Creation of Law no. 2021-1017 of August 2, 2021 - Art. 17

I.- A healthcare professional who decides to use, for a preventive, diagnostic or care procedure, a medical device with algorithmic data processing whose learning has been performed using big data shall ensure that the data subject has been informed of it and that he or she is, where applicable, informed of the resulting interpretation.

II.- The healthcare professionals concerned are informed of the use of this data processing. The patient data used in this processing and the results obtained from it are accessible to them.

III.- The designers of the algorithmic processing mentioned under point I ensure that its operation can be explained to users.

IV.- A decree of the French Minister of Health establishes, following the opinion of the National Authority for Health (HAS) and the Data Protection Authority (CNIL), the nature of the medical devices mentioned under point I and their methods of use.
# APPENDIX 4

Small Language Models (SLMs) are NLP models with a relatively small number of parameters compared to LLMs. They are designed to be lightweight, fast and easier to deploy on devices with limited resources. They can be adjusted and customized for specific tasks with fewer data and less computing power.

https://www.olvani.com/magazine/actualites-digitales/intelligence-artificielle-explication-des-differents-modeles-de-langages-llm-slm--article.

## Examples of SLMs

### 1. DistilBERT

- Capacity: Around 66 million parameters, 40% less than BERT.

- Use: Language comprehension tasks with limited resources.

- Resources: Available on Hugging Face.

### 2. TinyBERT

- Capacity: Even smaller than DistilBERT, optimized for mobile devices.

- Use: Mobile applications, IoT, where resources are limited.

- Resources: Available on Hugging Face.

### 3. ALBERT (A Lite BERT)

- Capacity: Fewer parameters than BERT, but with optimization techniques to maintain performance.

- Use: NLP tasks with less resources without sacrificing performance.

- Resources: Available on Hugging Face.
<figure><img src="image_4.png" /></figure>

Inserm