# Guidelines for conducting collective scientific assessments and advanced studies to inform public policy and public debate (version 2.1 – October 2023).

Catherine Donnars, Anaïs Tibi, Marc Antoine Caillaud, Regina Dashkina, Agnès Girard, Sophie Leenhardt, Virginie Lelievre, Sophie Le Perchec, Lise Paresys, Isabelle Savini, et al.

<figure><img src="image_1.png" /></figure>

## To cite this version:

Catherine Donnars, Anaïs Tibi, Marc Antoine Caillaud, Regina Dashkina, Agnès Girard, et al. Guidelines for conducting collective scientific assessments and advanced studies to inform public policy and public debate (version 2.1 – October 2023). [0] INRAE DEPE. 2023, 59 p. hal-04461470

HAL Id: hal-04461470

https://hal.inrae.fr/hal-04461470

Submitted on 16 Feb 2024

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>
<figure><img src="image_1.png" /></figure>

INRAE

<figure><img src="image_2.png" /></figure>

# Guidelines for collective scientific assessments and advanced studies at INRAE

## Assessments to inform public policy and debate

Unit for Collective Scientific Assessment, Foresight and Advanced Studies (DEPE)
Version 2.1 - October 2023
<figure><img src="image_3.png" /></figure>

INRAE

<figure><img src="image_4.png" /></figure>

# Guidelines for collective scientific assessments and advanced studies at INRAE

## Assessments to inform public policy and debate

Unit for Collective Scientific Assessment, Foresight and Advanced Studies (DEPE)
Version 2.1 - October 2023
**Authors :**

Catherine Donnars (coord.), Anaïs Tibi (coord.), Marc-Antoine Caillaud, Regina Dashkina, Agnès Girard, Sophie Leenhardt, Virginie Lelièvre, Sophie Le Perchec, Lise Paresys, Isabelle Savini, Guy Richard

**Publication Director:**

Guy Richard (INRAE, director of DEPE)

**Contact :**

DEPE-contact@inrae.fr

**To cite this document:**

INRAE-DEPE (2023). Guidelines for conducting collective scientific assessments and advanced studies to inform public policy and public debate (version 2.1 – October 2023). INRAE (France), 59 p.

DOI of document : https://dx.doi.org/10.17180/6x6d-wn26

This document is based on a previous version published in 2018

https://hal.inrae.fr/hal-03418920 and updated in 2021

https://dx.doi.org/10.15454/tkna-4w25
**Authors :**

Catherine Donnars (coord.), Anaïs Tibi (coord.), Marc-Antoine Caillaud, Regina Dashkina, Agnès Girard, Sophie Leenhardt, Virginie Lelièvre, Sophie Le Perchec, Lise Paresys, Isabelle Savini, Guy Richard

**Publication Director:**

Guy Richard (INRAE, director of DEPE)

**Contact :**

DEPE-contact@inrae.fr

**To cite this document:**

INRAE-DEPE (2023). Guidelines for conducting collective scientific assessments and advanced studies to inform public policy and public debate (version 2.1 – October 2023). INRAE (France), 59 p.

DOI of document : https://dx.doi.org/10.17180/6x6d-wn26

This document is based on a previous version published in 2018

https://hal.inrae.fr/hal-03418920 and updated in 2021

https://dx.doi.org/10.15454/tkna-4w25
<figure><img src="image_3.png" /></figure>

# Guidelines for collective scientific assessments and advanced studies to inform public policy and debate

Unit for Collective Scientific Assessment, Foresight and Advanced Studies (DEPE)
Version 2.1 - October 2023
<figure><img src="image_5.png" /></figure>

# Guidelines for collective scientific assessments and advanced studies to inform public policy and debate

Unit for Collective Scientific Assessment, Foresight and Advanced Studies (DEPE)
Version 2.1 - October 2023
<figure><img src="image_4.png" /></figure>
<figure><img src="image_6.png" /></figure>
<figure><img src="image_5.png" /></figure>

FORWORD

Like any research organisation, the French National Research Institute for Agriculture, Food and Environment (INRAE) has the primary mission of producing scientific knowledge in its fields of expertise: agriculture, the environment, the processing of agricultural products, food and the bioeconomy. Among the many missions resulting from this primary vocation, providing expertise, disseminating scientific culture, and shedding light on public policies and debates are important dimensions of the Institute's activities. Furthermore, society's expectations of INRAE continue to grow as the societal issues raised by INRAE's fields of activity are at the heart of many of the challenges facing contemporary society. The scientific assessments and foresight activities developed by INRA and Irstea since the early 2000s to address these issues are carried out by the Unit for Collective Scientific Expertise, Foresight and Advanced Studies (DEPE), working closely with the executive officer in charge of assessments and support for public policies. The central mission of DEPE is to provide scientific insights to respond to the questions raised by public authorities and society as a whole concerning agriculture and the use of its products, and the environment.

The role of public research organisations in supporting public policy is reflected in various actions, all based on the results of scientific research. Before public policies can be developed and in order to understand the agronomic, biological, environmental, economic and social issues in which public action is intended to intervene, decision-makers and all stakeholders need to be provided with a review of the available scientific knowledge, without overlooking what may be considered as acquired, uncertain, insufficiently documented or incomplete. This is the very principle of the collective scientific assessments conducted by INRAE. Complementary approaches such as advanced studies or foresight studies are intended to deepen or extend these knowledge bases in order to make them even more accessible and usable in the thinking of the various public policy stakeholders. This work requires the definition and implementation of a rigorous methodology shared by all stakeholders in accordance with the National Charter on Institutional Scientific Reports that INRA and Irstea, like their partners in public research, signed in 2010-2011. This methodology aims to ensure the credibility, legitimacy and relevance of the scientific elements provided via this intermediary to the public debate and made available to public and private decision makers.
<figure><img src="image_7.png" /></figure>

FORWORD

Like any research organisation, the French National Research Institute for Agriculture, Food and Environment (INRAE) has the primary mission of producing scientific knowledge in its fields of expertise: agriculture, the environment, the processing of agricultural products, food and the bioeconomy. Among the many missions resulting from this primary vocation, providing expertise, disseminating scientific culture, and shedding light on public policies and debates are important dimensions of the Institute's activities. Furthermore, society's expectations of INRAE continue to grow as the societal issues raised by INRAE's fields of activity are at the heart of many of the challenges facing contemporary society. The scientific assessments and foresight activities developed by INRA and Irstea since the early 2000s to address these issues are carried out by the Unit for Collective Scientific Expertise, Foresight and Advanced Studies (DEPE), working closely with the executive officer in charge of assessments and support for public policies. The central mission of DEPE is to provide scientific insights to respond to the questions raised by public authorities and society as a whole concerning agriculture and the use of its products, and the environment.

The role of public research organisations in supporting public policy is reflected in various actions, all based on the results of scientific research. Before public policies can be developed and in order to understand the agronomic, biological, environmental, economic and social issues in which public action is intended to intervene, decision-makers and all stakeholders need to be provided with a review of the available scientific knowledge, without overlooking what may be considered as acquired, uncertain, insufficiently documented or incomplete. This is the very principle of the collective scientific assessments conducted by INRAE. Complementary approaches such as advanced studies or foresight studies are intended to deepen or extend these knowledge bases in order to make them even more accessible and usable in the thinking of the various public policy stakeholders. This work requires the definition and implementation of a rigorous methodology shared by all stakeholders in accordance with the National Charter on Institutional Scientific Reports that INRA and Irstea, like their partners in public research, signed in 2010-2011. This methodology aims to ensure the credibility, legitimacy and relevance of the scientific elements provided via this intermediary to the public debate and made available to public and private decision makers.
In addition to the Charter on Scientific Expert Reports, DEPE bases its work on a set of procedures, described in the form of precise and detailed files, but whose format and volume make it difficult to publish in their current form. It has been deemed necessary to make the working methods and rules used to conduct these projects available to the various actors and users of the activities carried out by DEPE (sponsors, partners, scientific experts, stakeholders and various public audiences). A first version of this booklet was published in 2018 to publicise the way in which we conduct this type of complex and sensitive work with all possible rigour and impartiality for a constructive relationship between science and society. A number of reflections within DEPE have resulted in the clarification of certain aspects of the conduct of these delicate projects, and these have been integrated into this second version, which now appears as a new publication.

I would like to thank and congratulate the INRAE DEPE team for the quality of the work carried out over the last few years. Through this sharing of our practices, we hope to improve the understanding of and appreciation for the benefits as well as the limitations of the activities we carry out in this domain.

Philippe Mauguin, INRAE CEO
In addition to the Charter on Scientific Expert Reports, DEPE bases its work on a set of procedures, described in the form of precise and detailed files, but whose format and volume make it difficult to publish in their current form. It has been deemed necessary to make the working methods and rules used to conduct these projects available to the various actors and users of the activities carried out by DEPE (sponsors, partners, scientific experts, stakeholders and various public audiences). A first version of this booklet was published in 2018 to publicise the way in which we conduct this type of complex and sensitive work with all possible rigour and impartiality for a constructive relationship between science and society. A number of reflections within DEPE have resulted in the clarification of certain aspects of the conduct of these delicate projects, and these have been integrated into this second version, which now appears as a new publication.

I would like to thank and congratulate the INRAE DEPE team for the quality of the work carried out over the last few years. Through this sharing of our practices, we hope to improve the understanding of and appreciation for the benefits as well as the limitations of the activities we carry out in this domain.

Philippe Mauguin, INRAE CEO
TABLE OF CONTENTS

<table><tr><td>INTRODUCTION</td><td>09</td></tr><tr><td>1. DEVELOPMENT AND MONITORING OF AN ESCO OR A STUDY</td><td>13</td></tr><tr><td>1.1. Receipt of the request by INRAE</td><td>14</td></tr><tr><td>1.1.1. Institutional support of the project by INRAE</td><td>14</td></tr><tr><td>1.1.2. Coordination of the project by DEPE</td><td>14</td></tr><tr><td>1.1.3. Type of project</td><td>15</td></tr><tr><td>1.2. Development of the request</td><td>15</td></tr><tr><td>1.2.1. Developing the project specifications</td><td>15</td></tr><tr><td>1.2.2. Project specifications</td><td>16</td></tr><tr><td>1.3. Drawing up the agreement linking INRAE, its potential partners and the sponsors</td><td>17</td></tr><tr><td>1.4. Establishing monitoring of ESCos and advanced studies</td><td>18</td></tr><tr><td>1.4.1. Monitoring committee</td><td>18</td></tr><tr><td>1.4.2. Stakeholder advisory committee</td><td>19</td></tr><tr><td>2. CONSTITUTION AND ROLES OF ESCO AND ADVANCED STUDY WORKING GROUPS</td><td>20</td></tr><tr><td>2.1. Leadership of an ESCo or advanced study</td><td>20</td></tr><tr><td>2.1.1. Overall coordination: the DEPE project leader</td><td>21</td></tr><tr><td>2.1.2. Scientific coordination: the scientific leads</td><td>21</td></tr><tr><td>2.2. Conducting the ESCo or advanced study: the scientific expert committee</td><td>23</td></tr><tr><td>2.2.1. Identification of experts</td><td>23</td></tr><tr><td>2.2.2. Roles and responsibilities of experts</td><td>25</td></tr><tr><td>2.3. Support roles</td><td>26</td></tr><tr><td>2.4. Technical Committee</td><td>26</td></tr></table>
TABLE OF CONTENTS

<table><tr><td>INTRODUCTION</td><td>09</td></tr><tr><td>1. DEVELOPMENT AND MONITORING OF AN ESCO OR A STUDY</td><td>13</td></tr><tr><td>1.1. Receipt of the request by INRAE</td><td>14</td></tr><tr><td>1.1.1. Institutional support of the project by INRAE</td><td>14</td></tr><tr><td>1.1.2. Coordination of the project by DEPE</td><td>14</td></tr><tr><td>1.1.3. Type of project</td><td>15</td></tr><tr><td>1.2. Development of the request</td><td>15</td></tr><tr><td>1.2.1. Developing the project specifications</td><td>15</td></tr><tr><td>1.2.2. Project specifications</td><td>16</td></tr><tr><td>1.3. Drawing up the agreement linking INRAE, its potential partners and the sponsors</td><td>17</td></tr><tr><td>1.4. Establishing monitoring of ESCOs and advanced studies</td><td>18</td></tr><tr><td>1.4.1. Monitoring committee</td><td>18</td></tr><tr><td>1.4.2. Stakeholder advisory committee</td><td>19</td></tr><tr><td>2. CONSTITUTION AND ROLES OF ESCO AND ADVANCED STUDY WORKING GROUPS</td><td>20</td></tr><tr><td>2.1. Leadership of an ESCo or advanced study</td><td>20</td></tr><tr><td>2.1.1. Overall coordination: the DEPE project leader</td><td>21</td></tr><tr><td>2.1.2. Scientific coordination: the scientific leads</td><td>21</td></tr><tr><td>2.2. Conducting the ESCo or advanced study: the scientific expert committee</td><td>23</td></tr><tr><td>2.2.1. Identification of experts</td><td>23</td></tr><tr><td>2.2.2. Roles and responsibilities of experts</td><td>25</td></tr><tr><td>2.3. Support roles</td><td>26</td></tr><tr><td>2.4. Technical Committee</td><td>26</td></tr></table>
**3. IDENTIFICATION AND ANALYSIS OF THE EXPERTS LINKS OF INTEREST** 28

**3.1. Identifying links of interest** 28

**3.2. Analysing links of interest** 30

3.2.1. Review of individual declarations of interest 30

3.2.2. Analysis of the collective links of interest 30

3.2.3. Diversity of scientific approaches 31

**3.3. A reflective tool for the management of the project** 32

**4. COMPILATION AND ANALYSIS OF DOCUMENTARY SOURCES** 33

**4.1. Documentary sources** 33

4.1.1. Academic literature 33

4.1.2. Technical literature 35

4.1.3. Corpus of statutory texts 35

**4.2. Creation of the bibliographic corpus** 37

4.2.1. Successive steps 37

4.2.2. The main criteria for the selection of references 38

**4.3. Qualitative and quantitative analyses of the bibliographic corpus** 39

4.3.1. A rigorous, common reading grid 39

4.3.2. Use of bibliometric analyses 40

**5. 5. PREPARATION OF DELIVERABLES AND DISSEMINATION OF ESCO RESULTS AND ADVANCED STUDIES** 42

**5.1. Extended report** 42

**5.2. Condensed report** 44

**5.3. Summary report** 46

**5.4. Feedback symposium** 46

**5.5. Academic outputs from ESCos and advanced studies** 47

**5.6. Archiving of documents from ESCos and advanced studies** 47

**6. CONCLUSIONS** 48

**APPENDIX 1. ARRANGEMENTS FOR THE RETENTION AND ARCHIVING OF DOCUMENTS FROM DEPE PROJECTS** 49

**APPENDIX 2. ESCO AND STUDY ACTIVITIES CONDUCTED SINCE THE EARLY 2000S** 50
<table><tbody><tr><td><strong>3. IDENTIFICATION AND ANALYSIS OF THE EXPERTS LINKS OF INTEREST</strong></td><td><strong>28</strong></td></tr><tr><td><strong>3.1. Identifying links of interest</strong></td><td><strong>28</strong></td></tr><tr><td><strong>3.2. Analysing links of interest</strong></td><td><strong>30</strong></td></tr><tr><td>3.2.1. Review of individual declarations of interest</td><td>30</td></tr><tr><td>3.2.2. Analysis of the collective links of interest</td><td>30</td></tr><tr><td>3.2.3. Diversity of scientific approaches</td><td>31</td></tr><tr><td><strong>3.3. A reflective tool for the management of the project</strong></td><td><strong>32</strong></td></tr><tr><td><strong>4. COMPILATION AND ANALYSIS OF DOCUMENTARY SOURCES</strong></td><td><strong>33</strong></td></tr><tr><td><strong>4.1. Documentary sources</strong></td><td><strong>33</strong></td></tr><tr><td>4.1.1. Academic literature</td><td>33</td></tr><tr><td>4.1.2. Technical literature</td><td>35</td></tr><tr><td>4.1.3. Corpus of statutory texts</td><td>35</td></tr><tr><td><strong>4.2. Creation of the bibliographic corpus</strong></td><td><strong>37</strong></td></tr><tr><td>4.2.1. Successive steps</td><td>37</td></tr><tr><td>4.2.2. The main criteria for the selection of references</td><td>38</td></tr><tr><td><strong>4.3. Qualitative and quantitative analyses of the bibliographic corpus</strong></td><td><strong>39</strong></td></tr><tr><td>4.3.1. A rigorous, common reading grid</td><td>39</td></tr><tr><td>4.3.2. Use of bibliometric analyses</td><td>40</td></tr><tr><td><strong>5. PREPARATION OF DELIVERABLES AND DISSEMINATION OF ESCO RESULTS AND ADVANCED STUDIES</strong></td><td><strong>42</strong></td></tr><tr><td><strong>5.1. Extended report</strong></td><td><strong>42</strong></td></tr><tr><td><strong>5.2. Condensed report</strong></td><td><strong>44</strong></td></tr><tr><td><strong>5.3. Summary report</strong></td><td><strong>46</strong></td></tr><tr><td><strong>5.4. Feedback symposium</strong></td><td><strong>46</strong></td></tr><tr><td><strong>5.5. Academic outputs from ESCos and advanced studies</strong></td><td><strong>47</strong></td></tr><tr><td><strong>5.6. Archiving of documents from ESCos and advanced studies</strong></td><td><strong>47</strong></td></tr><tr><td><strong>6. CONCLUSIONS</strong></td><td><strong>48</strong></td></tr><tr><td><strong>APPENDIX 1. ARRANGEMENTS FOR THE RETENTION AND ARCHIVING OF DOCUMENTS FROM DEPE PROJECTS</strong></td><td><strong>49</strong></td></tr><tr><td><strong>APPENDIX 2. ESCO AND STUDY ACTIVITIES CONDUCTED SINCE THE EARLY 2000S</strong></td><td><strong>50</strong></td></tr></tbody></table>
<figure><img src="image_6.png" /></figure>

**Introduction: Collective scientific assessments, forecast studies and advanced studies to inform public policy and debate**

The mission of INRAE's Unit for Collective Scientific Assessment, Foresight and Advanced Studies (DEPE) is to inform public decision making on complex societal issues and, at the same time, to encourage the Institute to reflect on its own scientific directions.

As developed by INRA since 2000, **Collective Scientific Assessments** (known by their French initials ESCo) provide public decision-makers and the broader public with the most exhaustive possible review of validated scientific knowledge in response to a complex question. This question is referred to the Institute, and sometimes jointly to its scientific partners (e.g. CNRS, Cirad, Ifremer, Cemagref), in the form of a referral from one or more ministries, agencies, public or semi-public organisations. The referral generally occurs in advance of the implementation of a public policy (possibility of implementing a dedicated policy, adaptation of a regulation, environmental or public health issues, etc.). A committee of scientific experts conducts a multidisciplinary critical analysis of the international academic literature. The aim is to identify the results of research, the outstanding issues, the uncertainties that are still being studied, and to identify the unresolved controversies and gaps in scientific knowledge. A Collective Scientific Assessment thus constitutes an inventory of the knowledge produced by science and identifies future research needs. As conducted by INRAE, it does not go so far as to formulate recommendations to the sponsors.

Initiated a few years earlier at INRA (in 1993), foresight studies seek to inform decision-making with regard to possible futures by developing and exploring contrasting development scenarios at distant horizons, by identifying potential breaks and hypotheses about the future. To be rigorous and credible, it requires a solid foundation of knowledge in the field to be explored and therefore a good understanding of contemporary scientific knowledge and its uncertainties. These are shared within a working group, which is more of a participatory approach and therefore open to stakeholders during the scenario development phase.

More recently (since 2010), advanced studies have been carried out to overcome some of the limitations of the ESCo, particularly when the available scientific literature is insufficient to provide a precise answer to the question asked. These studies draw on sources of information other than academic literature (reports, articles in technical journals, etc.) and involve the processing of data, most often derived from numerical models. The scientific scope of these studies is generally narrower than that of the ESCo.
<figure><img src="image_8.png" /></figure>

**Introduction: Collective scientific assessments, forecast studies and advanced studies to inform public policy and debate**

The mission of INRAE's Unit for Collective Scientific Assessment, Foresight and Advanced Studies (DEPE) is to inform public decision making on complex societal issues and, at the same time, to encourage the Institute to reflect on its own scientific directions.

As developed by INRA since 2000, **Collective Scientific Assessments** (known by their French initials ESCo) provide public decision-makers and the broader public with the most exhaustive possible review of validated scientific knowledge in response to a complex question. This question is referred to the Institute, and sometimes jointly to its scientific partners (e.g. CNRS, Cirad, Ifremer, Cemagref), in the form of a referral from one or more ministries, agencies, public or semi-public organisations. The referral generally occurs in advance of the implementation of a public policy (possibility of implementing a dedicated policy, adaptation of a regulation, environmental or public health issues, etc.). A committee of scientific experts conducts a multidisciplinary critical analysis of the international academic literature. The aim is to identify the results of research, the outstanding issues, the uncertainties that are still being studied, and to identify the unresolved controversies and gaps in scientific knowledge. A Collective Scientific Assessment thus constitutes an inventory of the knowledge produced by science and identifies future research needs. As conducted by INRAE, it does not go so far as to formulate recommendations to the sponsors.

Initiated a few years earlier at INRA (in 1993), foresight studies seek to inform decision-making with regard to possible futures by developing and exploring contrasting development scenarios at distant horizons, by identifying potential breaks and hypotheses about the future. To be rigorous and credible, it requires a solid foundation of knowledge in the field to be explored and therefore a good understanding of contemporary scientific knowledge and its uncertainties. These are shared within a working group, which is more of a participatory approach and therefore open to stakeholders during the scenario development phase.

More recently (since 2010), advanced studies have been carried out to overcome some of the limitations of the ESCo, particularly when the available scientific literature is insufficient to provide a precise answer to the question asked. These studies draw on sources of information other than academic literature (reports, articles in technical journals, etc.) and involve the processing of data, most often derived from numerical models. The scientific scope of these studies is generally narrower than that of the ESCo.
Public dissemination and debate are essential to the credibility of these assessment activities in the broader sense. This is why, whatever the nature of the project, the results are made public and disseminated as widely as possible in order to inform public decision-making and debate.

While each of these three types of project has its own specific approach, there is a continuum between ESCos, advanced studies and foresight studies. While ESCos always focus on large-scale bibliographic syntheses, they may require the pooling of knowledge from more diverse sources to better contextualise the results acquired and the conclusions to be drawn from results that are otherwise too generic. Two types of study are then positioned as extensions of an ESCo:

* those that integrate data processing with a view to producing original or contextualised indicators from national databases, original simulations carried out using already existing and unmodified models, etc.;

* those which are closer, in terms of their purpose, to the foresight approach and which are based on more or less distant projections of a so-called 'reference' scenario, accompanied by sensitivity analyses via the development of 'alternative' scenarios.

For their part, the most recent foresight studies attempt to combine qualitative scenarios for exploring the future with simulations to quantify the consequences.

In addition, ESCos and advanced studies may propose an in-depth and sometimes quantitative analysis of the assembled bibliographic corpus by means of textual analysis, systematic review or meta-analysis methods.

All projects must comply with the four principles of the National Charter for Expertise: competence of experts, impartiality of outputs, diversity of disciplines and approaches, and transparency of methodologies. This publication focuses on the activities of ESCos and advanced studies, whose methodological principles are similar. Foresight studies are the subject of another publication currently in preparation.

This document presents the key elements, in a condensed format, of a guide to procedures for internal use, drawn up by INRA's DEPE and complying with the main specifications of the NF X50-110 standard. It is intended for all audiences interested in the approaches of ESCos and advanced studies, including public authorities, scientists, professionals, community activists, elected officials, etc. This is a second, updated 2023 version (the first in 2018).
Public dissemination and debate are essential to the credibility of these assessment activities in the broader sense. This is why, whatever the nature of the project, the results are made public and disseminated as widely as possible in order to inform public decision-making and debate.

While each of these three types of project has its own specific approach, there is a continuum between ESCos, advanced studies and foresight studies. While ESCos always focus on large-scale bibliographic syntheses, they may require the pooling of knowledge from more diverse sources to better contextualise the results acquired and the conclusions to be drawn from results that are otherwise too generic. Two types of study are then positioned as extensions of an ESCo:

* those that integrate data processing with a view to producing original or contextualised indicators from national databases, original simulations carried out using already existing and unmodified models, etc.;

* those which are closer, in terms of their purpose, to the foresight approach and which are based on more or less distant projections of a so-called 'reference' scenario, accompanied by sensitivity analyses via the development of 'alternative' scenarios.

For their part, the most recent foresight studies attempt to combine qualitative scenarios for exploring the future with simulations to quantify the consequences.

In addition, ESCos and advanced studies may propose an in-depth and sometimes quantitative analysis of the assembled bibliographic corpus by means of textual analysis, systematic review or meta-analysis methods.

All projects must comply with the four principles of the National Charter for Expertise: competence of experts, impartiality of outputs, diversity of disciplines and approaches, and transparency of methodologies. This publication focuses on the activities of ESCos and advanced studies, whose methodological principles are similar. Foresight studies are the subject of another publication currently in preparation.

This document presents the key elements, in a condensed format, of a guide to procedures for internal use, drawn up by INRA's DEPE and complying with the main specifications of the NF X50-110 standard. It is intended for all audiences interested in the approaches of ESCos and advanced studies, including public authorities, scientists, professionals, community activists, elected officials, etc. This is a second, updated 2023 version (the first in 2018).
Table 1 below shows the overall process of an ESCo or advanced study.

<table><caption>Table 1. Typical process for the execution of an ESCo or advanced study</caption><thead><tr><th>Phases</th><th>Development of the request</th><th>Conducting the assessment</th><th>Exploitation of results</th></tr></thead><tbody><tr><td>Duration</td><td>6 months to 1 year</td><td>1.5 to 2 years</td><td>6 months to 2 years</td></tr><tr><td>Referral and monitoring of the assessment</td><td>Drafting of project specifications Agreement</td><td>Follow-up with sponsors Consultation with stakeholders</td><td rowspan="3">Release of the extended report, condensed report for decision makers and summary report Public symposium Conferences Scientific publications</td></tr><tr><td>Work of the expert committee</td><td>Choice of scientific leads and experts, analysis of links of interest</td><td>Work of the expert committee alternating between reading, writing and meetings</td></tr><tr><td>Bibliography</td><td>Identification of thematic fields and key authors</td><td>Creation of the bibliographic corpus Adjustments and additional research Analysis of the final corpus</td></tr></tbody></table>

The process of examining the ESCo or study project is described in Chapter 1. It begins with the receipt of the request and gives rise to numerous exchanges and discussions within the Institute, with the initial and potential sponsors, and with research partners. The question arises as to INRAE's institutional and scientific positioning with respect to the request, its legitimacy to lead the project, and its scientific interest in the project. The challenge is to choose the most appropriate type of study (between ESCo, advanced study, or even foresight study) and the most appropriate form of management (DEPE, scientific directorate, department, etc.). Depending on the case, INRAE will accept or reject a project, propose reformulations or restrictions on the questions asked, broaden the request, suggest co-management or a scientific partnership, etc.

Two committees are set up during the appraisal phase to support the working group in charge of conducting the ESCo or advanced study: the monitoring committee enables the sponsors to follow the work's progress, and the stakeholder advisory committee informs the actors and stakeholders of the project's progress, as well as gathering their perspectives on the issues associated with the problem under examination. These are essential forums for exchange, which can help to identify useful work (particularly unreferenced reports) or data, and facilitate access to them.

Chapter 2 presents the rationale for setting up the working group. A multidisciplinary committee of scientific experts, specialised in the subject and chosen on the basis of their skills as demonstrated by their publications, is formed. It is led and coordinated by one or more scientific leads and by a DEPE project manager. It also relies on project management and documentary research skills. One of the challenges of leading the committee of experts is to build a shared vision of the questions posed by the
Table 1 below shows the overall process of an ESCo or advanced study.

<table><caption>Table 1. Typical process for the execution of an ESCo or advanced study</caption><thead><tr><th>Phases</th><th>Development of the request</th><th>Conducting the assessment</th><th>Exploitation of results</th></tr></thead><tbody><tr><td>Duration</td><td>6 months to 1 year</td><td>1.5 to 2 years</td><td>6 months to 2 years</td></tr><tr><td>Referral and monitoring of the assessment</td><td>Drafting of project specifications Agreement</td><td>Follow-up with sponsors Consultation with stakeholders</td><td rowspan="3">Release of the extended report, condensed report for decision makers and summary report Public symposium Conferences Scientific publications</td></tr><tr><td>Work of the expert committee</td><td>Choice of scientific leads and experts, analysis of links of interest</td><td>Work of the expert committee alternating between reading, writing and meetings</td></tr><tr><td>Bibliography</td><td>Identification of thematic fields and key authors</td><td>Creation of the bibliographic corpus Adjustments and additional research Analysis of the final corpus</td></tr></tbody></table>

The process of examining the ESCo or study project is described in Chapter 1. It begins with the receipt of the request and gives rise to numerous exchanges and discussions within the Institute, with the initial and potential sponsors, and with research partners. The question arises as to INRAE's institutional and scientific positioning with respect to the request, its legitimacy to lead the project, and its scientific interest in the project. The challenge is to choose the most appropriate type of study (between ESCo, advanced study, or even foresight study) and the most appropriate form of management (DEPE, scientific directorate, department, etc.). Depending on the case, INRAE will accept or reject a project, propose reformulations or restrictions on the questions asked, broaden the request, suggest co-management or a scientific partnership, etc.

Two committees are set up during the appraisal phase to support the working group in charge of conducting the ESCo or advanced study: the monitoring committee enables the sponsors to follow the work's progress, and the stakeholder advisory committee informs the actors and stakeholders of the project's progress, as well as gathering their perspectives on the issues associated with the problem under examination. These are essential forums for exchange, which can help to identify useful work (particularly unreferenced reports) or data, and facilitate access to them.

Chapter 2 presents the rationale for setting up the working group. A multidisciplinary committee of scientific experts, specialised in the subject and chosen on the basis of their skills as demonstrated by their publications, is formed. It is led and coordinated by one or more scientific leads and by a DEPE project manager. It also relies on project management and documentary research skills. One of the challenges of leading the committee of experts is to build a shared vision of the questions posed by the
sponsors, despite the divergent points of view and arguments that are specific to each scientific discipline. The findings and disagreements must be presented and explained as a result of the ESCo or the study.

Chapter 3 explains the procedures for collecting and analysing links of interest between experts, in accordance with the principles of transparency and impartiality. This includes identifying links of interest at the individual level and analysing links of interest at the level of the expert group.

Chapter 4 deals with the principles for developing and using the bibliographic corpus. The process of collecting, sorting and selecting references, from which the committee of experts will extract the most relevant ones in order to answer the question posed by the sponsors, determines the robustness of the project. Settling on a clear and transparent working strategy is crucial in view of the exponential growth in the number of scientific articles listed in bibliographic databases.

Each project leads to the production of three types of document, detailed in Chapter 5: an extended report (of several hundred pages) which brings together all the critical analyses written by the experts on the basis of the sorted and selected bibliographic corpus; a condensed report (around one hundred pages) intended for decision-makers and, more broadly, for all stakeholders in society concerned and/or interested in the issue (associations, professional organisations, etc.); and a summary report (around ten pages) which communicates the major conclusions of the work to a wider audience. These documents are disseminated at a public feedback symposium, which allows for discussion of the conclusions with stakeholders. The results of the work are also intended to be published in national and international scientific journals to promote their dissemination within the scientific community.

All ESCos and advanced studies conducted by INRA (and later INRAE) since the early 2000s are presented in *Appendix 2*.
sponsors, despite the divergent points of view and arguments that are specific to each scientific discipline. The findings and disagreements must be presented and explained as a result of the ESCo or the study.

Chapter 3 explains the procedures for collecting and analysing links of interest between experts, in accordance with the principles of transparency and impartiality. This includes identifying links of interest at the individual level and analysing links of interest at the level of the expert group.

Chapter 4 deals with the principles for developing and using the bibliographic corpus. The process of collecting, sorting and selecting references, from which the committee of experts will extract the most relevant ones in order to answer the question posed by the sponsors, determines the robustness of the project. Settling on a clear and transparent working strategy is crucial in view of the exponential growth in the number of scientific articles listed in bibliographic databases.

Each project leads to the production of three types of document, detailed in Chapter 5: an extended report (of several hundred pages) which brings together all the critical analyses written by the experts on the basis of the sorted and selected bibliographic corpus; a condensed report (around one hundred pages) intended for decision-makers and, more broadly, for all stakeholders in society concerned and/or interested in the issue (associations, professional organisations, etc.); and a summary report (around ten pages) which communicates the major conclusions of the work to a wider audience. These documents are disseminated at a public feedback symposium, which allows for discussion of the conclusions with stakeholders. The results of the work are also intended to be published in national and international scientific journals to promote their dissemination within the scientific community.

All ESCos and advanced studies conducted by INRA (and later INRAE) since the early 2000s are presented in *Appendix 2*.
<figure><img src="image_7.png" /></figure>

# CHAPTER 1. DEVELOPMENT AND MONITORING OF AN ESCO OR ADVANCED STUDY

ESCos and advanced studies are conducted in response to external requests. This demand mainly comes from public bodies (ministry divisions, agencies such as ADEME or OFB), but may also come from private bodies (technical institutes, professional associations) or those recognised as being in the public interest if the question raised concerns public policies.

ESCos and advanced studies may be conducted in partnership with one or more other public organisations with scientific expertise: research organisations, and sometimes agencies whose missions include research.

The sponsors finance the operating costs of the project<fnref n="1" />; the salaries of the DEPE staff and the experts mobilised remain the responsibility of INRAE or their parent organisation<fnref n="2" />. ESCos and advanced studies conducted by DEPE therefore differ from (full-cost) services that can be commissioned from a consultancy firm. The scope of work for an ESCo or an advanced study is developed jointly by the commissioning parties and INRAE. The results of the ESCo and the studies remain the property of the Institute, and the sponsors cannot restrict their dissemination, i.e. free access to all documents resulting from these activities (Chapter 5).

The project appraisal and development phase involves discussions with the sponsors, leading to the drafting of the ESCo or advanced study terms of reference and the signing of the agreement between INRAE and its potential partners and the external sponsors. The project officially starts on the date the agreement is signed.

The appraisal first consists of defining the Institute's position with regard to the request (institutional and scientific support), identifying the most appropriate format to respond to the request and the way in which the project will be conducted (section 1.1). It continues with the drafting of the specifications (section 1.2) and the agreement (section 1.3), and the setting up of the committees that will accompany the project (section 1.4). Sections 1.2 to 1.4 apply to cases where DEPE is designated to coordinate the project.

The entire process, which takes six to twelve months, entails many discussions and exchanges between the receipt of the request and the signing of the agreement. The terms of reference, which are co-constructed and validated by INRAE, its potential partner(s) and the sponsor(s), are attached to the agreement.

<footnote n="1">¹Costs related to staging meetings, publishing documents, organising public feedback symposium and, where applicable, the remuneration of a project manager contracted for the project.</footnote>

<footnote n="2">²As a result, the funding provided by the sponsors generally represents a maximum of around 30% of the total cost of the project.</footnote>
<figure><img src="image_9.png" /></figure>

# CHAPTER 1. DEVELOPMENT AND MONITORING OF AN ESCO OR ADVANCED STUDY

ESCos and advanced studies are conducted in response to external requests. This demand mainly comes from public bodies (ministry divisions, agencies such as ADEME or OFB), but may also come from private bodies (technical institutes, professional associations) or those recognised as being in the public interest if the question raised concerns public policies.

ESCos and advanced studies may be conducted in partnership with one or more other public organisations with scientific expertise: research organisations, and sometimes agencies whose missions include research.

The sponsors finance the operating costs of the project<fnref n="1" />; the salaries of the DEPE staff and the experts mobilised remain the responsibility of INRAE or their parent organisation<fnref n="2" />. ESCos and advanced studies conducted by DEPE therefore differ from (full-cost) services that can be commissioned from a consultancy firm. The scope of work for an ESCo or an advanced study is developed jointly by the commissioning parties and INRAE. The results of the ESCo and the studies remain the property of the Institute, and the sponsors cannot restrict their dissemination, i.e. free access to all documents resulting from these activities (Chapter 5).

The project appraisal and development phase involves discussions with the sponsors, leading to the drafting of the ESCo or advanced study terms of reference and the signing of the agreement between INRAE and its potential partners and the external sponsors. The project officially starts on the date the agreement is signed.

The appraisal first consists of defining the Institute's position with regard to the request (institutional and scientific support), identifying the most appropriate format to respond to the request and the way in which the project will be conducted (section 1.1). It continues with the drafting of the specifications (section 1.2) and the agreement (section 1.3), and the setting up of the committees that will accompany the project (section 1.4). Sections 1.2 to 1.4 apply to cases where DEPE is designated to coordinate the project.

The entire process, which takes six to twelve months, entails many discussions and exchanges between the receipt of the request and the signing of the agreement. The terms of reference, which are co-constructed and validated by INRAE, its potential partner(s) and the sponsor(s), are attached to the agreement.

<footnote n="1">¹Costs related to staging meetings, publishing documents, organising public feedback symposium and, where applicable, the remuneration of a project manager contracted for the project.</footnote>

<footnote n="2">²As a result, the funding provided by the sponsors generally represents a maximum of around 30% of the total cost of the project.</footnote>
## 1.1. Receipt of the request by INRAE

### 1.1.1. INRAE's institutional support for the project

Upon receipt of a request to the Institute, an initial phase of reflection is initiated to clarify the subject and identify the interest and legitimacy for INRAE to carry out the project at the institutional level. The Director of DEPE analyses the request with INRAE scientific Directors and, where appropriate, with the representatives of other research organisations involved. The following reasons may lead the INRAE general direction to decline the project:

* The subject is already being addressed by another body or organisation, and/or does not fall directly within INRAE's scientific fields of competence.

* The sponsor(s) and INRAE (and any other organisations involved) do not reach a satisfactory agreement on the referral;

* The subject involves questions that have not yet been fully explored by the scientific community. The available scientific knowledge is not sufficiently complete or robust to be the subject of a relevant synthesis.

* The request is of scientific interest but of little interest to society as a whole (the issues and concerns underlying the request are not motivated by the general interest and/or are not the subject of a public policy or debate to which an ESCo or advanced study would contribute).

### 1.1.2. Coordination of the project by DEPE

When INRAE assumes the institutional and scientific leadership of a project, coordination is entrusted to DEPE in the following cases:

* The nature of the request is to shed light on public policies and debates on the basis of syntheses of knowledge, and not on the basis of a collection of expert opinions;

* The questions are multidisciplinary, involving both the biological and social sciences;

* The academic scientific literature is sufficiently extensive for its analysis to provide an answer to the questions posed.
# 1.1. Receipt of the request by INRAE

## 1.1.1. INRAE's institutional support for the project

Upon receipt of a request to the Institute, an initial phase of reflection is initiated to clarify the subject and identify the interest and legitimacy for INRAE to carry out the project at the institutional level. The Director of DEPE analyses the request with INRAE scientific Directors and, where appropriate, with the representatives of other research organisations involved. The following reasons may lead the INRAE general direction to decline the project:

* The subject is already being addressed by another body or organisation, and/or does not fall directly within INRAE's scientific fields of competence.

* The sponsor(s) and INRAE (and any other organisations involved) do not reach a satisfactory agreement on the referral;

* The subject involves questions that have not yet been fully explored by the scientific community. The available scientific knowledge is not sufficiently complete or robust to be the subject of a relevant synthesis.

* The request is of scientific interest but of little interest to society as a whole (the issues and concerns underlying the request are not motivated by the general interest and/or are not the subject of a public policy or debate to which an ESCo or advanced study would contribute).

## 1.1.2. Coordination of the project by DEPE

When INRAE assumes the institutional and scientific leadership of a project, coordination is entrusted to DEPE in the following cases:

* The nature of the request is to shed light on public policies and debates on the basis of syntheses of knowledge, and not on the basis of a collection of expert opinions;

* The questions are multidisciplinary, involving both the biological and social sciences;

* The academic scientific literature is sufficiently extensive for its analysis to provide an answer to the questions posed.
### 1.1.3. Type of project

It is then necessary to define, a priori, the most appropriate form of response (ESCo, advanced study, or even foresight study) to the request. The decision is made on the basis of the following factors:

* whether the scope of the question is rather broad (ESCo) or, conversely, more focused (advanced study);

* the coverage of the subject by the academic literature (ESCo) or whether there is a need for extensive use of grey literature (advanced study);

* there is an explicit request for the creation of new data (advanced study), which requires simulations, statistical analyses, etc.

* there is an explicit request to make projections based on hypotheses of the evolution of the system under study (advanced study).

## 1.2. Development of the request

### 1.2.1. Developing the project specifications

Once INRAE has decided to respond favourably to the request, a second phase of exchanges with the sponsors is undertaken with the aim of refining the scope of the request and translating it into scientific questions. The outcome of these exchanges is formalised in a set of project specifications.

The exchanges are coordinated by the Director of DEPE. They involve the INRAE Scientific Director who will monitor the project, the representatives of the other institutes involved (if any), and the sponsors. When they are identified, the project leader and the lead scientists also participate<fnref n="3" />.

This development phase is supported by an initial diagnosis of the available bibliographic material<fnref n="4" />. This entails an assessment of the number of existing articles in order to identify:

* questions for which there is an abundance of literature, which may need to be clarified, reframed and circumscribed;

* questions on which the literature is limited and which, for lack of sufficient sources, should possibly be excluded from the scope of the project from the outset.

The exploration of the bibliography also aims to identify scientists who publish on the themes covered in the project specifications. These potential experts could be called upon to participate in the project (see Chapter 2).

<footnote n="3">³ Reflections aimed at identifying potential scientific leads can begin quite quickly, as soon as the decision is made to have INRAE lead the project at an institutional level.</footnote>

<footnote n="4">⁴ Mainly in the Web of Science or Scopus, and if necessary in databases specific to certain disciplines.</footnote>
### 1.1.3. Type of project

It is then necessary to define, a priori, the most appropriate form of response (ESCo, advanced study, or even foresight study) to the request. The decision is made on the basis of the following factors:

* whether the scope of the question is rather broad (ESCo) or, conversely, more focused (advanced study);

* the coverage of the subject by the academic literature (ESCo) or whether there is a need for extensive use of grey literature (advanced study);

* there is an explicit request for the creation of new data (advanced study), which requires simulations, statistical analyses, etc.

* there is an explicit request to make projections based on hypotheses of the evolution of the system under study (advanced study).

## 1.2. Development of the request

### 1.2.1. Developing the project specifications

Once INRAE has decided to respond favourably to the request, a second phase of exchanges with the sponsors is undertaken with the aim of refining the scope of the request and translating it into scientific questions. The outcome of these exchanges is formalised in a set of project specifications.

The exchanges are coordinated by the Director of DEPE. They involve the INRAE Scientific Director who will monitor the project, the representatives of the other institutes involved (if any), and the sponsors. When they are identified, the project leader and the lead scientists also participate<fnref n="3" />.

This development phase is supported by an initial diagnosis of the available bibliographic material<fnref n="4" />. This entails an assessment of the number of existing articles in order to identify:

* questions for which there is an abundance of literature, which may need to be clarified, reframed and circumscribed;

* questions on which the literature is limited and which, for lack of sufficient sources, should possibly be excluded from the scope of the project from the outset.

The exploration of the bibliography also aims to identify scientists who publish on the themes covered in the project specifications. These potential experts could be called upon to participate in the project (see Chapter 2).

<footnote n="3">³ Reflections aimed at identifying potential scientific leads can begin quite quickly, as soon as the decision is made to have INRAE lead the project at an institutional level.</footnote>

<footnote n="4">⁴ Mainly in the Web of Science or Scopus, and if necessary in databases specific to certain disciplines.</footnote>
In the more specific case of advanced studies, this development phase must also include an initial assessment of the data available as well as the suitability of various processing or simulation methods and tools required for the work to be carried out.

### 1.2.2. Project specifications

The specifications consist of a document of a few pages drafted jointly by INRAE, its possible partners and the sponsors. The DEPE director coordinates the drafting with the help of the project leader and the scientific leads (if they are identified). The specifications must include:

* the societal, political and regulatory context of the request and the associated issues. e.g. Why has the request been made? How do the sponsors intend to use the results? etc.

* the most precise possible description of the purpose and scope of the project (what is included and excluded from the project);

* all questions asked by the sponsors, to be examined subject to the existence of scientific literature<fnref n="5" />;

* a provisional schedule showing the duration of the various phases of the project

* a provisional budget;

* the arrangements for monitoring the project (see section 1.4).

These specifications have the status of a 'scoping paper'. At the time of the agreement's signing, they would not have been submitted to the committee of experts since the latter would not yet have been formed. This special status is a compromise that makes it possible to:

* have a stable concept, and therefore start on the basis of an initial document shared by INRAE, its potential partners and the sponsors;

* to have a sufficiently precise formulation of the questions, to be able to identify the disciplinary skills to be sought; and

* guarantee a certain margin of flexibility for the expert group, whose first task will be to draw up a refined diagnosis of the feasibility of addressing each of the questions posed.

Two pitfalls should be avoided when drawing up the project specifications:

* When the objective of the project is to examine the potential impacts of a phenomenon, the characteristics of the impacts to be included in the scope of the project must be defined. Indeed, an exhaustive list of impacts that might be worthy of analysis according to their nature (environmental, social, agronomic, health, etc.) and their targets (environmental compartments, organisms, etc.) can be a source of misunderstanding when the expert committee is faced with the material impossibility of taking on all the questions posed.

<footnote n="5">⁵The bibliographical exploration carried out at the time of this appraisal phase only provides an overview of the available literature; it does not guarantee the existence of literature on all the issues of concern to the sponsors.</footnote>
In the more specific case of advanced studies, this development phase must also include an initial assessment of the data available as well as the suitability of various processing or simulation methods and tools required for the work to be carried out.

### 1.2.2. Project specifications

The specifications consist of a document of a few pages drafted jointly by INRAE, its possible partners and the sponsors. The DEPE director coordinates the drafting with the help of the project leader and the scientific leads (if they are identified). The specifications must include:

* the societal, political and regulatory context of the request and the associated issues. e.g. Why has the request been made? How do the sponsors intend to use the results? etc.

* the most precise possible description of the purpose and scope of the project (what is included and excluded from the project);

* all questions asked by the sponsors, to be examined subject to the existence of scientific literature<fnref n="5" />;

* a provisional schedule showing the duration of the various phases of the project

* a provisional budget;

* the arrangements for monitoring the project (see section 1.4).

These specifications have the status of a 'scoping paper'. At the time of the agreement's signing, they would not have been submitted to the committee of experts since the latter would not yet have been formed. This special status is a compromise that makes it possible to:

* have a stable concept, and therefore start on the basis of an initial document shared by INRAE, its potential partners and the sponsors;

* to have a sufficiently precise formulation of the questions, to be able to identify the disciplinary skills to be sought; and

* guarantee a certain margin of flexibility for the expert group, whose first task will be to draw up a refined diagnosis of the feasibility of addressing each of the questions posed.

Two pitfalls should be avoided when drawing up the project specifications:

* When the objective of the project is to examine the potential impacts of a phenomenon, the characteristics of the impacts to be included in the scope of the project must be defined. Indeed, an exhaustive list of impacts that might be worthy of analysis according to their nature (environmental, social, agronomic, health, etc.) and their targets (environmental compartments, organisms, etc.) can be a source of misunderstanding when the expert committee is faced with the material impossibility of taking on all the questions posed.

<footnote n="5">⁵The bibliographical exploration carried out at the time of this appraisal phase only provides an overview of the available literature; it does not guarantee the existence of literature on all the issues of concern to the sponsors.</footnote>
* While it is legitimate for public authorities and stakeholders to want to have a cost-benefit analysis of a set of phenomena with potentially antagonistic effects, such an analysis can only be carried out if it has been the subject of specific scientific work that has already been published.

**1.3. Drawing up the agreement linking INRAE, its potential partners and the sponsors**

ESCos and advanced studies are subject to an agreement between the sponsor(s), INRAE and its possible partners. The agreement defines the purpose of the request made to the Institute, the terms and conditions for conducting the project and the funding allocated by the sponsors to INRAE (and its possible partners) to cover the project's operating budget. The signature of the agreement marks the end of the development phase and the start of the project.

The main points to consider concerning the content of the agreement are:

* The implementation period. This varies from 18 to 36 months from the date of signing and includes further value-adding activities following the feedback symposium (see Chapter 4).

* Delivery of intermediate outputs: some payments may be conditional on the delivery of outputs, for example, a progress report. The nature of these intermediate deliverables and their status must be specified in the agreement. They may not concern the preliminary results of the project and may not be distributed beyond the sponsors.

* The nature of the relationship between the signatories of the agreement: it does not constitute a commercial relationship, and the various deliverables from the project are public and not the property of the sponsors (see Chapter 4).

* The status of the deliverables<fnref n="6" />: the leads and scientific experts are responsible for writing the scientific report, while DEPE is responsible for writing the condensed report and the summary report. The condensed report is presented to the sponsors in an almost final draft for verification of its adequacy in relation to the project specifications, but is not subject to their approval in any way. The summary report is also sent to the sponsors for their opinions and suggestions before final validation by the CEO of INRAE and those of its eventual partners.

* Monitoring the progress of the project: the role of the committees on which the sponsors are represented is explicitly specified in the agreement (see Section 1.4).

<footnote n="6">⁶ Expertise is provided by INRAE and any partner institutions.</footnote>
* While it is legitimate for public authorities and stakeholders to want to have a cost-benefit analysis of a set of phenomena with potentially antagonistic effects, such an analysis can only be carried out if it has been the subject of specific scientific work that has already been published.

**1.3. Drawing up the agreement linking INRAE, its potential partners and the sponsors**

ESCos and advanced studies are subject to an agreement between the sponsor(s), INRAE and its possible partners. The agreement defines the purpose of the request made to the Institute, the terms and conditions for conducting the project and the funding allocated by the sponsors to INRAE (and its possible partners) to cover the project's operating budget. The signature of the agreement marks the end of the development phase and the start of the project.

The main points to consider concerning the content of the agreement are:

* The implementation period. This varies from 18 to 36 months from the date of signing and includes further value-adding activities following the feedback symposium (see Chapter 4).

* Delivery of intermediate outputs: some payments may be conditional on the delivery of outputs, for example, a progress report. The nature of these intermediate deliverables and their status must be specified in the agreement. They may not concern the preliminary results of the project and may not be distributed beyond the sponsors.

* The nature of the relationship between the signatories of the agreement: it does not constitute a commercial relationship, and the various deliverables from the project are public and not the property of the sponsors (see Chapter 4).

* The status of the deliverables<fnref n="6" />: the leads and scientific experts are responsible for writing the scientific report, while DEPE is responsible for writing the condensed report and the summary report. The condensed report is presented to the sponsors in an almost final draft for verification of its adequacy in relation to the project specifications, but is not subject to their approval in any way. The summary report is also sent to the sponsors for their opinions and suggestions before final validation by the CEO of INRAE and those of its eventual partners.

* Monitoring the progress of the project: the role of the committees on which the sponsors are represented is explicitly specified in the agreement (see Section 1.4).

<footnote n="6">⁶ Expertise is provided by INRAE and any partner institutions.</footnote>
## 1.4. Establishing the monitoring of ESCos and advanced studies: the monitoring committee and the stakeholder advisory committee

Two committees are established to monitor the progress of the project for the sponsors and to inform the socio-economic stakeholders of progress in the project. These discussion forums can also help to identify additional issues as well as studies (particularly unreferenced study reports) and useful data, and to facilitate access to them.

### 1.4.1. Monitoring committee

The monitoring committee provides the interface between the working group (see Chapter 2) and the sponsors. It is informed of the progress of the project and of any difficulties that the expert committee may encounter, advises the latter of changes in the political and regulatory context in which the application is made, and takes part in the discussion of the results.

The monitoring committee is composed of representatives of the sponsors, the INRAE executive management as the lead institute (the relevant scientific management and the DEPE director) and the executive management of the potential partners. This committee is set up as soon as the activities begin. Its meetings are chaired by the director of DEPE and prepared with the support of the project leader. It usually meets three times during the course of an ESCo or advanced study:

* When the agreement is signed, the committee discusses and validates the project specifications (see Section 1.2.2).

* During the course of the project, the leads and the overall project manager present the progress of the work and any provisional elements. These are not intermediate results, but rather aspects for reflection shared by the working group, which structure the work in progress and may lead to a slight reorientation. The leads and the project manager also report any difficulties that may prompt the monitoring committee to clarify or change certain parts of the questions listed in the project specifications.

* Shortly before the public release of the results, near-final versions of the condensed report and the summary report are sent to the members of the monitoring committee for review and comment. The monitoring committee also validates the provisional programme of the feedback symposium on the basis of a proposal drafted by DEPE (see Chapter 4).

The monitoring committee does not validate the content of the ESCo's deliverables or the advanced study. It is consulted for its opinion on the condensed report and the summary report. Its feedback should focus on the clarity of the elements presented
## 1.4. Establishing the monitoring of ESCos and advanced studies: the monitoring committee and the stakeholder advisory committee

Two committees are established to monitor the progress of the project for the sponsors and to inform the socio-economic stakeholders of progress in the project. These discussion forums can also help to identify additional issues as well as studies (particularly unreferenced study reports) and useful data, and to facilitate access to them.

### 1.4.1. Monitoring committee

The monitoring committee provides the interface between the working group (see Chapter 2) and the sponsors. It is informed of the progress of the project and of any difficulties that the expert committee may encounter, advises the latter of changes in the political and regulatory context in which the application is made, and takes part in the discussion of the results.

The monitoring committee is composed of representatives of the sponsors, the INRAE executive management as the lead institute (the relevant scientific management and the DEPE director) and the executive management of the potential partners. This committee is set up as soon as the activities begin. Its meetings are chaired by the director of DEPE and prepared with the support of the project leader. It usually meets three times during the course of an ESCo or advanced study:

* When the agreement is signed, the committee discusses and validates the project specifications (see Section 1.2.2).

* During the course of the project, the leads and the overall project manager present the progress of the work and any provisional elements. These are not intermediate results, but rather aspects for reflection shared by the working group, which structure the work in progress and may lead to a slight reorientation. The leads and the project manager also report any difficulties that may prompt the monitoring committee to clarify or change certain parts of the questions listed in the project specifications.

* Shortly before the public release of the results, near-final versions of the condensed report and the summary report are sent to the members of the monitoring committee for review and comment. The monitoring committee also validates the provisional programme of the feedback symposium on the basis of a proposal drafted by DEPE (see Chapter 4).

The monitoring committee does not validate the content of the ESCo's deliverables or the advanced study. It is consulted for its opinion on the condensed report and the summary report. Its feedback should focus on the clarity of the elements presented
and their relevance to the request. The working group remains responsible for
following up on the comments and suggestions made.

The monitoring committee applies strict confidentiality rules until the results are
published: any document received (draft condensed reports or summary reports,
minutes, etc.) cannot be circulated beyond the members who meet in session.

### 1.4.2. Stakeholder advisory committee

The Stakeholder Advisory Committee is the framework within which stakeholders are consulted about the direction and conclusions of the ESCo or advanced study. It is the forum for expressing stakeholders' concerns and questions about the project.

The composition of the stakeholder advisory committee is proposed by the DEPE director and validated by the monitoring committee (the members of the monitoring committee are themselves members). The stakeholder advisory committee brings together representatives of all stakeholders likely to be interested in the conclusions of the project and to use the results, such as ministry departments interested in the ESCo or the study without being the sponsor, French or European agencies, environmental or consumer associations, local authorities, professional organisations, economic stakeholders in the agri-food sector, scientific interest groups, etc. Each member of the stakeholder advisory committee participates as a representative of the organisation to which he/she belongs and not in a personal capacity.

Like the monitoring committee, the stakeholder advisory committee meetings are facilitated by the DEPE director and prepared with the project leader. The stakeholder advisory committee meets at least twice:

* at the time of the launch of the project, for a presentation to the participants in the demand, involving the issues, the organisation and planning of the project, as well as the framing elements resulting from the reflections of the working group. The objective of this session is twofold: to inform the stakeholders of the project and the methodology on the one hand; and on the other hand to gather their opinions with regard to their respective issues.

* at the end of the project, between the final meeting of the monitoring committee and the feedback symposium, for a presentation of the major conclusions of the project. This session allows the results to be shared and the initial reactions of the stakeholders to be collected, which is often useful for preparing the symposium (see Chapter 4).

A third meeting can be organised in the middle of the project to report on the progress of the work.
and their relevance to the request. The working group remains responsible for
following up on the comments and suggestions made.

The monitoring committee applies strict confidentiality rules until the results are
published: any document received (draft condensed reports or summary reports,
minutes, etc.) cannot be circulated beyond the members who meet in session.

### 1.4.2. Stakeholder advisory committee

The Stakeholder Advisory Committee is the framework within which stakeholders are consulted about the direction and conclusions of the ESCo or advanced study. It is the forum for expressing stakeholders' concerns and questions about the project.

The composition of the stakeholder advisory committee is proposed by the DEPE director and validated by the monitoring committee (the members of the monitoring committee are themselves members). The stakeholder advisory committee brings together representatives of all stakeholders likely to be interested in the conclusions of the project and to use the results, such as ministry departments interested in the ESCo or the study without being the sponsor, French or European agencies, environmental or consumer associations, local authorities, professional organisations, economic stakeholders in the agri-food sector, scientific interest groups, etc. Each member of the stakeholder advisory committee participates as a representative of the organisation to which he/she belongs and not in a personal capacity.

Like the monitoring committee, the stakeholder advisory committee meetings are facilitated by the DEPE director and prepared with the project leader. The stakeholder advisory committee meets at least twice:

* at the time of the launch of the project, for a presentation to the participants in the demand, involving the issues, the organisation and planning of the project, as well as the framing elements resulting from the reflections of the working group. The objective of this session is twofold: to inform the stakeholders of the project and the methodology on the one hand; and on the other hand to gather their opinions with regard to their respective issues.

* at the end of the project, between the final meeting of the monitoring committee and the feedback symposium, for a presentation of the major conclusions of the project. This session allows the results to be shared and the initial reactions of the stakeholders to be collected, which is often useful for preparing the symposium (see Chapter 4).

A third meeting can be organised in the middle of the project to report on the progress of the work.
<figure><img src="image_8.png" /></figure>

## CHAPTER 2. CONSTITUTION AND ROLES OF ESCO AND ADVANCED STUDY
### WORKING GROUPS

The working group of an ESCo or advanced study relies on a multidisciplinary committee of 20 to 40 scientific experts specialised in the subject. These experts identify and collect, with the help of information officers, the relevant international academic literature and extract elements that shed light on the questions raised by the sponsors. In the case of advanced studies, they complete this bibliographic synthesis by processing and assembling the data. Finally, they collectively write the scientific report of the project.

The work of the expert committee is co-led by a project leader from DEPE and one or more scientific leads. The leads set the scientific directions of the project, lead the collective production of results, ensure the use of available bibliographic material and knowledge, and draw up the general conclusions. They are responsible for producing the scientific report (see Chapter 5). The project leader is responsible for the overall coordination of the various stages of the project in accordance with the deadlines set and the methods and procedures established by DEPE. He/she is also responsible for the production of the condensed report and the summary report (see Chapter 5).

The project leader, the scientific leads and the expert committee are supported by information officers, a person responsible for the logistical and financial management of the project, and, depending on the project, one or more project officers (support for coordination, carrying out simulations and data analysis, analysis of technical documents, etc.).

If the analysis requires it, a technical committee can be set up to provide additional non-academic knowledge. It can also be called upon to collect, analyse and interpret data from the grey literature.

#### 2.1. Leadership of an ESCo or advanced study

The project leader and the scientific leads together manage the methodological and scientific aspects of an ESCo or study. Mutual trust and a common vision of their respective responsibilities as well as the objectives of the project guide the collaboration between the project leader and the scientific leads.
<figure><img src="image_10.png" /></figure>

## CHAPTER 2. CONSTITUTION AND ROLES OF ESCO AND ADVANCED STUDY WORKING GROUPS

The working group of an ESCo or advanced study relies on a multidisciplinary committee of 20 to 40 scientific experts specialised in the subject. These experts identify and collect, with the help of information officers, the relevant international academic literature and extract elements that shed light on the questions raised by the sponsors. In the case of advanced studies, they complete this bibliographic synthesis by processing and assembling the data. Finally, they collectively write the scientific report of the project.

The work of the expert committee is co-led by a project leader from DEPE and one or more scientific leads. The leads set the scientific directions of the project, lead the collective production of results, ensure the use of available bibliographic material and knowledge, and draw up the general conclusions. They are responsible for producing the scientific report (see Chapter 5). The project leader is responsible for the overall coordination of the various stages of the project in accordance with the deadlines set and the methods and procedures established by DEPE. He/she is also responsible for the production of the condensed report and the summary report (see Chapter 5).

The project leader, the scientific leads and the expert committee are supported by information officers, a person responsible for the logistical and financial management of the project, and, depending on the project, one or more project officers (support for coordination, carrying out simulations and data analysis, analysis of technical documents, etc.).

If the analysis requires it, a technical committee can be set up to provide additional non-academic knowledge. It can also be called upon to collect, analyse and interpret data from the grey literature.

### 2.1. Leadership of an ESCo or advanced study

The project leader and the scientific leads together manage the methodological and scientific aspects of an ESCo or study. Mutual trust and a common vision of their respective responsibilities as well as the objectives of the project guide the collaboration between the project leader and the scientific leads.
### 2.1.1. Overall coordination: the DEPE project leader

The project leader is a member of DEPE and is responsible for the institutional and functional management of the project. He/she ensures that the deadlines set with the sponsors and the working principles and methods developed by the DEPE are respected.

The production of an ESCo or advanced study depends primarily on the establishment of a sustainable working dynamic. The project leader must contribute to creating and maintaining links between the various members of the working group over time. Together with the scientific leads, he/she is responsible for:

* identifying the scientific experts (see Section 2.2);

* preparing the expert meetings and contributing to their facilitation;

* proposing frameworks for consideration to the expert committee as the project progresses: analysis framework, report structure, framework for general conclusions, guiding themes for the condensed report, etc.

* ensuring that the momentum of the work is maintained from one meeting to the next and that interest in the collective work is maintained;

* ensuring that the experts' contributions are written within the set deadlines, and reviewing all the contributions;

* checking the quality of the work produced, i.e. that it responds to the questions in the project specifications, that it cites the scientific literature and that it is written in accordance with the procedures.

Although not an expert on the subject, the project leader, with a scientific background, contributes to the development of the results by participating in the compilation of knowledge and the development of the arguments in response to the questions asked. He/she is responsible for writing the condensed report and the summary report.

### 2.1.2. Scientific coordination: the scientific leads scientifiques

The scientific leads direct the expert committee from a scientific perspective. There are usually two or possibly three, with complementary skills. They are identified on the basis of their scientific achievements and discussions led by the DEPE director with the relevant scientific directors and department heads. Ultimately, the leads are nominated by the INRAE CEO and by his/her counterparts from the partner institutions in the case where the project is carried out by several organisations.

The scientific leads are recognised for their scientific competence, their ability to step back from the questions posed, their broad knowledge of the scientific fields concerned, as well as for their open-mindedness and curiosity and their people skills in leading a group. The acceptance of the leads by the scientific community, and in
### 2.1.1. Overall coordination: the DEPE project leader

The project leader is a member of DEPE and is responsible for the institutional and functional management of the project. He/she ensures that the deadlines set with the sponsors and the working principles and methods developed by the DEPE are respected.

The production of an ESCo or advanced study depends primarily on the establishment of a sustainable working dynamic. The project leader must contribute to creating and maintaining links between the various members of the working group over time. Together with the scientific leads, he/she is responsible for:

* identifying the scientific experts (see Section 2.2);

* preparing the expert meetings and contributing to their facilitation;

* proposing frameworks for consideration to the expert committee as the project progresses: analysis framework, report structure, framework for general conclusions, guiding themes for the condensed report, etc.

* ensuring that the momentum of the work is maintained from one meeting to the next and that interest in the collective work is maintained;

* ensuring that the experts' contributions are written within the set deadlines, and reviewing all the contributions;

* checking the quality of the work produced, i.e. that it responds to the questions in the project specifications, that it cites the scientific literature and that it is written in accordance with the procedures.

Although not an expert on the subject, the project leader, with a scientific background, contributes to the development of the results by participating in the compilation of knowledge and the development of the arguments in response to the questions asked. He/she is responsible for writing the condensed report and the summary report.

### 2.1.2. Scientific coordination: the scientific leads scientifiques

The scientific leads direct the expert committee from a scientific perspective. There are usually two or possibly three, with complementary skills. They are identified on the basis of their scientific achievements and discussions led by the DEPE director with the relevant scientific directors and department heads. Ultimately, the leads are nominated by the INRAE CEO and by his/her counterparts from the partner institutions in the case where the project is carried out by several organisations.

The scientific leads are recognised for their scientific competence, their ability to step back from the questions posed, their broad knowledge of the scientific fields concerned, as well as for their open-mindedness and curiosity and their people skills in leading a group. The acceptance of the leads by the scientific community, and in
particular by the experts, is essential. Great care must also be taken to ensure that they have no conflicts of interest (see Chapter 3) with stakeholders relating to the issue being analysed. Finally, the leads must be reasonably available throughout the project, as their involvement is estimated to be around 30-40% of a full-time position. This estimate varies over time according to the project phase and the number of scientific leads.

The leads are responsible for the project's scientific coordination. To this end, they:

* They ensure the scientific coherence of the arguments developed in each contribution, in each chapter and at the overall level of the report, and make sure that the conclusions are well-founded (which must be supported by the scientific literature and, in the case of studies, by the results of the additional data processing and collation phase).

* They lead the discussion of the experts' individual contributions in order to structure the general conclusions of the work, for which they are responsible for drafting a first version (final chapter of the report).

* They ensure that the experts make a clear distinction in their analyses between knowledge considered to be acquired and stabilised as opposed to more uncertain knowledge.

* They ensure that the report reflects scientific controversies and identifies gaps in knowledge.

* In the case of a study, they define the methodology for processing, assembling data or simulations and coordinate its implementation with the experts concerned.

* They ensure that the experts adopt and adhere to the logic of the summary, structured by the project manager on the basis of the general conclusions of the report.

* Together with the experts, they are scientifically responsible for the content of all deliverables.

The scientific leads also represent the expert group on the monitoring committee, the stakeholder advisory committee and the technical committee if necessary.

Finally, a scientific coordination role continues beyond the public presentation of conclusions at the end of the project, in order to disseminate and promote them as widely as possible. The scientific leads are thus required to present the results of the project to various bodies: scientists (internally and externally), stakeholders, media, professional audiences, associations, etc. They also coordinate the strategy for the academic exploitation of the results, which may take the form of publications and scientific symposia.
particular by the experts, is essential. Great care must also be taken to ensure that they have no conflicts of interest (see Chapter 3) with stakeholders relating to the issue being analysed. Finally, the leads must be reasonably available throughout the project, as their involvement is estimated to be around 30-40% of a full-time position. This estimate varies over time according to the project phase and the number of scientific leads.

The leads are responsible for the project's scientific coordination. To this end, they:

* They ensure the scientific coherence of the arguments developed in each contribution, in each chapter and at the overall level of the report, and make sure that the conclusions are well-founded (which must be supported by the scientific literature and, in the case of studies, by the results of the additional data processing and collation phase).

* They lead the discussion of the experts' individual contributions in order to structure the general conclusions of the work, for which they are responsible for drafting a first version (final chapter of the report).

* They ensure that the experts make a clear distinction in their analyses between knowledge considered to be acquired and stabilised as opposed to more uncertain knowledge.

* They ensure that the report reflects scientific controversies and identifies gaps in knowledge.

* In the case of a study, they define the methodology for processing, assembling data or simulations and coordinate its implementation with the experts concerned.

* They ensure that the experts adopt and adhere to the logic of the summary, structured by the project manager on the basis of the general conclusions of the report.

* Together with the experts, they are scientifically responsible for the content of all deliverables.

The scientific leads also represent the expert group on the monitoring committee, the stakeholder advisory committee and the technical committee if necessary.

Finally, a scientific coordination role continues beyond the public presentation of conclusions at the end of the project, in order to disseminate and promote them as widely as possible. The scientific leads are thus required to present the results of the project to various bodies: scientists (internally and externally), stakeholders, media, professional audiences, associations, etc. They also coordinate the strategy for the academic exploitation of the results, which may take the form of publications and scientific symposia.
## 2.2. Conducting the ESCo or advanced study: the scientific expert committee

### 2.2.1. Identification des experts

The experts are scientists identified within French or foreign public research or higher education institutions (researchers, teacher-researchers, engineers). The identification of experts begins at the end of the preparation phase of the project in accordance with four main principles that underpin the project's activities:

* **Competence:** experts are first selected on the basis of their publications in peer-reviewed scientific journals and on themes consistent with the field of the project. For advanced studies, depending on the nature of the data collection and processing component, their skills in using certain tools or their expertise concerning the necessary data are also considered (use of models, conducting meta-analyses, construction of indicators, etc.).

* **The plurality of disciplines and approaches:** this is reflected in the diversity of the scientific disciplines represented and the diversity of the institutional origins of the experts. Experts not affiliated with INRAE and its partners should, if possible, represent at least 1/3 of the committee in order to avoid a certain institutional homogeneity of approaches. The integration of foreign experts is also desirable in order to go beyond a "Franco-French" approach.

* **Impartiality:** this is assessed at the level of the committee of experts on the basis of declarations of links of interest that each expert is likely to have with different spheres of society and stakeholders affected by the project (see Chapter 3).

* **Transparency:** the principles for setting up the expert committee are presented in this chapter. The mobilisation of experts is carried out in a transparent manner with regard to their scientific hierarchy.

The preliminary exploration of bibliographic databases carried out by information officers during the preparation phase allows for the identification of the authors who publish the most on the themes within the scope of the project. ESCos and advanced studies are, for now, mostly conducted in French. French authors are therefore examined as a priority, as well as foreign authors who speak French or understand French without necessarily writing it, in order to facilitate dialogue within the predominantly French committee. When the sponsors agree, the expert committee may be international, and the entire project is then conducted in English.
## 2.2. Conducting the ESCo or advanced study: the scientific expert committee

### 2.2.1. Identification des experts

The experts are scientists identified within French or foreign public research or higher education institutions (researchers, teacher-researchers, engineers). The identification of experts begins at the end of the preparation phase of the project in accordance with four main principles that underpin the project's activities:

* **Competence:** experts are first selected on the basis of their publications in peer-reviewed scientific journals and on themes consistent with the field of the project. For advanced studies, depending on the nature of the data collection and processing component, their skills in using certain tools or their expertise concerning the necessary data are also considered (use of models, conducting meta-analyses, construction of indicators, etc.).

* **The plurality of disciplines and approaches:** this is reflected in the diversity of the scientific disciplines represented and the diversity of the institutional origins of the experts. Experts not affiliated with INRAE and its partners should, if possible, represent at least 1/3 of the committee in order to avoid a certain institutional homogeneity of approaches. The integration of foreign experts is also desirable in order to go beyond a "Franco-French" approach.

* **Impartiality:** this is assessed at the level of the committee of experts on the basis of declarations of links of interest that each expert is likely to have with different spheres of society and stakeholders affected by the project (see Chapter 3).

* **Transparency:** the principles for setting up the expert committee are presented in this chapter. The mobilisation of experts is carried out in a transparent manner with regard to their scientific hierarchy.

The preliminary exploration of bibliographic databases carried out by information officers during the preparation phase allows for the identification of the authors who publish the most on the themes within the scope of the project. ESCos and advanced studies are, for now, mostly conducted in French. French authors are therefore examined as a priority, as well as foreign authors who speak French or understand French without necessarily writing it, in order to facilitate dialogue within the predominantly French committee. When the sponsors agree, the expert committee may be international, and the entire project is then conducted in English.
Solicitation of potential experts identified from the bibliographic databases is prioritised according to several criteria:

* the match between the expert's publications and the topic on which he/she is to be called upon to contribute;

* the scope of the topic on which the expert is likely to be competent (bearing in mind that the topic area assigned to a given expert in an ESCo or a study often exceeds that on which they publish);

* the expert's experience in writing bibliographic summaries and his/her experience of working collectively;

* the availability of the expert (involvement in other projects, responsibilities and assignments, etc.);

* the absence of known 'major' links of interest inferred from the expert's membership of certain think tanks, decision-making bodies, etc. (see Chapter 3).

Depending on the nature of the project and the scope of the request, additional skills may be identified. These may include engineers for more technical projects (conducting simulations, developing calculation methodologies, etc.). As they do not necessarily publish in peer-reviewed journals, they do not appear in the preliminary bibliographic exploration. The identification of additional skills therefore also involves the knowledge of internal teams and consultation with the scientific hierarchy. Any identification of additional experts during the course of the project (in the event that a need for additional skills is identified after the project has been launched) must follow the procedure described above.

Ultimately, around twenty experts are usually engaged in an ESCo or advanced study, although some projects may involve larger committees.

Once the list of potential experts has been finalised, the DEPE director seeks the agreement of the management of the experts' home institutions, and each expert is then contacted individually by the project leader and/or the scientific leads. It is important that each expert contacted is aware of the subject and the specific work that will be required of him/her. As soon as the expert confirms his or her participation in the project, two elements formalise his or her commitment:

* the declaration of the links of interest which they may have (see Chapter 3);

* a letter signed by INRAE's CEO (and if applicable, by the general direction of the partners), which has the status of a mission letter. This letter is copied to the expert's scientific superiors (unit director, head of department and scientific director concerned) to inform them of this engagement.
Solicitation of potential experts identified from the bibliographic databases is prioritised according to several criteria:

* the match between the expert's publications and the topic on which he/she is to be called upon to contribute;

* the scope of the topic on which the expert is likely to be competent (bearing in mind that the topic area assigned to a given expert in an ESCo or a study often exceeds that on which they publish);

* the expert's experience in writing bibliographic summaries and his/her experience of working collectively;

* the availability of the expert (involvement in other projects, responsibilities and assignments, etc.);

* the absence of known 'major' links of interest inferred from the expert's membership of certain think tanks, decision-making bodies, etc. (see Chapter 3).

Depending on the nature of the project and the scope of the request, additional skills may be identified. These may include engineers for more technical projects (conducting simulations, developing calculation methodologies, etc.). As they do not necessarily publish in peer-reviewed journals, they do not appear in the preliminary bibliographic exploration. The identification of additional skills therefore also involves the knowledge of internal teams and consultation with the scientific hierarchy. Any identification of additional experts during the course of the project (in the event that a need for additional skills is identified after the project has been launched) must follow the procedure described above.

Ultimately, around twenty experts are usually engaged in an ESCo or advanced study, although some projects may involve larger committees.

Once the list of potential experts has been finalised, the DEPE director seeks the agreement of the management of the experts' home institutions, and each expert is then contacted individually by the project leader and/or the scientific leads. It is important that each expert contacted is aware of the subject and the specific work that will be required of him/her. As soon as the expert confirms his or her participation in the project, two elements formalise his or her commitment:

* the declaration of the links of interest which they may have (see Chapter 3);

* a letter signed by INRAE's CEO (and if applicable, by the general direction of the partners), which has the status of a mission letter. This letter is copied to the expert's scientific superiors (unit director, head of department and scientific director concerned) to inform them of this engagement.
### 2.2.2. Roles and responsibilities of experts

In an ESCo, the experts analyse the published scientific literature, extract the acquired, uncertain or controversial knowledge, and detect gaps in scientific knowledge. In advanced studies, in addition to reviewing the scientific literature (certified or 'grey'), the experts conduct data analysis and processing. The experts collectively develop and are responsible for the scientific content of the ESCo or advanced study. Their work requires an openness to the diversity of bibliographic sources and approaches to the collective endeavour. Each expert is responsible for reporting on all perspectives and approaches, even minority ones, as long as they are scientifically sound.

On average, the investment of an expert is estimated at about 15% of his or her working time, with significant variability throughout the project. Each expert is expected to:

* establish, with the support of the information officers, a relevant bibliographical corpus to respond to the issues in question;

* read the selected references in full in order to provide answers to the questions posed by the sponsors;

* develop, with the other experts, a methodology to process and/or assemble data or even conduct simulations in the case of advanced studies;

* write a referenced summary (around 15 pages), thus participating in the collective drafting of the report;

* participate in the expert committee meetings (3 to 6 meetings during the project), in the collective discussion of the general conclusions and in the feedback symposium.

Subsequently, the experts participate in the dissemination of the results of the project, and are encouraged to publish them in peer-reviewed journals and further develop them by entering into new collaborations.

When the range of questions posed in the request requires the mobilisation of a large number of disciplinary skills, it is possible to set up expert committees in two circles: a first circle of coordinating experts and a second circle of contributing experts (whose role has already been defined above). In this configuration, the coordinating experts have scientific leadership responsibilities in support of the scientific leads, each coordinating the writing of a chapter of the report. They bring the chapter plan to the contributing experts whose work they coordinate. They are responsible, with the help of the project leader, for bringing together/contacting the contributing experts that they coordinate, to review the progress of their analysis and drafting and to check that the contributions have been properly integrated into the plan. They write an introduction and a conclusion to the chapter that they coordinate.
### 2.2.2. Roles and responsibilities of experts

In an ESCo, the experts analyse the published scientific literature, extract the acquired, uncertain or controversial knowledge, and detect gaps in scientific knowledge. In advanced studies, in addition to reviewing the scientific literature (certified or 'grey'), the experts conduct data analysis and processing. The experts collectively develop and are responsible for the scientific content of the ESCo or advanced study. Their work requires an openness to the diversity of bibliographic sources and approaches to the collective endeavour. Each expert is responsible for reporting on all perspectives and approaches, even minority ones, as long as they are scientifically sound.

On average, the investment of an expert is estimated at about 15% of his or her working time, with significant variability throughout the project. Each expert is expected to:

* establish, with the support of the information officers, a relevant bibliographical corpus to respond to the issues in question;

* read the selected references in full in order to provide answers to the questions posed by the sponsors;

* develop, with the other experts, a methodology to process and/or assemble data or even conduct simulations in the case of advanced studies;

* write a referenced summary (around 15 pages), thus participating in the collective drafting of the report;

* participate in the expert committee meetings (3 to 6 meetings during the project), in the collective discussion of the general conclusions and in the feedback symposium.

Subsequently, the experts participate in the dissemination of the results of the project, and are encouraged to publish them in peer-reviewed journals and further develop them by entering into new collaborations.

When the range of questions posed in the request requires the mobilisation of a large number of disciplinary skills, it is possible to set up expert committees in two circles: a first circle of coordinating experts and a second circle of contributing experts (whose role has already been defined above). In this configuration, the coordinating experts have scientific leadership responsibilities in support of the scientific leads, each coordinating the writing of a chapter of the report. They bring the chapter plan to the contributing experts whose work they coordinate. They are responsible, with the help of the project leader, for bringing together/contacting the contributing experts that they coordinate, to review the progress of their analysis and drafting and to check that the contributions have been properly integrated into the plan. They write an introduction and a conclusion to the chapter that they coordinate.
## 2.3. Support roles

Support roles have several dimensions.

The logistical and financial management of projects is carried out by the DEPE's secretary-managers. These logistical and financial managers organise the material conditions for conducting the work and manage all aspects related to the financing of the project. This work includes the organisation of the symposium.

One or more information officers accompany all phases of the project. DEPE has in-house expertise in scientific and technical documentation. INRAE information officers and/or those from other organisations are often also involved. The information officers interact with the experts to build up the bibliographic corpus that underpins the project: they develop the bibliographic database search strategy in collaboration with the experts, then provide them with the corpus. At the end of the project, they draw up the final list of references cited in the ESCo or advanced study and conduct a bibliometric analysis.

Finally, depending on the nature of the project and the skills required, one or more project managers, usually recruited on fixed-term contracts, may complete the work group. Depending on the project, they may participate in the general organisation of the project by supporting the project leader in organising meetings, drafting deliverables, etc. They may also provide specific skills to complement those of the expert committee, for example to carry out certain technical/calculative projects to produce quantitative results (preparation and processing of data, conducting of simulations, etc.) and/or to take charge of studies complementary to the analysis of the bibliography (textual analysis of the corpus, analysis of the context of the request based on the technical literature, etc.).

## 2.4. Technical Committee

Some projects may require the involvement of experts who do not belong to public research structures and who, for this reason, cannot be included in the committee of scientific experts. For example, the analysis of the geographical, regulatory, economic or social context in which the questions posed to the experts lie may require the input of technical centres or specific departments of ministries or quasi-public or professional bodies (statistical bodies, experts within agencies such as ADEME, CEREMA, experts from technical institutes, etc.). Another situation that may require the mobilisation of 'technical' experts is the analysis of data not directly accessible to the working group (for example, unpublished results of field experiments).
## 2.3. Support roles

Support roles have several dimensions.

The logistical and financial management of projects is carried out by the DEPE's secretary-managers. These logistical and financial managers organise the material conditions for conducting the work and manage all aspects related to the financing of the project. This work includes the organisation of the symposium.

One or more information officers accompany all phases of the project. DEPE has in-house expertise in scientific and technical documentation. INRAE information officers and/or those from other organisations are often also involved. The information officers interact with the experts to build up the bibliographic corpus that underpins the project: they develop the bibliographic database search strategy in collaboration with the experts, then provide them with the corpus. At the end of the project, they draw up the final list of references cited in the ESCo or advanced study and conduct a bibliometric analysis.

Finally, depending on the nature of the project and the skills required, one or more project managers, usually recruited on fixed-term contracts, may complete the work group. Depending on the project, they may participate in the general organisation of the project by supporting the project leader in organising meetings, drafting deliverables, etc. They may also provide specific skills to complement those of the expert committee, for example to carry out certain technical/calculative projects to produce quantitative results (preparation and processing of data, conducting of simulations, etc.) and/or to take charge of studies complementary to the analysis of the bibliography (textual analysis of the corpus, analysis of the context of the request based on the technical literature, etc.).

## 2.4. Technical Committee

Some projects may require the involvement of experts who do not belong to public research structures and who, for this reason, cannot be included in the committee of scientific experts. For example, the analysis of the geographical, regulatory, economic or social context in which the questions posed to the experts lie may require the input of technical centres or specific departments of ministries or quasi-public or professional bodies (statistical bodies, experts within agencies such as ADEME, CEREMA, experts from technical institutes, etc.). Another situation that may require the mobilisation of 'technical' experts is the analysis of data not directly accessible to the working group (for example, unpublished results of field experiments).
In order to benefit from these skills and/or data while guaranteeing the independence of the expert committee, it is possible to set up a technical committee. This committee provides support to the scientific experts in the discussion of certain choices made by the expert committee, in the interpretation of results from technical and/or field data, in giving an opinion on the choice of situations not described in the bibliography that are interesting to study, and in providing an opinion on the coherence of the work conducted within the 'study' component given their knowledge of the field, etc.

The members of this group participate in a personal capacity and not as representatives of their organisations (institutional representation is part of the stakeholder advisory committee, as described in Chapter 1). Their contributions are formalised in writing, with the expert committee remaining the sole judge of the follow-up to be given to the proposals made by the technical committee. If they are included in the scientific report, the special status of this information in relation to the rest of the elements comprising the report is explicitly stated.

The members of the technical committee do not co-sign the project deliverables and do not take responsibility for its conclusions. The information exchanged between the scientific expert committee and the technical committee remains confidential until publication of the study's results.
In order to benefit from these skills and/or data while guaranteeing the independence of the expert committee, it is possible to set up a technical committee. This committee provides support to the scientific experts in the discussion of certain choices made by the expert committee, in the interpretation of results from technical and/or field data, in giving an opinion on the choice of situations not described in the bibliography that are interesting to study, and in providing an opinion on the coherence of the work conducted within the 'study' component given their knowledge of the field, etc.

The members of this group participate in a personal capacity and not as representatives of their organisations (institutional representation is part of the stakeholder advisory committee, as described in Chapter 1). Their contributions are formalised in writing, with the expert committee remaining the sole judge of the follow-up to be given to the proposals made by the technical committee. If they are included in the scientific report, the special status of this information in relation to the rest of the elements comprising the report is explicitly stated.

The members of the technical committee do not co-sign the project deliverables and do not take responsibility for its conclusions. The information exchanged between the scientific expert committee and the technical committee remains confidential until publication of the study's results.
<figure><img src="image_9.png" /></figure>

## CHAPTER 3. IDENTIFICATION AND ANALYSIS OF THE EXPERTS' LINKS OF INTEREST

Two types of links of interest are likely to introduce bias into the experts' analyses and thus influence the lessons learned and the conclusions conveyed to public decision-makers: the links between experts and stakeholders in the subject under study, and the scientific links between experts. The principles of impartiality and transparency to which ESCos and advanced studies are subject require that these links be made explicit and that their analysis be transparent.

Considerable attention is paid to the examination of links of interest at the start of the project: first when the scientific leads are chosen, and then when the expert committee is formed.

The approach adopted by DEPE is based on:

* the completion of a standard form by each expert in which he/she indicates his/her links with stakeholders involved in the field under study;

* the identification of scientific links between experts and, as a corollary, the assessment of the diversity of scientific approaches within the committee of experts;

* an examination of these links by an ad hoc committee under the aegis of the person responsible for ethics at INRAE.

The examination of individual links of interest aims to identify possible conflicts of interest that would lead to the exclusion of the expert concerned. At the level of the committee as a whole, the analysis of links seeks to guard against possible biases tending to orient the collective, to alert to imbalances in the proximity of the experts to certain stakeholders and/or scientific communities working in the field in question. The conclusions of this examination are included in the extended report of the ESCO or advanced study in order to make this information public, thus contributing to the credibility of the study.

### 3.1. Identifying links of interest

As the name suggests, the Declaration of Interests (DLI) is the responsibility of the expert making the declaration. The form used by DEPE is available online at https://hal.inrae.fr/hal-03553051. It is based on the legal framework adopted by
<figure><img src="image_11.png" /></figure>

## CHAPTER 3. IDENTIFICATION AND ANALYSIS OF THE EXPERTS' LINKS OF INTEREST

Two types of links of interest are likely to introduce bias into the experts' analyses and thus influence the lessons learned and the conclusions conveyed to public decision-makers: the links between experts and stakeholders in the subject under study, and the scientific links between experts. The principles of impartiality and transparency to which ESCos and advanced studies are subject require that these links be made explicit and that their analysis be transparent.

Considerable attention is paid to the examination of links of interest at the start of the project: first when the scientific leads are chosen, and then when the expert committee is formed.

The approach adopted by DEPE is based on:

* the completion of a standard form by each expert in which he/she indicates his/her links with stakeholders involved in the field under study;

* the identification of scientific links between experts and, as a corollary, the assessment of the diversity of scientific approaches within the committee of experts;

* an examination of these links by an ad hoc committee under the aegis of the person responsible for ethics at INRAE.

The examination of individual links of interest aims to identify possible conflicts of interest that would lead to the exclusion of the expert concerned. At the level of the committee as a whole, the analysis of links seeks to guard against possible biases tending to orient the collective, to alert to imbalances in the proximity of the experts to certain stakeholders and/or scientific communities working in the field in question. The conclusions of this examination are included in the extended report of the ESCO or advanced study in order to make this information public, thus contributing to the credibility of the study.

### 3.1. Identifying links of interest

As the name suggests, the Declaration of Interests (DLI) is the responsibility of the expert making the declaration. The form used by DEPE is available online at https://hal.inrae.fr/hal-03553051. It is based on the legal framework adopted by
the Public Health Code (2012)<fnref n="7" /> and the Research Code (2021)<fnref n="8" />. It conforms to the standards of international research organisations. This form is completed by the leads and all members of the working group before the start of the project and ensures transparency regarding links with stakeholders in the field of the ESCo or advanced study over the previous five years. These links may be direct or indirect, and may be in a professional or personal capacity. The expert is expected to take a thorough approach to his or her engagements in relation to the subject matter of the ESCo or advanced study.

The form allows for qualification of the expert's activity: it may be a regular or occasional consultancy activity; the expert may be a member of a board of directors, a steering committee, a steering or evaluation committee, etc. He/she may be a shareholder in a start-up or a company, or hold a patent, etc. Speeches at conferences (or any other event) should also be reported.

Organisations that may be subject to links of interest include companies, associations, trade unions, consultancies, technical institutes, publishers, learned societies, clubs, foundations, local authorities, public and semi-public agencies, etc.

The existence of financial or in-kind support from private bodies is indicated in an appendix which is not made public. This information should include amounts received in the context of research projects or theses (e.g. Cifre contract). The remuneration or compensation allocated to the expert, either directly or indirectly via the institution to which he/she belongs, is also specified.

Research projects involving exclusively public research organisations, as well as assignments with public teaching bodies or agencies, make it possible to identify the scientific communities with which the expert works.

Finally, the expert must mention whether any of his close relatives are employees and/or have financial interests in any entity whose corporate purpose falls within the thematic scope of the project.

The DLIs filled in and signed by the experts must be in the possession of the DEPE at the beginning of the project so that they can be examined as soon as possible. DEPE archives all DLIs for a period of 5 years. Subject to regulations (CNIL), they can be viewed on request (with the exception of the financial appendix).

<footnote n="7">⁷ Information on the standard public declaration of interest document on transparency in public health and health safety is provided in the Public Health Code and was updated in March 2017. www.legifrance.gouv.fr/codes/article_lc/ LEGIARTI000041748484/ and www.legifrance.gouv.fr/jorf/id/JORFTEXT000034330604 (accessed in January 2021).</footnote>

<footnote n="8">⁸ Order of 17 December 2021 issued pursuant to Decree no. 2021-1448 of 4 November 2021 on the declaration of interests prior to the project of an expert mission provided for in Article L. 411-5 of the Research Code https://www.legifrance.gouv.fr/jorf/id/JORFTEXT000045159779</footnote>
the Public Health Code (2012)<fnref n="7" /> and the Research Code (2021)<fnref n="8" />. It conforms to the standards of international research organisations. This form is completed by the leads and all members of the working group before the start of the project and ensures transparency regarding links with stakeholders in the field of the ESCo or advanced study over the previous five years. These links may be direct or indirect, and may be in a professional or personal capacity. The expert is expected to take a thorough approach to his or her engagements in relation to the subject matter of the ESCo or advanced study.

The form allows for qualification of the expert's activity: it may be a regular or occasional consultancy activity; the expert may be a member of a board of directors, a steering committee, a steering or evaluation committee, etc. He/she may be a shareholder in a start-up or a company, or hold a patent, etc. Speeches at conferences (or any other event) should also be reported.

Organisations that may be subject to links of interest include companies, associations, trade unions, consultancies, technical institutes, publishers, learned societies, clubs, foundations, local authorities, public and semi-public agencies, etc.

The existence of financial or in-kind support from private bodies is indicated in an appendix which is not made public. This information should include amounts received in the context of research projects or theses (e.g. Cifre contract). The remuneration or compensation allocated to the expert, either directly or indirectly via the institution to which he/she belongs, is also specified.

Research projects involving exclusively public research organisations, as well as assignments with public teaching bodies or agencies, make it possible to identify the scientific communities with which the expert works.

Finally, the expert must mention whether any of his close relatives are employees and/or have financial interests in any entity whose corporate purpose falls within the thematic scope of the project.

The DLIs filled in and signed by the experts must be in the possession of the DEPE at the beginning of the project so that they can be examined as soon as possible. DEPE archives all DLIs for a period of 5 years. Subject to regulations (CNIL), they can be viewed on request (with the exception of the financial appendix).

<footnote n="7">⁷ Information on the standard public declaration of interest document on transparency in public health and health safety is provided in the Public Health Code and was updated in March 2017. www.legifrance.gouv.fr/codes/article_lc/ LEGIARTI000041748484/ and www.legifrance.gouv.fr/jorf/id/JORFTEXT000034330604 (accessed in January 2021).</footnote>

<footnote n="8">⁸ Order of 17 December 2021 issued pursuant to Decree no. 2021-1448 of 4 November 2021 on the declaration of interests prior to the project of an expert mission provided for in Article L. 411-5 of the Research Code https://www.legifrance.gouv.fr/jorf/id/JORFTEXT000045159779</footnote>
## 3.2. Analysing links of interest

The review of links of interest is threefold. The first part deals with individual links of interest. The second considers all the links of interest identified at the level of the expert committee, thus providing a consolidated picture of the risks of bias. The third part maps the scientific networks linking the experts to each other, making it possible to assess the diversity of scientific approaches within the expert committee.

### 3.2.1. Review of individual declarations of interest

A review panel is called upon as soon as the names of the scientific leads are identified. It is chaired by the INRAE ethics delegate and consists of at least one member of the INRAE ethics and scientific integrity committee, a representative of the INRAE scientific directorates charged with monitoring the project, the DEPE director and, if the project is conducted in partnership, a representative of each partner organisation. The commission meets twice, once to decide on the leads' declarations and once to examine the experts' declarations. The choice of the leads and the participation of the experts are validated only after examination of their declaration of interest.

Based on the declaration forms provided by the experts, the review panel examines the links that could be detrimental to the balance and independence expected in an expert study. The objective is not to exclude any expert with links of interest with stakeholders, as collaborations with civil society and with professional actors are frequent and encouraged in a targeted research organisation such as INRAE.

Indeed, these links ensure that the expert is familiar with the social, political and economic context of the ESCo or advanced study request.

The panel considers that a conflict of interest exists when the declared links are likely to bias the expert's assessment, influence their freedom of expression or call into question their legitimacy in relation to the expert committee. A conflict of interest is grounds for removal from an ESCo or advanced study. If in doubt, the commission may ask an expert to clarify his/her declaration. The declaration of interests nevertheless remains the responsibility of the expert.

### 3.2.2. Analysis of the collective links of interest

While the risks of conflicts of interest are dealt with individually, the committee also looks at the links of interest as a whole. Their diagnosis is based on the analysis provided by the project leader to the panel. Based on the DLIs, he/she draws up an anonymised overview of all of the links of interest in order to present, in a summarised and consolidated manner, the types of stakeholders with whom the committee of experts is in contact, and the respective influence of the stakeholders in these relationships.
## 3.2. Analysing links of interest

The review of links of interest is threefold. The first part deals with individual links of interest. The second considers all the links of interest identified at the level of the expert committee, thus providing a consolidated picture of the risks of bias. The third part maps the scientific networks linking the experts to each other, making it possible to assess the diversity of scientific approaches within the expert committee.

### 3.2.1. Review of individual declarations of interest

A review panel is called upon as soon as the names of the scientific leads are identified. It is chaired by the INRAE ethics delegate and consists of at least one member of the INRAE ethics and scientific integrity committee, a representative of the INRAE scientific directorates charged with monitoring the project, the DEPE director and, if the project is conducted in partnership, a representative of each partner organisation. The commission meets twice, once to decide on the leads' declarations and once to examine the experts' declarations. The choice of the leads and the participation of the experts are validated only after examination of their declaration of interest.

Based on the declaration forms provided by the experts, the review panel examines the links that could be detrimental to the balance and independence expected in an expert study. The objective is not to exclude any expert with links of interest with stakeholders, as collaborations with civil society and with professional actors are frequent and encouraged in a targeted research organisation such as INRAE.

Indeed, these links ensure that the expert is familiar with the social, political and economic context of the ESCo or advanced study request.

The panel considers that a conflict of interest exists when the declared links are likely to bias the expert's assessment, influence their freedom of expression or call into question their legitimacy in relation to the expert committee. A conflict of interest is grounds for removal from an ESCo or advanced study. If in doubt, the commission may ask an expert to clarify his/her declaration. The declaration of interests nevertheless remains the responsibility of the expert.

### 3.2.2. Analysis of the collective links of interest

While the risks of conflicts of interest are dealt with individually, the committee also looks at the links of interest as a whole. Their diagnosis is based on the analysis provided by the project leader to the panel. Based on the DLIs, he/she draws up an anonymised overview of all of the links of interest in order to present, in a summarised and consolidated manner, the types of stakeholders with whom the committee of experts is in contact, and the respective influence of the stakeholders in these relationships.
The diversity among the types of stakeholders shows the varied interactions of the experts with the non-academic sphere, which suggests that the expert committee collectively has the means to understand the social and professional issues in the field of the ESCo or advanced study in an open manner. The relative weight of the different categories of actors may highlight either a balance between, or an over-representation of, certain socio-economic sectors in the partnerships. The absence of certain categories of stakeholders may highlight a lack of appreciation or underestimation of certain issues. For example, the absence of consumer associations may suggest that the experts are less aware of demand issues than of supply issues arising from the world of agriculture or agri-food processing.

The number of links per expert gives an indication of the intensity of the experts' involvement in 'field' issues. A high frequency of financial links and their average amount can be an indication of possible precautionary positions taken more or less implicitly towards funders.

### 3.2.3. Diversity of scientific approaches

The commission also examines the composition of the expert committee and the scientific links between experts. This diagnosis is prepared by the project leader and the information officers involved in the ESCo or advanced study. Its main purpose is to verify the diversity of scientific approaches within the committee.

The description of the committee is based on the following indicators:

* the representation of the different scientific disciplines;

* the representation of experts according to gender and experience;

* the representation of INRAE experts and its partners;

* the representation of international experts.

The analysis of experts' links within the scientific community is documented using textual analysis software applied to experts' publications. These tools make it possible to identify the relationships between authors within the scientific community. The resulting map represents the networks of scientific relationships, whether present within the committee (direct co-publication links between experts) or, more indirectly, via the experts' collaborators (links between co-authors). The diversity of these networks is an indication of the diversity of scientific approaches and schools of thought with which the experts are in contact. The distance between the scientific communities present may indicate divergent theoretical approaches. The arrangements and the respective weight of the subsets may be rather balanced, or on the contrary reveal an over-representation of certain scientific communities, while others will appear to be in the minority or marginal.
The diversity among the types of stakeholders shows the varied interactions of the experts with the non-academic sphere, which suggests that the expert committee collectively has the means to understand the social and professional issues in the field of the ESCo or advanced study in an open manner. The relative weight of the different categories of actors may highlight either a balance between, or an over-representation of, certain socio-economic sectors in the partnerships. The absence of certain categories of stakeholders may highlight a lack of appreciation or underestimation of certain issues. For example, the absence of consumer associations may suggest that the experts are less aware of demand issues than of supply issues arising from the world of agriculture or agri-food processing.

The number of links per expert gives an indication of the intensity of the experts' involvement in 'field' issues. A high frequency of financial links and their average amount can be an indication of possible precautionary positions taken more or less implicitly towards funders.

### 3.2.3. Diversity of scientific approaches

The commission also examines the composition of the expert committee and the scientific links between experts. This diagnosis is prepared by the project leader and the information officers involved in the ESCo or advanced study. Its main purpose is to verify the diversity of scientific approaches within the committee.

The description of the committee is based on the following indicators:

* the representation of the different scientific disciplines;

* the representation of experts according to gender and experience;

* the representation of INRAE experts and its partners;

* the representation of international experts.

The analysis of experts' links within the scientific community is documented using textual analysis software applied to experts' publications. These tools make it possible to identify the relationships between authors within the scientific community. The resulting map represents the networks of scientific relationships, whether present within the committee (direct co-publication links between experts) or, more indirectly, via the experts' collaborators (links between co-authors). The diversity of these networks is an indication of the diversity of scientific approaches and schools of thought with which the experts are in contact. The distance between the scientific communities present may indicate divergent theoretical approaches. The arrangements and the respective weight of the subsets may be rather balanced, or on the contrary reveal an over-representation of certain scientific communities, while others will appear to be in the minority or marginal.
The review panel draws its conclusions from these different perspectives. It warns of the risks identified. It may suggest changes in the composition of the expert committee, recommend that additional experts be brought in on an *ad hoc* basis to compensate for certain shortcomings, complete the stakeholder advisory committee, etc.

### 3.3. A reflective tool for the management of the project

The methodological attention given to revealing links of interest, whether societal or scientific, also contributes to the conduct of the ESCo or the study.

The diagnosis carried out for the review panel is presented at a meeting of the expert committee, so that the experts can integrate it, in a reflective manner, as an element of their own functioning. This mirroring effect has a function of revealing the risks of bias. In a certain sense, the committee becomes aware of its position in the interplay of social and scientific actors. The project leader, the scientific leads and the expert committee can possibly take measures to counteract certain characteristics inherent to the expert group. This can lead, for example, to including stakeholders with whom the experts are not used to collaborating in the stakeholder advisory committee, in order to gather their views.

The conclusions of the analyses can also be communicated to the monitoring committee and the stakeholder advisory committee at the beginning of the project. Beyond the concern for transparency (the information will be included in the report published at the end of the project), this dialogue serves to raise awareness of the assessment of links of interest and helps to build confidence in the process.
The review panel draws its conclusions from these different perspectives. It warns of the risks identified. It may suggest changes in the composition of the expert committee, recommend that additional experts be brought in on an *ad hoc* basis to compensate for certain shortcomings, complete the stakeholder advisory committee, etc.

### 3.3. A reflective tool for the management of the project

The methodological attention given to revealing links of interest, whether societal or scientific, also contributes to the conduct of the ESCo or the study.

The diagnosis carried out for the review panel is presented at a meeting of the expert committee, so that the experts can integrate it, in a reflective manner, as an element of their own functioning. This mirroring effect has a function of revealing the risks of bias. In a certain sense, the committee becomes aware of its position in the interplay of social and scientific actors. The project leader, the scientific leads and the expert committee can possibly take measures to counteract certain characteristics inherent to the expert group. This can lead, for example, to including stakeholders with whom the experts are not used to collaborating in the stakeholder advisory committee, in order to gather their views.

The conclusions of the analyses can also be communicated to the monitoring committee and the stakeholder advisory committee at the beginning of the project. Beyond the concern for transparency (the information will be included in the report published at the end of the project), this dialogue serves to raise awareness of the assessment of links of interest and helps to build confidence in the process.
<figure><img src="image_10.png" /></figure>

# CHAPTER 4. COMPILATION AND ANALYSIS OF DOCUMENTARY SOURCES

This chapter presents the methodology for compiling the bibliographic corpus and sets out the principles adopted to guarantee the reliability of the documentary sources<fnref n="9" /> on which the results of the ESCo and advanced studies are based. The relevance of the bibliographical references cited in the report determines the credibility of the results. The ESCo or advanced study report provides a quantitative and qualitative assessment of the final bibliographic corpus.

## 4.1. Documentary sources

The methodology of the bibliographic search is as exhaustive as possible, by searching international scientific databases with the help of queries explained in the report. The documentary sources used must be 'certified', i.e. judged to be robust in terms of method (experimental protocol, choice of primary data, etc.) and interpretation of results. Review committees attached to scientific journals guarantee this certification by peer-reviewing manuscripts on scientific criteria.

### 4.1.1. Academic literature

The international academic literature forms the basis of the ESCo's documentary base. It mainly includes articles from peer-reviewed scientific journals and, to a lesser extent, books and book chapters. International peer-reviewed scientific journals are indexed in international databases (see Box 1). The lists provided by Clarivate Analytics<fnref n="10" /> and by the High Council for the Evaluation of Research and Higher Education (HCERES) are a guarantee of reliability.

The major international publishers (see Box 2) of academic books also have peer review committees or quality assessment processes in place for manuscripts.

The importance of international scientific journals differs between disciplines. Academic literature in the social sciences, in contrast to the life and earth sciences, relies significantly on books and journals published at the national level.

<footnote n="9">⁹ The term "documentary sources" covers all the documents analysed by the experts. The selected sources form a bibliographic corpus. The final corpus includes the "bibliographical references" mentioned in the expert or study report.</footnote>

<footnote n="10">¹⁰ https://clarivate.libguides.com/ld.php?content_id=48842741.</footnote>
<figure><img src="image_12.png" /></figure>

# CHAPTER 4. COMPILATION AND ANALYSIS OF DOCUMENTARY SOURCES

This chapter presents the methodology for compiling the bibliographic corpus and sets out the principles adopted to guarantee the reliability of the documentary sources<fnref n="9" /> on which the results of the ESCo and advanced studies are based. The relevance of the bibliographical references cited in the report determines the credibility of the results. The ESCo or advanced study report provides a quantitative and qualitative assessment of the final bibliographic corpus.

## 4.1. Documentary sources

The methodology of the bibliographic search is as exhaustive as possible, by searching international scientific databases with the help of queries explained in the report. The documentary sources used must be 'certified', i.e. judged to be robust in terms of method (experimental protocol, choice of primary data, etc.) and interpretation of results. Review committees attached to scientific journals guarantee this certification by peer-reviewing manuscripts on scientific criteria.

### 4.1.1. Academic literature

The international academic literature forms the basis of the ESCo's documentary base. It mainly includes articles from peer-reviewed scientific journals and, to a lesser extent, books and book chapters. International peer-reviewed scientific journals are indexed in international databases (see Box 1). The lists provided by Clarivate Analytics<fnref n="10" /> and by the High Council for the Evaluation of Research and Higher Education (HCERES) are a guarantee of reliability.

The major international publishers (see Box 2) of academic books also have peer review committees or quality assessment processes in place for manuscripts.

The importance of international scientific journals differs between disciplines. Academic literature in the social sciences, in contrast to the life and earth sciences, relies significantly on books and journals published at the national level.

<footnote n="9">⁹ The term "documentary sources" covers all the documents analysed by the experts. The selected sources form a bibliographic corpus. The final corpus includes the "bibliographical references" mentioned in the expert or study report.</footnote>

<footnote n="10">¹⁰ https://clarivate.libguides.com/ld.php?content_id=48842741.</footnote>
## 1.1. International bibliographic databases

* **HAL open archive:** online platform launched in 2001 by the CNRS, which provides access to publications from French and foreign teaching and research establishments and public and private scientific laboratories. Access is free, and INRAE has its own HAL-INRAE portal: https://hal.inrae.fr/

* **Web of Science™ Core Collection (WoS)**: scientific information platform managed by Clarivate Analytics (originally by Thomson Reuters). Since 1955, it has referenced more than 20,000 journals, mostly in the fields of science and technology, but with limited coverage of the social sciences.

* **Scopus**: trans-disciplinary bibliographic and bibliometric database launched by the scientific publisher Elsevier in 2004. The first references date back to 1970. Themes include physical, medical and life sciences, as well as social sciences and humanities.

* **Food Science Source**: platform focused on food science and agriculture.

* **PubMed**: database containing Medline, but enhanced with references not yet indexed in Medline https://bib.umontreal.ca/guides/bd/pubmed or https://www.nlm.nih.gov/bsd/difference.html

* **EconLit**: database focused on economic literature.

* **Repec (Research Papers in Economics)**: bibliographic database of research in economics: http://www.repec.org/

* **CAIRN**: database of journals and books in the humanities and social sciences: https://www.cairn.info/

* **CAB Abstracts®**: database published by CAB International and specialised in applied disciplines relating to the life sciences. More information at: https://www6.inrae.fr/reselec/Bases-de-donnees

## 2: Examples of publishers whose works can be considered as certified academic literature

* Cambridge University Press http://www.cambridge.org

* CABI Publishing http://www.cabi.org/

* Wageningen Academic Publishers http://www.wageningenacademic.com

* John Libbey Eurotext http://www.jle.com/fr/index.md

* Springer http://www.springer.com

* Wiley-Blackwell http://eu.wiley.com

* Elsevier http://www.elsevier.com/wps/find/authors.authors/bookauthorshome

* Quae https://www.quae.com/
## 1.1. International bibliographic databases

* **HAL open archive:** online platform launched in 2001 by the CNRS, which provides access to publications from French and foreign teaching and research establishments and public and private scientific laboratories. Access is free, and INRAE has its own HAL-INRAE portal: https://hal.inrae.fr/

* **Web of Science™ Core Collection (WoS)**: scientific information platform managed by Clarivate Analytics (originally by Thomson Reuters). Since 1955, it has referenced more than 20,000 journals, mostly in the fields of science and technology, but with limited coverage of the social sciences.

* **Scopus**: trans-disciplinary bibliographic and bibliometric database launched by the scientific publisher Elsevier in 2004. The first references date back to 1970. Themes include physical, medical and life sciences, as well as social sciences and humanities.

* **Food Science Source**: platform focused on food science and agriculture.

* **PubMed**: database containing Medline, but enhanced with references not yet indexed in Medline https://bib.umontreal.ca/guides/bd/pubmed or https://www.nlm.nih.gov/bsd/difference.html

* **EconLit**: database focused on economic literature.

* **Repec (Research Papers in Economics)**: bibliographic database of research in economics: http://www.repec.org/

* **CAIRN**: database of journals and books in the humanities and social sciences: https://www.cairn.info/

* **CAB Abstracts®**: database published by CAB International and specialised in applied disciplines relating to the life sciences. More information at: https://www6.inrae.fr/reselec/Bases-de-donnees

## 2: Examples of publishers whose works can be considered as certified academic literature

* Cambridge University Press http://www.cambridge.org

* CABI Publishing http://www.cabi.org/

* Wageningen Academic Publishers http://www.wageningenacademic.com

* John Libbey Eurotext http://www.jle.com/fr/index.md

* Springer http://www.springer.com

* Wiley-Blackwell http://eu.wiley.com

* Elsevier http://www.elsevier.com/wps/find/authors.authors/bookauthorshome

* Quae https://www.quae.com/
### 4.1.2. Technical literature

When the available academic scientific literature within the scope of the ESCo or the study is insufficient (poorly contextualised, unsettled or incomplete scientific knowledge), the experts may draw on so-called 'grey' literature<fnref n="11" />.

In particular, grey literature includes reports produced by bodies or working groups that carry out expert studies. These include reports from international bodies such as the FAO (HPLE reports) or the work of the European Joint Research Centre (JRC), European or international agencies. 'Grey' literature also includes documents of a much more diverse nature from government bodies (parliamentary and ministerial reports, reports from the Court of Auditors, etc.), the education sector, professional sectors, NGOs, think tanks, etc.

This literature can provide recent information and insights that are useful for supplementing the elements extracted from the academic scientific literature, particularly because it is contextualised and geographically situated. For example, the Agreste summaries produced by the Statistics and Forecasting Service of the Ministry of Agriculture and Food are frequently cited in ESCos and advanced study reports. The same applies to the literature produced by technical agricultural institutes when there is a need to contextualise the results of experimental or theoretical scientific research.

However, the use of non scientific literature needs to be kept to a minimum for ESCo projects. Grey literature must be validated by the expert committee. It should only be used to illustrate specific questions, and not to support major conclusions. It can be used in a more structuring way in advanced studies which, by construction, deal with subjects less documented in the academic literature.

### 4.1.3. Body of regulations

However, the use of grey literature should remain *ad hoc* in the case of an ESCo. Grey literature must be validated by the expert committee. It should only be used to illustrate specific questions and not to support major conclusions of the work. It can be used in a more structuring way in studies which, by their very nature, deal with subjects that are less documented in the academic literature.

<footnote n="11">¹¹ According to AFNOR, grey literature refers to any 'typed or printed document, often of a provisional nature, reproduced and distributed in fewer than a thousand copies, outside the commercial publishing and distribution channels'.</footnote>
### 4.1.2. Technical literature

When the available academic scientific literature within the scope of the ESCo or the study is insufficient (poorly contextualised, unsettled or incomplete scientific knowledge), the experts may draw on so-called 'grey' literature<fnref n="11" />.

In particular, grey literature includes reports produced by bodies or working groups that carry out expert studies. These include reports from international bodies such as the FAO (HPLE reports) or the work of the European Joint Research Centre (JRC), European or international agencies. 'Grey' literature also includes documents of a much more diverse nature from government bodies (parliamentary and ministerial reports, reports from the Court of Auditors, etc.), the education sector, professional sectors, NGOs, think tanks, etc.

This literature can provide recent information and insights that are useful for supplementing the elements extracted from the academic scientific literature, particularly because it is contextualised and geographically situated. For example, the Agreste summaries produced by the Statistics and Forecasting Service of the Ministry of Agriculture and Food are frequently cited in ESCos and advanced study reports. The same applies to the literature produced by technical agricultural institutes when there is a need to contextualise the results of experimental or theoretical scientific research.

However, the use of non scientific literature needs to be kept to a minimum for ESCo projects. Grey literature must be validated by the expert committee. It should only be used to illustrate specific questions, and not to support major conclusions. It can be used in a more structuring way in advanced studies which, by construction, deal with subjects less documented in the academic literature.

### 4.1.3. Body of regulations

However, the use of grey literature should remain *ad hoc* in the case of an ESCo. Grey literature must be validated by the expert committee. It should only be used to illustrate specific questions and not to support major conclusions of the work. It can be used in a more structuring way in studies which, by their very nature, deal with subjects that are less documented in the academic literature.

<footnote n="11">¹¹ According to AFNOR, grey literature refers to any 'typed or printed document, often of a provisional nature, reproduced and distributed in fewer than a thousand copies, outside the commercial publishing and distribution channels'.</footnote>
<table><caption>Table 2. Stages in the compilation of the corpus bibliographic</caption><thead><tr><th>Bibliographic queries</th><th>State of corpus</th><th>Use of corpus</th></tr></thead><tbody><tr><td>Search the WoS/Scopus databases using keywords identified in the referral and defined by the project leader, information officers and scientific leads</td><td>Exploratory corpus</td><td>Quantify the scientific articles published on the subject. Clarify the questions and scope of the ESCo. Identify potential experts</td></tr><tr><td>Improve queries based on keywords and selection criteria defined by the experts</td><td>Initial corpus</td><td>Distribution of the corpus among experts according to disciplines and issues addressed</td></tr><tr><td>Progressively refine the initial corpus in order to select/add relevant references to meet the project specifications</td><td></td><td></td></tr><tr><td>✓ Temporal limits of the bibliographic search (generally between 25 and 10 years prior)</td><td>Intermediate corpus</td><td>Bibliometric analysis of the corpus of each chapter to inform the discussion within the expert committee</td></tr><tr><td>✓ Identification of syntheses and meta-analyses that have already dealt with the scientific literature</td><td>Versions 1, 2, 3, 4, etc.</td><td>Qualitative diagnosis of the bibliographic corpus of the different chapters by the expert committee</td></tr><tr><td>✓ Elimination of irrelevant or redundant references</td><td>Validated version</td><td>Validation of the bibliographic corpus of the different chapters by the expert committee</td></tr><tr><td>✓ Addition of references, identified by the experts, that were not captured by the queries</td><td></td><td></td></tr><tr><td>✓ Addition of references derived from the literature review carried out during the project</td><td></td><td></td></tr><tr><td>✓ Addition of documents from the grey</td><td></td><td></td></tr><tr><td>Compile the corpus containing all references cited in the report</td><td>Final corpus</td><td>Bibliometric analysis published in the report</td></tr></tbody></table>
<table><caption>Table 2. Stages in the compilation of the corpus bibliographic</caption><thead><tr><th>Bibliographic queries</th><th>State of corpus</th><th>Use of corpus</th></tr></thead><tbody><tr><td>Search the WoS/Scopus databases using keywords identified in the referral and defined by the project leader, information officers and scientific leads</td><td>Exploratory corpus</td><td>Quantify the scientific articles published on the subject. Clarify the questions and scope of the ESCo. Identify potential experts</td></tr><tr><td>Improve queries based on keywords and selection criteria defined by the experts</td><td>Initial corpus</td><td>Distribution of the corpus among experts according to disciplines and issues addressed</td></tr><tr><td>Progressively refine the initial corpus in order to select/add relevant references to meet the project specifications</td><td></td><td></td></tr><tr><td>✓ Temporal limits of the bibliographic search (generally between 25 and 10 years prior)</td><td>Intermediate corpus</td><td>Bibliometric analysis of the corpus of each chapter to inform the discussion within the expert committee</td></tr><tr><td>✓ Identification of syntheses and meta-analyses that have already dealt with the scientific literature</td><td>Versions 1, 2, 3, 4, etc.</td><td>Qualitative diagnosis of the bibliographic corpus of the different chapters by the expert committee</td></tr><tr><td>✓ Elimination of irrelevant or redundant references</td><td>Validated version</td><td>Validation of the bibliographic corpus of the different chapters by the expert committee</td></tr><tr><td>✓ Addition of references, identified by the experts, that were not captured by the queries</td><td></td><td></td></tr><tr><td>✓ Addition of references derived from the literature review carried out during the project</td><td></td><td></td></tr><tr><td>✓ Addition of documents from the grey</td><td></td><td></td></tr><tr><td>Compile the corpus containing all references cited in the report</td><td>Final corpus</td><td>Bibliometric analysis published in the report</td></tr></tbody></table>
## 4.2. Creation of the bibliographic corpus

The creation of the bibliographic corpus follows a formalised working method based on a search of bibliographic databases using queries, then on successive refinements through exchanges between information officers and the experts. The tracking through the various steps guarantees the transparency of the approach and choices leading to the final bibliographic corpus, which will be cited in the scientific report.

### 4.2.1. Successive steps

Table 2 presents the general approach, from the first queries of the bibliographic databases to the assembly of the final corpus.

During the initial appraisal of the ESCo or advanced study, the information officers, the project leader and the scientific leads define an initial list of keywords based on the vocabulary of the referral and the main publications to which it refers. The exploratory corpus obtained from the queries permits an assessment of the state of the scientific literature on the subject. This exploration may lead to the refinement of the scientific questions raised by the referral, or even to a narrowing of the scope depending on the quantity of literature to be processed. The exploratory corpus also makes it possible to identify the main authors in the field likely to be called upon to form the expert committee. When the project begins, these initial queries are completed and adjusted based on word lists suggested by the experts according to their themes and disciplines. Particularly relevant articles provided by the experts help to target new keywords. At this stage, the corpus (or the sum of the corpora distributed among experts) is classified as initial, in the sense that it is the result of queries reworked by the entire committee of experts in order to collect as many relevant references as possible for the ESCo or the advanced study.

The experts (or chapter coordinators, depending on the structuring of the project) establish an initial diagnosis of this corpus by highlighting its scope and limits, and make some very general observations about the temporal dynamics, the diversity of research institutions, geographical areas, disciplines and research fields represented. This diagnosis also includes the main features of the strategy identified for selecting the articles to be analysed.

Once this initial corpus has been distributed among the experts, a sorting and refining phase begins. The objective is to select the relevant references in order to best answer the questions in the project specifications. To do this, the experts carry out an initial sorting based on the titles and abstracts of the articles. In the frequent case where the quantity of literature exceeds the analysis capacity of the committee of experts, several strategies aimed at limiting the number of references to be
## 4.2. Creation of the bibliographic corpus

The creation of the bibliographic corpus follows a formalised working method based on a search of bibliographic databases using queries, then on successive refinements through exchanges between information officers and the experts. The tracking through the various steps guarantees the transparency of the approach and choices leading to the final bibliographic corpus, which will be cited in the scientific report.

### 4.2.1. Successive steps

Table 2 presents the general approach, from the first queries of the bibliographic databases to the assembly of the final corpus.

During the initial appraisal of the ESCo or advanced study, the information officers, the project leader and the scientific leads define an initial list of keywords based on the vocabulary of the referral and the main publications to which it refers. The exploratory corpus obtained from the queries permits an assessment of the state of the scientific literature on the subject. This exploration may lead to the refinement of the scientific questions raised by the referral, or even to a narrowing of the scope depending on the quantity of literature to be processed. The exploratory corpus also makes it possible to identify the main authors in the field likely to be called upon to form the expert committee. When the project begins, these initial queries are completed and adjusted based on word lists suggested by the experts according to their themes and disciplines. Particularly relevant articles provided by the experts help to target new keywords. At this stage, the corpus (or the sum of the corpora distributed among experts) is classified as initial, in the sense that it is the result of queries reworked by the entire committee of experts in order to collect as many relevant references as possible for the ESCo or the advanced study.

The experts (or chapter coordinators, depending on the structuring of the project) establish an initial diagnosis of this corpus by highlighting its scope and limits, and make some very general observations about the temporal dynamics, the diversity of research institutions, geographical areas, disciplines and research fields represented. This diagnosis also includes the main features of the strategy identified for selecting the articles to be analysed.

Once this initial corpus has been distributed among the experts, a sorting and refining phase begins. The objective is to select the relevant references in order to best answer the questions in the project specifications. To do this, the experts carry out an initial sorting based on the titles and abstracts of the articles. In the frequent case where the quantity of literature exceeds the analysis capacity of the committee of experts, several strategies aimed at limiting the number of references to be
analysed are discussed: relate in particular to the nature of the articles (favouring literature reviews and meta-analyses that have already been published and limiting the exploration of primary articles to questions that are not covered by the former) and to the time frame of the bibliographic investigation (depending on the questions and scientific developments, the references may go back 25, 20, 15 or 10 years). In all cases, the ESCo or advanced study must provide an updated overview of current scientific knowledge, i.e. it must shed light on the most recent findings. The experts then gradually refine their selection by reading all the pre-screened references. They amend and complete each sub-corpus through interactions with the other experts and information officers. The refinement of the intermediate corpus thus continues throughout the entire project. The experts may also need to re-interrogate the bibliographic databases using additional queries (more targeted, specific to certain questions, etc.) or to query new bibliographic databases. The corpus thus goes through 'intermediate' states before being finalised.

The final corpus includes all bibliographic references cited in the ESCo or advanced study report.

#### 4.2.2. The main criteria for the selection of references

At the beginning of the project, each expert therefore has a list of references corresponding to the issue(s) he or she is responsible for dealing with. To give a rough idea of the size of the list, an expert generally reviews several hundred publications and selects a range of between 50 and 150.

The following references will be excluded from the corpus:

* not relevant (topic, geographical context);

* methodologically unreliable or not transferable to the context of this project.

The following are not given priority:

* redundant references, in particular certain earlier references whose results are repeated and updated in more recent articles;

* primary sources cited in reviews and meta-analysis, the latter being examined as a priority because they already constitute a synthetic analysis of the scientific literature.

Conversely, references not captured by the initial bibliographic queries can be added to the bibliographic corpus:

* references identified in the bibliographies of the publications obtained from the queries;

* articles collected by the information officers in the course of the project;
analysed are discussed: relate in particular to the nature of the articles (favouring literature reviews and meta-analyses that have already been published and limiting the exploration of primary articles to questions that are not covered by the former) and to the time frame of the bibliographic investigation (depending on the questions and scientific developments, the references may go back 25, 20, 15 or 10 years). In all cases, the ESCo or advanced study must provide an updated overview of current scientific knowledge, i.e. it must shed light on the most recent findings. The experts then gradually refine their selection by reading all the pre-screened references. They amend and complete each sub-corpus through interactions with the other experts and information officers. The refinement of the intermediate corpus thus continues throughout the entire project. The experts may also need to re-interrogate the bibliographic databases using additional queries (more targeted, specific to certain questions, etc.) or to query new bibliographic databases. The corpus thus goes through 'intermediate' states before being finalised.

The final corpus includes all bibliographic references cited in the ESCo or advanced study report.

#### 4.2.2. The main criteria for the selection of references

At the beginning of the project, each expert therefore has a list of references corresponding to the issue(s) he or she is responsible for dealing with. To give a rough idea of the size of the list, an expert generally reviews several hundred publications and selects a range of between 50 and 150.

The following references will be excluded from the corpus:

* not relevant (topic, geographical context);

* methodologically unreliable or not transferable to the context of this project.

The following are not given priority:

* redundant references, in particular certain earlier references whose results are repeated and updated in more recent articles;

* primary sources cited in reviews and meta-analysis, the latter being examined as a priority because they already constitute a synthetic analysis of the scientific literature.

Conversely, references not captured by the initial bibliographic queries can be added to the bibliographic corpus:

* references identified in the bibliographies of the publications obtained from the queries;

* articles collected by the information officers in the course of the project;
* references known to the experts through their research activities, in particular those from journals not indexed in international databases or from grey literature, as well as references relating to related fields and deemed essential to shed light on the issues under investigation.

Experts are expected to explain their reasons for rejecting or adding references, if possible involving a dual reading of titles, keywords and summaries. These arguments should be validated collectively. In particular, care must be taken to ensure that the process does not eliminate references that would give a complete picture of the scientific controversies on the issue in question. Similarly, additions may tend to broaden the subject beyond the specifications (by going into the field of the expert's research, for example), potentially biasing the answers given to the sponsors. The traceability and explicitness of the choices motivating the list of selected sources are therefore necessary to ensure the credibility of the results of the ESCo or advanced study.

**4.3. Qualitative and quantitative analyses of the bibliographic corpus**

The experts write a general statement on the bibliographic corpus by chapter: main subjects dealt with in the scientific literature, main journals publishing on the issue, most cited authors or institutions, existence of authoritative works in the field, geographic settings of the studies, evolution of the way in which science has dealt with the given questions, identified controversies or gaps, etc.

#### 4.3.1. A rigorous, common reading grid

The critical examination of the documentary sources follows the rules of scientific work. For each article selected, the experts assess the robustness of the methodologies, the suitability of the analytical tools used, and the rigour of the methods of interpretation and discussion of the results. Each expert defines this review framework according to the prevailing standards in his or her scientific discipline. The analysis of grey literature requires increased rigour and collective validation by the committee of experts.

In addition to these standard rules for analysing scientific literature, there is a specific reading grid for expert assessments: experts must clearly distinguish, among the scientific findings, those that can be considered as acquired, i.e. currently established and agreed upon. They must point out the uncertainties and variability that may affect certain results or conclusions.

They must explicitly explain disagreements between disciplines and authors. Finally, they must identify the gaps in scientific knowledge in relation to the questions posed by the sponsors. These four dimensions (achievements, uncertainties, controversies, knowledge gaps) are considered necessary by DEPE to fully clarify the state of scientific knowledge.
* references known to the experts through their research activities, in particular those from journals not indexed in international databases or from grey literature, as well as references relating to related fields and deemed essential to shed light on the issues under investigation.

Experts are expected to explain their reasons for rejecting or adding references, if possible involving a dual reading of titles, keywords and summaries. These arguments should be validated collectively. In particular, care must be taken to ensure that the process does not eliminate references that would give a complete picture of the scientific controversies on the issue in question. Similarly, additions may tend to broaden the subject beyond the specifications (by going into the field of the expert's research, for example), potentially biasing the answers given to the sponsors. The traceability and explicitness of the choices motivating the list of selected sources are therefore necessary to ensure the credibility of the results of the ESCo or advanced study.

**4.3. Qualitative and quantitative analyses of the bibliographic corpus**

The experts write a general statement on the bibliographic corpus by chapter: main subjects dealt with in the scientific literature, main journals publishing on the issue, most cited authors or institutions, existence of authoritative works in the field, geographic settings of the studies, evolution of the way in which science has dealt with the given questions, identified controversies or gaps, etc.

### 4.3.1. A rigorous, common reading grid

The critical examination of the documentary sources follows the rules of scientific work. For each article selected, the experts assess the robustness of the methodologies, the suitability of the analytical tools used, and the rigour of the methods of interpretation and discussion of the results. Each expert defines this review framework according to the prevailing standards in his or her scientific discipline. The analysis of grey literature requires increased rigour and collective validation by the committee of experts.

In addition to these standard rules for analysing scientific literature, there is a specific reading grid for expert assessments: experts must clearly distinguish, among the scientific findings, those that can be considered as acquired, i.e. currently established and agreed upon. They must point out the uncertainties and variability that may affect certain results or conclusions.

They must explicitly explain disagreements between disciplines and authors. Finally, they must identify the gaps in scientific knowledge in relation to the questions posed by the sponsors. These four dimensions (achievements, uncertainties, controversies, knowledge gaps) are considered necessary by DEPE to fully clarify the state of scientific knowledge.
The review of the literature in an ESCo or advanced study is part of a progressive and collective approach to assessing scientific results. In this sense, it differs from systematic review or meta-analysis approaches. Indeed, the latter rely on a process of systematically screening publications using a pre-established analysis grid, which provides guarantees of transparency and rigour. However, the implementation of such an analytical process is generally not sufficient to meet the expectations of an ESCo or advanced study given the broad nature of their scope, questions and criteria for analysis.

### 4.3.2. Use of bibliometric analysis

The rapid increase in the number of scientific articles published each year increases the difficulty of the corpus selection process as well as the critical synthesis work required of experts. Automatic processing methods can therefore be used to describe and analyse the corpus. The descriptions are based on information referenced in bibliographic databases.

The WoS platform provides tools to analyse the frequency of certain indicators such as the thematic fields of journals, the titles of publications, the authors and their areas of affiliation, institutions, funding sources, etc.

ITextual analyses can be performed for scientific articles by looking at words in the title and abstract, keywords, authors, their institutions of affiliation, etc. and performing statistical analysis. Time sequence analyses offer a retrospective look at how the literature is evolving. The results are usually represented in graphical form, which allows the characteristics of a large number of texts to be quickly visualised. Using measures of term co-occurrence, it is possible to construct collaborative networks between authors or thematic networks. DEPE is currently using WoS tools and the digital platform CorTexT Manager<fnref n="12" />, which combine language processing, information extraction, complex network analysis and scientometrics.

Bibliometric analysis tools are used for several purposes, depending on the stage of the project. At the start of the project (initial corpus), they are useful for highlighting the main features of the scientific literature on the subject to be dealt with in order to initiate the discussion within the committee of experts. They can also facilitate the distribution of the overall corpus among the experts according to the topics they are to deal with. The analyses can also lead to new bibliographical requests or more detailed investigations on a given aspect. As the work progresses, the bibliographic analyses make it possible to objectify the evolution of the corpus and to identify

<footnote n="12">¹² https://www.cortex.net/projects/cortex-manager/</footnote>
The review of the literature in an ESCo or advanced study is part of a progressive and collective approach to assessing scientific results. In this sense, it differs from systematic review or meta-analysis approaches. Indeed, the latter rely on a process of systematically screening publications using a pre-established analysis grid, which provides guarantees of transparency and rigour. However, the implementation of such an analytical process is generally not sufficient to meet the expectations of an ESCo or advanced study given the broad nature of their scope, questions and criteria for analysis.

### 4.3.2. Use of bibliometric analysis

The rapid increase in the number of scientific articles published each year increases the difficulty of the corpus selection process as well as the critical synthesis work required of experts. Automatic processing methods can therefore be used to describe and analyse the corpus. The descriptions are based on information referenced in bibliographic databases.

The WoS platform provides tools to analyse the frequency of certain indicators such as the thematic fields of journals, the titles of publications, the authors and their areas of affiliation, institutions, funding sources, etc.

ITextual analyses can be performed for scientific articles by looking at words in the title and abstract, keywords, authors, their institutions of affiliation, etc. and performing statistical analysis. Time sequence analyses offer a retrospective look at how the literature is evolving. The results are usually represented in graphical form, which allows the characteristics of a large number of texts to be quickly visualised. Using measures of term co-occurrence, it is possible to construct collaborative networks between authors or thematic networks. DEPE is currently using WoS tools and the digital platform CorTexT Manager<fnref n="12" />, which combine language processing, information extraction, complex network analysis and scientometrics.

Bibliometric analysis tools are used for several purposes, depending on the stage of the project. At the start of the project (initial corpus), they are useful for highlighting the main features of the scientific literature on the subject to be dealt with in order to initiate the discussion within the committee of experts. They can also facilitate the distribution of the overall corpus among the experts according to the topics they are to deal with. The analyses can also lead to new bibliographical requests or more detailed investigations on a given aspect. As the work progresses, the bibliographic analyses make it possible to objectify the evolution of the corpus and to identify

<footnote n="12">¹² https://www.cortex.net/projects/cortex-manager/</footnote>
any selection biases in terms of both themes and communities of authors. Particular attention is paid to self-citations and the relative share of the different networks of authors cited.

At the end of the project, bibliometric analyses applied to the final corpus provide key information on the sources on which the ESCo or advanced study is based. They represent a significant result from the ESCo or advanced study, as they shed light on the current state of scientific knowledge. Conducted by the information officers and included in the final report, the analyses cover the following items:

* the category of documents: academic articles, reviews, conference proceedings, statistics, reports, etc;

* the temporal distribution of the cited references (it is expected that they are as up-to-date as possible). This analysis can be further developed by describing the temporal evolution of research questions on the subject;

* the allocation between themes and the number of references per chapter;

* the principal authors cited by chapter;

* the comparison between the final corpus and the initial corpus, with regard to the main reasons for the orientations, deletions or additions.

Finally, the keywords and/or thematic descriptors of the references are indicated (usually in the appendix of the report) so that the queries can be reproduced.
any selection biases in terms of both themes and communities of authors. Particular attention is paid to self-citations and the relative share of the different networks of authors cited.

At the end of the project, bibliometric analyses applied to the final corpus provide key information on the sources on which the ESCo or advanced study is based. They represent a significant result from the ESCo or advanced study, as they shed light on the current state of scientific knowledge. Conducted by the information officers and included in the final report, the analyses cover the following items:

* the category of documents: academic articles, reviews, conference proceedings, statistics, reports, etc;

* the temporal distribution of the cited references (it is expected that they are as up-to-date as possible). This analysis can be further developed by describing the temporal evolution of research questions on the subject;

* the allocation between themes and the number of references per chapter;

* the principal authors cited by chapter;

* the comparison between the final corpus and the initial corpus, with regard to the main reasons for the orientations, deletions or additions.

Finally, the keywords and/or thematic descriptors of the references are indicated (usually in the appendix of the report) so that the queries can be reproduced.
<figure><img src="image_11.png" /></figure>

## CHAPTER 5. PREPARATION OF DELIVERABLES AND DISSEMINATION OF ESCO RESULTS AND ADVANCED STUDIES

ESCo and advanced study deliverables do not include opinions or recommendations, unlike those of risk assessment agencies such as health agencies, which are usually required to issue them. These studies yield three types of deliverables, as shown in Figure 1:

1. the scientific report, which is often extensive (500 to 1 000 pages), brings together all contributions and critical analyses written by the experts on the basis of the bibliographic corpus (Chapter 4) as well as the list of references cited. In the case of advanced studies, it also includes a description of the data processing and assembly methodologies developed by the experts, as well as details of the results of their implementation;

2. the condensed report (approximately 100 pages) presents the main findings and results of the report. In addition to those who commissioned the project (political leaders and decision-makers, ministry or agency officials), it is intended for all societal stakeholders concerned and/or interested in the issue (associations, professional organisations, industry stakeholders, etc.);

3. the summary report (usually around ten pages) communicates the main conclusions of the work more widely.

Deliverables can be checked with anti-plagiarism software.

The results and conclusions of ESCos and advanced studies are always made public, and are discussed with stakeholders at a symposium open to anyone. The condensed report and summary reports are then disseminated. Finally, the scientific exploitation of the results in the form of academic publications is also a major objective of these projects, and is envisaged and planned from the outset of the project.

### 5.1. Extended report

The drafting of the scientific report is the responsibility of the scientific leads and the experts. The outline of the scientific report is developed collectively by the experts.

Each expert writes a contribution that provides a critical synthesis of the elements that they have extracted from the bibliographic corpus that has been assigned to them.
<figure><img src="image_13.png" /></figure>

## CHAPTER 5. PREPARATION OF DELIVERABLES AND DISSEMINATION OF ESCO RESULTS AND ADVANCED STUDIES

ESCo and advanced study deliverables do not include opinions or recommendations, unlike those of risk assessment agencies such as health agencies, which are usually required to issue them. These studies yield three types of deliverables, as shown in Figure 1:

1. the scientific report, which is often extensive (500 to 1 000 pages), brings together all contributions and critical analyses written by the experts on the basis of the bibliographic corpus (Chapter 4) as well as the list of references cited. In the case of advanced studies, it also includes a description of the data processing and assembly methodologies developed by the experts, as well as details of the results of their implementation;

2. the condensed report (approximately 100 pages) presents the main findings and results of the report. In addition to those who commissioned the project (political leaders and decision-makers, ministry or agency officials), it is intended for all societal stakeholders concerned and/or interested in the issue (associations, professional organisations, industry stakeholders, etc.);

3. the summary report (usually around ten pages) communicates the main conclusions of the work more widely.

Deliverables can be checked with anti-plagiarism software.

The results and conclusions of ESCos and advanced studies are always made public, and are discussed with stakeholders at a symposium open to anyone. The condensed report and summary reports are then disseminated. Finally, the scientific exploitation of the results in the form of academic publications is also a major objective of these projects, and is envisaged and planned from the outset of the project.

### 5.1. Extended report

The drafting of the scientific report is the responsibility of the scientific leads and the experts. The outline of the scientific report is developed collectively by the experts.

Each expert writes a contribution that provides a critical synthesis of the elements that they have extracted from the bibliographic corpus that has been assigned to them.
<figure><img src="image_12.png" /><figcaption>Figure 1. Schematic flow of the preparation of the extended, condensed, and summary reports of an ESCo or advanced study</figcaption></figure>

A contribution must explicitly summarise (i) the current state of knowledge, indicating the level of evidence or uncertainty, (ii) the scientific debates identified in the literature, (iii) and the gaps in scientific knowledge in relation to the issue explored. The review concludes with key findings from the analysis and the formulation of possible research and/or data needs. In the case of advanced studies, the bibliographical analysis is accompanied by specific statistical analyses or simulations carried out in addition to the bibliographical analysis. The project leader and the scientific leads coordinate the process of drafting each contribution and ensure, together with the information officers, that each statement is well supported by evidence.

The extended report generally adheres to the following standard structure:

* The introduction, which may be extended by a framing chapter, presents the context of the ESCo or advanced study request and the issues that motivated the request. It also specifies the work process, the composition of the committee, the results of the examination of the links of interest and the highlights of the bibliometric analysis (number of references, most cited journals, etc.). The introduction is usually handled by the project leader and the scientific leads.

* The following chapters are organised around the scientific issues addressed by the project. Each chapter begins with a review of the bibliography and concludes with the key aspects of the various contributions to the chapter. Each expert first writes a short summary which is discussed at the expert committee meeting. On this basis, the expert writes a contribution of approximately 10 to 20 pages with explicit links to the references selected to support the analysis. Cross-reading of the contributions by the project
<figure><img src="image_14.png" /><figcaption>Figure 1. Schematic flow of the preparation of the extended, condensed, and summary reports of an ESCo or advanced study</figcaption></figure>

A contribution must explicitly summarise (i) the current state of knowledge, indicating the level of evidence or uncertainty, (ii) the scientific debates identified in the literature, (iii) and the gaps in scientific knowledge in relation to the issue explored. The review concludes with key findings from the analysis and the formulation of possible research and/or data needs. In the case of advanced studies, the bibliographical analysis is accompanied by specific statistical analyses or simulations carried out in addition to the bibliographical analysis. The project leader and the scientific leads coordinate the process of drafting each contribution and ensure, together with the information officers, that each statement is well supported by evidence.

The extended report generally adheres to the following standard structure:

* The introduction, which may be extended by a framing chapter, presents the context of the ESCo or advanced study request and the issues that motivated the request. It also specifies the work process, the composition of the committee, the results of the examination of the links of interest and the highlights of the bibliometric analysis (number of references, most cited journals, etc.). The introduction is usually handled by the project leader and the scientific leads.

* The following chapters are organised around the scientific issues addressed by the project. Each chapter begins with a review of the bibliography and concludes with the key aspects of the various contributions to the chapter. Each expert first writes a short summary which is discussed at the expert committee meeting. On this basis, the expert writes a contribution of approximately 10 to 20 pages with explicit links to the references selected to support the analysis. Cross-reading of the contributions by the project
leader, the scientific leads, the other experts and the information officers
results in the content being reworked until it is finally formulated.

* The conclusions of the report, which generally take the form of a final chapter, offer a transversal overview of the analytical chapters in response to the sponsors' request. The conclusions are therefore expressed in a concise manner (in the order of 10 to 20 pages) and refer to the various sections of the report as necessary. The scientific leads generally draft the first version of the conclusions, which are eventually validated by the entire expert committee.

* The appendices include the bibliometric analysis of the cited corpus as well as any other methodological contributions, data or complementary analyses.

* The report includes a description of the entire work group (names, contact details, roles): scientific leads, members of the expert committee, possible one-off scientific contributors, the project leader, information officers, managers, and other participants.

There are two ways of citing an extended report depending on the manner in which it was produced:

* Scientific leaders (coord.), experts in alphabetical order / information officers / project leaders (coord.) (date). Title of the document. ESCo/advanced study report, INRAE - possible partners (France), xxx pages.

* Scientific leaders (coord.), project leader (coord.), experts in alphabetical order / information officers / project leaders (date). Title of the document. ESCo/study report, INRAE - possible partners (France), xxx pages.

## 5.2. Condensed report

The condensed report goes beyond simply presenting the general conclusions of the extended report. It also provides an account of the approach and the more sector-specific conclusions of the analysis. It can also introduce generic elements that are useful to help the non-specialist reader understand the reasoning presented. The condensed report should be readable without the need to refer to the extended report. In fact, it provides an entry point to the extended report, with the reader consulting the sections of the latter that he or she wishes to explore further. The synthesis cites a limited number of bibliographical references.

The condensed report is published in French and English, except in cases where the experts publish a peer-reviewed scientific article, based on the condensed report, in an internationally refereed journal<fnref n="13" />.

<footnote n="13">¹³ In this case, releasing the English version of the condensed report may prejudice the submission of a scientific paper based on it.</footnote>
leader, the scientific leads, the other experts and the information officers
results in the content being reworked until it is finally formulated.

* The conclusions of the report, which generally take the form of a final chapter, offer a transversal overview of the analytical chapters in response to the sponsors' request. The conclusions are therefore expressed in a concise manner (in the order of 10 to 20 pages) and refer to the various sections of the report as necessary. The scientific leads generally draft the first version of the conclusions, which are eventually validated by the entire expert committee.

* The appendices include the bibliometric analysis of the cited corpus as well as any other methodological contributions, data or complementary analyses.

* The report includes a description of the entire work group (names, contact details, roles): scientific leads, members of the expert committee, possible one-off scientific contributors, the project leader, information officers, managers, and other participants.

There are two ways of citing an extended report depending on the manner in which it was produced:

* Scientific leaders (coord.), experts in alphabetical order / information officers / project leaders (coord.) (date). Title of the document. ESCo/advanced study report, INRAE - possible partners (France), xxx pages.

* Scientific leaders (coord.), project leader (coord.), experts in alphabetical order / information officers / project leaders (date). Title of the document. ESCo/study report, INRAE - possible partners (France), xxx pages.

## 5.2. Condensed report

The condensed report goes beyond simply presenting the general conclusions of the extended report. It also provides an account of the approach and the more sector-specific conclusions of the analysis. It can also introduce generic elements that are useful to help the non-specialist reader understand the reasoning presented. The condensed report should be readable without the need to refer to the extended report. In fact, it provides an entry point to the extended report, with the reader consulting the sections of the latter that he or she wishes to explore further. The synthesis cites a limited number of bibliographical references.

The condensed report is published in French and English, except in cases where the experts publish a peer-reviewed scientific article, based on the condensed report, in an internationally refereed journal<fnref n="13" />.

<footnote n="13">¹³ In this case, releasing the English version of the condensed report may prejudice the submission of a scientific paper based on it.</footnote>
The project leader drafts the condensed report in close collaboration with the scientific leads and the project managers. The experts amend the proposed plan and then the intermediate versions, and validate the final version. The drafting process begins as soon as the general conclusions of the report are formulated.

A very advanced working version is reviewed by:

* the sponsors, for an opinion on the clarity of the document, its relevance to the questions in the project specifications, and the appropriateness of its content for the purpose of informing public decision-making. The sponsors do not intervene in the content of either the analyses or the conclusions;

* INRAE scientific directors who monitor the project, and if necessary, their counterparts in the partner organisations, in order to access the contents as the public presentation of the results approaches, and;

* usually, by scientific reviewers from outside the working group in the interests of scientific rigour.

The authors of the condensed report are not compelled to accept the remarks of the reviewers but, if they do, they must justify why.

The project leader, the scientific leads, the members of the expert committee, the information officers and the project managers are the authors of the condensed report, except in special cases. DEPE is responsible for its publication. The condensed report is published by Editions Quæ in a version that is often revised in French and/or English (with the possibility of also engaging an English publisher).

The condensed report is cited (general case) as follows:

Project leader (coord.), scientific leads (coord.), experts and other members of the working group who contributed to the synthesis in alphabetical order (date). Title of the document. Condensed report of the ESCo/study report, INRAE - possible partners (France), xxx pages.
The project leader drafts the condensed report in close collaboration with the scientific leads and the project managers. The experts amend the proposed plan and then the intermediate versions, and validate the final version. The drafting process begins as soon as the general conclusions of the report are formulated.

A very advanced working version is reviewed by:

* the sponsors, for an opinion on the clarity of the document, its relevance to the questions in the project specifications, and the appropriateness of its content for the purpose of informing public decision-making. The sponsors do not intervene in the content of either the analyses or the conclusions;

* INRAE scientific directors who monitor the project, and if necessary, their counterparts in the partner organisations, in order to access the contents as the public presentation of the results approaches, and;

* usually, by scientific reviewers from outside the working group in the interests of scientific rigour.

The authors of the condensed report are not compelled to accept the remarks of the reviewers but, if they do, they must justify why.

The project leader, the scientific leads, the members of the expert committee, the information officers and the project managers are the authors of the condensed report, except in special cases. DEPE is responsible for its publication. The condensed report is published by Editions Quæ in a version that is often revised in French and/or English (with the possibility of also engaging an English publisher).

The condensed report is cited (general case) as follows:

Project leader (coord.), scientific leads (coord.), experts and other members of the working group who contributed to the synthesis in alphabetical order (date). Title of the document. Condensed report of the ESCo/study report, INRAE - possible partners (France), xxx pages.