CoARA French Chapter

Fabien Borget, Armelle Regnault, Nicolas Fressengeas

► To cite this version:

Fabien Borget, Armelle Regnault, Nicolas Fressengeas. CoARA French Chapter. 2023. [https://hal.hal.science/hal-04264538v1](https://hal.hal.science/hal-04264538v1)

HAL Id: hal-04264538

[https://amu.hal.science/hal-04264538v1](https://amu.hal.science/hal-04264538v1)

Submitted on 30 Oct 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
<figure><img src="image_2.png" /></figure>

MINISTÈRE
DE L'ENSEIGNEMENT
SUPÉRIEUR
ET DE LA RECHERCHE

Liberté
Égalité
Fraternité

(Aix*Marseille
université
Socialement engagée

Inserm

La science pour la santé

From science to health

# CoARA French Chapter

Fabien BORGET & Armelle REGNAULT
Nicolas FRESSENGEAS

<figure><img src="image_3.png" /></figure>
# French assessment

## Different types of assessments at different level by different actors

* HCERES

* Universities (hiring, promotions, projects ...)

* NROs

* National by CNU (National Council of Universities) : half of the individual promotions

* Projects

<figure><img src="image_4.png" /></figure>

<figure><img src="image_5.png" /></figure>

<figure><img src="image_6.png" /></figure>

<figure><img src="image_7.png" /></figure>

<figure><img src="image_8.png" /></figure>

<figure><img src="image_9.png" /></figure>

Over a hundred ways of performing assessment
Coordination on its way
# Individual Assessment: Local and Nationwide

## Hiring in universities : two uncorrelated steps

* Discipline specific nationwide step

* Position specific local step

## Promotion in universities : two steps with the same file

* Discipline specific nationwide step

* Trans-discipline local step

## National research organizations

* Each has its own assessment process
# Collective assessment

HCERES : (Haut Conseil de l'évaluation de la recherche et de l'enseignement supérieur, High Council for Research and Higher Education assessment)

* Nationwide

* Assesses all collective research organizations

* Great tool for coordination
# Project assessment

## National funding agencies diversity

* Leader : Agence Nationale de la Recherche (ANR)

* Common roadmap towards Open Science

* Assessment coordination on a good start

## University or national Research organisation specific projects

Each has its own specific assessment

* Sometimes internally

* Sometimes with network

* Sometimes international

* Sometimes by foundation ...
# WG DORA creation

**Why:**

2021 creation of a specific WG for OSEC during the French presidency of the EU

Discussion between different actors

Political group

**Who:** institutions who signed DORA or will do

Universities

NROs

Funding Agencies

Evaluation agencies

**How:**

Discussion and presentation of each usage
Specialists interventions.

<figure><img src="image_10.png" /></figure>

MINISTÈRE
DE L'ENSEIGNEMENT
SUPÉRIEUR
ET DE LA RECHERCHE

Liberté
Égalité
Fraternité

<figure><img src="image_11.png" /></figure>

Signatory of
DORA

## Actual Pilots

<figure><img src="image_12.png" /><figcaption>Armelle Regnault, INSERM</figcaption></figure>

<figure><img src="image_13.png" /><figcaption>Fabien Borget, AMU</figcaption></figure>

25 members, from NROs, agencies, Universities

<figure><img src="image_14.png" /></figure>

<figure><img src="image_15.png" /></figure>

anr
agence nationale
de la recherche
AU SERVICE DE LA SCIENCE

**Inserm**
La science pour la santé
From science to health

<figure><img src="image_16.png" /></figure>

<figure><img src="image_17.png" /></figure>

Previous Pilots: Cherifa Boukacem (Université Lyon 1) and Frédéric Gaymard (INRAE)
# WG DORA delivrables

* ✓ Paris Call text : OSEC2022

* ✓ Needs to become operational

* ✓ Evolution into the new framework of CoARA

* ✓ Two DORA WG members were in the Core group to design CoARA texts : Sylvie Rousset (in CoARA board) and Bernard Larrouturou.

OSEC 2022
Open Science European Conference

## Paris Call on Research Assessment

This text was prepared by the French Open Science Committee and presented to the Paris Open Science European Conference (OSEC) held in Paris on 4th and 5th February 2022, organised in the context of the French Presidency of the Council of the European Union, following the publication of the UNECE recommendation on Open Science and the publication by the European Commission of a reform of the research assessment system: scope, report.

In the conclusions of its meeting of December 1st 2020 on the New European Research Area, the Competitiveness Council of the European Union highlighted that Open Science has a crucial role in boosting impact, quality, efficiency, transparency and integrity of research and innovation, and brings science and society closer together. The Council emphasised that bibliodiversity and multilingualism and the acknowledgement of all scientific productions are relevant elements of an European Research Area policy on Open Science.

The current system for assessing research, researchers and research institutions, however, does not incentivise or reward enough the quality of all research outputs in their diversity. It often relies on the quantity of publications in journals with high journal impact Factor and citations as mere proxies for quality and impact, thereby underestimating the value of other contributions, lowering reproducibility and holding back researchers from open sharing and collaboration.

The Open Science European Conference (OSEC) 2022, under the auspices of the French Presidency of the Council of the European Union:

* Recognises that openness improves the quality, efficiency and impact of research, and fosters team science;

* Reaffirms the need to align what we assess with what we value;

* Calls for an assessment system where research proposals, researchers, research units and research institutions are evaluated on the basis of their intrinsic merits and impact, rather than on the number of publications and where they are published, promoting qualitative judgment provided by peers, supported by a responsible use of quantitative indicators;

* Calls therefore for a research assessment system that:

* rewards quality and the various impacts of research;

* ensures that research meets the highest standards of ethics and integrity;

* values the diversity of research activities and outputs such as publications and preprints, data, methods, software, code and patents, as well as their societal impacts and activities related to training, innovation and public engagement;

* uses assessment criteria and processes that respect the variety of research disciplines;

* rewards not only research outputs, but also the appropriate conduct of research, and values good practices, in particular open practices for sharing research results and methodologies whenever possible;

* values collaborative work, as well as cross-disciplinarity and citizen science, when appropriate;

* supports a diversity of researcher profiles and career paths.

* Calls for the creation of a coalition of research funding organisations, research performing organisations, and assessment authorities, willing and committed to reform the current research assessment system along commonly agreed objectives, principles and actions (such as mutual learning, shared documentation and commonly agreed monitoring effort). The success of such a coalition will be deeply connected to its capacity to prepare concrete implementation processes and to its capacity to associate and involve researchers at all levels.

<footnote>[1] https://unesdoc.unesco.org/ark:/48223/pt/000379949.localen</footnote>

<footnote>[1] https://data.europa.eu/eurol/10.2777/70f440</footnote>
# CoARA - France

<figure><img src="image_18.png" /></figure>

## CoARA

### French member in CoARA board:

* Sylvie Rousset, DDOR

### French signatories
August 31st 2023

* 32 universities

* 4 schools or schools association

* 13 NROs, agencies and associations (including FU, UDICE, CNRS, ANR, INSERM ...)

### National Structuration :

* Submission of the national chapter

### In forthcoming weeks:

* Coordination of CoARA-EU WG participation

* Operational: exchanges of good practices, web site

* Dialogue avec CoARA-EU
# Creation of the CoARA French Chapter

<figure><img src="image_19.png" /></figure>

**Base:** DORA WG

**Socle:** nationwide multi-organisations

**Minister as facilitator**

Letters to the signatories governance to propose member for the chapter

<figure><img src="image_20.png" /></figure>

RÉPUBLIQUE
FRANÇAISE

Liberté
Egalité
Fraternité

Since CoARA creation, we extended the members of the DORA WG, accepting any signatories who made the request

Larger scope of the French chapter

More participants (49 signatories)

**Operational:** CoARA WG are in progress
# Roadmap

<figure><img src="image_21.png" /></figure>

1. Identify institutional needs and help generating resources to meet them

2. Exchange of information between members,

3. Exchanges of information on ongoing work at European level,

4. Encourage the development of experiments,

5. Play the role of a network of actors to create synergy between specialists to avoid isolation in their functions

6. Follow-up, at the French level, the actions carried out in the establishments and organizations, in relation to the CoARA commitments
# Work plan for the French Chapter

<figure><img src="image_22.png" /></figure>

1. Presentation of the state of the art with a public display (State of the art in each institution plus a summary of the state of the art in France on the COARA FRANCE website)

2. List of resources and skills available in each institution to consider training of evaluators

3. Training of trainers

4. Creation of a space for sharing resources (COARA France website)

5. Translation of the COARA EU documents into French so that each institution can appropriate them.

6. Conceive guidelines on the ethics of evaluation for every stakeholders
# Actions

<figure><img src="image_23.png" /></figure>

## Place to discuss and diffuse the results of CoARA

* ✓ Web site for diffusion in coordination with CoARA secretriat ESF

* ✓ Diffusion of the CoARA documents

* ✓ Translation of CoARA documents

* ✓ Results of the French Chapter discussion

## Place to coordinate and exchange on the CoARA WG contents

* ✓ Having French members in each CoARA WG

* ✓ Could propose some mirror WG at national level

* ✓ Could have specific discussion for national implementation

* ✓ Thinking shared guidelines based on CoARA commitments
# Points for attention

**Identify roadblocks** in the process and find solutions to surpass them

CoARA commitments: Integrity and data sharing -> **Research on Research** have to be taken into account

Assessment is at the heart of **extremely complex system**

Keeping a **bottom-up process**

Taking care of **national laws**: use a comparative law methodology, coordination with CoARA WG

**Reviewer fatigue**: taking care of the growing numbers of evaluations, time consuming and evaluation rhythm

Include CoARA commitments in HRS4R policy ?
# Thank you for your attention

## Staying available for any question

Armelle.Regnault@inserm.fr

Fabien.Borget@univ-amu.fr