Report of the Working Group on Electronic Lab
Notebooks

Gilles Mathieu, Tovo Rabemanantsoa, Christophe Chipeaux, Simon Duvillard, Jean-François Peyrat, Célya Gruson-Daniel, Marie-Emilia Herbet, Moussa Seydi, Véronique Theisen, Agnès Pinet, et al.

<figure><img src="image_1.png" /></figure>

To cite this version:

Gilles Mathieu, Tovo Rabemanantsoa, Christophe Chipeaux, Simon Duvillard, Jean-François Peyrat, et al.. Report of the Working Group on Electronic Lab Notebooks: French Committee for Open Science. Comité pour la science ouverte. 2021. hal-03959882

HAL Id: hal-03959882

https://hal-lara.archives-ouvertes.fr/hal-03959882

Submitted on 27 Jan 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution - ShareAlike 4.0 International License
<figure><img src="image_3.png" /></figure>

Ouvrir
la science !

Report of the
Working Group on
Electronic Lab
Notebooks

French Committee for
Open Science

September 2021

<figure><img src="image_4.png" /></figure>
Report of the Working
Group on Electronic Lab
Notebooks

French Committee for Open Science

Tovo Rabemanantsoa - Pilote
INRAE

Dominique Pigeon - Pilote
Inserm

Gilles Mathieu - Pilote
Inserm

Christophe CHIPEAUX
ICOS/INRAE

Simon DUVILLARD
Réseau CURIE

Célya GRUSON-DANIEL
Université Lyon I

Marie HERBET
Université Lyon I

Arnaud LEGRAND
INRIA/CNRS

Nathalie LEON
CNRS

Domenico LIBRI
CNRS/INSB

Jean-Baptiste LILY
Université Lyon I
Jean-François PEYRAT
Université Paris Saclay

Agnès PINET
CEA

François SABOT
IRD

Moussa SEYDI
IRD

Véronique THEISEN
CEA

This document is the translation of a report first published in French :

Gilles Mathieu, Dominique Pigeon, Tovo Rabemanantsoa, Christophe Chipeaux, Simon Duvillard, et al.. Rapport du groupe de travail sur les cahiers de laboratoires électroniques. [Rapport de recherche] Comité pour la science ouverte. 2021, 68 p.,
DOI : [10.52949/3](https://doi.org/10.52949/3)

---

September 2021

---

Conception graphique : opixido

<figure><img src="image_5.png" /></figure>
# Table of contents

<table><tr><td>1</td><td>Document background and objectives</td><td>6</td></tr><tr><td>1.1</td><td>Background</td><td>6</td></tr><tr><td>1.2</td><td>The working group's objective</td><td>6</td></tr><tr><td>1.3</td><td>Group composition</td><td>7</td></tr><tr><td>1.4</td><td>How the WG worked</td><td>8</td></tr><tr><td>1.5</td><td>Purpose of the report</td><td>9</td></tr><tr><td>2</td><td>Definition and scope: a shared vision of the electronic lab notebook</td><td>10</td></tr><tr><td>2.1</td><td>Definition</td><td>10</td></tr><tr><td>2.2</td><td>Scope</td><td>10</td></tr><tr><td>3</td><td>Recommendations for interoperability between different tools</td><td>13</td></tr><tr><td>3.1</td><td>Interoperability</td><td>13</td></tr><tr><td>3.2</td><td>Recommendations</td><td>13</td></tr><tr><td>3.3</td><td>Summary</td><td>15</td></tr><tr><td>4</td><td>Recommendations for the tool selection procedure</td><td>16</td></tr><tr><td>4.1</td><td>Needs</td><td>16</td></tr><tr><td>4.2</td><td>Establish the selection criteria</td><td>20</td></tr><tr><td>4.3</td><td>Identify commercial options</td><td>23</td></tr><tr><td>4.4</td><td>Assess options based on stated criteria</td><td>25</td></tr><tr><td>5</td><td>Comparative list of tools</td><td>31</td></tr><tr><td>5.1</td><td>Selection of tools studied</td><td>31</td></tr><tr><td>5.2</td><td>Results</td><td>33</td></tr><tr><td>5.3</td><td>Comments</td><td>33</td></tr><tr><td>6</td><td>Successfully conducting an ELN implementation project</td><td>36</td></tr><tr><td>6.1</td><td>Introduction</td><td>36</td></tr><tr><td>6.2</td><td>Project stakeholders and leadership</td><td>36</td></tr><tr><td>6.3</td><td>Pre-project</td><td>36</td></tr><tr><td>6.4</td><td>Pilot phase</td><td>37</td></tr><tr><td>6.5</td><td>Call for proposals</td><td>37</td></tr><tr><td>6.6</td><td>Deployment</td><td>37</td></tr><tr><td>6.7</td><td>Change of tools</td><td>37</td></tr><tr><td>6.8</td><td>Change management</td><td>38</td></tr><tr><td>6.9</td><td>HR profiles and scaling</td><td>40</td></tr><tr><td>6.10</td><td>Joint supervision</td><td>40</td></tr><tr><td>7</td><td>Conclusion</td><td>40</td></tr><tr><td>A</td><td>Appendices</td><td>42</td></tr><tr><td>A.1</td><td>Non-WG contributors</td><td>42</td></tr><tr><td>A.2</td><td>WG mission statement</td><td>42</td></tr></table>
<table><tr><td>1.</td><td>Background</td><td>45</td></tr><tr><td>2.</td><td>Objectives and deliverables</td><td>45</td></tr><tr><td>3.1.</td><td>Group composition</td><td>45</td></tr><tr><td>3.2.</td><td>How it works</td><td>46</td></tr><tr><td>A.3</td><td>Detailed look at uses - observational approach</td><td>47</td></tr><tr><td>A.4</td><td>Description of the criteria</td><td>50</td></tr><tr><td>A.5</td><td>List of tools analysed and not analysed</td><td>56</td></tr><tr><td>B</td><td>Glossary and abbreviations</td><td>60</td></tr></table>
# 1 Document background and objectives

## 1.1 Background

The laboratory notebook is a logbook used to record the day-to-day activities of research projects. It serves to track experiment descriptions and employed protocols, as well as log individual contributions. Using it ensures that requirements in terms of quality and scientific integrity are met by guaranteeing the traceability of the scientific methods employed and the reproducibility of research data and results. It therefore provides patent offices with proof of an invention, of its inventors, and thus of its rightholders.

The electronic lab notebook is the dematerialised digital version of the lab notebook. The term "electronic", often replaced by "digital" in other fields, is the most frequently used adjective for lab notebooks.

Due to its crucial place in research activities and its role in the management and protection of scientific knowledge, the electronic lab notebook is an essential strategic tool that is fully in line with open science. Moreover, it must meet the same scientific objectives and challenges as the physical notebook, which is why the choice of tool, its configuration, and how it is used are so important.

In order to inform scientific teams in choosing between the various solutions currently available, a working group was set up as part of the Research Data College of the Committee for Open Science within the French Ministry of Research and Higher Education.

**Terminology:** In the remainder of this document and in a ll the work produced by the working group, the Electronic Lab Notebook is referred to by the acronym ELN.

## 1.2 The working group's objective

The group's mission statement (in the appendices) specifies the objective of the working group, which is to propose an analysis method to scientific teams that have to choose an electronic lab notebook to meet their specific needs. To carry out this study, the working group defined the following objectives:

* Establish a shared vision of the definition, framing, uses and functional scope of the "electronic lab notebook" (see section 2).

* Produce a set of recommendations for interoperability between different electronic lab notebook tools and other tools or information systems already in use. Laboratory notebooks must be able to seamlessly merge with existing computer environments (including data
repositories), especially in a French academic multi-funder<fnref n="1" /> context (see section 3).

* Produce a set of recommendations on the criteria for choosing a tool, based on functional needs, disciplines, and fields of research, as well as institutional requirements (see section 4).

* Draw up a comparative list of some existing tools based on the recommended criteria in order to measure their effectiveness in and/or suitability to different usage contexts (see section 5).

* Produce a set of recommendations to successfully implement an ELN (see section 6).

Note that the group's objective is to propose decision-making support criteria and not to provide definitive methods of comparison between different tools, nor to advocate the choice of a single tool for the entire higher education and research community.

## 1.3 Group composition

The working group is composed of 16 people from different institutions, representing the scientific diversity of higher education and research. Each member, according to their discipline and experience, was able to contribute their expertise, from the following points of view:

* **Scientific/experimental:** with a hands-on view, as a user

* **Legal and strategic:** consideration of aspects related to scientific integrity, intellectual property, and content promotion.

* **Technical:** both in terms of infrastructure and tool hosting, and of issues relating to data management, interoperability and semantics.

<footnote n="1">¹especially in France where academic research is driven and funded by different bodies and organisations.</footnote>
### 1.3.1 Leaders

The following people have been commissioned by the Committee for Open Science to lead the group's work:

* Gilles Mathieu, Research engineer in computer science at the Inserm Information System Department (ISD)

* Dominique Pigeon, Research engineer in computer science at the Inserm Information System Department (ISD)

* Tovo Rabemanantsoa, Member of the INRAE Directorate for Open Science (DipSO)

### 1.3.2 Members

The following people have been members of the working group:

* Christophe Chipeaux (ICOS/INRAE)

* Simon Duvillard (Réseau CURIE)

* Célya Gruson-Daniel (Inno³/UTC)

* Marie-Emilia Herbet (Université Lyon I)

* Arnaud Legrand (INRIA/CNRS)

* Nathalie Leon (CNRS)

* Domenico Libri (CNRS/INSB)

* Jean-Baptiste Lily (ICOS/INRAE)

* Jean-François Peyrat (Université Paris Saclay)

* Agnès Pinet (CEA)

* François Sabot (IRD)

* Moussa Seydi (IRD)

* Véronique Theisen (CEA)

## 1.4 How the WG worked

The working group carried out its work between November 2020 and July 2021.

Discussions took place by videoconference only, in the following format:

* plenary meetings held monthly for a duration of one hour.

* workshops that were offered to WG members, to work on particular topics.

* Exceptionally, external participants took part in sessions or were asked to discuss certain topics or contribute their thoughts on them. The list of persons concerned is provided in the appendices.

In addition, the work was coordinated with that of the "Successfully appropriating
open science"<fnref n="2" /> working group led by Anne Vanet at the Committee for Open Science and with the "Electronic Lab Notebooks" project which is underway at the CNRS, led by Nathalie Léon and Domenico Libri. To this end, the working group included a member appointed from the Data College, a member of the "Successfully appropriating open science" working group, as well as the two CNRS project co-leaders.

**1.5 Purpose of the report**

The working group has drawn up recommendations for electronic laboratory notebooks in two forms :

* Firstly, this report, which sets out the working group's approach, presents the tools and examples of their use.

* Secondly, a methodological tool: a *grid of criteria* to help choose an electronic lab notebook (editable table, with calculation formulas included); these criteria are accompanied by proposals for weighting according to use.

Two files make up this tool: "ELN WG - Tools - Criteria Grid.odt" provides instructions for the use of the different tabs in "ELN WG - Tools - Criteria Grid.ods".

<footnote n="2">² https://www.ouvirlascience.fr/reussir-lappropriation-de-la-science-ouverte/</footnote>
# 2 Definition and scope: a shared vision of the electronic lab notebook

## 2.1 Definition

An *Electronic Lab Notebook*, or ELN, is a software program that collects and pools all the essential information for the reproduction of a scientific experiment. Its archival value and its ability to provide proof for patent applications echo those of a physical notebook, but the ELN goes much further, creating an "upgraded" and "amplified" notebook comparable to a complete laboratory knowledge management tool. The most significant differences are found in its collaborative capabilities (online collaboration), its improved traceability (use of standard protocols or data models) and time savings (using a search engine to look up experiments). In addition to combining operating procedures, the electronic notebook provides an integrated approach to scientific work. It establishes continuity between an experiment and its raw data, without being designed to store the latter. It facilitates data processing thanks to the possibility of interfacing with specialist software, laboratory instruments and databases. Finally, when the ELN is paired with an LIMS-style solution (see section 2.2.3 "related tools" below), its scientific role is complemented by logistical features likely to improve the organisation of the laboratory in terms of day-to-day stock management (substances, samples, animals, etc.) or equipment management (booking, maintenance).

## 2.2 Scope

### 2.2.1 Persons involved

An ELN is used by the following people:

* **The primary research participants**, for whom the ELN is primarily intended. These are the main users of the ELN, as far as tracking and logging their work is concerned;

* **The team leader**, acting as a specialist or administrator. The term principal investigator (PI) may also be used;

* **The signatory**, who will ultimately be responsible for validating the research recorded in the ELN;

* **The Legal and Promotion departments**, who will be able to rely on the ELN to establish or confirm the date and authorship of the research work recorded in the ELN, particularly for patent filing processes;

* **The different partners** of the principal research participants (communities, industrial partners, hospitals, etc.), who will be able to use the ELN as a collaborative and information sharing tool;

* **Archivists**, who will be responsible for ensuring the long-term preservation of the information stored in the ELN.
### 2.2.2 Uses

The uses of an ELN differ enormously according to scientific discipline, but also— and especially—according to research practices. The names field notebook, handling book, logbook, archeology field notebook are therefore preferred to "lab notebook", manifesting the various forms (digital in particular) and purposes according to the requirements of the respective communities. Outside the "laboratory", field surveys involve the use of a number of materials and tools whose functions differ depending on research methods. These can serve as:

* an aid for gathering a variety of field observations or analytical notes throughout "qualitative" research, whether individually or collectively as part of ethnographic, sociological, or anthropological approaches

* a tool for the systematic collection and organisation of data in archaeology and environmental sciences, resulting in databases and information systems

Whether it be note-taking using a text editor or the use of geographic information systems, the nature of the digital tools employed varies according to practice.

In the field of humanities and social sciences (sociology, anthropology, etc.) or life sciences (ecology, etc.), software tools have begun to replace paper in the field for taking observational or analytical notes, etc. While not having the same needs as experimental research, these disciplines can benefit from the use of ELNs as regards data security, indexing, integration with other digital tools, etc. A well-documented and non-exhaustive overview of these topics is provided in the appendices.

In terms of research practices, across all scientific fields, the working group identified three principal approaches that characterise the various uses of ELNs:

* Predominantly **experimental** research. The research is primarily based on controlled and deliberate testing, whether under laboratory or real-world conditions.

* Predominantly **observational** research, where the main focus of the research is observing objects, phenomena and behaviours outside of a controlled setting.

* Predominantly **analytical** research, which mainly focuses on computer analysis and processing of data, whether of an observational or experimental origin.

Note that most scientific disciplines have areas and topics of research relating to these three approaches and that there is no strict boundary between them.

### 2.2.3 Related tools

With advances in digital technology, the various tools available are seen less and less as separate objects: at different scales, digital work environments are being developed that emphasise the interoperability of different software. The lab notebook in a digital environment is therefore far from being isolated like a physical notebook, and it works with various tools and research infrastructures, allowing for fluid and secure data management.
The following tools are among those most likely to interact with an ELN:

* **The LIMS** = (*Lab Information Management System*), These are integrated management software packages that manage, for example, field data entry and sample traceability, users, instruments (mainly robots), stocks and supplies, and product and equipment tracking. For research units such as laboratories, working with a primarily experimental approach, it is particularly useful to have features dedicated to the management of this information. It should be noted that many ELN solutions integrate LIMS-like features.

* **Notebooks**. These are interactive programming interfaces that make it possible to combine natural language text content with sections in computer language. These tools are often used by research communities working in a predominantly analytical manner.

* **Software forges**. These are systems for the collaborative management and maintenance of code, text and documents often linked to an IT project or software product. These tools are mainly used in analytical research.

* **Discipline-specific computer systems**, such as GIS (Geographic Information Systems), used in archaeology.

* **Data repositories**, allowing the link between the research process as recorded in the ELN and the final and potentially published data resulting from this work.

Linking the recorded progress of a project's research work with the data handled, the equipment used, or the code used, is often necessary to meet traceability and reproducibility requirements. Depending on the context, interoperability with these different tools may be necessary for the correct use of an ELN.
# 3 Recommendations for interoperability between different tools

## 3.1 Interoperability

Interoperability, as widely understood<fnref n="3" />, is a product or system's ability to work with other existing or future products or systems without restrictions on access or implementation. Interoperability concerns only the external behaviour of products and does not influence their internal design. This capability is ensured by known standards and formats. In France, these standards can be found in the general interoperability framework (RGI – référentiel général de l'interopérabilité)<fnref n="4" /> the current version of which was approved by the Ministerial Order of 20th April 2016⁵. The RGI is a document that outlines a set of standards and good practices used by French administrative bodies in the field of information technology.

The challenges of interoperability lie in the formats of the data being exchanged between different products as well as in the communication interfaces and protocols.

A research laboratory generally has a rich digital ecosystem, often referred to as an information system (IS). It can be composed of data management infrastructures (databases, document management, repository, etc.), computing infrastructures (HPC, Notebook, processing chains, etc.), virtual research environments (VRE) or simply office tools. Research staff also have access to resources outside of the laboratory such as public data repositories, catalogues and storage and/or computing space. In order for the adoption of an ELN to add value, it needs to seamlessly interface with existing IS components. The ELN's capacities in terms of interoperability are key to its integration into the IS.

## 3.2 Recommendations

### 3.2.1 Joint supervision

When each supervisor already recommends or uses an ELN, it is important that each ELN be able to export its content in a standard, open and documented format

This will allow different scientists to distribute and share content, and thus decompartmentalise research work.

As there is currently no standard format for ELN content, the accessibility and usability of these export formats is paramount in order to create bridges between existing ELNs.

<footnote n="3">³ https://en.wikipedia.org/wiki/Interoperability</footnote>

<footnote n="4">⁴ https://www.numerique.gouv.fr/publications/interoperabilite/</footnote>

<footnote n="5">⁵ https://www.legifrance.gouv.fr/jorf/id/JORFTEXT000032438896</footnote>
It is essential that the ELN can at least assign a Persistent IDentifier (PID) and a
timestamp to each of its inputs. Ideally, the ELN should be able to address these
identifiers directly through repositories (DOI, ORCID, etc.)

3.2.2 Integration

One should Prioritise ELNs that integrate seamlessly with tools already in use in the institution.

This integration can be done natively without the need for programming, but can
also be achieved through the use of programming interfaces (APIs). In this case, favour
an ELN with well-documented APIs.

The ELN must be able to interface with existing systems such as identity
management, EDMs, VREs, repositories (Dataverse, software forge, etc.) or
computational environments (notebooks such as Jupyter and Scilab, R-studio, etc).

3.2.3 Access to data

Consider the ability of the ELN to access and read data stored outside the system - especially large datasets that cannot be imported into it.

It is important that the ELN can access data and metadata stored in repositories
(Dataverse, CKAN, etc.) to enable reuse of research data.

3.2.4 Backup and archiving

If the ELN includes a backup mechanism, it must be configurable and operable
without limitation for the user institution's managers. Backed-up items must be
accessible and restorable at will.

Otherwise, the ELN's structure must make it possible. Backup of ELN data must be
possible externally through images of the ELN file system tree and the ELN's databases.
These images must be restorable, at least, on the same version of the ELN and, ideally,
with backward and forward compatibility.

In any case, it is in the interest of the user institution to be able to access a history
of the contents of the ELN it uses. In the event of corruption or loss of data, it must be
able to cope through a restoration of this history.

The ELN must be able to export all or part of its entries in an open format in order
to establish a persistent archive. These exports must include all necessary metadata,
associated media and references so that they can be used independently of the ELN.
## 3.3 Summary

Ensure that the ELN can store and handle data in open input AND output formats.

Ensure that the ELN provides a well-documented API to facilitate its integration
into the IT ecosystem if it does not do so natively.

Ensure that the content of the ELN can be backed up and archived.
# 4 Recommendations for the tool selection procedure

## 4.1 Needs

### 4.1.1 Determine the wishes of stakeholders

Since the objective is to meet the needs of stakeholders who want an electronic solution suited to specific research practices, one of the first phases is to broadly identify functional requirements. This process involves a study to gather user needs and define the technical specifications derived from the contributions of the different stakeholders (management, scientists, legal, information systems, security, etc.) In order to ensure the traceability and scientific integrity of research results, careful consideration must be given to providing all the security, confidentiality, and traceability necessary to protect the results and scientific heritage of the institution and of its research partners. Also, collaboration between partner institutions and most often unit co-supervisors is essential in order to issue shared recommendations facilitating access to the tools for the primary target, i.e. the users.

The managing body must provide guidelines for the expected objective(s), the scope of the study or need, and the overall organisational structure necessary to respond to the achievement of this or these objectives according to indicators which will be set in advance. These will define the arbitration and validation steps allowing for consistent progress of the study with regard to preliminary results and allowing for a certain agility in adjusting milestones over time if necessary.

### 4.1.2 Take into account the research discipline and its practices

Usually used for bench or field experiments, the electronic version of the lab notebook has wider applicability, as shown in the "uses" section (2.2.2) of this report. Most tools have a shared core set of features (creation of experiments, advanced search, experiment models, etc.), supplemented by specialised functionalities that are sometimes very specific to a given discipline.

Selecting a tool can thus be difficult due to specialised requirements in the different fields and sub-fields of research. Within a single discipline,

expectations can be very different: a synthetic chemist will not use the ELN in the same way as a chemist who does analytical work with code. However, these two profiles may very well be found within the same laboratory. Consequently, the choice should hinge on research practices rather than the discipline itself.

Staying with the example of chemistry and more specifically synthetic chemistry, the added value of an ELN lies in features such as a stoichiometric table performing mass and quantity calculations automatically, an efficient search by chemical structure and substructure, a molecule editor and a solution for managing inventories of
reagents and products used during tests. It is this comprehensive set of discipline-specific tools that can make the ELN an asset in a researcher's daily work.

The inventory management mentioned above is a functionality that relates more to LIMS-style (Laboratory Information Management System) tools. However, most electronic lab notebook solutions currently available include stock management features, with varying degrees of sophistication. In situations where the laboratory is required to manage products, equipment or apparatus on a daily basis, as may be the case in chemical or biology laboratories, the choice of ELN may be partly determined by this feature.

For researchers from analytical and software development fields, tools that strictly correspond to the definition of an electronic notebook are not very suitable: advanced calculation, editing and execution of code, version control, etc.; these are indispensable features that do not exist at time of writing-. Moreover, many very powerful tools performing most of these functions exist and are already in use by various communities. Examples include Git for version control, Gitlab for collaborative development, Emacs for code editing and Jupyter Notebook for code formatting and execution. The ELN cannot replace these tools, but can complement them. Some ELN publishers provide an API that allows the tool to be interconnected with other software. It may be of interest to analytical researchers to see if they are able to "piggyback" their tools onto the ELN using the API.

Therefore, the importance to be placed on a particular feature will vary according to the discipline and the research methods favoured by the target audience.

### 4.1.3 Consider the level of security required

One of the most important things to consider when choosing is the expected level of security for the storage, processing and retrieval of data. Depending on the technical, scientific, and human circumstances specific to each study, it will be necessary to establish the extent of the security requirements. To this end, it is recommended to use a risk analysis process such as that from the French National Agency for the Security of Information Systems (ANSSI - Agence Nationale de la Sécurité des Systèmes d'Information), also known as the EBIOS<fnref n="6" /> method, while referring to standardised criteria such as those found in the ISO/CEI 27001⁷ standard. Whether or not a laboratory has restricted access policies should also be taken into account when choosing. The scope of an ELN's functions tends to imply high security requirements, particularly according to the four commonly applied assessment criteria:

* The confidentiality of processed data must be ensured through an authorisation system : only authorised persons should be able to access the information intended for them (through access rights or permissions) and any unwanted access must be prevented.

<footnote n="6">⁶ https://www.ssi.gouv.fr/administration/management-du-risque/la-methode-ebios-risk-manager/</footnote>

<footnote n="7">⁷ https://www.iso.org/isoiec-27001-information-security.html</footnote>
* **Authenticity** must be ensured via an **authentication** system: users must prove their identity through the use of an access code. This allows the management of access rights to the resources in question, and maintains a level of confidence when sharing.

* **Integrity**: the data must be as expected, and must not be altered in an accidental, illicit or malicious way. This means that the data in question must be accurate and complete. Usually checksums or hashes are used for this purpose.

* **Availability**: the ability to access information system resources must be maintained continuously and without interruption during planned periods of use. Services and resources should be accessible quickly and regularly.

Security requirements may also be increased according to the nature of the data being processed, for example data subject to professional secrecy, industrial and commercial secrets, personal data and data requiring protection to safeguard scientific potential. The country and conditions in which the data in the notebook is hosted will require careful scrutiny, as will the conditions for its storage and access by third parties.

In addition to these four main criteria, there are other aspects to be considered in the selection of an ELN, including:

* **Traceability** (or "proof"): a guarantee that access and attempts to access the data in question are traceable and that these traces are stored and usable.

* **Non-repudiation** and **imputation**: no user should be able to disavow tasks that they carried out as part of the activities they were authorised to perform, and no third party should be able to attribute the tasks of another user to themselves.

#### 4.1.4 Consider the extent of the tool's deployment

A distinction must be made between the level of deployment of the ELN and that of its hosting:

* An ELN can be created for each contributor to a research project or, more commonly, for each research team; it is less common for a multi-team research unit or even an entire research centre to have a single ELN.

* ELNs can be set up on a researcher's workstation, in the building hosting the research unit, in a regional or national data centre, or even in the cloud.

Having the ELN set up on an individual workstation can provide its owner with autonomy when it comes to backing up data and ensuring data quality (e.g. timestamping activities). However, this makes it difficult to share data.-While one of the advantages of an ELN over its physical counterpart is the ability to share information within a research team. An advantage of having the ELN on an individual workstation is the ability to work offline or during network outages. Setting up ELNs on a central server makes sharing easier but also makes them dependent on the quality of networking between the workstations and the server. As various combinations of the above are possible, they will need to be assessed with the following in mind:

* Sharing information
* Data security

* Potential interconnectivity with other digital tools

* Efficiency, reliability

#### 4.1.5 Identify the systems that will interact with the ELN

Depending on the context, methods, and fields of the research in question, the chosen ELN should be able to interact with different third party systems. This will be the case for the related tools described in section 2.2.3. Recommendations covering the interoperability of the chosen solution, as described in section 3, would therefore need to be closely followed.
## 4.2 Establish the selection criteria

### 4.2.1 Technical and functional characteristics

The process of choosing an ELN starts with drawing up a list of desired features and any essential requirements and technical specifications. The following list of features was drawn up by the working group:

<table><caption>Table 1: shortlisted features</caption><thead><tr><td>Feature</td></tr></thead><tbody><tr><td>Description of experiments and results</td></tr><tr><td>Ability to duplicate experiments (without using research output), such as a model/template for a new experiment</td></tr><tr><td>Description of protocols</td></tr><tr><td>Ability to duplicate protocols as a model/template for a new protocol</td></tr><tr><td>Attachment management: office (PDF, audio, video, etc.) or scientific and technical (git, chromatograms, spectra, images, etc.)</td></tr><tr><td>Ability to export materials from the ELN to a printable format</td></tr><tr><td>Access to external data (i.e. not stored in the ELN), raw or not, for reading and for reference</td></tr><tr><td>Data structuring with internal search engine and filter; use of keywords (tags)</td></tr><tr><td>Accessible content (for mining)</td></tr><tr><td>Possibility to associate metadata with the data sets being handled, whether they are generic (dates, authors, etc.) or specific (ontologies, taxonomies, etc.)</td></tr><tr><td>Use and maintenance of equipment (including notifications)</td></tr><tr><td>Stock management (reagents, cell lines, etc.)</td></tr><tr><td>Tools for simple calculations (basic mathematical functions, unit conversion, etc.)</td></tr><tr><td>Tools for complex calculations (statistics or analyses requiring a programming language)</td></tr><tr><td>Molecule building/design tool</td></tr><tr><td>Notes on experiments or protocol (during experimentation), reports, etc.</td></tr><tr><td>Ability to cite content from bibliographies (such as labtex.bib Text)</td></tr><tr><td>Access to workstation office tools</td></tr><tr><td>Management of software tools and codes (continuous integration, continuous deployment, version control, etc.)</td></tr><tr><td>Management of sharing by users</td></tr><tr><td>Shared agenda</td></tr><tr><td>Collaborative document editing (online and simultaneously by several users)</td></tr><tr><td>Annotation of content and comment insertion</td></tr></tbody></table>
As for technical specifications, the group came up with the following list:

<table><caption>Table 2: shortlisted technical specifications</caption><thead><tr><td>Category</td><td>Specification</td></tr></thead><tbody><tr><td>Data protection</td><td>Compliance with the GDPR for personal user data<br>Compliance with the GDPR for research data: personal data of research participants if stored in the ELN<br>Have a managed, redundant and backed-up storage (remote sites)</td></tr><tr><td>Integrity</td><td>Compliance with regulatory or legislative requirements governing the hosting location of research data<br>Enable document signing (hashes)<br>Enable activity and document timestamping<br>Enable unique input identification (protocol, experiment, note, etc.) – UUID<br>Enable versioning/change tracking<br>Enable the implementation of predefined workflows (e.g. a set of required actions in the description of an experiment or result)</td></tr><tr><td>Authentication, security</td><td>Enable electronic signature by participants<br>Use a secure authentication system<br>Enable the creation of user management rules: access permissions by role, project, and life cycle (arrival/departure of participant, project launch/completion, etc.)</td></tr><tr><td>Data backup</td><td>Encrypt data transport (TLS, RGS-compliant algorithms)<br>Encrypt data<br>Include regular backups and restores<br>Define data retention periods and archiving or destruction terms</td></tr><tr><td>Confidentiality</td><td>Ensure the confidentiality of exported data for backup or archiving.<br>Safeguard the confidentiality of information that may represent valuable know-how or confidential business information (safeguard the innovation in the event of a patentable invention).<br>Safeguard the confidentiality of information exchanged with a partner in the context of a collaboration</td></tr></tbody></table>
<table><tr><td rowspan="6">Compatibility, interoperability</td><td>Compatibility with different operating systems (Windows, Linux, etc.), for servers</td></tr><tr><td>Allow data to be managed (Import/Export) in a structured format (e.g. in case of a software or administrative modification)</td></tr><tr><td>Allow integration with a local information system (databases, directories, etc.)</td></tr><tr><td>Enable links with data repositories</td></tr><tr><td>When available, use open standards for input or output data to allow information to be permanently re-used.</td></tr><tr><td>Enable data sharing through data integration and exposition services, via WebServices or REST APIs if possible</td></tr><tr><td rowspan="6">Use, usage</td><td>Ability to easily install updates and upgrades, while guaranteeing accessibility, if using a web browser</td></tr><tr><td>Ergonomic and intuitive</td></tr><tr><td>Enable modularity/customisation/flexibility</td></tr><tr><td>Have the necessary documentation (guide, tutorials, instruction manuals, etc.)</td></tr><tr><td>Access to source code in order to manage, if necessary, the development of the tool, its maintenance, and most importantly the data it contains</td></tr><tr><td>Supported by a community of users</td></tr><tr><td rowspan="4"></td><td>Multilingual support (at least English in addition to the local language)</td></tr><tr><td>Allow offline use with subsequent data synchronisation</td></tr><tr><td>Be usable on the move (on tablets and smartphones)</td></tr><tr><td>Take environmental footprint into account : electricity consumption of servers, etc.</td></tr></table>

Of course, the various features and technical specifications listed will not have the same weight in the final choice. They should mainly serve as a starting point for the creation of an assessment grid, which will be outlined below. Further details on some of these features and specifications can be found in the appendices.

### 4.2.2 Other criteria

Features and technical requirements are not the only criteria to consider. For a comprehensive assessment, it is also important to consider:

* The overall cost of the solution, whether financial costs or resource requirements, such as human resources. This cost is to be estimated for the acquisition, deployment, and long-term use of the chosen solution. In addition
to the potential licence costs for a proprietary solution, the study must also
include maintenance costs related to hosting, setting up the required technical
environment, support, and change management.

* **Ease of adoption and available support.** This can mean the quality of support provided by the publisher, if any, the availability of documentation, but also the size and vitality of the user community.

* **Subjective suitability criteria specific to the target users** While difficult to quantify, it is nevertheless a major factor in successfully choosing a solution. For some communities this can mean the ability of the ELN in adapting to the typical user workflow, and this can lead to a more effortless interplay between the thought processes of the researcher and the actual input of information into the ELN. It is difficult to objectively assess this factor and it is often necessary to carry out a use case analysis with test users followed by feedback and opinion surveys.

•

4.3 Identify commercial options

The ELN market is booming. Between 2014 and 2020, revenue in this sector more
than doubled in the US from $222 million to $576 million<fnref n="8" />. This dynamism was largely
driven by the pharmaceutical industry's digital transformation due to its early
recognition of the potential of ELNs as a driver for improving quality processes,
innovation, and competitiveness. Some industry giants such as BMS, Astra Zeneca,
Johnson & Johnson, GSK, Eli Lilly and Company, and Roche, therefore opted for
electronic lab notebooks in the early 2000s.

This drive for transformation resulted in a boom in available solutions. In 2006, a US report counted under 30 suppliers of electronic lab notebooks<fnref n="9" />. In 2017, a study conducted by the University of Southampton counted 103 different solutions<fnref n="10" />.

The ELN software market is so intricate as to complicate the selection of a solution,
and this is reflected in the many current attempts at comparative analysis (i.e.
*benchmarking*). Between 2018 and 2021, Harvard Medical School and the Boston
University Medical Campus conducted a benchmarking study with 33 individual
products to “help researchers (...) effectively identify ELN tools that meet the specific
needs of their research.”<fnref n="11" /> The resulting table is still available online ¹²and was created
by sending questionnaires to the relevant publishers. Other such efforts are also freely
available, such as the more succinct survey by Cambridge University<fnref n="13" />, which includes

<footnote n="8">⁸ https://www.grandviewresearch.com/industry-analysis/electronic-lab-notebook-eln-market</footnote>

<footnote n="9">⁹ https://journals.sagepub.com/doi/10.1016/j.jala.2009.01.002</footnote>

<footnote n="10">¹⁰ https://jcheminf.biomedcentral.com/articles/10.1186/s13321-017-0221-3</footnote>

<footnote n="11">¹¹ https://zenodo.org/record/4723753#.Y9Ks562ZPIV</footnote>

<footnote n="12">¹² The working group behind the document decided to stop updating it after April 2021: https://docs.google.com/spreadsheets/d/1ar8fgwagOh30E31EAPL-Gorwn_g6XNf81g3VDQnQ_I8/</footnote>

<footnote n="13">¹³ https://www.data.cam.ac.uk/data-management-guide/electronic-research-notebooks/electronic-research-notebook-products</footnote>
short case studies focusing on four products (Onenote, Hivebench, Benchling, and LabArchives)<fnref n="14" />. The latter software, which is used by Monash University in Melbourne, Australia, is extensively featured on the university library's website<fnref n="15" />. Similarly, the university libraries of Lyon 1 and Grenoble Alpes have drawn up a comprehensive overview of eight electronic lab notebook solutions, some of which excel at providing features specifically suited to chemistry<fnref n="16" />.

An initial assessment of the different products available can involve separating ELNs according to usage, business model, and host type:

<table><tr><td rowspan="3">Usage</td><td>General-purpose</td></tr><tr><td>Discipline-specific</td></tr><tr><td>Integrated LIMS features</td></tr><tr><td rowspan="2">Business model</td><td>Open Source</td></tr><tr><td>Proprietary</td></tr><tr><td rowspan="3">Host type</td><td>Cloud provided by publisher only</td></tr><tr><td>In-house hosting only</td></tr><tr><td>Cloud or in-house hosting</td></tr></table>

This basic grid enables products to be filtered out based on the features of currently available products and the priorities of individual establishments. The template provided below includes four examples of ELN that are representative of the variety of products on offer:

<table><thead><tr><th>Criterion / Solution</th><th></th><th>Chemotion</th><th>elabFTW</th><th>Mbook</th><th>Labstep</th></tr></thead><tbody><tr><td rowspan="3">Usage</td><td>General-purpose</td><td></td><td>X</td><td></td><td>X</td></tr><tr><td>Discipline-specific</td><td>X</td><td></td><td>X</td><td></td></tr><tr><td>Integrated LIMS features</td><td></td><td></td><td>X</td><td>X</td></tr><tr><td rowspan="2">Business model</td><td>Open Source</td><td>X</td><td>X</td><td></td><td></td></tr><tr><td>Proprietary</td><td></td><td></td><td>X</td><td>X</td></tr><tr><td rowspan="2">Business Host type</td><td>Cloud provided by publisher only</td><td></td><td></td><td></td><td>X</td></tr><tr><td>In-house only</td><td>X</td><td></td><td></td><td></td></tr></tbody></table>

<footnote n="14">¹⁴ https://www.data.cam.ac.uk/data-management-guide/electronic-research-notebooks/electronic-notebook-case-studies</footnote>

<footnote n="15">¹⁵ https://www.monash.edu/library/researchers/data-collection-management/eln</footnote>

<footnote n="16">¹⁶ https://www.datac.org/bonnes-pratiques/utiliser-un-cahier-de-laboratoire-numerique/</footnote>
<table><tr><td>Cloud or in-house hosting</td><td></td><td>X</td><td>X</td><td></td></tr></table>

Given the diversity of available solutions, it is not possible to identify a general trend favouring one particular tool in recent years in the field of research and higher education. Some institutions have favoured open source solutions (elabFTW at Inria, the Curie Institute, the Institut Jacques Monod, the University of Düsseldorf, Graz University of Technology, etc.), while others have gone for proprietary solutions (Labguru at Inserm, eLabJournal at the Pasteur Institute and the University of Geneva [where they also use Rspace]<fnref n="17" />, etc.). Among the software with purposefully discipline-specific designs, we see products primarily catering to the needs of biologists and chemists.

The ELN market is also experiencing an incursion by large publishers, some of whom are buying up products initially developed by start-ups. Examples include the acquisition of Hivebench by Elsevier in 2016 and Digital Science's purchase of shares in Biodata (publisher of Labguru), whose parent company is Holtzbrinck. ELNs are also part of the business strategies of global groups such as Dassault (Biovia), PerkinElmer (Signals), and Agilent (Slims).

These solutions stand in contrast to strategies pursued by the academic sector, which have developed in recent years: elabFTW (Curie Institute), Chemotion (Karlsruhe Institute of Technology) and OpenBIS (University of Zurich).

## 4.4 Assess options based on stated criteria

The comparison of tools in technical terms (i.e. features and specifications) should ideally rely on a list of objective criteria that allows solutions to be classified according to the weight given to a particular criterion in the context of the study.

This requires a two-tiered analysis: firstly with regard to the desired scale and granularity of criteria (scoring), and secondly with regard to the relative importance of criteria (weighting).

### 4.4.1 Scoring

The minimum degree to which a tool can be rated according to the established criteria is binary: the assessed tool either has a feature or does not; it either meets a requirement or doesn't. Solutions can therefore be assessed by feature as part of an objective assessment that does not imply any kind of value judgement.

However, depending on the study's context, different scoring scales can be employed according to the profile of the assessors or the level of detail required. The template grid provided by the working group presents a scoring system with four tiers:

<footnote n="17">¹⁷ https://www.unige.ch/researchdata/fr/collecter-organiser/eln/</footnote>
* 0 = does not meet requirement or information unavailable

* 1 = partially meets requirement

* 2 = meets requirement

* 3 = meets requirement with merit, or has specific qualities

If the study involves numerous assessors, a continuous scale (e.g. from 0 to 10) could be used to get an average of the scores given by the different assessors for each feature.

In any case, the type of scoring scale should be explained to assessors in order to avoid confusion and misuse.

### 4.4.2 Weighting

While scoring allows different tools to be compared based on a specific criterion, when making the final choice a distinction must be made between essential criteria and those of a more auxiliary nature. For example, it makes sense to place more emphasis on the "allows description of experiments" feature (without which an ELN would not be an ELN) than on the ability to export text in a printable format.

Consequently, to get a rounded impression of a solution's quality these individual scores should be weighted in a way that treats the results of the assessment as a whole.

Two weighting scales have been put forward by the working group. The first has four tiers:

* 1 = useless

* 2 = secondary

* 3 = important

* 4 = essential

The second is a non-linear scale, put forward in order to allow for a more detailed analysis. It works by summing the weighting values to 100 and then dividing the values according to their corresponding importance. This scale is used in the grid shown in the following paragraph.

### 4.4.3 Presentation of results

Proper use of the grid should allow for the results of the technical comparative study to be displayed. Depending on needs and context, the results can be displayed in several ways:

* A binary "does/doesn't" table: this format can prove effective and useful as a first step in filtering a large list of tools (see section 5)

* An overall score for each tool, calculated by multiplying the score for each criterion by its weighting value and then summing the results.

* A graphical representation of each tool's strengths and weaknesses. This could take the form of a radar chart.
Figures 1 and 2 show examples of a scoring grid and are based on some of the tools discussed in section 5.

A radar chart can be created by comparing the subtotals for all features and the subtotals for each category of specifications with the maximum score of 2 per criterion (3 meaning that based on stated requirements, the assessed solution surpasses 100% suitability), and then scaling them to 5. The eight values obtained are represented on the radar chart in a way that can be particularly useful for comparing different solutions. However, it is worth noting the bias inherent to the fact that the relative importance of the different groups of specifications has not been taken into account. Examples of a radar chart and the table of values used to generate one are shown in Figures 3 and 4.

#### 4.4.4 Comments:

* In the example given above (Figure 1), we can see that the features of the tool being assessed are potentially better suited to an experimental approach, while its technical specifications are not significantly different.

* We can see that there are a few items with a score of 0 (Figure 2). This is actually because the ELN in question is only available as a hosted solution in the United States (i.e. as SaaS - Software as a Service), since its publisher is American. Although the terms of use state that the publisher is compliant with the GDPR with respect to users' personal data, they make no mention of the protection of research data stored in the ELN. It is therefore reasonable to assume that personal research data is not protected. However, neither the documentation nor the publisher's website indicate whether content is encrypted or not. This information is therefore missing.

* As for the items with a score of 1: with the "use and maintenance of equipment" feature in mind, this ELN does indeed allow access to the laboratory's equipment inventory and also allows us to see whether equipment is free or in use, but we are not able to see whether a piece of equipment is overdue for inspection.

* For items with a score of 3 (meaning this specific feature or specification of the ELN exceeds expectations): if we look at the "Molecule building/design tool" feature as an example, we see that this ELN exceeds the typical functionality of such features as it also enables the verification/validation of atomic bonds, molar mass calculation, etc.
<table><caption>Figure 1 – Example of feature scoring</caption><thead><tr><th rowspan="2">Feature</th><th rowspan="2">Score assigned</th><th rowspan="2">Exp. Weighted score</th><th rowspan="2">Analyt. Weighted score</th><th rowspan="2">Obs. Weighted score</th><th colspan="3">Weighting per approach</th></tr><tr><th>experimental</th><th>analytical</th><th>observational</th></tr></thead><tbody><tr><td>Description of experiments and results</td><td>3</td><td>30</td><td>30</td><td>30</td><td>10</td><td>10</td><td>10</td></tr><tr><td>Ability to duplicate experiments (without the research output) as a model/template for a new experiment</td><td>3</td><td>15</td><td>21</td><td>12</td><td>5</td><td>7</td><td>4</td></tr><tr><td>Description of protocols</td><td>1</td><td>10</td><td>7</td><td>10</td><td>10</td><td>7</td><td>10</td></tr><tr><td>Ability to duplicate protocols as a model/template for a new protocol</td><td>2</td><td>10</td><td>14</td><td>10</td><td>5</td><td>7</td><td>5</td></tr><tr><td>Attachment management: office (PDF, audio, video, etc.) or scientific and technical (git, chromatograms, spectra, images, etc.)</td><td>3</td><td>21</td><td>15</td><td>21</td><td>7</td><td>5</td><td>7</td></tr><tr><td>Ability to export materials from the ELN to a printable format</td><td>2</td><td>4</td><td>4</td><td>4</td><td>2</td><td>2</td><td>2</td></tr><tr><td>Access to external data (i.e. not stored in the ELN), raw or not, for reading and for reference</td><td>2</td><td>20</td><td>20</td><td>20</td><td>10</td><td>10</td><td>10</td></tr><tr><td>Data structuring with internal search engine and filter; use of keywords (tags)</td><td>2</td><td>14</td><td>12</td><td>16</td><td>7</td><td>6</td><td>8</td></tr><tr><td>Accessible content (for mining)</td><td>0</td><td>0</td><td>0</td><td>0</td><td>2</td><td>2</td><td>3</td></tr><tr><td>Ability to associate metadata with processed datasets, whether generic (dates, authors, etc.) or specific (ontologies, taxonomies, etc.)</td><td>2</td><td>16</td><td>12</td><td>16</td><td>8</td><td>6</td><td>8</td></tr><tr><td>Use and maintenance of equipment (including notifications)</td><td>1</td><td>3</td><td>1</td><td>1</td><td>3</td><td>1</td><td>1</td></tr><tr><td>Stock management (reagents, cell lines, etc.)</td><td>2</td><td>6</td><td>0</td><td>4</td><td>3</td><td>0</td><td>2</td></tr><tr><td>Tools for simple calculations (basic mathematical functions, unit conversion, etc.)</td><td>3</td><td>9</td><td>9</td><td>9</td><td>3</td><td>3</td><td>3</td></tr><tr><td>Tools for complex calculations (statistics or analyses requiring a programming language)</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>10</td><td>5</td></tr><tr><td>Molecule building/design tool</td><td>3</td><td>12</td><td>3</td><td>0</td><td>4</td><td>1</td><td>0</td></tr><tr><td>Notes on experiments or protocol (during experimentation), reports, etc.</td><td>1</td><td>2</td><td>1</td><td>3</td><td>2</td><td>1</td><td>3</td></tr><tr><td>Ability to cite content from bibliographies (such as labtex.bib Text)</td><td>1</td><td>3</td><td>2</td><td>4</td><td>3</td><td>2</td><td>4</td></tr><tr><td>Access to workstation office tools</td><td>0</td><td>0</td><td>0</td><td>0</td><td>3</td><td>2</td><td>4</td></tr><tr><td>Management of software tools and code (continuous integration, continuous deployment, version control, etc.)</td><td>0</td><td>0</td><td>0</td><td>0</td><td>2</td><td>10</td><td>0</td></tr><tr><td>Management of sharing by users</td><td>2</td><td>10</td><td>10</td><td>10</td><td>5</td><td>5</td><td>5</td></tr><tr><td>Shared agenda</td><td>0</td><td>0</td><td>0</td><td>0</td><td>2</td><td>1</td><td>2</td></tr><tr><td>Collaborative document editing (online and simultaneously by several users)</td><td>0</td><td>0</td><td>0</td><td>0</td><td>2</td><td>1</td><td>2</td></tr><tr><td>Annotation of content and comment insertion</td><td>1</td><td>2</td><td>1</td><td>2</td><td>2</td><td>1</td><td>2</td></tr></tbody><tfoot><tr><td>Features subtotal</td><td>34</td><td>187</td><td>162</td><td>172</td><td>100</td><td>100</td><td>100</td></tr></tfoot></table>
<table><caption>Figure 2 – Example of technical specifications scoring</caption><thead><tr><th colspan="2">Technical specifications</th><th colspan="7">Weighting per approach</th></tr><tr><th>Category</th><th>Specification</th><th>Score assigned</th><th>Exp. Weighted score</th><th>Analyst. Weighted score</th><th>Obs. Weighted score</th><th>experimental</th><th>analytical</th><th>observational</th></tr></thead><tbody><tr><td rowspan="3">Data protection</td><td>Compliance with the GDPR for personal user data</td><td>2</td><td>8</td><td>8</td><td>8</td><td>4</td><td>4</td><td>4</td></tr><tr><td>Compliance with the GDPR for research data: personal data of research participants if stored in the ELN</td><td>0</td><td>0</td><td>0</td><td>0</td><td>4</td><td>4</td><td>4</td></tr><tr><td>Have managed, redundant, and backed-up storage (remote sites)</td><td>2</td><td>8</td><td>8</td><td>8</td><td>4</td><td>4</td><td>4</td></tr><tr><td rowspan="6">Integrity</td><td>Compliance with regulatory or legislative requirements governing the hosting location of research data</td><td>2</td><td>8</td><td>8</td><td>8</td><td>4</td><td>4</td><td>4</td></tr><tr><td>Enable document signing (hashing)</td><td>0</td><td>0</td><td>0</td><td>0</td><td>4</td><td>4</td><td>4</td></tr><tr><td>Enable timestamping of activities and timestamping of documents</td><td>1</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td></tr><tr><td>Enable unique input identification (protocol, experiment, score, etc.) - UUID</td><td>0</td><td>0</td><td>0</td><td>0</td><td>4</td><td>4</td><td>4</td></tr><tr><td>Ensure versioning/monitoring of changes</td><td>1</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td></tr><tr><td>Enable the implementation of predefined workflows (e.g. a set of required actions in the description of an experiment or result)</td><td>1</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td></tr><tr><td rowspan="2">Authentication/security</td><td>Enable electronic signature by participants</td><td>1</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td></tr><tr><td>Use a secure authentication system</td><td>2</td><td>6</td><td>6</td><td>6</td><td>3</td><td>3</td><td>3</td></tr><tr><td rowspan="4">Data backup</td><td>Enable the creation of user management rules: access permissions by role, project, and life cycle (participant arrival/departure, project launch/completion, etc.)</td><td>1</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td></tr><tr><td>Encrypt data transport (TLS, RGS-compliant algorithms)</td><td>2</td><td>8</td><td>8</td><td>8</td><td>4</td><td>4</td><td>4</td></tr><tr><td>Encrypt data</td><td>0</td><td>0</td><td>0</td><td>0</td><td>3</td><td>3</td><td>3</td></tr><tr><td>Include regular backups and restores</td><td>2</td><td>8</td><td>8</td><td>8</td><td>4</td><td>4</td><td>4</td></tr><tr><td rowspan="3">Confidentiality</td><td>Define data retention periods and conditions for archiving or destruction</td><td>2</td><td>4</td><td>4</td><td>4</td><td>2</td><td>2</td><td>2</td></tr><tr><td>Ensure the confidentiality of exported data for backup or archiving.</td><td>0</td><td>0</td><td>0</td><td>0</td><td>4</td><td>4</td><td>4</td></tr><tr><td>Safeguard the confidentiality of information that may represent valuable know-how or confidential business information (safeguard innovation in the event of a patentable invention).</td><td>0</td><td>0</td><td>0</td><td>0</td><td>4</td><td>4</td><td>4</td></tr><tr><td rowspan="5">Compatibility/interoper</td><td>Safeguard the confidentiality of information exchanged with a partner in the context of a collaboration</td><td>1</td><td>2</td><td>2</td><td>2</td><td>2</td><td>2</td><td>2</td></tr><tr><td>Compatibility with different operating systems (Windows, Linux, etc.), for servers</td><td>1</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td></tr><tr><td>Allow data to be managed (Import/Export) in a structured format (e.g. in case of software or administrative modifications)</td><td>1</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td></tr><tr><td>Allow integration with a local information system (databases, directories, etc.)</td><td>2</td><td>8</td><td>8</td><td>8</td><td>4</td><td>4</td><td>4</td></tr><tr><td>Enable links with data repositories</td><td>0</td><td>0</td><td>0</td><td>0</td><td>3</td><td>3</td><td>3</td></tr><tr><td rowspan="5">Use/Usage</td><td>When available, use open standards for input or output data to allow information to be permanently re-used.</td><td>1</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td></tr><tr><td>Enable data sharing through data integration and exposition services, via WebServices or REST APIs if possible</td><td>1</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td></tr><tr><td>Ability to easily install updates and upgrades, while guaranteeing accessibility, if used from a web browser</td><td>2</td><td>6</td><td>6</td><td>6</td><td>3</td><td>3</td><td>3</td></tr><tr><td>Ergonomic and intuitive</td><td>2</td><td>8</td><td>8</td><td>8</td><td>4</td><td>4</td><td>4</td></tr><tr><td>Enable modularity/customisation/flexibility</td><td>1</td><td>2</td><td>2</td><td>2</td><td>2</td><td>2</td><td>2</td></tr><tr><td rowspan="10">Specifications subtotal</td><td>Have the necessary documentation (guide, tutorials, instruction manuals, etc.)</td><td>3</td><td>12</td><td>12</td><td>12</td><td>4</td><td>4</td><td>4</td></tr><tr><td>Access to source code in order to manage, if necessary, the development of the tool, its maintenance, and most importantly, the data it contains</td><td>0</td><td>0</td><td>0</td><td>0</td><td>3</td><td>3</td><td>3</td></tr><tr><td>Supported by a user community</td><td>1</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td><td>3</td></tr><tr><td>Multilingual support (at least English in addition to the local language)</td><td>1</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td><td>4</td></tr><tr><td>Allow offline use with subsequent data synchronisation</td><td>0</td><td>0</td><td>0</td><td>0</td><td>4</td><td>4</td><td>4</td></tr><tr><td>Be usable on the move (on tablets and smartphones)</td><td>2</td><td>8</td><td>8</td><td>8</td><td>4</td><td>4</td><td>4</td></tr><tr><td>Take environmental footprint into account: consumption of electricity by servers, etc.</td><td>0</td><td>0</td><td>0</td><td>0</td><td>2</td><td>2</td><td>2</td></tr><tr><td>Score assigned</td><td>38</td><td>136</td><td>134</td><td>136</td><td>123</td><td>122</td><td>123</td></tr><tr><td>Grand Total</td><td>72</td><td>323</td><td>296</td><td>308</td><td></td><td></td><td></td></tr><tr><td>Score out of 100</td><td></td><td>72</td><td>67</td><td>69</td><td></td><td></td><td></td></tr></tbody></table>
<table><caption>Figure 3 – Example of calculations for graphical representation</caption><thead><tr><th colspan="4">experimental approach</th></tr><tr><th>Category</th><th>score per cat.</th><th>max score</th><th>score out of 5</th></tr></thead><tbody><tr><td>Features</td><td>187</td><td>200</td><td>4,7</td></tr><tr><td>Data protection</td><td>2</td><td>24</td><td>0,4</td></tr><tr><td>Integrity</td><td>1</td><td>44</td><td>0,1</td></tr><tr><td>Authentication/security</td><td>1</td><td>22</td><td>0,2</td></tr><tr><td>Data backup</td><td>2</td><td>26</td><td>0,4</td></tr><tr><td>Confidentiality</td><td>1</td><td>20</td><td>0,3</td></tr><tr><td>Compatibility/interop</td><td>1</td><td>44</td><td>0,1</td></tr><tr><td>Use/Usage</td><td>0</td><td>66</td><td>0,0</td></tr></tbody></table><table><thead><tr><th colspan="4">analytical approach</th></tr><tr><th>Category</th><th>score per cat.</th><th>max score</th><th>score out of 5</th></tr></thead><tbody><tr><td>Features</td><td>162</td><td>200</td><td>4,1</td></tr><tr><td>Data protection</td><td>16</td><td>24</td><td>3,3</td></tr><tr><td>Integrity</td><td>18</td><td>44</td><td>2,0</td></tr><tr><td>Authentication/security</td><td>14</td><td>22</td><td>3,2</td></tr><tr><td>Data backup</td><td>20</td><td>26</td><td>3,8</td></tr><tr><td>Confidentiality</td><td>2</td><td>20</td><td>0,5</td></tr><tr><td>Compatibility/interop</td><td>23</td><td>44</td><td>2,6</td></tr><tr><td>Use/Usage</td><td>41</td><td>64</td><td>3,2</td></tr></tbody></table><table><thead><tr><th colspan="4">observational approach</th></tr><tr><th>Category</th><th>score per cat.</th><th>max score</th><th>score out of 5</th></tr></thead><tbody><tr><td>Features</td><td>172</td><td>200</td><td>4,3</td></tr><tr><td>Data protection</td><td>16</td><td>24</td><td>3,3</td></tr><tr><td>Integrity</td><td>18</td><td>44</td><td>2,0</td></tr><tr><td>Authentication/security</td><td>14</td><td>22</td><td>3,2</td></tr><tr><td>Data backup</td><td>20</td><td>26</td><td>3,8</td></tr><tr><td>Confidentiality</td><td>2</td><td>20</td><td>0,5</td></tr><tr><td>Compatibility/interop</td><td>23</td><td>44</td><td>2,6</td></tr><tr><td>Use/Usage</td><td>43</td><td>66</td><td>3,3</td></tr></tbody></table><table><thead><tr><th colspan="4">Score out of 5 per approach</th></tr><tr><th>Category</th><th>experimental</th><th>analytical</th><th>observational</th></tr></thead><tbody><tr><td>Features</td><td>4,7</td><td>4,1</td><td>4,3</td></tr><tr><td>Data protection</td><td>0,4</td><td>3,3</td><td>3,3</td></tr><tr><td>Integrity</td><td>0,1</td><td>2,0</td><td>2,0</td></tr><tr><td>Authentication/security</td><td>0,2</td><td>3,2</td><td>3,2</td></tr><tr><td>Data backup</td><td>0,4</td><td>3,8</td><td>3,8</td></tr><tr><td>Confidentiality</td><td>0,3</td><td>0,5</td><td>0,5</td></tr><tr><td>Compatibility/interop</td><td>0,1</td><td>2,6</td><td>2,6</td></tr><tr><td>Use/Usage</td><td>0,0</td><td>3,2</td><td>3,3</td></tr></tbody></table>
<figure><img src="image_6.png" /><figcaption>Figure 4 – Example of graphical representation using a radar chart</figcaption></figure>

#### 4.4.5 Relative importance of different criteria

While an assessment grid makes it possible to thoroughly and objectively assess functional and technical criteria, it is important to bear in mind that it should not be used as the sole basis for ELN selection; it should be compared with other relevant criteria (cost, ease of adoption, etc.). The relative importance of these different factors can only be established with a clear understanding of the economic and policy context of the ELN implementation project.

### 5 Comparative list of tools

The working group compared a selection of available solutions based on the methods described in section 4.

#### 5.1 Selection of tools studied
The tools selected are listed in the table below. The analyses were carried out on the software versions available in May-June 2021, unless stated otherwise:

<table><caption>Table 3: list of assessed tools</caption><thead><tr><td>Name</td><td>Publisher</td><td>Version</td><td>Type</td></tr></thead><tbody><tr><td>BIOVIA</td><td>Dassault Systèmes</td><td></td><td>Commercial</td></tr><tr><td>Chemotion</td><td>Karlsruhe Institute of Technology</td><td>0.5.0</td><td>Open Source</td></tr><tr><td>ElabFTW_dev</td><td>Deltablot</td><td>4.0</td><td>Open Source</td></tr><tr><td>ElabFTW_QeR</td><td>Deltablot</td><td>3.6.6</td><td>Open Source</td></tr><tr><td>Elog</td><td></td><td></td><td>Open Source</td></tr><tr><td>FindMolecule</td><td>FindMolecule, inc.</td><td></td><td>Commercial</td></tr><tr><td>Jogl</td><td>Just One Giant Lab MVAC</td><td></td><td>Open Source</td></tr><tr><td>Jupyter notebook</td><td></td><td></td><td>Open Source</td></tr><tr><td>LabArchives</td><td>Labarchives LLC</td><td></td><td>Commercial</td></tr><tr><td>LabBook</td><td>eNovalys</td><td></td><td>Commercial</td></tr><tr><td>LabCollector</td><td>Agilebio</td><td>6.032 with ELN add-on 4.13</td><td>Commercial</td></tr><tr><td>Labforward</td><td>LabForward gmbh</td><td></td><td>Commercial</td></tr><tr><td>LabGuru</td><td>Biodata</td><td></td><td>Commercial</td></tr><tr><td>Mbook</td><td>mestrelab</td><td>2.1.1</td><td>Commercial</td></tr><tr><td>OSF.io</td><td>Center for Open Science</td><td></td><td>non-profit or organisation</td></tr><tr><td>RSpace</td><td>ResearchSpace</td><td></td><td>Commercial</td></tr></tbody></table>

The tools that we studied were chosen with an even distribution of several criteria in mind:

* Tools under open licence and commercial licence

* Tools specialising in certain scientific fields (biology, chemistry, etc.) and general-purpose tools

* Tools only available online (Software as a Service – SaaS) and tools that can be installed on-site (on-premise software).

We also chose to focus on tools that are already in use or deployed at our own institutions in order to ensure the highest assessment quality possible.

Additionally, we also included a few outsiders that were not originally ELNs, but ended up being used for this purpose.

Appendix 5 contains links to the publishers of the assessed solutions and a list of other tools that were considered but could not be analysed due to time limitations.
## 5.2 Results

An overview of the results is provided in Figures 5 and 6. We chose the simplest scoring method—a colour code:

* green = the solution meets the requirement with varying degrees of success

* red = the solution does not meet the requirement or the information is unavailable

*Note: We did not have the means to fully assess eLog, so its technical specifications column is blank.*

## 5.3 Comments

This type of table does not take into account the relative weight of the criteria or the requirements specific to each research approach. It can therefore only be used to eliminate solutions that do not meet a criterion or a set of criteria that would be considered essential. The working group was not expected to give out of context recommendations for one tool or another.

The list of solutions is therefore non exhaustive and scoring remains incomplete. This is partly because it is divorced from real-world conditions, and partly because assessments were conducted in a limited period of time by a diverse group of people.

Nevertheless, these assessments still serve to emphasise the importance of correctly establishing selection criteria and their weight in a thorough assessment process.
<table><caption>Figure 5 – Feature scores for the 16 assessed solutions</caption><thead><tr><th>Feature</th><th>Biovia</th><th>Chemotion</th><th>eLabFTW_dev</th><th>eLabFTW_QeR</th><th>elog</th><th>FindMolecule</th><th>Jogl</th><th>Jupyter</th><th>LabArchive</th><th>LabBook</th><th>LabCollector</th><th>LabForward</th><th>Labguru</th><th>Mbook</th><th>OSF</th><th>RSpace</th></tr></thead><tbody><tr><td>Description of experiments and results</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Ability to duplicate experiments (without the research results) as a model/template for a new experiment</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Description of protocols</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Ability to duplicate protocols as a model/template for a new protocol</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Attachment management: office (PDF, audio, video, etc.) or scientific and technical (git, chromatograms, spectra, images, etc.)</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Ability to export materials from the ELN to a printable format</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Access to external data (i.e. not stored in the ELN), raw or not, for reading and for reference</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Data structuring with internal search engine and filter; use of keywords (tags)</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Accessible content (for mining)</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Ability to associate metadata with processed datasets, whether generic (dates, authors, etc.) or specific (ontologies, taxonomies, etc.)</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Use and maintenance of equipment (including notifications)</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Stock management (reagents, cell lines, etc.)</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Tools for simple calculations (basic mathematical functions, unit conversion, etc.)</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Tools for complex calculations (statistics or analyses requiring a programming language)</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Molecule building/design tool</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Notes on experiments or protocol (during experimentation), reports, etc.</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Ability to cite content from bibliographies (such as labtex.bib Text)</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Access to workstation office tools</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Management of software tools and code (continuous integration, continuous deployment, version control, etc.)</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Management of sharing by users</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Shared agenda</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Collaborative document editing (online and simultaneously by several users)</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Annotation of content and comment insertion</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr></tbody></table>
<table><caption>Figure 6 – Technical specifications scores for the 16 assessed solutions</caption><thead><tr><th>Category</th><th>Technical specifications</th><th>Biovia</th><th>Chemotion</th><th>eLabFTW/d</th><th>eLabFTW/d</th><th>FindMolecule</th><th>Joni</th><th>JupMer</th><th>Lab/Archive</th><th>LabBook</th><th>LabCollecto</th><th>Labforward</th><th>Labguru</th><th>Mbook</th><th>OSF</th><th>RSpace</th></tr></thead><tbody><tr><td rowspan="3">Data protection</td><td>Compliance with the GDPR for personal user data</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Compliance with the GDPR for research data: personal data of research participants if stored in the ELN</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Have managed, redundant, and backed-up storage (remote sites)</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td rowspan="6">Integrity</td><td>Compliance with regulatory or legislative requirements governing the hosting location of research data</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Enable document signing (hashing)</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Enable timestamping of activities and timestamping of documents</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Enable unique input identification (protocol, experiment, score, etc.) - UUID</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Ensure versioning/monitoring of changes</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Enable the implementation of predefined workflows (e.g. a set of required actions in the description of an experiment or result)</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td rowspan="3">Authentication/security</td><td>Enable electronic signature by participants</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Use a secure authentication system</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Enable the creation of user management rules: access permissions by role, project, and life cycle (participant arrival/departure, project launch/completion, etc.)</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td rowspan="4">Data backup</td><td>Encrypt data transport (TLS, RGS-compliant algorithms)</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Encrypt data</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Include regular backups and restores</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Define data retention periods and conditions for archiving or destruction</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td rowspan="2">Confidentiality</td><td>Ensure the confidentiality of exported data for backup or archiving.</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Safeguard the confidentiality of information that may represent valuable know-how or confidential business information (safeguard innovation in the event of a patentable invention).</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td rowspan="7">Compatibility/interoper</td><td>Safeguard the confidentiality of information exchanged with a partner in the context of a collaboration</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Compatibility with different operating systems (Windows, Linux, etc.), for servers</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Allow data to be managed (Import/Export) in a structured format (e.g. in case of software or administrative modifications)</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Allow integration with a local information system (databases, directories, etc.)</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Enable links with data repositories</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>When available, use open standards for input or output data to allow information to be permanently re-used.</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Enable data sharing through data integration and exposition services, via WebServices or REST APIs if possible</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td rowspan="10">Use/Usage</td><td>Ability to easily install updates and upgrades, while guaranteeing accessibility, if used from a web browser</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Ergonomic and intuitive</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Enable modularity/customisation/flexibility</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Have the necessary documentation (guide, tutorials, instruction manuals, etc.)</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Access to source code in order to manage, if necessary, the development of the tool, its maintenance, and most importantly, the data it contains</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Supported by a user community</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Multilingual support (at least English in addition to the local language)</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Allow offline use with subsequent data synchronisation</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Be usable on the move (on tablets and smartphones)</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Take environmental footprint into account: consumption of electricity by servers, etc.</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr></tbody></table>
# 6 Successfully conducting an ELN implementation project

## 6.1 Introduction

The aim of this chapter is not to dictate how a project should be executed in general, but rather to underline the particularities and challenges inherent to a project of this nature.

## 6.2 Project stakeholders and leadership

In addition to the stakeholders typically involved in a software implementation project (i.e. the purchasing department, legal department, and information systems department), it is also important to involve very early on the departments that stand to gain the most from the following benefits of an ELN's use:

* Quality

* Open science

* Promotion

* Scientific integrity

* Archiving

* Data security

* Data management

Stakeholders and project leaders need the input of people that represent future users. Support from the management team of either a research unit or a national research institution is a guarantee of the project's success.

It can be difficult to reconcile the duties of legal experts (who need to ensure GDPR compliance) with the principles of Open Science.

## 6.3 Pre-project

In the pre-project phase, it is necessary to:

* Find out about any recommendations from the relevant supervisory ministry(ies) or institutions: do they recommend or require a particular solution or selection method?

* Find out what the situation is at other institutions of potential co-supervisors: do they recommend or require a particular solution?

* Properly identify target users: e.g. based on discipline and also on research approach (experimental, analytical, or observational); see paragraph 2.2.2

* Consider the appropriate number of solutions: the simplest option would be to choose not one, but several solutions that
could better address the needs of different communities. This
could also be achieved by choosing a simple solution that has
additional modules.

* Consider whether the use of the selected tool(s) is mandatory or not.

* Check data security constraints (e.g. hosting).

* Think about how ELNs will be allocated: to a research project, a team, a sub-group, a unit, or to a particular device.

## 6.4 Pilot phase

ELN deployment is easier if the solution was chosen by a panel of users during a pilot phase. This can start with the use of a small number (2 to 4) of selected tools by a group of future users during a pre-study phase (see Chapter 4). This group should be as representative as possible of the range of future applications of the tool (team structure, disciplines, research approaches, location, etc.). This phase can last between a quarter and a year with periodic reviews and user satisfaction surveys.

## 6.5 Call for proposals

Institutions or laboratories can use the features and technical specifications grid in Chapter 4 as a guide for their requirements, so that they can analyse tools that may meet their needs and ensure that factors are not omitted. If a tool hosted by a third party is being sought, the requirements should stipulate that details regarding the location, hosting conditions, access conditions, etc. are to be provided.

It can be beneficial to request:

* that the response to the call for proposals include a planned development roadmap for the submitted solution. This will allow an assessment, for example, of whether the solution could be supplemented with components that could make it similar to a LIMS, or whether its insertion into the researcher's digital working environment is likely to be easy.

* that the tool's use can be maintained in read-only mode once the contract has ended.

## 6.6 Deployment

Deployment of the chosen solution is best done gradually so that the quality of the technical architecture and available support can be assessed prior to heavy usage.

## 6.7 Change of tools

One major consideration to keep in mind is that the limitations of public tenders can
mean that it may be necessary to switch to a different tool once the conditions for
renewal of the licence are exhausted<fnref n="18" />. This can be very laborious if data transfer
cannot be automated and/or if there is a large number of notebooks. The integrity of
all data must be guaranteed. Training all users to work with a new ELN can also have
significant financial costs and require considerable effort.

In short, research would be slower down during the transition from one ELN to
another

6.8 Change management

This is an important part of a project's success. For many researchers, the standard
laboratory notebook is an everyday tool. This is why it is important for them to quickly
master the electronic lab notebook and to be able to quickly find answers to their
questions. Additionally, certain ELN features can lead to significant changes in the way
work is organised: collaborative work (notebooks are usually shared by a team and are
constantly accessible by the Principal Investigator (PI)), remote access (enabling
telecommuting), etc. Therefore, it is necessary to provide appropriate support for
both the end user and the notebook administrator (team leader, PI).

To start with, it seems sensible to (re)acquaint users with good practices regarding
the use of a lab notebook, whether in physical or electronic form. One specific
obstacle involves teams that have never used physical laboratory notebooks because
the tool was considered to be totally unsuitable for their work, and they did not
recognise its value for the protection and promotion of their results.

<footnote n="18">¹⁸ This is the case when the purpose of the contract is to acquire an ELN solution and not to implement a given solution</footnote>
<figure><img src="image_7.png" /><figcaption>**Figure 7 - Factors for successful software deployment** (taken from the book "Pro en Conduite du changement", Juliette Ricou and Valérie Moissonnier, 2018, Vuibert)</figcaption></figure>

The primary fears regarding the use of such a tool are to do with aspects related to computing in general: ergonomics, difficulties in implementation (during field assignments in particular), costs (licences, equipment), human resources (technical skills, assistance, support), as well as a lack of confidence in the reliability of computing tools and the continuity of research results.

The change management process must be structured and planned, and involves two major project phases:

* assistance and troubleshooting during the setup phase and in the event of technical problems;

* assistance and user support during the usage phase if difficulties arise when using the tool.

In the first case, several options are possible depending on the implementation of theselected tool: support can be provided externally (by a dedicated service provider or by a solution provided as part of the tool's acquisition) or internally by local or national information systems departments, etc.

In the second case, support can be provided internally through training, education, and dialogue with the user community. The publisher of the selected tool may also
provide support in the event of technical problems (chat, online help, hotline, etc.)

Change management should also be provided in various forms:

* learning initiatives, presentations of the study and choices made, individual and group training in the use of the tool,

* the creation of user communities,

* availability of documentation and video tutorials. In the context of Higher Education, Research, and Innovation, the creation of MOOCs (Massive Open Online Courses - remote learning that can accommodate a large number of participants) could be explored as a means of validating students' knowledge, for example.

Additionally, it is necessary to set up support services to address users' difficulties and troubleshooting needs during the installation phase or when using the tool.

## 6.9 HR profiles and scaling

The relatively rapid turnover of doctoral and post-doctoral students in laboratories means that support services are essential.

The institution must have one (or more) designated point(s) of contact whose responsibility is to handle questions regarding research data traceability and management (a specific department, committee, technical advisor, etc.), data security, etc.

A *Lab Data Manager* role (which covers more than just the ELN) can be recommended for laboratories in order to manage data, configure the ELN, and ensure that newcomers are familiar with the system.

## 6.10 Joint supervision

How does one deal with situations where a laboratory's co-supervisors recommend different solutions? In such circumstances a list should be made of the policies and strategies employed by the unit's co-supervisors, if they exist. The next step would be to check whether a co-supervisor recommends an electronic solution and whether its use would be worthwhile to the unit in terms of technical functionality and deployment (access, storage, and security conditions, etc.) If necessary, the solutions can be reviewed taking into account the unit's stated needs and expectations and in accordance with its rules (e.g. competition rules or procurement rules).

## 7 Conclusion

The fact that so many ELN options have recently been developed and that research organisations are keen to adopt such solutions clearly demonstrates the growing importance of ELNs in the development of digital tools for research teams, in data
quality assurance, and in research practices. Given the proliferation of available options, the diversity of research approaches, and the need to ensure interoperability both with other tools and other ELNs, making a well-informed choice is of paramount importance.

With this in mind, the working group produced:

* Lists of the features and technical specifications expected of an ELN

* An example of how to weight these criteria according to the relevant approach: experimental, analytical, or observational.

* An example of how to score ELNs and graphically represent these scores to allow for comparison.

* A comparison of around fifteen proprietary and open source tools, depending on whether or not they met the required criteria.

* Recommendations on how to go about the implementation of an ELN after its selection.

These materials are freely available and reusable, and represent the main deliverables of the group's assignment, which ended in July 2021.

This report represents all of the work that was carried out, and it was delivered in its present state in September 2021.
# A Appendices

## A.1 Non-WG contributors

In addition to the members of the working group, its co-leads would like to thank
the following people for their contribution:

* Jérémie Leonard, Data Processing Officer, University Lyon 1

* Nicolas Argento, Applications Team Leader, École polytechnique fédérale de Lausanne

* Philippe Rizand, ISD Director, Curie Institute

* Nicolas Carpi, designer of eLabFTW, UMR144, CNRS, and Curie Institute

* Nicolas Renard, UMR6521, CNRS

* Alain Rivet, UPR530 - Henri Valeins, UMR5536 - Yaël Hersant - UMR6296, members of the Qualité en Recherche network, CNRS

* Ambra Mari, team leader, CNRS and University Lyon 1

* Frédéric Leroux, director, UMR7042, CNRS

* Auriane Denis Méyère, Céline Charavay, Eric Faudry - ELN administration team, UMR 5075 - IRIG, CEA

* Fanny Brizzi, ELN project leader, DSI, Inserm

The WG co-leads would also like to thank the CNRS electronic lab notebook WG,
which shared its results before they were made final.

## A.2 WG mission statement

The mission statement sent to the project leaders, as well as the working group's
roadmap, are included in full on the following 4 pages.
<figure><img src="image_8.png" /></figure>

MINISTÈRE
DE L'ENSEIGNEMENT
SUPÉRIEUR,
DE LA RECHERCHE
ET DE L'INNOVATION

Liberté

Egalité

Fraternité

Direction générale
de la recherche et de l'innovation

Affaire suivie par :
Claire LEYMONERIE
Tél : 01 55 55 80 51
Mél : claire.leymonerie@recherche.gouv.fr

1 rue Descartes
75231 Paris SP 05

Paris, le 17 novembre 2020

Messieurs,

Les cahiers de laboratoire sont des outils scientifiques et juridiques qui permettent de garantir la traçabilité des expériences de laboratoire, de maintenir la continuité du cheminement intellectuel aboutissant aux résultats scientifiques, d'identifier les contributions de chacun et d'établir les dates d'acquisition des résultats. Ils contribuent à la reproductibilité et au caractère cumulatif des connaissances scientifiques et s'inscrivent donc pleinement dans une démarche de science ouverte.

Il importe donc d'éclairer les choix qui sont faits par les équipes scientifiques parmi les différentes solutions actuellement disponibles.

Nous vous confions à cet effet le pilotage d'un groupe de travail qui aura pour objectifs d'établir une vision partagée sur la définition, le périmètre fonctionnel et les usages du cahier de laboratoire électronique, de produire des recommandations sur les critères de choix d'un outil et son interopérabilité avec d'autres outils, dont notamment les entrepôts de données. Vous établirez ainsi une analyse comparative d'outils existants (sans viser à l'exhaustivité) et en vous appuyant sur les études déjà réalisées par certains établissements ou le réseau CURIE.

Vos travaux s'inscriront dans le cadre du Plan national pour la science ouverte et seront menés en lien avec la Direction générale de la recherche et de l'innovation du ministère de l'Enseignement supérieur, de la Recherche et de l'Innovation, et le collège des données de la recherche du Comité national pour la science ouverte. Vous présenterez les résultats des travaux menés devant le Secrétariat permanent de la science ouverte.

Gilles Mathieu
INSERM, Département Systèmes d'Information

Dominique Pigeon
INSERM, Département Systèmes d'information

Tovo Rabemanantsoa,
INRAE, Direction pour la science ouverte

Copies :
Véronique Stoll et Pierre-Yves Arnould, pilotes du collège données de la recherche du Comité pour la science ouverte
Anne Vanet, responsable du groupe de travail « réussir l'appropriation de la science ouverte »

Pièce jointe : Feuille de route du groupe de travail « Cahiers de laboratoire »
Vous veillerez à coordonner vos travaux avec ceux du groupe de travail « réussir l'appropriation de la science ouverte » piloté par Anne Vanet au sein du Comité pour la science ouverte et avec le projet « cahiers de laboratoire électronique » en cours au CNRS. Pour cela, votre groupe de travail comptera dans sa composition un membre nommé au sein du collège de données et du groupe de travail « réussir l'appropriation de la science ouverte ». Plus globalement, la composition du groupe devra autant que possible représenter la diversité disciplinaire et structurelle de l'ESR.

Vous trouverez joint à ce courrier une feuille de route précisant les objectifs et les modalités de fonctionnement du groupe de travail.

Nous vous remercions pour votre engagement dans cette mission et vous prions de recevoir, Messieurs, nos cordiales salutations.

<figure><img src="image_9.png" /></figure>

Isabelle Blanc
P/O le bureau de la Science Ouverte
## 1. Background

The laboratory notebook is a logbook used to record the day-to-day activities of research projects, experiments, and employed protocols. For patent offices in particular, it provides proof of an invention, of its inventors, and therefore of its rightholders. The notebook simplifies quality processes, contributes to the reproducibility of data and research results, and meets legal and contractual requirements.

Due to its crucial place in research activities, the electronic lab notebook represents a strategic tool for the team and unit, as well for institutional supervisors. It is an essential tool for research projects committed to the open science movement. The choice of tool, its configuration, and above all how it is used are of capital importance

## 2. Objectives and deliverables

The primary objectives of the "electronic lab notebooks" working group will be to:

* **Establish a shared vision of the definition, functional scope, and purposes of the "electronic lab notebook".** It will be particularly important to establish what such a tool does and does not do, what it is suitable for, and—according to the context—which tools can be used to supplement it.

* **Produce a set of recommendations on the criteria for choosing a tool.** The criteria will be established in an objective manner according to functional needs, disciplines, and fields of research, as well as institutional requirements. These criteria will also be used for comparative analyses of different tools and to measure their effectiveness in and/or suitability to different usage contexts.

* **Produce a set of recommendations for interoperability between different tools.** In the French academic multi-supervisor context, the use of different tools should not result in research work being compartmentalised and isolated. Laboratory notebooks must be able to seamlessly (meaning without the need for specific licences, reverse engineering, etc.) merge with existing computer environments and allow, in particular, for data movement (i.e. data exposition) to data repositories.

* Draw up a (non-exhaustive) comparative list of tools based on the recommended criteria.

This list can serve as an example of how to conduct a selection study.

## 3. Methodology

### 3.1. Group composition

It is recommended that a group of between 8 and 12 people be assembled, with each person representing one or more of the following:

* **Fieldwork:** users or potential users of notebooks, preferably from different scientific communities
* Scientific and legal integrity, and intellectual property: lawyers, data specialists, quality managers, etc.

* **Technical and infrastructural expertise:** service operators who are experienced in efficiency, backups, security, etc.

* **Technical and data expertise:** data managers, platform managers who are experienced in formatting, interoperability, archiving, etc.

* **Strategic and content promotion expertise:** institutional or scientific representatives, content promotion managers, etc.

The group's composition should represent the disciplinary and structural diversity of the higher education and research community as best as possible. Distribution by establishment of origin and by scientific discipline will therefore serve as selection criteria for the group's final composition. The group should also include a member of the Data College.

## 3.2. How it works

The group will carry out its work between October 2020 and June 2021.

Due to the dispersed nature of the group, the majority of meetings and workshops are expected to be held by videoconference, according to a schedule that will be jointly determined once the group is established.

In addition to these regular meetings there will be quarterly progress and feedback meetings, in January 2021 and April 2021, with the MESRI open science team and the Data College. The group's findings and final deliverable will be presented at the end of their work in June 2021.
## A.3 Detailed look at uses – observational approach

### A.3.1 Ethnographic approach and "qualitative" analysis (sociology, anthropology, design, participatory research, etc.)

In ethnographic research (e.g. in anthropology or so-called "qualitative" sociology), the "logbook" or "field notebook" provides a framework for both recording the data collected and explaining the rationale behind decisions made during the research. These documents play an important role in structuring thought processes throughout research. Elucidating this data collection process and its subsequent analysis represents a benchmark in terms of quality and reliability in the research communities employing these approaches (Vanlint, 2019).

During research, ethnographers may use several physical notebooks or loosely formatted documents (Weber, 1991) that serve different functions (observational or analytical notes, etc.). In a digital medium, these often consist of a variety of text files containing analysis reports/memos, notes from interviews, comments, etc. These text files (from a text editor) may also include sketches, graphics, and audio or video recordings. There are currently few digital ready-to-use tools that assemble these sometimes disparate notes and files. General-purpose online note-taking tools (such as Evernote) can be used. In the field of design, some platforms are being developed that facilitate collective note-taking in various formats (audio, text, video) and provide a "shared history" of events for participatory research projects.

During the analysis of observations obtained and data gathered, Qualitative Data Analysis (QDA) tools are used to elucidate interpretations of the coding of information from a body of text (verbatim accounts, etc.).

#### 7.1.1 A few examples:

* An example of a physical sociological research field notebook used in participatory research:
  https://reflexivites.hypotheses.org/11932

* The "Lévi-Strauss and his Nambikwara fieldnotes – NAMBIKWARA" project.

An ANR (French National Research Agency) project to decipher, digitise, and produce a scholarly edition of Claude Lévi-Strauss' field notebooks written during his second visit to the Nambikwara people (May 1938 – January 1939):
https://anr.fr/Projet-ANR-18-CE27-0017

* Various digitised and anonymised surveys undertaken as part of the BeQuali project (a qualitative social science survey bank):
https://bequali.fr/en/

For example, the qualitative survey on "The working conditions of pregnant women" by Anne Marie Devreux (with anonymisation assistance from the BeQuali team):
https://cdsp.sciences-po.fr/en/ressources-en-ligne/ressource/cdsp_bq_s12/

* L'Atelier des Chercheurs ("Researchers' Workshop") provides a number of open source prototypes employed in the context of a variety of standard and collaborative observational practices in design projects:

https://latelier-des-chercheurs.fr/infos

* Summary of tools aimed at assisting content analysis, interpretation of real-life accounts, or interviews in qualitative sociology:

https://www.squash.uliege.be/logiciels/

### A.3.2 Data collection and recording in challenging environments (archaeology, environmental sciences)

In archaeology, excavation notebooks used to exist in a format that allowed observations to be shared as research progressed. In the 1980s, these observational practices became more systematic and standardised, leading to the introduction of physical files that allowed for an analytical overview of the situation in the field and the recording of information under defined headings (Desachy, 2016). This information is now directly recorded digitally using field apps/tablets, resulting in the creation of relational databases and the development of field-based Geographic Information Systems (GIS).

Field data collection practices in challenging environments are also a central issue in ecology and the environmental sciences. What we now call "electronic field notebooks" actually represent various media and tools associated with these approaches. For example, these include mobile devices (phones, tablets, etc.) that are designed to withstand extreme environments. These are used with data collection and organisation software. The latter often include features like an "offline" mode and postponed data sync (for situations in which a network connection is not available), as well as collaborative features allowing the shared input of data into the research project's databases and GIS.

#### 7.1.2 A few examples:

* OpenData Kit is an open source application for the collection, management, and use of data in challenging environments.

* Pi4X4 is a rugged tablet project built using a Raspberry Pi.

https://odk-x.org/

* Presentations of various projects in ecology and the environmental sciences during an "electronic field notebook" workshop in March 2018. https://rbdd.cnrs.fr/spip.php?article270

* An example of data collection in a public health project in Africa using mobile phones and open data kit. https://www.youtube.com/watch?v=vB0jHQgVkEs

The tools most closely related to observational and analytical research work, as well as to field and investigative research, take a variety of forms depending on knowledge production modes and their validity. The idea of reproducing an experiment
(which underlies the notion of a "laboratory notebook") is antithetical to qualitative approaches that seek to observe and interpret social phenomena, either through the study of specific communities or people, or through the critical analysis of a body of text.

Nevertheless, there are a number of needs that digital tools are able to meet depending on specific usage methods and practices, e.g. use on mobile devices with "offline" mode, collected data backup and protection, internal search engine, linking to data repositories, collaborative features, and access rights management.

### 7.1.3 Further information:

* Baribeau, Colette. 2005. *L'instrumentation dans la collecte de données - Le journal de bord du chercheur* (Instrumentation in data collection - The researcher's logbook) Hors-Série (2): 17.

* Desachy Bruno. 2016. *Du Carnet de Fouilles Aux Systèmes d'information Archéologiques de Terrain: (From the Excavation notebook to Field Archeology Information Systems:)* Quelques Remarques Sur l'évolution de l'enregistrement et l'impact de l'informatisation (Some remarks on the evolution of information registration and the impact of computerisation) Médiathèque de La MSH Mondes · Médiathèque MSH Mondes.

http://mediatheque.mae.cnrs.fr/s/fr/item/883

* Gruson-Daniel, Célya. 2019. *Les outils numériques d'enquête (Digital research tools).* In Guide décolonisé et pluriversel de formation à la recherche en sciences sociales et humaines (decolonised and pluriversal research training in the social and human sciences). Éditions science et bien commun (Science and common good editions).

https://scienceetbiencommun.pressbooks.pub/projetthese/chapter/les-outils-numeriques-denquete/

* Olivier de Sardan, Jean-Pierre. 1995. "La politique du terrain Sur la production des données en anthropologie" ("Fieldwork policy for the generation of data in anthropology"). Survey. Archives de la revue Enquête, issue 1 (October): 71-109.

https://doi.org/10.4000/enquete.263

* Pecqueux, Anthony. n.d. *L'écoute comme politique de l'enquête* (Listening as a policy of inquiry): 5/ Écoute acousmatique, ou Le nez dans le carnet ou dans le guidon ? (5/Acousmatic listening: absorbed with the notebook or absorbed in the task) Bulletin. Espaces réflexifs (blog). Visited on 27 May 2020.

https://reflexivites.hypotheses.org/11932

* Vanlint, Alice. 2019. "Le journal de bord ou de terrain." dans Guide décolonisé et pluriversel de formation à la recherche en sciences sociales et humaines. ("The log or field log." in "A decolonised and pluriversal guide to research training in the social sciences and humanities"). Éditions science et bien commun (Science and common
good editions).

https://scienceetbiencommun.pressbooks.pub/projetthese/chapter/le-journal-de-bord-ou-de-terrain/

* Weber, Florence. 1991. L'enquête, la recherche et l'intime ou : pourquoi censurer son journal de terrain ? ("Investigation, Research, and Intimacy, or Why You Should Censor Your Field-Diary?") Espace Temps 47 (1): 71–81.

https://doi.org/10.3406/espat.1991.3788

## A.4 Description of the criteria

The features and specifications listed in section 4 are further described below when deemed necessary:

<table><thead><tr><td>Feature</td><td>Description</td></tr></thead><tbody><tr><td>Description of experiments and results</td><td></td></tr><tr><td>Ability to duplicate experiments (from search results) as a model/template for a new experiment Description of protocols</td><td></td></tr><tr><td>Ability to duplicate protocols as a model/template for a new protocol new protocol</td><td></td></tr><tr><td>Attachment management: office (PDF, audio, video, etc.) or scientific and technical (git, chromategrams, spectra, images, etc.)</td><td></td></tr><tr><td>Ability to export materials from the ELN to a printable format</td><td>The software must allow the export of all its content for subsequent printing. This means formatting all kinds of materials (images, diagrams, texts, etc.).</td></tr><tr><td>Access to external data (i.e. not stored in the ELN), raw or not, for reading and for reference</td><td>The ability to save links to data sets, ideally with a DOI (digital object identifier)</td></tr><tr><td>Data structuring with internal search engine and filter; use of keywords (tags)</td><td>The ability to execute simple searches of material stored in the notebooks, primarily via the user interface</td></tr><tr><td>Accessible content (for mining)</td><td>A TDM (Text and Data Mining) tool can access the data in the ELN in order to run searches.</td></tr></tbody></table>
<table><tr><td>Ability to associate metadata with processed datasets, whether generic (dates, authors, etc.) or specific (ontologies, taxonomies, etc.)</td><td></td></tr><tr><td>Use and maintenance of equipment (including notifications)</td><td>A CMMS component (Computerised Maintenance Management System) must be proposed to the user. This will allow for the management of equipment reservation and use schedules, as well as the management of equipment servicing and maintenance. A notification feature (equipment not returned, preventive maintenance or calibration necessary, equipment not returned from supplier, etc.) must also be included.</td></tr><tr><td>Stock management (reagents, cell lines, etc.)</td><td>A stock management component must be available in the electronic lab notebook. It provides users with an overview of supplies and allows them to plan the procurement of consumables (reagents, cell lines, chemical products, etc.).</td></tr><tr><td>Tools for simple calculations (basic mathematical functions, unit conversion, etc.)</td><td></td></tr><tr><td>Tools for complex calculations (statistics or analyses requiring a programming language) programming language)</td><td></td></tr><tr><td>Molecule building/design tool</td><td></td></tr><tr><td>Notes on experiments or protocol (during experimentation), reports, etc.</td><td>Ability to create simple documents, linked to others.</td></tr><tr><td>Ability to cite content from bibliographies (such as labtex.bib Text)</td><td>Ideally, it should be possible to connect to a bibliographic database, (e.g. PubMed or HAL).</td></tr><tr><td>Access to workstation office tools</td><td>This allows files to be viewed or modified without leaving the ELN. Clicking on the file in the ELN opens the corresponding application, if it is installed on the workstation.</td></tr><tr><td>Management of software tools and code (continuous integration, continuous deployment, version control, etc.)</td><td>A feature rarely found in ELNs, the alternative being a connection to a specialised server such as GitLab or SourceSup.</td></tr></table>
<table><tr><td>Management of sharing by users</td><td>Each user must be able to allow other users access to information related to a project or experiment. This allows for quick but fully informed sharing of information with rightholders.</td></tr><tr><td>Shared agenda</td><td>If an electronic lab notebook includes a project, for example, and this project includes milestones that need to be met or experiments that need to be carried out (i.e. involving several people), the availability of a shared agenda for the rightholders is essential.</td></tr><tr><td>Collaborative document editing (online and simultaneously by several users)</td><td>The electronic lab notebook must allow several users to edit documents online and simultaneously. This ensures that all changes are taken into account when editing simultaneously and prevents a document from being locked because someone is already editing it.</td></tr><tr><td>Annotation of content and comment insertion</td><td>This involves adding to descriptions of experiments or protocols without creating a new document.</td></tr></table>

The list of technical specifications cited in section 4 is detailed below:

<table><thead><tr><td>Specification</td><td>Description</td></tr></thead><tbody><tr><td>Compliance with the GDPR for personal user data</td><td></td></tr><tr><td>Compliance with the GDPR for research data: personal data of research participants if stored in the ELN</td><td></td></tr><tr><td>Have managed, redundant, and backed-up storage (remote sites)</td><td></td></tr><tr><td>Compliance with regulatory or legislative requirements governing the hosting location of research data</td><td></td></tr></tbody></table>
<table><tr><td>Enable document signing (hashes)</td><td>This is not an individual's signature (see "Authentication/security/Enable electronic signature by participants" criterion) but a signature by a program that generates a sequence of characters specific to the file (its "hash"). If the file is made available to a third party, the third party must be able to retrieve the same sequence of characters by using the same signing mechanism (even if the file was encrypted and decrypted), thus proving that the authenticity of the file has been protected. See https://www.cnil.fr/fr/securite-chiffrer-garantir-lintegrite-ou-signe</td></tr><tr><td>Enable activity and document timestamping</td><td></td></tr><tr><td>Enable unique input identification (protocol, experiment, score, etc.) – UUID</td><td></td></tr><tr><td>Enable versioning/change tracking</td><td>Each action carried out must be indicated, for example with incremental version numbers and/or a "Last Modified" date. Versioning ensures that no action can be carried out without being recorded.</td></tr><tr><td>Enable the implementation of predefined workflows (e.g. a set of required actions in the description of an experiment or result)</td><td></td></tr><tr><td>Enable electronic signature by participants</td><td>Each person using the electronic lab notebook must be able to sign it electronically. Typically, a witness (a unit head, a colleague not working on the project, or someone from another team, etc.) must regularly sign the notebook in order to certify what is recorded in it, thereby guaranteeing its legal value.</td></tr><tr><td>Use a secure authentication system</td><td></td></tr><tr><td>Enable the creation of user management rules: access permissions by role, project, and life cycle (participant arrival/departure, project launch/completion, etc.)</td><td></td></tr></table>
<table><tr><td>Encrypt data transport (TLS, RGS-compliant algorithms)</td><td></td></tr><tr><td>Encrypt data</td><td>Each data backup must be protected by encryption to ensure confidentiality. It is inconceivable that a backup be used without any restrictions, just as applies to the electronic lab notebook.</td></tr><tr><td>Include regular backups and restores</td><td>In order to safeguard the contents of the electronic lab notebook, the user should be able to set up regular backups (with a specified regularity and possibly location) that are then automatically executed. It must also be possible to restore content in the event of errors.</td></tr><tr><td>Define data retention periods and data archiving or destruction data</td><td></td></tr><tr><td>Ensure the confidentiality of exported data for backup or archiving.</td><td></td></tr><tr><td>Safeguard the confidentiality of information that may represent valuable know-how or confidential business information (safeguard innovation in the event of a patentable invention).</td><td></td></tr><tr><td>Safeguard the confidentiality of information exchanged with a partner in the context of a collaboration.</td><td></td></tr><tr><td>Compatibility with different operating systems (Windows, Linux, etc.), for servers</td><td></td></tr><tr><td>Allow data management (Import/Export) in a structured format (e.g. in case of a software or administrative modification)</td><td></td></tr><tr><td>Allow integration with a local information system (databases, directories, etc.)<br/>Enable links with data repositories<br/>When available, use open standards for input or output data to allow information to be permanently re-used.</td><td></td></tr></table>
<table><tr><td>Enable data sharing through data integration and exposition services, via WebServices or REST APIs if possible.</td><td></td></tr><tr><td>Ability to easily install updates and upgrades, while guaranteeing accessibility, if used from a web browser</td><td></td></tr><tr><td>Ergonomic and intuitive</td><td></td></tr><tr><td>Enable modularity/customisation/flexibility</td><td></td></tr><tr><td>Have the necessary documentation (guide, tutorials, instruction manuals, etc.)</td><td></td></tr><tr><td>Access to source code in order to manage, if necessary, the development of the tool, its maintenance, and most importantly the data it contains</td><td>The proposed solution is open source, meaning that the software's source code is made available under a free, open licence. Ideally the source code is available on a software forge and is documented. The open source aspect can also be combined with a community development process. As far as the data itself is concerned, the Terms of Use and "data policy" specify the conditions of data use and retention. If there are any APIs, these are also documented and open.</td></tr><tr><td>Support by a community of users</td><td>It is crucial for the user to be able to contact other users (e.g. via an online forum) to find a solution to any questions or problems they may have. The user should also be able to post any solutions they find themselves.</td></tr></table>
<table><tr><td>Multilingual support (at least English)</td><td>As with most scientific communities, internationalism represents a fundamental aspect. Not everyone speaks (or is sufficiently fluent in) the home language of the laboratory they are working in, so it is essential for users to be able to select the language they use. This setting obviously only affects the UI of the electronic lab notebook and not its content. So it is fair to say that at least English should be an option.</td></tr><tr><td>Allow offline use with subsequent data synchronisation</td><td>When network access is not available (e.g. when conducting experiments outdoors or in a building with no network connection), the user must be able to use the electronic lab notebook in offline mode. Once a network connection is established, any new data and/or changes must then be synchronised.</td></tr><tr><td>Be usable on the move (on tablets and smartphones)</td><td>The electronic lab notebook must be compatible with mobile devices such as tablets and smartphones. This implies that the software's ergonomics are compatible with touchscreens and varying screen resolutions (icons not too small, numpad, etc.). It is also essential that the software can run in offline mode and can sync data afterwards.</td></tr><tr><td>Taking the environmental footprint into account: electricity consumption of servers, etc.</td><td></td></tr></table>

## A.5 List of tools analysed and not analysed

The list of tools assessed in section 5 is given below, along with notes for each tool:
<table><thead><tr><td>Name</td><td>Publisher</td><td>Type</td><td>Score</td></tr></thead><tbody><tr><td>BIOVIA<sup>19</sup></td><td>Dassault Systems</td><td>Commercial</td><td>USA storage by default.</td></tr><tr><td>Chemotion<sup>20</sup></td><td>Karlsruhe Institute of Technology</td><td>Open Source</td><td>Originally developed for organic chemistry, but can also be used for other fields. The development of new discipline-specific solutions is in progress.</td></tr><tr><td>ElabFTW_dev<sup>21</sup></td><td>Deltablot</td><td>Open Source</td><td>Standard version. Examples of usages are available<sup>22</sup></td></tr><tr><td>ElabFTW_QeR</td><td>Deltablot</td><td>Open Source</td><td>Version of the CNRS Qualité en Recherche network.</td></tr><tr><td>Elog<sup>23</sup></td><td></td><td>Open Source</td><td>Open source tool developed and maintained by the particle and nuclear physics community as a whole.</td></tr><tr><td>FindMolecule<sup>24</sup></td><td>FindMolecule, inc.</td><td>Commercial</td><td>ELN and chemical substances inventory management. Under Quebec law, free for the academic community</td></tr><tr><td>Jogl<sup>25</sup></td><td>Just One Giant Lab MVAC</td><td>Open Source</td><td>SaaS project management solution; cannot replace an ELN but can complement one.</td></tr><tr><td>Jupyter notebook<sup>26</sup></td><td></td><td>Open Source</td><td>Allows for development, documenting, and execution of code, as well as sharing of results.</td></tr><tr><td>LabArchives<sup>27</sup></td><td>Labarchives LLC</td><td>Commercial</td><td></td></tr><tr><td>LabBook<sup>28</sup></td><td>eNovalys</td><td>Commercial</td><td>Tool used by units in the DR10 of the CNRS and by universities (Polytechnic University of Hauts-de-France, Paris Sud, Strasbourg).</td></tr><tr><td>LabCollector<sup>29</sup></td><td>Agilebio</td><td>Commercial</td><td>Used at the CEA (French Alternative Energies and Atomic Energy Commission), mainly in biology. https://lab-collector.com/</td></tr><tr><td>Labforward<sup>30</sup></td><td>LabForward gmbh</td><td>commercial</td><td></td></tr><tr><td>LabGuru<sup>31</sup></td><td>Biodata</td><td>Commercial</td><td>Deployed at Inserm. ELN with some LIMS features.</td></tr></tbody></table>
<table><tr><td>Mbook<sup>32</sup></td><td>mestrelab</td><td>Commercial</td><td>for synthetic chemists.</td></tr><tr><td>OSF.io<sup>33</sup></td><td>Center for Open Science</td><td>non-profit organisation</td><td></td></tr><tr><td>RSpace<sup>34</sup></td><td>ResearchSpace</td><td>Commercial</td><td>Interfacing with repositories already implemented - Under British legislation.</td></tr></table>

The following list contains tools that were selected but not assessed by the group:

<table><thead><tr><td>Name</td><td>Publisher</td><td>Type</td><td>Score</td></tr></thead><tbody><tr><td>Scinote<sup>35</sup></td><td>scinote llc</td><td>Commercial</td><td>only available in cloud mode</td></tr><tr><td>Benchling<sup>36</sup></td><td>benchling</td><td>Commercial</td><td>mainly used in the private sector (including Sanofi, Syngenta, BP)</td></tr><tr><td>Labstep<sup>37</sup></td><td>Labstep Ltd.</td><td>Commercial</td><td>Only available in cloud mode, free for the academic community</td></tr><tr><td>ISIA</td><td></td><td></td><td>AnaEE France consortium</td></tr></tbody></table>

<footnote n="19">19 https://www.3ds.com/products-services/biovia/products/laboratory-informatics/electronic-lab-notebooks/biovia-notebook/</footnote>

<footnote n="20">20 https://www.chemotion.net/chemotionsaurus/</footnote>

<footnote n="21">21 https://www.elabftw.net/</footnote>

<footnote n="22">22 https://archive.fosdem.org/2021/schedule/event/open_research_using_elabftw/</footnote>

<footnote n="23">23 https://elog.psi.ch/elog/</footnote>

<footnote n="24">24 https://findmolecule.com/</footnote>

<footnote n="25">25 https://jogl.io/fr</footnote>

<footnote n="26">26 https://jupyter-notebook.readthedocs.io/en/stable/</footnote>

<footnote n="27">27 https://www.labarchives.com/eln-for-research/</footnote>

<footnote n="28">28 https://www.lab-book.org/</footnote>

<footnote n="29">29 https://labcollector.com/</footnote>

<footnote n="30">30 https://labforward.io/</footnote>

<footnote n="31">31 https://www.labguru.com/</footnote>

<footnote n="32">32 https://mestrelab.com/software/mbook/</footnote>

<footnote n="33">33 https://osf.io/</footnote>

<footnote n="34">34 https://www.researchspace.com/</footnote>

<footnote n="35">35 https://www.scinote.net/</footnote>

<footnote n="36">36 https://www.benchling.com/notebook</footnote>

<footnote n="37">37 https://www.labstep.com/</footnote>
<table><tr><td>Cassandre<sup>38</sup>, NVivo<sup>39</sup></td><td>University de Liège</td><td>Open Source</td><td>Cassandre and NVivo are Open Source programs for analysing qualitative data, used in HSS.</td></tr><tr><td>Hivebench</td><td>Elsevier</td><td>Commercial</td><td></td></tr><tr><td>Bookitlab<sup>40</sup></td><td>Prog4biz Software Ltd</td><td>Commercial</td><td></td></tr><tr><td>OpenDataKit<sup>41</sup></td><td></td><td>Open Source</td><td></td></tr><tr><td>Elabjournal<sup>42</sup></td><td>Bio-itech-Ependorf</td><td>Commercial</td><td>Used by the Pasteur Institute</td></tr><tr><td>Laby<sup>43</sup></td><td>KYLI SAS</td><td>commercial</td><td>Has some LIMS features. Developed in France.</td></tr><tr><td>Evernote<sup>44</sup></td><td>Evernote corporation</td><td>Commercial</td><td></td></tr></table>

<footnote n="38">38 https://github.com/Hypertopic/Cassandre</footnote>

<footnote n="39">39 https://www.qsrinternational.com/nvivo-qualitative-data-analysis-software/home</footnote>

<footnote n="40">40 https://bookit-lab.com/</footnote>

<footnote n="41">41 https://getodk.org/#features</footnote>

<footnote n="42">42 https://www.elabnext.com/products/elabjournal/</footnote>

<footnote n="43">43 https://laby.io/</footnote>

<footnote n="44">44 https://evernote.com/intl/fr</footnote>
B Glossary and abbreviations

<table><thead><tr><td>Term or Concept</td><td>Translation</td><td>Meaning</td></tr></thead><tbody><tr><td>ANSSI - Agence Nationale de la Sécurité des Systèmes d'Information</td><td>French National Agency for the Security of Information Systems</td><td>French national authority for the security and defence of information systems.</td></tr><tr><td>API</td><td>Application Programming Interface</td><td>Standardised set of classes, methods, features, and constants that serve as a front end through which software provides services to other software</td></tr><tr><td>Checksum, hashing, hash</td><td>n.a.</td><td>A process that uses input data to generate a digital signature that can be used to quickly identify the initial data, similar to the way a signature identifies a person.</td></tr><tr><td>CKAN</td><td>Comprehensive Knowledge Archive Network</td><td>Web application enabling the storage and sharing of data such as spreadsheets or database content.</td></tr><tr><td>CoSO - Comité pour la Science Ouverte</td><td>French Committee for Open Science</td><td>Attached to the Directorate-General for Research and Innovation of the French Ministry of Higher Education, Research, and Innovation.</td></tr><tr><td>Dataverse</td><td>Comprehensive Knowledge Archive Network</td><td>Open source web application for storing, sharing, citing, searching, and analysing research data.</td></tr><tr><td>DMP</td><td>Data management plan</td><td>A living document that helps the researcher or research project leader to establish a plan for the management of data used and generated during their research activity or project.</td></tr></tbody></table>
<table><tr><td>DOI</td><td>Digital Object Identifier</td><td>System for identifying resources, which can be digital resources, such as a film, a report, or scientific articles, but can also be people or any other type of object.</td></tr><tr><td>EBIOS (Expression des Besoins et Identification des Objectifs de Sécurité)</td><td>Expression of Needs and Identification of Security Objectives</td><td>Comprehensive information systems security risk management tool, compliant with the RGS and the latest ISO standards. It enables such risks to be assessed and dealt with.</td></tr><tr><td>ELN</td><td>Electronic Lab Notebook</td><td>See chapter 1 of the report.</td></tr><tr><td>Emacs</td><td>n.a.</td><td>A series of text editors with an expandable set of features. It is popular among programmers and more generally among people with more advanced computer skills.</td></tr><tr><td>ESRI (Enseignement Supérieur, Recherche et Innovation)</td><td>Higher Education, Research, and Innovation</td><td></td></tr><tr><td>EDM</td><td>Electronic Document Management</td><td>A computerised process for identifying, creating, producing, organising, distributing, and managing information and physical or electronic documents within an organisation.</td></tr><tr><td>Git</td><td>When asked why he called his software "git", which means "unpleasant person" in British English slang, Linus Torvalds quipped: "I'm an egotistical bastard, and I therefore name all my projects after myself. First 'Linux', now 'Git'."</td><td>Free distributed version control software.</td></tr></table>
<table><tr><td>GitLab</td><td>n.a.</td><td>Free forge software based on Git providing wiki functionality, a bug tracking system, continuous integration, and continuous delivery.</td></tr><tr><td>CMMS</td><td>Computerised Maintenance Management System</td><td>A software-assisted management method for a company's maintenance departments to help them in their work.</td></tr><tr><td>HAL (Hyper Articles en Ligne)</td><td>n.a.</td><td>Multidisciplinary open archive intended for the upload and dissemination of research-related scientific articles (published or not) and theses from French or foreign academic and research institution, as well as from public or private laboratories.</td></tr><tr><td>HPC</td><td>High-performance computing</td><td>Supercomputing.</td></tr><tr><td>ICOS</td><td>Integrated Carbon Observation System</td><td>European infrastructure dedicated to observing and quantifying greenhouse gas fluxes.</td></tr><tr><td>INSB (Institut National des Sciences Biologiques)</td><td>French National Institute of Biological Sciences</td><td>One of the CNRS' ten specialised institutes.</td></tr><tr><td>LIMS</td><td>Laboratory Information Management System</td><td>See paragraph 2.2.3 of the report.</td></tr><tr><td>ORCID</td><td>Open Researcher and Contributor ID</td><td>Non-proprietary alphanumeric code to uniquely and perpetually identify researchers and authors of scholarly and scientific contributions.</td></tr></table>
<table><tr><td>Benchmarking</td><td></td><td>The practice of finding companies that perform a given process or task most efficiently, studying them, and then tailoring this process to one's own company.</td></tr><tr><td>PI</td><td>Principal Investigator</td><td>In North America, refers to the holder of an independent grant and the lead researcher for the grant project, usually in the sciences; often synonymous with "head of the laboratory" or "research group leader".</td></tr><tr><td>PID</td><td>Persistent IDentifier</td><td>A string of alphanumeric characters that persistently identifies a resource regardless of its location. This resource can be any real or conceptual object (a person, a structure, research results, etc.).</td></tr><tr><td>PubMed</td><td>n.a.</td><td>Leading search engine for bibliographic data from all fields of biology and medicine. PubMed is free and provides access to the MEDLINE bibliographic database, which contains citations and abstracts of biomedical research articles. Developed and hosted by the National Institutes of Health (NIH, USA).</td></tr><tr><td>REST</td><td>Representational state transfer</td><td>A software architectural style that defines a set of constraints for creating web services. Web services that conform to the REST architectural style (also called RESTful web services) allow for interoperability between computers on the Internet.</td></tr></table>
<table><tr><td>RGI (Référentiel Général de l'Interopérabilité)</td><td>General Interoperability Framework</td><td>A set of recommendations listing norms and standards that bolster interoperability within the French government's information systems.</td></tr><tr><td>GDPR</td><td>General Data Protection Regulation</td><td>EU regulation on the protection of natural persons with regard to the processing of personal data and on the free movement of such data.</td></tr><tr><td>RGS (Référentiel Général de Sécurité)</td><td>General security framework</td><td>Regulatory framework for establishing trust in electronic exchanges between the French government and citizens.</td></tr><tr><td>RStudio</td><td>n.a.</td><td>Free, open, and multi-platform development environment for R, a programming language used for data processing and statistical analysis.</td></tr><tr><td>HSS</td><td>Humanities and social sciences</td><td>A group of disciplines studying various aspects of human experience from both individual and collective perspectives.</td></tr><tr><td>SourceSup</td><td>n.a.</td><td>Software management platform for research and higher education institutions. It is managed by the RENATER public interest group and allows software with open source licences to be made public, and projects to be hosted without making them public.</td></tr><tr><td>TDM</td><td>Text and Data Mining</td><td>The process of obtaining information from material read by a computer. It works by copying large quantities of textual material, extracting the data, and combining it to identify patterns.</td></tr></table>
<table><tr><td>TLS</td><td>Transport Layer Security</td><td>A protocol that allows communications security over computer networks, particularly over the Internet.</td></tr><tr><td>UUID</td><td>Universally Unique IDentifier</td><td>A system that allows distributed systems to uniquely identify a piece of information without depending on central coordination.</td></tr><tr><td>VRE</td><td>Virtual Research Environment</td><td>An online system that helps researchers collaborate.</td></tr></table>