# Annotating scientific data using artificial intelligence

Essmay Touami, Giulia Di Gennaro, Matthieu Chavent, Pierre Poulain

► To cite this version:

Essmay Touami, Giulia Di Gennaro, Matthieu Chavent, Pierre Poulain. Annotating scientific data using artificial intelligence. SFCi 2025 - 12e journées de la Société Française de Chémoinformatique, Dec 2025, Paris, France., 2025. hal-05424145

HAL Id: hal-05424145

https://hal.science/hal-05424145v1

Submitted on 18 Dec 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
<figure><img src="image_2.png" /></figure>

<figure><img src="image_3.png" /></figure>

# Annotating scientific data using artificial intelligence

Essmay Touami<fnref n="1" />, Giulia Di Gennaro<fnref n="1" />, Matthieu Chavent<fnref n="2" />, Pierre Poulain<fnref n="1" />

## ABSTRACT

In the context of Open Science, large volumes of research data are increasingly accessible across scientific fields. Yet, their reuse is limited by missing or inconsistent metadata. Within the European project LUMEN, we develop an AI-based methodology using Large Language Models (LLMs) to automatically annotate molecular dynamics (MD) simulations, without any manually annotated training dataset.

Our approach extracts key metadata entities such as molecules (MOL), software informations (SOFTNAME, SOFTVERS), simulation times (STIME), temperatures (TEMP), and force fields and models (FFM), and applies a structured validation layer to enhance LLM output reliability. We benchmarked open-source Python libraries and state-of-the-art LLMs (GPT-5, Gemini, DeepSeek, etc.) to evaluate their performance, robustness, and resistance to hallucinations.

The method is domain-agnostic and aims to integrate validated metadata into MDverse<fnref n="1" />, a data catalogue for MD simulations, to improve dataset discoverability and reusability.

MDverse

shares data

redirects to

zenodo

<figure><img src="image_4.png" /></figure>

OSF

<figure><img src="image_5.png" /></figure>

figshare

<figure><img src="image_6.png" /></figure>

## CONCLUSIONS

* Enforcing strict prompt rules and output schemas improves reliability.

* Structured-output validation strongly reduces hallucinations, especially for smaller models.

* Instructor is the best-performing extractor framework.

* Gemini-3-Pro achieves the best entity-detection performance, but also has the highest computation time.

<figure><img src="image_7.png" /></figure>

* Quantify LLM performances vs. expert consensus.

* Compare LLMs with fine-tuned BERT³-like model on our dataset.

* Explore LLM agents for voting & judging.

## BACKGROUND

MDverse & Metadatas Challenge: MD dataset descriptions are freely written by authors without standardization, making it difficult to identify molecules, software, and simulation parameters. Named Entity Recognition (NER) is used to extract this information automatically. NER is a Natural Language Processing (NLP) technique used to identify and classify key elements in text into predefined categories<fnref n="2" />.

<figure><img src="image_8.png" /></figure>

### NER Approaches:

* Fine-tuning models (accurate, but needs a lot of annotated data)

* Prompt-engineering (highly adaptable and training-free, but produces non-deterministic results)

One prompt→many possible outputs

<figure><img src="image_9.png" /></figure>

Correct annotation: Alkaline dipeptide is the only molecule simulated for 87 µs at 300 K using the AMBER99SB-ILDN force field with GROMACS version 2021.4

Unbaked simulation of Alkaline dipeptide: MOL in gas phase, 87 microsecond, 198 ns long unbaked Molecular Dynamics (MD) trajectory of alkaline dipeptide: MOL in gas phase (sim_comp_id: 300 K, TEMP, Force Field: AMBER99SB-ILDN, FMM: There are 30+ back and forth transitions between the C_req and the C_tax state. Simulations were performed using GROMACS, SOFTWARE: 2021.4

Uncorrect annotation: Cine and dipeptide are wrongly listed; time and temperature are missing or incorrect. The force field, software, and main molecule are correct, but the GROMACS version is missing.

<figure><img src="image_10.png" /></figure>

<figure><img src="image_11.png" /></figure>

How can we guide LLMs to extract structured and validated metadatas automatically?

## PIPELINE

<figure><img src="image_12.png" /></figure>

Papers

Data descriptions

## RESULTS

We tested popular Extractor libraries (Instructor, LlamaIndex, PydanticAI) :
→Instructor performed best and was used for all further experiments

<figure><img src="image_13.png" /><figcaption>Evaluation of 50 MD simulation dataset descriptions manually annotated by experts</figcaption></figure>

* Gemini 3 Pro remains highly accurate with or without validation (~85% correct entities)

* Smaller models benefit the most (DeepSeek: 40%→73.5% correct entities with validation)

## REFERENCES

<figure><img src="image_14.png" /></figure>

<figure><img src="image_15.png" /></figure>

<figure><img src="image_16.png" /></figure>

<figure><img src="image_17.png" /></figure>

<footnote n="1">¹ Université Paris Cité, CNRS, Institut, de Biologie Physico-Chimique, Laboratoire de Biochimie Théorique, 75005, Paris, France</footnote>

<footnote n="2">² Université de Toulouse, CNRS, Centre de Biologie Intégrative, 31400, Toulouse, France</footnote>

<footnote>(1) Johanna KS Tiemann, et al. MDverse, shedding light on the dark matter of molecular dynamics simulations. eLife, 2024. DOI: 10.7554/eLife.90061.3</footnote>

<footnote>(2) What is named entity recognition? (2023, August 26). IBM. https://www.ibm.com/think/topics/named-entity-recognition</footnote>

<footnote>(3) Lee, Jinhyuk, et al. "BioBERT: a pre-trained biomedical language representation model for biomedical text mining." Bioinformatics 36.4 (2020): 1234-1240.</footnote>