Détection d’Hallucinations dans les Dossiers Médicaux Générés Automatiquement : Une Approche d’Optimisation pour les Couches Sémantiques et les Seuils Adaptatifs (2025)

Abstract: Les Modèles de Langage (LLM) sont susceptibles aux hallucinations, générant parfois des informations inexactes d’où un risque non négligeable, notamment dans le domaine médical où la fiabilité est essentielle. Cet article aborde deux objectifs : améliorer la qualité des dossiers médicaux et renforcer la fiabilité des cohortes de recherche. Nous présentons un système de détection des hallucinations dans les résumés médicaux générés par IA en optimisant les couches sémantiques de BERT. Notre méthodologie exploite BERT Score pour évaluer la similarité entre les phrases des rapports générés et des transcriptions originales. Notre contribution principale introduit un mécanisme à double seuil critique et alerte optimisé par l’algorithme Tree Parzen Estimator, contrairement aux approches traditionnelles à seuil unique. Les résultats démontrent des améliorations significatives dans la détection des hallucinations, avec une précision et un rappel supérieur aux méthodes de référence. Bien que notre étude soit limitée à la langue française, le système proposé assure améliore la fiabilité des informations médicales, répondant aux objectifs d’amélioration de la qualité documentaire et d’intégrité des données de recherche.

## introduction hallucination in large language models (llms) represents one of the major challenges in developing artificial intelligence technologies today. this phenomenon, where systems generate incorrect information presented as factual, raises fundamental questions about their reliability across various application domains. in the medical context, this issue takes on particular importance. medical summaries require absolute precision as they guide clinical decisions and patient monitoring. the integration of llms in this sector offers promising prospects for improving administrative efficiency and documentation, but the critical nature of healthcare amplifies the risks associated with hallucinations. inaccurate information in a medical report can lead to serious consequences, from inappropriate diagnoses to inadequate treatments @xcite . our work focuses on two primary objectives. first, we aim to improve the quality of medical records by developing a novel hallucination detection system that is used for the french language, which has fewer resources than english. this system leverages bertscore, a metric that evaluates the semantic similarity between generated and reference texts using the contextual embeddings from bert models. by optimizing the semantic layers of bert models specifically for hallucination detection, we seek to enhance the accuracy of generated medical text and minimize the impact of hallucinations, ultimately ensuring the reliability of ai-generated content in critical healthcare settings. second, we seek to enhance the reliability of automatically generated medical notes from consultation transcriptions by implementing a dual-threshold mechanism, critical and alert, that is algorithmically optimized rather than relying on traditional single empirically fixed thresholds. by leveraging bertscore @xcite to evaluate semantic similarity between pairs of sentences from automated reports and original transcriptions, our approach offers a more nuanced and effective solution to the hallucination problem in medical text summarization. this system tackles the specific challenges posed by specialized medical terminology, intricate causal relationships, and varied care pathways, all of which require robust verification mechanisms for tasks such as generating medical documents, summaries, or reports using llms. ## related works the development of artificial intelligence technologies has transformed natural language processing (nlp), particularly through the emergence of transformer-based architectures @xcite . models such as bert @xcite , gpt-2 @xcite gpt-3 @xcite and @xcite have revolutionized nlp across various domains, including the medical sector. these advances have enabled the automation of medical summary creation, significantly facilitating clinical documentation and administrative tasks in healthcare facilities. alongside the advancement of llm models, attention toward their limitations and potential risks has also increased. one of the most significant challenges with llms is the phenomenon of hallucination, where models generate content that is unfaithful to the source information or factually incorrect @xcite . @xcite demonstrated that llm can be prompted to extract and generate private information from their training data, such as email addresses and phone numbers. this memorization behavior qualifies as hallucination since the model produces content unfaithful to the source input, generating private details absent from the immediate context, which also raises significant privacy concerns. in the medical field, hallucinations present many risks that extend beyond privacy concerns @xcite . advanced models like llama 3 @xcite and gpt-4o have demonstrated impressive capabilities in generating meaningful medical content and passing medical examinations @xcite ), yet they remain susceptible to critical reliability issues. researchers have identified two distinct categories of hallucinations affecting medical applications @xcite factual hallucinations, where generated information contradicts verifiable medical knowledge, and faithfulness hallucinations, where content deviates from the specific patient context provided. this distinction is especially critical given the highly contextualized and personalized nature of medical records. to overcome these key challenges, researchers have developed various approaches for hallucination detection in llm-generated medical content. these methods can be broadly categorized into reference-dependent and reference-free approaches. reference-dependent metrics compare model outputs against verified knowledge sources, with specialized benchmarks like med-halt @xcite specifically addressing hallucinations in medical contexts. these approaches evaluate the fidelity of generated content by measuring discrepancies against established medical knowledge databases or source documents. reference-free approaches offer alternative detection methods that don't require external reference materials. uncertainty-based methods, as proposed by @xcite , analyze token probabilities, operating under the assumption that lowconfidence predictions correlate with hallucinated content. more applicable to black-box scenarios, consistency-based detection methods generate multiple responses to the same prompt and measure their agreement. these techniques employ various similarity metrics, including bleu-based variation ratio @xcite , n-gram approximation @xcite , and bertscore @xcite . bertscore has proven particularly valuable for medical applications by leveraging contextual embeddings to assess semantic similarity between texts rather than relying on lexical matching-an important distinction in medical contexts where terminology variations are common but semantic integrity remains essential. among the bert-based models specialized for medical applications, several french language variants have been developed. clinicalbert @xcite adaptations for french address challenges like gender agreement and terminology variations specific to the french healthcare system. camembert-bio (touchent et al., 2023) extends the original camembert (antoun et al., 2024) with french biomedical texts, improving performance on medical entity recognition in french clinical documentation. additionally, drbert was designed for french clinical applications, trained on medical reports from french hospitals @xcite , while flaubert @xcite has been fine-tuned for healthcare applications. the original camembert has also been applied to medical tasks through domain adaptation. bart-base-french leverages a denoising autoencoder architecture and is fine-tuned on domain-specific medical data to generate accurate french medical summaries while reducing hallucinations @xcite . these models are valuable for french-speaking healthcare systems, where english-based models often fail to capture linguistic nuances and specialized french medical vocabulary. while these bert-based models show promise for french medical nlp, research on the optimization of bert layers has shown important findings. @xcite found that using the middle layers of bert rather than the final layer produces better correlation with human judgments when evaluating text similarity. this underscores the need for careful optimization of both the bert layers and consequently, the threshold settings to enhance performance in detection tasks. however, despite the recognized importance of threshold selection in classification and detection systems, research on optimizing thresholds specifically for hallucination detection remains limited. this gap is particularly significant in medical contexts, where the consequences of false positives and false negatives vary greatly in severity. identifying appropriate thresholds represents a crucial task that has received insufficient attention in the current literature on medical hallucination detection. moreover, recent advances such as selfcheckgpt @xcite have introduced zero-resource, black-box hallucination detection methods that leverage the generative model's outputs without requiring external references or access to training data. these approaches demonstrate promising directions toward scalable, model-agnostic detection frameworks that can complement threshold-based optimization and improve reliability in clinical nlp applications. these developments underscore the importance of combining robust hallucination detection techniques with careful calibration of detection thresholds to ensure reliability, especially in sensitive medical contexts. building on this foundation, our work focuses on optimizing threshold settings for hallucination detection in french clinical nlp models, leveraging both reference-free detection methods and layer-wise analysis of bert representations to enhance performance and trustworthiness. ## proposed methodology in this section, we present a novel approach to hallucination detection in llm through targeted optimization of semantic representations and detection thresholds, which is illustrated in figure 1 . current literature indicates that the most effective semantic representations in transformer-based models are not necessarily stored in the final layer, with multiple studies demonstrating that intermediate layers often contain more relevant information for specific semantic tasks. building on this insight, our research implements a two-phase optimization process. phase 1 focuses on optimizing the model layers by evaluating multiple bert variants across all their respective layers, using semantic similarity corpora to determine which specific combination yields the most effective vector representations. ## comparative analysis with state-of-the-art approaches we benchmarked our optimized bioclinical-bert method against established baselines on a comprehensive dataset of authentic clinical consultations and reports. the evaluation dataset consists of real transcription-report pairs from medical practice, with all techniques assessed using identical ground truth annotations to ensure a fair comparison. in particular, we tested two variants of selfcheckgpt : selfcheckgpt-roberta and selfcheckgpt-gpt2, which represent recent stateof-the-art approaches for zero-resource, black-box hallucination detection. ## system limitations while the dual-threshold optimization system we developed for hallucination detection shows promising results, several limitations warrant further investigation. our current approach relies on pairwise sentence-level comparisons between the generated medical report and the original transcription. however, this method may struggle to accurately handle cases where a single sentence in the report effectively condenses or synthesizes information from multiple sentences in the transcription. such summarization is common in clinical practice, as healthcare professionals frequently integrate diverse pieces of information into concise, coherent conclusions. this inherent complexity poses challenges for the alignment and evaluation process, potentially leading to under-detection or misclassification of hallucinations in synthesized content. ## conclusion and perspectives this study presents a novel dual-threshold approach for detecting hallucinations in medical documentation generated by large language models. by optimizing the semantic layers of bert and implementing critical and alert thresholds through the tree parzen estimator, our system achieves superior performance in hallucination detection compared to traditional methods and state-of-the-art approaches such as selfcheckgpt variants. in clinical settings, this technology can enhance patient safety by ensuring accurate medical summaries. future work should explore applications across diverse medical specialties and examine effectiveness in multiple languages beyond french. testing our system on authentic medical consultations and records will provide more robust validation of its practical utility in clinical environments. additionally, integrating explainable ai techniques could further enhance trust and adoption by healthcare professionals. the development of such systems represents an important step toward responsible ai deployment in healthcare, where information accuracy directly impacts patient outcomes and treatment decisions.