# Improving Social Determinants of Health Documentation in French EHRs Using Large Language Models

Adrien Bazoge, Pacôme Constant Dit Beaufils, Mohammed Hmitouch, Romain Bourcier, Emmanuel Morin, Richard Dufour, Béatrice Daille, Pierre-Antoine Gourraud, Matilde Karakachoff

► To cite this version:

Adrien Bazoge, Pacôme Constant Dit Beaufils, Mohammed Hmitouch, Romain Bourcier, Emmanuel Morin, et al. Improving Social Determinants of Health Documentation in French EHRs Using Large Language Models. Scientific Reports, 2025, 15 (45427), 10.1038/s41598-025-29987-z. http://dx.doi.org/10.1038/s41598-025-29987-z

HAL Id: hal-05468221

https://hal.science/hal-05468221v1

Submitted on 20 Jan 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC0 1.0 - Universal - International License
# Improving Social Determinants of Health Documentation in French EHRs Using Large Language Models

Adrien Bazoge<fnref n="1" />,<fnref n="2" />

Pacôme Constant dit Beaufils<fnref n="1" />,<fnref n="3" />,<fnref n="4" />

Mohammed Hmitouch<fnref n="1" />,<fnref n="2" />

Romain Bourcier<fnref n="3" />

Emmanuel Morin<fnref n="2" />

Richard Dufour<fnref n="2" />

Béatrice Daille<fnref n="2" />

Pierre-Antoine Gourraud<fnref n="1" />

Matilde Karakachoff<fnref n="1" />

## Abstract

Social determinants of health (SDOH) significantly influence health outcomes, shaping disease progression, treatment adherence, and health disparities. However, their documentation in structured electronic health records (EHRs) is often incomplete or missing. This study presents an approach based on large language models (LLMs) for extracting 13 SDoH categories from French clinical notes. We trained Flan-T5-Large on annotated social history sections from clinical notes at Nantes University Hospital, France. We evaluated the model at two levels: (i) identification of SDoH categories and associated values, and (ii) extraction of detailed SDoH with associated temporal and quantitative information. The model performance was assessed across four datasets, including two that we publicly release as open resources. The model achieved strong performance for identifying well-documented categories such as living condition, marital status, descendants, job, tobacco, and alcohol use (F1 score > 0.80). Performance was lower for categories with limited training data or highly variable expressions, such as employment status, housing, physical activity, income, and education. Our model identified 95.8% of patients with at least one SDoH, compared to 2.8% for ICD-10 codes from structured EHR data. Our error analysis showed that performance limitations were linked to annotation inconsistencies, reliance on English-centric tokenizer, and reduced generalizability due to the model being trained on social history sections only. These results demonstrate the effectiveness of NLP in improving the completeness of real-world SDoH data in a non-English EHR system.

## 1 Introduction

Despite a continuous increase in overall life expectancy, social inequalities in health persist and are widening throughout the life course [1, 2]. Medical progress can prolong the life expectancy of individuals with severe diseases, but care management cannot be disconnected from the socio-economic environments where patients live [3]. The outcomes of chronic diseases are shaped by a combination of behaviors and exposures, resulting in a complex socio-biological process that determines both individual and societal health status [4]. Social determinants of health (SDOH) are the conditions in which people grow, live, work, and age that influence their quality of life and health [4]. These determinants encompass a broad range of socioeconomic factors, including family support, employment status, and education, as well as health-related behaviors, such as substance use and physical activity [5]. In addition to shaping behaviors, socioeconomic conditions also determine

<footnote n="1">¹Data Clinic, University Hospital of Nantes, France</footnote>

<footnote n="2">²Nantes Université, École Centrale Nantes, CNRS, LS2N, France</footnote>

<footnote n="3">³Department of Neuroradiology, University Hospital of Nantes, Thorax Institute, France</footnote>

<footnote n="4">⁴Nantes Université, CNRS, INSERM, Thorax Institute, France</footnote>
individuals' exposure to environmental risks, such as air pollution, noise, and extreme weather events, which further impact health outcomes [6]. Together, social and behavioral factors are major drivers of health disparities, contributing to 47% and 34% of patient outcomes, respectively [7]. In clinical settings, SDoH are documented in the electronic health records (EHR) through both structured data (e.g., coded fields) and unstructured data (e.g., clinical notes) [8]. However, unstructured clinical notes provide more intricate and detailed representations of SDoH than structured data [9]. To enable large-scale secondary use of EHR data, there is an increasing need for automated methods to extract and structure patient SDoH, enhancing our ability to study the impact of social inequalities on health [10]. In this context, studying SDoH in clinical sciences is essential for understanding how factors like income, education, housing, and social environments shape health outcomes beyond technico-biological variables. Integrating insights from SDoH into clinical practice highlights opportunities for early intervention, holistic patient care, and addressing health inequities at their root. Furthermore, recognizing the actionable nature of SDoH enables the development of evidence-based public health policies that target structural barriers and improve population health at scale [11].

In recent years, automatic extraction of SDoH has been widely studied in the English language using natural language processing (NLP) [12]. Progress has been accelerated by the dissemination of annotated corpora and shared task challenges, for example the i2b2 NLP Smoking Challenge on identifying patients' smoking status with a corpus of 502 clinical notes [13] and the 2022 n2c2 exploring the extraction of SDoH from 4,405 social history sections from clinical notes, including substance use (alcohol, drugs, and tobacco), employment and living conditions [14]. Most subsequent studies have focused on U.S. hospital settings, showing that NLP methods applied to unstructured data from EHRs can reliably identify key social risk factors [15, 16, 17, 18, 19]. In terms of coverage, the most commonly addressed SDoH are smoking status, substance abuse (alcohol and drug) and housing instability [12], which are well known for their impact on health, and thus are well documented in EHRs [7]. In contrast, other SDoH, such as education, employment status, social support and isolation, remain underexplored and present ongoing challenges for NLP systems [12]. To address SDoH identification and extraction, a range of NLP methods have been developed. Early approaches relied on rule-based systems and keyword matching, offering high precision but limited generalizability [20]. Subsequently, semantic word embedding methods such as word2vec [21] enabled more nuanced lexical representations, supporting downstream machine learning classifiers for SDoH identification [22, 23, 24, 25, 26]. More advanced approaches have leveraged deep learning architectures, CNNs [27], LSTMs [27], and transformer-based models such as BERT [27, 28, 29, 30], which have demonstrated improved performance in extracting contextually rich SDoH information.

Recent studies have also shown the potential of large language models (LLMs) for the identification and classification of SDoH in clinical notes. Decoder-only models such as GPT-4 [31] have been used in zero- and few-shot settings [32], while encoder-decoder models such as Flan-T5 [33] have been applied for generative SDoH extraction tasks, and are the current state-of-the-art approaches for SDoH extraction [14, 19, 30, 32, 34, 35]. However, most of available work are focused on English language, while resources for other languages remain scarce [12, 36]. The extraction of smoking status from clinical narrative texts has been studied in Spanish [37], Finnish [38], Swedish [39], and in a Korean-English bilingual setting [40], but none of the underlying corpora are available. To our knowledge, the extraction of SDoH in French clinical texts has not been addressed.

In this work, we propose a sequence-to-sequence approach based on a large language model for extracting SDoH from French clinical texts. Our study focuses on 13 SDoH categories: living condition, marital status, descendants, employment status, occupation, tobacco use, alcohol use, drug use, housing, education, physical activity, income, and ethnicity/country of birth. To support model development and evaluation, we constructed and manually annotated four datasets consisting of social history sections from clinical notes. Two of these datasets are publicly released to promote reproducibility and support the development of new methods for SDoH extraction in French.
# 2 Methods

## 2.1 Data

To train and evaluate the proposed SDoH model, we used four datasets: MUSCADET-InHouse, MUSCADET-Synthetic, UW-FrenchSDOH and InHouse Tuberculosis/ALS.

**MUSCADET-InHouse** This dataset was obtained from clinical notes from the Nantes biomedical data warehouse (NBDW), as summarized in Figure 1. The NBDW encompasses nearly 1.5 million patients who received care at the Nantes University Hospital, over the past 20 years. It includes different dimensions of patient-related data: structured data (e.g. *Classification Commune des Actes Médicaux* billing codes, a French coding system of clinical procedures; ICD-10 codes; laboratory results; drug administrations), and unstructured data such as outpatient and inpatient clinical notes, radiology and operative reports [41]. The NBDW was authorized by the French authority of data protection (*Commission Nationale de l'Informatique et des Libertés*) (Registration code n°920242). The present study is compliant with French regulatory and General Data Protection Regulation requirements, including informed consent. A total of 1,144,443 clinical notes were selected from the NBDW according to the following inclusion criteria: age ≥ 18 years, the presence of clinical notes within the NBDW between August 1, 2018, and June 1, 2022. The non-inclusion criterion was patient opposition to data reuse. We then focused on semi-structured clinical notes containing predefined sections (e.g., 'History', 'Medications', 'Social History', etc.), with a particular emphasis on two categories: consultation reports and hospital stay reports. These two types of notes span multiple medical specialties, resulting in a total of 206,973 clinical notes covering diverse patient profiles. The clinical notes selected in the previous step were filtered to retain only those containing a social history section, for a total of 32,666 clinical notes. The social history section was extracted using a rule-based approach. Finally, 1,700 social history sections were randomly selected to constitute our corpus for annotation. This dataset was randomly divided into training (70%), validation (10%) and test (20%) sets for our experiments.

<figure><img src="image_2.png" /><figcaption>Figure 1: MUSCADET-InHouse corpus construction flow-chart.</figcaption></figure>

To assess the generalization capabilities of our model, we constructed three external test datasets. For reproducibility, we introduce two open-source datasets: MUSCADET-Synthetic and UW-FrenchSDOH.
**MUSCADET-Synthetic** This dataset comprises synthetic social history section texts written by a physician. These synthetic documents follow the template of real medical records but were entirely written from scratch, ensuring that they do not reference any real patient. The corpus includes 340 documents, matching the test set size of MUSCADET-InHouse.

**UW-FrenchSDOH** This dataset is an automatically translated version of an existing dataset from the University of Washington (UW) [42, 43]. It consists of 364 social history sections collected from MTSamples. The dataset was translated into French using GPT-4o (gpt-4o-2024-11-20) and manually corrected during annotation.

**InHouse Tuberculosis and ALS** Since other datasets focus only on social history sections, it was essential to assess the model's effectiveness in a broader clinical context, particularly on non-SDoH texts, to determine its propensity for false positives. To this end, we applied the model to full clinical notes in two use cases, focusing on patients hospitalized for tuberculosis or amyotrophic lateral sclerosis (ALS). These diseases were selected due to the significant impact of SDoH on their outcomes [44, 45, 46]. Both groups of patients and related clinical notes were selected on ICD-10 criteria: A15-A19 for tuberculosis and G12.2 for ALS. A total of 1,186 patients were identified for tuberculosis and 647 for ALS. We included the first clinical note recorded for each patient visit associated with the respective ICD-10 code as the principal diagnosis. For each disease, 200 clinical notes were fully annotated to serve as a test set, for a total of 400 clinical notes.

## 2.2 Annotation Scheme

The annotation scheme was designed to provide a broad coverage of SDoH, with a fine-grained description of the determinants. The annotation scheme includes entities, attributes, and relations between entities.

The annotation scheme comprises 25 entities related to SDoH, covering 13 SDoH categories (living condition, marital status, descendants, employment status, job, tobacco use, alcohol use, drug use, housing, education, physical activity, income and ethnicity/country of birth) and 6 entities related to relations (StatusTime, History, Duration, Amount, Frequency, Type). SDoH entities are either span-only (Job, Income, Education, Ethnicity, Alcohol, Tobacco, Drug) or labeled (Living, MaritalStatus, Descendants, Employment, Housing, PhysicalActivity), while all relation entities are span-only. Table 1 presents all entities.

<table><caption>Table 1: Entities description.</caption><thead><tr><th>Entity</th><th>Definition</th><th>Examples</th><th>Examples translated into English</th></tr></thead><tbody><tr><td>Living_Along</td><td>The patient is described as living alone.</td><td>Vit seul</td><td>Living alone</td></tr><tr><td>Living_WithOthers</td><td>The patient is described as living with other people. This doesn't include animals but includes nursing home.</td><td>Vit avec sa soeur; Vit avec ses parents; Vit avec son épouse et ses enfants ; Vit en EHPAD</td><td>Lives with his/her sister; Lives with his/her parents; Lives with his wife and children; Lives in nursing home</td></tr><tr><td>MaritalStatus_Single</td><td>The patient described as being single (i.e. no regular partner).</td><td>Célibataire sans enfant; Pas de conjoint</td><td>Single without children; No partner</td></tr><tr><td>MaritalStatus_InRelationship</td><td>The patient is described as married, registered partnership or with a regular partner.</td><td>Mariée; Vit en concubinage; Vit avec sa compagne; Pacsé</td><td>Married ; Lives with partner; Lives with his girlfriend; Civil union</td></tr><tr><td>MaritalStatus_Divorced</td><td>The patient is described as divorced or separated from a regular partner (except from non-regular partnership).</td><td>Divorcé; Plus de contact avec le père de ses enfants</td><td>Divorced; No more contact with the father of her children</td></tr><tr><td>MaritalStatus_Widowed</td><td>The patient is described as widowed.</td><td>Veuf; Veuve</td><td>Widower, Widow</td></tr><tr><td>Descendants_Yes</td><td>The patient has descendants.</td><td>Deux filles; 1 fils adopté; Trois petits-enfants; Courses faites par sa fille</td><td>Two daughters; 1 adopted son; Three grandchildren; Shopping done by daughter</td></tr><tr><td>Descendants_No</td><td>The patient has no descendants.</td><td>Pas d'enfant; Sans enfant; Enceinte de son premier enfant</td><td>No children; Childless; Pregnant with first child</td></tr></tbody><tfoot><tr><td colspan="4">Continued on next page</td></tr></tfoot></table>
<table><thead><tr><th>Entity</th><th>Definition</th><th>Examples</th><th>Examples translated into English</th></tr></thead><tbody><tr><td>Job</td><td>All the patient's jobs, previous and actual.</td><td>Courtier dans le textile ;<br/>Travaille dans le bâtiment;<br/>Soudeur</td><td>Textile broker; Construction<br/>worker; Welder</td></tr><tr><td>Last_job</td><td>The patient's latest job.</td><td></td><td></td></tr><tr><td>Employment Working</td><td>Any documentation that the patient is currently working. It includes short/finite period of time off from work as sick leave, sab- batical, maternity leave or only information about job.</td><td>Travaille toujours; En reconver-<br/>sion professionnelle ; Agent de<br/>ménage ; Maître d'œuvre</td><td>Still working; Retraining;<br/>Housekeeper; Prime contractor</td></tr><tr><td>Employment Unemployed</td><td>The patient is described as un- employed.</td><td>Vient de terminer son CDD ; Il est au chômage</td><td>Has just finished his fixed-term<br/>contract; He is unemployed</td></tr><tr><td>Employment Student</td><td>The patient is described as a stu- dent.</td><td>Actuellement lycéen; ... en pre-<br/>mière année de BTS ; Patient étu-<br/>diant</td><td>Currently in high school; ... in<br/>first year of associated degree ;<br/>Student patient</td></tr><tr><td>Employment Pensioner</td><td>The patient is described as pen- sioner. It also includes mentions such as "former" + job refer- ences, without explicit mention of pensioner.</td><td>Retraité; A la retraite; Ancien<br/>magasinier</td><td>Retired; Retired; Former ware-<br/>houseman</td></tr><tr><td>Employment Other</td><td>The patient is not described with previous employment labels nor job. This entity describes other employment situations: unclear work status without temporality, disability, long sick leave, irreg- ular situation, housewife, volun- teer work, etc.).</td><td>Il a été courtier dans le textile ;<br/>Elle a travaillé comme dentiste ;<br/>Travailleur handicapé et ne tra-<br/>vaille plus depuis 2011 ; En in-<br/>validité ; Mère au foyer</td><td>He was a textile broker; She worked as a dentist; Disabled worker and hasn't worked since 2011; On disability; Housewife</td></tr><tr><td>Alcohol</td><td>The alcohol use is annotated as an event. Only the trigger an- chor is annotated here. The sta- tus of alcohol use is annotated through a relationship with the StatusTime entity.</td><td>Alcool; Ethylisme; CAD; OH;<br/>Intoxication ethylique</td><td>Alcohol; Ethylism; CAD; OH;<br/>Ethyl intoxication</td></tr><tr><td>Tobacco</td><td>The tobacco use is annotated as an event. Only the trigger an- chor is annotated here. The sta- tus of tobacco use is annotated through a relationship with the StatusTime entity.</td><td>Tabac; Tabagisme ; Consomma-<br/>tion tabagique</td><td>Tobacco; Smoking; Tobacco<br/>consumption</td></tr><tr><td>Drug</td><td>The drug use is annotated as an event. Only the trigger anchor is annotated here. The status of drug use is annotated through a relationship with the StatusTime entity.</td><td>Héroïne en fumette; Cannabis;<br/>drogue</td><td>Heroin smoking; Cannabis;<br/>Drugs</td></tr><tr><td>Housing_Yes</td><td>The patient is described as hav- ing a housing stability.</td><td>Habite à Nantes; Pas d'aides à<br/>domicile ; Vit dans une maison</td><td>Lives in Nantes; No home help;<br/>Lives in a house</td></tr><tr><td>Housing_No</td><td>The patient is described as hav- ing a housing instability.</td><td>Sans domicile fixe; Hébergé<br/>chez des tiers</td><td>Homeless; Living with a third<br/>party</td></tr><tr><td>PhysicalActivity_Yes</td><td>The patient is described as exer- cising regularly.</td><td>Fait du vélo; Marche 30min par<br/>jour</td><td>Cycling; Walking 30min a day</td></tr><tr><td>PhysicalActivity_No</td><td>The patient is described as not exercising regularly.</td><td>Marche avec une canne, sort peu<br/>; Pas d'activité sportive; Séden-<br/>taire ; Performance status 3</td><td>Walks with a cane, goes out lit-<br/>tle; No sports activity; Seden-<br/>tary; Performance status 3</td></tr><tr><td>Income</td><td>Any documentation about pa- tient's financial resources. It in- cludes social aids.</td><td>RSA; AAH</td><td>Active solidarity income; Dis-<br/>abled adult allowance</td></tr><tr><td>Education</td><td>Any documentation about pa- tient's education.</td><td>Certificat d'étude; BTS Informa-<br/>tique; Licence en droit</td><td>Certificate of Study; Associate<br/>degree in Computer Science;<br/>Bachelor's Degree in Law</td></tr><tr><td>Ethnicity/Country of birth</td><td>The patient's country of birth.</td><td>Originaire du Maroc; Origi-<br/>naire d'Érythrée</td><td>From Morocco ; From Eritrea</td></tr></tbody><tfoot><tr><td colspan="4">Continued on next page</td></tr></tfoot></table>
<table><thead><tr><th>Entity</th><th>Definition</th><th>Examples</th><th>Examples<br>English</th><th>translated</th><th>into</th></tr></thead><tbody><tr><td>StatusTime</td><td>The status of substance consumption (alcohol, tobacco and drug). An attribute is associated to this entity with possible values: current, past, none.</td><td>Ne consomme pas d'alcool</td><td>No alcohol</td><td></td><td></td></tr><tr><td>History</td><td>Any date of occurrence of an event.</td><td>Il y a 8 ans; Il a arrêté depuis 3 ans</td><td>8 years ago; He stopped 3 years ago</td><td></td><td></td></tr><tr><td>Duration</td><td>Any exposure duration (to substance use, house instability, etc.).</td><td>Il a fumé pendant 20 ans.</td><td>He smoked for 20 years.</td><td></td><td></td></tr><tr><td>Amount</td><td>Any amount related to SDoH, such as Descendants or substance use.</td><td>20 paquets de cigarettes; 3 verres de vin; 2 enfants</td><td>20 packs of cigarettes; 3 glasses of wine; 2 children;</td><td></td><td></td></tr><tr><td>Frequency</td><td>Any frequency related to SDoH, such as physical activity or substance use.</td><td>Par jour; par mois; occasionnellement</td><td>Daily; monthly; occasionally</td><td></td><td></td></tr><tr><td>Type</td><td>Specify certain broad entities to get fine-grained details, such as the type of descendants, the type of drugs or alcohol used.</td><td>Enfants; petits-enfants; vin; bière; cigarette; cocaine; cannabis</td><td>Children; grandchildren; wine; beer; cigarettes; cocaine; cannabis</td><td></td><td></td></tr></tbody></table>

The annotation scheme also includes six relations (Table 2):

* Status: encodes the status (current, past, or none) of the substance use (alcohol, tobacco, or drugs).

* History: links any event to its date of occurrence

* Duration: encodes the duration of exposure to substance use

* Amount: encodes the quantity of substance use or the number of children in a lineage. Units of measurement: number of glasses, number of cigarettes, number of children, grams, etc.

* Frequency: gives the frequency of an event's occurrence. This relation is also used when the amount related to substance use is not precise enough. For example: drinks occasionally

* Type: details certain entities, such as the type of lineage or the type of substance use.

Following the work on SHAC corpus [47], we annotated the substance use (tobacco, alcohol, and drugs) using an event-based scheme characterized by a trigger entity and status-related attributes. We used the BRAT Rapid Annotation Tool (BRAT) for datasets annotation [48]. For the MUSCADET-InHouse dataset, the annotation process was carried out in three phases, with inter-annotator agreement calculated at the end of the first two phases: (1) a preliminary annotation phase on 100 documents by three annotators (PCDB, a physician; AB, an NLP researcher; and MK, an epidemiologist) to evaluate the annotation scheme and refine the guidelines; (2) a second annotation phase on 200 documents by two annotators (PCDB, AB) to validate the modifications made to the guidelines following the first phase; and (3) a final annotation phase during which each annotator worked independently according to the finalized annotation guidelines.

For the MUSCADET-Synthetic dataset, all texts were annotated by three annotators (PCDB, AB, MK). The UW-FrenchSDOH dataset was annotated by a single annotator (AB), while the InHouse Tuberculosis and ALS dataset was annotated by two annotators (PCDB, AB). We computed inter-annotator agreement (IAA) values for entities using F-measure from the open-source tool bratiaa<fnref n="1" /> for each annotator pair. For relation annotations, F-measure scores were performed using an in-house script. The BRAT configuration files and the scripts for IAA computation are available in the project's repository<fnref n="2" />.

## 2.3 Annotation Statistics

Table 3 presents the distribution of entity types across all datasets. The most frequent entities are Living_WithOthers, MaritalStatus_InRelationship, Descendants_Yes, Job, Tobacco, Alcohol and

<footnote n="1">¹https://github.com/kldtz/bratiaa</footnote>

<footnote n="2">²https://github.com/CliniqueDesDonnees/SDoH</footnote>
<table><caption>Table 2: Relations. Involved entities in bold indicates the relations is required when the SDoH entity is annotated. * indicates that any entity can be linked.</caption><thead><tr><th>Relation type</th><th>Involved entities</th></tr></thead><tbody><tr><td>Status</td><td>Tobacco - StatusTime<br>Alcohol - StatusTime<br>Drug - StatusTime</td></tr><tr><td>Amount</td><td>Descendants_Yes - Amount<br>Tobacco - Amount<br>Alcohol - Amount<br>Drug - Amount</td></tr><tr><td>Duration</td><td>Tobacco - Amount<br>Alcohol - Amount<br>Drug - Amount</td></tr><tr><td>Frequency</td><td>Physical_Activity - Frequency<br>Tobacco - Frequency<br>Alcohol - Frequency<br>Drug - Frequency</td></tr><tr><td>History</td><td>* - History</td></tr><tr><td>Type</td><td>Descendants_Yes - Type<br>Tobacco - Type<br>Alcohol - Type<br>Drug - Type</td></tr></tbody></table>

Housing_Yes, while the remaining entities appear less frequently. Similarly, Table 4 presents the distribution of relation types across all datasets. The most common relations are Status, Amount, and Type, the latter two being highly associated with the entity Descendants_Yes.

<table><caption>Table 3: Entities distribution in all datasets. *n* corresponds to the number of documents.</caption><thead><tr><th rowspan="3">Entities</th><th colspan="3">MUSCADET-InHouse</th><th>MUSCADET-Synthetic</th><th>UW-FrenchSDOH</th><th>InHouse Tuberculosis and ALS</th></tr><tr><th colspan="3">(n = 1,700)</th><th colspan="2">(n = 340)</th><th colspan="2">(n = 364)</th><th colspan="2">(n = 400)</th></tr><tr><th>Train</th><th>Dev</th><th>Test</th><th>Test</th><th>Test</th><th>Test</th><th>Test</th><th>Test</th><th>Test</th></tr></thead><tbody><tr><td>Living_Alone</td><td>194</td><td>32</td><td>55</td><td>37</td><td>17</td><td>17</td><td>17</td><td>10</td><td>10</td></tr><tr><td>Living_WithOthers</td><td>412</td><td>61</td><td>128</td><td>89</td><td>73</td><td>73</td><td>73</td><td>44</td><td>44</td></tr><tr><td>MaritalStatus_Single</td><td>50</td><td>9</td><td>14</td><td>20</td><td>20</td><td>20</td><td>20</td><td>6</td><td>6</td></tr><tr><td>MaritalStatus_InRelationship</td><td>629</td><td>72</td><td>184</td><td>132</td><td>127</td><td>127</td><td>127</td><td>72</td><td>72</td></tr><tr><td>MaritalStatus_Divorced</td><td>70</td><td>13</td><td>14</td><td>17</td><td>18</td><td>18</td><td>18</td><td>8</td><td>8</td></tr><tr><td>MaritalStatus_Widowed</td><td>69</td><td>8</td><td>17</td><td>18</td><td>9</td><td>9</td><td>9</td><td>3</td><td>3</td></tr><tr><td>Descendants_Yes</td><td>845</td><td>101</td><td>226</td><td>154</td><td>80</td><td>80</td><td>80</td><td>64</td><td>64</td></tr><tr><td>Descendants_No</td><td>98</td><td>13</td><td>25</td><td>33</td><td>5</td><td>5</td><td>5</td><td>6</td><td>6</td></tr><tr><td>Job</td><td>828</td><td>109</td><td>247</td><td>216</td><td>95</td><td>95</td><td>95</td><td>58</td><td>58</td></tr><tr><td>Last_job</td><td>751</td><td>100</td><td>217</td><td>204</td><td>85</td><td>85</td><td>85</td><td>49</td><td>49</td></tr><tr><td>Employment Working</td><td>348</td><td>53</td><td>92</td><td>114</td><td>56</td><td>56</td><td>56</td><td>15</td><td>15</td></tr><tr><td>Employment_Unemployed</td><td>113</td><td>18</td><td>31</td><td>17</td><td>9</td><td>9</td><td>9</td><td>7</td><td>7</td></tr><tr><td>Employment_Student</td><td>30</td><td>4</td><td>14</td><td>17</td><td>8</td><td>8</td><td>8</td><td>0</td><td>0</td></tr><tr><td>Employment_Pensioner</td><td>305</td><td>45</td><td>91</td><td>48</td><td>34</td><td>34</td><td>34</td><td>23</td><td>23</td></tr><tr><td>Employment_Other</td><td>82</td><td>3</td><td>32</td><td>21</td><td>16</td><td>16</td><td>16</td><td>8</td><td>8</td></tr><tr><td>Alcohol</td><td>498</td><td>70</td><td>127</td><td>193</td><td>251</td><td>251</td><td>251</td><td>42</td><td>42</td></tr><tr><td>Tobacco</td><td>627</td><td>94</td><td>164</td><td>219</td><td>263</td><td>263</td><td>263</td><td>60</td><td>60</td></tr><tr><td>Drug</td><td>78</td><td>11</td><td>20</td><td>101</td><td>136</td><td>136</td><td>136</td><td>17</td><td>17</td></tr><tr><td>Housing_Yes</td><td>682</td><td>86</td><td>203</td><td>122</td><td>48</td><td>48</td><td>48</td><td>137</td><td>137</td></tr><tr><td>Housing_No</td><td>16</td><td>3</td><td>3</td><td>13</td><td>0</td><td>0</td><td>0</td><td>2</td><td>2</td></tr><tr><td>PhysicalActivity_Yes</td><td>154</td><td>23</td><td>33</td><td>63</td><td>21</td><td>21</td><td>21</td><td>4</td><td>4</td></tr><tr><td>PhysicalActivity_No</td><td>37</td><td>6</td><td>11</td><td>14</td><td>3</td><td>3</td><td>3</td><td>0</td><td>0</td></tr><tr><td>Income</td><td>24</td><td>3</td><td>13</td><td>8</td><td>0</td><td>0</td><td>0</td><td>2</td><td>2</td></tr><tr><td>Education</td><td>58</td><td>5</td><td>18</td><td>20</td><td>3</td><td>3</td><td>3</td><td>1</td><td>1</td></tr><tr><td>Ethnicity</td><td>69</td><td>13</td><td>21</td><td>18</td><td>1</td><td>1</td><td>1</td><td>27</td><td>27</td></tr></tbody></table>
<table><caption>Table 4: Relations distribution in all datasets. *n* corresponds to the number of documents.</caption><thead><tr><th rowspan="3">Relations</th><th colspan="3">MUSCADET-InHouse</th><th>MUSCADET-Synthetic</th><th>UW-FrenchSDOH</th><th>InHouse Tuberculosis and ALS</th></tr><tr><td colspan="3">(n = 1,700)</td><td colspan="2">(n = 340)</td><td colspan="2">(n = 364)</td><td colspan="2">(n = 400)</td></tr><tr><th>Train</th><th>Dev</th><th>Test</th><th>Test</th><th>Test</th><th>Test</th><th>Test</th><th>Test</th></tr></thead><tbody><tr><td>Status</td><td>1,206</td><td>175</td><td>314</td><td>514</td><td>646</td><td>117</td><td></td><td></td></tr><tr><td>Amount</td><td>1,182</td><td>155</td><td>301</td><td>253</td><td>148</td><td>82</td><td></td><td></td></tr><tr><td>Duration</td><td>51</td><td>8</td><td>18</td><td>12</td><td>33</td><td>10</td><td></td><td></td></tr><tr><td>Frequency</td><td>489</td><td>68</td><td>124</td><td>124</td><td>135</td><td>35</td><td></td><td></td></tr><tr><td>History</td><td>262</td><td>30</td><td>73</td><td>44</td><td>55</td><td>13</td><td></td><td></td></tr><tr><td>Type</td><td>1,096</td><td>146</td><td>289</td><td>261</td><td>154</td><td>96</td><td></td><td></td></tr></tbody></table>

## 2.4 Experiment

Following recent studies on SDoH extraction leveraging large language models (LLMs) [32, 34], we used Flan-T5-Large model [33] in our experiments. We formulated SDoH extraction as a text-to-structure translation task, where the model receives a social history section from a clinical note and generates a linearized sequence of SDoH events. This sequence-to-sequence (seq2seq) formulation allows the model to jointly predict entities, attributes, and their relations within a single decoding pass. As illustrated in Figure 2, the output events are ordered from left to right according to their token offsets in the original text. To train the model, Flan-T5-Large was fine-tuned on the MUSCADET-InHouse training set, with input-output representations presented in Figure 2. The model was fine-tuned for 10 epochs on two 24GB NVIDIA RTX 4090 GPUs.

<figure><img src="image_3.png" /><figcaption>Figure 2: Example of (1) annotated social history section and (2) the corresponding structured sequence of SDoH events. Example translated into English: *Social History: Lives with his wife and two daughters. Does the housework, gardening, and drives. Has been tobacco-free for 33 years and alcohol-free for 15 years.*</figcaption></figure>

During inference for evaluation, the sequence of SDoH events generated by the model was post-processed to recover token offsets corresponding to entities and relations. The source text was then searched for tokens matching the SDoH events in the output sequence. However, the output sequence tokens often matched multiple offsets in the source text. Ambiguities were resolved by applying distinct strategies for entities and relations. For entities, when multiple non-overlapping matches were found, we selected the leftmost occurrence in the source text that had not already been extracted. For relations, we selected the match nearest to the associated entity to ensure contextual accuracy. For the InHouse Tuberculosis and ALS dataset, we evaluated the model using both the full clinical notes and the social history sections alone after preprocessing to assess its robustness on non-social history section texts.
## 2.5 Evaluation

We conducted the evaluation at two levels: (i) SDoH factors and associated values, and (ii) fine-grained SDoH extraction including all entities and relations.

In the level 1 evaluation, we assessed the exact match presence of labeled entities in the gold standard and the model's predictions. For alcohol, tobacco, and drug use, we included the corresponding Status relations to convert span-only entities into labeled entities (e.g., for Tobacco: Tobacco_StatusTime:current, Tobacco_StatusTime:past, Tobacco_StatusTime:none).

In the level 2 evaluation, we evaluated the extraction of SDoH as a slot-filling task, following prior work on evaluating SDoH extraction models in the context of 2022 n2c2/UW Shared Task [14]. This approach allows for multiple equivalent span annotations. Figure 3 illustrates this by presenting the same sentence with two equivalent sets of annotations. Event equivalence was defined using two criteria for model evaluation: exact-match spans and overlap-match spans. In the exact match setting, two events were considered equivalent if the entity offsets matched exactly between the gold standard and predictions, and their associated relation offsets also matched exactly. In the overlap match setting, two events were considered equivalent if the entity offsets shared at least one overlapping character between the gold standard and predictions, and their associated relation offsets also shared at least one overlapping character. The performance of the model for all evaluation settings was measured using precision (P), recall (R), and F1-score (F1).

<figure><img src="image_4.png" /><figcaption>Figure 3: Examples of substance use annotated as events. Annotations (a) and (b) are considered equivalent. English translation of the example: *Active smoking at 17 cigarettes per day*.</figcaption></figure>

## 2.6 Comparison with Structured EHR Data

To assess the completeness of SDoH documentation in structured versus unstructured EHR data, we collected Z-codes for all patients in the MUSCADET-InHouse dataset. Z-codes are ICD-10 codes that describe factors that influence health status and healthcare utilization when the primary reason for the encounter is not a specific disease or injury, which partially include SDoH-related codes. All collected Z-codes for MUSCADET-InHouse patients were manually mapped to SDoH categories if relevant (see Supplementary Table 8). We compared the presence of one or more SDoH categories in manually annotated text from MUSCADET-InHouse against the corresponding patient's Z-codes from structured EHR data.
# 3 Results

## 3.1 Inter-Annotator Agreement

Table 5 presents the inter-annotator agreement scores (F-measure). During the first phase of annotation for the MUSCADET-InHouse dataset, the average entity agreement was 0.689 before adjudication; it improved to 0.725 in the second phase. A similar trend was observed for relations, with an F-measure of 0.795 in the first phase, increasing to 0.829 in the second phase. For the MUSCADET-Synthetic dataset, the average agreement was 0.742 for entities and 0.788 for relations.

<table><caption>Table 5: Inter-annotator agreement.</caption><thead><tr><th></th><th>Entities - F-Measure</th><th>Relations - F-Measure</th></tr></thead><tbody><tr><td><strong>MUSCADET-InHouse</strong></td><td></td><td></td></tr><tr><td>Phase 1 - 100 documents</td><td>0.689</td><td>0.795</td></tr><tr><td>Phase 2 - 200 documents</td><td>0.725</td><td>0.829</td></tr><tr><td><strong>MUSCADET-Synthetic</strong></td><td>0.742</td><td>0.788</td></tr></tbody></table>

## 3.2 Model Performance

Table 6 shows the macro-averaged performance of the fine-tuned Flan-T5-Large model on all datasets. The model demonstrates stable performance when evaluated on the social history sections alone, achieving a macro-F1 score ranging from 0.7618 to 0.7863 in the level 1 evaluation setting (SDOH entities with associated values), and from 0.3934 to 0.4804 in the level 2 (SDOH extraction with all entities and relations) under the exact match criteria. However, when applied to full clinical notes from the InHouse Tuberculosis and ALS dataset, the model produces a high number of false positives, resulting in a substantial drop in performance, yielding a macro-F1 of 0.4017 in level 1, and 0.0451 in level 2 evaluation. The model's performance on the UW-FrenchSDOH dataset is slightly lower in the level 2 evaluation, likely due to the dataset being a translated dataset from English to French. Although the SDoH-related terms are accurately translated, the word order and writing style retain Anglophone patterns. Since the model was not trained on such translated or non-native-like data, it struggles to extract SDoH text spans precisely.

<table><caption>Table 6: Macro-averaged precision, recall, and F1 metrics of the seq2seq FlanT5-large model.</caption><thead><tr><th>Dataset</th><th>Precision</th><th>Recall</th><th>F1-score</th></tr></thead><tbody><tr><td><strong>MUSCADET-InHouse</strong></td><td></td><td></td><td></td></tr><tr><td>SDOH entities with associated values (level 1)</td><td>0.8133</td><td>0.7708</td><td>0.7863</td></tr><tr><td>All Entities and relations - Exact match (level 2)</td><td>0.5145</td><td>0.4790</td><td>0.4804</td></tr><tr><td>All Entities and relations - Overlap match (level 2)</td><td>0.6738</td><td>0.6313</td><td>0.6290</td></tr><tr><td><strong>MUSCADET-Synthetic</strong></td><td></td><td></td><td></td></tr><tr><td>SDOH entities with associated values (level 1)</td><td>0.8851</td><td>0.7344</td><td>0.7795</td></tr><tr><td>All Entities and relations - Exact match (level 2)</td><td>0.5343</td><td>0.4429</td><td>0.4629</td></tr><tr><td>All Entities and relations - Overlap match (level 2)</td><td>0.7222</td><td>0.5978</td><td>0.6297</td></tr><tr><td><strong>UW-FrenchSDOH</strong></td><td></td><td></td><td></td></tr><tr><td>SDOH entities with associated values (level 1)</td><td>0.7732</td><td>0.7972</td><td>0.7618</td></tr><tr><td>All Entities and relations - Exact match (level 2)</td><td>0.3985</td><td>0.4788</td><td>0.3934</td></tr><tr><td>All Entities and relations - Overlap match (level 2)</td><td>0.5716</td><td>0.6314</td><td>0.5446</td></tr><tr><td><strong>InHouse Tuberculosis and ALS - Raw full texts</strong></td><td></td><td></td><td></td></tr><tr><td>SDOH entities with associated values (level 1)</td><td>0.4167</td><td>0.5765</td><td>0.4017</td></tr><tr><td>All Entities and relations - Exact match (level 2)</td><td>0.0471</td><td>0.0521</td><td>0.0451</td></tr><tr><td>All Entities and relations - Overlap match (level 2)</td><td>0.1059</td><td>0.1518</td><td>0.1058</td></tr><tr><td><strong>InHouse Tuberculosis and ALS - Social history sections only</strong></td><td></td><td></td><td></td></tr><tr><td>SDOH entities with associated values (level 1)</td><td>0.8499</td><td>0.7487</td><td>0.7893</td></tr><tr><td>All Entities and relations - Exact match (level 2)</td><td>0.4847</td><td>0.4325</td><td>0.4426</td></tr><tr><td>All Entities and relations - Overlap match (level 2)</td><td>0.7316</td><td>0.6497</td><td>0.6700</td></tr></tbody></table>
Table 7 presents the model’s performance for each SDoH category. Some categories, such as living condition, marital status, descendants, job, tobacco and alcohol use, are well modeled by Flan-T5-Large with F1 scores over 0.80. These SDoH categories are often expressed in consistent, structured ways in clinical documents, making it easier for the model to learn their patterns. In contrast, categories such as employment status, housing, physical activity, income, and education present greater challenges. The model struggles to achieve consistent performance on these categories, likely due to the greater linguistic variability and contextual diversity in how these factors are documented in the clinical notes. This variability, along with the scarcity of certain SDoH categories, make generalization more difficult, especially when annotations vary in phrasing or context.

## 3.3 Impact of Applying Model on Non-Social History Section Text

Applying the model to entire clinical documents significantly increases the number of incorrect predictions (false positives). Specifically, the model's performance on full-text documents is considerably lower than when applied to social history sections only, with a macro-F1 of 0.4017 compared to 0.7893 in the level 1 evaluation setting. This discrepancy is expected, as the model was trained exclusively on social history sections and does not generalize effectively to other parts of the clinical notes. While restricting inference to social history sections improves precision and overall performance, this approach risks missing important SDoH information that may appear elsewhere in the document. Thus, there is a trade-off between achieving high precision and ensuring comprehensive recall of patient-related social information.

To assess the extent of information missed under this constraint, we compared the number of SDoH annotations in the InHouse Tuberculosis and ALS dataset across full documents versus social history sections only. Across the full dataset, 665 annotations were identified, of which 461 (69.3%) were located within the social history sections. This indicates that 204 annotations (30.7%) lie outside these sections. Among these, 87 annotations occurred in documents that do not include a social history section, while the remaining 117 were found outside the social history section in documents that did include one. Notably, 81 of the 117 were redundant —i.e., SDoH categories that were already mentioned within the corresponding social history section. The remaining 36 annotations represented unique SDoH information not captured in the social history sections. These were primarily related to substance use (tobacco, alcohol, and drug use), commonly discussed in sections such as medical history or risk factors. In total, restricting model inference to social history sections results in 123 missed unique SDoH annotations (87 from documents without a social history section, and 36 unique mentions from documents with one), accounting for approximately 18.5% of all SDoH annotations in the InHouse Tuberculosis and ALS dataset.

## 3.4 Error Analysis

Supplementary Table 9 provides an overview of the primary differences observed between the model outputs and the reference annotations. Through qualitative inspection, we categorized these discrepancies into eight distinct error types: (1) human annotation errors, (2) false positives, (3) false negatives, (4) difficulties in adhering to the structured output format, and (5) cases where the predicted text span was correct but the associated label was incorrect. Additional discrepancies labeled as errors were, in fact, not entirely incorrect; these resulted from (6) post-processing rules—for instance, when multiple identical text spans were present—or from (7) model predictions that differed from the ground truth annotation in terms of text spans but were nonetheless valid in the context of the slot-filling task. A small number of errors also stemmed from (8) limitations of the tokenizer, which did not support several French characters, such as “i”. This led the model to generate incorrect forms, such as producing “cocane” instead of “cocaïne”, thereby introducing errors in post-processing.

## 3.5 Comparison with Structured EHR Data

Manual annotation of the MUSCADET-InHouse dataset identified at least one SDoH category in 98.5% of patients (1621/1646). In contrast, structured EHR data, based on Z-codes, captured SDoH information in only 2.8% of cases (46/1646). Among these, 17 SDoH mentions overlapped between the two sources. The remaining non-overlapping instances from the structured data were mainly associated with Z-codes such as Z29.0 and Z60.20, which correspond to living alone.
<table><caption>Table 7: Performance of Flan-T5-Large on SDoH categories.</caption><thead><tr><th rowspan="3">SDoH category</th><th colspan="3">SDoH entities with associated values (level 1)</th><th colspan="6">All entities and relations (level 2)</th></tr><tr><th rowspan="2">P</th><th rowspan="2">R</th><th rowspan="2">F1</th><th colspan="3">Exact match</th><th colspan="3">Overlap match</th></tr><tr><th>P</th><th>R</th><th>F1</th><th>P</th><th>R</th><th>F1</th></tr></thead><tbody><tr><td><strong>Living condition</strong></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>MUSCADET-InHouse</td><td>0.9141</td><td>0.9088</td><td>0.9113</td><td>0.8228</td><td>0.8088</td><td>0.8157</td><td>0.9101</td><td>0.8947</td><td>0.9023</td></tr><tr><td>MUSCADET-Synthetic</td><td>0.8939</td><td>0.9112</td><td>0.9024</td><td>0.8439</td><td>0.8606</td><td>0.8521</td><td>0.8884</td><td>0.9056</td><td>0.8968</td></tr><tr><td>UW-FrenchSDOH</td><td>0.8513</td><td>0.8727</td><td>0.8603</td><td>0.8422</td><td>0.8316</td><td>0.8308</td><td>0.8422</td><td>0.8316</td><td>0.8308</td></tr><tr><td>InHouse Tuberc./ALS – Raw full texts</td><td>0.4861</td><td>0.6387</td><td>0.5265</td><td>0.1266</td><td>0.1568</td><td>0.1363</td><td>0.2873</td><td>0.3296</td><td>0.3007</td></tr><tr><td>InHouse Tuberc./ALS – Social history sections</td><td>0.9630</td><td>0.6950</td><td>0.8073</td><td>0.8704</td><td>0.6274</td><td>0.7292</td><td>0.9445</td><td>0.6815</td><td>0.7917</td></tr><tr><td><strong>Marital status</strong></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>MUSCADET-InHouse</td><td>0.8681</td><td>0.9358</td><td>0.9007</td><td>0.6131</td><td>0.8089</td><td>0.6759</td><td>0.6701</td><td>0.8677</td><td>0.7337</td></tr><tr><td>MUSCADET-Synthetic</td><td>0.9550</td><td>0.9206</td><td>0.9370</td><td>0.7300</td><td>0.7140</td><td>0.7211</td><td>0.8317</td><td>0.8213</td><td>0.8255</td></tr><tr><td>UW-FrenchSDOH</td><td>0.9196</td><td>0.9115</td><td>0.9154</td><td>0.6647</td><td>0.5910</td><td>0.6138</td><td>0.7507</td><td>0.6783</td><td>0.7005</td></tr><tr><td>InHouse Tuberc./ALS – Raw full texts</td><td>0.8385</td><td>0.5821</td><td>0.6778</td><td>0.0147</td><td>0.0104</td><td>0.0122</td><td>0.3199</td><td>0.2222</td><td>0.2587</td></tr><tr><td>InHouse Tuberc./ALS – Social history sections</td><td>0.9500</td><td>0.7212</td><td>0.8131</td><td>0.8809</td><td>0.6330</td><td>0.7313</td><td>0.9341</td><td>0.6840</td><td>0.7834</td></tr><tr><td><strong>Descendants</strong></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>MUSCADET-InHouse</td><td>0.9838</td><td>0.9891</td><td>0.9864</td><td>0.5769</td><td>0.6176</td><td>0.5963</td><td>0.8831</td><td>0.9371</td><td>0.9091</td></tr><tr><td>MUSCADET-Synthetic</td><td>0.9733</td><td>0.9696</td><td>0.9715</td><td>0.6711</td><td>0.6730</td><td>0.6721</td><td>0.9548</td><td>0.9580</td><td>0.9564</td></tr><tr><td>UW-FrenchSDOH</td><td>0.8114</td><td>0.9688</td><td>0.8831</td><td>0.4098</td><td>0.5210</td><td>0.4556</td><td>0.7221</td><td>0.8881</td><td>0.7919</td></tr><tr><td>InHouse Tuberc./ALS – Raw full texts</td><td>0.7375</td><td>0.6919</td><td>0.6857</td><td>0.0886</td><td>0.0728</td><td>0.0784</td><td>0.2527</td><td>0.2359</td><td>0.2390</td></tr><tr><td>InHouse Tuberc./ALS – Social history sections</td><td>0.9878</td><td>0.9045</td><td>0.9424</td><td>0.6814</td><td>0.6374</td><td>0.6574</td><td>0.9214</td><td>0.8756</td><td>0.8963</td></tr><tr><td><strong>Housing</strong></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>MUSCADET-InHouse</td><td>0.4272</td><td>0.4147</td><td>0.4209</td><td>0.2158</td><td>0.2094</td><td>0.2125</td><td>0.3655</td><td>0.3547</td><td>0.3600</td></tr><tr><td>MUSCADET-Synthetic</td><td>0.6286</td><td>0.4993</td><td>0.4824</td><td>0.3809</td><td>0.1983</td><td>0.2106</td><td>0.6091</td><td>0.4770</td><td>0.4615</td></tr><tr><td>UW-FrenchSDOH</td><td>0.4265</td><td>0.6744</td><td>0.5225</td><td>0.1159</td><td>0.1667</td><td>0.1368</td><td>0.3623</td><td>0.5208</td><td>0.4274</td></tr><tr><td>InHouse Tuberc./ALS – Raw full texts</td><td>0.1537</td><td>0.3281</td><td>0.2093</td><td>0.0156</td><td>0.0219</td><td>0.0182</td><td>0.0518</td><td>0.0730</td><td>0.0606</td></tr><tr><td>InHouse Tuberc./ALS – Social history sections</td><td>0.4768</td><td>0.4184</td><td>0.4457</td><td>0.2322</td><td>0.1831</td><td>0.2047</td><td>0.4286</td><td>0.3381</td><td>0.3780</td></tr><tr><td><strong>Employment</strong></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>MUSCADET-InHouse</td><td>0.8707</td><td>0.6892</td><td>0.7482</td><td>0.5945</td><td>0.5029</td><td>0.5387</td><td>0.7949</td><td>0.6282</td><td>0.6855</td></tr><tr><td>MUSCADET-Synthetic</td><td>0.8248</td><td>0.6995</td><td>0.7275</td><td>0.6004</td><td>0.4370</td><td>0.4674</td><td>0.8158</td><td>0.5807</td><td>0.6339</td></tr><tr><td>UW-FrenchSDOH</td><td>0.7138</td><td>0.7369</td><td>0.7054</td><td>0.6180</td><td>0.6042</td><td>0.5816</td><td>0.6368</td><td>0.6229</td><td>0.6018</td></tr><tr><td>InHouse Tuberc./ALS – Raw full texts</td><td>0.1731</td><td>0.5366</td><td>0.2470</td><td>0.0332</td><td>0.0756</td><td>0.0447</td><td>0.1325</td><td>0.4086</td><td>0.1940</td></tr><tr><td>InHouse Tuberc./ALS – Social history sections</td><td>0.8352</td><td>0.7303</td><td>0.7506</td><td>0.3728</td><td>0.3798</td><td>0.3744</td><td>0.8400</td><td>0.7138</td><td>0.7454</td></tr><tr><td><strong>Alcohol</strong></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>MUSCADET-InHouse</td><td>0.8561</td><td>0.8169</td><td>0.8357</td><td>0.4472</td><td>0.4348</td><td>0.4405</td><td>0.7068</td><td>0.6834</td><td>0.6943</td></tr><tr><td>MUSCADET-Synthetic</td><td>0.9257</td><td>0.6504</td><td>0.7443</td><td>0.4192</td><td>0.3318</td><td>0.3656</td><td>0.7526</td><td>0.5839</td><td>0.6490</td></tr><tr><td>UW-FrenchSDOH</td><td>0.9379</td><td>0.8182</td><td>0.8708</td><td>0.0555</td><td>0.0662</td><td>0.0577</td><td>0.4988</td><td>0.4683</td><td>0.4763</td></tr><tr><td>InHouse Tuberc./ALS – Raw full texts</td><td>0.3423</td><td>0.4998</td><td>0.3287</td><td>0.0563</td><td>0.0533</td><td>0.0497</td><td>0.1325</td><td>0.1584</td><td>0.1225</td></tr><tr><td>InHouse Tuberc./ALS – Social history sections</td><td>0.9364</td><td>0.8714</td><td>0.9000</td><td>0.1777</td><td>0.1655</td><td>0.1708</td><td>0.6574</td><td>0.6205</td><td>0.6277</td></tr><tr><td><strong>Tobacco</strong></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>MUSCADET-InHouse</td><td>0.9825</td><td>0.8888</td><td>0.9332</td><td>0.7019</td><td>0.6374</td><td>0.6663</td><td>0.8367</td><td>0.7599</td><td>0.7945</td></tr><tr><td>MUSCADET-Synthetic</td><td>0.8944</td><td>0.7566</td><td>0.8181</td><td>0.6076</td><td>0.5358</td><td>0.5481</td><td>0.7310</td><td>0.6537</td><td>0.6853</td></tr><tr><td>UW-FrenchSDOH</td><td>0.8664</td><td>0.8597</td><td>0.8607</td><td>0.4437</td><td>0.4122</td><td>0.4241</td><td>0.6566</td><td>0.6214</td><td>0.6343</td></tr><tr><td>InHouse Tuberc./ALS – Raw full texts</td><td>0.5460</td><td>0.6102</td><td>0.5500</td><td>0.0414</td><td>0.0590</td><td>0.0475</td><td>0.0749</td><td>0.0911</td><td>0.0799</td></tr><tr><td>InHouse Tuberc./ALS – Social history sections</td><td>1.0000</td><td>0.8975</td><td>0.9445</td><td>0.6893</td><td>0.6072</td><td>0.6379</td><td>0.9025</td><td>0.7890</td><td>0.8318</td></tr><tr><td><strong>Drug</strong></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>MUSCADET-InHouse</td><td>0.6000</td><td>0.5048</td><td>0.5444</td><td>0.6945</td><td>0.4222</td><td>0.5126</td><td>0.7445</td><td>0.4556</td><td>0.5526</td></tr><tr><td>MUSCADET-Synthetic</td><td>0.9097</td><td>0.5598</td><td>0.6663</td><td>0.3304</td><td>0.1439</td><td>0.1906</td><td>0.6220</td><td>0.2911</td><td>0.3728</td></tr><tr><td>UW-FrenchSDOH</td><td>0.7211</td><td>0.6354</td><td>0.5716</td><td>0.2852</td><td>0.2245</td><td>0.2324</td><td>0.5378</td><td>0.3661</td><td>0.3938</td></tr><tr><td>InHouse Tuberc./ALS – Raw full texts</td><td>0.4468</td><td>0.5516</td><td>0.3404</td><td>0.0000</td><td>0.0000</td><td>0.0000</td><td>0.0000</td><td>0.0000</td><td>0.0000</td></tr><tr><td>InHouse Tuberc./ALS – Social history sections</td><td>1.0000</td><td>1.0000</td><td>1.0000</td><td>0.4167</td><td>0.2315</td><td>0.2778</td><td>0.6667</td><td>0.4537</td><td>0.5093</td></tr><tr><td><strong>Physical activity</strong></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>MUSCADET-InHouse</td><td>0.8167</td><td>0.7891</td><td>0.7955</td><td>0.1179</td><td>0.2689</td><td>0.1514</td><td>0.2504</td><td>0.5531</td><td>0.3177</td></tr><tr><td>MUSCADET-Synthetic</td><td>0.9606</td><td>0.6429</td><td>0.7660</td><td>0.2249</td><td>0.1044</td><td>0.1374</td><td>0.3713</td><td>0.1996</td><td>0.2528</td></tr><tr><td>UW-FrenchSDOH</td><td>0.7106</td><td>0.6970</td><td>0.6667</td><td>0.2302</td><td>0.1052</td><td>0.1302</td><td>0.6111</td><td>0.2083</td><td>0.2730</td></tr><tr><td>InHouse Tuberc./ALS – Raw full texts</td><td>0.0259</td><td>0.7500</td><td>0.0500</td><td>0.0000</td><td>0.0000</td><td>0.0000</td><td>0.0046</td><td>0.1667</td><td>0.0089</td></tr><tr><td>InHouse Tuberc./ALS – Social history sections</td><td>0.5000</td><td>0.5000</td><td>0.5000</td><td>0.0000</td><td>0.0000</td><td>0.0000</td><td>0.8333</td><td>0.8333</td><td>0.8333</td></tr><tr><td><strong>Job</strong></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>MUSCADET-InHouse</td><td>-</td><td>-</td><td>-</td><td>0.7456</td><td>0.6883</td><td>0.7158</td><td>0.9342</td><td>0.8623</td><td>0.8968</td></tr><tr><td>MUSCADET-Synthetic</td><td>-</td><td>-</td><td>-</td><td>0.8010</td><td>0.7454</td><td>0.7722</td><td>0.8856</td><td>0.8241</td><td>0.8537</td></tr><tr><td>UW-FrenchSDOH</td><td>-</td><td>-</td><td>-</td><td>0.6702</td><td>0.6632</td><td>0.6667</td><td>0.8191</td><td>0.8105</td><td>0.8148</td></tr><tr><td>InHouse Tuberc./ALS – Raw full texts</td><td>-</td><td>-</td><td>-</td><td>0.0379</td><td>0.0862</td><td>0.0526</td><td>0.1061</td><td>0.2414</td><td>0.1474</td></tr><tr><td>InHouse Tuberc./ALS – Social history sections</td><td>-</td><td>-</td><td>-</td><td>0.6667</td><td>0.5833</td><td>0.6222</td><td>0.8810</td><td>0.7708</td><td>0.8222</td></tr><tr><td><strong>Last job</strong></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>MUSCADET-InHouse</td><td>-</td><td>-</td><td>-</td><td>0.701</td><td>0.659</td><td>0.6793</td><td>0.8676</td><td>0.8157</td><td>0.8409</td></tr><tr><td>MUSCADET-Synthetic</td><td>-</td><td>-</td><td>-</td><td>0.7919</td><td>0.7647</td><td>0.7781</td><td>0.8680</td><td>0.8382</td><td>0.8529</td></tr><tr><td>UW-FrenchSDOH</td><td>-</td><td>-</td><td>-</td><td>0.6818</td><td>0.7059</td><td>0.6936</td><td>0.8295</td><td>0.8588</td><td>0.8439</td></tr><tr><td>InHouse Tuberc./ALS – Raw full texts</td><td>-</td><td>-</td><td>-</td><td>0.0315</td><td>0.0816</td><td>0.0455</td><td>0.0866</td><td>0.2245</td><td>0.1250</td></tr><tr><td>InHouse Tuberc./ALS – Social history sections</td><td>-</td><td>-</td><td>-</td><td>0.6316</td><td>0.5854</td><td>0.6076</td><td>0.8158</td><td>0.7561</td><td>0.7848</td></tr><tr><td><strong>Income</strong></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>MUSCADET-InHouse</td><td>-</td><td>-</td><td>-</td><td>0.2500</td><td>0.0769</td><td>0.1177</td><td>0.3750</td><td>0.1154</td><td>0.1765</td></tr><tr><td>MUSCADET-Synthetic</td><td>-</td><td>-</td><td>-</td><td>0.5000</td><td>0.1250</td><td>0.2000</td><td>0.5000</td><td>0.1250</td><td>0.2000</td></tr><tr><td>UW-FrenchSDOH</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td></tr><tr><td>InHouse Tuberc./ALS – Raw full texts</td><td>-</td><td>-</td><td>-</td><td>0.0000</td><td>0.0000</td><td>0.0000</td><td>0.0000</td><td>0.0000</td><td>0.0000</td></tr><tr><td>InHouse Tuberc./ALS – Social history sections</td><td>-</td><td>-</td><td>-</td><td>0.0000</td><td>0.0000</td><td>0.0000</td><td>0.0000</td><td>0.0000</td><td>0.0000</td></tr><tr><td><strong>Education</strong></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>MUSCADET-InHouse</td><td>-</td><td>-</td><td>-</td><td>0.3889</td><td>0.3889</td><td>0.3889</td><td>0.6111</td><td>0.6111</td><td>0.6111</td></tr><tr><td>MUSCADET-Synthetic</td><td>-</td><td>-</td><td>-</td><td>0.3478</td><td>0.4000</td><td>0.3721</td><td>0.4348</td><td>0.5000</td><td>0.4651</td></tr><tr><td>UW-FrenchSDOH</td><td>-</td><td>-</td><td>-</td><td>0.0526</td><td>0.3333</td><td>0.0909</td><td>0.0526</td><td>0.3333</td><td>0.0909</td></tr><tr><td>InHouse Tuberc./ALS – Raw full texts</td><td>-</td><td>-</td><td>-</td><td>0.0000</td><td>0.0000</td><td>0.0000</td><td>0.0000</td><td>0.0000</td><td>0.0000</td></tr><tr><td>InHouse Tuberc./ALS – Social history sections</td><td>-</td><td>-</td><td>-</td><td>0.5000</td><td>1.0000</td><td>0.6667</td><td>0.5000</td><td>1.0000</td><td>0.6667</td></tr><tr><td><strong>Ethnicity/Country of birth</strong></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>MUSCADET-InHouse</td><td>-</td><td>-</td><td>-</td><td>0.7500</td><td>0.5714</td><td>0.6486</td><td>0.9375</td><td>0.7143</td><td>0.8108</td></tr><tr><td>MUSCADET-Synthetic</td><td>-</td><td>-</td><td>-</td><td>0.2308</td><td>0.1667</td><td>0.1935</td><td>0.8462</td><td>0.6111</td><td>0.7097</td></tr><tr><td>UW-FrenchSDOH</td><td>-</td><td>-</td><td>-</td><td>0.1111</td><td>1.0000</td><td>0.2000</td><td>0.1111</td><td>1.0000</td><td>0.2000</td></tr><tr><td>InHouse Tuberc./ALS – Raw full texts</td><td>-</td><td>-</td><td>-</td><td>0.2143</td><td>0.1111</td><td>0.1463</td><td>0.4286</td><td>0.2222</td><td>0.2927</td></tr><tr><td>InHouse Tuberc./ALS – Social history sections</td><td>-</td><td>-</td><td>-</td><td>0.6667</td><td>0.4211</td><td>0.5161</td><td>0.9167</td><td>0.5789</td><td>0.7097</td></tr></tbody></table>
## 4 Discussion

We developed a sequence-to-sequence model to extract 13 SDoH categories from French clinical notes, demonstrating the potential of large language models for enhancing the collection of real-world SDoH data. Our model performed well in identifying SDoH mentions in clinical notes and showed consistent performance across four datasets, including two that are publicly available to the research community. SDoH mentions extracted from clinical notes identified 95.8% patients with relevant information, compared to 2.8% for ICD-10 codes from structured EHR data, underscoring the added value of unstructured data. These results highlight the effectiveness of NLP approaches in leveraging unstructured clinical notes to improve the completeness of real-world data, which is often missing or sparsely represented in structured EHR data. For example, ICD-10 Z-codes describing SDoH (e.g. 'Problems relating to housing and economic circumstances') are used in less than 5% of cases by clinicians in routine discharge coding practice, whereas automated NLP systems can recover comparable information with far less effort, requiring about one day of processing versus nine person-days per physician [49]. Yet clinician documentation habits remain a bottleneck: in a U.S. study of >5 million patients, structured data such as address and race were well-documented, while housing, income, and social isolation were mentioned in less than 5% of records [50]. Greater clinician awareness and consistent recording of these factors are therefore essential.

Our model achieved strong performance (macro-F1 > 0.80) in identifying well-documented SDoH categories (living condition, marital status, descendants, smoking status, alcohol use, employment and physical activity) but lower scores for housing status and drug use. These discrepancies were primarily due to inconsistencies in human annotation, limited training data, and highly variable language, ranging from direct mentions (e.g., "apartment," "house") to more indirect or context-dependent references (e.g., "nursing home," "home nurse," "in-home assistance"). These results highlight the strengths of our approach in extracting high-level SDoH categories, which is particularly relevant for secondary use applications. Since the output is already structured for each SDoH category, it can be directly integrated into clinical databases and research cohorts without requiring additional post-processing. However, when more granular detail is needed, fine-grained SDoH extraction with entities and relations involves additional post-processing steps and result in lower and less stable performance across SDoH categories.

Our error analysis revealed several limitations in using language models to extract SDoH from French clinical notes. While such models are generally capable of identifying the presence of relevant concepts, they often struggle to precisely extract detailed information. This performance gap may be explained by multiple factors, including the relatively small number of models' parameters compared to state-of-the-art architectures, and quality issues in the annotated data. Indeed, model performance is inherently limited by the quality and consistency of the annotations, which are challenging in the SDoH domain due to its conceptual complexity. Annotator bias and inconsistency further reduce reliability and, consequently, model accuracy. Additional errors stem from the use of English-based tokenizers, which often mishandle accented characters. As a result, post-processing becomes difficult: the predicted spans cannot be reliably aligned with gold annotations, and character offsets are often miscalculated during evaluation. These issues underscore the need for tokenizers and models tailored to specific languages, as most publicly available models are English-centric and may not generalize well to other languages or multilingual contexts [51, 52]. Moreover, the generation-based approach introduces alignment errors during post-processing. Specifically, selecting the leftmost matching text span to align the generated SDoH outputs can result in incorrect mappings. Similarly, associating predicted entities with their nearest potential relation arguments can introduce a proximity bias, potentially overlooking longer-range dependencies.

Applying NLP to EHR data poses a persistent challenge of transferability. Models often struggle to maintain consistent performance across different patient sub-populations within the same institution, and even more so across hospitals or over time as clinical language and practices evolve. This limits the reliability, equity, and generalizability of NLP-driven insights, underscoring the importance of adaptable, continuously validated models in clinical settings. This highlights the need for ongoing ad-hoc validation studies, underlying the importance of methodological transparency in studies like ours, including the release of code and data.

Another key challenge in SDoH research is the scarcity of resources in languages other than English, which limits the development of NLP methods that account for social and cultural variability across healthcare systems. SDoH are deeply context-dependent, shaped by language, culture, policy, and
local healthcare practice, making it essential to develop corpora that reflect diverse populations [53]. A major motivation behind our work is to address this gap by providing a French-language SDoH corpus that is openly accessible and free from legal constraints. Given the sensitivity of medical data under the General Data Protection Regulation (GDPR), we adopted a dual approach to ensure compliance: generating synthetic social history sections authored by a physician and translating publicly available English-language a dataset from the University of Washington into French. This approach enables us to uphold privacy standards while advancing FAIR research principles. By introducing a corpus tailored to the French clinical context, we aim to promote inclusivity and facilitate the development of NLP methods for French-speaking populations and foster multilingual research in SDoH extraction.

Our study has several limitations that affect the generalizability of our findings. First, our training dataset was derived from a predominantly Caucasian population treated at one comprehensive center, Nantes University hospital. This demographic skew impacted certain SDoH categories, such as ethnicity, which are more likely to be documented for non-caucasian individuals. In addition, ethnic data are not usually collected by French physicians unless deemed relevant for healthcare purposes. In general, we observed variation in the amount of SDoH information available across populations. On average, patients born in France (n = 1397, mean = 5.45 (SD 2.32) SDoH mentions) had more SDoH information recorded (P < 0.01 using a Student's t-test) than patients born outside France (n = 239, mean = 4.87 (SD 2.29) SDoH mentions), suggesting possible disparities in data completeness. A second limitation is that we trained our model only on the social history sections of clinical notes, rather than on full-text documents. While this decision reduced annotation effort, it limited the model's ability to generalize to other sections. As a result, the model is not directly applicable to raw clinical notes and requires a preprocessing step to isolate the relevant sections prior to inference. This design choice may also reduce recall, as SDoH can also appear in other sections of clinical notes. Furthermore, because not all clinical notes include a social history section, information may be missed for certain patients.

The availability and quality of SDoH documentation in EHRs is often limited and inconsistent. Real-world data are primarily collected for clinical and administrative purposes by physicians during patient care, rather than for secondary use in research. Consequently, certain SDoH categories, apart from substance use, which is well known as a risk factor, are often overlooked during consultations. Several factors contribute to this under-documentation: lack of awareness among healthcare providers about the relevance of social factors to health outcomes, discomfort with asking about these factors, and restricted resources, staffing, and time to conduct screenings, which often compete with medical priorities [54]. Moreover, as Nantes University Hospital serves as the comprehensive center in our region, physicians there tend to focus more on medical care and less on the social environment than general practitioners [55]. As a result, SDoH information is frequently missing or incomplete, even in unstructured formats within the EHR. This under-documentation limits the ability to study social determinants at scale and hinders efforts to reduce health disparities. It also impairs the capacity of health systems to implement targeted, equity-oriented interventions based on complete, representative patient data.

# 5 Conclusion

Social determinants of health have a profound impact on both individual and population health outcomes, influencing morbidity, mortality, and healthcare access. Yet, SDoH are often under-documented in EHRs, particularly in structured data. In this work, we developed and evaluated a sequence-to-sequence language model to extract 13 SDoH categories from French clinical notes, demonstrating the effectiveness of NLP for improving the completeness of real-world health data. Our model consistently outperformed structured EHR data by identifying the majority of relevant SDoH across all patients and showed robust performance across multiple datasets, including publicly available benchmarks.

Future work will explore data augmentation techniques and the use of synthetic clinical text to improve the model's generalization, and facilitate the open-source release of both the model and annotated training dataset to support reproducibility and multilingual SDoH research. Ultimately, advancing automated SDoH extraction from unstructured clinical text can support more equitable healthcare by enabling richer, more representative data for research, policy-making, and population health interventions.
## Acknowledgments

This work was financially supported, in part, by the Agence Nationale de la Recherche (ANR) AIBy4 under contract ANR-20-THIA-0011, ANR MALADES under contract ANR-23-IAS1-0005, a grant from the French Ministry of Health (DAtAE2023-15744432) and the cluster DELPHI - NExT under contract ANR-16-IDEX-0007, integrated to France 2030 plan, by Région Pays de la Loire and by Nantes Métropole. We would also like to thank Emilie Varey for her administrative support in managing the project.

## Authors Contribution

AB: Conceptualization, Methodology (model training and evaluation), Investigation, Resources, Data curation, Writing- Original draft, Formal analysis, Visualization. PCDB: Conceptualization, Investigation, Resources, Data curation, Writing- Original draft preparation, Formal analysis, Visualization. Funding acquisition. MH: Resources, Methodology, Investigation, Formal analysis. RB: Supervision, Writing- Reviewing and Editing. EM: Supervision, Writing- Reviewing and Editing. RD: Supervision, Writing- Reviewing and Editing. BD: Supervision, Writing- Reviewing and Editing. PAG: Supervision, Writing- Reviewing and Editing. MK: Conceptualization, Methodology, Investigation, Writing- Reviewing and Editing, Project administration, Supervision.

## Conflict of Interest Disclosure

P.A. Gourraud is the founder of Methodomics (2008, www.methodomics.com) and of its spin-off Big Data Santé (2018, commercial name: “Octopize”). He acts as a consultant and/or contributor for several pharmaceutical and medical device companies. All related activities are conducted under institutional (university or hospital) contracts with the following entities: AstraZeneca, Amgen, Biogen, Boston Scientific, Cemka, Cook, Docaposte/Heva, Edimark, Ellipses, Elsevier, Grunenthal, Janssen, IAGE, Lek, Methodomics, Merck, Mérieux, Novartis, Octopize, Sanofi-Genzyme, Lifen, TuneInsight, and Aspire UAE. He serves as an unpaid board member of AXA mutual insurance (since 2021). He does not prescribe medications or medical devices and receives no personal remuneration. All other authors declare no competing interests.

## References

[1] Ophélie Merville, Florian Bonnet, Guy Launoy, Carlo Giovanni Camarda, and Emmanuelle Cambois. Unpacking occupational and sex divides to understand the moderate progress in life expectancy in recent years (france, 2010's). *International Journal for Equity in Health*, 23(1):1–12, 2024.

[2] Raj Chetty, Michael Stepner, Sarah Abraham, Shelby Lin, Benjamin Scuderi, Nicholas Turner, Augustin Bergeron, and David Cutler. The association between income and life expectancy in the united states, 2001-2014. *Jama*, 315(16):1750–1766, 2016.

[3] Michael Marmot. Social determinants of health inequalities. *The lancet*, 365(9464):1099–1104, 2005.

[4] Neal Halfon, Kandyce Larson, Michael Lu, Ericka Tullis, and Shirley Russ. Lifecourse health development: past, present and future. *Maternal and child health journal*, 18:344–365, 2014.

[5] Hugh Alderwick and Laura M Gottlieb. Meanings and misunderstandings: a social determinants of health lexicon for health care systems. *The Milbank Quarterly*, 97(2):407, 2019.

[6] Daniel Nettle. Why are there social gradients in preventative health behavior? a perspective from behavioral ecology. *PLoS One*, 5(10):e13371, 2010.

[7] Carlyn M Hood, Keith P Gennuso, Geoffrey R Swain, and Bridget B Catlin. County health rankings: relationships between determinant factors and health outcomes. *American journal of preventive medicine*, 50(2):129–135, 2016.
[8] Michael Wang, Matthew S Pantell, Laura M Gottlieb, and Julia Adler-Milstein. Documentation and review of social determinants of health data in the ehr: measures and associated insights. *Journal of the American Medical Informatics Association*, 28(12):2608–2616, 2021.

[9] Elizabeth S Chen, Sharad Manaktala, Indra Neil Sarkar, and Genevieve B Melton. A multi-site content analysis of social history information in clinical notes. In *AMIA Annual Symposium Proceedings*, volume 2011, page 227, 2011.

[10] Adrien Bazoge, Emmanuel Morin, Béatrice Daille, and Pierre-Antoine Gourraud. Applying natural language processing to textual data from clinical data warehouses: Systematic review. *JMIR medical informatics*, 11:e42477, 2023.

[11] Karen Hacker, John Auerbach, Robin Ikeda, Celeste Philip, and Debra Houry. Social determinants of health—an approach taken at cdc. *Journal of public health management and practice*, 28(6):589–594, 2022.

[12] Braja G Patra, Mohit M Sharma, Veer Vekaria, Prakash Adekkanattu, Olga V Patterson, Benjamin Glicksberg, Lauren A Lepow, Euijung Ryu, Joanna M Biernacka, Al'ona Furmanchuk, et al. Extracting social determinants of health from electronic health records using natural language processing: a systematic review. *Journal of the American Medical Informatics Association*, 28(12):2716–2727, 2021.

[13] Özlem Uzuner, Ira Goldstein, Yuan Luo, and Isaac Kohane. Identifying patient smoking status from medical discharge records. *Journal of the American Medical Informatics Association*, 15(1):14–24, 2008.

[14] Kevin Lybarger, Meliha Yetisgen, and Özlem Uzuner. The 2022 n2c2/uw shared task on extracting social determinants of health. *Journal of the American Medical Informatics Association*, 30(8):1367–1378, 2023.

[15] Charlie M Wray, Marzieh Vali, Louise C Walter, Lee Christensen, Samir Abdelrahman, Wendy Chapman, and Salomeh Keyhani. Examining the interfacility variation of social determinants of health in the veterans health administration. *Federal Practitioner*, 38(1):15, 2021.

[16] Daniel J Feller, Jason Zucker, Bharat Srikishan, Roxana Martinez, Henry Evans, Michael T Yin, Peter Gordon, Noémie Elhadad, et al. Towards the inference of social and behavioral determinants of sexual health: development of a gold-standard corpus with semi-supervised learning. In *AMIA Annual Symposium Proceedings*, volume 2018, page 422, 2018.

[17] Majid Afshar, Andrew Phillips, Niranjan Karnik, Jeanne Mueller, Daniel To, Richard Gonzalez, Ron Price, Richard Cooper, Cara Joyce, and Dmitriy Dligach. Natural language processing and machine learning to identify alcohol misuse from the electronic health record in trauma patients: development and internal validation. *Journal of the American Medical Informatics Association*, 26(3):254–261, 2019.

[18] Rachel Stemerman, Jaime Arguello, Jane Brice, Ashok Krishnamurthy, Mary Houston, and Rebecca Kitzmiller. Identification of social determinants of health using multi-label classification of electronic health record clinical notes. *JAMIA open*, 4(3):00aa069, 2021.

[19] Braja Gopal Patra, Lauren A Lepow, Praneet Kasi Reddy Jagadeesh Kumar, Veer Vekaria, Mohit Manoj Sharma, Prakash Adekkanattu, Brian Fennessy, Gavin Hynes, Isotta Landi, Jorge A Sanchez-Ruiz, et al. Extracting social support and social isolation information from clinical psychiatry notes: comparing a rule-based natural language processing system and a large language model. *Journal of the American Medical Informatics Association*, 32(1):218–226, 2025.

[20] Mike Conway, Salomeh Keyhani, Lee Christensen, Brett R South, Marzieh Vali, Louise C Walter, Danielle L Mowery, Samir Abdelrahman, and Wendy W Chapman. Moonstone: a novel natural language processing system for inferring social risk from clinical narratives. *Journal of biomedical semantics*, 10:1–10, 2019.

[21] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed representations of words and phrases and their compositionality. *Advances in neural information processing systems*, 26, 2013.
[22] Cosmin A Bejan, John Angiolillo, Douglas Conway, Robertson Nash, Jana K Shirey-Rice, Loren Lipworth, Robert M Cronin, Jill Pulley, Sunil Kripalani, Shari Barkin, et al. Mining 100 million notes to find homelessness and adverse childhood experiences: 2 case studies of rare and severe social determinants of health in electronic health records. *Journal of the American Medical Informatics Association*, 25(1):61–71, 2018.

[23] Maxim Topaz, Ludmila Murga, Ofrit Bar-Bachar, Kenrick Cato, and Sarah Collins. Extracting alcohol and substance abuse status from clinical notes: The added value of nursing data. In *MEDINFO 2019: Health and Wellbeing e-Networks for All*, pages 1056–1060. IOS Press, 2019.

[24] Adi V Gundlapalli, Marjorie E Carter, Miland Palmer, Thomas Ginter, Andrew Redd, Steven Pickard, Shuying Shen, Brett South, Guy Divita, Scott Duvall, et al. Using natural language processing on the free text of clinical documents to screen for evidence of homelessness among us veterans. In *AMIA Annual Symposium Proceedings*, volume 2013, page 537, 2013.

[25] Christopher J Rouillard, Mahmoud A Nasser, Haihong Hu, and Douglas W Roblin. Evaluation of a natural language processing approach to identify social determinants of health in electronic health records in a diverse community cohort. *Medical care*, 60(3):248–255, 2022.

[26] Daniel J Feller, Oliver J Bear Don't Walk, Jason Zucker, Michael T Yin, Peter Gordon, Noémie Elhadad, et al. Detecting social and behavioral determinants of health with structured and free-text clinical data. *Applied clinical informatics*, 11(01):172–181, 2020.

[27] Sifei Han, Robert F Zhang, Lingyun Shi, Russell Richie, Haixia Liu, Andrew Tseng, Wei Quan, Neal Ryan, David Brent, and Fuchiang R Tsui. Classifying social determinants of health from unstructured electronic health records using deep learning-based natural language processing. *Journal of biomedical informatics*, 127:103984, 2022.

[28] Zehao Yu, Xi Yang, Chong Dang, Songzi Wu, Prakash Adekkanattu, Jyotishman Pathak, Thomas J George, William R Hogan, Yi Guo, Jiang Bian, et al. A study of social and behavioral determinants of health in lung cancer patients using transformers-based natural language processing models. In *AMIA Annual Symposium Proceedings*, volume 2021, page 1225, 2022.

[29] Russell Richie, Victor M Ruiz, Sifei Han, Lingyun Shi, and Fuchiang Tsui. Extracting social determinants of health events with transformer-based multitask, multilabel named entity recognition. *Journal of the American Medical Informatics Association*, 30(8):1379–1388, 2023.

[30] Lei Gong, Jaren Bresnick, Aidong Zhang, Cathy Wu, and Kishlay Jha. Boosting social determinants of health extraction with semantic knowledge augmented large language model. In *AMIA Annual Symposium Proceedings*, volume 2024, page 453, 2025.

[31] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. *arXiv preprint arXiv:2303.08774*, 2023.

[32] Marco Guevara, Shan Chen, Spencer Thomas, Tafadzwa L Chaunzwa, Idalid Franco, Benjamin H Kann, Shalini Moningi, Jack M Qian, Madeleine Goldstein, Susan Harper, et al. Large language models to identify social determinants of health in electronic health records. *NPJ digital medicine*, 7(1):6, 2024.

[33] Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction-finetuned language models. *Journal of Machine Learning Research*, 25(70):1–53, 2024.

[34] Brian Romanowski, Asma Ben Abacha, and Yadan Fan. Extracting social determinants of health from clinical note text with classification and sequence-to-sequence approaches. *Journal of the American Medical Informatics Association*, 30(8):1448–1455, 2023.

[35] Vipina K Keloth, Salih Selek, Qingyu Chen, Christopher Gilman, Sunyang Fu, Yifang Dang, Xinghan Chen, Xinyue Hu, Yujia Zhou, Huan He, et al. Social determinants of health extraction from clinical notes across institutions using large language models. *npj Digital Medicine*, 8(1):1–13, 2025.
[36] Min Chen, Xuan Tan, and Rema Padman. Social determinants of health in electronic health records and their impact on analysis and risk prediction: a systematic review. *Journal of the American Medical Informatics Association*, 27(11):1764–1773, 2020.

[37] Rosa L Figueroa, Diego A Soto, and Esteban J Pino. Identifying and extracting patient smoking status information from clinical narrative texts in spanish. In *2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society*, pages 2710–2713. IEEE, 2014.

[38] A Karlsson, A Ellonen, H Irjala, V Väliaho, K Mattila, L Nissi, E Kytö, S Kurki, R Ristamäki, P Vihinen, et al. Impact of deep learning-determined smoking status on mortality of cancer patients: never too late to quit. *ESMO open*, 6(3):100175, 2021.

[39] Andrea Caccamisi, Leif Jørgensen, Hercules Dalianis, and Mats Rosenlund. Natural language processing and machine learning to enable automatic extraction and classification of patients’ smoking status from electronic medical records. *Upsala journal of medical sciences*, 125(4):316–324, 2020.

[40] Ye Seul Bae, Kyung Hwan Kim, Han Kyul Kim, Sae Won Choi, Taehoon Ko, Hee Hwa Seo, Hae-Young Lee, and Hyojin Jeon. Keyword extraction algorithm for classifying smoking status from unstructured bilingual electronic health records based on natural language processing. *Applied Sciences*, 11(19):8812, 2021.

[41] Matilde Karakachoff, Thomas Goronflot, Sandrine Coudol, Delphine Toublant, Adrien Bazoge, Pacôme Constant Dit Beaufils, Emilie Varey, Christophe Leux, Nicolas Mauduit, Matthieu Wargny, et al. Implementing a biomedical data warehouse from blueprint to bedside in a regional french university hospital setting: Unveiling processes, overcoming challenges, and extracting clinical insight. *JMIR Medical Informatics*, 12(1):e50194, 2024.

[42] Meliha Yetisgen, Esther Pellicer, David R. Crosslin, and Lucy Vanderwende. Automatic identification of lifestyle and environmental factors from social history in clinical text. In *Proceedings of the AMIA 2016 Joint Summits on Translational Science*, San Francisco, CA, 2016.

[43] Meliha Yetisgen and Lucy Vanderwende. Automatic identification of substance abuse from social history in clinical text. In *Conference on Artificial Intelligence in Medicine in Europe*, pages 171–181. Springer, 2017.

[44] Fiona A Költringer, Kristi Sidney Annerstedt, Delia Boccia, Daniel J Carter, and William E Rudgard. The social determinants of national tuberculosis incidence rates in 116 countries: a longitudinal ecological study between 2005–2015. *BMC public health*, 23(1):337, 2023.

[45] James R Hargreaves, Delia Boccia, Carlton A Evans, Michelle Adato, Mark Petticrew, and John DH Porter. The social determinants of tuberculosis: from evidence to action. *American journal of public health*, 101(4):654–662, 2011.

[46] Alvaro Alonso, Giancarlo Logroscino, Susan S Jick, and Miguel A Hernán. Association of smoking with amyotrophic lateral sclerosis risk and survival in men and women: a prospective study. *BMC neurology*, 10:1–6, 2010.

[47] Kevin Lybarger, Mari Ostendorf, and Meliha Yetisgen. Annotating social determinants of health using active learning, and characterizing determinants using neural event extraction. *Journal of Biomedical Informatics*, 113:103631, 2021.

[48] Pontus Stenetorp, Sampo Pyysalo, Goran Topić, Tomoko Ohta, Sophia Ananiadou, and Jun'ichi Tsujii. Brat: a web-based tool for nlp-assisted text annotation. In *Proceedings of the Demonstrations at the 13th Conference of the European Chapter of the Association for Computational Linguistics*, pages 102–107, 2012.

[49] Marie-Pier Gauthier, Jennifer H Law, Lisa W Le, Janice JN Li, Sajda Zahir, Sharon Nirmalakumar, Mike Sung, Christopher Pettengell, Steven Aviv, Ryan Chu, et al. Automating access to real-world evidence. *JTO Clinical and Research Reports*, 3(6):100340, 2022.
[50] Elham Hatef, Masoud Rouhizadeh, Iddrisu Tia, Elyse Lasser, Felicia Hill-Briggs, Jill Marsteller, Hadi Kharrazi, et al. Assessing the availability of data on social and behavioral determinants in structured and unstructured electronic health records: a retrospective analysis of a multilevel health care system. *JMIR medical informatics*, 7(3):e13802, 2019.

[51] Xiang Zhang, Senyu Li, Bradley Hauer, Ning Shi, and Grzegorz Kondrak. Don't trust ChatGPT when your question is not in English: A study of multilingual abilities and types of LLMs. In *Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing*, pages 7915–7927, Singapore, December 2023. Association for Computational Linguistics.

[52] Chris Wendler, Veniamin Veselovsky, Giovanni Monea, and Robert West. Do llamas work in english? on the latent language of multilingual transformers. In *Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)*, pages 15366–15394, 2024.

[53] Fahim Faisal and Antonios Anastasopoulos. Geographic and geopolitical biases of language models. In Duygu Ataman, editor, *Proceedings of the 3rd Workshop on Multi-lingual Representation Learning (MRL)*, pages 139–163, Singapore, December 2023. Association for Computational Linguistics.

[54] Jeongyoung Park, Robert Phillips, and Andrew Bazemore. Factors associated with documenting social determinants of health in electronic health records by family physicians, 2024.

[55] Stephanie Loo, Melanie Molina, N Jia Ahmad, Maeve Swanton, Olivia Chen, Krislyn M Boggs, Carlos A Camargo, and Margaret Samuels-Kalow. Implementing social determinants of health screening in us emergency departments. *JAMA Network Open*, 8(3):e250137–e250137, 2025.
A Supplemental Material

<table><caption>Table 8: Z-codes mapping to SDoH categories.</caption><thead><tr><td>SDoH categories</td><td>Z-codes</td></tr></thead><tbody><tr><td>Alcohol (current, past)</td><td>Z502 (alcohol withdrawal), Z714 (advice and monitoring for alcoholism)</td></tr><tr><td>Tobacco (current)</td><td>Z720 (smoking-related difficulties)</td></tr><tr><td>Drug</td><td>Z503 (rehabilitation for drug addicts and after drug abuse)</td></tr><tr><td>Living condition (alone)</td><td>Z290 (isolation), Z602 (difficulties linked to loneliness)</td></tr><tr><td>Descendants (yes)</td><td>Z370 (single birth, living child), Z372 (twin birth, twins born alive), Z391 (breastfeeding care and examinations), Z392 (routine postpartum check-up)</td></tr><tr><td>Housing (no)</td><td>Z590 (difficulties associated with homelessness), Z598 (other difficulties related to housing and economic conditions)</td></tr><tr><td>Ethnicity/Country of birth</td><td>Z603 (acculturation difficulties)</td></tr></tbody></table>

<table><caption>Table 9: Most common types of discrepancies between ground truth and model predictions.</caption><thead><tr><td>Error type</td><td>Count</td></tr></thead><tbody><tr><td>Human annotation errors</td><td>105</td></tr><tr><td>False negatives</td><td>224</td></tr><tr><td>False positives</td><td>125</td></tr><tr><td>Output not structured as specified during model training</td><td>63</td></tr><tr><td>Incorrect labels</td><td>58</td></tr><tr><td>Postprocessing error (multiple matches in the source text)</td><td>26</td></tr><tr><td>Correct label but different text spans</td><td>94</td></tr><tr><td>Tokenization-related errors</td><td>3</td></tr></tbody></table>