# Prediction of post-traumatic stress disorder in family members of ICU patients: a machine learning approach

Thibault Dupont, Nancy Kentish-Barnes, Frédéric Pochard, Edouard

Duchesnay, Elie Azoulay

► To cite this version:

Thibault Dupont, Nancy Kentish-Barnes, Frédéric Pochard, Edouard Duchesnay, Elie Azoulay. Prediction of post-traumatic stress disorder in family members of ICU patients: a machine learning approach. Intensive Care Medicine, 2023, 50 (1), pp.114-124. 10.1007/s00134-023-07288-1. hal-04995402

HAL Id: hal-04995402

https://hal.science/hal-04995402v1

Submitted on 18 Mar 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
<figure><img src="image_2.png" /></figure>

# Prediction of post-traumatic stress disorder in family members of ICU patients: a machine learning approach

Thibault Dupont¹,<fnref n="2" />,³, Nancy Kentish-Barnes<fnref n="2" />,³, Frédéric Pochard<fnref n="2" />, Edouard Duchesnay¹ and Elie Azoulay<fnref n="2" />,³*

© 2023 Springer-Verlag GmbH Germany, part of Springer Nature

**Abstract**

**Purpose:** Post-traumatic stress disorder (PTSD) can affect family members of patients admitted to the intensive care unit (ICU). Easily accessible patient's and relative's information may help develop accurate risk stratification tools to direct relatives at higher risk of PTSD toward appropriate management.

**Methods:** PTSD was measured 90 days after ICU discharge using validated instruments (Impact of Event Scale and Impact of Event Scale-Revised) in 2374 family members. Various supervised machine learning approaches were used to predict PTSD in family members and evaluated on an independent held-out test dataset. To better understand variables' contributions to PTSD predicted probability, we used machine learning interpretability methods on the best predictive algorithm.

**Results:** Non-linear ensemble learning tree-based methods showed better predictive performances (Random Forest—area under curve, AUC = 0.73 [0.68–0.77] and XGBoost—AUC = 0.73 [0.69–0.78]) than regularized linear models, kernel-based models, or deep learning models. In the best performing algorithm, most important features that positively contributed to PTSD's predicted probability were all non-modifiable factors, namely, lower patient's age, longer duration of ICU stay, relative's female sex, lower relative's age, relative being a spouse/child, and patient's death in ICU. A sensitivity analysis in bereaved relatives did not alter the algorithm's predictive performance.

**Conclusion:** We propose a machine learning-based approach to predict PTSD in relatives of ICU patients at an individual level. In this model, PTSD is mostly influenced by non-modifiable factors.

**Keywords:** Machine learning, Algorithm, Post-ICU burden, Prediction, Mechanical ventilation

**Introduction**

Post-traumatic stress disorder (PTSD) is an anxiety dis-
order that can affect individuals who have experienced
or witnessed a traumatic event [1]. While traditionally
associated with direct victims of physical injury or severe
mental or emotional distress, such as military combat,

violent assault, natural disaster, or other life-threatening
events, emerging research has shed light on the signifi-
cant impact of PTSD on family members of intensive
care unit (ICU) patients [2–4]. It is now a known fact
that the ICU environment, with its inherent stressors and
emotional challenges, can trigger psychological distress
and lead to the development of PTSD symptoms among
family members [5].

Studies have provided valuable insights into the
prevalence and risk factors associated with PTSD in
family members of ICU patients [2–4]. The findings
revealed a notable prevalence rate of 30% to 50%.
Furthermore, some demographic factors contributed

Full author information is available at the end of the article
Thibault Dupont and Nancy Kentish-Barnes have contributed equally to
this work. Edouard Duchesnay and Elie Azoulay have contributed equally
to this work.

<figure><img src="image_3.png" /></figure>

Springer

<footnote>*Correspondence: elie.azoulay@aphp.fr</footnote>

<footnote n="2">² Famirea Study Group, Médecine Intensive et Réanimation, Hôpital Saint-Louis, 1 Avenue Claude Vellefaux, 75010 Paris, France</footnote>
to a higher likelihood of developing PTSD, including being a spouse, female gender, and being the relative of a patient who died. Additionally, symptom's control and suboptimal communication between family members and the ICU team emerged as significant factors associated with increased rates of PTSD. Within the context of bereavement, the detrimental impact of being unable to say goodbye to the patient, and witnessing the patient's distressful breathing patterns have been reported [6]. These aggravating factors further emphasized the complex nature of PTSD development in this specific setting, distinguishing it from prolonged grief [6, 7].

Building upon the insights gained from previous research, the present study aims to delve deeper into understanding and predicting PTSD among family members of ICU patients. For instance, three randomized controlled trials performed in dying patients showed a significant PTSD reduction in family members [8–10], whereas other trials performed in patients at high risk of death were either negative [11–13], or associated with deleterious effect from the intervention [14–16]. These important discrepancies across studies raise the concern of non-modifiable factors associated with PTSD in family members of ICU patients. Deciphering the contribution of these non-modifiable factors to family PTSD would enable clinicians and researchers take heed of the complexity in averting PTSD, and influence research priorities toward management more than prevention.

To exemplify the contribution of non-modifiable factors to family PTSD, we have used a novel approach by leveraging machine learning technologies on a large dataset of relatives of ICU patients followed up within 3 months of ICU discharge. The utilization of machine learning methodologies holds promise in elucidating the complex interplay of various risk factors, including demographic, clinical, and environmental variables. Incorporating risk prediction systems at an individual level using machine learning algorithms may help identify individuals at higher risk of developing PTSD within this population, and direct targeted interventions or support systems to mitigate the risk and manage PTSD in family members of ICU patients.

# Methods

## Patients, data sources collection, and aggregation

We included patient's and relative's data issued from nine clinical studies (March 2003–October 2020) of the FAMIREA study group (flowchart, Fig. 1, supplemental Table S1). Raw epidemiologic, demographic, and medical data were collected from both patients and relatives, and all common variables were aggregated in the form of an anonymized dataset.

## Take-home message

In family members of ICU patients, interventions to alleviate the symptoms of PTSD should take into account the importance of non-modifiable predictors.

All studies were approved by respective institutional review boards and all family members provided informed consent to participate to these studies.

## Post-traumatic stress disorder evaluation and characterization

All relatives experienced the ICU admission of a close relative, thereby satisfying criterion A of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5) for PTSD, namely, the exposure of a close family member to a traumatic event [22]. The outcome of interest was the presence of PTSD-related symptoms in relatives 90 days after ICU discharge and was assessed using the raw value of the Impact of Events (IES) scale or its revised version (IES-R) according to which metric was used in each study. We chose a threshold of 24 for IES and 26 for IES-R, above which relatives were considered to have significant PTSD-related symptoms [23–25]. The distribution of the PTSD outcome variable in each study along with the clinical evaluation scale used is summarized in supplemental Table S2. All studies had a PTSD evaluation metric at day 90 except for one study in which relatives were evaluated at day 30 [15]. These data was kept in the final aggregated dataset under the assumption that this difference in timepoint would not greatly influence PTSD outcome.

## Data preprocessing and variable selection

Variables on the final aggregated dataset with more than 40% of missing values were not included in the final model, leaving 16 variables of interest (supplemental Table S4). Dataset was then dummy encoded and variables with missing data (supplemental Fig. S2, Table S5) were imputed using a K-Nearest Neighbor imputation method with one nearest-neighbor, stratified on patient's sex and ICU mortality status (supplemental Fig. S2). A sensitivity analysis using multiple imputation as an alternative imputation method was also conducted (supplemental Fig. S11). A correlation matrix with the Pearson's correlation coefficient of all features included in the classification models was also generated (supplemental Fig. S3).

## Statistical analysis

For univariable analysis, continuous variables are described as median and inter-quartile ranges (IQR) and
<figure><img src="image_4.png" /><figcaption>**Fig. 1** Study flow diagram. *References of all nine studies included: [2], [8], [10], [15], [17–21]</figcaption></figure>

compared using the Mann–Whitney or the Kruskal–Wallis test. Categorical variables are described by counts and frequencies and compared using Fisher's exact test. All tests were two-sided, and a *P* value less than 0.05 was considered significant. *P* values were adjusted for multiple testing using Bonferroni's correction. Multivariable analysis using a non-regularized logistic regression model was conducted, using PTSD symptoms as the outcome variable and all 16 variables of interest as regressors. Results were shown as odds ratios (OR) with their respective 95% confidence intervals (CI95%) and displayed as a forest plot (supplemental Fig. S8 and Table S8).

## Machine learning study design

Our study was conducted in accordance with the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) checklist [26]. An initial random train/test split of the final dataset of 2734 patients was performed (80%/20%), stratified on both the presence of PTSD symptoms and the study from which the data originated (supplemental Fig. S2). Univariate analysis comparing patients' characteristics between the training and test sets was conducted (supplemental Table S3). We included 16 covariates of interest to train machine learning models (supplemental Table S4). Each supervised classification model was trained using

a machine learning pipeline constructed with multiple steps: the KNN imputer, the scaling method (either standard or min-max scaling), and finally the machine learning model. In total, we used seven different classification models, using both linear (logistic regression with L2 regularization, and ElasticNet regularization) and non-linear approaches with a kernel-based method [Support Vector Machine (SVM)] and ensemble learning methods with XGBoost [27] and Random Forests [28]). Optimal hyperparameter tuning for each model was performed using tenfold grid search cross-validation on the training set on a range of chosen hyperparameters (supplemental Table S6). Performances of machine learning algorithms were then evaluated on the independent held-out test set with the following metrics: Area Under the Curve of the Receiving Operator Characteristic (ROC-AUC), recall, specificity, and balanced accuracy. Aleatoric variability estimates (mean and CI 95%) of each performance metric were estimated by training and evaluating all models on 1000 bootstrap samples with various random train/test splits (Monte Carlo cross-validation). A sensitivity analysis of models' performance on 1000 bootstrap samples of the original dataset was conducted. Briefly, we used the bootstrap sample as the train set and the out of bag sample as the test set. ROC-AUCs estimates (mean and CI95%) were then computed (supplemental Table S9). A calibration analysis was conducted
on the final XGBoost algorithm with a diagnosis reliability diagram, and the model was recalibrated using an isotonic method (supplemental Fig. S9). All statistical analyses and machine learning models were computed using Python 3.9 and the following packages: *pandas*, * numpy*, * matplotlib*, *statsmodels*, *scikit-learn* [29], *xgboost*, and *shap*.

## Model interpretability with feature importance and SHAP values

To gain further insight on how features affect model's output in the best performing model, we used SHapley Additive exPlanation (SHAP) values [30, 31], computed on the trained XGboost model with the dummy encoded, imputed, and scaled final dataset. SHAP values correspond to the average marginal contribution of each feature across all possible features' permutations. They not only allow to quantify the magnitude of impact of a given feature on a model's output, but also the direction of effect (toward higher or lower predicted probability if SHAP is positive or negative, respectively). We computed mean SHAP values of the XGBoost model on 1000 bootstrap samples with random train/test splits, to obtain mean and CI95% estimates. Feature importance corresponds to the average value of all absolute SHAP values for each feature and allows to rank variables according to the magnitude of impact they have on a model's output. We also derived dependence scatterplots (supplemental Fig. S7), illustrating the effect of a given numerical or categorical feature on the SHAP value. For numerical variables (patient's age, relative's age, and duration of ICU stay), we fitted a linear regression model on the dependence scatterplots to identify the critical feature value above or below which the SHAP value becomes either positive or negative. This helped us identify indicative cutpoints for numerical features that drive the model's decision output. Additionally, to further illustrate the cumulative effect of features to the model's output, we retrieved subsets of relatives in two distinct situations of lowest and highest predicted PTSD probabilities. We then built SHAP decision plots using these specific subsets to visualize the model's final decision with cumulative features of increasing importance (Fig. 3). Also, a decision curve analysis (DCA) was conducted to extract the net benefit over the range of threshold probabilities of all models and is shown in supplemental Fig. S10.

## Results

### Patients' characteristics and PTSD prevalence

Across all nine studies, 4591 family members were screened, 2992 (65.2%) were included, and 2734 (59.6%)

underwent a PTSD evaluation 90 days after ICU discharge and were kept in the final aggregated dataset (flow chart, Fig. 1, supplemental Table S1). Patients' characteristics are described in Table 1 and supplemental S7. The prevalence of family members with PTSD-related symptoms 90 days after ICU discharge was 33.5% (916/2734) (Table 1). Patient's median age was 68 years [57–76] and 63% were male. Half the patients were managed in ICUs from university hospitals (n = 1325). The median number of ICU beds was 16 [12–16, 22–29]. Length of ICU stay was 7 days [4.7–14], 1825 patients required mechanical ventilation (67%), and mortality rate was 70% (1909 deaths). Compared to family members without PTSD, those with PTSD-related symptoms were less frequently men [180 (21.2) vs. 542 (36.1), *p* < 0.001], and more frequently unemployed (111 (15) vs 144 (11.6), *p* = 0.034). PTSD affected significantly more frequently the relatives of younger patients (66 [55–76] vs. 68 [59–77], *p* < 0.001), those with cancer (34% vs 29.8%, *p* = 0.028), longer ICU length of stay (76.7% vs 66.3%, *p* < 0.001), more frequent need for mechanical higher (72.2% vs 64.1%, *p* < 0.001), longer duration of ICU stay (7.4 days [4.7, 15] vs 6.5 days [4.7, 12], *p* = 0.002), and more frequent death in ICU (76.7% vs 66.3%, *p* < 0.001). The presence of a psychologist dedicated to the ICU was less frequent in family members with PTSD-related symptoms (48.2% vs 55.2%, *p* = 0.011).

### Assessment of machine learning models to predict PTSD in relatives

Among all supervised learning classification models (Table 2, supplemental Fig. S4), the KNN classifier was less efficient (AUC 0.68 [0.64–0.73]. Regularized linear approaches with L2 logistic regression (AUC=0.71 [0.67–0.76]) and ElasticNet logistic regression (AUC=0.71 [0.67–0.75]) showed higher predictive performance. A deep learning approach with the Multilayer Perceptron (MLP) (AUC=0.71 [0.67–0.75]) and a kernel-based method with Support Vector Machine (AUC=0.72 [0.68–0.76]) showed similar performances. Highest predictive power was achieved using ensemble learning tree-based methods, namely, Random Forest (AUC=0.727 [0.683–0.773]), and XGBoost (AUC=0.731 [0.688–0.776]) (supplemental Figs. S4 and S5B). Highest balanced accuracy was reached with the XGBoost (0.68 [0.64–0.72]) and SVM (0.68 [0.64–0.71]) models, followed by L2 Logistic Regression (0.66 [0.61–0.7]), ElasticNet (0.65 [0.62–0.69]), Random Forest (0.65 [0.61–0.69]), Multilayer Perceptron (0.61 [0.56–0.64]), and KNN (0.56 [0.52–0.59]). In a sensitivity analysis with decision curve analysis (DCA), all models showed a net benefit in true positives in the [0.2–0.5] probability
<table><caption>Table 1 Relative's characteristics according to PTSD-related symptoms at day 90</caption><thead><tr><th>Median (IQR) or numbers (%)</th><th>Relatives without PTSD-related symptoms (n = 1818)</th><th>Relatives with PTSD-related symptoms (n = 916)</th><th>p value</th></tr></thead><tbody><tr><td>Age, median [Q1, Q3]</td><td>68 [59,77]</td><td>66 [55,76]</td><td>&lt; 0.001</td></tr><tr><td>Male sex, n (%)</td><td>1137 (62.5)</td><td>598 (65.3)</td><td>0.173</td></tr><tr><td>ICU death, n (%)</td><td>1206 (66.3)</td><td>703 (76.7)</td><td>&lt; 0.001</td></tr><tr><td>Performance status, n (%)</td><td></td><td></td><td></td></tr><tr><td>Good (1–2)</td><td>1207 (68.1)</td><td>579 (65.2)</td><td>0.149</td></tr><tr><td>Poor (3–4)</td><td>566 (31.9)</td><td>309 (34.8)</td><td></td></tr><tr><td>History of cancer, n (%)</td><td>532 (29.8)</td><td>305 (34)</td><td>0.028</td></tr><tr><td>Center type, n (%)</td><td></td><td></td><td></td></tr><tr><td>University</td><td>845 (50.9)</td><td>480 (55.8)</td><td>0.001</td></tr><tr><td>General</td><td>382 (23)</td><td>213 (24.8)</td><td></td></tr><tr><td>Private</td><td>434 (26.1)</td><td>167 (19.4)</td><td></td></tr><tr><td>Number of beds, median [Q1, Q3]</td><td>16 [12,24]</td><td>16 [12,24]</td><td>0.011</td></tr><tr><td>Psychologist, n (%)</td><td>910 (55.2)</td><td>415 (48.2)</td><td>0.001</td></tr><tr><td>Mechanical ventilation, n (%)</td><td>1164 (64.1)</td><td>661 (72.2)</td><td>&lt; 0.001</td></tr><tr><td>Treatment limitation, n (%)</td><td>1018 (57.9)</td><td>592 (66.3)</td><td>&lt; 0.001</td></tr><tr><td>Duration of stay, median [Q1, Q3]</td><td>6.5 [4.7,12]</td><td>7.4 [4.7,15]</td><td>0.002</td></tr><tr><td>Relationship, n (%)</td><td></td><td></td><td></td></tr><tr><td>Spouse</td><td>549 (36.1)</td><td>409 (48.1)</td><td>&lt; 0.001</td></tr><tr><td>Children</td><td>565 (37.1)</td><td>284 (33.4)</td><td></td></tr><tr><td>Parent</td><td>102 (6.7)</td><td>82 (9.6)</td><td></td></tr><tr><td>Other</td><td>305 (20.1)</td><td>76 (8.9)</td><td></td></tr><tr><td>Relative's age, median [Q1, Q3]</td><td>53 [44,64]</td><td>53 [43.8,63]</td><td>0.433</td></tr><tr><td>Relative's male sex, n (%)</td><td>542 (36.1)</td><td>180 (21.2)</td><td>&lt; 0.001</td></tr><tr><td>Relative's activity status, n (%)</td><td></td><td></td><td></td></tr><tr><td>Active</td><td>737 (59.5)</td><td>437 (59.1)</td><td>0.034</td></tr><tr><td>Unemployed</td><td>144 (11.6)</td><td>111 (15)</td><td></td></tr><tr><td>Retired</td><td>313 (25.3)</td><td>157 (21.2)</td><td></td></tr><tr><td>Other</td><td>45 (3.6)</td><td>35 (4.7)</td><td></td></tr><tr><td>Relative has children, n (%)</td><td>956 (80.2)</td><td>509 (82.2)</td><td>0.328</td></tr></tbody></table>

<table><caption>Table 2 Model performance metrics</caption><thead><tr><th>Model</th><th>AUC</th><th>Balanced accuracy</th><th>Recall</th><th>Specificity</th></tr></thead><tbody><tr><td>KNN</td><td>0.68 [0.64–0.73]</td><td>0.56 [0.52–0.59]</td><td>0.21 [0.15–0.27]</td><td>0.90 [0.87–0.93]</td></tr><tr><td>Logistic regression L2</td><td>0.71 [0.67–0.76]</td><td>0.66 [0.61–0.70]</td><td>0.67 [0.59–0.74]</td><td>0.64 [0.60–0.69]</td></tr><tr><td>Logistic regression ElasticNet</td><td>0.71 [0.67–0.75]</td><td>0.65 [0.62–0.69]</td><td>0.68 [0.62–0.75]</td><td>0.63 [0.59–0.68]</td></tr><tr><td>SVM</td><td>0.72 [0.68–0.76]</td><td>0.68 [0.64–0.71]</td><td>0.64 [0.57–0.70]</td><td>0.71 [0.66–0.76]</td></tr><tr><td>Multilayer Perceptron</td><td>0.71 [0.67–0.75]</td><td>0.61 [0.56–0.64]</td><td>0.34 [0.26–0.39]</td><td>0.88 [0.84–0.91]</td></tr><tr><td>Random forest</td><td>0.73 [0.68–0.77]</td><td>0.65 [0.61–0.69]</td><td>0.46 [0.39–0.54]</td><td>0.83 [0.80–0.87]</td></tr><tr><td>XGBoost</td><td>0.73 [0.69–0.78]</td><td>0.68 [0.64–0.72]</td><td>0.65 [0.58–0.71]</td><td>0.71 [0.67–0.77]</td></tr></tbody></table>

threshold range (supplemental Fig. S10). In a sensitivity analysis using bootstrapping of the original data, classification performance metrics were similar across all models, with Random Forest (0.69 [0.662, 0.718], *p* < 0.001)

and Regularized Logistic Regression (0.691 [0.664, 0.719], *p* < 0.001) performing slightly better than the XGBoost model (supplemental Table S8).
<figure><img src="image_5.png" /><figcaption>**Fig. 2** SHAP summary dot plot of the XGBoost model. The SHAP summary dot plot contains all observations represented as dots for each feature. Models' features are ranked on the y-axis from the highest to the lowest contributing variables in terms of feature importances. The distance of each dot from the center of the x-axis represents the magnitude of impact (SHAP value) on the model's output. Dots are colored according to the feature's value ranging from low (in blue) to high (in red) for numerical variables and No (in blue) to Yes (in red) for binary dummy encoded categorical variables. SHAP Shapley additive explanations, XGBoost extreme gradient boosting</figcaption></figure>

## Model interpretability with SHAP values

Overall, the XGBoost model showed the best predictive performances with both the highest mean ROC-AUC (supplemental Fig. S5) and balanced accuracy on the test set using the Monte Carlo validation method. Figure 2 reports the summary of SHAP values for each feature in the XGBoost model. Most influential positive contributors to PTSD with the highest feature importance included: younger patient's age, increased length of ICU stay, patient receiving mechanical ventilation, relative's female sex, being a spouse/concubine or a patient's child, younger relative's age, and patient's death

in the ICU (supplemental Fig. S5A). Using multivariate analysis with logistic regression (supplemental Table S8 and Fig. S8), we found the same contributors ranked in terms of feature importance. Variables positively associated with PTSD included being a spouse (OR [CI 95%], 3.05 [2.29–4.06]), a patient's child (2.54 [1.95–3.58]), a patient's parent (2.11 [1.46–3.04]), relative's female sex (2.07 [1.68–2.56]), duration of stay > 8 days (1.49 [1.25–1.77]), history of cancer (1.33 [1.1–1.59]), and mechanical ventilation (1.28 [1.06–1.55]). Diagnosis reliability diagrams showed an underestimation of true probabilities of PTSD for all classification models used, which was
improved following isotonic calibration (supplemental Fig. S9). This calibration method did not influence the predictive performance of these models. In particular, the final XGBoost model showed a comparable ROC-AUC (AUC = 0.72) before and after calibration.

Supplemental Figure S6 reports the results of the sensitivity analysis within the 1909 bereaved relatives. In this specific population, the XGBoost model remained the most performant model (AUC = 0.71). Being a patient's spouse or child, length of ICU stay, relatives as female and patient's younger age were the most significant PTSD positive contributors. We also performed a sensitivity analysis using a more conventional multiple imputation strategy that yielded similar results in terms of predictive performance (AUC = 0.73, for the XGBoost model), and interpretability (supplemental Fig. S11). Most important features remained being a spouse, patient's death in the

ICU, relative's sex, and ICU duration of stay, though in a different ranking order according to feature importance.

Figure 3 illustrates the cumulative effect of features driving to either a very low or very high probability of PTSD. In this specific subset of relatives with lowest and highest (Fig. 3) risk of PTSD (predicted probability < 0.15 or > 0.80, respectively), the most contributing features, either positively or negatively were patient's age, being a spouse/concubine or a patient's child, patient's history of cancer, duration of ICU stay, relative's sex, relative's age, and patient's ICU death. For numerical variables with highest feature importance (patient's age, duration of stay, relative's age), we identified indicative threshold values for which the SHAP value switches signs, thereby shifting the direction of effect for these features (supplemental Fig. S7, Fig. 3). Accordingly, patient's age > 60 years, relative's age > 53 years, and duration of stay < 8 days were all

<figure><img src="image_6.png" /></figure>
indicative values above or below which the association
shifted toward lower probability of PTSD.

Discussion

Post-traumatic stress disorder significantly affects the
mental well-being of family members of ICU patients,
highlighting the importance of targeted support and
interventions to mitigate its impact. Predicting those
relatives at higher risk of developing PTSD is crucial
to better direct them to comprehensive psychological
care. This study analyzed a comprehensive dataset of
2734 family members of ICU patients who underwent
a 3-month follow-up to assess the presence of PTSD-
related symptoms using two validated instruments.
The results suggest that clinical, easily accessible,
non-modifiable factors widely influence the predicted
probability of PTSD in the setting of a machine learning
approach. This influence is maintained in subgroups of
bereaved relatives and relatives with highest and lowest
predicted probability of PTSD. By developing an artificial
intelligence algorithm to predict PTSD, this study sheds
light on the predictive capabilities of machine learning,
the influence of non-modifiable predictors, and the
impact of patient and family characteristics on the
development of PTSD.

The 33% overall PTSD prevalence is in agreement with
studies performed by other groups [3, 32, 33]. Moreover,
the significantly higher PTSD in family members of
patients who died is in line with the previous results [34,
35].

Our results suggest that machine learning and
traditional linear statistical models show similar
performances in predicting PTSD using accessible
demographic and clinical information. By leveraging the
power of complex pattern recognition to uncover hidden
relationships, machine learning is able to accurately
predict outcomes. In the case of PTSD prediction, the
XGBoost algorithm achieved good performance. This
underscores the potential of machine learning as a
valuable tool in clinical settings, particularly for mental
health-related assessments.

One significant contribution of this study is the
identification of several non-modifiable predictors
of PTSD. For instance, spouses of younger patients,
especially those receiving mechanical ventilation, were
at higher risk of PTSD. Some other characteristics such
as family sex or being in a university affiliated hospital
are other non-modifiable factors that contribute to the
risk of PTSD. Bereaved family members are particularly
vulnerable to PTSD, suggesting the profound emotional
toll that the loss of a loved one during ICU admission
can have on family members [1, 2, 5]. In this setting,
three trials to assist bereaved family members in coping

with their grief at the time of death have shown that
tailored interventions led to significant reduction in
PTSD rates [8–10], whereas a trial performed 3 weeks
after death resulted in opposite results. Factors such
as communication between the family and healthcare
providers or the involvement of family members in the
decision-making process are inconstantly associated with
PTSD and could not be included in our study [9, 32]. By
study design, the final number of covariates included
in our models was restricted to the maximal number of
common variables with the least amount of missing data
across all studies and it is likely that covariates, such as
ICU interventions, that could improve the predictive
ability of the models have not been included. These
findings should not be considered as a slammed door on
possible control over certain risk factors that predispose
individuals to develop PTSD. They highlight that factors,
such as genetic predisposition, personal history of mental
health disorders, and family history of PTSD, might
impact the likelihood of developing PTSD among family
members of ICU patients and warrant specific studies to
be done in the field [36]. These non-modifiable predictors
emphasize the importance of targeted interventions and
support systems for individuals who are at higher risk,
amuing to mitigate the potential impact of PTSD on their
mental well-being.

A single but major modifiable factor that significantly
contributed to PTSD prediction is the length of ICU
stay. It was the sixth factor in all family members, but
the first to contribute to PTSD in family members of
patients who died. Studies have shown that barriers in
implementing decisions to withhold or withdraw life
sustaining therapies lead to overuse of ICU resources
[37, 38]. Moreover, Gerstel et al. have reported that
duration in life-support withdrawal was associated
with family satisfaction with care [39]. In agreement
with these findings, a multicomponent family-support
intervention that was associated with a shorter length
of stay in the ICU was significantly improved the
quality of communication and the patient perception
of centeredness [12]. Palliative care studies targeting
the length of ICU stay in patients dying in the ICU are
warranted.

The implications of our findings extend beyond the
academic realm, carrying important implications for
clinical practice and healthcare policy. The predictive
capabilities of machine learning algorithms provide an
opportunity for early identification of family members
at higher risk of developing PTSD, allowing for targeted
interventions and support resources to be allocated more
effectively. Using information readily available at ICU
discharge or death in the prediction algorithm, early
detection of PTSD-related symptoms in close relatives
could occur in the first days following ICU discharge. By leveraging machine learning technologies, healthcare providers can proactively identify individuals who may benefit from early interventions, such as psychological counseling, psychoeducation, post-ICU follow-up, and support groups, ultimately improving mental health outcomes in this vulnerable population. Future studies to identify personal and familial risk factors for PTSD will help fine-tune this algorithm. Furthermore, a personalized approach that includes genetic studies to identify molecular signatures of PTSD will add accuracy and relevance to PTSD reduction.

Our study has several limitations. First, the generalizability of our findings may be limited to the context of French ICUs. Further research is needed to validate our results in different healthcare settings and cultural contexts. Also, we included nine studies of varying design and population and some studies focused solely on relatives of deceased patients which can influence both outcome and PTSD management and therefore hinder the generalizability of our findings. Also, these findings might be sensitive to the different methods available for data preprocessing and model postprocessing, namely, missing data management and model calibration. In our study, we chose nearest-neighbor imputation stratified on patient's sex and ICU mortality. Multiple imputation in preprocessing and isotonic calibration in postprocessing were evaluated in a sensitivity analysis and did not affect the predictive performance of the models, but we cannot exclude that other methods would not influence our results. Second, while machine learning algorithms demonstrate good predictive performance, their complexity and lack of transparency raise concerns regarding interpretability. Learning models such as XGBoost or Random Forest help solve prediction problems at an individual level. The use of interpretability methods such as SHAP values allow to gain further insight on the inherent positive and negative drivers of the model's output. However, these methods can rarely answer causal questions. Efforts should be made to enhance the interpretability of machine learning models to facilitate their integration into routine clinical practice. Third, PTSD was defined using IES or IES-R scales. However, the DSM-5 now pays more attention to the behavioral symptoms that accompany PTSD and proposes four distinct diagnostic clusters (re-experiencing, avoidance, negative cognitions and mood, and arousal [24]).

Finally, this algorithm will need to be re-assessed with more variables depicting the individual and familial psychiatric background, as well with more modifiable factors that relate to the ICU experience and to interactions with healthcare providers. Incorporating

genetic factors to this algorithm will allow to identify the molecular signatures of PTSD and offer a personalized approach to PTSD prediction and management.

In summary, our study highlights the potential of machine learning algorithms in predicting PTSD in family members of ICU patients. The identification of non-modifiable predictors underscores the importance of targeted interventions for at-risk individuals, while the impact of the ICU length of stay warrants that it becomes the outcome variable for a series of palliative care interventions. By harnessing the power of machine learning, healthcare providers can enhance the identification, prevention, and management of PTSD, ultimately improving the overall well-being of family members affected.

**Supplementary Information**

The online version contains supplementary material available at https://doi.org/10.1007/s00134-023-07288-1.

**Author details**

**Author contributions**

All authors have made significant contribution for this manuscript, to the study design; and interpretation of data for the work; all have also contributed to drafting the work or reviewing it critically for important intellectual content; all approved the final version to be published; all authors agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. EA, NKB, and FP are the study PI, TD and ED are the methodologists dedicated to machine learning technologies. All authors have taken part to every single step of the development and validation of the algorithm. EA and ED directly accessed and verified the underlying data reported in the manuscript. All authors have full access to all the data in the study and accept responsibility to submit for publication. This manuscript has not been published elsewhere. However, as mentioned in the Methods and Fig. 1, the database was built using consecutive trials and cohort studies, each having been published.

**Funding**

The study sponsor is the French government. It had no role in the study design; the collection, analysis, interpretation of data; writing of the report; and in the decision to submit the paper for publication.

**Data availability**

Data are available on request to the corresponding author. Authorization from the steering committee will be required. The data collected for the study, including individual participant data (deidentified participant data) and a data dictionary defining each field in the set, will be made available in a web repository with publication, with investigator support. The proposal will be submitted to the steering committee and a data agreement will be signed.

**Declarations**

**Conflicts of interest**

EA reports support for the current manuscript from Programme Hospitalier de Recherche Clinique of the French Ministry of Health (PHRC) and ANR-RHU-5th Wave (RHU FAME 2021); outside the submitted work, grants from MSD-AVENIR and Alexion, and lecture honoraria from Pfizer, Alexion, and Sanofi. All other authors report no conflict of interest.

<footnote n="1">¹ NeuroSpin, CEA, Université Paris-Saclay, 91191 Gif-Sur-Yvette, France.</footnote>

<footnote n="2">² Famirea Study Group, Médecine Intensive et Réanimation, Hôpital Saint-Louis, 1 Avenue Claude Vellefaux, 75010 Paris, France. ³ Université Paris Cité, Paris, France.</footnote>
**Publisher's Note**

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Received: 19 August 2023 Accepted: 18 November 2023

Published: 19 December 2023

**References**

1. Yehuda R (2002) Post-traumatic stress disorder. N Engl J Med 346:108-114. https://doi.org/10.1056/NEJMra012941

2. Azoulay E, Pochard F, Kentish-Barnes N et al (2005) Risk of post-traumatic stress symptoms in family members of intensive care unit patients. Am J Respir Crit Care Med 171:987-994. https://doi.org/10.1164/rccm.200409-12950C

3. Jones C, Skirrow P, Griffiths RD et al (2004) Post-traumatic stress disorder-related symptoms in relatives of patients following intensive care. Intensive Care Med 30:456-460. https://doi.org/10.1007/s00134-003-2149-5

4. Wendlandt B, Ceppe A, Choudhury S et al (2018) Risk factors for post-traumatic stress disorder symptoms in surrogate decision-makers of patients with chronic critical illness. Ann Am Thorac Soc 15:1451-1458. https://doi.org/10.1513/AnnalsATS.201806-4200C

5. Herridge MS, Azoulay E (2023) Outcomes after critical illness. N Engl J Med 388:913-924. https://doi.org/10.1056/NEJMra2104669

6. Kentish-Barnes N, Chaize M, Seegers V et al (2015) Complicated grief after death of a relative in the intensive care unit. Eur Respir J 45:1341-1352. https://doi.org/10.1183/09031936.00160014

7. Siegel MD, Hayes E, Vanderwerker LC et al (2008) Psychiatric illness in the next of kin of patients who die in the intensive care unit. Crit Care Med 36:1722-1728. https://doi.org/10.1097/CCM.0b013e318174da72

8. Lautrette A, Darmon M, Megarbane B et al (2007) A communication strategy and brochure for relatives of patients dying in the ICU. N Engl J Med 356:469-478. https://doi.org/10.1056/NEJMoa063446

9. Jabre P, Belpomme V, Azoulay E et al (2013) Family presence during cardiopulmonary resuscitation. N Engl J Med 368:1008-1018. https://doi.org/10.1056/NEJMoa1203366

10. Kentish-Barnes N, Chevret S, Valade S et al (2022) A three-step support strategy for relatives of patients dying in the intensive care unit: a cluster randomised trial. Lancet 399:656-664. https://doi.org/10.1016/S0140-6736(21)02176-0

11. Azoulay E, Forel J, Vinatier I et al (2018) Questions to improve family-staff communication in the ICU: a randomized controlled trial. Intensive Care Med 44:1879-1887. https://doi.org/10.1007/s00134-018-5423-2

12. White DB, Angus DC, Shields A-M et al (2018) A randomized trial of a family-support intervention in intensive care units. N Engl J Med 378:2365-2375. https://doi.org/10.1056/NEJMoa1802637

13. Curtis JR, Treece PD, Nielsen EL et al (2016) Randomized trial of communication facilitators to reduce family distress and intensity of end-of-life care. Am J Respir Crit Care Med 193:154-162. https://doi.org/10.1164/rccm.201505-09000C

14. Curtis JR, Back AL, Ford DW et al (2013) Effect of communication skills training for residents and nurse practitioners on quality of communication with patients with serious illness: a randomized trial. JAMA 310:2271-2281. https://doi.org/10.1001/jama.2013.282081

15. Kentish-Barnes N, Chevret S, Champigneulle B et al (2017) Effect of a con-dolence letter on grief symptoms among relatives of patients who died in the ICU: a randomized clinical trial. Intensive Care Med 43:473-484. https://doi.org/10.1007/s00134-016-4669-9

16. Carson SS, Cox CE, Wallenstein S et al (2016) Effect of palliative care-led meetings for families of patients with chronic critical illness: a randomized clinical trial. JAMA 316:51-62. https://doi.org/10.1001/jama.2016.8474

17. de Miranda S, Pochard F, Chaize M et al (2011) Postintensive care unit psychological burden in patients with chronic obstructive pulmonary disease and informal caregivers: A multicenter study. Crit Care Med 39(1):112-118. https://doi.org/10.1097/CCM.0b013e3181feb824

18. Ehooman F, Biard L, Lemiale V et al (2019) Long-term health-related quality of life of critically ill patients with haematological malignancies: a prospective observational multicenter study. Ann Intensive Care 9(1):2. https://doi.org/10.1186/s13613-018-0478-3

19. Azoulay E, Kouatchet A, Jaber S et al (2013) Noninvasive mechanical ventilation in patients having declined tracheal intubation. Intensive Care Med 39(2):292-301. https://doi.org/10.1007/s00134-012-2746-2

20. Azoulay E, Resche-Rigon M, Megarbane B et al (2022) Association of COVID-19 acute respiratory distress syndrome with symptoms of post-traumatic stress disorder in family members after ICU discharge. JAMA 327(11):1042-1050. https://doi.org/10.1001/jama.2022.17

21. Kentish-Barnes N, Seegers V, Legriel S et al (2016) CAESAR: a new tool to assess relatives' experience of dying and death in the ICU. Intensive Care Med 42(6):995-1002. https://doi.org/10.1007/s00134-016-4260-4

22. Marx BP, Lee DJ, Norman SB et al (2022) Reliable and clinically significant change in the clinician-administered PTSD Scale for DSM-5 and PTSD Checklist for DSM-5 among male veterans. Psychol Assess 34:197-203. https://doi.org/10.1037/pas0001098

23. Creamer M, Bell R, Failla S (2003) Psychometric properties of the Impact of Event Scale—revised. Behav Res Ther 41:1489-1496. https://doi.org/10.1016/j.brat.2003.07.010

24. Hoge CW, Riviere LA, Wilk JE et al (2014) The prevalence of post-traumatic stress disorder (PTSD) in US combat soldiers: a head-to-head comparison of DSM-5 versus DSM-IV-TR symptom criteria with the PTSD checklist. Lancet Psychiatry 1:269-277. https://doi.org/10.1016/S2215-0366(14)70235-4

25. Witteveen AB, Bramsen I, Hovens JE, van der Ploeg HM (2005) Utility of the impact of event scale in screening for posttraumatic stress disorder. Psychol Rep 97:297-308. https://doi.org/10.2466/pr0.97.1.297-308

26. Collins GS, Reitsma JB, Altman DG, Moons KGM (2015) Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. Ann Intern Med 162(1):55-63. https://doi.org/10.7326/M14-0697

27. Chen T, Guestrin C (2016) XGBoost: a scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, New York, NY, USA, pp 785-794

28. Breiman L (2001) Random forests. Mach Learn 45:5-32. https://doi.org/10.1023/A:1010933404324

29. Pedregosa F, Varoquaux G, Gramfort A et al (2011) Scikit-learn: machine learning in Python. J Mach Learn Res 12:2825-2830

30. Lundberg SM, Lee S-I (2017) A unified approach to interpreting model predictions. In: Guyon I, Luxburg UV, Bengio S et al (eds) Advances in neural information processing systems 30. Curran Associates Inc, pp 4765-4774

31. Lundberg SM, Erion G, Chen H et al (2020) From local explanations to global understanding with explainable AI for trees. Nat Mach Intell 2:2522-5839

32. Anderson WG, Arnold RM, Angus DC, Bryce CL (2008) Posttraumatic stress and complicated grief in family members of patients in the intensive care unit. J Gen Intern Med 23:1871-1876. https://doi.org/10.1007/s11606-008-0770-2

33. Cameron JI, Chu LM, Matte A et al (2016) One-year outcomes in caregivers of critically ill patients. N Engl J Med 374:1831-1841. https://doi.org/10.1056/NEJMoa1511160

34. Robert R, Le Gouge A, Kentish-Barnes N et al (2017) Terminal weaning or immediate extubation for withdrawing mechanical ventilation in critically ill patients (the ARREVE observational study). Intensive Care Med. https://doi.org/10.1007/s00134-017-4891-0

35. Probst DR, Gustin JL, Goodman LF et al (2016) ICU versus non-ICU hospital death: family member complicated grief, posttraumatic stress, and depressive symptoms. J Palliat Med 19:387-393. https://doi.org/10.1089/jpm.2015.0120

36. Nievergelt CM, Maihofer AX, Klengel T et al (2019) International meta-analysis of PTSD genome-wide association studies identifies sex- and ancestry-specific genetic risk loci. Nat Commun 10:4558. https://doi.org/10.1038/s41467-019-12576-w
37. The SUPPORT Principal Investigators (1995) A controlled trial to improve care for seriously ill hospitalized patients. The study to understand prog-noses and preferences for outcomes and risks of treatments (SUPPORT). The SUPPORT Principal Investigators. JAMA 274:1591-1598

38. Nelson JE, Azoulay E (2019) Intensive care unit provision at the end of life: miles travelled, miles to go. Lancet Respir Med 7:560-562. https://doi.org/10.1016/S2213-2600(19)30168-7

39. Gerstel E, Engelberg RA, Koepsell T, Curtis JR (2008) Duration of with-drawal of life support in the intensive care unit and association with family satisfaction. Am J Respir Crit Care Med 178:798-804. https://doi.org/10.1164/rccm.200711-1617OC